Known-issues sheets for local language models Rev 0.1.0 · reviewed every 7 days
llm-picker

Model datasheet · Gemma 3

Gemma 3 4B

Superseded

Superseded by Gemma 4 E4B (Google's next generation at the same 8GB-laptop size, with vision again). Kept as the record of what the previous generation scored — pleasant chat and image understanding on small machines, and weak at math and code.

Vendor
Google
Architecture
Dense · 4B
Context
131,072 tokens
License
Gemma Terms of Use
Released
2025-03-12
Vision
Yes

→ GGUF (llama.cpp / LM Studio / Ollama) → MLX (Apple Silicon)

Architecture, from its own config.json

34 layers · 29 sliding window · 5 full attention

8 attention heads over 4 KV heads (GQA 2:1), head dim 256 · dense feed-forward · vocabulary 262,208

Where the memory goes: weights by band above, plus a KV cache of 334MB at 8K · 814MB at 32K · 2.7GB at 128K, plus the runtime's own working memory — dominated by the output projection over that vocabulary.

Inside one layer, from its own tensors

34 × layer

29 sliding-window layers, 5 full-attention layers — the same block either way. What differs is how far back the mask lets attention look, not what the layer is built from.

input_layernormattentionq_projk_projv_projsoftmax(q·kᵀ/√d) · vq_norm · k_norm before the dotproduct8 query heads over 4 key/value headso_projpost_attention_layernormpre_feedforward_layernormfeed-forwardgate_proj · up_proj → down_proj10,240 widepost_feedforward_layernorm

The green line down the right of each half is the residual: it leaves the spine above the norm, skips the operation entirely and rejoins at the +. Where attention is drawn open, q, k and v are scaled to their real output widths — a model that shares key and value heads across many query heads shows one wide box beside two narrow ones.

Around the stack: an embedding of 262,208 × 2,560, a final norm, and an output projection that reuses the embedding (tied weights).

§1 Characteristics by quantization band

Q2–Q3 — 2–3 bit

2.1GB weights · e.g. Q3_K_M + KV: 334MB at 8K · 814MB at 32K · 2.7GB at 128K

— Not yet rated · contributions welcome —

Q4–Q5 — 4–5 bit

2.5GB weights · e.g. Q4_0 , Q4_K_M , mlx-4bit + KV: 334MB at 8K · 814MB at 32K · 2.7GB at 128K
Math & reasoning 3/10
data checked Aug 2026
  • quant-degradation-community — community consensus @ Q4_0 · aggregated · 2026-08-30

    Editorial derivation: already weak at full precision; QAT Q4_0 preserves what there is.

Coding 4/10
data checked Aug 2026
Creative writing 4/10
data checked Oct 2026
  • quant-degradation-community — editorial band derivation @ Q4_0 · aggregated · 2026-10-04

    Editorial derivation: carried from the EQ-Bench-anchored full band; QAT Q4_0 limits quant loss.

Q6–Q8 — 6–8 bit

3.2GB weights · e.g. Q6_K , Q8_0 + KV: 334MB at 8K · 814MB at 32K · 2.7GB at 128K

— Not yet rated · contributions welcome —

FP16 — full precision (fp16/bf16)

7.8GB weights · e.g. bf16 + KV: 334MB at 8K · 814MB at 32K · 2.7GB at 128K
Math & reasoning 3/10
data checked Aug 2026
Coding 4/10
data checked Aug 2026
Factuality 5/10
data checked Aug 2026
Creative writing 4/10
data checked Oct 2026
  • eqbench-creative — EQ-Bench Creative Writing v3 rubric score (0-20) = 13.91 @ bf16 · aggregated · 2026-10-04

    The weakest of the Gemma 3 sizes (12B 14.63, 27B 15.33), but well ahead of Llama 3.2 3B (8.18) at a similar size. Normalised Elo 1068.0, 114th of 140, retrieved 2026-10-04. Independent; rubric judged by Claude Sonnet 4. EQ-Bench's normalised Elo moves as models are added, so the rubric score is the figure stored. Quant not stated by EQ-Bench (presumed full-precision API serving). Data file behind https://eqbench.com/creative_writing.html; the date the row was added is not published.

  • eqbench-creative — EQ-Bench Creative Writing Longform score (0-100) = 34.4 @ bf16 · aggregated · 2026-10-04

    Longform: below the 12B and 27B (41.8, 41.9), ahead of Qwen3 4B (17.0). Data file behind the Longform page on eqbench.com. Quant not stated (presumed full precision).

§2 Known issues & what fixes them

ISSUE-01 severe Q2–Q3 / Q4–Q5 / Q6–Q8 / FP16 · math, coding

Weakest math and coding in its size class (MATH 24.2, HumanEval 36.0 — vendor's own numbers). Its strengths are chat, languages, and vision; STEM is not on the list.

Fix
Code interpreter / calculator tool [partial]

A calculator tool helps with arithmetic — though check tool-calling works in your setup, since Gemma's function calling is prompt-based.

No fix
No known fix — choose a different model

For math or code at 4B, use Qwen3-4B or Phi-4-mini instead.

Evidence · 2 sources · community-consensus
ISSUE-02 moderate Q2–Q3 / Q4–Q5 / Q6–Q8 / FP16 · factuality

Thin world knowledge plus a 67.3% answer rate on HHEM — it either doesn't know or declines to say, a third of the time.

Fix
RAG (retrieval) [strong]

Grounding raises both accuracy and answer rate.

Evidence · 2 sources · community-consensus