Model datasheet · Gemma 3
Gemma 3 12B
SupersededSuperseded by Gemma 4 12B, Google's next generation at the same size. Kept as the record of what the previous generation scored: natural prose, multilingual chat and built-in vision, weaker than same-size rivals at code and tool use.
- Vendor
- Architecture
- Dense · 12.2B
- Context
- 131,072 tokens
- License
- Gemma Terms of Use
- Released
- 2025-03-12
- Vision
- Yes
→ GGUF (llama.cpp / LM Studio / Ollama) → MLX (Apple Silicon)
Architecture, from its own config.json
48 layers · 40 sliding window · 8 full attention
16 attention heads over 8 KV heads (GQA 2:1), head dim 256 · dense feed-forward · vocabulary 262,208
Where the memory goes: weights by band above, plus a KV cache of 992MB at 8K · 2.5GB at 32K · 8.5GB at 128K, plus the runtime's own working memory — dominated by the output projection over that vocabulary.
Inside one layer, from its own tensors
48 × layer
40 sliding-window layers, 8 full-attention layers — the same block either way. What differs is how far back the mask lets attention look, not what the layer is built from.
The green line down the right of each half is the residual: it leaves the spine above the norm, skips the operation entirely and rejoins at the +. Where attention is drawn open, q, k and v are scaled to their real output widths — a model that shares key and value heads across many query heads shows one wide box beside two narrow ones.
Around the stack: an embedding of 262,208 × 3,840, a final norm, and an output projection that reuses the embedding (tied weights).
§1 Characteristics by quantization band
data checked Aug 2026
- quant-degradation-community — degradation reports @ Q3_K_M · aggregated · 2026-08-30
Editorial derivation: standard ≤3-bit math cliff.
data checked Oct 2026
- quant-degradation-community — editorial band derivation @ Q3_K_M · aggregated · 2026-10-04
Editorial derivation: one point below the mid band, as before; prose survives low quants better than reasoning does, but Q3_K_M is not a QAT build.
Q4–Q5 — 4–5 bit
7GB weights · e.g. Q4_0 , Q4_K_M , mlx-4bit + KV: 992MB at 8K · 2.5GB at 32K · 8.5GB at 128Kdata checked Oct 2026
- quant-degradation-community — editorial band derivation @ Q4_0 · aggregated · 2026-10-04
Editorial derivation: carried from the EQ-Bench-anchored full band; Google ships QAT Q4_0 weights trained to preserve quality.
data checked Aug 2026
- community-consensus — community reputation (no IFEval-style benchmark published for this model) @ Q4_0 · aggregated · 2026-08-30
Editorial score: chat-instruction reliability is a consistent community finding; QAT weights limit quant loss. No numeric benchmark backs this cell.
data checked Aug 2026
- quant-degradation-community — community consensus @ Q4_0 · aggregated · 2026-08-30
Editorial derivation from the vendor-reported full-band score; QAT limits quant loss.
data checked Aug 2026
- quant-degradation-community — community consensus @ Q4_0 · aggregated · 2026-08-30
Editorial derivation: small loss vs the vendor-reported full band.
— Not yet rated · contributions welcome —
FP16 — full precision (fp16/bf16)
24GB weights · e.g. bf16 + KV: 992MB at 8K · 2.5GB at 32K · 8.5GB at 128Kdata checked Aug 2026
- vendor-model-card — MATH (4-shot) = 43.3 @ bf16 · vendor-reported · 2026-08-30
GSM8K (8-shot): 71.0.
data checked Aug 2026
- vendor-model-card — HumanEval (0-shot) = 45.7 @ bf16 · vendor-reported · 2026-08-30
MBPP (3-shot): 60.4. Absent from the Aider polyglot leaderboard (the 27B scores 4.9% there).
data checked Aug 2026
- vectara-hallucination — HHEM hallucination rate % (lower is better) = 4.4 @ bf16 · aggregated · 2026-08-30
95.6% factual consistency, 97.4% answer rate.
data checked Oct 2026
- eqbench-creative — EQ-Bench Creative Writing v3 rubric score (0-20) = 14.63 @ bf16 · aggregated · 2026-10-04
Below Gemma 3 27B (15.33) and above Gemma 3 4B (13.91); Nemotron 3.5 Lightning's NVFP4 build is close (14.39). Normalised Elo 1125.5, 109th of 140, retrieved 2026-10-04. Independent; rubric judged by Claude Sonnet 4. EQ-Bench's normalised Elo moves as models are added, so the rubric score is the figure stored. Quant not stated by EQ-Bench (presumed full-precision API serving). Data file behind https://eqbench.com/creative_writing.html; the date the row was added is not published.
- eqbench-creative — EQ-Bench Creative Writing Longform score (0-100) = 41.8 @ bf16 · aggregated · 2026-10-04
Longform: level with Gemma 3 27B (41.9), so the gap between the two sizes is in short-form quality, not stamina. Data file behind the Longform page on eqbench.com. Quant not stated (presumed full precision).
§2 Known issues & what fixes them
Tool calling is unreliable at every precision — Gemma 3 was not trained with a strong native function-calling format, so agents built on it misfire.
Strict system prompts with exact JSON examples improve reliability somewhat.
For serious agent/MCP work, pick a Qwen3 model instead — this is a training gap, not a quant artifact.
Evidence · 2 sources · community-consensus
- BFCL — Gemma 3 12B is absent from the function-calling leaderboard entirely (checked 2026-08-30) (leaderboard, 2026-08-30)
- r/LocalLLaMA threads on Gemma function-calling reliability (reddit, 2026-08-30)
Already mid-pack at math, and the ≤3-bit cliff makes multi-step arithmetic untrustworthy.
Google's QAT Q4_0 build is only ~1GB larger and much stronger — there is little reason to run this model below 4-bit.
Offload arithmetic to a tool — but note this model's tool calling is itself weak, so verify your setup works.
Evidence · 2 sources · community-consensus
- llama.cpp quantization quality discussions (github, 2026-08-30)
- r/LocalLLaMA consensus on ≤3-bit degradation (reddit, 2026-08-30)