Model datasheet · Gemma 3
Gemma 3 12B
The conversationalist with eyes. Natural prose, good multilingual chat, and built-in image understanding. Weaker than same-size rivals at code and tool calling — pick it for writing and vision, not agents.
- Vendor
- Architecture
- Dense · 12.2B
- Context
- 131,072 tokens
- License
- Gemma Terms of Use
- Released
- 2025-03-12
- Vision
- Yes
→ GGUF (llama.cpp / LM Studio / Ollama) → MLX (Apple Silicon)
§1 Characteristics by quantization band
data checked Aug 2026
- quant-degradation-community — degradation reports @ Q3_K_M · aggregated · 2026-08-30
Editorial derivation: standard ≤3-bit math cliff.
data checked Aug 2026
- quant-degradation-community — community consensus @ Q3_K_M · aggregated · 2026-08-30
Editorial derivation: prose survives low quants better than reasoning does.
Q4–Q5 — 4–5 bit
7GB weights · e.g. Q4_0 , Q4_K_M , mlx-4bit + KV: 832MB at 8K · 2.3GB at 32K · 8.3GB at 128Kdata checked Aug 2026
- quant-degradation-community — community consensus @ Q4_0 · aggregated · 2026-08-30
Editorial derivation: Google ships QAT Q4_0 weights trained to preserve quality; prose holds up well at 4-bit.
data checked Aug 2026
- community-consensus — community reputation (no IFEval-style benchmark published for this model) @ Q4_0 · aggregated · 2026-08-30
Editorial score: chat-instruction reliability is a consistent community finding; QAT weights limit quant loss. No numeric benchmark backs this cell.
data checked Aug 2026
- quant-degradation-community — community consensus @ Q4_0 · aggregated · 2026-08-30
Editorial derivation from the vendor-reported full-band score; QAT limits quant loss.
data checked Aug 2026
- quant-degradation-community — community consensus @ Q4_0 · aggregated · 2026-08-30
Editorial derivation: small loss vs the vendor-reported full band.
— Not yet rated · contributions welcome —
FP16 — full precision (fp16/bf16)
24GB weights · e.g. bf16 + KV: 832MB at 8K · 2.3GB at 32K · 8.3GB at 128Kdata checked Aug 2026
- vendor-model-card — MATH (4-shot) = 43.3 @ bf16 · vendor-reported · 2026-08-30
GSM8K (8-shot): 71.0.
data checked Aug 2026
- vendor-model-card — HumanEval (0-shot) = 45.7 @ bf16 · vendor-reported · 2026-08-30
MBPP (3-shot): 60.4. Absent from the Aider polyglot leaderboard (the 27B scores 4.9% there).
data checked Aug 2026
- vectara-hallucination — HHEM hallucination rate % (lower is better) = 4.4 @ bf16 · aggregated · 2026-08-30
95.6% factual consistency, 97.4% answer rate.
data checked Aug 2026
- community-consensus — community reputation (no public benchmark currently lists this model) @ bf16 · aggregated · 2026-08-30
Editorial score: Gemma 3's prose quality is a consistent community finding, but the model is absent from the current EQ-Bench Creative Writing v3 list (checked 2026-08-30). Treat as unverified-by-benchmark.
§2 Known issues & what fixes them
Tool calling is unreliable at every precision — Gemma 3 was not trained with a strong native function-calling format, so agents built on it misfire.
Strict system prompts with exact JSON examples improve reliability somewhat.
For serious agent/MCP work, pick a Qwen3 model instead — this is a training gap, not a quant artifact.
Evidence · 2 sources · community-consensus
- BFCL — Gemma 3 12B is absent from the function-calling leaderboard entirely (checked 2026-08-30) (leaderboard, 2026-08-30)
- r/LocalLLaMA threads on Gemma function-calling reliability (reddit, 2026-08-30)
Already mid-pack at math, and the ≤3-bit cliff makes multi-step arithmetic untrustworthy.
Google's QAT Q4_0 build is only ~1GB larger and much stronger — there is little reason to run this model below 4-bit.
Offload arithmetic to a tool — but note this model's tool calling is itself weak, so verify your setup works.
Evidence · 2 sources · community-consensus
- llama.cpp quantization quality discussions (github, 2026-08-30)
- r/LocalLLaMA consensus on ≤3-bit degradation (reddit, 2026-08-30)