Model datasheet · Gemma 4
Gemma 4 12B
The 'Unified' one: an encoder-free architecture that takes image and audio straight into the embedding space, at a size that leaves room to breathe on a 16GB machine. Punches near the 26B MoE on reasoning while being the only model here that hears.
- Vendor
- Architecture
- Dense · 11.95B
- Context
- 262,144 tokens
- License
- Apache-2.0
- Released
- 2026-07-02
- Vision
- Yes
→ GGUF (llama.cpp / LM Studio / Ollama) → MLX (Apple Silicon)
§1 Characteristics by quantization band
— Not yet rated · contributions welcome —
Q4–Q5 — 4–5 bit
7GB weights · e.g. Q4_0 , Q4_K_M , mlx-4bit + KV: 832MB at 8K · 2.3GB at 32K · 8.3GB at 128Kdata checked Aug 2026
- quant-degradation-community — community consensus @ Q4_0 · aggregated · 2026-08-30
Editorial derivation: QAT Q4_0 is the intended local build; one point for quant loss, consistent with the family.
data checked Aug 2026
- quant-degradation-community — community consensus @ Q4_0 · aggregated · 2026-08-30
Editorial derivation: QAT limits quant loss.
data checked Aug 2026
- quant-degradation-community — community consensus @ Q4_0 · aggregated · 2026-08-30
Editorial derivation: carried from the full band.
— Not yet rated · contributions welcome —
FP16 — full precision (fp16/bf16)
24GB weights · e.g. bf16 + KV: 832MB at 8K · 2.3GB at 32K · 8.3GB at 128Kdata checked Aug 2026
- vendor-model-card — AIME 2026 (no tools) = 77.5 @ bf16 · vendor-reported · 2026-08-30
GPQA Diamond: 78.8, MMLU-Pro: 77.2. A 12B scoring 77.5 on AIME 2026 is the headline of this release.
data checked Aug 2026
- vendor-model-card — LiveCodeBench v6 = 72 @ bf16 · vendor-reported · 2026-08-30
Vendor-reported; no Aider polyglot entry as of 2026-08-30.
data checked Aug 2026
- vendor-model-card — Tau2 average (with tools) = 69 @ bf16 · vendor-reported · 2026-08-30
Essentially level with the 26B-A4B (68.2) at half the footprint.
data checked Aug 2026
- vendor-model-card — MRCR v2 @128K (8 needle) = 43.4 @ bf16 · vendor-reported · 2026-08-30
Same story as the 26B-A4B: a 256K window whose retrieval is already shaky at 128K.
§2 Known issues & what fixes them
The 12B ships a Multi-Token-Prediction drafter for speculative decoding, and that path has its own bug reports in llama.cpp — enabling it (`-fit on`) can break generation on this size specifically. The speed feature is the risk, not the model.
Run without MTP/speculative decoding until the issue closes, or track a llama.cpp build that has the Gemma4 MTP fix. Plain Q4_0 decoding is unaffected.
Evidence · 2 sources · community-consensus
- Eval bug: Gemma4 MTP (github, 2026-08-30)
- unsloth Gemma 4 12B QAT GGUF — ships the MTP drafter for speculative decoding (other, 2026-08-30)
Shares the family's rocky local-runtime history: Google's refreshed official GGUFs (2026-07-15) failed to load in LM Studio 0.4.19, and the family produces gibberish past ~230 tokens when KV offload (-nkvo) is on.
Take a re-cut community quant and a current llama.cpp; don't run Gemma 4 with -nkvo.
Evidence · 2 sources · community-consensus
- Latest official Google Gemma 4 GGUF models fail to load in LM Studio 0.4.19 (github, 2026-08-30)
- Eval bug: Gemma4 models produce gibberish at some point with -nkvo (github, 2026-08-30)
The 256K window oversells it: retrieval accuracy at 128K is 43.4% (vendor's own MRCR v2 number), essentially the same as the 26B-A4B. Don't stuff the window and expect recall.
Retrieve and paste the relevant chunk rather than the whole corpus.
Evidence · 1 source · anecdotal
- Gemma 4 model card — MRCR v2 @128K per variant (vendor, 2026-08-30)