Model datasheet · Gemma 4
Gemma 4 31B
The Gemma 4 family's dense flagship, and much stronger at long context than its own 26B sibling: 66.4% MRCR at 128K against 44.1%. Official QAT Q4_0 is 17.7GB, so the whole thing fits a 32GB machine.
- Vendor
- Architecture
- Dense · 30.7B
- Context
- 262,144 tokens
- License
- Apache-2.0
- Released
- 2026-03-31
- Vision
- Yes
→ GGUF (llama.cpp / LM Studio / Ollama) → MLX (Apple Silicon)
Architecture, from its own config.json
60 layers · 50 sliding window · 10 full attention
32 attention heads over 16 KV heads (GQA 2:1), head dim 256 — its full-attention layers use a different shape again, 4 KV heads of 512, which is the shape that decides the cache at long context · dense feed-forward · vocabulary 262,144
Where the memory goes: weights by band above, plus a KV cache of 1.8GB at 8K · 3.7GB at 32K · 11.2GB at 128K, plus the runtime's own working memory — dominated by the output projection over that vocabulary.
Inside one layer, from its own tensors
50 × sliding-window layer
10 × full-attention layer
The green line down the right of each half is the residual: it leaves the spine above the norm, skips the operation entirely and rejoins at the +. Where attention is drawn open, q, k and v are scaled to their real output widths — a model that shares key and value heads across many query heads shows one wide box beside two narrow ones.
Around the stack: an embedding of 262,144 × 5,376, a final norm, and an output projection that reuses the embedding (tied weights).
Also in every block and not drawn, because their tensor names do not fix where they belong: layer_scalar. Guessing an order is the one thing this diagram will not do.
§1 Characteristics by quantization band
— Not yet rated · contributions welcome —
Q4–Q5 — 4–5 bit
18GB weights · e.g. Q4_0 , Q4_K_M , mlx-4bit + KV: 1.8GB at 8K · 3.7GB at 32K · 11.2GB at 128Kdata checked Aug 2026
- quant-degradation-community — community consensus @ Q4_0 · aggregated · 2026-08-30
Editorial derivation: QAT Q4_0 is Google's intended local build (17.7GB); small loss vs full. Mirrors the derivation used for gemma-4-26b-a4b.
data checked Aug 2026
- quant-degradation-community — community consensus @ Q4_0 · aggregated · 2026-08-30
Editorial derivation: QAT limits quant loss.
data checked Aug 2026
- quant-degradation-community — community consensus @ Q4_0 · aggregated · 2026-08-30
Editorial derivation: carried down one point, consistent with the family.
— Not yet rated · contributions welcome —
FP16 — full precision (fp16/bf16)
61GB weights · e.g. bf16 + KV: 1.8GB at 8K · 3.7GB at 32K · 11.2GB at 128Kdata checked Aug 2026
- vendor-model-card — AIME 2026 (no tools) = 89.2 @ bf16 · vendor-reported · 2026-08-30
GPQA Diamond: 84.3, MMLU-Pro: 85.2. Marginally ahead of the 26B-A4B sibling (88.3) — same editorial score, since the gap is inside vendor-reporting noise and neither has independent replication.
data checked Aug 2026
- vendor-model-card — LiveCodeBench v6 = 80 @ bf16 · vendor-reported · 2026-08-30
Codeforces ELO 2150. Absent from Aider polyglot as of 2026-08-30, so no real-world code-editing check exists yet — the dimension that historically embarrassed Gemma 3.
data checked Aug 2026
- vendor-model-card — Tau2 average (with tools) = 76.9 @ bf16 · vendor-reported · 2026-08-30
Best Tau2 in the family. Held at 7 rather than 8 because llama.cpp users report duplicated tool calls on this size — see weaknesses/gemma-4-31b.json.
data checked Aug 2026
- vendor-model-card — MRCR v2 @128K (8 needle) = 66.4 @ bf16 · vendor-reported · 2026-08-30
The standout number in this catalog: 66.4 where the 26B-A4B scores 44.1 and most models here are unrated. Still not a licence to fill 256K — a third of the needles are missed at half the window.
data checked Oct 2026
- eqbench-creative — EQ-Bench Creative Writing v3 rubric score (0–20) = 16.01 · aggregated · 2026-10-04
Independent; row google/gemma-4-31B-it. Rubric rank 66 of 140 (board median 15.96); by Elo (1368.2, relative, judged pairwise by Claude Sonnet 4.6) it ranks 85. The rubric is the stabler figure and is what is stored. Quant and mode not stated (presumed full-precision API). Data file behind https://eqbench.com/creative_writing.html.
- eqbench-creative — EQ-Bench Creative Writing Longform score (0–100) = 56.5 · aggregated · 2026-10-04
Rank 80 of 140 on the longform board (median 59.1). Mid-table prose: solid rubric scores, weaker in pairwise comparison and at chapter length.
data checked Oct 2026
- vectara-hallucination — HHEM hallucination rate % (lower is better) = 7.4 @ bf16 · aggregated · 2026-10-04
Independent; row google/gemma-4-31b-it, README 'Last updated on September 22, 2026' (rows present since at least the 2026-05-11 README commit). Factual consistency 92.6%, answer rate 100.0%. Summarisation faithfulness at temperature 0 via API; mode not stated. Higher than its own 26B-A4B sibling (5.2) and identical to Gemma 3 27B (7.4), despite the stronger MMLU-Pro (85.2) — HHEM measures faithfulness to a source document, not stored knowledge.
§2 Known issues & what fixes them
Emits the same tool call twice back-to-back, and sometimes closes a thinking block only to immediately open another right before tool handling breaks. Reported on both the 31B and its 26B-A4B sibling — slightly less often on the 31B — so agent harnesses need to be idempotent or de-duplicate. The llama.cpp issue cited here was closed on 2026-05-10 after parser fixes; a separate streaming tool-call report against the family opened on 2026-09-29 (see the streaming entry), so the parser is still settling.
Track current llama.cpp; the Gemma 4 tool-call and template handling has been patched repeatedly since launch.
Make the tools themselves idempotent, or have the harness drop an identical repeat call, so a duplicate is harmless rather than a double write.
Evidence · 2 sources · community-consensus
- Eval bug: More Gemma4 edge cases (duplicate tool calls, thinking-block restarts) (github, 2026-08-30)
- Problems with using the model in llama.cpp (other, 2026-08-30)
Local Gemma 4 builds have been a moving target. Google's refreshed official GGUFs (2026-07-15) would not load in LM Studio 0.4.19 at all, and with KV-offload (-nkvo) enabled the family emits gibberish after roughly 230 tokens. Both look like model defects and are not. The churn continued in September: after a llama.cpp update the 31B ran out of Metal memory on a 128GB Mac, traced to an auto-fit change (#28849, merged 2026-09-16) that sized the starting context from the training context; it was reverted on 2026-09-26 (#29437).
Use a llama.cpp build newer than the -nkvo regression and a re-uploaded quant (unsloth and bartowski both re-cut theirs after the upstream fixes). Avoid -nkvo on Gemma 4 until the issue closes.
Evidence · 5 sources · community-consensus
- Latest official Google Gemma 4 GGUF models fail to load in LM Studio 0.4.19 (github, 2026-08-30)
- Eval bug: Gemma4 models produce gibberish at some point with -nkvo (github, 2026-08-30)
- New uploads add llama.cpp fixes (other, 2026-08-30)
- macOS Metal OOM on Gemma 4 31B after update (large default n_ctx) (github, 2026-10-04)
- Revert "Change max context length for auto-fitting with unified KV" (merged 2026-09-26) (github, 2026-10-04)
Best-in-catalog long-context retrieval is still only 66.4% MRCR v2 at 128K — on a model advertised at 256K. A third of the needles are missed at half the window.
Retrieve the relevant passages instead of pasting the corpus. Cheaper and more accurate than relying on the tail of a 256K window.
Evidence · 1 source · anecdotal
- Gemma 4 model card — MRCR v2 @128K per variant (vendor, 2026-08-30)
An open llama.cpp report (2026-09-29, Gemma 4 26B MoE, Vulkan) describes unstable streaming tool calls when arguments span several lines or the model uses unquoted keys — edits and file writes come out malformed. The parser maintainer is reworking the Gemma 4 parser (PR #29307, open) but was sceptical of the reporter's patch, and the reporter could not reproduce the parse failures on master on 2026-10-02. Treat as unconfirmed, and as a llama.cpp parser issue rather than a model defect.
Track llama.cpp for the Gemma 4 parser rework (PR #29307). Until then, prefer non-streaming tool calls in agent harnesses if multi-line arguments misparse — a workaround inferred from the report, not maintainer-verified.
Evidence · 2 sources · anecdotal
In multi-tool-call turns, the reasoning trace can grow progressively longer trailing garbage until the server returns an error. The llama.cpp parser maintainer traced it to a leading newline the chat template injects before the end-of-thought marker on later turns, which the model then imitates. Reported on a Q8_0 31B; the issue is open.
Use the edited chat template from the thread, which trims the thinking text before closing the channel. The reporter confirmed it fixed the problem; it has not been merged into Google's template.
Evidence · 1 source · anecdotal
- Eval bug: gemma4 thinking starts to emit progressively long trailing garbage (github, 2026-10-04)