Known-issues sheets for local language models Rev 0.1.0 · reviewed every 7 days
llm-picker

Model datasheet · Gemma 4

Gemma 4 31B

The Gemma 4 family's dense flagship, and the best long-context model in this catalog by a wide margin — 66.4% MRCR at 128K where its own 26B sibling manages 44.1%. Official QAT Q4_0 is 17.7GB, so the whole thing fits a 32GB machine.

Vendor
Google
Architecture
Dense · 30.7B
Context
262,144 tokens
License
Apache-2.0
Released
2026-07-02
Vision
Yes

→ GGUF (llama.cpp / LM Studio / Ollama) → MLX (Apple Silicon)

§1 Characteristics by quantization band

Q2–Q3 — 2–3 bit

15GB weights · e.g. Q3_K_M + KV: 2GB at 8K · 5.8GB at 32K · 20.8GB at 128K

— Not yet rated · contributions welcome —

Q4–Q5 — 4–5 bit

18GB weights · e.g. Q4_0 , Q4_K_M , mlx-4bit + KV: 2GB at 8K · 5.8GB at 32K · 20.8GB at 128K
Math & reasoning 7/10
data checked Aug 2026
  • quant-degradation-community — community consensus @ Q4_0 · aggregated · 2026-08-30

    Editorial derivation: QAT Q4_0 is Google's intended local build (17.7GB); small loss vs full. Mirrors the derivation used for gemma-4-26b-a4b.

Coding 7/10
data checked Aug 2026
Tool calling / agentic 6/10
data checked Aug 2026
  • quant-degradation-community — community consensus @ Q4_0 · aggregated · 2026-08-30

    Editorial derivation: carried down one point, consistent with the family.

Q6–Q8 — 6–8 bit

25GB weights · e.g. Q6_K + KV: 2GB at 8K · 5.8GB at 32K · 20.8GB at 128K

— Not yet rated · contributions welcome —

FP16 — full precision (fp16/bf16)

61GB weights · e.g. bf16 + KV: 2GB at 8K · 5.8GB at 32K · 20.8GB at 128K
Math & reasoning 8/10
data checked Aug 2026
  • vendor-model-card — AIME 2026 (no tools) = 89.2 @ bf16 · vendor-reported · 2026-08-30

    GPQA Diamond: 84.3, MMLU-Pro: 85.2. Marginally ahead of the 26B-A4B sibling (88.3) — same editorial score, since the gap is inside vendor-reporting noise and neither has independent replication.

Coding 8/10
data checked Aug 2026
  • vendor-model-card — LiveCodeBench v6 = 80 @ bf16 · vendor-reported · 2026-08-30

    Codeforces ELO 2150. Absent from Aider polyglot as of 2026-08-30, so no real-world code-editing check exists yet — the dimension that historically embarrassed Gemma 3.

Tool calling / agentic 7/10
data checked Aug 2026
  • vendor-model-card — Tau2 average (with tools) = 76.9 @ bf16 · vendor-reported · 2026-08-30

    Best Tau2 in the family. Held at 7 rather than 8 because llama.cpp users report duplicated tool calls on this size — see weaknesses/gemma-4-31b.json.

Long context 6/10
data checked Aug 2026
  • vendor-model-card — MRCR v2 @128K (8 needle) = 66.4 @ bf16 · vendor-reported · 2026-08-30

    The standout number in this catalog: 66.4 where the 26B-A4B scores 44.1 and most models here are unrated. Still not a licence to fill 256K — a third of the needles are missed at half the window.

§2 Known issues & what fixes them

ISSUE-01 moderate Q2–Q3 / Q4–Q5 / Q6–Q8 / FP16 · tool-calling

Emits the same tool call twice back-to-back, and sometimes closes a thinking block only to immediately open another right before tool handling breaks. Reported on both the 31B and its 26B-A4B sibling — slightly less often on the 31B — so agent harnesses need to be idempotent or de-duplicate.

Fix
Update your runtime [partial]

Track current llama.cpp; the Gemma 4 tool-call and template handling has been patched repeatedly since launch.

Fix
MCP / external tools [strong]

Make the tools themselves idempotent, or have the harness drop an identical repeat call, so a duplicate is harmless rather than a double write.

Evidence · 2 sources · community-consensus
ISSUE-02 severe Q2–Q3 / Q4–Q5 / Q6–Q8 · instruction-following, coding

Local Gemma 4 builds have been a moving target. Google's refreshed official GGUFs (2026-07-15) would not load in LM Studio 0.4.19 at all, and with KV-offload (-nkvo) enabled the family emits gibberish after roughly 230 tokens. Both look like model defects and are not.

Fix
Update your runtime [strong]

Use a llama.cpp build newer than the -nkvo regression and a re-uploaded quant (unsloth and bartowski both re-cut theirs after the upstream fixes). Avoid -nkvo on Gemma 4 until the issue closes.

Evidence · 3 sources · community-consensus
ISSUE-03 mild Q2–Q3 / Q4–Q5 / Q6–Q8 / FP16 · long-context

Best-in-catalog long-context retrieval is still only 66.4% MRCR v2 at 128K — on a model advertised at 256K. A third of the needles are missed at half the window.

Fix
RAG (retrieval) [strong]

Retrieve the relevant passages instead of pasting the corpus. Cheaper and more accurate than relying on the tail of a 256K window.

Evidence · 1 source · anecdotal