Known-issues sheets for local language models Rev 0.1.0 · reviewed every 7 days

Model datasheet · Gemma 4

Gemma 4 E4B

Phone-class Gemma: Per-Layer Embeddings mean 8B of weights behave like 4.5B at inference, and the official QAT build is 5.15GB. Text, image and audio in the 8GB tier — but the benchmarks are honest about the ceiling.

Vendor
Google
Architecture
Dense · 8B
Context
131,072 tokens
License
Apache-2.0
Released
2026-07-02
Vision
Yes

→ GGUF (llama.cpp / LM Studio / Ollama) → MLX (Apple Silicon)

§1 Characteristics by quantization band

Q2–Q3 — 2–3 bit

4GB weights · e.g. Q3_K_M + KV: 132MB at 8K · 420MB at 32K · 1.5GB at 128K

— Not yet rated · contributions welcome —

Q4–Q5 — 4–5 bit

5GB weights · e.g. Q4_0 , Q4_K_M , mlx-4bit + KV: 132MB at 8K · 420MB at 32K · 1.5GB at 128K
Math & reasoning 3/10
data checked Aug 2026
  • quant-degradation-community — community consensus @ Q4_0 · aggregated · 2026-08-30

    Editorial derivation: QAT Q4_0 (5.15GB) is the intended build, but small models have less redundancy to lose — one point down.

Coding 3/10
data checked Aug 2026
  • quant-degradation-community — community consensus @ Q4_0 · aggregated · 2026-08-30

    Editorial derivation: carried down one point from the full band.

Tool calling / agentic 2/10
data checked Aug 2026
  • quant-degradation-community — community consensus @ Q4_0 · aggregated · 2026-08-30

    Editorial derivation: an already-marginal tool layer at 4-bit. Treat tool use as unsupported here.

Q6–Q8 — 6–8 bit

6.5GB weights · e.g. Q6_K + KV: 132MB at 8K · 420MB at 32K · 1.5GB at 128K

— Not yet rated · contributions welcome —

FP16 — full precision (fp16/bf16)

16GB weights · e.g. bf16 + KV: 132MB at 8K · 420MB at 32K · 1.5GB at 128K
Math & reasoning 4/10
data checked Aug 2026
  • vendor-model-cardAIME 2026 (no tools) = 42.5 @ bf16 · vendor-reported · 2026-08-30

    GPQA Diamond: 58.6, MMLU-Pro: 69.4. Strong for 4.5B effective parameters, weak in absolute terms.

Coding 4/10
data checked Aug 2026
  • vendor-model-cardLiveCodeBench v6 = 52 @ bf16 · vendor-reported · 2026-08-30

    Competition-style problems only; no agentic or repo-level evidence at this size.

Tool calling / agentic 3/10
data checked Aug 2026
Long context 2/10
data checked Aug 2026

§2 Known issues & what fixes them

ISSUE-01 moderate Q2–Q3 / Q4–Q5 / Q6–Q8 / FP16 · tool-calling

Tau2 42.2 against the 12B's 69.0 — by the vendor's own measurement, roughly two in five tool-use episodes go wrong. This is a capacity ceiling at 4.5B effective parameters, not a bug, and no runtime update moves it.

Fix
Prompt technique [partial]

One or two tools with unambiguous schemas, single-step calls, and validate every argument before executing. Multi-step agent loops are out of reach at this size.

No fix
No known fix — choose a different model [strong]

If the workload genuinely needs reliable tool use, move up to Gemma 4 12B (Tau2 69.0) — a 9GB rather than 7GB footprint.

Evidence · 1 source · anecdotal
ISSUE-02 severe Q2–Q3 / Q4–Q5 / Q6–Q8 / FP16 · long-context

The 128K context window is nominal: MRCR v2 at 128K is 25.4%, so three quarters of the needles are missed. Loading a long document and asking questions about it will silently produce confident, wrong answers.

Fix
RAG (retrieval) [strong]

Chunk and retrieve — feed it a page, not a book. This is the single highest-value mitigation for the E-series.

Evidence · 1 source · anecdotal
ISSUE-03 moderate Q2–Q3 / Q4–Q5 / Q6–Q8 · instruction-following

Inherits the family's local-runtime churn — official GGUF refreshes that would not load, and gibberish past ~230 tokens with KV offload enabled.

Fix
Update your runtime [strong]

Current llama.cpp, a re-cut community quant, and no -nkvo.

Evidence · 2 sources · community-consensus