Known-issues sheets for local language models Rev 0.1.0 · reviewed every 7 days
llm-picker

Model datasheet · Granite 4.2

Granite 4.2 30B

IBM's enterprise reasoner with a thinking switch — one checkpoint that either reasons step by step or answers directly. Trained to use tools inside sandboxed environments rather than on transcripts of tool use, which shows in its function-calling scores. Five days old at time of writing; every number here is IBM's.

Vendor
IBM
Architecture
Dense · 29B
Context
131,072 tokens
License
Apache-2.0
Released
2026-08-25
Vision
No

→ GGUF (llama.cpp / LM Studio / Ollama) → MLX (Apple Silicon)

§1 Characteristics by quantization band

Q2–Q3 — 2–3 bit

14GB weights · e.g. Q3_K_M + KV: 2GB at 8K · 8GB at 32K · 32GB at 128K

— Not yet rated · contributions welcome —

Q4–Q5 — 4–5 bit

18GB weights · e.g. Q4_K_M , mlx-4bit + KV: 2GB at 8K · 8GB at 32K · 32GB at 128K
Coding 5/10
data checked Aug 2026
  • quant-degradation-community — community consensus @ Q4_K_M · aggregated · 2026-08-30

    Editorial derivation: one point down from full. Q4_K_M is 18.03GB — the band that fits a 32GB machine.

Tool calling / agentic 6/10
data checked Aug 2026
  • quant-degradation-community — community consensus @ Q4_K_M · aggregated · 2026-08-30

    Editorial derivation: one point down; structured output degrades before prose does.

Q6–Q8 — 6–8 bit

24GB weights · e.g. Q6_K + KV: 2GB at 8K · 8GB at 32K · 32GB at 128K
Coding 6/10
data checked Aug 2026
  • quant-degradation-community — community consensus @ Q6_K · aggregated · 2026-08-30

    Editorial derivation: carried from full. Note the open llama.cpp report of Q6/Q8 Granite 4 failing at very large contexts.

Tool calling / agentic 7/10
data checked Aug 2026
  • quant-degradation-community — community consensus @ Q6_K · aggregated · 2026-08-30

    Editorial derivation: carried from the full band.

FP16 — full precision (fp16/bf16)

59GB weights · e.g. bf16 + KV: 2GB at 8K · 8GB at 32K · 32GB at 128K
Coding 6/10
data checked Aug 2026
  • vendor-model-card — SWE-bench Verified = 57 @ bf16 · vendor-reported · 2026-08-30

    Well behind Qwen3.6 27B (77.2) at a comparable footprint — Granite's pitch is governance and tool reliability, not raw coding. Vendor-reported, five days after release, no independent replication.

Tool calling / agentic 7/10
data checked Aug 2026
  • vendor-model-card — BFCL v3 = 73.68 @ bf16 · vendor-reported · 2026-08-30

    Leads the Granite line on BFCL v3. IBM trained the 8B and 30B to act with tools inside sandboxed environments rather than on transcripts — treat as a vendor claim until the Berkeley leaderboard lists it independently.

Factuality 6/10
data checked Aug 2026
  • vendor-model-card — MMLU = 80.2 @ bf16 · vendor-reported · 2026-08-30

    Pre-trained on ~15T tokens over five phases. Absent from Vectara HHEM as of 2026-08-30, so this is a knowledge proxy, not a hallucination measurement.

§2 Known issues & what fixes them

ISSUE-01 mild Q2–Q3 / Q4–Q5 / Q6–Q8 / FP16 · coding, tool-calling

Released 2026-08-25 — five days before this entry was written. Every score here is IBM's own; no independent leaderboard lists it, and the BFCL v3 figure in particular is a vendor number for a benchmark that has a public leaderboard it has not yet appeared on.

No fix
No known fix — choose a different model

Not a defect — a caveat. Scores will be revisited when independent measurements land.

Evidence · 1 source · anecdotal
ISSUE-02 moderate Q6–Q8 / FP16 · long-context

The Granite 4 line has an open llama.cpp report of Q6/Q8 builds failing outright at very large contexts (100K–200K) — which is most of the advertised 131K window. Separately, Granite 4 chat-template auto-detection has failed in llama.cpp, producing subtly wrong prompting rather than an error.

Fix
Update your runtime [partial]

Track current llama.cpp and pass the chat template explicitly rather than relying on auto-detection. Both reports are runtime-side, not weights-side.

Fix
RAG (retrieval) [strong]

Retrieve rather than filling 100K+ of context — this sidesteps the failure entirely.

Evidence · 2 sources · community-consensus