Model datasheet · Granite 4.2
Granite 4.2 8B
The tool-calling specialist of the 8GB tier: Apache-2.0, a thinking switch, and function-calling trained in live sandboxes rather than scraped transcripts. Its predecessor's 8B matched a 32B MoE across ten benchmarks — this is that lineage, five days old.
- Vendor
- IBM
- Architecture
- Dense · 9B
- Context
- 131,072 tokens
- License
- Apache-2.0
- Released
- 2026-08-25
- Vision
- No
→ GGUF (llama.cpp / LM Studio / Ollama) → MLX (Apple Silicon)
§1 Characteristics by quantization band
— Not yet rated · contributions welcome —
Q4–Q5 — 4–5 bit
5.5GB weights · e.g. Q4_K_M , mlx-4bit + KV: 1.3GB at 8K · 5GB at 32K · 20GB at 128Kdata checked Aug 2026
- quant-degradation-community — community consensus @ Q4_K_M · aggregated · 2026-08-30
Editorial derivation: one point down. Q4_K_M is 5.54GB — comfortably inside the 8GB tier.
data checked Aug 2026
- quant-degradation-community — community consensus @ Q4_K_M · aggregated · 2026-08-30
Editorial derivation: one point down; smaller models have less headroom for the token-level precision that structured calls need.
data checked Aug 2026
- quant-degradation-community — community consensus @ Q6_K · aggregated · 2026-08-30
Editorial derivation: carried from the full band.
data checked Aug 2026
- quant-degradation-community — community consensus @ Q6_K · aggregated · 2026-08-30
Editorial derivation: carried from the full band. At 7.52GB this is the band to run if you have 16GB.
FP16 — full precision (fp16/bf16)
18GB weights · e.g. bf16 + KV: 1.3GB at 8K · 5GB at 32K · 20GB at 128Kdata checked Aug 2026
- vendor-model-card — SWE-bench Verified = 47.67 @ bf16 · vendor-reported · 2026-08-30
An 8B-class model resolving nearly half of SWE-bench Verified is notable for the size; still far below the 27B-class models here. Vendor-reported.
data checked Aug 2026
- vendor-model-card — BFCL v3 = 68.27 @ bf16 · vendor-reported · 2026-08-30
The reason to pick this model at 8B: function calling is the design goal, not an afterthought. Independent BFCL listing pending as of 2026-08-30.
data checked Aug 2026
- vendor-model-card — MMLU = 73.8 @ bf16 · vendor-reported · 2026-08-30
Knowledge proxy only — no hallucination benchmark lists this model yet. Nine billion parameters is a hard ceiling on stored facts.
§2 Known issues & what fixes them
Released 2026-08-25. Every score here is IBM's own — including the BFCL v3 number, which is the model's main selling point and comes from a benchmark whose public leaderboard has not yet listed it.
Not a defect — a caveat. Revisit when independent numbers land.
Evidence · 1 source · anecdotal
- Granite 4.2 LLMs: How They're Built (IBM's own benchmark reporting) (vendor, 2026-08-30)
Granite 4 chat-template auto-detection has failed in llama.cpp, and the Granite 4 line also has an open report of quantized builds failing at very large contexts. A mis-detected template degrades tool calling quietly — the model looks dumber than it is.
Pass the Granite chat template explicitly instead of relying on auto-detection, and keep llama.cpp current.
Evidence · 2 sources · community-consensus
- Eval bug: Granite 4 template detection fails (github, 2026-08-30)
- Eval bug: granite 4 quantized builds fail on very large context (github, 2026-08-30)
MMLU 73.8 against the 30B sibling's 80.2. Nine billion parameters caps how much world knowledge it can hold, and no hallucination benchmark lists it, so the factuality score is a knowledge proxy rather than a measurement.
Ground it in retrieved documents — this model is built for enterprise RAG and tool use, which is exactly the workload that avoids the ceiling.
Evidence · 1 source · anecdotal
- Granite 4.2 LLMs: How They're Built (MMLU per variant) (vendor, 2026-08-30)