Known-issues sheets for local language models Rev 0.1.0 · reviewed every 7 days
llm-picker

Model datasheet · MiniCPM5

MiniCPM5 2B

The smallest model here that is not a compromise: 1.6GB at Q4_K_M, and OpenBMB's table puts it ahead of every 2B-class rival and level with several 4B ones on coding, math and tool use. A plain Llama-shaped dense transformer, so every runtime already handles it. The gap it cannot close is stored knowledge — 2.5B parameters hold what 2.5B parameters hold.

Vendor
OpenBMB
Architecture
Dense · 2.5B
Context
131,072 tokens
License
Apache-2.0
Released
2026-09-05
Vision
No

→ GGUF (llama.cpp / LM Studio / Ollama) → MLX (Apple Silicon)

Architecture, from its own config.json

42 layers · 42 full attention

16 attention heads over 2 KV heads (GQA 8:1), head dim 128 · dense feed-forward · vocabulary 130,560

Where the memory goes: weights by band above, plus a KV cache of 336MB at 8K · 1.3GB at 32K · 5.3GB at 128K, plus the runtime's own working memory — dominated by the output projection over that vocabulary.

Inside one layer, from its own tensors

42 × full-attention layer

input_layernormattentionq_projkvsoftmax(q·kᵀ/√d) · v16 query heads over 2 key/valueheadso_projpost_attention_layernormfeed-forwardgate_proj · up_proj → down_proj6,144 wide

The green line down the right of each half is the residual: it leaves the spine above the norm, skips the operation entirely and rejoins at the +. Where attention is drawn open, q, k and v are scaled to their real output widths — a model that shares key and value heads across many query heads shows one wide box beside two narrow ones.

Around the stack: an embedding of 130,560 × 2,048, a final norm, and a separate output projection.

On the norm names: despite what it is called, post_attention_layernorm is drawn after the residual add here, which is where this family applies it — it normalises the input to the feed-forward, not the output of attention. Models that carry a separate pre_feedforward_layernorm use it the other way, and are drawn that way.

§1 Characteristics by quantization band

Q2–Q3 — 2–3 bit

1.3GB weights · e.g. Q3_K_M + KV: 336MB at 8K · 1.3GB at 32K · 5.3GB at 128K
Math & reasoning 4/10
data checked Sep 2026
  • quant-degradation-community — editorial band derivation @ Q3_K_M · aggregated · 2026-09-12

    Editorial derivation: two points down, and only 0.3GB smaller than Q4_K_M — a bad trade at this size.

Coding 3/10
data checked Sep 2026
  • quant-degradation-community — editorial band derivation @ Q3_K_M · aggregated · 2026-09-12

    Editorial derivation: two points down.

Tool calling / agentic 3/10
data checked Sep 2026
  • quant-degradation-community — editorial band derivation @ Q3_K_M · aggregated · 2026-09-12

    Editorial derivation: two points down; structured output degrades fastest of all at 3-bit on a small dense model.

Q4–Q5 — 4–5 bit

1.6GB weights · e.g. Q4_K_M , mlx-4bit + KV: 336MB at 8K · 1.3GB at 32K · 5.3GB at 128K
Math & reasoning 7/10
data checked Sep 2026
  • quant-degradation-community — editorial band derivation @ Q4_K_M · aggregated · 2026-09-12

    Editorial derivation: one point down. Small dense models have the least redundancy to spend, and math goes first.

Coding 5/10
data checked Sep 2026
Tool calling / agentic 5/10
data checked Sep 2026
Long context 6/10
data checked Sep 2026
Instruction following 6/10
data checked Sep 2026
  • quant-degradation-community — editorial band derivation @ Q4_K_M · aggregated · 2026-09-12

    Editorial derivation: carried; format compliance survives 4-bit better than arithmetic does.

Q6–Q8 — 6–8 bit

2.7GB weights · e.g. Q8_0 , mlx-8bit + KV: 336MB at 8K · 1.3GB at 32K · 5.3GB at 128K
Math & reasoning 8/10
data checked Sep 2026
  • quant-degradation-community — editorial band derivation @ Q8_0 · aggregated · 2026-09-12

    Editorial derivation: carried from full precision. At 2.7GB this is the band to run if you have 8GB.

Coding 6/10
data checked Sep 2026
  • quant-degradation-community — editorial band derivation @ Q8_0 · aggregated · 2026-09-12

    Editorial derivation: carried from full precision.

Tool calling / agentic 6/10
data checked Sep 2026
  • quant-degradation-community — editorial band derivation @ Q8_0 · aggregated · 2026-09-12

    Editorial derivation: carried from full precision.

Long context 6/10
data checked Sep 2026
  • quant-degradation-community — editorial band derivation @ Q8_0 · aggregated · 2026-09-12

    Editorial derivation: carried; long context is not quant-sensitive here, but the cache is: 42 full-attention layers cost 43KB per token, so 128K of context is 5.5GB — more than three times the weights.

Instruction following 6/10
data checked Sep 2026
  • quant-degradation-community — editorial band derivation @ Q8_0 · aggregated · 2026-09-12

    Editorial derivation: carried from full precision.

FP16 — full precision (fp16/bf16)

5GB weights · e.g. fp16 + KV: 336MB at 8K · 1.3GB at 32K · 5.3GB at 128K
Math & reasoning 8/10
data checked Sep 2026
  • vendor-model-card — AIME 2026 = 86.5 @ bf16 · vendor-reported · 2026-09-12

    Vendor-reported, thinking mode. The 4B-class models in OpenBMB's own comparison set post 82.7 (Qwen3.5-4B) and 45.0 (Gemma-4-E4B); this is the strongest math result per gigabyte on this site.

  • vendor-model-card — MATH-500 = 94.6 @ bf16 · vendor-reported · 2026-09-12

    Secondary math check on the same table.

Coding 6/10
data checked Sep 2026
  • vendor-model-card — LiveCodeBench v6 = 69.1 @ bf16 · vendor-reported · 2026-09-12

    Ahead of every 2B and 4B model in OpenBMB's comparison set.

  • vendor-model-card — SWE-bench Verified = 46.4 @ bf16 · vendor-reported · 2026-09-12

    Agentic repository work. Held below the 27B-class models here — granite-4.2-30b posts 57.0 and scores 6 — but extraordinary for 2.5B parameters.

Tool calling / agentic 6/10
data checked Sep 2026
  • vendor-model-card — BFCL v4 = 66.6 @ bf16 · vendor-reported · 2026-09-12

    Function calling, vendor-reported. Comparable to Phi-4-mini's 70.3 on the older BFCL.

  • vendor-model-card — τ²-Bench Telecom = 97.1 @ bf16 · vendor-reported · 2026-09-12

    Multi-turn tool use in one narrow domain; the banking variant on the same table is 20.8, so read this as a ceiling rather than a typical result.

Instruction following 6/10
data checked Sep 2026
Long context 6/10
data checked Sep 2026
  • vendor-model-card — NoLiMa = 68.1 @ bf16 · vendor-reported · 2026-09-12

    Long-context retrieval without lexical overlap — the hard version. Every other model in OpenBMB's comparison set scores under 44, several under 6.

  • vendor-model-card — LongBench v2 = 43.7 @ bf16 · vendor-reported · 2026-09-12

    Second long-context check on the same table.

Factuality 4/10
data checked Sep 2026
  • vendor-model-card — MMLU-Redux = 84.7 @ bf16 · vendor-reported · 2026-09-12

    Knowledge proxy only — no hallucination benchmark lists this model. 2.5B parameters is a hard ceiling on stored facts, and MMLU-Pro on the same table is 70.8.

§2 Known issues & what fixes them

ISSUE-01 moderate Q2–Q3 / Q4–Q5 / Q6–Q8 / FP16 · tool-calling, instruction-following

Two separate things, and an earlier version of this entry conflated them. llama.cpp needs no MiniCPM5-specific template support: the chat template ships inside the GGUF (a PR adding one, #28595, was closed unmerged on 2026-09-08 by its author for that reason). The XML-flavoured tool calls do need the minicpm5 parser, merged in June. The real defect was in the template itself: a <tool_sep> branch could drop assistant text that followed a tool call (OpenBMB/MiniCPM#379, still open). OpenBMB fixed the template in the original-weights repo on 2026-09-29 (HF discussion #19, opened 2026-09-21). The official GGUF repo was last changed on 2026-09-12, so the template embedded in those files probably predates the fix — inferred from dates, not checked inside the file.

Fix
Update your runtime [partial]

Use a llama.cpp build with the minicpm5 tool-call parser (merged 2026-06-28). For tool use, override the embedded template with the fixed chat_template.jinja from openbmb/MiniCPM5-2B (--chat-template-file) until the GGUFs are re-uploaded.

Evidence · 4 sources · community-consensus
ISSUE-02 moderate Q2–Q3 / Q4–Q5 / Q6–Q8 / FP16 · instruction-following

Can fall into repetition loops in llama.cpp — one user reported an infinite loop on the official Q8_0 GGUF. OpenBMB's GGUF README (updated 2026-09-12) attributes this to llama.cpp's default min_p=0.05, which can discard exactly the low-probability tokens needed to break out of a loop; other runtimes such as SGLang default to 0.

Fix
Prompt technique [strong]

Set the sampler explicitly, as OpenBMB recommends: temperature 1.0, top_p 0.95, min_p 0.0 (--min-p 0 in llama.cpp). If repetition persists, add repetition_penalty 1.05.

Evidence · 2 sources · community-consensus
ISSUE-03 moderate Q2–Q3 / Q4–Q5 / Q6–Q8 / FP16 · factuality

2.5B parameters cannot hold much world knowledge, and the benchmark suite the vendor publishes is all reasoning and agentic work — no hallucination benchmark lists this model. The scores that make it remarkable are the ones where a tool or a retrieved document does the remembering.

Fix
RAG (retrieval) [strong]

Ground it in retrieved documents. Its long-context scores (NoLiMa 68.1) say it can actually use what you give it, which is the harder half.

Fix
MCP / external tools [strong]

Let it look things up rather than recall them.

Evidence · 1 source · anecdotal
ISSUE-04 mild Q2–Q3 / Q4–Q5 / Q6–Q8 / FP16 · coding, math, tool-calling, long-context, instruction-following

Released 2026-09-05. Every score here is OpenBMB's own, and their table marks several rows as reproduced internally rather than taken from the benchmark's own leaderboard. No independent listing exists yet.

No fix
No known fix — choose a different model

Not a defect — a caveat. Revisit when independent numbers land.

Evidence · 1 source · anecdotal
ISSUE-05 severe Q2–Q3 · math, coding, tool-calling

Q3_K_M saves 0.3GB over Q4_K_M and costs about two points everywhere that matters. A 2.5B dense model has no spare capacity for 3-bit to take: this is the clearest case on the site of a band not worth running.

Fix
Move up a quant band [strong]

Run Q4_K_M at 1.6GB, or Q8_0 at 2.7GB. Both fit machines that could hold the 3-bit file, so there is nothing to gain.

Fix
Code interpreter / calculator tool [weak]

If arithmetic is what broke, hand it to a tool — though tool calling is also down at this band, so this helps less than usual.

Evidence · 2 sources · community-consensus