Known-issues sheets for local language models Rev 0.1.0 · reviewed every 7 days
llm-picker

Model datasheet · Phi

Phi-4-mini

Strong math and code for a 3.8B — and the worst measured hallucination rate on this site. A capable calculator that should never be trusted on facts without retrieval.

Vendor
Microsoft
Architecture
Dense · 3.8B
Context
131,072 tokens
License
MIT
Released
2025-02-27
Vision
No

→ GGUF (llama.cpp / LM Studio / Ollama) → MLX (Apple Silicon)

Architecture, from its own config.json

32 layers · 32 full attention

24 attention heads over 8 KV heads (GQA 3:1), head dim 128 · dense feed-forward · vocabulary 200,064

Where the memory goes: weights by band above, plus a KV cache of 1GB at 8K · 4GB at 32K · 16GB at 128K, plus the runtime's own working memory — dominated by the output projection over that vocabulary.

Inside one layer, from its own tensors

32 × full-attention layer

input_layernormattentionqkv_proj — q, k and v in one tensorsoftmax(q·kᵀ/√d) · v24 query heads over 8 key/valueheadso_projpost_attention_layernormfeed-forwardgate_proj · up_proj → down_proj8,192 wide

The green line down the right of each half is the residual: it leaves the spine above the norm, skips the operation entirely and rejoins at the +. Where attention is drawn open, q, k and v are scaled to their real output widths — a model that shares key and value heads across many query heads shows one wide box beside two narrow ones.

Around the stack: an embedding of 200,064 × 3,072, a final norm, and an output projection that reuses the embedding (tied weights).

On the norm names: despite what it is called, post_attention_layernorm is drawn after the residual add here, which is where this family applies it — it normalises the input to the feed-forward, not the output of attention. Models that carry a separate pre_feedforward_layernorm use it the other way, and are drawn that way.

§1 Characteristics by quantization band

Q2–Q3 — 2–3 bit

2.1GB weights · e.g. Q3_K_M + KV: 1GB at 8K · 4GB at 32K · 16GB at 128K

— Not yet rated · contributions welcome —

Q4–Q5 — 4–5 bit

2.5GB weights · e.g. Q4_K_M , mlx-4bit + KV: 1GB at 8K · 4GB at 32K · 16GB at 128K
Math & reasoning 5/10
data checked Aug 2026
  • quant-degradation-community — KLD/perplexity delta vs bf16 @ Q4_K_M · aggregated · 2026-08-30

    Editorial derivation: small models lose proportionally more at 4-bit.

Coding 5/10
data checked Aug 2026
  • quant-degradation-community — KLD/perplexity delta vs bf16 @ Q4_K_M · aggregated · 2026-08-30

    Editorial derivation: small loss at 4-bit.

Tool calling / agentic 5/10
data checked Aug 2026

Q6–Q8 — 6–8 bit

3.2GB weights · e.g. Q6_K , Q8_0 + KV: 1GB at 8K · 4GB at 32K · 16GB at 128K

— Not yet rated · contributions welcome —

FP16 — full precision (fp16/bf16)

7.7GB weights · e.g. fp16 + KV: 1GB at 8K · 4GB at 32K · 16GB at 128K
Math & reasoning 6/10
data checked Aug 2026
  • vendor-model-card — MATH = 64 @ bf16 · vendor-reported · 2026-08-30

    GSM8K: 88.6 — best-in-class for 4B.

Coding 6/10
data checked Aug 2026
  • vendor-model-card — HumanEval = 74.4 @ bf16 · vendor-reported · 2026-08-30

    MBPP: 65.3, BigCodeBench-Complete: 43.0 (tech report).

Instruction following 5/10
data checked Aug 2026
Tool calling / agentic 6/10
data checked Aug 2026
Factuality 2/10
data checked Aug 2026

§2 Known issues & what fixes them

ISSUE-01 severe Q2–Q3 / Q4–Q5 / Q6–Q8 / FP16 · factuality

Hallucinates at 23.5% even when summarizing text it was handed (Vectara HHEM) — roughly five times the rate of same-size peers. Its fluent confidence makes this worse.

Fix
RAG (retrieval) [partial]

Grounding helps but does not fix it — verify anything factual it produces.

No fix
No known fix — choose a different model

For factual Q&A, use Gemma 3 4B or Qwen3-4B instead; keep Phi-4-mini for math and code.

Evidence · 1 source · anecdotal