Model datasheet · Phi
Phi-4
The exam ace with no life experience: superb STEM reasoning per GB, near-zero world knowledge, and only a 16K window. Excellent at closed problems you paste in; pair with retrieval for everything else.
- Vendor
- Microsoft
- Architecture
- Dense · 14B
- Context
- 16,384 tokens
- License
- MIT
- Released
- 2024-12-12
- Vision
- No
→ GGUF (llama.cpp / LM Studio / Ollama) → MLX (Apple Silicon)
Architecture, from its own config.json
40 layers · 40 full attention
40 attention heads over 10 KV heads (GQA 4:1), head dim 128 · dense feed-forward · vocabulary 100,352
Where the memory goes: weights by band above, plus a KV cache of 1.6GB at 8K, plus the runtime's own working memory — dominated by the output projection over that vocabulary.
Inside one layer, from its own tensors
40 × full-attention layer
The green line down the right of each half is the residual: it leaves the spine above the norm, skips the operation entirely and rejoins at the +. Where attention is drawn open, q, k and v are scaled to their real output widths — a model that shares key and value heads across many query heads shows one wide box beside two narrow ones.
Around the stack: an embedding of 100,352 × 5,120, a final norm, and a separate output projection.
On the norm names: despite what it is called, post_attention_layernorm is drawn after the residual add here, which is where this family applies it — it normalises the input to the feed-forward, not the output of attention. Models that carry a separate pre_feedforward_layernorm use it the other way, and are drawn that way.
§1 Characteristics by quantization band
— Not yet rated · contributions welcome —
Q4–Q5 — 4–5 bit
9GB weights · e.g. Q4_K_M , mlx-4bit + KV: 1.6GB at 8K · 3.1GB at 32K · 3.1GB at 128Kdata checked Aug 2026
- quant-degradation-community — KLD/perplexity delta vs bf16 @ Q4_K_M · aggregated · 2026-08-30
Editorial derivation: small loss at 4-bit.
data checked Aug 2026
- quant-degradation-community — KLD/perplexity delta vs bf16 @ Q4_K_M · aggregated · 2026-08-30
Editorial derivation: small loss at 4-bit.
— Not yet rated · contributions welcome —
FP16 — full precision (fp16/bf16)
29GB weights · e.g. bf16 + KV: 1.6GB at 8K · 3.1GB at 32K · 3.1GB at 128Kdata checked Aug 2026
- vendor-model-card — MATH = 80.4 @ bf16 · vendor-reported · 2026-08-30
MGSM: 80.6, GPQA: 56.1 — exceptional per-GB math.
data checked Aug 2026
- vendor-model-card — HumanEval = 82.6 @ bf16 · vendor-reported · 2026-08-30
Absent from the Aider polyglot leaderboard as of 2026-08-30.
data checked Aug 2026
- vectara-hallucination — HHEM hallucination rate % (lower is better) = 3.7 @ bf16 · aggregated · 2026-08-30
Grounds provided text very well.
- vendor-model-card — SimpleQA = 3 @ bf16 · vendor-reported · 2026-08-30
…but knows almost nothing: open-domain recall is among the lowest measured. The split score reflects both.
§2 Known issues & what fixes them
Near-zero open-domain knowledge (SimpleQA 3.0 — vendor's own number). It will confidently invent facts it was never taught, even though it grounds provided text well (HHEM 3.7%).
Feed it the facts and it behaves: its grounding is excellent. Never use it as an encyclopedia.
Web-search tooling covers the recall gap for current-events questions.
Evidence · 2 sources · community-consensus
- Phi-4 model card (SimpleQA 3.0, self-reported) (vendor, 2026-08-30)
- Vectara HHEM leaderboard (grounding vs recall split) (leaderboard, 2026-08-30)
16K context window — a quarter of what 2025+ peers offer. Long documents and long agent sessions simply don't fit.
Chunked retrieval keeps each request inside 16K.
For genuinely long context, pick a 128K model.
Evidence · 1 source · anecdotal
- Phi-4 model card (16K context) (vendor, 2026-08-30)
Benchmark scores flatter it: in longer interactive sessions it drifts off instructions and feels less capable than its numbers suggest.
Use it for one-shot, well-specified tasks; restate constraints each turn in longer sessions.
Evidence · 1 source · anecdotal
- Tried Phi-4 — it didn't impress (interactive-use writeup) (blog, 2026-08-30)