Model datasheet · Mistral Small
Mistral Small 4
Mistral folded Magistral, Pixtral and Devstral into one model: reasoning, vision and agentic coding, Apache licensed, 6.5B active of 119B. The catch is attention. It kept full multi-head attention where everything else its size uses grouped-query, so its KV cache costs 576KB per token — the most expensive on this site, and more than a 358B model's. Cheap to compute, dear to give context to.
- Vendor
- Mistral AI
- Architecture
- MoE · 119B total / ~6.5B active
- Context
- 262,144 tokens
- License
- Apache-2.0
- Released
- 2026-03-16
- Vision
- Yes
Architecture, from its own config.json
32 attention heads, multi-head latent attention — it caches one 320-wide compressed vector per layer instead of per-head keys and values, which is why its cache is small for its size · mixture of experts, 6.5B of 119B active per token across 4 experts · vocabulary 131,072
Where the memory goes: weights by band above, plus a KV cache of 180MB at 8K · 720MB at 32K · 2.8GB at 128K, plus the runtime's own working memory — dominated by the output projection over that vocabulary.
No per-layer split is drawn because latent attention does not have one in the usual sense: every layer caches the same compressed vector, so there is no full-versus-sliding distinction to show.
Inside one layer, from its own tensors
36 × every layer
The green line down the right of each half is the residual: it leaves the spine above the norm, skips the operation entirely and rejoins at the +. Where attention is drawn open, q, k and v are scaled to their real output widths — a model that shares key and value heads across many query heads shows one wide box beside two narrow ones.
Around the stack: an embedding of 131,072 × 4,096, a final norm, and a separate output projection.
On the norm names: despite what it is called, post_attention_layernorm is drawn after the residual add here, which is where this family applies it — it normalises the input to the feed-forward, not the output of attention. Models that carry a separate pre_feedforward_layernorm use it the other way, and are drawn that way.
§1 Characteristics by quantization band
data checked Aug 2026
- quant-degradation-community — editorial band derivation @ Q3_K_M · aggregated · 2026-08-31
Editorial derivation: two points down. At 3-bit a sparse MoE loses expert-routing precision first, which surfaces as dropped tool arguments and broken multi-file edits before prose visibly suffers.
data checked Aug 2026
- quant-degradation-community — editorial band derivation @ Q4_K_M · aggregated · 2026-08-31
Editorial derivation: one point down from the measured bf16 figure. 4-bit is the band a model this size is actually run at, and the loss is real but modest.
data checked Aug 2026
- quant-degradation-community — editorial band derivation @ Q4_K_M · aggregated · 2026-08-31
Editorial derivation: one point down from the measured bf16 figure. 4-bit is the band a model this size is actually run at, and the loss is real but modest.
data checked Oct 2026
- quant-degradation-community — editorial band derivation @ Q4_K_M · aggregated · 2026-10-04
Editorial derivation: carried from the EQ-Bench-anchored full band; creative writing is not quant-sensitive.
data checked Aug 2026
- quant-degradation-community — editorial band derivation @ Q6_K · aggregated · 2026-08-31
Editorial derivation: no drop from the measured bf16 figure. 6-8 bit is effectively lossless on a model this size.
data checked Aug 2026
- quant-degradation-community — editorial band derivation @ Q6_K · aggregated · 2026-08-31
Editorial derivation: no drop from the measured bf16 figure. 6-8 bit is effectively lossless on a model this size.
data checked Aug 2026
- quant-degradation-community — editorial band derivation @ Q6_K · aggregated · 2026-08-31
Editorial derivation: no drop from the measured bf16 figure. 6-8 bit is effectively lossless on a model this size.
data checked Aug 2026
- quant-degradation-community — editorial band derivation @ Q6_K · aggregated · 2026-08-31
Editorial derivation: no drop from the measured bf16 figure. 6-8 bit is effectively lossless on a model this size.
FP16 — full precision (fp16/bf16)
238GB weights · e.g. bf16 + KV: 180MB at 8K · 720MB at 32K · 2.8GB at 128Kdata checked Aug 2026
- vendor-model-card — AIME 2025 = 83.8 @ bf16 · vendor-reported · 2026-08-31
GPQA Diamond 71.2. Reasoning is toggled per request with reasoning_effort, so the figure assumes it is on.
data checked Aug 2026
- vendor-model-card — LiveCodeBench = 63.6 @ bf16 · vendor-reported · 2026-08-31
Mistral claim it beats gpt-oss-120b here while producing 20% less output. No SWE-bench figure is published, which is the number that would matter most for the agentic coding it absorbed from Devstral.
data checked Aug 2026
- vendor-model-card — MMLU-Pro = 78 @ bf16 · vendor-reported · 2026-08-31
MMMU-Pro 60 on the vision side. No hallucination-rate benchmark is published, so this is a knowledge score rather than a truthfulness one.
data checked Aug 2026
- artificial-analysis — AA-LCR long-context reasoning (link-only — no numbers republished) @ bf16 · aggregated · 2026-08-31
Cited by link under their terms. Mistral report it as matching gpt-oss-120b. Worth reading against this model's KV cost: the context is affordable to reason over and expensive to hold.
data checked Oct 2026
- eqbench-creative — EQ-Bench Creative Writing Longform score (0-100) = 43.2 @ bf16 · aggregated · 2026-10-04
Row mistralai/Mistral-Small-4-119B-2603, the model this sheet describes. Not on the short-form Creative Writing v3 board, so this longform figure is the only measurement; scored against Gemma 3 12B (41.8, scored 5) and Gemma 3 4B (34.4, scored 4) on the same board. Quant not stated by EQ-Bench (presumed full-precision API serving). Data file behind the Longform page on eqbench.com.
§2 Known issues & what fixes them
The most expensive KV cache in this catalogue, and by a long way. Its config keeps full multi-head attention — 32 key-value heads against 32 attention heads — where every comparable model uses grouped-query attention to shrink exactly this. The result is 576KB per token: 18GB of cache at 32K context, about 72GB at 128K, on top of 74GB of 4-bit weights. A 358B model with grouped-query attention costs less. The sparsity that makes it fast to run does nothing for the memory it needs to hold a conversation.
Architectural. Budget the cache explicitly — the RUNS AT line prices it at the context you pick — or run it at short context, which is where its 6.5B active parameters pay off.
Evidence · 2 sources · community-consensus
- Mistral Small 4 config.json: num_key_value_heads 32 equals num_attention_heads 32 (vendor, 2026-08-31)
- unsloth GGUF sizes: UD-Q4_K_M 73.8GB of weights before any cache (other, 2026-08-31)
Sold as the model that absorbed Devstral's agentic coding, and published with no tool-calling or instruction-following benchmark to show for it — no BFCL, no IFEval, no SWE-bench. Those cells are blank here rather than estimated, which the coverage discount then penalises. Every figure that does exist is Mistral's own; no independent replication was found.
A gap in the evidence, not a defect in the model. Treat the agentic claim as unverified until someone benchmarks it.
Evidence · 2 sources · community-consensus
- Mistral Small 4 announcement: AIME, MMLU-Pro, GPQA and LiveCodeBench only (vendor, 2026-08-31)
- Model card publishes no BFCL, IFEval or SWE-bench figure (vendor, 2026-08-31)