Model datasheet · gpt-oss
gpt-oss-20b
OpenAI's open MoE, shipped natively in MXFP4 (~4-bit) — the quant band question mostly answers itself. Fast, strong at STEM and tool use, and startlingly ignorant of the world: built to look things up, not to know them.
- Vendor
- OpenAI
- Architecture
- MoE · 20.9B total / ~3.6B active
- Context
- 131,072 tokens
- License
- Apache-2.0
- Released
- 2025-08-05
- Vision
- No
→ GGUF (llama.cpp / LM Studio / Ollama) → MLX (Apple Silicon)
Architecture, from its own config.json
24 layers · 12 sliding window · 12 full attention
64 attention heads over 8 KV heads (GQA 8:1), head dim 64 · mixture of experts, 3.6B of 20.9B active per token across 4 experts · vocabulary 201,088
Where the memory goes: weights by band above, plus a KV cache of 210MB at 8K · 786MB at 32K · 3GB at 128K, plus the runtime's own working memory — dominated by the output projection over that vocabulary.
Inside one layer, from its own tensors
24 × layer
12 sliding-window layers, 12 full-attention layers — the same block either way. What differs is how far back the mask lets attention look, not what the layer is built from.
The green line down the right of each half is the residual: it leaves the spine above the norm, skips the operation entirely and rejoins at the +. Where attention is drawn open, q, k and v are scaled to their real output widths — a model that shares key and value heads across many query heads shows one wide box beside two narrow ones.
Around the stack: an embedding of 201,088 × 2,880, a final norm, and a separate output projection.
On the norm names: despite what it is called, post_attention_layernorm is drawn after the residual add here, which is where this family applies it — it normalises the input to the feed-forward, not the output of attention. Models that carry a separate pre_feedforward_layernorm use it the other way, and are drawn that way.
Also in every block and not drawn, because their tensor names do not fix where they belong: sinks. Guessing an order is the one thing this diagram will not do.
§1 Characteristics by quantization band
data checked Aug 2026
- vendor-model-card — AIME 2024 (high reasoning, no tools) = 42.1 @ mxfp4 · vendor-reported · 2026-08-30
61.2 WITH tools — the tool-use gain is the design intent. Measured at native MXFP4, so this anchors the mid band directly.
data checked Aug 2026
- vendor-model-card — SWE-bench Verified = 60.7 @ mxfp4 · vendor-reported · 2026-08-30
Codeforces Elo 2230. Absent from Aider polyglot (the 120b appears; 20b does not).
data checked Aug 2026
- vendor-model-card — Tau-Bench Retail = 54.8 @ mxfp4 · vendor-reported · 2026-08-30
Tool use is the model's organizing principle — browsing, python, and functions are first-class in its harmony format.
data checked Aug 2026
- vendor-model-card — SimpleQA = 6.7 @ mxfp4 · vendor-reported · 2026-08-30
PersonQA hallucination rate: 53.2% — OpenAI's own numbers. Absent from Vectara HHEM (only the 120b is listed).
data checked Oct 2026
- eqbench-creative — EQ-Bench Creative Writing v3 rubric score (0-20) = 7.87 @ mxfp4 · aggregated · 2026-10-04
Normalised Elo 665.6, 137th of 140, retrieved 2026-10-04: among the weakest rows on the board, below Llama 3.2 3B (8.18). Native MXFP4 weights, so the measurement anchors this band directly. Independent; rubric judged by Claude Sonnet 4. EQ-Bench's normalised Elo moves as models are added, so the rubric score is the figure stored. Data file behind https://eqbench.com/creative_writing.html; the date the row was added is not published.
- eqbench-creative — EQ-Bench Creative Writing Longform score (0-100) = 21.2 @ mxfp4 · aggregated · 2026-10-04
Longform: above only Qwen3 4B (17.0) among models on this site. Data file behind the Longform page on eqbench.com.
§2 Known issues & what fixes them
Severe factual hallucination and thin world knowledge — SimpleQA 6.7 and 53.2% PersonQA hallucination are OpenAI's own figures, and community testing found it 'unbelievably ignorant' offline.
Give it web search — it was explicitly designed to look things up rather than know them, and its tool use is good enough to make this work.
Grounded document Q&A sidesteps the recall gap entirely.
Evidence · 2 sources · community-consensus
- Community discussion on offline knowledge gaps (other, 2026-08-30)
- Critique of gpt-oss world knowledge (blog, 2026-08-30)
STEM-narrow: general chat feels stiff, multilingual ability is weak, and prose is functional at best.
Model character. For chat or writing, Gemma or Mistral at similar memory are better company.
Evidence · 2 sources · community-consensus
- Critique of gpt-oss general use (blog, 2026-08-30)
- Community discussion of chat/multilingual gaps (other, 2026-08-30)