Model datasheet · Qwen3

Qwen3 32B

Superseded

Superseded by Qwen3.8 27B, which covers the same 24–32GB machines with a newer dense design. Kept because it still holds one of the very few independent Aider polyglot scores for a model this size (40.0%).

Vendor: Alibaba
Architecture: Dense · 32.8B
Context: 32,768 tokens
License: Apache-2.0
Released: 2025-04-29
Vision: No

→ GGUF (llama.cpp / LM Studio / Ollama) → MLX (Apple Silicon)

Architecture, from its own config.json

64 layers · 64 full attention

64 attention heads over 8 KV heads (GQA 8:1), head dim 128 · dense feed-forward · vocabulary 151,936

Where the memory goes: weights by band above, plus a KV cache of 2GB at 8K · 8GB at 32K, plus the runtime's own working memory — dominated by the output projection over that vocabulary.

Inside one layer, from its own tensors

64 × full-attention layer

The green line down the right of each half is the residual: it leaves the spine above the norm, skips the operation entirely and rejoins at the +. Where attention is drawn open, q, k and v are scaled to their real output widths — a model that shares key and value heads across many query heads shows one wide box beside two narrow ones.

Around the stack: an embedding of 151,936 × 5,120, a final norm, and a separate output projection.

On the norm names: despite what it is called, post_attention_layernorm is drawn after the residual add here, which is where this family applies it — it normalises the input to the feed-forward, not the output of attention. Models that carry a separate pre_feedforward_layernorm use it the other way, and are drawn that way.

§1 Characteristics by quantization band

Q2–Q3 — 2–3 bit

16GB weights · e.g. Q3_K_M + KV: 2GB at 8K · 8GB at 32K · 8GB at 128K

— Not yet rated · contributions welcome —

Q4–Q5 — 4–5 bit

20GB weights · e.g. Q4_K_M , Q5_K_M , mlx-4bit + KV: 2GB at 8K · 8GB at 32K · 8GB at 128K

Math & reasoning

7/10

data checked Aug 2026

quant-degradation-community — KLD/perplexity delta vs bf16 @ Q4_K_M · aggregated · 2026-08-30
Editorial derivation: large dense models quantize gracefully; small loss at 4-bit.

Coding

7/10

data checked Aug 2026

quant-degradation-community — KLD/perplexity delta vs bf16 @ Q4_K_M · aggregated · 2026-08-30
Editorial derivation: small loss at 4-bit.

Tool calling / agentic

6/10

data checked Aug 2026

quant-degradation-community — community consensus @ Q4_K_M · aggregated · 2026-08-30
Editorial derivation: occasional argument-format slips at 4-bit.

Instruction following

7/10

data checked Aug 2026

quant-degradation-community — community consensus @ Q4_K_M · aggregated · 2026-08-30
Editorial derivation: IF is not quant-sensitive.

Creative writing

5/10

data checked Oct 2026

quant-degradation-community — editorial band derivation @ Q4_K_M · aggregated · 2026-10-04
Editorial derivation: carried from the EQ-Bench-anchored full band; creative writing is not quant-sensitive.

Q6–Q8 — 6–8 bit

27GB weights · e.g. Q6_K + KV: 2GB at 8K · 8GB at 32K · 8GB at 128K

— Not yet rated · contributions welcome —

FP16 — full precision (fp16/bf16)

66GB weights · e.g. bf16 + KV: 2GB at 8K · 8GB at 32K · 8GB at 128K

Math & reasoning

8/10

data checked Aug 2026

vendor-model-card — AIME'24 (thinking) = 81.4 @ bf16 · vendor-reported · 2026-08-30
MATH-500: 97.2 thinking; Codeforces rating 1977.

Coding

7/10

data checked Aug 2026

vendor-model-card — LiveCodeBench v5 (thinking) = 65.7 @ bf16 · vendor-reported · 2026-08-30
aider-polyglot — percent correct (diff format) = 40 @ bf16 · aggregated · 2026-08-30
The only sub-35B model on the Aider polyglot board — independent real-world editing evidence.

Tool calling / agentic

7/10

data checked Aug 2026

vendor-model-card — BFCL v3 (thinking) = 70.3 @ bf16 · vendor-reported · 2026-08-30

Instruction following

7/10

data checked Aug 2026

vendor-model-card — IFEval strict prompt (thinking) = 85 @ bf16 · vendor-reported · 2026-08-30
Arena-Hard: 93.8.

Factuality

5/10

data checked Aug 2026

vectara-hallucination — HHEM hallucination rate % (lower is better) = 5.9 @ bf16 · aggregated · 2026-08-30

Long context

7/10

data checked Aug 2026

vendor-model-card — RULER avg (non-thinking) = 93.7 @ bf16 · vendor-reported · 2026-08-30
91.0 thinking. Native 32K; 128K via YaRN.

Creative writing

5/10

data checked Oct 2026

eqbench-creative — EQ-Bench Creative Writing Longform score (0-100) = 40.5 @ bf16 · aggregated · 2026-10-04
Row qwen/qwen3-32b. Not on the short-form Creative Writing v3 board, so this longform figure is the only measurement; scored against Gemma 3 12B (41.8, scored 5) and Gemma 3 4B (34.4, scored 4) on the same board. Quant not stated by EQ-Bench (presumed full-precision API serving). Data file behind the Longform page on eqbench.com.

§2 Known issues & what fixes them

ISSUE-01 mild Q2–Q3 / Q4–Q5 / Q6–Q8 / FP16 · instruction-following

Thinking mode burns thousands of tokens per answer — painful at the <10 tok/s this dense 32B manages on consumer hardware, especially in agent loops.

Fix

Prompt technique [strong]

Set enable_thinking=False or append /no_think for tasks that don't need deep reasoning; consider Qwen3-30B-A3B for ~5× the speed at similar quality.

Evidence · 2 sources · community-consensus

NVIDIA docs on Qwen3 thinking-budget control (vendor, 2026-08-30)
Qwen3-30B-A3B vs 32B speed/quality analysis (blog, 2026-08-30)

ISSUE-02 mild Q2–Q3 / Q4–Q5 / Q6–Q8 / FP16 · instruction-following

Endless repetition under greedy decoding — the model card warns against it explicitly.

Fix

Prompt technique [strong]

Card samplers (0.6/0.95/20 thinking; 0.7/0.8/20 non-thinking); presence_penalty up to 1.5.

Evidence · 2 sources · community-consensus

Qwen3-32B card (greedy-decoding warning) (vendor, 2026-08-30)
llama.cpp issue: Qwen3 GGUF repetition (github, 2026-08-30)