Known-issues sheets for local language models Rev 0.1.0 · reviewed every 7 days
llm-picker

Model datasheet · Qwen3

Qwen3 4B

Superseded

Superseded by Qwen3.5 4B, the same size one generation on. Kept as the record of what the previous generation scored: with thinking on, it reasoned like models twice its size, and it was the default answer for 8GB laptops.

Vendor
Alibaba
Architecture
Dense · 4B
Context
32,768 tokens
License
Apache-2.0
Released
2025-04-29
Vision
No

→ GGUF (llama.cpp / LM Studio / Ollama) → MLX (Apple Silicon)

Architecture, from its own config.json

36 layers · 36 full attention

32 attention heads over 8 KV heads (GQA 4:1), head dim 128 · dense feed-forward · vocabulary 151,936

Where the memory goes: weights by band above, plus a KV cache of 1.1GB at 8K · 4.5GB at 32K, plus the runtime's own working memory — dominated by the output projection over that vocabulary.

Inside one layer, from its own tensors

36 × full-attention layer

input_layernormattentionq_projkvsoftmax(q·kᵀ/√d) · vq_norm · k_norm before the dotproduct32 query heads over 8 key/valueheadso_projpost_attention_layernormfeed-forwardgate_proj · up_proj → down_proj9,728 wide

The green line down the right of each half is the residual: it leaves the spine above the norm, skips the operation entirely and rejoins at the +. Where attention is drawn open, q, k and v are scaled to their real output widths — a model that shares key and value heads across many query heads shows one wide box beside two narrow ones.

Around the stack: an embedding of 151,936 × 2,560, a final norm, and an output projection that reuses the embedding (tied weights).

On the norm names: despite what it is called, post_attention_layernorm is drawn after the residual add here, which is where this family applies it — it normalises the input to the feed-forward, not the output of attention. Models that carry a separate pre_feedforward_layernorm use it the other way, and are drawn that way.

§1 Characteristics by quantization band

Q2–Q3 — 2–3 bit

2.1GB weights · e.g. Q3_K_M + KV: 1.1GB at 8K · 4.5GB at 32K · 4.5GB at 128K

— Not yet rated · contributions welcome —

Q4–Q5 — 4–5 bit

2.5GB weights · e.g. Q4_K_M , Q5_K_M , mlx-4bit + KV: 1.1GB at 8K · 4.5GB at 32K · 4.5GB at 128K
Math & reasoning 6/10
data checked Aug 2026
  • quant-degradation-community — KLD/perplexity delta vs bf16 @ Q4_K_M · aggregated · 2026-08-30

    Editorial derivation: small models lose proportionally more at 4-bit.

Coding 5/10
data checked Aug 2026
  • quant-degradation-community — KLD/perplexity delta vs bf16 @ Q4_K_M · aggregated · 2026-08-30

    Editorial derivation: small loss at 4-bit.

Tool calling / agentic 5/10
data checked Aug 2026
Instruction following 7/10
data checked Aug 2026
  • quant-degradation-community — community consensus @ Q4_K_M · aggregated · 2026-08-30

    Editorial derivation: IF is not quant-sensitive.

Creative writing 2/10
data checked Oct 2026
  • quant-degradation-community — editorial band derivation @ Q4_K_M · aggregated · 2026-10-04

    Editorial derivation: carried from the EQ-Bench-anchored full band; creative writing is not quant-sensitive.

Q6–Q8 — 6–8 bit

3.3GB weights · e.g. Q6_K , Q8_0 + KV: 1.1GB at 8K · 4.5GB at 32K · 4.5GB at 128K

— Not yet rated · contributions welcome —

FP16 — full precision (fp16/bf16)

8GB weights · e.g. bf16 + KV: 1.1GB at 8K · 4.5GB at 32K · 4.5GB at 128K
Math & reasoning 7/10
data checked Aug 2026
Coding 6/10
data checked Aug 2026
Tool calling / agentic 6/10
data checked Aug 2026
Instruction following 7/10
data checked Aug 2026
Factuality 5/10
data checked Aug 2026
Long context 6/10
data checked Aug 2026
Creative writing 2/10
data checked Oct 2026
  • eqbench-creative — EQ-Bench Creative Writing Longform score (0-100) = 17 @ bf16 · aggregated · 2026-10-04

    Row qwen/qwen3-4b: the lowest Longform figure of any model on this site. Not on the short-form Creative Writing v3 board, so this longform figure is the only measurement; scored against Gemma 3 12B (41.8, scored 5) and Gemma 3 4B (34.4, scored 4) on the same board. Quant not stated by EQ-Bench (presumed full-precision API serving). Data file behind the Longform page on eqbench.com.

§2 Known issues & what fixes them

ISSUE-01 moderate Q2–Q3 / Q4–Q5 / Q6–Q8 / FP16 · instruction-following

Falls into endless repetition loops under greedy decoding — the card itself warns against it — and quantized builds hit it hardest at this size.

Fix
Prompt technique [strong]

Use the card's samplers (temp 0.6 / top_p 0.95 / top_k 20 thinking; 0.7 / 0.8 / 20 non-thinking); presence_penalty up to 1.5 if loops persist.

Evidence · 2 sources · community-consensus
ISSUE-02 moderate Q2–Q3 / Q4–Q5 / Q6–Q8 / FP16 · factuality

The family's math/code tilt comes at the cost of world knowledge — a 4B slice of it especially. Expect confident wrong answers on open-domain facts.

Fix
RAG (retrieval) [strong]

Grounding fixes most of it — its HHEM consistency (94.3%) shows it respects provided text.

Evidence · 2 sources · community-consensus