Model datasheet · Qwen3

Qwen3 4B

The strongest 4B all-rounder: with thinking mode on, it reasons like models twice its size. The default answer for 8GB laptops.

Vendor: Alibaba
Architecture: Dense · 4B
Context: 32,768 tokens
License: Apache-2.0
Released: 2025-04-29
Vision: No

→ GGUF (llama.cpp / LM Studio / Ollama) → MLX (Apple Silicon)

§1 Characteristics by quantization band

Q2–Q3 — 2–3 bit

2.1GB weights · e.g. Q3_K_M + KV: 1.1GB at 8K · 4.5GB at 32K · 4.5GB at 128K

— Not yet rated · contributions welcome —

Q4–Q5 — 4–5 bit

2.5GB weights · e.g. Q4_K_M , Q5_K_M , mlx-4bit + KV: 1.1GB at 8K · 4.5GB at 32K · 4.5GB at 128K

Math & reasoning

6/10

data checked Aug 2026

quant-degradation-community — KLD/perplexity delta vs bf16 @ Q4_K_M · aggregated · 2026-08-30
Editorial derivation: small models lose proportionally more at 4-bit.

Coding

5/10

data checked Aug 2026

quant-degradation-community — KLD/perplexity delta vs bf16 @ Q4_K_M · aggregated · 2026-08-30
Editorial derivation: small loss at 4-bit.

Tool calling / agentic

5/10

data checked Aug 2026

quant-degradation-community — community consensus @ Q4_K_M · aggregated · 2026-08-30
Editorial derivation: small loss at 4-bit.

Instruction following

7/10

data checked Aug 2026

quant-degradation-community — community consensus @ Q4_K_M · aggregated · 2026-08-30
Editorial derivation: IF is not quant-sensitive.

Q6–Q8 — 6–8 bit

3.3GB weights · e.g. Q6_K , Q8_0 + KV: 1.1GB at 8K · 4.5GB at 32K · 4.5GB at 128K

— Not yet rated · contributions welcome —

FP16 — full precision (fp16/bf16)

8GB weights · e.g. bf16 + KV: 1.1GB at 8K · 4.5GB at 32K · 4.5GB at 128K

Math & reasoning

7/10

data checked Aug 2026

vendor-model-card — AIME'24 (thinking mode) = 73.8 @ bf16 · vendor-reported · 2026-08-30
MATH-500: 97.0 thinking. Non-thinking drops to 25.0 — thinking mode is doing the lifting.

Coding

6/10

data checked Aug 2026

vendor-model-card — LiveCodeBench v5 (thinking) = 54.2 @ bf16 · vendor-reported · 2026-08-30
Non-thinking: 21.3. Absent from Aider polyglot.

Tool calling / agentic

6/10

data checked Aug 2026

vendor-model-card — BFCL v3 (thinking) = 65.9 @ bf16 · vendor-reported · 2026-08-30
Non-thinking: 57.6.

Instruction following

7/10

data checked Aug 2026

vendor-model-card — IFEval strict prompt (thinking) = 81.9 @ bf16 · vendor-reported · 2026-08-30

Factuality

5/10

data checked Aug 2026

vectara-hallucination — HHEM hallucination rate % (lower is better) = 5.7 @ bf16 · aggregated · 2026-08-30
Grounds well; open-domain knowledge is thin for the whole family.

Long context

6/10

data checked Aug 2026

vendor-model-card — RULER avg 4K–128K (thinking) = 83.5 @ bf16 · vendor-reported · 2026-08-30
85.2 non-thinking. Native window is 32K; 128K needs YaRN.

§2 Known issues & what fixes them

ISSUE-01 moderate Q2–Q3 / Q4–Q5 / Q6–Q8 / FP16 · instruction-following

Falls into endless repetition loops under greedy decoding — the card itself warns against it — and quantized builds hit it hardest at this size.

Fix

Prompt technique [strong]

Use the card's samplers (temp 0.6 / top_p 0.95 / top_k 20 thinking; 0.7 / 0.8 / 20 non-thinking); presence_penalty up to 1.5 if loops persist.

Evidence · 2 sources · community-consensus

Qwen3-4B card ('DO NOT use greedy decoding') (vendor, 2026-08-30)
llama.cpp issue: Qwen3 GGUF repetition (github, 2026-08-30)

ISSUE-02 moderate Q2–Q3 / Q4–Q5 / Q6–Q8 / FP16 · factuality

The family's math/code tilt comes at the cost of world knowledge — a 4B slice of it especially. Expect confident wrong answers on open-domain facts.

Fix

RAG (retrieval) [strong]

Grounding fixes most of it — its HHEM consistency (94.3%) shows it respects provided text.

Evidence · 2 sources · community-consensus

Family-wide knowledge-gap discussion (other, 2026-08-30)
Qwen3 release analysis (SimpleQA weakness) (blog, 2026-08-30)