Model datasheet · Qwen3
Qwen3 8B
The default mid-size all-rounder. Strong reasoning and coding for its size, hybrid thinking mode, solid tool calling. The community's most common answer to 'which model should I start with?' on 16GB machines.
- Vendor
- Alibaba
- Architecture
- Dense · 8.2B
- Context
- 32,768 tokens
- License
- Apache-2.0
- Released
- 2025-04-29
- Vision
- No
→ GGUF (llama.cpp / LM Studio / Ollama) → MLX (Apple Silicon)
§1 Characteristics by quantization band
Q2–Q3 — 2–3 bit
4GB weights · e.g. IQ3_XXS , Q3_K_M + KV: 1.1GB at 8K · 4.5GB at 32K · 4.5GB at 128Kdata checked Aug 2026
- quant-degradation-community — KLD delta + degradation reports @ Q3_K_M · aggregated · 2026-08-30
Editorial derivation: multi-step arithmetic degrades sharply at ≤3-bit; consistent community finding across Qwen-family models.
data checked Aug 2026
- quant-degradation-community — KLD delta + degradation reports @ Q3_K_M · aggregated · 2026-08-30
Editorial derivation: noticeably more syntax errors and logic slips at 3-bit.
data checked Aug 2026
- quant-degradation-community — community consensus @ Q3_K_M · aggregated · 2026-08-30
Editorial derivation: malformed JSON arguments become common at ≤3-bit.
data checked Aug 2026
- quant-degradation-community — community consensus @ Q3_K_M · aggregated · 2026-08-30
Editorial derivation: IF holds up better than math/coding at low quants.
Q4–Q5 — 4–5 bit
5GB weights · e.g. Q4_K_M , Q5_K_M , mlx-4bit + KV: 1.1GB at 8K · 4.5GB at 32K · 4.5GB at 128Kdata checked Aug 2026
- quant-degradation-community — KLD/perplexity delta vs bf16 @ Q4_K_M · aggregated · 2026-08-30
Editorial derivation from the full-band score: community KLD data shows Q4_K_M retains most reasoning ability with a small consistent loss.
data checked Aug 2026
- quant-degradation-community — KLD/perplexity delta vs bf16 @ Q4_K_M · aggregated · 2026-08-30
Editorial derivation: Q4_K_M coding loss vs bf16 is minor per community testing.
data checked Aug 2026
- quant-degradation-community — community consensus @ Q4_K_M · aggregated · 2026-08-30
Editorial derivation: tool-call formatting stays reliable at 4-bit for this model per community reports.
data checked Aug 2026
- quant-degradation-community — community consensus @ Q4_K_M · aggregated · 2026-08-30
Editorial derivation: IF is not quant-sensitive; carried from full band.
— Not yet rated · contributions welcome —
FP16 — full precision (fp16/bf16)
16GB weights · e.g. bf16 + KV: 1.1GB at 8K · 4.5GB at 32K · 4.5GB at 128Kdata checked Aug 2026
- vendor-model-card — AIME'24 (thinking mode) = 76 @ bf16 · vendor-reported · 2026-08-30
Qwen3 Technical Report Table 17. Non-thinking mode drops to 29.1 — the score reflects both modes.
- vendor-model-card — MATH-500 (thinking mode) = 97.4 @ bf16 · vendor-reported · 2026-08-30
data checked Aug 2026
- vendor-model-card — LiveCodeBench v5 (thinking mode) = 57.5 @ bf16 · vendor-reported · 2026-08-30
Non-thinking mode: 22.8. Model absent from the Aider polyglot leaderboard as of 2026-08-30.
data checked Aug 2026
- vendor-model-card — BFCL v3 (thinking mode) = 68.1 @ bf16 · vendor-reported · 2026-08-30
Non-thinking mode: 60.2.
data checked Aug 2026
- vendor-model-card — IFEval strict prompt (thinking mode) = 85 @ bf16 · vendor-reported · 2026-08-30
data checked Aug 2026
- vectara-hallucination — HHEM hallucination rate % (lower is better) = 4.8 @ bf16 · aggregated · 2026-08-30
95.2% factual consistency, 99.9% answer rate.
data checked Aug 2026
- vendor-model-card — RULER @128K = 77.4 @ bf16 · vendor-reported · 2026-08-30
RULER @32K: 91.2 — accuracy degrades past 64K.
§2 Per-quant deviations
IQ3_XXS · Math & reasoning drops to 2/10
IQ3_XXS is materially worse than Q3_K_M for arithmetic despite sharing the band — the extreme compression hits attention precision hardest.
§3 Known issues & what fixes them
At 3-bit and below, multi-step arithmetic degrades sharply — expect dropped carries, wrong intermediate steps, and confidently wrong answers.
Route calculations to a Python tool; the model plans fine, it just can't execute arithmetic reliably at this quant.
Q4_K_M needs only ~7GB and recovers most math ability.
Evidence · 2 sources · community-consensus
- llama.cpp quantization quality discussions (KLD/perplexity by quant type) (github, 2026-08-30)
- r/LocalLLaMA recurring consensus on ≤3-bit math degradation (reddit, 2026-08-30)
At ≤3-bit, code gains subtle logic slips and tool calls start producing malformed JSON arguments.
The 4–5 bit band fixes most of this for ~1GB more memory.
Constrain output with explicit format examples; helps somewhat with tool-call formatting.
Evidence · 2 sources · community-consensus
- llama.cpp quantization quality discussions (github, 2026-08-30)
- r/LocalLLaMA reports on tool-calling reliability at low quants (reddit, 2026-08-30)
Prose is competent but flat, with repetitive phrasing in longer outputs. This is the model's character, not a quant artifact.
No tool meaningfully changes voice. If prose quality is the goal, Gemma 3 12B is the better pick at similar memory.
Evidence · 1 source · anecdotal
- r/LocalLLaMA model-recommendation threads for writing (reddit, 2026-08-30)
Qwen3 models fall into endless repetition loops under greedy decoding — the model card itself warns 'DO NOT use greedy decoding'. Quantized builds hit it more often.
Use the card's sampler settings (temp 0.6 / top_p 0.95 / top_k 20 in thinking mode; 0.7 / 0.8 / 20 non-thinking) and presence_penalty up to 1.5 if loops persist.
Evidence · 2 sources · community-consensus
- Qwen3 model card (greedy-decoding warning + recommended samplers) (vendor, 2026-08-30)
- llama.cpp issue: Qwen3 GGUF repetition reports (github, 2026-08-30)