Model datasheet · Qwen3-Next
Qwen3-Coder-Next 80B-A3B
Three billion active parameters out of eighty, and 70.6 on SWE-bench Verified — within striking distance of models that activate ten times as much. The 49GB 4-bit build is the cheapest way onto this site's upper tier, and the sparsity that makes it fast is also what makes it fragile at low quant.
- Vendor
- Alibaba
- Architecture
- MoE · 80B total / ~3B active
- Context
- 262,144 tokens
- License
- Apache-2.0
- Released
- 2026-02-03
- Vision
- No
→ GGUF (llama.cpp / LM Studio / Ollama) → MLX (Apple Silicon)
Architecture, from its own config.json
48 layers · 36 linear / recurrent · 12 full attention
Only 12 of 48 layers hold a cache at all, which is why its memory grows so much more slowly with context than its size suggests.
16 attention heads over 2 KV heads (GQA 8:1), head dim 256 · mixture of experts, 3B of 80B active per token across 10 experts · vocabulary 151,936
Where the memory goes: weights by band above, plus a KV cache of 192MB at 8K · 768MB at 32K · 3GB at 128K, plus the runtime's own working memory — dominated by the output projection over that vocabulary.
Inside one layer, from its own tensors
36 × linear / recurrent layer
12 × full-attention layer
The green line down the right of each half is the residual: it leaves the spine above the norm, skips the operation entirely and rejoins at the +. Where attention is drawn open, q, k and v are scaled to their real output widths — a model that shares key and value heads across many query heads shows one wide box beside two narrow ones.
Around the stack: an embedding of 151,936 × 2,048, a final norm, and a separate output projection.
On the norm names: despite what it is called, post_attention_layernorm is drawn after the residual add here, which is where this family applies it — it normalises the input to the feed-forward, not the output of attention. Models that carry a separate pre_feedforward_layernorm use it the other way, and are drawn that way.
§1 Characteristics by quantization band
data checked Aug 2026
- quant-degradation-community — editorial band derivation @ Q3_K_M · aggregated · 2026-08-31
Editorial derivation: two points down. At 3-bit a sparse MoE loses expert-routing precision first, which surfaces as dropped tool arguments and broken multi-file edits before prose visibly suffers.
data checked Aug 2026
- quant-degradation-community — editorial band derivation @ Q3_K_M · aggregated · 2026-08-31
Editorial derivation: two points down. At 3-bit a sparse MoE loses expert-routing precision first, which surfaces as dropped tool arguments and broken multi-file edits before prose visibly suffers.
data checked Aug 2026
- quant-degradation-community — editorial band derivation @ Q4_K_M · aggregated · 2026-08-31
Editorial derivation: one point down from the measured bf16 figure. 4-bit is the band a model this size is actually run at, and the loss is real but modest.
data checked Aug 2026
- quant-degradation-community — editorial band derivation @ Q4_K_M · aggregated · 2026-08-31
Editorial derivation: one point down from the measured bf16 figure. 4-bit is the band a model this size is actually run at, and the loss is real but modest.
data checked Aug 2026
- quant-degradation-community — editorial band derivation @ Q6_K · aggregated · 2026-08-31
Editorial derivation: no drop from the measured bf16 figure. 6-8 bit is effectively lossless on a model this size — the differences that show up at 4-bit and below are not measurable here. Recorded explicitly rather than left to band fallback, which would otherwise borrow the 4-bit number and subtract a point.
data checked Aug 2026
- quant-degradation-community — editorial band derivation @ Q6_K · aggregated · 2026-08-31
Editorial derivation: no drop from the measured bf16 figure. 6-8 bit is effectively lossless on a model this size — the differences that show up at 4-bit and below are not measurable here. Recorded explicitly rather than left to band fallback, which would otherwise borrow the 4-bit number and subtract a point.
FP16 — full precision (fp16/bf16)
159GB weights · e.g. bf16 + KV: 192MB at 8K · 768MB at 32K · 3GB at 128Kdata checked Aug 2026
- vendor-model-card — SWE-bench Verified = 70.6 @ bf16 · vendor-reported · 2026-08-31
SWE-bench Pro 44.3. Qwen claim performance comparable to models with 10-20x more active parameters, and on SWE-bench Verified the figure supports it.
data checked Aug 2026
- vendor-model-card — Terminal-Bench 2.0 = 36.2 @ bf16 · vendor-reported · 2026-08-31
Below GLM-4.7's 41.0 and well below Qwen3.5-397B's 52.5 — the gap that 3B active parameters buys you.
§2 Known issues & what fixes them
Only 3B of 80B parameters are active per token, routed across 512 experts. That extreme sparsity is what makes the model fast and small to run, and it is also what makes 3-bit risky: every routing decision is made on a thin slice of the network, and quantization noise there costs more than it would in a dense model of the same footprint.
Run the 49.3GB 4-bit build. Unsloth's own guidance is more than 45GB of unified memory or RAM/VRAM for 4-bit, which is a modest ask for this tier.
Evidence · 2 sources · community-consensus
- Unsloth Qwen3-Coder-Next guide: >45GB recommended for 4-bit quants (blog, 2026-08-31)
- unsloth Qwen3-Coder-Next-GGUF: UD-IQ2_XXS 23.3GB, Q3_K_M 38.3GB, UD-Q4_K_M 49.3GB (other, 2026-08-31)
This is a coding model and its published evidence is coding-only: SWE-bench Verified, SWE-bench Pro and Terminal-Bench, and nothing else. There is no AIME, GPQA, MMLU, IFEval or hallucination figure for it, so five of the seven dimensions on this site are genuinely unmeasured rather than merely low. The coverage discount on its cards reflects that, and it should — do not read the blank cells as strength.
Use it for what it was measured on. For general work, a Qwen3.5 model of similar size has evidence across the board.
Evidence · 2 sources · community-consensus
- Qwen3-Coder-Next model card: SWE-bench and Terminal-Bench only (vendor, 2026-08-31)
- Unsloth Qwen3-Coder-Next guide (blog, 2026-08-31)