Model datasheet · Qwen3.5
Qwen3.5 122B-A10B
The Qwen3.5 that fits: 76GB at 4-bit puts it inside a 96GB machine with room for context, and it gives up surprisingly little to the 397B — 72.0 on SWE-bench against 76.4. The best accuracy-per-gigabyte on this site above 70B.
- Vendor
- Alibaba
- Architecture
- MoE · 122B total / ~10B active
- Context
- 262,144 tokens
- License
- Apache-2.0
- Released
- 2026-02-24
- Vision
- Yes
→ GGUF (llama.cpp / LM Studio / Ollama) → MLX (Apple Silicon)
Architecture, from its own config.json
48 layers · 36 linear / recurrent · 12 full attention
Only 12 of 48 layers hold a cache at all, which is why its memory grows so much more slowly with context than its size suggests.
32 attention heads over 2 KV heads (GQA 16:1), head dim 256 · mixture of experts, 10B of 122B active per token across 8 experts · vocabulary 248,320
Where the memory goes: weights by band above, plus a KV cache of 192MB at 8K · 768MB at 32K · 3GB at 128K, plus the runtime's own working memory — dominated by the output projection over that vocabulary.
Inside one layer, from its own tensors
36 × linear / recurrent layer
12 × full-attention layer
The green line down the right of each half is the residual: it leaves the spine above the norm, skips the operation entirely and rejoins at the +. Where attention is drawn open, q, k and v are scaled to their real output widths — a model that shares key and value heads across many query heads shows one wide box beside two narrow ones.
Around the stack: an embedding of 248,320 × 3,072, a final norm, and a separate output projection.
On the norm names: despite what it is called, post_attention_layernorm is drawn after the residual add here, which is where this family applies it — it normalises the input to the feed-forward, not the output of attention. Models that carry a separate pre_feedforward_layernorm use it the other way, and are drawn that way.
§1 Characteristics by quantization band
data checked Aug 2026
- quant-degradation-community — editorial band derivation @ Q3_K_M · aggregated · 2026-08-31
Editorial derivation: two points down. At 3-bit a sparse MoE loses expert-routing precision first, which surfaces as dropped tool arguments and broken multi-file edits before prose visibly suffers.
data checked Aug 2026
- quant-degradation-community — editorial band derivation @ Q3_K_M · aggregated · 2026-08-31
Editorial derivation: two points down. At 3-bit a sparse MoE loses expert-routing precision first, which surfaces as dropped tool arguments and broken multi-file edits before prose visibly suffers.
data checked Aug 2026
- quant-degradation-community — editorial band derivation @ Q4_K_M · aggregated · 2026-08-31
Editorial derivation: one point down from the measured bf16 figure. 4-bit is the band a model this size is actually run at, and the loss is real but modest.
data checked Aug 2026
- quant-degradation-community — editorial band derivation @ Q4_K_M · aggregated · 2026-08-31
Editorial derivation: one point down from the measured bf16 figure. 4-bit is the band a model this size is actually run at, and the loss is real but modest.
data checked Aug 2026
- quant-degradation-community — editorial band derivation @ Q6_K · aggregated · 2026-08-31
Editorial derivation: no drop from the measured bf16 figure. 6-8 bit is effectively lossless on a model this size — the differences that show up at 4-bit and below are not measurable here. Recorded explicitly rather than left to band fallback, which would otherwise borrow the 4-bit number and subtract a point.
data checked Aug 2026
- quant-degradation-community — editorial band derivation @ Q6_K · aggregated · 2026-08-31
Editorial derivation: no drop from the measured bf16 figure. 6-8 bit is effectively lossless on a model this size — the differences that show up at 4-bit and below are not measurable here. Recorded explicitly rather than left to band fallback, which would otherwise borrow the 4-bit number and subtract a point.
data checked Aug 2026
- quant-degradation-community — editorial band derivation @ Q6_K · aggregated · 2026-08-31
Editorial derivation: no drop from the measured bf16 figure. 6-8 bit is effectively lossless on a model this size — the differences that show up at 4-bit and below are not measurable here. Recorded explicitly rather than left to band fallback, which would otherwise borrow the 4-bit number and subtract a point.
data checked Aug 2026
- quant-degradation-community — editorial band derivation @ Q6_K · aggregated · 2026-08-31
Editorial derivation: no drop from the measured bf16 figure. 6-8 bit is effectively lossless on a model this size — the differences that show up at 4-bit and below are not measurable here. Recorded explicitly rather than left to band fallback, which would otherwise borrow the 4-bit number and subtract a point.
data checked Aug 2026
- quant-degradation-community — editorial band derivation @ Q6_K · aggregated · 2026-08-31
Editorial derivation: no drop from the measured bf16 figure. 6-8 bit is effectively lossless on a model this size — the differences that show up at 4-bit and below are not measurable here. Recorded explicitly rather than left to band fallback, which would otherwise borrow the 4-bit number and subtract a point.
FP16 — full precision (fp16/bf16)
244GB weights · e.g. bf16 + KV: 192MB at 8K · 768MB at 32K · 3GB at 128Kdata checked Aug 2026
- vendor-model-card — SWE-bench Verified = 72 @ bf16 · vendor-reported · 2026-08-31
LiveCodeBench v6 78.9, Terminal Bench 2 49.4. Within 4.4 points of the 397B at less than a third of the weights.
data checked Aug 2026
- vendor-model-card — MMLU-Pro = 86.7 @ bf16 · vendor-reported · 2026-08-31
Knowledge benchmark only; no hallucination-rate figure is published for this model.
data checked Aug 2026
- vendor-model-card — IFEval = 93.4 @ bf16 · vendor-reported · 2026-08-31
IFBench 76.1. The highest IFEval figure in this catalogue — above the 397B's 92.6 and Llama 3.3 70B's 92.1.
data checked Aug 2026
- vendor-model-card — GPQA Diamond = 86.6 @ bf16 · vendor-reported · 2026-08-31
MMLU-Pro 86.7. Two points behind the 397B on GPQA.
data checked Aug 2026
- vendor-model-card — BFCL-V4 = 72.2 @ bf16 · vendor-reported · 2026-08-31
Terminal Bench 2 49.4 — essentially level with the 397B's 72.9 on BFCL.
§2 Known issues & what fixes them
3-bit is where the agentic behaviour goes. With 10B active parameters spread over 256 experts, routing decisions are made on a small slice of the model, and 3-bit quantization blurs exactly that slice — malformed tool arguments and dropped call sequences show up well before the prose reads worse.
The 76.5GB 4-bit build is the one to run; it fits a 96GB machine with context to spare, which is the whole argument for this model over the 397B.
Grammar-constrained tool schemas recover much of the malformed-argument loss.
Evidence · 2 sources · community-consensus
- unsloth Qwen3.5-122B-A10B-GGUF: UD-IQ2_M 39.1GB, Q3_K_M 56.4GB, Q4_K_M 76.5GB (other, 2026-08-31)
- Unsloth Qwen3.5 run guide (blog, 2026-08-31)
Every score here is Alibaba's own, published on release day and not independently replicated in this dataset. The claim that a 122B model lands within a few points of a 397B one is the interesting part and the part most in need of outside confirmation. Long-context and factuality cells are blank rather than estimated — no RULER or hallucination result exists for it.
A caveat on the evidence, not on the model.
Evidence · 2 sources · community-consensus
- Qwen3.5-122B-A10B model card: all figures vendor-reported (vendor, 2026-08-31)
- unsloth GGUF distribution; no independent benchmarks published (other, 2026-08-31)