Model datasheet · Qwen3.8
Qwen3.8 Flash-Next
Alibaba's preview of the Qwen4 architecture. Alibaba's own table gives it SWE-bench Multilingual 81.0 and LiveCodeBench v6 91.9; LiveBench's independent run puts its plain coding nearer the pack, while its agentic coding, math and instruction following hold up well. Only 6B of it computes per token, out of 125B of transformer plus a 51B n-gram embedding table: 111GB at 4-bit. Whether all of that must be held depends on the runtime. llama.cpp on CUDA or CPU now reads the table (about 27 GiB at 4-bit) from disk as needed, but on a Mac current GGUFs still count it against GPU memory because of an open bug. The sizes here assume the Mac case, so on a PC they overstate what must be resident. Twelve of 48 layers cache, so context is the cheap part. Metal also has an open bug that stops decode after one token at long context.
- Vendor
- Alibaba
- Architecture
- MoE · 180B total / ~6B active
- Context
- 262,144 tokens
- License
- Qwen Community License 1.0
- Released
- 2026-08-24
- Vision
- Yes
→ GGUF (llama.cpp / LM Studio / Ollama) → MLX (Apple Silicon)
Architecture, from its own config.json
48 layers · 36 linear / recurrent · 12 full attention
Only 12 of 48 layers hold a cache at all, which is why its memory grows so much more slowly with context than its size suggests.
24 attention heads over 2 KV heads (GQA 12:1), head dim 256 · mixture of experts, 6B of 180B active per token across 10 experts · vocabulary 248,320
Where the memory goes: weights by band above, plus a KV cache of 192MB at 8K · 768MB at 32K · 3GB at 128K, plus the runtime's own working memory — dominated by the output projection over that vocabulary.
Inside one layer
Not drawn. This model names its per-layer norm norm, which fixes no position — it could be the one before attention or the one before the feed-forward, and the name does not say. The memory figures above come from its tensors either way; the block order does not, so it is left out rather than guessed.
§1 Characteristics by quantization band
Q2–Q3 — 2–3 bit
90GB weights · e.g. Q3_K_XL , Q2_K_XL + KV: 192MB at 8K · 768MB at 32K · 3GB at 128Kdata checked Sep 2026
- quant-degradation-community — editorial band derivation @ Q3_K_XL · aggregated · 2026-09-12
Editorial derivation: two points down from bf16. 90GB against 111GB — worth it only if 4-bit will not fit.
data checked Sep 2026
- quant-degradation-community — editorial band derivation @ Q3_K_XL · aggregated · 2026-09-12
Editorial derivation: two points down.
data checked Sep 2026
- quant-degradation-community — editorial band derivation @ Q3_K_XL · aggregated · 2026-09-12
Editorial derivation: two points down.
Q4–Q5 — 4–5 bit
111GB weights · e.g. Q4_K_XL , mlx-4bit + KV: 192MB at 8K · 768MB at 32K · 3GB at 128Kdata checked Sep 2026
- quant-degradation-community — editorial band derivation @ Q4_K_XL · aggregated · 2026-09-12
Editorial derivation: one point down. At 111GB this is the band a 128GB machine runs, and the one almost everyone will use.
data checked Sep 2026
- quant-degradation-community — editorial band derivation @ Q4_K_XL · aggregated · 2026-09-12
Editorial derivation: one point down; 6B active parameters leaves less slack at 4-bit than a dense model of the same footprint would have.
data checked Sep 2026
- quant-degradation-community — editorial band derivation @ Q4_K_XL · aggregated · 2026-09-12
Editorial derivation: one point down.
data checked Sep 2026
- quant-degradation-community — editorial band derivation @ Q4_K_XL · aggregated · 2026-09-12
Editorial derivation: carried; format compliance survives 4-bit better than arithmetic.
data checked Sep 2026
- quant-degradation-community — editorial band derivation @ Q6_K_XL · aggregated · 2026-09-12
Editorial derivation: carried from bf16.
data checked Sep 2026
- quant-degradation-community — editorial band derivation @ Q6_K_XL · aggregated · 2026-09-12
Editorial derivation: carried from bf16.
data checked Sep 2026
- quant-degradation-community — editorial band derivation @ Q6_K_XL · aggregated · 2026-09-12
Editorial derivation: carried from bf16.
data checked Sep 2026
- quant-degradation-community — editorial band derivation @ Q6_K_XL · aggregated · 2026-09-12
Editorial derivation: carried from bf16.
FP16 — full precision (fp16/bf16)
354GB weights · e.g. bf16 + KV: 192MB at 8K · 768MB at 32K · 3GB at 128Kdata checked Oct 2026
- vendor-model-card — SWE-bench Multilingual = 81 @ bf16 · vendor-reported · 2026-09-12
Alibaba's table. Ahead of Qwen3.8-27B (73.8) and of Claude Opus 4.6 (77.5) on the row as they report it.
- vendor-model-card — LiveCodeBench v6 = 91.9 @ bf16 · vendor-reported · 2026-09-12
Competitive coding, against Qwen3.8-27B's 90.3 — which this site scores 9.
- vendor-model-card — SWE-bench Pro = 62.5 @ bf16 · vendor-reported · 2026-09-12
Agentic repository work.
- livebench — LiveBench Coding (category average, as displayed) = 72.6 · aggregated · 2026-10-04
Per task: code_completion 76.087, code_generation 69.014. code_generation is far below Alibaba's LiveCodeBench v6 91.9, but the problem sets and scoring differ, so this is not a like-for-like replication. Agentic Coding on the same release is 61.6 (javascript 68.182, typescript 46.667, python 70.0), level with Qwen3.8 27B (61.4) and the best of the four catalog models on this release. Held at 9 on the agentic result and the vendor SWE-bench rows; plain coding alone would sit level with Qwen3.6 27B (71.8, scored 8). LiveBench release 2026-06-25, category figure as displayed on livebench.ai on 2026-10-04 (the site averages the per-task scores in https://livebench.ai/table_2026_06_25.csv, file last modified 2026-09-29; the GitHub mirror is stale). Independent. Quant and reasoning mode not stated by LiveBench (presumed full-precision API serving).
data checked Oct 2026
- vendor-model-card — GPQA Diamond = 91.7 @ bf16 · vendor-reported · 2026-09-12
Vendor-reported, above Qwen3.8-27B (89.2).
- vendor-model-card — Humanity's Last Exam (HLE) = 35.9 @ bf16 · vendor-reported · 2026-09-12
Broad expert-level reasoning on the same table.
- livebench — LiveBench Mathematics (category average, as displayed) = 85.8 · aggregated · 2026-10-04
Independent support for the vendor tier. Per task: AMPS_Hard 99.0, math_comp 94.118, olympiad 91.166, integrals_with_game 59.0. LiveBench release 2026-06-25, category figure as displayed on livebench.ai on 2026-10-04 (the site averages the per-task scores in https://livebench.ai/table_2026_06_25.csv, file last modified 2026-09-29; the GitHub mirror is stale). Independent. Quant and reasoning mode not stated by LiveBench (presumed full-precision API serving).
data checked Sep 2026
- vendor-model-card — Toolathlon Verified (Pass@1) = 73.5 @ bf16 · vendor-reported · 2026-09-12
Real-world tool use, averaged over three runs by the vendor.
- vendor-model-card — Agents' Last Exam (Pass@1) = 24.3 @ bf16 · vendor-reported · 2026-09-12
Frontier agentic tasks; a low absolute number for every model on the row, and above Qwen3.8-27B's 20.4.
data checked Oct 2026
- vendor-model-card — IFBench = 81.3 @ bf16 · vendor-reported · 2026-09-12
Ahead of Qwen3.8-27B (79.5).
- livebench — LiveBench Instruction Following (category average, as displayed) = 77.1 · aggregated · 2026-10-04
Fifth-highest IF figure in the release table as displayed, ahead of every Claude and GPT entry. Per task: paraphrase 79.467, simplify 70.25, story_generation 81.367, summarize 77.35. Held at 8 rather than raised, because LiveBench's IF tasks are rewriting tasks, narrower than IFBench. LiveBench release 2026-06-25, category figure as displayed on livebench.ai on 2026-10-04 (the site averages the per-task scores in https://livebench.ai/table_2026_06_25.csv, file last modified 2026-09-29; the GitHub mirror is stale). Independent. Quant and reasoning mode not stated by LiveBench (presumed full-precision API serving).
§2 Known issues & what fixes them
On Apple silicon, decode can emit a single token and then stop once the context grows long, after a prefill that looks healthy. The reporter traced it on their own setup to a quantized KV cache (-ctk/-ctv q4_0) combined with MTP speculative decoding: verification takes a fast path that assumes unquantized keys. Their fix is a local patch, and the issue is still open as of 2026-10-04. On CUDA, a separate open report has decode slowing linearly with context rather than staying flat.
Until there is an upstream fix: keep the KV cache unquantized (drop -ctk/-ctv q4_0) or run without the MTP draft model. This follows from the reporter's diagnosis and has not been confirmed by a maintainer. An f16 cache at 256K did not fit the reporter's 64GB machines, so expect to trade context for it. Check output length rather than assuming a reply finished.
Evidence · 3 sources · community-consensus
- Eval bug: qwen4exp (Qwen3.8-Flash-Next) on Metal: decode emits 1 token then EOS at long context (open) (github, 2026-10-04)
- Reporter's root cause: quantized KV cache plus MTP verification fast path (github, 2026-10-04)
- Eval bug: qwen4exp (Qwen3.8-Flash-Next) CUDA: decode slows linearly with context (open) (github, 2026-10-04)
51B of the 180B parameters are an n-gram embedding table rather than transformer weights, about 27 GiB at 4-bit. Whether you have to hold it depends on the runtime. Since late August, llama.cpp reads the table lazily from disk by default, so on CUDA and CPU it does not need to sit in RAM or VRAM. On Apple silicon it still counts: current Unsloth files place the table between GPU tensors, and Metal maps the whole span, so UD-Q4_K_XL takes 106,166 MiB of GPU working set before any context. That is over the roughly 96 GiB a 128GB Mac allows by default (open bug). The sizes on this site include the table, which is right for a Mac and overstates what a PC must hold. Separately, one earlier llama.cpp build silently did not load the table at all (since fixed).
Use a build from after 2026-09-04 and confirm the loader reports the n-gram tensors; a model that answers but has lost 51B parameters will not announce it.
On a Mac, a community repack that moves the table to the end of the file (bit-identical tensors) cut the Metal mapping to 78,701 MiB in the bug report; otherwise wait for the loader fix.
Evidence · 4 sources · community-consensus
- llama: model_loader: add TENSOR_READ_LAZY (merged 2026-08-27) (github, 2026-10-04)
- Metal: per-file mmap mapping spans CPU-resident tensors, so a 27 GiB lazy-read table counts against the GPU working set (Qwen3.8-Flash-Next OOM) (open) (github, 2026-10-04)
- Qwen3.8-Flash-Next: 51B N-gram embedding table not loaded after build 10665 (closed 2026-09-04) (github, 2026-10-04)
- Qwen3.8-Flash-Next model card (vendor, 2026-09-12)
Tool-calling scores are still Alibaba's own: the model is absent from the Agents' Last Exam leaderboard, and no independent agentic benchmark lists it. Coding, math and instruction following now have one independent check, LiveBench, where math and instruction following hold up and plain coding comes in well under the vendor's LiveCodeBench figure. The model is also explicitly an experimental preview of the Qwen4 architecture, not a settled release.
Not a defect, a caveat. Qwen3.6 27B is on this site partly because it has had months of independent checking; this has had one leaderboard.
Evidence · 3 sources · anecdotal
- Qwen3.8-Flash-Next model card (vendor benchmark table) (vendor, 2026-09-12)
- Agents' Last Exam leaderboard (model not listed as of 2026-10-04) (leaderboard, 2026-10-04)
- LiveBench leaderboard, release 2026-06-25 (leaderboard, 2026-10-04)
Six billion active parameters doing the work of a 180B file means 3-bit has little to take from. The band saves 21GB over 4-bit, which only matters on a machine that cannot reach 111GB at all.
Q4_K_XL at 111GB if the machine can hold it — that is the difference between a 96GB and a 128GB machine here.
Where arithmetic broke, hand it to a tool; tool calling still scores 5 at this band, so the fix mostly works.
Evidence · 2 sources · community-consensus
- llama.cpp quantization quality discussions (sparse MoE KLD tables) (github, 2026-09-12)
- Qwen3.8-Flash-Next quant sizes: UD-Q3_K_XL 90GB against UD-Q4_K_XL 111GB (other, 2026-09-12)
Two user reports of the model losing track of reality inside agent loops. One describes it inventing unrelated user tasks, misreading tool output it had just received, and reconstructing system instructions that were never given; that report says Qwen3.8 27B does it too. The other describes it inventing URLs for web-fetch calls. Neither has a response from Qwen as of 2026-10-04.
No fix reported. In an agent harness, check tool results and fetched URLs against what the model claims before acting on them.