Known-issues sheets for local language models Rev 0.1.0 · reviewed every 7 days
llm-picker

Model datasheet · Qwen3.5

Qwen3.5 4B

Replaces Qwen3 4B: vision, 256K context, and a cheap cache, since only 8 of its 32 layers hold KV. Two competitors have re-run it and broadly agree with Alibaba's numbers (OpenBMB measured IFBench 59.0 against the vendor's 59.2). It is a thinking model first, and newer small models now beat it on agentic coding: OpenBMB's own run puts its SWE-bench Verified at 33.6.

Vendor
Alibaba
Architecture
Dense · 4B
Context
262,144 tokens
License
Apache-2.0
Released
2026-03-02
Vision
Yes

→ GGUF (llama.cpp / LM Studio / Ollama) → MLX (Apple Silicon)

Architecture, from its own config.json

32 layers · 24 linear / recurrent · 8 full attention

Only 8 of 32 layers hold a cache at all, which is why its memory grows so much more slowly with context than its size suggests.

16 attention heads over 4 KV heads (GQA 4:1), head dim 256 · dense feed-forward · vocabulary 248,320

Where the memory goes: weights by band above, plus a KV cache of 256MB at 8K · 1GB at 32K · 4GB at 128K, plus the runtime's own working memory — dominated by the output projection over that vocabulary.

Inside one layer, from its own tensors

24 × linear / recurrent layer

input_layernormgated delta-net mixerin_proj · conv1d · out_proja fixed-size recurrent state, sothis layer does not grow withcontextpost_attention_layernormfeed-forwardgate_proj · up_proj → down_proj9,216 wide

8 × full-attention layer

input_layernormattentionq_projkvsoftmax(q·kᵀ/√d) · vq_norm · k_norm before the dotproduct16 query heads over 4 key/valueheadso_projpost_attention_layernormfeed-forwardgate_proj · up_proj → down_proj9,216 wide

The green line down the right of each half is the residual: it leaves the spine above the norm, skips the operation entirely and rejoins at the +. Where attention is drawn open, q, k and v are scaled to their real output widths — a model that shares key and value heads across many query heads shows one wide box beside two narrow ones.

Around the stack: an embedding of 248,320 × 2,560, a final norm, and an output projection that reuses the embedding (tied weights).

On the norm names: despite what it is called, post_attention_layernorm is drawn after the residual add here, which is where this family applies it — it normalises the input to the feed-forward, not the output of attention. Models that carry a separate pre_feedforward_layernorm use it the other way, and are drawn that way.

§1 Characteristics by quantization band

Q2–Q3 — 2–3 bit

2.3GB weights · e.g. Q3_K_M + KV: 256MB at 8K · 1GB at 32K · 4GB at 128K
Math & reasoning 5/10
data checked Oct 2026
  • quant-degradation-community — editorial band derivation @ Q3_K_M · aggregated · 2026-10-04

    Editorial derivation: two points down, and Q3_K_M is only 0.45GB smaller than Q4_K_M at this size.

Coding 3/10
data checked Oct 2026
  • quant-degradation-community — editorial band derivation @ Q3_K_M · aggregated · 2026-10-04

    Editorial derivation: two points down.

Tool calling / agentic 3/10
data checked Oct 2026
  • quant-degradation-community — editorial band derivation @ Q3_K_M · aggregated · 2026-10-04

    Editorial derivation: two points down; structured output degrades fastest at 3-bit on a small dense model.

Q4–Q5 — 4–5 bit

2.7GB weights · e.g. Q4_K_M , mlx-4bit + KV: 256MB at 8K · 1GB at 32K · 4GB at 128K
Math & reasoning 6/10
data checked Oct 2026
  • quant-degradation-community — editorial band derivation @ Q4_K_M · aggregated · 2026-10-04

    Editorial derivation: one point down. Small dense models have the least redundancy to spend, and math goes first.

Coding 4/10
data checked Oct 2026
Tool calling / agentic 4/10
data checked Oct 2026
Instruction following 6/10
data checked Oct 2026
  • quant-degradation-community — editorial band derivation @ Q4_K_M · aggregated · 2026-10-04

    Editorial derivation: carried; format compliance survives 4-bit better than arithmetic does.

Long context 5/10
data checked Oct 2026
  • quant-degradation-community — editorial band derivation @ Q4_K_M · aggregated · 2026-10-04

    Editorial derivation: carried; the cache is fp16 regardless of the weight quant.

Q6–Q8 — 6–8 bit

3.5GB weights · e.g. Q6_K , Q8_0 + KV: 256MB at 8K · 1GB at 32K · 4GB at 128K
Math & reasoning 7/10
data checked Oct 2026
  • quant-degradation-community — editorial band derivation @ Q6_K · aggregated · 2026-10-04

    Editorial derivation: carried from the full band.

Coding 5/10
data checked Oct 2026
  • quant-degradation-community — editorial band derivation @ Q6_K · aggregated · 2026-10-04

    Editorial derivation: carried from the full band.

Tool calling / agentic 5/10
data checked Oct 2026
  • quant-degradation-community — editorial band derivation @ Q6_K · aggregated · 2026-10-04

    Editorial derivation: carried from the full band.

Instruction following 6/10
data checked Oct 2026
  • quant-degradation-community — editorial band derivation @ Q6_K · aggregated · 2026-10-04

    Editorial derivation: carried from the full band.

Long context 5/10
data checked Oct 2026
  • quant-degradation-community — editorial band derivation @ Q6_K · aggregated · 2026-10-04

    Editorial derivation: carried; long context is not quant-sensitive here.

FP16 — full precision (fp16/bf16)

8.4GB weights · e.g. bf16 + KV: 256MB at 8K · 1GB at 32K · 4GB at 128K
Math & reasoning 7/10
data checked Oct 2026
  • vendor-model-card — GPQA Diamond = 76.2 @ bf16 · vendor-reported · 2026-10-04

    Thinking mode: the card says Qwen3.5 runs in thinking mode by default and compares it against thinking models, though no row states its mode. HMMT Feb 25 74.0 and HMMT Nov 25 76.8 on the same card.

  • vendor-model-card — AIME 2026 = 82.7 @ bf16 · vendor-reported · 2026-10-04

    OpenBMB's own run of Qwen3.5-4B on the MiniCPM5-2B card, i.e. a competitor's reproduction rather than Alibaba's. Cells marked with a dagger there are Artificial Analysis figures and are not used; this one is not marked.

  • vendor-model-card — GPQA Diamond = 78.2 @ bf16 · vendor-reported · 2026-10-04

    Nanbeige LLM Lab's run of Qwen3.5-4B in the Nanbeige4.2 technical report (Table 3), i.e. a competitor's measurement rather than Alibaba's.

Coding 5/10
data checked Oct 2026
  • vendor-model-card — LiveCodeBench v6 = 55.8 @ bf16 · vendor-reported · 2026-10-04

    Thinking mode: the card says Qwen3.5 runs in thinking mode by default and compares it against thinking models, though no row states its mode.

  • vendor-model-card — LiveCodeBench v6 = 56.4 @ bf16 · vendor-reported · 2026-10-04

    OpenBMB's own run of Qwen3.5-4B on the MiniCPM5-2B card, i.e. a competitor's reproduction rather than Alibaba's. Cells marked with a dagger there are Artificial Analysis figures and are not used; this one is not marked.

  • vendor-model-card — SWE-bench Verified = 33.6 @ bf16 · vendor-reported · 2026-10-04

    OpenBMB's own run of Qwen3.5-4B on the MiniCPM5-2B card, i.e. a competitor's reproduction rather than Alibaba's. Cells marked with a dagger there are Artificial Analysis figures and are not used; this one is not marked. Alibaba publishes no SWE-bench figure for this size.

  • vendor-model-card — SWE-bench Verified = 38.8 @ bf16 · vendor-reported · 2026-10-04

    Nanbeige LLM Lab's run of Qwen3.5-4B in the Nanbeige4.2 technical report (Table 3), i.e. a competitor's measurement rather than Alibaba's. Absent from the Aider polyglot leaderboard as of 2026-10-04.

Tool calling / agentic 5/10
data checked Oct 2026
  • vendor-model-card — BFCL v4 = 50.3 @ bf16 · vendor-reported · 2026-10-04

    Thinking mode: the card says Qwen3.5 runs in thinking mode by default and compares it against thinking models, though no row states its mode. TAU2-Bench 79.9 on the same card.

  • vendor-model-card — BFCL v4 = 56.8 @ bf16 · vendor-reported · 2026-10-04

    OpenBMB's own run of Qwen3.5-4B on the MiniCPM5-2B card, i.e. a competitor's reproduction rather than Alibaba's. Cells marked with a dagger there are Artificial Analysis figures and are not used; this one is not marked.

Instruction following 6/10
data checked Oct 2026
  • vendor-model-card — IFBench = 59.2 @ bf16 · vendor-reported · 2026-10-04

    Thinking mode: the card says Qwen3.5 runs in thinking mode by default and compares it against thinking models, though no row states its mode. IFEval 89.8 on the same card.

  • vendor-model-card — IFBench = 59 @ bf16 · vendor-reported · 2026-10-04

    OpenBMB's own run of Qwen3.5-4B on the MiniCPM5-2B card, i.e. a competitor's reproduction rather than Alibaba's. Cells marked with a dagger there are Artificial Analysis figures and are not used; this one is not marked. IFEval 90.2 in the same column.

  • vendor-model-card — IFBench = 41.4 @ bf16 · vendor-reported · 2026-10-04

    Nanbeige LLM Lab's run of Qwen3.5-4B in the Nanbeige4.2 technical report (Table 3), i.e. a competitor's measurement rather than Alibaba's. Eighteen points below the other two runs, with no setup difference stated; treated as an outlier.

Long context 5/10
data checked Oct 2026
  • vendor-model-card — LongBench v2 = 50 @ bf16 · vendor-reported · 2026-10-04

    Thinking mode: the card says Qwen3.5 runs in thinking mode by default and compares it against thinking models, though no row states its mode. The card's AA-LCR row is an Artificial Analysis benchmark and is not recorded.

  • vendor-model-card — NoLiMa = 43.5 @ bf16 · vendor-reported · 2026-10-04

    OpenBMB's own run of Qwen3.5-4B on the MiniCPM5-2B card, i.e. a competitor's reproduction rather than Alibaba's. Cells marked with a dagger there are Artificial Analysis figures and are not used; this one is not marked. LongBench v2 47.3 and LongBenchPro 58.4 in the same column.

§2 Known issues & what fixes them

ISSUE-01 mild Q2–Q3 / Q4–Q5 / Q6–Q8 / FP16 · math, coding, long-context

Qwen recommend keeping at least 128K tokens of context "to preserve thinking capabilities", and 32K to 81K of output for hard problems. The cache is cheap, 32KB per token, but 128K of it is still 4.3GB, more than the Q4_K_M weights, which matters on the 8GB machines this size is for.

Fix
Prompt technique [partial]

On a small machine, run a shorter context and accept shorter reasoning, or switch thinking off (enable_thinking: false) for tasks that do not need it.

Evidence · 1 source · anecdotal
ISSUE-02 mild Q2–Q3 / Q4–Q5 / Q6–Q8 / FP16 · instruction-following, tool-calling

The two competitor re-runs mostly agree with Alibaba, but not everywhere: Nanbeige measured IFBench at 41.4 against 59.2 (vendor) and 59.0 (OpenBMB), and OpenBMB measured BFCL v4 at 56.8 against the vendor's 50.3. Harness and mode settings are not reconciled, so treat single-digit differences as noise. Absent from Aider polyglot and Vectara HHEM as of 2026-10-04.

No fix
No known fix — choose a different model

A caveat, not a defect.

Evidence · 2 sources · community-consensus
ISSUE-03 moderate Q2–Q3 · math, coding, tool-calling

At 4B dense there is little redundancy for 3-bit to eat into, and in the Qwen3.5 hybrid layout the attention tensors are the ones most sensitive to quantization. Q3_K_M saves only 0.45GB over Q4_K_M here, so the band costs about two points on every quant-sensitive dimension for almost nothing.

Fix
Move up a quant band [strong]

Q4_K_M is 2.7GB and fits any machine that fits Q3_K_M. Run that, or Q6_K at 3.5GB.

Fix
Code interpreter / calculator tool [partial]

Where arithmetic is what broke, hand it to a tool instead of the weights.

Evidence · 2 sources · community-consensus