Known-issues sheets for local language models Rev 0.1.0 · reviewed every 7 days
llm-picker

Model datasheet · GLM-5

GLM-5.3-Flash

Replaces GLM-4.7 in the same size class: 320B against 358B, MIT again. Z.ai reports Terminal-Bench 2.1 at 84.3, but the one independent rerun so far got 68.3% per trial, and the card does not say how 84.3 was counted. The architectural change is what matters here: 34 of 45 layers are linear attention and the other 11 cache a 512-wide latent, so a million tokens of context costs about 11GB where GLM-4.7 charged 368KB per token. The weights are the problem instead: 109GB at 2-bit, 200GB at 4-bit. llama.cpp support was merged on 2026-09-30 and is still settling.

Vendor
Z.ai
Architecture
MoE · 321B total / ~18B active
Context
1,048,576 tokens
License
MIT
Released
2026-08-25
Vision
Yes

→ GGUF (llama.cpp / LM Studio / Ollama) → MLX (Apple Silicon)

Architecture, from its own config.json

45 layers · 34 linear / recurrent · 11 full attention

Only 11 of 45 layers hold a cache at all, which is why its memory grows so much more slowly with context than its size suggests.

64 attention heads, multi-head latent attention — it caches one 512-wide compressed vector per layer instead of per-head keys and values, which is why its cache is small for its size · mixture of experts, 18B of 321B active per token across 8 experts · vocabulary 154,880

Where the memory goes: weights by band above, plus a KV cache of 88MB at 8K · 352MB at 32K · 1.4GB at 128K, plus the runtime's own working memory — dominated by the output projection over that vocabulary.

Inside one layer, from its own tensors

34 × linear / recurrent layer

input_layernormattentionq_projk_projv_projsoftmax(q·kᵀ/√d) · v64 query heads over 64 key/valueheadso_projpost_attention_layernormmixture of expertsrouter picks 8 of 288 expertseach 2,048 wide: gate·up → downplus a shared expert, run for everytoken

11 × full-attention layer

input_layernormlatent attentionq_a_proj →q_a_layernorm →q_b_projkv_a_proj_with_mqa →kv_a_layernorm →kv_b_projsoftmax(q·kᵀ/√d) · vone 512-wide latent is cached, notper-head keys and valueso_projpost_attention_layernormmixture of expertsrouter picks 8 of 288 expertseach 2,048 wide: gate·up → downplus a shared expert, run for everytoken

The green line down the right of each half is the residual: it leaves the spine above the norm, skips the operation entirely and rejoins at the +. Where attention is drawn open, q, k and v are scaled to their real output widths — a model that shares key and value heads across many query heads shows one wide box beside two narrow ones.

Around the stack: an embedding of 154,880 × 4,096, a final norm, and a separate output projection. The first 3 layers use a dense feed-forward instead of the experts.

On the norm names: despite what it is called, post_attention_layernorm is drawn after the residual add here, which is where this family applies it — it normalises the input to the feed-forward, not the output of attention. Models that carry a separate pre_feedforward_layernorm use it the other way, and are drawn that way.

Also in every block and not drawn, because their tensor names do not fix where they belong: hc_attn_base, hc_attn_fn, hc_attn_scale, hc_ffn_base, hc_ffn_fn, hc_ffn_scale, A_log, b_proj, dt_bias, f_a_proj, f_b_proj, g_a_proj, g_b_proj, k_conv1d, o_norm, q_conv1d, v_conv1d, e_score_correction_bias, index_kpool_compress_ape, index_kpool_compress_gate, weights_proj, wk, wq_b. Guessing an order is the one thing this diagram will not do.

§1 Characteristics by quantization band

Q2–Q3 — 2–3 bit

120GB weights · e.g. IQ3_XXS , Q2_K_XL + KV: 88MB at 8K · 352MB at 32K · 1.4GB at 128K
Coding 6/10
data checked Sep 2026
  • quant-degradation-community — editorial band derivation @ Q2_K_XL · aggregated · 2026-09-12

    Editorial derivation: two points down from bf16, and the only band most machines can reach at all: 120GB against 200GB at 4-bit.

Tool calling / agentic 5/10
data checked Sep 2026
  • quant-degradation-community — editorial band derivation @ Q2_K_XL · aggregated · 2026-09-12

    Editorial derivation: two points down; 2-bit is the most aggressive band this site will quote and the one where agent loops break first.

Math & reasoning 6/10
data checked Oct 2026
  • quant-degradation-community — editorial band derivation @ Q2_K_XL · aggregated · 2026-10-04

    Editorial derivation: two points down from bf16, as for coding.

Instruction following 4/10
data checked Oct 2026
  • quant-degradation-community — editorial band derivation @ Q2_K_XL · aggregated · 2026-10-04

    Editorial derivation: one point down at 2-bit, from an already weak full-precision figure.

Q4–Q5 — 4–5 bit

200GB weights · e.g. Q4_K_XL + KV: 88MB at 8K · 352MB at 32K · 1.4GB at 128K
Coding 8/10
data checked Sep 2026
  • quant-degradation-community — editorial band derivation @ Q4_K_XL · aggregated · 2026-09-12

    Editorial derivation: one point down.

Tool calling / agentic 7/10
data checked Sep 2026
  • quant-degradation-community — editorial band derivation @ Q4_K_XL · aggregated · 2026-09-12

    Editorial derivation: one point down.

Math & reasoning 7/10
data checked Oct 2026
  • quant-degradation-community — editorial band derivation @ Q4_K_XL · aggregated · 2026-10-04

    Editorial derivation: one point down, as for coding.

Instruction following 5/10
data checked Oct 2026
  • quant-degradation-community — editorial band derivation @ Q4_K_XL · aggregated · 2026-10-04

    Editorial derivation: carried; format compliance survives 4-bit better than arithmetic.

Q6–Q8 — 6–8 bit

292GB weights · e.g. Q6_K_XL , Q8_0 + KV: 88MB at 8K · 352MB at 32K · 1.4GB at 128K
Coding 9/10
data checked Sep 2026
  • quant-degradation-community — editorial band derivation @ Q6_K_XL · aggregated · 2026-09-12

    Editorial derivation: carried from bf16.

Tool calling / agentic 8/10
data checked Sep 2026
  • quant-degradation-community — editorial band derivation @ Q6_K_XL · aggregated · 2026-09-12

    Editorial derivation: carried from bf16.

Math & reasoning 8/10
data checked Oct 2026
  • quant-degradation-community — editorial band derivation @ Q6_K_XL · aggregated · 2026-10-04

    Editorial derivation: carried from bf16.

Instruction following 5/10
data checked Oct 2026
  • quant-degradation-community — editorial band derivation @ Q6_K_XL · aggregated · 2026-10-04

    Editorial derivation: carried from bf16.

FP16 — full precision (fp16/bf16)

642GB weights · e.g. bf16 + KV: 88MB at 8K · 352MB at 32K · 1.4GB at 128K
Coding 9/10
data checked Oct 2026
  • vendor-model-card — Terminal-Bench 2.1 = 84.3 @ bf16 · vendor-reported · 2026-09-12

    Z.ai's own evaluation chart, run in Claude Code 2.1.207 by their stated protocol. Above GLM-5.2 (81.0) and within a point of the closed models they compare against. The card does not say how 84.3 was aggregated (pass@1, trial mean or pass@k). The one independent rerun found so far got 68.31% trial-level accuracy over 89 tasks x 5 trials (pass@5 93.26%) through Z.ai's own API; see the known issue.

  • vendor-model-card — DeepSWE v1.1 = 63.4 @ bf16 · vendor-reported · 2026-09-12

    Agentic software engineering under the mini-swe-agent harness, against GLM-5.2's 46.2.

  • livebench — LiveBench Coding (category average, as displayed) = 79 · aggregated · 2026-10-04

    The highest Coding category figure of the four catalog models on this release (Qwen3.8 27B 75.7, Qwen3.8 Flash Next 72.6, Qwen3.6 27B 71.8); per task, code_completion 80.435, code_generation 77.465. Agentic Coding on the same release is 56.8 (javascript 63.636, typescript 46.667, python 60.0). Held at 9: the independent coding picture is as strong as Qwen3.8 Flash Next's, which also scores 9, even though the headline vendor Terminal-Bench figure has not been reproduced. LiveBench release 2026-06-25, category figure as displayed on livebench.ai on 2026-10-04 (the site averages the per-task scores in https://livebench.ai/table_2026_06_25.csv, file last modified 2026-09-29; the GitHub mirror is stale). Independent. Quant and reasoning mode not stated by LiveBench (presumed full-precision API serving).

Tool calling / agentic 8/10
data checked Sep 2026
  • vendor-model-card — AutomationBench v1.0.6 = 48.8 @ bf16 · vendor-reported · 2026-09-12

    Tool-driven automation; the highest open-weight figure on Z.ai's chart and behind Gemini 3.7 Flash (52.3).

  • vendor-model-card — Agents' Last Exam = 26.3 @ bf16 · vendor-reported · 2026-09-12

    Frontier agentic tasks — a low absolute number for every model on the chart.

Math & reasoning 8/10
data checked Oct 2026
  • livebench — LiveBench Mathematics (category average, as displayed) = 81.2 · aggregated · 2026-10-04

    The first math measurement of this model anywhere: Z.ai published none. Per task: AMPS_Hard 97.0, math_comp 93.137, olympiad 89.841, integrals_with_game 45.0. Scored level with Qwen3.6 27B (79.9 on the same release, 8 here) and Qwen3.8 Flash Next (85.8, 8 here); a frontier-heavy board, where the median is about 90. LiveBench release 2026-06-25, category figure as displayed on livebench.ai on 2026-10-04 (the site averages the per-task scores in https://livebench.ai/table_2026_06_25.csv, file last modified 2026-09-29; the GitHub mirror is stale). Independent. Quant and reasoning mode not stated by LiveBench (presumed full-precision API serving).

Instruction following 5/10
data checked Oct 2026
  • livebench — LiveBench Instruction Following (category average, as displayed) = 52.8 · aggregated · 2026-10-04

    The first instruction-following measurement of this model, and a weak one: the lowest IF figure in the release table as displayed, just under Qwen3.6 27B (53.2) and far under Qwen3.8 Flash Next (77.1). Per task: paraphrase 47.983, simplify 46.95, story_generation 59.883, summarize 56.467. LiveBench's IF tasks are rewriting tasks graded for constraint adherence, which is narrower than IFBench. LiveBench release 2026-06-25, category figure as displayed on livebench.ai on 2026-10-04 (the site averages the per-task scores in https://livebench.ai/table_2026_06_25.csv, file last modified 2026-09-29; the GitHub mirror is stale). Independent. Quant and reasoning mode not stated by LiveBench (presumed full-precision API serving).

§2 Known issues & what fixes them

ISSUE-01 moderate Q2–Q3 / Q4–Q5 / Q6–Q8 / FP16 · coding, tool-calling

llama.cpp support was merged on 2026-09-30 and first shipped in release b11279, so the model runs on stock builds now, but the code is days old and still moving. A follow-up commit reserves worst-case memory for the indexer cache, so a context that used to load can now run out of VRAM (open). On Apple silicon, decode stalled for 1-9 seconds per token; that is fixed on master and in releases from b11372, with a graph-reallocation problem above about 2,048 tokens still being investigated. CUDA illegal memory access on long prefill (Blackwell) and a SOFT_MAX failure on Turing cards are still open as of 2026-10-04.

Fix
Update your runtime [partial]

Use b11372 or later, which has the Metal stall fix. If a context that used to fit now runs out of memory, lower -c or -ub until the VRAM regression is resolved; the reporter's 307,200-token setup loaded at 204,800.

No fix
No known fix — choose a different model

Nothing on the prompt side fixes a backend that crashes on prefill; on Blackwell and Turing cards, wait for the open CUDA fixes.

Evidence · 5 sources · community-consensus
ISSUE-02 moderate Q2–Q3 / Q4–Q5 / Q6–Q8 / FP16 · long-context

The opposite problem from GLM-4.7. Only 11 of 45 layers cache, and each holds one 512-wide latent, so a full million-token context costs about 11GB — roughly a thirtieth of what the previous generation charged per token. What you cannot get around is 120GB of weights at the smallest band this site will quote.

No fix
No known fix — choose a different model

Architectural, and in your favour for once: if the weights fit, the context will too. The RUNS AT line prices both.

Evidence · 2 sources · community-consensus
ISSUE-03 mild Q2–Q3 / Q4–Q5 / Q6–Q8 / FP16 · coding, tool-calling, factuality

Z.ai publishes six benchmark results for this model as a chart and nothing else: no table, no per-benchmark methodology beyond footnotes, and no statement of how Terminal-Bench was aggregated. Math and instruction following are rated here from LiveBench alone, because Z.ai published no measurement of either. Factuality is unrated because nobody has measured it, not because the model is weak at it.

No fix
No known fix — choose a different model

Not a defect, a caveat. The blank is honest.

Evidence · 1 source · anecdotal
ISSUE-04 mild Q2–Q3 / Q4–Q5 / Q6–Q8 / FP16 · coding

The headline Terminal-Bench 2.1 figure (84.3) has not been reproduced. One user ran all 89 tasks five times each through Z.ai's own API with the harness Z.ai names (Claude Code 2.1.207, reasoning effort max) and got 68.31% of trials passing; 93.26% of tasks passed at least once in five tries. Z.ai's card does not say which of those 84.3 measures, and the question was unanswered as of 2026-10-04. One reproducer, so treat it as a flag rather than a verdict.

No fix
No known fix — choose a different model

Not a defect, a caveat. The coding score leans on LiveBench, which is independent, rather than on this number.

Evidence · 1 source · anecdotal