Known-issues sheets for local language models Rev 0.1.0 · reviewed every 7 days
llm-picker

Model datasheet · GLM

GLM-4.7-Flash

The agentic-coding MoE of the moment: SWE-bench 59.2 from 3B active params, MIT-licensed, ~200K context. Watch it at long context, where its tool calls start fraying.

Vendor
Z.ai
Architecture
MoE · 30B total / ~3B active
Context
202,752 tokens
License
MIT
Released
2026-01-19
Vision
No

→ GGUF (llama.cpp / LM Studio / Ollama) → MLX (Apple Silicon)

Architecture, from its own config.json

20 attention heads, multi-head latent attention — it caches one 576-wide compressed vector per layer instead of per-head keys and values, which is why its cache is small for its size · mixture of experts, 3B of 30B active per token across 4 experts · vocabulary 154,880

Where the memory goes: weights by band above, plus a KV cache of 423MB at 8K · 1.7GB at 32K · 6.6GB at 128K, plus the runtime's own working memory — dominated by the output projection over that vocabulary.

No per-layer split is drawn because latent attention does not have one in the usual sense: every layer caches the same compressed vector, so there is no full-versus-sliding distinction to show.

Inside one layer, from its own tensors

47 × every layer

input_layernormlatent attentionq_a_proj →q_a_layernorm →q_b_projkv_a_proj_with_mqa →kv_a_layernorm →kv_b_projsoftmax(q·kᵀ/√d) · vone 576-wide latent is cached, notper-head keys and valueso_projpost_attention_layernormmixture of expertsrouter picks 4 of 64 expertseach 1,536 wide: gate·up → downplus a shared expert, run for everytoken

The green line down the right of each half is the residual: it leaves the spine above the norm, skips the operation entirely and rejoins at the +. Where attention is drawn open, q, k and v are scaled to their real output widths — a model that shares key and value heads across many query heads shows one wide box beside two narrow ones.

Around the stack: an embedding of 154,880 × 2,048, a final norm, and a separate output projection. The first 1 layer use a dense feed-forward instead of the experts.

On the norm names: despite what it is called, post_attention_layernorm is drawn after the residual add here, which is where this family applies it — it normalises the input to the feed-forward, not the output of attention. Models that carry a separate pre_feedforward_layernorm use it the other way, and are drawn that way.

§1 Characteristics by quantization band

Q2–Q3 — 2–3 bit

15GB weights · e.g. Q3_K_M + KV: 423MB at 8K · 1.7GB at 32K · 6.6GB at 128K

— Not yet rated · contributions welcome —

Q4–Q5 — 4–5 bit

18GB weights · e.g. Q4_K_M , mlx-4bit + KV: 423MB at 8K · 1.7GB at 32K · 6.6GB at 128K
Math & reasoning 7/10
data checked Aug 2026
  • quant-degradation-community — KLD/perplexity delta vs bf16 @ Q4_K_M · aggregated · 2026-08-30

    Editorial derivation: MoE expert weights lose a little at 4-bit.

Coding 6/10
data checked Aug 2026
  • quant-degradation-community — KLD/perplexity delta vs bf16 @ Q4_K_M · aggregated · 2026-08-30

    Editorial derivation: small loss at 4-bit.

Tool calling / agentic 6/10
data checked Aug 2026
  • quant-degradation-community — community consensus @ Q4_K_M · aggregated · 2026-08-30

    Editorial derivation: quant plus long context compounds the tool-call fraying — keep context modest at 4-bit.

Creative writing 5/10
data checked Oct 2026
  • quant-degradation-community — editorial band derivation @ Q4_K_M · aggregated · 2026-10-04

    Editorial derivation: carried from the EQ-Bench-anchored full band; creative writing is not quant-sensitive.

Q6–Q8 — 6–8 bit

25GB weights · e.g. Q6_K + KV: 423MB at 8K · 1.7GB at 32K · 6.6GB at 128K

— Not yet rated · contributions welcome —

FP16 — full precision (fp16/bf16)

60GB weights · e.g. bf16 + KV: 423MB at 8K · 1.7GB at 32K · 6.6GB at 128K
Math & reasoning 8/10
data checked Aug 2026
Coding 7/10
data checked Aug 2026
Tool calling / agentic 7/10
data checked Aug 2026
  • vendor-model-card — Tau2-Bench = 79.5 @ bf16 · vendor-reported · 2026-08-30

    Strong at short context; see the known issue on long-context degradation.

Factuality 4/10
data checked Aug 2026
Creative writing 5/10
data checked Oct 2026
  • eqbench-creative — EQ-Bench Creative Writing v3 rubric score (0-20) = 13.68 @ bf16 · aggregated · 2026-10-04

    Normalised Elo 1124.7, 110th of 140, retrieved 2026-10-04: level with Gemma 3 12B (1125.5, scored 5), though its rubric score is lower (14.63 for the Gemma). Quant not stated by EQ-Bench (presumed full-precision API serving). Independent; rubric judged by Claude Sonnet 4. EQ-Bench's normalised Elo moves as models are added, so the rubric score is the figure stored. Data file behind https://eqbench.com/creative_writing.html; the date the row was added is not published.

  • eqbench-creative — EQ-Bench Creative Writing Longform score (0-100) = 47.8 @ bf16 · aggregated · 2026-10-04

    Longform: ahead of all three Gemma 3 sizes (34.4-41.9), so it holds up better over long pieces than its short-form rubric suggests. Quant not stated by EQ-Bench (presumed full-precision API serving). Data file behind the Longform page on eqbench.com.

§2 Known issues & what fixes them

ISSUE-01 moderate Q2–Q3 / Q4–Q5 / Q6–Q8 / FP16 · tool-calling, long-context

Output quality and tool-call formatting degrade as context grows — malformed JSON and artifacts appear well before the ~200K window is full.

Fix
Update your runtime [partial]

Recent llama.cpp builds fixed part of this (flash-attention path); update first.

Fix
Prompt technique [partial]

Cap working context well below the maximum and summarize long agent sessions periodically.

Evidence · 2 sources · community-consensus
ISSUE-02 moderate Q2–Q3 / Q4–Q5 / Q6–Q8 / FP16 · factuality

Hallucinates more than its peers (HHEM 9.3% vs 5–7% for Qwen3/Gemma 3) — the price of its agentic tuning.

Fix
RAG (retrieval) [strong]

Ground factual work; keep it in its lane (code and agents).

Evidence · 1 source · anecdotal