Known-issues sheets for local language models Rev 0.1.0 · reviewed every 7 days
llm-picker

Model datasheet · DeepSeek-V4

DeepSeek-V4-Flash-0731

DeepSeek's 284B MoE with 13B active, in its text-only July release. The experts were trained in MXFP4 and ship that way, so Unsloth's 162GB UD-Q8_K_XL is a lossless copy of the release, and the 155-156GB 4-bit files sit close to it (mean KLD 0.013). The cache cannot be worked out from config.json, because three cache types compress at different ratios, so the figure here is read from llama.cpp's own startup log for this checkpoint at 64,000 tokens. In that log, 21 layers hold one 512-wide entry per 4 tokens plus a 128-wide index entry per 4, and 20 layers hold one 512-wide entry per 128 tokens. At 2 bytes per value that is 6,880 bytes per token, about 7GB at a million tokens. The log's total of 423 MiB for those caches agrees, once padding is counted. Small fixed buffers (the 128-token sliding window on every layer, compressor state) are priced separately or not at all. DeepSeek publishes agentic numbers only. Mainline mlx-lm cannot load this architecture yet, so no MLX link is given.

Vendor
DeepSeek
Architecture
MoE · 284B total / ~13B active
Context
1,048,576 tokens
License
MIT
Released
2026-07-31
Vision
No

→ GGUF (llama.cpp / LM Studio / Ollama)

Architecture, from its own config.json

64 attention heads over 1 KV heads (GQA 64:1), head dim 512 · mixture of experts, 13B of 284B active per token across 6 experts · vocabulary 129,280

Where the memory goes: weights by band above, plus a KV cache of 86MB at 8K · 247MB at 32K · 892MB at 128K, plus the runtime's own working memory — dominated by the output projection over that vocabulary.

Its per-layer attention pattern is not published in a form we can derive, so none is drawn. The cache figures above still come from its own config.

Inside one layer

Not drawn. This model names its per-layer norm norm, which fixes no position — it could be the one before attention or the one before the feed-forward, and the name does not say. The memory figures above come from its tensors either way; the block order does not, so it is left out rather than guessed.

§1 Characteristics by quantization band

Q2–Q3 — 2–3 bit

128GB weights · e.g. Q3_K_XL , Q2_K_XL + KV: 86MB at 8K · 247MB at 32K · 892MB at 128K
Coding 6/10
data checked Oct 2026
  • quant-degradation-community — editorial band derivation @ Q3_K_XL · aggregated · 2026-10-04

    Editorial derivation: two points down. Below 4 bits this model loses real precision, because its experts were 4-bit to begin with. Unsloth's table: UD-Q3_K_XL mean KLD 0.158 with 87% top-token agreement, UD-Q2_K_XL 0.408 and 79%. The score is for Q3_K_XL. Q2_K_XL is worse.

Tool calling / agentic 5/10
data checked Oct 2026
  • quant-degradation-community — editorial band derivation @ Q3_K_XL · aggregated · 2026-10-04

    Editorial derivation: two points down on the same KLD evidence. Agent loops break first at low bits, and no one has published a tool-calling run below 4 bits.

Q4–Q5 — 4–5 bit

156GB weights · e.g. Q4_K_XL , mxfp4 + KV: 86MB at 8K · 247MB at 32K · 892MB at 128K
Coding 8/10
data checked Oct 2026
  • quant-degradation-community — editorial band derivation @ Q4_K_XL · aggregated · 2026-10-04

    Editorial derivation: carried from the vendor checkpoint. Unsloth measures UD-Q4_K_XL at mean KLD 0.013 against it, with 96% of tokens matching the top choice. Per Unsloth, the two files differ only in the attention and other non-expert layers, which are BF16 in Q8_K_XL and Q8_0 here.

Tool calling / agentic 7/10
data checked Oct 2026
  • quant-degradation-community — editorial band derivation @ Q4_K_XL · aggregated · 2026-10-04

    Editorial derivation: carried, on the same KLD 0.013 evidence.

Q6–Q8 — 6–8 bit

162GB weights · e.g. Q8_K_XL + KV: 86MB at 8K · 247MB at 32K · 892MB at 128K
Coding 8/10
data checked Oct 2026
  • vendor-model-card — Terminal-Bench 2.1 = 82.7 @ mxfp4 · vendor-reported · 2026-10-04

    DeepSeek's checkpoint is MXFP4 experts with FP8 elsewhere, and there is no BF16 release. Unsloth's UD-Q8_K_XL reproduces it exactly (KLD 0 in their table, and a second converter reports bit-for-bit agreement), so the vendor's figures attach to this band.

    Thinking mode: the card says its code-agent rows ran at the 'max' reasoning effort, temperature 1.0, top_p 0.95, in the minimal mode of DeepSeek Harness. Against GLM-5.3-Flash's 84.3.

  • vendor-model-card — DeepSWE = 54.4 @ mxfp4 · vendor-reported · 2026-10-04

    As above: UD-Q8_K_XL is the vendor checkpoint.

    Max reasoning effort. The preview checkpoint scored 7.3 in the same table, so this release is a large step. Below GLM-5.3-Flash's 63.4, which is why this sits a point under that model's 9. NL2Repo 54.2 in the same table.

Tool calling / agentic 7/10
data checked Oct 2026
  • vendor-model-card — Toolathlon-Verified = 70.3 @ mxfp4 · vendor-reported · 2026-10-04

    As above: UD-Q8_K_XL is the vendor checkpoint.

    The card states the reasoning effort for its code-agent rows only (max), and does not say for this one. Against 49.7 for the preview.

  • vendor-model-card — Agents' Last Exam = 25.2 @ mxfp4 · vendor-reported · 2026-10-04

    As above: UD-Q8_K_XL is the vendor checkpoint.

    Low absolute numbers for every model on this benchmark. The card also lists AutomationBench Public at 25.1, a different variant from the v1.0.6 other vendors report, so the two cannot be compared.

§2 Known issues & what fixes them

ISSUE-01 moderate Q2–Q3 / Q4–Q5 / Q6–Q8 · coding, tool-calling, long-context

Mainline llama.cpp has run V4-Flash since 29 June, but several faults tied to particular hardware or settings were still open on 4 October. Reported problems: garbled output on Strix Halo under ROCm with unified memory, deterministic character-level corruption when copying exact strings on DGX Spark (GB10), the model forgetting earlier conversation when the server runs a unified KV cache across several slots, and VRAM growing by about 10MB per turn with the DSpark drafter until it runs out.

Fix
Update your runtime [partial]

Rebuild often and read the open issues for your backend before blaming the weights. Reporters avoided the forgetting with -np 1 or --no-kv-unified, and the memory growth by running without the DSpark drafter.

No fix
No known fix — choose a different model

Nothing on the prompt side fixes a backend bug.

Evidence · 4 sources · community-consensus
ISSUE-02 moderate Q2–Q3 / Q4–Q5 / Q6–Q8 · tool-calling, long-context

In long agent sessions with many tools (onsets reported between 92K and 350K tokens), the model can stall at the point where it should emit a tool call, repeating a short line such as 'Run.' until it is cut off. One team reproduced this from a single fixed conversation on three serving stacks that share no code, including 6 of 6 trials on a hosted endpoint, which points at the checkpoint rather than a runtime. A second team confirmed it on their own vLLM deployment. Not yet reported on llama.cpp specifically.

Fix
Prompt technique [partial]

Both teams suggest stripping the repeated sentences out of the history you resend. The second measured it, with frequency_penalty 0.1, on live sessions and says it broke the escalation. The first notes that frequency_penalty alone stops verbatim loops but lets reworded ones come back. This needs control over the agent harness.

Evidence · 1 source · anecdotal
ISSUE-03 moderate Q2–Q3 · coding, tool-calling

The experts were trained in 4-bit and ship that way, so a 2-3 bit file is not trimming spare precision. It is cutting into the model as released. Unsloth's measurements against the lossless file: UD-Q4_K_XL mean KLD 0.013 with 96% of tokens matching the top choice, UD-Q3_K_XL 0.158 and 87%, UD-Q2_K_XL 0.408 and 79%. Those are distance measures, not task scores. No one has published a task benchmark for the smaller files.

Fix
Move up a quant band [strong]

UD-Q4_K_XL is 155GB against 128GB for UD-Q3_K_XL. If the extra 27GB fits, it recovers nearly all of the gap.

Evidence · 2 sources · anecdotal
ISSUE-04 mild Q2–Q3 / Q4–Q5 / Q6–Q8 · math, factuality, instruction-following

DeepSeek's card for this checkpoint lists agentic coding and tool-use benchmarks only. The technical report has math, knowledge and long-context results, but for the April preview checkpoint, which this one replaces, so they are not used here. Math, factuality, instruction following, long context and writing are unrated because nothing was published for 0731, not because the model is known to be weak at them.

No fix
No known fix — choose a different model

Not a defect. It is a caveat, and the blank cells are honest.

Evidence · 2 sources · anecdotal