Known-issues sheets for local language models Rev 0.1.0 · reviewed every 7 days
llm-picker

Model datasheet · Ling 3.0

Ling 3.0 Flash

124B parameters with 5.1B active, and only 7 of its 42 layers keep a KV cache, so 128K of context adds about 1GB to a 77GB Q4_K_M file. That puts the 4-bit file on 128GB machines. inclusionAI's own table, in thinking mode, puts it ahead of Nemotron 3 Super on every row the two share, including SWE-bench Pro (56.6 against 34.1) and MRCR at 128K (90.8 against 40.8). No independent figure is republished here yet. llama.cpp has run it since 2026-08-17, but the dedicated Ling 3.0 chat parser only merged on 2026-09-19 and schema-constrained output on 2026-10-03, so older builds undersell its tool use. There is no mlx-community build.

Vendor
inclusionAI (Ant Group)
Architecture
MoE · 124B total / ~5.1B active
Context
262,144 tokens
License
MIT
Released
2026-08-02
Vision
No

→ GGUF (llama.cpp / LM Studio / Ollama)

Architecture, from its own config.json

42 layers · 35 linear / recurrent · 7 full attention

Only 7 of 42 layers hold a cache at all, which is why its memory grows so much more slowly with context than its size suggests.

32 attention heads, multi-head latent attention — it caches one 576-wide compressed vector per layer instead of per-head keys and values, which is why its cache is small for its size · mixture of experts, 5.1B of 124B active per token across 8 experts · vocabulary 157,184

Where the memory goes: weights by band above, plus a KV cache of 63MB at 8K · 252MB at 32K · 1008MB at 128K, plus the runtime's own working memory — dominated by the output projection over that vocabulary.

Inside one layer, from its own tensors

35 × linear / recurrent layer

input_layernormattentionq_projk_projv_projsoftmax(q·kᵀ/√d) · v32 query heads over 32 key/valueheadso_projpost_attention_layernormmixture of expertsrouter picks 8 of 512 expertseach 768 wide: gate·up → downplus a shared expert, run for everytoken

7 × full-attention layer

input_layernormlatent attentionq_a_proj →q_a_layernorm →q_b_projkv_a_proj_with_mqa →kv_a_layernorm →kv_b_projsoftmax(q·kᵀ/√d) · vone 576-wide latent is cached, notper-head keys and valueso_projpost_attention_layernormmixture of expertsrouter picks 8 of 512 expertseach 768 wide: gate·up → downplus a shared expert, run for everytoken

The green line down the right of each half is the residual: it leaves the spine above the norm, skips the operation entirely and rejoins at the +. Where attention is drawn open, q, k and v are scaled to their real output widths — a model that shares key and value heads across many query heads shows one wide box beside two narrow ones.

Around the stack: an embedding of × , a final norm, and a separate output projection. The first 2 layers use a dense feed-forward instead of the experts.

On the norm names: despite what it is called, post_attention_layernorm is drawn after the residual add here, which is where this family applies it — it normalises the input to the feed-forward, not the output of attention. Models that carry a separate pre_feedforward_layernorm use it the other way, and are drawn that way.

Also in every block and not drawn, because their tensor names do not fix where they belong: g_proj, q_conv1d, dt_bias, k_conv1d, f_proj, A_log, v_conv1d, b_proj, o_norm, expert_bias, dense. Guessing an order is the one thing this diagram will not do.

§1 Characteristics by quantization band

Q2–Q3 — 2–3 bit

59GB weights · e.g. Q3_K_M + KV: 63MB at 8K · 252MB at 32K · 1008MB at 128K
Math & reasoning 6/10
data checked Oct 2026
  • quant-degradation-community — editorial band derivation @ Q3_K_M · aggregated · 2026-10-04

    Editorial derivation: three points down. The same community run measured +4.04% perplexity for a Q3_K mix that kept a Q6_K attention path, and +13.92% at Q2_K. A stock Q3_K_M quantizes that path harder, so expect at least this much loss.

Coding 5/10
data checked Oct 2026
  • quant-degradation-community — editorial band derivation @ Q3_K_M · aggregated · 2026-10-04

    Editorial derivation: three points down, as for math.

Tool calling / agentic 4/10
data checked Oct 2026
  • quant-degradation-community — editorial band derivation @ Q3_K_M · aggregated · 2026-10-04

    Editorial derivation: four points down. Structured output degrades fastest at 3-bit, on top of a parser that only recently stopped being the weak link.

Instruction following 7/10
data checked Oct 2026

Q4–Q5 — 4–5 bit

77GB weights · e.g. Q4_K_M + KV: 63MB at 8K · 252MB at 32K · 1008MB at 128K
Math & reasoning 8/10
data checked Oct 2026
  • quant-degradation-community — editorial band derivation @ Q4_K_M · aggregated · 2026-10-04

    Editorial derivation: one point down. A community perplexity run on a Q4_K_M-class mix (Q8_0 attention path, Q4_K experts) measured 4.3849 against 4.3532 at F16, +0.73%. That is small, but 5.1B active parameters leave less redundancy than the 124B total suggests.

Coding 7/10
data checked Oct 2026
  • quant-degradation-community — editorial band derivation @ Q4_K_M · aggregated · 2026-10-04

    Editorial derivation: one point down, as for math.

Tool calling / agentic 7/10
data checked Oct 2026
  • quant-degradation-community — editorial band derivation @ Q4_K_M · aggregated · 2026-10-04

    Editorial derivation: one point down, as for math. The parser fixes in llama.cpp matter more here than the quant does.

Instruction following 8/10
data checked Oct 2026
  • quant-degradation-community — editorial band derivation @ Q4_K_M · aggregated · 2026-10-04

    Editorial derivation: carried. Format compliance survives 4-bit better than arithmetic does.

Long context 7/10
data checked Oct 2026
  • quant-degradation-community — editorial band derivation @ Q4_K_M · aggregated · 2026-10-04

    Editorial derivation: as for the high band. The newline collapse was reproduced on Q4_K_M.

Q6–Q8 — 6–8 bit

104.8GB weights · e.g. Q6_K , Q8_0 + KV: 63MB at 8K · 252MB at 32K · 1008MB at 128K
Math & reasoning 9/10
data checked Oct 2026
  • quant-degradation-community — editorial band derivation @ Q6_K · aggregated · 2026-10-04

    Editorial derivation: carried from full precision.

Coding 8/10
data checked Oct 2026
  • quant-degradation-community — editorial band derivation @ Q6_K · aggregated · 2026-10-04

    Editorial derivation: carried from full precision.

Tool calling / agentic 8/10
data checked Oct 2026
  • quant-degradation-community — editorial band derivation @ Q6_K · aggregated · 2026-10-04

    Editorial derivation: carried from full precision.

Instruction following 8/10
data checked Oct 2026
  • quant-degradation-community — editorial band derivation @ Q6_K · aggregated · 2026-10-04

    Editorial derivation: carried from full precision.

Long context 7/10
data checked Oct 2026
  • quant-degradation-community — editorial band derivation @ Q6_K · aggregated · 2026-10-04

    Editorial derivation: one point below the vendor figure. Not a precision loss: the only long-context reports from quantized builds (Q4_K_M and Q8_0) describe a newline collapse near 150K tokens, and they come with a sampler fix.

FP16 — full precision (fp16/bf16)

255.1GB weights · e.g. bf16 + KV: 63MB at 8K · 252MB at 32K · 1008MB at 128K
Math & reasoning 9/10
data checked Oct 2026
  • vendor-model-card — AIME26 = 93.2 @ bf16 · vendor-reported · 2026-10-04

    Vendor evaluation table (image on the card), thinking mode, which the card says is on by default. Nemotron-3-Super-120B-A12B posts 91.7 on the same row.

  • vendor-model-card — HMMT-Feb26 = 87 @ bf16 · vendor-reported · 2026-10-04

    Same table. Nemotron-3-Super-120B-A12B 84.9.

  • vendor-model-card — IMO-AnswerBench = 83.7 @ bf16 · vendor-reported · 2026-10-04

    Same table. Nemotron-3-Super-120B-A12B 77.0.

Coding 8/10
data checked Oct 2026
  • vendor-model-card — SWE-Bench Pro = 56.6 @ bf16 · vendor-reported · 2026-10-04

    Vendor evaluation table (image on the card), thinking mode, which the card says is on by default. OpenHands harness with tailored prompts, 256K context. Nemotron-3-Super-120B-A12B 34.1 on the same row. No SWE-bench Verified or Aider figure is published.

  • vendor-model-card — SWE-Bench Multilingual = 72.4 @ bf16 · vendor-reported · 2026-10-04

    Same harness. Nemotron-3-Super-120B-A12B 42.7.

  • vendor-model-card — LiveCodeBench (2408-2505) = 82.8 @ bf16 · vendor-reported · 2026-10-04

    Same table, listed under Reasoning.

  • vendor-model-card — Terminal-Bench 2.1 = 57 @ bf16 · vendor-reported · 2026-10-04

    inclusionAI's own run with the Terminus 2 harness. The card says it followed the Artificial Analysis protocol (2-hour timeout, mean of 3 runs), but the figure is the vendor's measurement, not an Artificial Analysis one.

  • vals-ai — Vals AI model page (SWE-bench, LiveCodeBench, Terminal-Bench 2.1) · aggregated · 2026-10-04

    Independent results exist here. Link only: Vals AI figures are not republished on this site.

Tool calling / agentic 8/10
data checked Oct 2026
  • vendor-model-card — BFCL-v4 = 73 @ bf16 · vendor-reported · 2026-10-04

    Vendor evaluation table (image on the card), thinking mode, which the card says is on by default. Level with Claude-Sonnet-4.6 (max) at 73.1 on the same row, ahead of Nemotron-3-Super-120B-A12B at 60.6.

  • vendor-model-card — MCP-Atlas = 65.5 @ bf16 · vendor-reported · 2026-10-04

    500-task public set, official v1 harness, 20-turn limit. The same table also lists SkillsBench at 44.8.

  • vendor-model-card — Tau3-banking-AA (value at source) @ bf16 · vendor-reported · 2026-10-04

    Multi-turn agentic banking, run by inclusionAI under an Artificial Analysis protocol. Neither the value nor how it compares with other models is republished here.

Long context 8/10
data checked Oct 2026
  • vendor-model-card — MRCR_128K = 90.8 @ bf16 · vendor-reported · 2026-10-04

    Vendor evaluation table (image on the card), thinking mode, which the card says is on by default. Multi-round coreference at 128K tokens. Nemotron-3-Super-120B-A12B 40.8.

  • vendor-model-card — MRCR_256k = 81.1 @ bf16 · vendor-reported · 2026-10-04

    Same benchmark at 256K, the trained context. Nemotron-3-Super-120B-A12B 35.8.

  • vendor-model-card — AA-LCR (value at source) @ bf16 · vendor-reported · 2026-10-04

    An Artificial Analysis long-context benchmark on the same table. The value is not republished here.

Instruction following 8/10
data checked Oct 2026
  • vendor-model-card — IFBench = 74.5 @ bf16 · vendor-reported · 2026-10-04

    Vendor evaluation table (image on the card), thinking mode, which the card says is on by default. Nemotron-3-Super-120B-A12B 72.6, DeepSeek-V4-Flash-Preview 79.2.

  • vendor-model-card — Multi-IF = 87.7 @ bf16 · vendor-reported · 2026-10-04

    Multi-turn instruction following. The same table lists SysBench at 93.6 and LIFEBench at 77.3.

§2 Known issues & what fixes them

ISSUE-01 moderate Q2–Q3 / Q4–Q5 / Q6–Q8 / FP16 · tool-calling, instruction-following

Before llama.cpp had a dedicated Ling 3.0 parser, the generic one let the model open a tool call inside its reasoning and never close it, so the whole turn disappeared into reasoning_content. The dedicated parser fixed that when it merged on 2026-09-19. It then ignored response_format and json_schema until a follow-up merged on 2026-10-03. On an older build the model looks worse at tool use and structured output than it is.

Fix
Update your runtime [strong]

Use a llama.cpp build from 2026-10-03 or later, with --jinja. Ollama and LM Studio need a release that bundles it.

Evidence · 4 sources · community-consensus
ISSUE-02 moderate Q2–Q3 / Q4–Q5 / Q6–Q8 / FP16 · long-context, coding

Several users running long agent sessions report the model collapsing into a flood of newlines at around 150K tokens of context, at both Q4_K_M and Q8_0. One reproduced it byte-for-byte across seeds. repeat_penalty 1.10 did not stop it. The same user reports that the sampler change below works on llama.cpp, vLLM and SGLang, which suggests this is the model rather than a quant.

Fix
Prompt technique [partial]

A sampler setting, not a prompt: set frequency_penalty to 0.1-0.3 (0.05 was not enough), or on llama.cpp use DRY with the newline removed from the sequence breakers: --dry-multiplier 0.8 --dry-base 1.75 --dry-allowed-length 2 --dry-sequence-breaker ':' --dry-sequence-breaker '"' --dry-sequence-breaker '*'. The default DRY breakers include the newline, which makes default DRY useless against this failure.

No fix
No known fix — choose a different model

If a workload must sit above about 150K tokens without sampler changes, there is no other reported fix.

Evidence · 2 sources · community-consensus
ISSUE-03 moderate Q2–Q3 · math, coding, tool-calling

The only community quantization measurement puts a 3-bit mix at +4% perplexity against F16 and Q2_K at +14%, against +0.7% at 4-bit. The 59GB Q3_K_M is still worth knowing about. On a 96GB Mac, the default GPU allowance holds it and not the 77GB Q4_K_M.

Fix
Move up a quant band [strong]

Q4_K_M at 77GB where the machine allows it. IQ4_XS at 68.7GB is a middle step.

Fix
Code interpreter / calculator tool [weak]

Recovers arithmetic, but tool calling is also down at this band, so it helps less than usual.

Evidence · 2 sources · anecdotal
ISSUE-04 mild Q2–Q3 / Q4–Q5 / Q6–Q8 / FP16 · math, coding, tool-calling, long-context, instruction-following

Every stored score is inclusionAI's own, from a table published as an image. Independent results exist at Vals AI and Artificial Analysis, but neither may be republished here. No SimpleQA or other factuality figure has been published, and a community request for one is open, so factuality is unrated rather than estimated.

No fix
No known fix — choose a different model

Not a defect but a caveat. The blanks are honest, not zeros. The Vals AI and Artificial Analysis pages are linked for anyone who wants an independent view.

Evidence · 4 sources · anecdotal