Known-issues sheets for local language models Rev 0.1.0 · reviewed every 7 days
llm-picker

Model datasheet · Ling 3.0

Ling 3.0 Tiny

1.3B active parameters out of 7.9B, and the vendor reports it running at 86-90 tokens/s on an M4 Pro laptop in about 8.3GB. Three linear-attention layers for every latent-attention one means only 6 of 24 layers cache, so long context is nearly free. Runtime support has caught up — llama.cpp has had a dedicated Ling 3.0 chat parser since 2026-09-19 (release b11057) — but looping reports remain open.

Vendor
inclusionAI (Ant Group)
Architecture
MoE · 7.9B total / ~1.3B active
Context
131,072 tokens
License
MIT
Released
2026-08-10
Vision
No

→ GGUF (llama.cpp / LM Studio / Ollama)

Architecture, from its own config.json

24 layers · 18 linear / recurrent · 6 full attention

Only 6 of 24 layers hold a cache at all, which is why its memory grows so much more slowly with context than its size suggests.

16 attention heads, multi-head latent attention — it caches one 576-wide compressed vector per layer instead of per-head keys and values, which is why its cache is small for its size · mixture of experts, 1.3B of 7.9B active per token across 8 experts · vocabulary 157,184

Where the memory goes: weights by band above, plus a KV cache of 54MB at 8K · 216MB at 32K · 864MB at 128K, plus the runtime's own working memory — dominated by the output projection over that vocabulary.

Inside one layer, from its own tensors

18 × linear / recurrent layer

input_layernormattentionq_projk_projv_projsoftmax(q·kᵀ/√d) · v16 query heads over 16 key/valueheadso_projpost_attention_layernormmixture of expertsrouter picks 8 of 128 expertseach 512 wide: gate·up → downplus a shared expert, run for everytoken

6 × full-attention layer

input_layernormlatent attentionq_a_proj →q_a_layernorm →q_b_projkv_a_proj_with_mqa →kv_a_layernorm →kv_b_projsoftmax(q·kᵀ/√d) · vone 576-wide latent is cached, notper-head keys and valueso_projpost_attention_layernormmixture of expertsrouter picks 8 of 128 expertseach 512 wide: gate·up → downplus a shared expert, run for everytoken

The green line down the right of each half is the residual: it leaves the spine above the norm, skips the operation entirely and rejoins at the +. Where attention is drawn open, q, k and v are scaled to their real output widths — a model that shares key and value heads across many query heads shows one wide box beside two narrow ones.

Around the stack: an embedding of × , a final norm, and a separate output projection. The first 1 layer use a dense feed-forward instead of the experts.

On the norm names: despite what it is called, post_attention_layernorm is drawn after the residual add here, which is where this family applies it — it normalises the input to the feed-forward, not the output of attention. Models that carry a separate pre_feedforward_layernorm use it the other way, and are drawn that way.

Also in every block and not drawn, because their tensor names do not fix where they belong: A_log, b_proj, dt_bias, f_proj, g_proj, k_conv1d, o_norm, q_conv1d, v_conv1d, dense, expert_bias. Guessing an order is the one thing this diagram will not do.

§1 Characteristics by quantization band

Q2–Q3 — 2–3 bit

3.8GB weights · e.g. Q3_K_M + KV: 54MB at 8K · 216MB at 32K · 864MB at 128K
Math & reasoning 4/10
data checked Sep 2026
  • quant-degradation-community — editorial band derivation @ Q3_K_M · aggregated · 2026-09-12

    Editorial derivation: two points down — and the file is only 1.1GB smaller than Q4_K_M.

Coding 3/10
data checked Sep 2026
  • quant-degradation-community — editorial band derivation @ Q3_K_M · aggregated · 2026-09-12

    Editorial derivation: two points down.

Tool calling / agentic 3/10
data checked Sep 2026
  • quant-degradation-community — editorial band derivation @ Q3_K_M · aggregated · 2026-09-12

    Editorial derivation: two points down, on top of a tool-call parser that is already the weak link.

Q4–Q5 — 4–5 bit

4.9GB weights · e.g. Q4_K_M + KV: 54MB at 8K · 216MB at 32K · 864MB at 128K
Math & reasoning 6/10
data checked Sep 2026
  • quant-degradation-community — editorial band derivation @ Q4_K_M · aggregated · 2026-09-12

    Editorial derivation: one point down. 1.3B active parameters is the least redundancy of any model on this site.

Coding 4/10
data checked Sep 2026
Tool calling / agentic 5/10
data checked Sep 2026
Instruction following 6/10
data checked Sep 2026
  • quant-degradation-community — editorial band derivation @ Q4_K_M · aggregated · 2026-09-12

    Editorial derivation: carried; format compliance holds up better than arithmetic under 4-bit.

Q6–Q8 — 6–8 bit

6.8GB weights · e.g. Q6_K , Q8_0 + KV: 54MB at 8K · 216MB at 32K · 864MB at 128K
Math & reasoning 7/10
data checked Sep 2026
  • quant-degradation-community — editorial band derivation @ Q6_K · aggregated · 2026-09-12

    Editorial derivation: carried from full precision. 6.8GB, so this is the band an 8GB machine runs if the context stays short.

Coding 5/10
data checked Sep 2026
  • quant-degradation-community — editorial band derivation @ Q6_K · aggregated · 2026-09-12

    Editorial derivation: carried from full precision.

Tool calling / agentic 6/10
data checked Sep 2026
  • quant-degradation-community — editorial band derivation @ Q6_K · aggregated · 2026-09-12

    Editorial derivation: carried from full precision.

Instruction following 6/10
data checked Sep 2026
  • quant-degradation-community — editorial band derivation @ Q6_K · aggregated · 2026-09-12

    Editorial derivation: carried from full precision.

FP16 — full precision (fp16/bf16)

15.8GB weights · e.g. bf16 + KV: 54MB at 8K · 216MB at 32K · 864MB at 128K
Math & reasoning 7/10
data checked Sep 2026
  • vendor-model-card — GPQA Diamond = 73.4 @ bf16 · vendor-reported · 2026-09-12

    Vendor table, thinking mode. Behind Qwen3.5-9B (80.60) on the same table, ahead of Gemma-4-E4B (57.60) — on 1.3B active parameters.

  • vendor-model-card — HMMT-Feb26 = 70.31 @ bf16 · vendor-reported · 2026-09-12

    Competition math on the same table.

  • vendor-model-card — IMO-AnswerBench = 71.03 @ bf16 · vendor-reported · 2026-09-12

    Ahead of every comparison model inclusionAI lists, Qwen3.5-9B (70.00) included.

Coding 5/10
data checked Sep 2026
Tool calling / agentic 6/10
data checked Sep 2026
  • vendor-model-card — BFCL-v4 (FC) = 62.72 @ bf16 · vendor-reported · 2026-09-12

    Function calling. Level with Qwen3.5-4B (62.47), behind Qwen3.5-9B (65.83). The site's tool-calling caveat for this model is the runtime parser, not the weights.

  • vendor-model-card — TAU3-Banking-AA (value at source) @ bf16 · vendor-reported · 2026-09-12

    Multi-turn agentic banking, reported by inclusionAI from an Artificial Analysis protocol; neither the value nor how it compares with other models is republished here.

Instruction following 6/10
data checked Sep 2026
  • vendor-model-card — IFBench = 63.61 @ bf16 · vendor-reported · 2026-09-12

    Ahead of Qwen3.5-4B (52.00), behind Gemma-4-12B (73.50).

  • vendor-model-card — Multi-IF = 83.15 @ bf16 · vendor-reported · 2026-09-12

    Multi-turn instruction following, ahead of Qwen3.5-9B (83.03).

§2 Known issues & what fixes them

ISSUE-01 mild Q2–Q3 / Q4–Q5 / Q6–Q8 / FP16 · tool-calling, instruction-following

llama.cpp runs the architecture (BailingMoeV3 support merged 2026-08-05), and since 2026-09-19 it has a dedicated Ling 3.0 chat parser (PR #28682, first release b11057). Before that, the generic parser could emit a tool call without a reasoning terminator, which broke agent loops; a user on that report confirmed tool calling is 'much better now' after the merge, though the issue is still technically open. json_schema support in the new parser followed on 2026-10-03 (#29813). Builds older than b11057 still have the old behaviour.

Fix
Update your runtime [strong]

Use llama.cpp b11057 or later for the dedicated parser, and a build from 2026-10-03 or later if you rely on json_schema-constrained output.

Evidence · 3 sources · community-consensus
ISSUE-02 moderate Q2–Q3 / Q4–Q5 / Q6–Q8 · coding, instruction-following

Quantized builds have been reported degenerating into repeated slashes, users on the model's own discussion board report it looping indefinitely on logic problems with reasoning on or off, and a separate report has Vulkan on Intel Arc producing garbage depending on how many layers sit on the GPU. All open as of 2026-10-04.

Fix
Update your runtime [partial]

Keep llama.cpp current; if you are on Vulkan, try moving layers off the GPU to confirm it is the backend rather than the weights.

Fix
Move up a quant band [partial]

Reports cluster on the smaller quants — Q6_K is 6.8GB and worth trying first.

Evidence · 3 sources · community-consensus
ISSUE-03 mild Q2–Q3 / Q4–Q5 / Q6–Q8 / FP16 · math, coding, tool-calling, instruction-following

Every score here comes from inclusionAI's own table, and the two rows that would say most about knowledge and long context are Artificial Analysis measurements whose values this site does not republish. Factuality and long context are therefore unrated rather than estimated.

No fix
No known fix — choose a different model

Not a defect — a caveat. The blanks are honest; they are not zeros.

Evidence · 1 source · anecdotal
ISSUE-04 severe Q2–Q3 · math, coding, tool-calling

1.3B active parameters is the thinnest compute path on this site. Three-bit takes about two points off every quant-sensitive dimension to save 1.1GB against Q4_K_M — which fits anywhere the 3-bit file does.

Fix
Move up a quant band [strong]

Q4_K_M at 4.9GB, or Q6_K at 6.8GB. Neither asks much more of the machine.

Fix
Code interpreter / calculator tool [partial]

Where arithmetic broke, a calculator tool recovers more than a quant change would — if the tool-call path is working.

Evidence · 2 sources · community-consensus