Known-issues sheets for local language models Rev 0.1.0 · reviewed every 7 days
llm-picker

Model datasheet · Spark-X2.5

Spark-X2.5 4B

A 4.1B dense model, 2.6GB at Q4_K_M, with a 1M-token context on paper. SparkLLM's table, all in thinking mode, has it level with MiniCPM5-2B on BFCL v4 (65.1 against 66.6), ahead on IFBench and AIME 2026, and behind on SWE-bench Verified. Those figures come from two vendors' tables, so read the comparison loosely. Three of every four layers use a 512-token sliding window, but the nine full-attention layers still cache about 4.8GB at 128K, which is more than the weights. It also thinks at length: community runs report answers cut off by token budgets, and loops that DRY sampling breaks.

Vendor
SparkLLM (XHToken)
Architecture
Dense · 4.1B
Context
1,048,576 tokens
License
Apache-2.0
Released
2026-08-24
Vision
No

→ GGUF (llama.cpp / LM Studio / Ollama) → MLX (Apple Silicon)

Architecture, from its own config.json

36 layers · 27 sliding window · 9 full attention

16 attention heads over 4 KV heads (GQA 4:1), head dim 256 · dense feed-forward · vocabulary 131,072

Where the memory goes: weights by band above, plus a KV cache of 396MB at 8K · 1.2GB at 32K · 4.6GB at 128K, plus the runtime's own working memory — dominated by the output projection over that vocabulary.

Inside one layer, from its own tensors

36 × layer

27 sliding-window layers, 9 full-attention layers — the same block either way. What differs is how far back the mask lets attention look, not what the layer is built from.

input_layernormattentionq_k_v_proj — q, k and v in one tensorsoftmax(q·kᵀ/√d) · v16 query heads over 4 key/valueheadso_projpost_attention_layernormfeed-forwardgate_proj · up_proj → down_proj10,240 wide

The green line down the right of each half is the residual: it leaves the spine above the norm, skips the operation entirely and rejoins at the +. Where attention is drawn open, q, k and v are scaled to their real output widths — a model that shares key and value heads across many query heads shows one wide box beside two narrow ones.

Around the stack: an embedding of 131,072 × 2,560, a final norm, and an output projection that reuses the embedding (tied weights).

On the norm names: despite what it is called, post_attention_layernorm is drawn after the residual add here, which is where this family applies it — it normalises the input to the feed-forward, not the output of attention. Models that carry a separate pre_feedforward_layernorm use it the other way, and are drawn that way.

Also in every block and not drawn, because their tensor names do not fix where they belong: g_proj, out_proj. Guessing an order is the one thing this diagram will not do.

§1 Characteristics by quantization band

Q2–Q3 — 2–3 bit

2.2GB weights · e.g. Q3_K_M + KV: 396MB at 8K · 1.2GB at 32K · 4.6GB at 128K
Math & reasoning 5/10
data checked Oct 2026
  • quant-degradation-community — editorial band derivation @ Q3_K_M · aggregated · 2026-10-04

    Editorial derivation: three points down. In the same table, the 3.5-bpw mix reaches a mean KLD of 1.20, with only 52% top-token agreement, and its MMLU-Redux falls to 40.4 against 63.3 at Q8_0. The stock Q3_K_M is about 4.2 bpw and should lose less than that, but it saves only 0.43GB over Q4_K_M.

Coding 4/10
data checked Oct 2026
  • quant-degradation-community — editorial band derivation @ Q3_K_M · aggregated · 2026-10-04

    Editorial derivation: two points down from the 4-bit band, on the same evidence.

Tool calling / agentic 4/10
data checked Oct 2026
  • quant-degradation-community — editorial band derivation @ Q3_K_M · aggregated · 2026-10-04

    Editorial derivation: two points down from full precision. Structured output degrades fastest.

Q4–Q5 — 4–5 bit

2.6GB weights · e.g. Q4_K_M , mlx-4bit + KV: 396MB at 8K · 1.2GB at 32K · 4.6GB at 128K
Math & reasoning 7/10
data checked Oct 2026
  • quant-degradation-community — editorial band derivation @ Q4_K_M · aggregated · 2026-10-04

    Editorial derivation: one point down. This model measures as unusually quant-sensitive. eaddario's 4.5-bpw mix has a mean KLD of 0.34 against F16, with the top token matching only 74% of the time. The 5.5-bpw mix is at 0.15. The stock Q4_K_M is 2.60GB, about 5.1 bpw, so it should sit between the two. These are custom mixes on a fork, not stock quants.

Coding 5/10
data checked Oct 2026
  • quant-degradation-community — editorial band derivation @ Q4_K_M · aggregated · 2026-10-04

    Editorial derivation: one point down, as for math.

Tool calling / agentic 5/10
data checked Oct 2026
  • quant-degradation-community — editorial band derivation @ Q4_K_M · aggregated · 2026-10-04

    Editorial derivation: one point down, as for math.

Instruction following 6/10
data checked Oct 2026
  • quant-degradation-community — editorial band derivation @ Q4_K_M · aggregated · 2026-10-04

    Editorial derivation: one point down. Usually carried at 4-bit, but the divergence here is several times what a 4B model typically shows.

Q6–Q8 — 6–8 bit

4.4GB weights · e.g. Q8_0 , Q6_K + KV: 396MB at 8K · 1.2GB at 32K · 4.6GB at 128K
Math & reasoning 8/10
data checked Oct 2026
  • quant-degradation-community — editorial band derivation @ Q8_0 · aggregated · 2026-10-04

    Editorial derivation: carried. The only KL-divergence table for this model (eaddario, target-bpw mixes) has Q8_0 at a mean KLD of 0.0066 and a 6.5-bpw mix at 0.051 against F16.

Coding 6/10
data checked Oct 2026
  • quant-degradation-community — editorial band derivation @ Q8_0 · aggregated · 2026-10-04

    Editorial derivation: carried from full precision.

Tool calling / agentic 6/10
data checked Oct 2026
  • quant-degradation-community — editorial band derivation @ Q8_0 · aggregated · 2026-10-04

    Editorial derivation: carried from full precision.

Instruction following 7/10
data checked Oct 2026
  • quant-degradation-community — editorial band derivation @ Q8_0 · aggregated · 2026-10-04

    Editorial derivation: carried from full precision.

FP16 — full precision (fp16/bf16)

8.2GB weights · e.g. bf16 + KV: 396MB at 8K · 1.2GB at 32K · 4.6GB at 128K
Math & reasoning 8/10
data checked Oct 2026
Coding 6/10
data checked Oct 2026
  • vendor-model-card — SWE-Bench Verified = 41.6 @ bf16 · vendor-reported · 2026-10-04

    Vendor table; the card says all evaluations are in thinking mode. Qwen3.5-9B 53.1 and Qwen3.5-4B 38.8 (both quoted from Qwen's own cards).

  • vendor-model-card — SWE-Bench Pro = 44.4 @ bf16 · vendor-reported · 2026-10-04

    Same table. Note that the vendor's SWE-bench Pro figure is above its SWE-bench Verified one, which is unusual: Pro is normally the harder set. Recorded as published, not corrected.

  • vendor-model-card — SWE-Bench Multilingual = 53.3 @ bf16 · vendor-reported · 2026-10-04

    Same table.

  • vendor-model-card — SciCode = 34.7 @ bf16 · vendor-reported · 2026-10-04

    Same table. Gemma4-12B 39.8.

Tool calling / agentic 6/10
data checked Oct 2026
  • vendor-model-card — BFCL-V4 = 65.1 @ bf16 · vendor-reported · 2026-10-04

    Vendor table; the card says all evaluations are in thinking mode. Qwen3.5-9B 66.1, Qwen3.5-4B 50.3 (quoted).

  • vendor-model-card — τ²-bench = 75.1 @ bf16 · vendor-reported · 2026-10-04

    Same table. Qwen3.5-4B 79.9 (quoted).

  • vendor-model-card — MCP-Atlas = 54.6 @ bf16 · vendor-reported · 2026-10-04

    Same table. Qwen3.5-9B 47.4 (quoted). The table also lists BrowseComp at 40.9.

Instruction following 7/10
data checked Oct 2026
  • vendor-model-card — IFEval = 93 @ bf16 · vendor-reported · 2026-10-04

    Vendor table; the card says all evaluations are in thinking mode. Gemma4-12B 94.8.

  • vendor-model-card — IFBench = 75 @ bf16 · vendor-reported · 2026-10-04

    Same table. Gemma4-12B 73.5 (quoted), Qwen3.5-9B 64.5.

§2 Known issues & what fixes them

ISSUE-01 moderate Q2–Q3 / Q4–Q5 / Q6–Q8 / FP16 · math, coding, tool-calling

Thinking is on by default and runs long. One Q4_K_M math run with an 8,192-token cap scored 15 of 30, and all 15 misses were answers that never arrived because the budget ran out. Other users report a loop that spent 8K tokens on one trivial decision at Q4_K_M, agent tasks stalling on overthinking, and hard repetition loops at Q8_0. The vendor has acknowledged the overthinking and says a fix is in progress.

Fix
Prompt technique [partial]

Sampler settings, not prompt wording. One measurement at Q8_0 found DRY broke the loops (--dry-multiplier 0.8 --dry-base 1.75 --dry-penalty-last-n 1024), cutting the repeat rate from 0.27 to 0.085, and adding --reasoning-budget 4096 lowered it to 0.039. Give maths and coding a large output budget.

Fix
Prompt technique [weak]

For simple turns, send enable_thinking: false. Every published score is in thinking mode, so expect less than the table shows.

Evidence · 4 sources · community-consensus
ISSUE-02 mild Q2–Q3 / Q4–Q5 / Q6–Q8 / FP16 · instruction-following

Under greedy decoding the base weights can emit token 128486, which starts with an orphan UTF-8 byte. It has been reproduced mid-word in Greek, at bf16 and fp32, and on an F16 GGUF. The output is then invalid UTF-8, and llama-server --jinja answers with HTTP 500. Lower-bit quants sometimes pick a different token there, which hides the defect without fixing it. Only one team has reported it so far.

Fix
Prompt technique [partial]

A request setting, not prompt wording: a logit_bias of minus infinity on the 502 orphan-byte token ids the reporter published. They found a system prompt does not help.

Evidence · 2 sources · anecdotal
ISSUE-03 mild Q4–Q5 · math, coding, tool-calling, instruction-following

By the one KL-divergence table available, 4-bit moves this model's output further from full precision than is typical at 4B: a mean KLD of 0.34 for a 4.5-bpw mix, against 0.051 at 6.5 bpw and 0.0066 at Q8_0. Q8_0 is 4.4GB, so on most machines that fit 2.6GB it is affordable.

Fix
Move up a quant band [strong]

Q6_K at 3.4GB (community builds) or the vendor's Q8_0 at 4.4GB.

Evidence · 1 source · anecdotal
ISSUE-04 severe Q2–Q3 · math, coding, tool-calling

At 3.5 bpw the measured KL divergence reaches 1.20, with the top token agreeing barely half the time, and MMLU-Redux falls from 63.3 to 40.4. The stock Q3_K_M is a little larger and should fare better, but it saves only 0.43GB against Q4_K_M, which fits anywhere it does.

Fix
Move up a quant band [strong]

Q4_K_M at 2.6GB, or better Q8_0 at 4.4GB.

Fix
Code interpreter / calculator tool [weak]

Recovers arithmetic only if tool calling still works, and it is also down at this band.

Evidence · 2 sources · anecdotal
ISSUE-05 mild Q2–Q3 / Q4–Q5 / Q6–Q8 / FP16 · tool-calling, instruction-following

spark2_5 is a new architecture. llama.cpp support merged on 2026-09-06 (build b10828), after the vendor's GGUFs were already up, and older LM Studio and Ollama runtimes refuse the file with 'unknown model architecture'. On a Mac, the mlx-community build needs the mlx-optiq package imported before mlx_lm.load. Stock mlx-lm does not know the spark2_5 architecture.

Fix
Update your runtime [strong]

llama.cpp b10828 or later, Ollama 0.34.1 or later, or LM Studio runtime 2.34.0 or later, per the model card.

Evidence · 4 sources · community-consensus
ISSUE-06 mild Q2–Q3 / Q4–Q5 / Q6–Q8 / FP16 · math, coding, tool-calling, long-context, factuality, instruction-following

Every stored score is SparkLLM's own. Long context is unrated despite the 1M-token claim: the only long-context row on the card is an Artificial Analysis benchmark (AA-LCR), whose values this site does not republish. No hallucination benchmark is published, so factuality is unrated too. The card's comparison columns mix figures it measured with figures quoted from other vendors' cards (marked with an asterisk there).

No fix
No known fix — choose a different model

Not a defect but a caveat. The blanks are honest, not zeros. Revisit when independent numbers land.

Evidence · 1 source · anecdotal