Known-issues sheets for local language models Rev 0.1.0 · reviewed every 7 days

Model datasheet · Llama 3.3

Llama 3.3 70B

The largest dense model in this catalogue — everything above it is a sparse MoE. Best instruction-following evidence on the site at IFEval 92.1, and it needs a 96GB machine to be comfortable at any real context, because its KV cache costs 10GB at 32K before the weights are counted.

Vendor
Meta
Architecture
Dense · 70B
Context
131,072 tokens
License
Llama 3.3 Community License
Released
2024-12-06
Vision
No

→ GGUF (llama.cpp / LM Studio / Ollama) → MLX (Apple Silicon)

§1 Characteristics by quantization band

Q2–Q3 — 2–3 bit

34.3GB weights · e.g. Q3_K_M + KV: 2.5GB at 8K · 10GB at 32K · 40GB at 128K
Instruction following 7/10
data checked Aug 2026
  • quant-degradation-community — community consensus @ Q3_K_M · aggregated · 2026-08-30

    Editorial derivation: two points down. 3-bit on a 70B still beats 4-bit on many smaller models, which is the argument for running one at all.

Math & reasoning 5/10
data checked Aug 2026
  • quant-degradation-community — community consensus @ Q3_K_M · aggregated · 2026-08-30

    Editorial derivation: two points down from the full band.

Q4–Q5 — 4–5 bit

42.5GB weights · e.g. Q4_K_M , mlx-4bit + KV: 2.5GB at 8K · 10GB at 32K · 40GB at 128K
Instruction following 8/10
data checked Aug 2026
  • quant-degradation-community — community consensus @ Q4_K_M · aggregated · 2026-08-30

    Editorial derivation: one point down. At 42.5GB this is the only band most people will realistically run, and large dense models tolerate 4-bit better than small ones.

Math & reasoning 6/10
data checked Aug 2026
  • quant-degradation-community — community consensus @ Q4_K_M · aggregated · 2026-08-30

    Editorial derivation: one point down; multi-step arithmetic is the first thing 4-bit costs you.

Tool calling / agentic 6/10
data checked Aug 2026
  • quant-degradation-community — community consensus @ Q4_K_M · aggregated · 2026-08-30

    Editorial derivation: one point down from the full band.

Q6–Q8 — 6–8 bit

57.9GB weights · e.g. Q6_K + KV: 2.5GB at 8K · 10GB at 32K · 40GB at 128K
Instruction following 9/10
data checked Aug 2026
  • quant-degradation-community — community consensus @ Q6_K · aggregated · 2026-08-30

    Editorial derivation: 6-bit is near-lossless, and a 70B has more redundancy to spare than anything else here. Carried unchanged.

Math & reasoning 7/10
data checked Aug 2026
  • quant-degradation-community — community consensus @ Q6_K · aggregated · 2026-08-30

    Editorial derivation: carried from the full band.

FP16 — full precision (fp16/bf16)

141.1GB weights · e.g. bf16 + KV: 2.5GB at 8K · 10GB at 32K · 40GB at 128K
Instruction following 9/10
data checked Aug 2026
  • vendor-model-cardIFEval = 92.1 @ bf16 · vendor-reported · 2026-08-30

    The highest instruction-following figure on this site, on a dimension only a third of the catalogue has any evidence for at all. IFEval is program-checkable, so there is less room for interpretation than most benchmarks.

Math & reasoning 7/10
data checked Aug 2026
  • vendor-model-cardMATH (CoT) = 77 @ bf16 · vendor-reported · 2026-08-30

    MGSM 91.1, MMLU (CoT) 86.0, MMLU-Pro 68.9. GPQA Diamond is only 50.5 — well behind the 2026 models here, which is what two years of progress looks like on graduate science questions.

Coding 6/10
data checked Aug 2026
  • vendor-model-cardHumanEval = 88.4 @ bf16 · vendor-reported · 2026-08-30

    MBPP EvalPlus 87.6. Scored well below those numbers on purpose: HumanEval is saturated and predates the agentic benchmarks the 2026 models are measured on, and this model has no SWE-bench or Terminal-Bench figure to compare against them.

Tool calling / agentic 7/10
data checked Aug 2026
  • vendor-model-cardBFCL v2 = 77.3 @ bf16 · vendor-reported · 2026-08-30

    Measured on BFCL v2; most entries here quote v3 or v4, which are not directly comparable.

§2 Known issues & what fixes them

ISSUE-01 severe Q2–Q3 / Q4–Q5 / Q6–Q8 / FP16 · long-context

Eighty layers with eight KV heads and no sliding window or linear attention: 320KB of cache per token, the most expensive in this catalogue. That is 2.5GB at 8K, 10GB at 32K and 40GB at 128K — on top of 42.5GB of 4-bit weights. Filling the advertised 128K window needs more memory for the cache than most machines have in total.

Fix
RAG (retrieval) [strong]

Retrieve rather than fill the window. On this model the context is what breaks your memory budget, not the weights.

Fix
Move up a quant band [partial]

Inverted advice: on a 64GB machine, dropping from Q4_K_M to Q3_K_M buys 8GB of weights, which is 25K more tokens of context. The quant you can afford depends on the context you need.

Evidence · 1 source · anecdotal
ISSUE-02 moderate Q2–Q3 / Q4–Q5 / Q6–Q8 / FP16 · coding, math

Released December 2024 and measured on the benchmarks of that moment — HumanEval, MBPP, BFCL v2. It has no SWE-bench Verified or Terminal-Bench figure, so it cannot be compared with the 2026 agentic models on the work people now use these for. Where the comparison does exist it shows its age: GPQA Diamond 50.5 against 84–88 for the current 27–31B models.

No fix
No known fix — choose a different model

Not a defect to fix. It remains the strongest instruction-follower here; it is simply not the model to reach for on agentic coding.

Evidence · 1 source · anecdotal
ISSUE-03 mild Q2–Q3 / Q4–Q5 / Q6–Q8 / FP16 · instruction-following

Llama 3.3 Community License, not Apache-2.0 or MIT. Commercial use is allowed but the terms carry conditions, including an attribution requirement and a monthly-active-user threshold above which a separate licence is needed.

No fix
No known fix — choose a different model

Read the licence before shipping on it. If a permissive licence is a hard requirement, the Apache-2.0 and MIT models here are the alternatives.

Evidence · 1 source · anecdotal