Known-issues sheets for local language models Rev 0.1.0 · reviewed every 7 days
llm-picker

Model datasheet · Gemma 3

Gemma 3 27B

Superseded

Superseded by Gemma 4 31B and Gemma 4 26B-A4B, Google's next generation in the same size class. Kept for its receipts: best-in-family prose and real vision, and the clearest case on this site of a vendor coding number (HumanEval 48.8) contradicted by an independent one (Aider polyglot 4.9%).

Vendor
Google
Architecture
Dense · 27B
Context
131,072 tokens
License
Gemma Terms of Use
Released
2025-03-12
Vision
Yes

→ GGUF (llama.cpp / LM Studio / Ollama) → MLX (Apple Silicon)

Architecture, from its own config.json

62 layers · 52 sliding window · 10 full attention

32 attention heads over 16 KV heads (GQA 2:1), head dim 128 · dense feed-forward · vocabulary 262,208

Where the memory goes: weights by band above, plus a KV cache of 1.2GB at 8K · 3.1GB at 32K · 10.6GB at 128K, plus the runtime's own working memory — dominated by the output projection over that vocabulary.

Inside one layer, from its own tensors

62 × layer

52 sliding-window layers, 10 full-attention layers — the same block either way. What differs is how far back the mask lets attention look, not what the layer is built from.

input_layernormattentionq_projk_projv_projsoftmax(q·kᵀ/√d) · vq_norm · k_norm before the dotproduct32 query heads over 16 key/valueheadso_projpost_attention_layernormpre_feedforward_layernormfeed-forwardgate_proj · up_proj → down_proj21,504 widepost_feedforward_layernorm

The green line down the right of each half is the residual: it leaves the spine above the norm, skips the operation entirely and rejoins at the +. Where attention is drawn open, q, k and v are scaled to their real output widths — a model that shares key and value heads across many query heads shows one wide box beside two narrow ones.

Around the stack: an embedding of 262,208 × 5,376, a final norm, and an output projection that reuses the embedding (tied weights).

§1 Characteristics by quantization band

Q2–Q3 — 2–3 bit

13GB weights · e.g. Q3_K_M + KV: 1.2GB at 8K · 3.1GB at 32K · 10.6GB at 128K

— Not yet rated · contributions welcome —

Q4–Q5 — 4–5 bit

17GB weights · e.g. Q4_0 , Q4_K_M , mlx-4bit + KV: 1.2GB at 8K · 3.1GB at 32K · 10.6GB at 128K
Creative writing 6/10
data checked Oct 2026
  • quant-degradation-community — editorial band derivation @ Q4_0 · aggregated · 2026-10-04

    Editorial derivation: carried from the EQ-Bench-anchored full band; Google's QAT Q4_0 preserves prose quality well.

Math & reasoning 4/10
data checked Aug 2026
  • quant-degradation-community — community consensus @ Q4_0 · aggregated · 2026-08-30

    Editorial derivation: small loss vs full band; QAT helps.

Coding 4/10
data checked Aug 2026

Q6–Q8 — 6–8 bit

22GB weights · e.g. Q6_K + KV: 1.2GB at 8K · 3.1GB at 32K · 10.6GB at 128K

— Not yet rated · contributions welcome —

FP16 — full precision (fp16/bf16)

54GB weights · e.g. bf16 + KV: 1.2GB at 8K · 3.1GB at 32K · 10.6GB at 128K
Math & reasoning 5/10
data checked Aug 2026
Coding 4/10
data checked Aug 2026
Factuality 5/10
data checked Aug 2026
Creative writing 6/10
data checked Oct 2026
  • eqbench-creative — EQ-Bench Creative Writing v3 rubric score (0-20) = 15.33 @ bf16 · aggregated · 2026-10-04

    Best of the Gemma 3 sizes (12B 14.63, 4B 13.91) but behind every Gemma 4 model (16.01-16.03 for 31B and 26B-A4B) and well behind Muse Glimmer 30B (16.26). Normalised Elo 1265.7, 98th of 140, retrieved 2026-10-04. Independent; rubric judged by Claude Sonnet 4. EQ-Bench's normalised Elo moves as models are added, so the rubric score is the figure stored. Quant not stated by EQ-Bench (presumed full-precision API serving). Data file behind https://eqbench.com/creative_writing.html; the date the row was added is not published.

  • eqbench-creative — EQ-Bench Creative Writing Longform score (0-100) = 41.9 @ bf16 · aggregated · 2026-10-04

    Longform: level with Gemma 3 12B (41.8), below Gemma 4 31B (56.5) and Qwen3.8 27B (53.3). Data file behind the Longform page on eqbench.com. Quant not stated (presumed full precision).

§2 Known issues & what fixes them

ISSUE-01 severe Q2–Q3 / Q4–Q5 / Q6–Q8 / FP16 · coding

Real-world code editing is dismal — 4.9% on the Aider polyglot board, near the bottom, despite a respectable-looking HumanEval. Snippet benchmarks flatter it; repository work exposes it.

No fix
No known fix — choose a different model

Use a Qwen3 model for coding. Gemma's lane is chat, languages, writing, and vision.

Evidence · 2 sources · community-consensus
ISSUE-02 moderate Q2–Q3 / Q4–Q5 / Q6–Q8 / FP16 · tool-calling

No native function-calling tokens or template — tool use rides on prompt-following, which works surprisingly well but breaks more than trained-in tool calling.

Fix
Prompt technique [partial]

Give exact JSON schemas with examples in the system prompt — community benchmarks show it follows them decently.

No fix
No known fix — choose a different model

For heavy MCP/agent work, pick a model with trained-in tool calling.

Evidence · 2 sources · community-consensus
ISSUE-03 mild Q2–Q3 / Q4–Q5 / Q6–Q8 / FP16 · factuality

Family-wide tendency to hallucinate names and specific factual details; HHEM (7.4%) is oddly worse than the 4B.

Fix
RAG (retrieval) [strong]

Ground factual work; treat unsourced names/dates with suspicion.

Evidence · 1 source · anecdotal