Model datasheet · Gemma 4
Gemma 4 E4B
Phone-class Gemma: Per-Layer Embeddings mean 8B of weights behave like 4.5B at inference, and the official QAT build is 5.15GB. Text, image and audio in the 8GB tier — but the benchmarks are honest about the ceiling.
- Vendor
- Architecture
- Dense · 8B
- Context
- 131,072 tokens
- License
- Apache-2.0
- Released
- 2026-03-31
- Vision
- Yes
→ GGUF (llama.cpp / LM Studio / Ollama) → MLX (Apple Silicon)
Architecture, from its own config.json
42 layers · 20 sliding window · 4 full attention · 18 sharing an earlier layer's cache
Its last 18 of 42 layers have no keys and values of their own — they read those of an earlier layer, so they add nothing to the cache no matter how long the context gets.
8 attention heads over 2 KV heads (GQA 4:1), head dim 256 — its full-attention layers use a different shape again, 2 KV heads of 512, which is the shape that decides the cache at long context · mixture of experts, 4.5B of 8B active per token · vocabulary 262,144
Where the memory goes: weights by band above, plus a KV cache of 168MB at 8K · 552MB at 32K · 2GB at 128K, plus the runtime's own working memory — dominated by the output projection over that vocabulary.
Inside one layer, from its own tensors
42 × layer
35 sliding-window layers, 7 full-attention layers — the same block either way. What differs is how far back the mask lets attention look, not what the layer is built from.
The green line down the right of each half is the residual: it leaves the spine above the norm, skips the operation entirely and rejoins at the +. Where attention is drawn open, q, k and v are scaled to their real output widths — a model that shares key and value heads across many query heads shows one wide box beside two narrow ones.
Around the stack: an embedding of 262,144 × 2,560, a final norm, and an output projection that reuses the embedding (tied weights).
Also in every block and not drawn, because their tensor names do not fix where they belong: layer_scalar, per_layer_input_gate, per_layer_projection, post_per_layer_input_norm. Guessing an order is the one thing this diagram will not do.
§1 Characteristics by quantization band
— Not yet rated · contributions welcome —
Q4–Q5 — 4–5 bit
5GB weights · e.g. Q4_0 , Q4_K_M , mlx-4bit + KV: 168MB at 8K · 552MB at 32K · 2GB at 128Kdata checked Aug 2026
- quant-degradation-community — community consensus @ Q4_0 · aggregated · 2026-08-30
Editorial derivation: QAT Q4_0 (5.15GB) is the intended build, but small models have less redundancy to lose — one point down.
data checked Aug 2026
- quant-degradation-community — community consensus @ Q4_0 · aggregated · 2026-08-30
Editorial derivation: carried down one point from the full band.
data checked Aug 2026
- quant-degradation-community — community consensus @ Q4_0 · aggregated · 2026-08-30
Editorial derivation: an already-marginal tool layer at 4-bit. Treat tool use as unsupported here.
— Not yet rated · contributions welcome —
FP16 — full precision (fp16/bf16)
16GB weights · e.g. bf16 + KV: 168MB at 8K · 552MB at 32K · 2GB at 128Kdata checked Aug 2026
- vendor-model-card — AIME 2026 (no tools) = 42.5 @ bf16 · vendor-reported · 2026-08-30
GPQA Diamond: 58.6, MMLU-Pro: 69.4. Strong for 4.5B effective parameters, weak in absolute terms.
data checked Aug 2026
- vendor-model-card — LiveCodeBench v6 = 52 @ bf16 · vendor-reported · 2026-08-30
Competition-style problems only; no agentic or repo-level evidence at this size.
data checked Oct 2026
- vendor-model-card — Tau2 average (with tools) = 42.2 @ bf16 · vendor-reported · 2026-08-30
Half the 12B's score. Usable for one or two well-described tools, not for agent loops.
- vendor-model-card — BFCL-v4 (FC) = 41.64 · aggregated · 2026-10-04
Terminal-Bench 2.1 1.90 in the same table. Consistent with the vendor Tau2 42.2: tool use is marginal at this size. Third-party measurement by inclusionAI, published as an image in the Ling-3.0-tiny model card (column 'Gemma-4-E4B-it (Thinking)'). A competitor-run comparison, not Google's number and not a neutral leaderboard; thinking mode, quant not stated (presumed full precision). The -AA rows in the same table are deliberately not recorded.
data checked Aug 2026
- vendor-model-card — MRCR v2 @128K (8 needle) = 25.4 @ bf16 · vendor-reported · 2026-08-30
The 128K window is nominal — a quarter of needles found.
data checked Oct 2026
- vendor-model-card — IFBench = 44.2 · aggregated · 2026-10-04
LIFEBench 52.80, Multi-IF 80.92 in the same table — fine on multi-turn adherence, weak on IFBench's harder verifiable constraints. Third-party measurement by inclusionAI, published as an image in the Ling-3.0-tiny model card (column 'Gemma-4-E4B-it (Thinking)'). A competitor-run comparison, not Google's number and not a neutral leaderboard; thinking mode, quant not stated (presumed full precision). The -AA rows in the same table are deliberately not recorded.
§2 Known issues & what fixes them
Tau2 42.2 against the 12B's 69.0 — by the vendor's own measurement, roughly two in five tool-use episodes go wrong. This is a capacity ceiling at 4.5B effective parameters, not a bug, and no runtime update moves it.
One or two tools with unambiguous schemas, single-step calls, and validate every argument before executing. Multi-step agent loops are out of reach at this size.
If the workload genuinely needs reliable tool use, move up to Gemma 4 12B (Tau2 69.0) — a 9GB rather than 7GB footprint.
Evidence · 1 source · anecdotal
- Gemma 4 model card — Tau2 per variant (E2B 24.5, E4B 42.2, 12B 69.0) (vendor, 2026-08-30)
The 128K context window is nominal: MRCR v2 at 128K is 25.4%, so three quarters of the needles are missed. Loading a long document and asking questions about it will silently produce confident, wrong answers.
Chunk and retrieve — feed it a page, not a book. This is the single highest-value mitigation for the E-series.
Evidence · 1 source · anecdotal
- Gemma 4 model card — MRCR v2 @128K per variant (vendor, 2026-08-30)
Inherits the family's local-runtime churn — official GGUF refreshes that would not load, and gibberish past ~230 tokens with KV offload enabled.
Current llama.cpp, a re-cut community quant, and no -nkvo.
Evidence · 2 sources · community-consensus
- Eval bug: Gemma4 models produce gibberish at some point with -nkvo (github, 2026-08-30)
- Latest official Google Gemma 4 GGUF models fail to load in LM Studio 0.4.19 (github, 2026-08-30)