Methodology

§1 Which models are here

Open-weight dense and MoE models up to 400B total parameters — the ones that actually fit on a machine you own. Everything above 70B here is a sparse MoE, because that is what the field built: total parameters decide what you have to hold in memory, active parameters decide how fast it runs, and both are on every sheet. Within that bound we list what's worth running now, not the history of how we got here.

So when a family supersedes itself, only the survivor gets a sheet. Granite 4.0 and 4.1 are absent because Granite 4.2 replaced them outright; Qwen2.5-Coder 32B and QwQ-32B because later Qwen releases cover the same ground better on the same hardware; OLMo 2 32B because Olmo 3 exists; Mistral Small 3.2 because Mistral Small 4 replaced it — though the successor is five times the size and so did not look like one. A catalog that keeps every generation becomes an archive, and the question here is "what do I download tonight."

Where a model was already on the site when its replacement arrived, the sheet stays and is marked SUPERSEDED instead of disappearing — the picker stops offering it, the explore table hides it unless you ask, and the evidence behind it remains readable. Ten are marked that way: Nemotron 3 Nano 30B-A3B and GLM-4.7, replaced by Nemotron 3.5 Lightning and GLM-5.3-Flash; Gemma 3 4B, 12B and 27B, replaced by Gemma 4; Qwen3 4B, 8B, 30B-A3B and 32B, replaced by Qwen3.5 4B, Qwen3.5 9B, Qwen3.6 35B-A3B and Qwen3.8 27B; and Qwen3.6 27B, replaced by Qwen3.8 27B. A sheet that vanishes takes its provenance with it, and someone who already downloaded those weights still deserves to read what we knew about them.

Exclusion is not a verdict on quality. Those weights still run fine, and if one is already on your disk there's no reason to delete it — it only means someone starting fresh, on the same hardware, has a better option from the same maker. Competition between families is judged differently: a model keeps its slot if it's the best answer for some machine or some workload, even when a rival vendor beats it overall. That's why several models here are outscored by others on this very site.

§2 Quantization bands

Every measurement is recorded at the exact quant/precision it was taken at, but scores and recommendations are presented per quantization band, because that's the granularity a human can act on:

• Q2–Q3 — 2–3 bit (Q2_K, Q3_K_M, IQ3_XXS, MLX 3-bit, …) — 1-bit formats are deliberately not rated here
• Q4–Q5 — 4–5 bit (Q4_K_M, MLX 4-bit, MXFP4, AWQ, …) — the everyday local sweet spot
• Q6–Q8 — 6–8 bit (Q6_K, Q8_0, MLX 8-bit, FP8) — near-lossless
• FP16 — full precision (fp16/bf16) — what published benchmarks measure

Bands are format-agnostic: GGUF (llama.cpp, LM Studio, Ollama), MLX (Apple Silicon), and AWQ/GPTQ quants all map into the same band by bits-per-weight, and every measurement records the exact format it was taken at. When a specific quant deviates meaningfully from its band (e.g. IQ3_XXS vs Q3_K_M), we record a per-quant override and surface it as a warning on the model sheet.

§3 Does it fit? — the memory model

"Fits" is computed for the machine you describe, not looked up from a bucket. Two halves:

• budget = min(GPU ceiling, RAM − reserve)
• demand = weights + KV cache at your context + runtime overhead

On Apple Silicon the ceiling is the real constraint: macOS lets the GPU wire down about 75% of unified memory and llama.cpp treats that as a hard limit — roughly 24GB on a 32GB Mac, not 32. That 75% is our assumption, not your machine's answer. The real figure varies by chip and capacity — reported between two thirds and three quarters, and measured at 78% on one 32GB M2 Max — and you do not have to take ours: Metal will tell you, via mx.device_info()['max_recommended_working_set_size']. On any specific machine that number beats every percentage on this page, including this one. You can raise it, and when a model is blocked by nothing else we say so and give you the command. How far you can raise it is not something this site knows. Apple documents the setting nowhere, so there is no official guidance and no number here is authoritative — ours is arithmetic, deliberately more aggressive than the published advice we show beside it, and every option past that advice says so. What makes it approachable is that it reverts on reboot: treat it as a session, raise it, run the model, put it back. The reserve (what's left for the OS and your other apps) is proportional rather than flat, because a fixed allowance is wrong at both ends of the range.

The KV cache is the half everyone forgets. It grows with context, and it varies enormously by architecture: Qwen3.6 27B caches only 16 of its 64 layers, so it needs 2GB at 32K where a conventional model its size needs four times that. Every model's figure is computed from its published config.json and linked from its sheet. A model with no measured figure is not recommended at all rather than assigned a guess.

Those figures are at f16, which is how llama.cpp, Ollama and LM Studio store the cache unless told otherwise. You can tell the picker otherwise: set KV cache to q8_0 and the cache is priced at 34 bytes per 32 values instead of 64, 0.53×, which is the block layout's arithmetic rather than a measured saving. It is the cheapest way to buy context, and the quality cost is widely reported as hard to measure; we have not measured it. q4_0 is not offered, because it is reported to break math.

Sliding-window layers are priced at what llama.cpp actually allocates for them, which is not the window. It holds the window plus one micro-batch, rounded up to 256 cells: a 1,024-token window keeps 1,536 tokens, and a 128-token window keeps 768. The rule is one line of its source, and this site priced the window alone until a real startup log disagreed.

The third term is the one most calculators leave out, and it is derived rather than guessed. A running model needs somewhere to put its working numbers, and the pieces are knowable: the final vocabulary projection for one micro-batch, the peak live activations through the widest part of the network, and the attention mask, which is the only part that grows with your context. The first is the one nobody expects and usually the largest — turning each token into a score for every word in a 256,000-word vocabulary is half a gigabyte before anything else is counted.

The check on all this is that llama.cpp publishes a worked example: a 507MB compute buffer for Gemma 2 9B. The vocabulary projection alone predicts 500MB. That is the mechanism, to within 1.4%, with nothing fitted to it. Only one input is not derived — llama.cpp and its kernels, which published figures put between 0.5 and 1.5GB — so the estimate is a range, and any verdict about whether a model fits takes the pessimistic end.

It has not been checked against a real machine. It rests on its mechanism, which anyone can verify against llama.cpp's published example. To test it properly, the procedure is in the repo — and record the wired limit in force when you run it, since a raised cap changes the budget the reading is measured against.

Strix Halo is the platform where the ceiling depends on the operating system. On Windows it is the UEFI VRAM split, up to 96GB of 128GB, which is the 75% used here. On Linux, AMD's own guidance is to keep the firmware split tiny and raise the shared GTT limit instead, which starts at about half of RAM. An untouched Linux install therefore holds less than we assume, and a configured one holds more. Set the limit you actually use.

Two assumptions come with it, both stated on every card that depends on them: flash attention is on, which is llama.cpp's default and which saves gigabytes at long context; and one serving slot at the default 512 micro-batch. Running --parallel N multiplies the KV cache by N, which we do not model.

Verdicts are comfortable, tight (it runs, but you'll be closing other apps — or it's spilling onto your GPU's slow path) or won't fit. A model that runs at 8K but not 128K is a useful answer with a condition attached, not a yes or a no.

§4 How fast? — the bandwidth model

Fitting is only half a buying question. 210 of the 217 machines listed here share a memory capacity with another one, so "does it fit" gives the same answer for most of the list. Speed is what separates them, and it comes from one number:

• bytes read / token = active weights + KV cache at your context
• seconds / token = bytes read ÷ memory bandwidth + layers × per-layer overhead

Generating a token means reading every active weight once and doing about two arithmetic operations per byte, so the machine spends most of its time waiting on memory. Not all of it: the second term is everything that happens per layer regardless of how much weight is read — kernel launches, attention over the cache, sampling, expert routing — and it is added rather than applied as a percentage.

That distinction matters more than it sounds. This site used to derate bandwidth by a fixed 45–70%, and a fraction is the wrong shape for a fixed cost: it flatters anything that reads little per token. Sparse mixture-of-experts models read a tenth of themselves, so they were being predicted at roughly twice their real speed — a published Qwen3.5-35B-A3B llama.cpp measurement on an M4 Max is 70–72 tok/s where the old arithmetic said 120–187. The per-layer form gets it right without a special case, and it explains the same puzzle in llama.cpp's own table, where 4-bit looks less "efficient" than FP16 on identical hardware.

The overhead is charged per layer because that is what the measurements show: solved as a flat per-token cost it comes to 6–14ms on a 32-layer 7B and 25–41ms on an 80-layer 70B, and divided by layer count both land in the same few tenths of a millisecond. A flat per-token cost misses the 70B rows entirely. It is set per platform, since CUDA dispatches more cheaply than Metal, and AMD and Intel currently sit inside the CUDA figure, which is probably optimistic for them.

On Apple Silicon it is 0.05–0.30ms per layer, set from 92 published measurements attributed to a named chip, across llama.cpp and MLX, and it brackets 76 of them. The range is wide because the overhead is not one number: routing a mixture-of-experts costs two to three times what a dense layer costs, and a current llama.cpp build dispatches about half as expensively as the 2023 build in its own table. What escapes the range is dense models under MLX on M3 Ultra and M4 chips, which run up to a tenth quicker than its top, and prompts of 15K tokens or more, where attention over the cache pushes below its bottom. The previous figure, 0.21–0.45ms, was fitted to a 70B on an M2 Ultra including a 2-bit row that is bound by dequantisation rather than memory, and it left 75 of the 92 above the range.

Two consequences worth knowing. A mixture-of-experts model reads only the experts it activates, which is why a 122B-A10B generates like a 10B while occupying 122B of memory. And the KV cache is read in full on every token, so the same model on the same machine is markedly slower at 128K context than at 4K — a cost most tokens-per-second calculators leave out, and one this site can price because it already computes your cache exactly.

A model too big for a graphics card is split, not abandoned: the card holds what it can and the CPU runs the remaining layers itself, out of system RAM. This applies to a graphics card in a machine with its own RAM, and not to unified memory. On Apple Silicon there is one pool: the CPU adds no memory and no bandwidth, and reads that pool more slowly than the GPU does — 220–240GB/s against the GPU's ~330 on an M1 Max's 400GB/s — a figure that is not assumed either, since the same llama.cpp table gives 332GB/s on an M2 Max and 338 on an M3 Max at FP16. llama.cpp's maintainer puts it plainly, that if the GPU has already saturated the bandwidth then the CPU cannot help; nor is there any parallelism to win, since each layer needs the one before it. So a split is possible there and pointless, and the site does not offer one. On those machines the lever is raising the wired limit, which we say instead. Both processors work on every token, so the estimate is a weighted average of their two speeds — and because system RAM is several times slower than a card's memory, the CPU's share dominates. That is the arithmetic behind the warning that a half-split model is usually not worth running.

The spilled layers are not streamed across PCIe; they stay in system RAM and the CPU reads them there, which is why your system memory speed is what prices them. A published -ngl sweep is the check: measured at 2.89 tok/s on the CPU alone and 43.18 on the card alone, this model predicts the measured 83%-on-card point at 12.99 against 12.5 actual, with nothing fitted.

Which makes the speed of your system RAM part of the answer, so the picker asks for it. Two channels of DDR5 read at about 90GB/s and an eight-channel workstation at about 384 — a fourfold spread that decides how much a split costs you. One case really is PCIe-bound, and it is the one nobody chooses: a driver silently spilling graphics memory into system RAM behind your back is far worse than splitting the model deliberately.

On a Mac that number comes from the chip, never from the capacity. A 32GB Mac is anything from an M4 at 120GB/s to an M5 Max at 614 — a fivefold spread — so the picker asks which chip you have rather than guessing from the memory. Bandwidth does not track generation either: the M3 Pro reads at 150GB/s where the M1 and M2 Pro both managed 200. Every figure is a published Apple spec, held in a table because no formula reproduces it.

These figures assume a GGUF runtime — llama.cpp, LM Studio, Ollama — because that is what every measurement behind them was taken on, apart from the Apple Silicon range above, which was set on MLX as well. The per-layer overhead is set separately for CUDA and for Metal, from llama.cpp's own Apple Silicon benchmark table and a published 70B study, rather than measuring one platform and applying it everywhere — which is what the previous model did and its own note called its weakest joint. What remains unmeasured is AMD and Intel, which currently borrow the CUDA figure and are probably slower than it suggests.

MLX is a different runtime, and the Apple Silicon range holds both. Controlled same-machine comparisons at 4-bit on both sides put MLX at 1.0–1.2× llama.cpp on most models — 1.04× on Qwen3-8B, 1.03–1.19× on Qwen3-30B-A3B, 1.09× on Qwen3-32B — and at 1.6–1.9× on Qwen3.5, whose linear-attention layers llama.cpp runs less well: 126–132 against 70–72 tok/s on a 35B mixture-of-experts and 1.63× on a dense 9B. So on a Mac running MLX, expect the top of the range on most models, and above it on Qwen3.5.

One number worth distrusting is the 3× you will see quoted. That comes from Ollama switching its own backend, and roughly half of it is Ollama's wrapper rather than MLX: on the same machine, raw llama.cpp reaches 89.4 tok/s where Ollama reaches 43.5. It is a real speedup for Ollama users and not a measure of MLX against llama.cpp.

It gets no multiplier of its own because a single multiplier would be the wrong shape. MLX ships its own quantizations at their own bits-per-weight — 4-bit is 4.5 bits per weight exactly, where Q4_K_M is about 4.85, so the same model is 17GB as Q4_K_M and 15.8GB in MLX, which is free speed from reading less and most of the 1.0–1.2× — and the rest moves with architecture and runtime version. This project has measured none of it on its own hardware; the range is set from other people's published runs, and the document above lists every one.

Prompt processing gets a class — fast, moderate or slow — rather than a number. It is compute-bound where generation is bandwidth-bound, and the two genuinely invert: a DGX Spark ingests a long prompt several times quicker than a Ryzen AI Max+ while generating no faster. A tok/s figure there would depend on the backend and is honestly measured for only a handful of machines, so we grade it instead of inventing it.

The grade is per platform, with overrides where a machine breaks from its platform, and on Apple silicon the M5 is that break. Its GPU cores carry matrix units, and on llama.cpp's own 7B prompt benchmark a 40-core M5 Max ingests 3,220 tokens/s where a 40-core M4 Max measures 886 and an 80-core M3 Ultra 1,471 — prefill no longer tracks bandwidth at all. So the M5 Pro, Max and Ultra are graded fast; the base M5, at 723, posts a 32-core M4 Max's figure and stays moderate. The grade assumes a current llama.cpp on macOS 26, which is what reaches those units; an older build gets M4-class figures on the same chip, and the picker says so whenever an M5 is selected.

Every estimate assumes speculative decoding is off. It is off by default in llama.cpp, and the formula above matches llama.cpp at its defaults: 16–17 tok/s measured for Qwen3.8 27B on an M5 Pro and an M3 Max, where this site estimates 13–16 and 15–20. Speculative decoding breaks the formula rather than tuning it. A small draft model proposes several tokens and the real one checks them all in a single pass over its weights, so bandwidth no longer caps the speed. How many tokens each pass yields depends on the text, not the machine. Ollama's MLX engine turns this on by default for Gemma 4 and Qwen3.5 on Macs, and Splash, an Apple-silicon engine for two models only, uses it on every token.

So for Splash the picker shows measured speeds, not an estimate, and says who measured them. The spread is the reason. On one M3 Max, independent tests of Qwen3.8 27B ran from 32 tok/s on varied technical prose to 83 on repetitive text, while the vendor reports 92 on coding prompts; Qwen3.6 35B-A3B gained almost nothing over plain engines on prose and 2.7× on repetition. The memory side is computed as usual, from Splash's own published packages: the draft model and vision encoder are added, and the KV cache is priced at half, because Splash stores it at 8-bit by default. Splash's SSD cache does not add context, by its own documentation, so it is not counted.

Speed is one axis of a buying decision, and rarely the deciding one. Nothing here accounts for price, performance per pound, power draw, heat, noise, physical size or portability. A four-card 3090 rig and a laptop can post similar numbers and suit completely different lives — one of them wants a 1600W supply and a room that can take the heat. Treat these figures as one input you can compute, not as advice on what to buy.

§5 Scores are editorial, evidence is cited

Each 0–10 score is a human judgment over the cited evidence — not an automated composite, because averaging incommensurable benchmarks would be false precision. Every score carries its provenance: source, metric, raw value, retrieval date, the precision it was measured at, and a method tag — aggregated (from a public benchmark), owner-verified (run on our own hardware, so only for models that fit it), or vendor-reported (the maker's own numbers, trust accordingly).

Published benchmarks run at full precision, so they anchor the full quantization band directly. Lower-band scores are editorial derivations from community quantization evidence (KL-divergence and perplexity tables, degradation reports), each labeled as such. Cells with no public evidence stay blank — blank is honest; a guessed number is not.

When the picker ranks models for a workload, it weights the dimensions that workload needs — and most models are unrated on most dimensions. Averaging only the cells that have evidence rewards ignorance: a model rated on a tenth of a task, scoring 8 there, would beat one rated on seven tenths averaging 7. That is an implicit claim that the 90% nobody measured is as good as the 10% somebody did.

So unmeasured weight counts as merely average instead, and each recommendation shows the score we measured alongside how much of the workload it covers. No cell is invented and nothing is stored — the blanks above stay blank. It changes only what gets recommended, and it is a weaker assumption than the one it replaces.

Nothing here has been spot-checked on our own hardware yet. Every score is either vendor-reported or aggregated from a public benchmark — zero cells carry an owner-verified provenance entry. The warning sits on every recommendation, at every hardware tier, and comes off a model only when a real spot-check replaces it.

§6 Known issues & fixes

The point of this site. Weakness entries require citations (community-consensus claims need at least two), and every workaround is a real, linked technique. When nothing fixes a weakness, the entry says "no fix — choose a different model" rather than pretending.

§7 Source licensing

We only store numbers from sources whose terms permit republication, always with attribution. Sources that forbid it (e.g. Artificial Analysis) are cited by link only — our validation pipeline structurally rejects their values.

§8 Revision cadence

The site updates every 7 days: an automated pipeline proposes score refreshes as a pull request; a human reviews conflicts and merges. Last editorial review: 2026-10-04. Evidence older than 60 days is flagged in the data and on the sheets.