Methodology

§1 Which models are here

Open-weight dense and MoE models up to 400B total parameters — the ones that actually fit on a machine you own. Everything above 70B here is a sparse MoE, because that is what the field built: total parameters decide what you have to hold in memory, active parameters decide how fast it runs, and both are on every sheet. Within that bound we list what's worth running now, not the history of how we got here.

So when a family supersedes itself, only the survivor gets a sheet. Granite 4.0 and 4.1 are absent because Granite 4.2 replaced them outright; Qwen2.5-Coder 32B and QwQ-32B because later Qwen releases cover the same ground better on the same hardware; OLMo 2 32B because Olmo 3 exists; Mistral Small 3.2 because Mistral Small 4 replaced it — a case that took a reader to spot, since the successor is five times the size and so did not look like a replacement. A catalog that keeps every generation becomes an archive, and the question here is "what do I download tonight."

Exclusion is not a verdict on quality. Those weights still run fine, and if one is already on your disk there's no reason to delete it — it only means someone starting fresh, on the same hardware, has a better option from the same maker. Competition between families is judged differently: a model keeps its slot if it's the best answer for some machine or some workload, even when a rival vendor beats it overall. That's why several models here are outscored by others on this very site.

§2 Quantization bands

Every measurement is recorded at the exact quant/precision it was taken at, but scores and recommendations are presented per quantization band, because that's the granularity a human can act on:

• Q2–Q3 — 2–3 bit (Q2_K, Q3_K_M, IQ3_XXS, MLX 3-bit, …) — 1-bit formats are deliberately not rated here
• Q4–Q5 — 4–5 bit (Q4_K_M, MLX 4-bit, MXFP4, AWQ, …) — the everyday local sweet spot
• Q6–Q8 — 6–8 bit (Q6_K, Q8_0, MLX 8-bit, FP8) — near-lossless
• FP16 — full precision (fp16/bf16) — what published benchmarks measure

Bands are format-agnostic: GGUF (llama.cpp, LM Studio, Ollama), MLX (Apple Silicon), and AWQ/GPTQ quants all map into the same band by bits-per-weight, and every measurement records the exact format it was taken at. When a specific quant deviates meaningfully from its band (e.g. IQ3_XXS vs Q3_K_M), we record a per-quant override and surface it as a warning on the model sheet.

§3 Does it fit? — the memory model

"Fits" is computed for the machine you describe, not looked up from a bucket. Two halves:

• budget = min(GPU ceiling, RAM − reserve)
• demand = weights + KV cache at your context + runtime overhead

On Apple Silicon the ceiling is the real constraint: macOS lets the GPU wire down about 75% of unified memory and llama.cpp treats that as a hard limit — roughly 24GB on a 32GB Mac, not 32. That 75% is our assumption, not your machine's answer. The real figure varies by chip and capacity — reported between two thirds and three quarters, and measured at 78% on one 32GB M2 Max — and you do not have to take ours: Metal will tell you, via mx.device_info()['max_recommended_working_set_size']. On any specific machine that number beats every percentage on this page, including this one. You can raise it, and when a model is blocked by nothing else we say so and give you the command. How far you can raise it is not something this site knows. Apple documents the setting nowhere, so there is no official guidance and no number here is authoritative — ours is arithmetic, deliberately more aggressive than the published advice we show beside it, and every option past that advice says so. What makes it approachable is that it reverts on reboot: treat it as a session, raise it, run the model, put it back. The reserve (what's left for the OS and your other apps) is proportional rather than flat, because a fixed allowance is wrong at both ends of the range.

The KV cache is the half everyone forgets. It grows with context, and it varies enormously by architecture: Qwen3.6 27B caches only 16 of its 64 layers, so it needs 2GB at 32K where a conventional model its size needs four times that. Every model's figure is computed from its published config.json and linked from its sheet. A model with no measured figure is not recommended at all rather than assigned a guess.

The third term is the one most calculators leave out, and it is derived rather than guessed. A running model needs somewhere to put its working numbers, and the pieces are knowable: the final vocabulary projection for one micro-batch, the peak live activations through the widest part of the network, and the attention mask, which is the only part that grows with your context. The first is the one nobody expects and usually the largest — turning each token into a score for every word in a 256,000-word vocabulary is half a gigabyte before anything else is counted.

The check on all this is that llama.cpp publishes a worked example: a 507MB compute buffer for Gemma 2 9B. The vocabulary projection alone predicts 500MB. That is the mechanism, to within 1.4%, with nothing fitted to it. Only one input is not derived — llama.cpp and its kernels, which published figures put between 0.5 and 1.5GB — so the estimate is a range, and any verdict about whether a model fits takes the pessimistic end.

This replaced a term of the form fixed cost plus a share of the weights, and the replacement matters more than a refinement would. Parameter count does not size a compute buffer; vocabulary, width and batch do. The old shape charged a 397B mixture-of-experts 15.6GB of scratch it never uses, charged a model a third as wide three times more than a wider one, and undercharged the small models with the largest vocabularies.

It has not been checked against a real machine. This page previously claimed corroboration from a measured ~80K context on a 32GB Mac. That reading was taken with the wired limit raised, so it was never a 24GB-budget measurement and could not test this term at all — the claim is withdrawn rather than reworded, and the same reading was what the old term had been fitted to. The derivation stands on its mechanism, which anyone can check against llama.cpp's published example. If you want to test it properly, the procedure is in the repo — and record the wired limit in force when you run it.

Two assumptions come with it, both stated on every card that depends on them: flash attention is on, which is llama.cpp's default and which saves gigabytes at long context; and one serving slot at the default 512 micro-batch. Running --parallel N multiplies the KV cache by N, which we do not model.

Verdicts are comfortable, tight (it runs, but you'll be closing other apps — or it's spilling onto your GPU's slow path) or won't fit. A model that runs at 8K but not 128K is a useful answer with a condition attached, not a yes or a no.

§4 How fast? — the bandwidth model

Fitting is only half a buying question. 176 of the 181 machines listed here share a memory capacity with another one, so "does it fit" gives the same answer for most of the list. Speed is what separates them, and it comes from one number:

• bytes read / token = active weights + KV cache at your context
• tokens / second = memory bandwidth ÷ bytes read, derated for real engines

Generating a token means reading every active weight once and doing about two arithmetic operations per byte, so the machine spends its time waiting on memory and compute barely enters. Two consequences worth knowing. A mixture-of-experts model reads only the experts it activates, which is why a 122B-A10B generates like a 10B while occupying 122B of memory. And the KV cache is read in full on every token, so the same model on the same machine is markedly slower at 128K context than at 4K — a cost most tokens-per-second calculators leave out, and one this site can price because it already computes your cache exactly.

A model too big for a graphics card is split, not abandoned: the card holds what it can and the CPU runs the remaining layers itself, out of system RAM. This applies to a graphics card in a machine with its own RAM, and not to unified memory. On Apple Silicon there is one pool: the CPU adds no memory and no bandwidth, and reads that pool more slowly than the GPU does — 220–240GB/s against the GPU's ~330 on an M1 Max's 400GB/s. llama.cpp's maintainer puts it plainly, that if the GPU has already saturated the bandwidth then the CPU cannot help; nor is there any parallelism to win, since each layer needs the one before it. So a split is possible there and pointless, and the site does not offer one. On those machines the lever is raising the wired limit, which we say instead. Both processors work on every token, so the estimate is a weighted average of their two speeds — and because system RAM is several times slower than a card's memory, the CPU's share dominates. That is the arithmetic behind the warning that a half-split model is usually not worth running.

This site used to price that slower half at PCIe bandwidth, about 32GB/s, on the theory that the spilled weights crossed the bus every token. They do not — they stay in system RAM and are read there — and the error was large enough to be visible from inside the data: below about 67% of a model on the card, the old arithmetic reported a machine with an RTX 3090 as slower than the same site's machine with no graphics card at all. Correcting it raises split-model estimates by roughly two to three times. A published -ngl sweep is the check: measured at 2.89 tok/s on the CPU alone and 43.18 on the card alone, the corrected model predicts the measured 83% point at 12.99 against 12.5 actual, with nothing fitted. The old one predicted 9.27.

Which makes the speed of your system RAM part of the answer, so the picker asks for it. Two channels of DDR5 read at about 90GB/s and an eight-channel workstation at about 384 — a fourfold spread that decides how much a split costs you. One case really is PCIe-bound, and it is the one nobody chooses: a driver silently spilling graphics memory into system RAM behind your back is far worse than splitting the model deliberately.

On a Mac that number comes from the chip, never from the capacity. A 32GB Mac is anything from an M4 at 120GB/s to an M5 Max at 614 — a fivefold spread — so the picker asks which chip you have rather than guessing from the memory. Bandwidth does not track generation either: the M3 Pro reads at 150GB/s where the M1 and M2 Pro both managed 200. Every figure is a published Apple spec, held in a table because no formula reproduces it.

These figures assume a GGUF runtime — llama.cpp, LM Studio, Ollama — because that is what the efficiency was measured on: an 8B at 4-bit on an RTX 4090, which reaches about half of its theoretical peak. That one measurement is then applied to every platform, which is the model's weakest joint. llama.cpp on Metal measures 60–67% of theoretical across four Apple chips, so Mac figures here are likely low at the bottom of the range.

MLX is a different runtime and is not modelled. It is widely reported as faster, and on mixture-of-experts models at short context it clearly is — but the published comparisons disagree badly. Reduced to a fraction of theoretical bandwidth, MLX lands at 50% on one machine and 65% on another, against a steady 60–67% for llama.cpp; and the frequently quoted 1.4–1.6× cannot hold, since 65% of peak multiplied by 1.45 is 94%, which sustained weight streaming does not reach. The advantage is also reported to fall away past roughly 40K of context. Picking a number out of that spread would be invention, so there is none here. Nor is it safe to call these a floor: at the low end of that spread MLX sits below llama.cpp, not above it. If you run MLX at short context, expect these numbers or somewhat better; at long context, expect these numbers.

Prompt processing gets a class — fast, moderate or slow — rather than a number. It is compute-bound where generation is bandwidth-bound, and the two genuinely invert: a DGX Spark ingests a long prompt several times quicker than a Ryzen AI Max+ while generating no faster. A tok/s figure there would depend on the backend and is honestly measured for only a handful of machines, so we grade it instead of inventing it.

Speed is one axis of a buying decision, and rarely the deciding one. Nothing here accounts for price, performance per pound, power draw, heat, noise, physical size or portability. A four-card 3090 rig and a laptop can post similar numbers and suit completely different lives — one of them wants a 1600W supply and a room that can take the heat. Treat these figures as one input you can compute, not as advice on what to buy.

§5 Scores are editorial, evidence is cited

Each 0–10 score is a human judgment over the cited evidence — not an automated composite, because averaging incommensurable benchmarks would be false precision. Every score carries its provenance: source, metric, raw value, retrieval date, the precision it was measured at, and a method tag — aggregated (from a public benchmark), owner-verified (run on our M1 Max 32GB, so only for models that fit it), or vendor-reported (the maker's own numbers, trust accordingly).

Published benchmarks run at full precision, so they anchor the full quantization band directly. Lower-band scores are editorial derivations from community quantization evidence (KL-divergence and perplexity tables, degradation reports), each labeled as such. Cells with no public evidence stay blank — blank is honest; a guessed number is not.

When the picker ranks models for a workload, it weights the dimensions that workload needs — and most models are unrated on most dimensions. Averaging only the cells that have evidence rewards ignorance: a model rated on a tenth of a task, scoring 8 there, would beat one rated on seven tenths averaging 7. That is an implicit claim that the 90% nobody measured is as good as the 10% somebody did.

So unmeasured weight counts as merely average instead, and each recommendation shows the score we measured alongside how much of the workload it covers. No cell is invented and nothing is stored — the blanks above stay blank. It changes only what gets recommended, and it is a weaker assumption than the one it replaces.

Nothing here has been spot-checked on our own hardware yet. Every score is either vendor-reported or aggregated from a public benchmark — zero cells carry an owner-verified provenance entry. We used to print that warning only on the largest hardware tier, which implied the smaller ones had been verified. They had not. The warning is now on every recommendation, and it comes off a model only when a real spot-check replaces it.

§6 Known issues & fixes

The point of this site. Weakness entries require citations (community-consensus claims need at least two), and every workaround is a real, linked technique. When nothing fixes a weakness, the entry says "no fix — choose a different model" rather than pretending.

§7 Source licensing

We only store numbers from sources whose terms permit republication, always with attribution. Sources that forbid it (e.g. Artificial Analysis) are cited by link only — our validation pipeline structurally rejects their values.

§8 Revision cadence

The site updates every 7 days: an automated pipeline proposes score refreshes as a pull request; a human reviews conflicts and merges. Last editorial review: 2026-08-30. Evidence older than 60 days is flagged in the data and on the sheets.