What LLM your machine can run — and which one is a good fit for a specific job.

All models

• DeepSeek R1-0528 Qwen3-8B R1's reasoning distilled into a Qwen3-8B body: frontier-class competition math in 8B — AIME'24 86.0 matches models 30× its size. A specialist: brilliant at math and logic, awkward at everything agentic.
• DeepSeek-V4-Flash-0731 DeepSeek's 284B MoE with 13B active, in its text-only July release. The experts were trained in MXFP4 and ship that way, so Unsloth's 162GB UD-Q8_K_XL is a lossless copy of the release, and the 155-156GB 4-bit files sit close to it (mean KLD 0.013). The cache cannot be worked out from config.json, because three cache types compress at different ratios, so the figure here is read from llama.cpp's own startup log for this checkpoint at 64,000 tokens. In that log, 21 layers hold one 512-wide entry per 4 tokens plus a 128-wide index entry per 4, and 20 layers hold one 512-wide entry per 128 tokens. At 2 bytes per value that is 6,880 bytes per token, about 7GB at a million tokens. The log's total of 423 MiB for those caches agrees, once padding is counted. Small fixed buffers (the 128-token sliding window on every layer, compressor state) are priced separately or not at all. DeepSeek publishes agentic numbers only. Mainline mlx-lm cannot load this architecture yet, so no MLX link is given.
• Gemma 3 12B vision Superseded by Gemma 4 12B, Google's next generation at the same size. Kept as the record of what the previous generation scored: natural prose, multilingual chat and built-in vision, weaker than same-size rivals at code and tool use.
• Gemma 3 27B vision Superseded by Gemma 4 31B and Gemma 4 26B-A4B, Google's next generation in the same size class. Kept for its receipts: best-in-family prose and real vision, and the clearest case on this site of a vendor coding number (HumanEval 48.8) contradicted by an independent one (Aider polyglot 4.9%).
• Gemma 3 4B vision Superseded by Gemma 4 E4B (Google's next generation at the same 8GB-laptop size, with vision again). Kept as the record of what the previous generation scored — pleasant chat and image understanding on small machines, and weak at math and code.
• Gemma 4 12B vision The 'Unified' one: an encoder-free architecture that takes image and audio straight into the embedding space, at a size that leaves room to breathe on a 16GB machine. Punches near the 26B MoE on reasoning while being the only model here that hears.
• Gemma 4 26B-A4B vision Google's answer to the fast-MoE era: 26B of knowledge at 4B speed, vision, 256K window, QAT 4-bit weights, and finally an Apache license. The strongest all-rounder in the 16GB class right now.
• Gemma 4 31B vision The Gemma 4 family's dense flagship, and much stronger at long context than its own 26B sibling: 66.4% MRCR at 128K against 44.1%. Official QAT Q4_0 is 17.7GB, so the whole thing fits a 32GB machine.
• Gemma 4 E2B vision The smallest Gemma 4, at 3.35GB in Google's QAT build — genuinely multimodal on hardware that cannot run anything else in this catalog. Use it for dictation, captioning and simple chat; its own vendor numbers rule out reasoning and tools.
• Gemma 4 E4B vision Phone-class Gemma: Per-Layer Embeddings mean 8B of weights behave like 4.5B at inference, and the official QAT build is 5.15GB. Text, image and audio in the 8GB tier — but the benchmarks are honest about the ceiling.
• GLM-4.7-Flash The agentic-coding MoE of the moment: SWE-bench 59.2 from 3B active params, MIT-licensed, ~200K context. Watch it at long context, where its tool calls start fraying.
• GLM-4.7 Superseded by GLM-5.3-Flash (2026-08-25): same vendor, same MIT licence, same size class at 320B against 358B, and a newer generation. Kept as the record of what the previous architecture cost — all 92 layers full attention, 368KB of KV per token, where the replacement caches on 11 layers of 45. Its SWE-bench Verified 73.8 is still the highest measured coding score here.
• GLM-5.3-Flash vision Replaces GLM-4.7 in the same size class: 320B against 358B, MIT again. Z.ai reports Terminal-Bench 2.1 at 84.3, but the one independent rerun so far got 68.3% per trial, and the card does not say how 84.3 was counted. The architectural change is what matters here: 34 of 45 layers are linear attention and the other 11 cache a 512-wide latent, so a million tokens of context costs about 11GB where GLM-4.7 charged 368KB per token. The weights are the problem instead: 109GB at 2-bit, 200GB at 4-bit. llama.cpp support was merged on 2026-09-30 and is still settling.
• gpt-oss-120b The big sibling of gpt-oss-20b, and the same bargain at four times the size: shipped natively in MXFP4, so there is no quant ladder to choose from — one 63GB file is the model. Reasoning and tool use are excellent; world knowledge is not. It was built to look things up.
• gpt-oss-20b OpenAI's open MoE, shipped natively in MXFP4 (~4-bit) — the quant band question mostly answers itself. Fast, strong at STEM and tool use, and startlingly ignorant of the world: built to look things up, not to know them.
• Granite 4.2 30B IBM's enterprise reasoner with a thinking switch — one checkpoint that either reasons step by step or answers directly. Trained to use tools inside sandboxed environments rather than on transcripts of tool use, though its own BFCL v4 figure (61.39) is mid-pack rather than a standout. Every number here is IBM's, and IBM restated part of its table after launch.
• Granite 4.2 8B Apache-2.0, a thinking switch, and function calling trained in live sandboxes rather than scraped transcripts. IBM's own numbers are strong on math and instruction following for the size, but its BFCL v4 score (52.39 on the card, 50.29 on the blog) trails other 8–9B models here. Every number is IBM's.
• K2-Horizon 7B Does not run in stock llama.cpp, Ollama or LM Studio yet: support is an open pull request (llama.cpp #29535), so the GGUFs need IFM's own fork; the MLX build runs in stock mlx-lm with --trust-remote-code. IFM's own table is remarkable for the size (SWE-bench Verified 72.4, AIME 2026 90.2) and unconfirmed by anyone else. Named for its 7B backbone; with its two 1B embedding tables the file holds 9.0B parameters. It also caches heavily, about 18GB at 128K, and loses more to plain 4-bit than most models.
• K2-Horizon MoVA 36B-A4B Does not run in stock llama.cpp, Ollama or LM Studio yet: support is an open pull request (llama.cpp #29535), so the GGUFs need IFM's own fork. The MLX build runs in stock mlx-lm with --trust-remote-code. Fully open (weights, data and training code, Apache 2.0), but few of IFM's numbers are recordable here, and the one independent test found it looping on agentic coding and losing recall past the middle of a 256K context. Every layer caches, so the KV cache is heavy: about 24GB at 128K.
• KAT-Coder-V2.5-Dev Qwen3.6-35B-A3B with a coding-specific post-train bolted on — same architecture, same footprint, tuned hard for agentic software work and reported to use far fewer tokens getting there. Vision was stripped from the open release. A useful demonstration of what post-training alone is worth.
• Ling 3.0 Flash 124B parameters with 5.1B active, and only 7 of its 42 layers keep a KV cache, so 128K of context adds about 1GB to a 77GB Q4_K_M file. That puts the 4-bit file on 128GB machines. inclusionAI's own table, in thinking mode, puts it ahead of Nemotron 3 Super on every row the two share, including SWE-bench Pro (56.6 against 34.1) and MRCR at 128K (90.8 against 40.8). No independent figure is republished here yet. llama.cpp has run it since 2026-08-17, but the dedicated Ling 3.0 chat parser only merged on 2026-09-19 and schema-constrained output on 2026-10-03, so older builds undersell its tool use. There is no mlx-community build.
• Ling 3.0 Tiny 1.3B active parameters out of 7.9B, and the vendor reports it running at 86-90 tokens/s on an M4 Pro laptop in about 8.3GB. Three linear-attention layers for every latent-attention one means only 6 of 24 layers cache, so long context is nearly free. Runtime support has caught up — llama.cpp has had a dedicated Ling 3.0 chat parser since 2026-09-19 (release b11057) — but looping reports remain open.
• Llama 3.1 8B The compatibility baseline. Every local tool, tutorial, and fine-tune supports it, and its tool calling is still excellent — but 2025–26 peers beat it on raw reasoning. Pick it for ecosystem, not scores.
• Llama 3.2 3B Meta's edge-class model: built for summarization, rewriting, and on-device assistants, and honest about it. Runs on almost anything; don't ask it to be a scientist.
• Llama 3.3 70B The largest dense model in this catalogue — everything above it is a sparse MoE. Best instruction-following evidence on the site at IFEval 92.1, and it needs a 96GB machine to be comfortable at any real context, because its KV cache costs 10GB at 32K before the weights are counted.
• MiMo-V2.6-Flash (MOPD) vision Xiaomi's 309B MoE with 15B active, as the MOPD checkpoint: the 27 September follow-up to the RL release, retrained to stop the repeated tool calls the RL files were reported for. Xiaomi ships the experts in MXFP4, so the MXFP4 GGUF (167-175GB, depending on how the other layers are stored) is the model as released and there is nothing bigger worth downloading; the only smaller stock file is a 126GB Q2_K. 39 of 48 layers use a 128-token sliding window, so the cache stays small: the 9 global layers hold 23KB per token, about 22GB at a million tokens. That figure is computed from config.json, with keys 192 wide and values 128 wide, and is not a measured llama.cpp log. Every benchmark on this sheet was published for the RL checkpoint, not this one. Mainline mlx-lm cannot load this architecture yet, so no MLX link is given.
• MiniCPM5 2B The smallest model here that is not a compromise: 1.6GB at Q4_K_M, and OpenBMB's table puts it ahead of every 2B-class rival and level with several 4B ones on coding, math and tool use. A plain Llama-shaped dense transformer, so every runtime already handles it. The gap it cannot close is stored knowledge — 2.5B parameters hold what 2.5B parameters hold.
• Mistral Small 4 vision Mistral folded Magistral, Pixtral and Devstral into one model: reasoning, vision and agentic coding, Apache licensed, 6.5B active of 119B. The catch is attention. It kept full multi-head attention where everything else its size uses grouped-query, so its KV cache costs 576KB per token — the most expensive on this site, and more than a 358B model's. Cheap to compute, dear to give context to.
• Muse Glimmer 30B vision Meta's agent-first comeback under a real Apache license: strong tool use, screen understanding, and a speculative-decoding sidecar for speed. Knowledge is its weak leg — it's built to act, not to recall.
• Nanbeige4.2 3B A looped transformer: 22 layers run twice, so 3B of parameters do the work of a much deeper model. Reported to beat Qwen3.5 9B and Gemma 4 12B on agentic coding at a third of the size. The loop is not free — it doubles the KV cache, which is the largest memory cost here by far.
• Nemotron 3 Nano 30B-A3B Superseded on 2026-08-11 by Nemotron 3.5 Lightning 30B-A3B, which NVIDIA's own table beats it with on every row — SWE-bench Verified 51.6 against 34.1, Terminal-Bench 24.6 against 8.3 — at the same size and a more permissive licence. Kept for its RULER result, still the only million-token measurement on this site: 86.3% at 1M tokens, on a hybrid Mamba-Transformer where only 6 of 52 layers hold a KV cache. The successor publishes no comparable long-context figure, so this sheet is the record of one.
• Nemotron 3 Super 120B-A12B The Super tier of Nemotron 3, scaling the Nano's hybrid Mamba-MoE design to 120B. Only 8 of its 88 layers are attention, so the KV cache costs 8KB per token — a quarter of what a dense model this size would ask, and the reason a 256K context stays affordable on a machine that can hold the weights.
• Nemotron 3.5 Lightning 30B-A3B Replaces Nemotron 3 Nano, which it beats on every row of NVIDIA's own comparison — SWE-bench Verified 51.6 against 34.1, Terminal-Bench 24.6 against 8.3. Same hybrid Mamba-MoE shape and the same cheap cache: 6 of 52 layers hold KV, so 256K of context costs about 1.5GB. Licensed OpenMDW rather than the old Nemotron licence.
• Ornith-1.5 35B-A3B vision The highest SWE-bench Verified score in this catalog — 79.0, from 3B active parameters, under MIT. Trained by having the model invent its own tasks and scaffolds rather than learning from a fixed human-written set. Eleven days old, and every number is DeepReinforce's own.
• Ornith-1.5 9B 70.6 on SWE-bench Verified from a 9B dense model that fits a 16GB machine at 4-bit — a number that would have belonged to a 30B a year ago. Text only, MIT, and only its maker has measured it so far.
• Phi-4-mini Strong math and code for a 3.8B — and the worst measured hallucination rate on this site. A capable calculator that should never be trusted on facts without retrieval.
• Phi-4-reasoning-vision vision Microsoft's selective-reasoning vision model — it decides when to think. llama.cpp has run its vision tower since 2026-03-11, but only with the separate mmproj projector file loaded alongside the weights; a GGUF without one is text-only. No text benchmark scores are recorded yet.
• Phi-4 The exam ace with no life experience: superb STEM reasoning per GB, near-zero world knowledge, and only a 16K window. Excellent at closed problems you paste in; pair with retrieval for everything else.
• Qwen3 14B The sweet spot of the dense Qwen3 line: near-32B quality in a 16GB-friendly footprint, with the family's strong long-context showing (RULER 94.6 non-thinking).
• Qwen3 30B-A3B Superseded by Qwen3.6 35B-A3B, the same 3B-active MoE design at a similar size, two generations on. Kept as the record of the model that made small-active MoE the default answer for fast local inference.
• Qwen3 32B Superseded by Qwen3.8 27B, which covers the same 24–32GB machines with a newer dense design. Kept because it still holds one of the very few independent Aider polyglot scores for a model this size (40.0%).
• Qwen3 4B Superseded by Qwen3.5 4B, the same size one generation on. Kept as the record of what the previous generation scored: with thinking on, it reasoned like models twice its size, and it was the default answer for 8GB laptops.
• Qwen3 8B Superseded by Qwen3.5 9B, which covers the same 8–12GB machines one generation on. Kept as the best-documented small Qwen — the Qwen3 technical report publishes thinking and non-thinking scores for every size — and as the source of this site's one per-quant override.
• Qwen3-Coder-Next 80B-A3B Three billion active parameters out of eighty, and 70.6 on SWE-bench Verified — within striking distance of models that activate ten times as much. The 49GB 4-bit build is the cheapest way onto this site's upper tier, and the sparsity that makes it fast is also what makes it fragile at low quant.
• Qwen3.5 122B-A10B vision The Qwen3.5 that fits: 76GB at 4-bit puts it inside a 96GB machine with room for context, and it gives up surprisingly little to the 397B — 72.0 on SWE-bench against 76.4. The best accuracy-per-gigabyte on this site above 70B.
• Qwen3.5 397B-A17B vision The largest model on this site and the top of the Qwen3.5 line. Gated DeltaNet plus sparse MoE: only 15 of its 60 layers are full attention, so 256K of context costs about 8GB of cache — remarkably cheap for its size. The weights are the problem, not the cache.
• Qwen3.5 4B vision Replaces Qwen3 4B: vision, 256K context, and a cheap cache, since only 8 of its 32 layers hold KV. Two competitors have re-run it and broadly agree with Alibaba's numbers (OpenBMB measured IFBench 59.0 against the vendor's 59.2). It is a thinking model first, and newer small models now beat it on agentic coding: OpenBMB's own run puts its SWE-bench Verified at 33.6.
• Qwen3.5 9B vision The most-downloaded local model of 2026 (12M+/month): vision, 256K context, and vendor claims that beat gpt-oss-20b from 9B dense. Sparse independent data so far — most cells honestly unrated.
• Qwen3.6 27B vision Superseded by Qwen3.8 27B (2026-08-14), the same size and the same hybrid DeltaNet design one generation on. Kept because it is the better-evidenced of the two: four months of independent results, where its successor's numbers are still mostly the vendor's own.
• Qwen3.6 35B-A3B vision Near-flagship agentic coding at 3B-active speed: 73.4 on SWE-bench Verified while activating a twelfth of its weights. The trade is memory, not quality — you hold all 35B in RAM to run 3B worth of compute, and it carries the 3.6 family's tool-calling defect.
• Qwen3.8 27B vision The current Qwen open flagship under 35B: hybrid DeltaNet attention, vision, 256K context, and vendor benchmarks that look absurd — none independently replicated yet. Two weeks old; treat every number as a claim.
• Qwen3.8 Flash-Next vision Alibaba's preview of the Qwen4 architecture. Alibaba's own table gives it SWE-bench Multilingual 81.0 and LiveCodeBench v6 91.9; LiveBench's independent run puts its plain coding nearer the pack, while its agentic coding, math and instruction following hold up well. Only 6B of it computes per token, out of 125B of transformer plus a 51B n-gram embedding table: 111GB at 4-bit. Whether all of that must be held depends on the runtime. llama.cpp on CUDA or CPU now reads the table (about 27 GiB at 4-bit) from disk as needed, but on a Mac current GGUFs still count it against GPU memory because of an open bug. The sizes here assume the Mac case, so on a PC they overstate what must be resident. Twelve of 48 layers cache, so context is the cheap part. Metal also has an open bug that stops decode after one token at long context.
• Spark-X2.5 1.7B The case for this one is narrow. On the four rows both vendors publish (AIME 2026, BFCL v4, SWE-bench Verified, IFBench), it trails MiniCPM5-2B on three and ties IFBench at 66.3. MiniCPM5-2B's Q4_K_M file is only 0.5GB larger. What it has is size: 1.1GB at Q4_K_M, with a full-context cache about a third of MiniCPM5-2B's per token (about 1.9GB at 128K). It is for machines where those gigabytes decide. It is ahead of Gemma 4 E2B on nearly every row of its own table. Quantization costs it more than most models: see the known issues before going below Q8_0, which is 1.8GB.
• Spark-X2.5 4B A 4.1B dense model, 2.6GB at Q4_K_M, with a 1M-token context on paper. SparkLLM's table, all in thinking mode, has it level with MiniCPM5-2B on BFCL v4 (65.1 against 66.6), ahead on IFBench and AIME 2026, and behind on SWE-bench Verified. Those figures come from two vendors' tables, so read the comparison loosely. Three of every four layers use a 512-token sliding window, but the nine full-attention layers still cache about 4.8GB at 128K, which is more than the weights. It also thinks at length: community runs report answers cut off by token budgets, and loops that DRY sampling breaks.
• Xing4.0 29B-A4B Does not run in stock llama.cpp, Ollama or LM Studio yet: support is an open pull request (llama.cpp #29012), so you need the vendor's own llama.cpp build, and the only official GGUF is one IQ4_NL file. On paper it is strong for its size, with SWE-bench Verified 75.0 and AIME 2026 90.0 from 4B active parameters, but those are China Telecom's own thinking-mode numbers and nobody has checked them independently yet.