Which local LLMs can the Apple M2 Ultra run?
Unified memory — by default the GPU can use about 70% of it
Assumes 8k context and an f16 KV cache. Verdicts are shown for each memory size.
Specs
- Memory options
- 64 GB · 128 GB · 192 GB
- Memory bandwidth
- 800.0 GB/s
- FP16 compute
- 54.0 TFLOPS
- Launch year
- 2023
Verdicts at a glance
At Q4_K_M (or the closest available quant) with 8k context.
- 64 GBRuns great19 modelsRuns well5 modelsRuns slowly1 modelWon't run4 models2 more models run at a lower quant.
- 128 GBRuns great22 modelsRuns well5 modelsRuns slowly1 modelWon't run1 model1 more model runs at a lower quant.
- 192 GBRuns great22 modelsRuns well5 modelsRuns slowly2 modelsWon't run0 models
How to read the verdicts
- Runs great
- Fully on the GPU at 20 tok/s or more
- Runs well
- Fully on the GPU at 8–20 tok/s; MoE experts in system RAM at 20 tok/s or more; or at least 90% on the GPU at 8 tok/s or more
- Runs slowly
- 2–8 tok/s; CPU-only; less than 90% on the GPU; or MoE experts in system RAM below 20 tok/s
- Won't run
- Does not fit, or under 2 tok/s
Every model on the Apple M2 Ultra
Scroll sideways to see every column.
| Model | Verdict | Speed | Quant | Memory | Runs as | Context | Notes |
|---|---|---|---|---|---|---|---|
| EXAONE 4.0 1.2B | Runs great | est. 268.0 tok/s214.4–321.6calibrated estimate ±20% | Q4_K_M | 1.9 / 134.4 GB | Unified memory | up to 64k | — |
| HyperCLOVA X SEED 1.5B | Runs great | est. 193.9 tok/s155.1–232.7calibrated estimate ±20% | Q4_K_M | 2.4 / 134.4 GB | Unified memory | up to 16k | — |
| Qwen3.5 35B-A3B | Runs great | est. 112.8 tok/s90.3–135.4calibrated estimate ±20% | Q4_K_M | 23.4 / 134.4 GB | Unified memory | up to 256k | — |
| gpt-oss-20b | Runs great | est. 101.4 tok/s81.1–121.7calibrated estimate ±20% | MXFP4 | 12.9 / 134.4 GB | Unified memory | up to 128k |
|
| Qwen3 30B-A3B (2507) | Runs great | est. 82.5 tok/s66.0–99.0calibrated estimate ±20% | Q4_K_M | 19.9 / 134.4 GB | Unified memory | up to 64k | — |
| Gemma 4 26B-A4B | Runs great | est. 80.3 tok/s64.2–96.3calibrated estimate ±20% | Q4_K_M | 17.9 / 134.4 GB | Unified memory | up to 128k | — |
| gpt-oss-120b | Runs great | 79.7 tok/s8k estimate 75.6 tok/s (60.5–90.7, calibrated estimate ±20%)measured (1 run, 2k context) | MXFP4 | 64.3 / 134.4 GB | Unified memory | up to 128k |
|
| Kanana 1.5 15.7B-A3B | Runs great | est. 75.6 tok/s60.5–90.7calibrated estimate ±20% | Q4_K_M | 12.1 / 134.4 GB | Unified memory | up to 32k | — |
| DeepSeek R1 Distill Llama 8B | Runs great | est. 58.7 tok/s47.0–70.5calibrated estimate ±20% | Q4_K_M | 6.6 / 134.4 GB | Unified memory | up to 32k |
|
| Kanana 1.5 8B | Runs great | est. 58.7 tok/s47.0–70.5calibrated estimate ±20% | Q4_K_M | 6.6 / 134.4 GB | Unified memory | up to 32k | — |
| Llama 3.1 8B | Runs great | est. 58.7 tok/s47.0–70.5calibrated estimate ±20% | Q4_K_M | 6.6 / 134.4 GB | Unified memory | up to 64k | — |
| Qwen3.5 9B | Runs great | est. 57.3 tok/s45.9–68.8calibrated estimate ±20% | Q4_K_M | 6.7 / 134.4 GB | Unified memory | up to 256k | — |
| Qwen3 8B | Runs great | est. 56.4 tok/s45.1–67.7calibrated estimate ±20% | Q4_K_M | 6.8 / 134.4 GB | Unified memory | up to 32k | — |
| Gemma 4 12B | Runs great | est. 46.0 tok/s36.8–55.2calibrated estimate ±20% | Q4_K_M | 8.2 / 134.4 GB | Unified memory | up to 128k | — |
| Gemma 3 12B | Runs great | est. 44.9 tok/s35.9–53.9calibrated estimate ±20% | Q4_K_M | 8.4 / 134.4 GB | Unified memory | up to 128k | — |
| Qwen3.5 122B-A10B | Runs great | est. 35.9 tok/s28.7–43.1calibrated estimate ±20% | Q4_K_M | 79.0 / 134.4 GB | Unified memory | up to 128k | — |
| HyperCLOVA X SEED Think 14B | Runs great | est. 34.5 tok/s24.2–44.9theoretical estimate ±30% | Q4_K_M | 10.8 / 134.4 GB | Unified memory | up to 16k |
|
| Qwen3 14B | Runs great | est. 34.0 tok/s27.2–40.8calibrated estimate ±20% | Q4_K_M | 10.9 / 134.4 GB | Unified memory | up to 32k | — |
| Phi-4 | Runs great | est. 32.8 tok/s26.2–39.4calibrated estimate ±20% | Q4_K_M | 11.3 / 134.4 GB | Unified memory | up to 8k |
|
| Solar Open 100B | Runs great | est. 26.1 tok/s20.9–31.3calibrated estimate ±20% | Q4_K_M | 64.4 / 134.4 GB | Unified memory | up to 16k | — |
| Mistral Small 3.2 24B | Runs great | est. 22.5 tok/s18.0–27.0calibrated estimate ±20% | Q4_K_M | 16.2 / 134.4 GB | Unified memory | up to 16k | — |
| Gemma 3 27B | Runs great | est. 20.4 tok/s16.4–24.5calibrated estimate ±20% | Q4_K_M | 17.8 / 134.4 GB | Unified memory | up to 8k | — |
| Qwen3.5 27B | Runs well | est. 19.9 tok/s16.0–23.9calibrated estimate ±20% | Q4_K_M | 18.2 / 134.4 GB | Unified memory | up to 256k |
|
| EXAONE 4.0 32B | Runs well | est. 17.7 tok/s14.2–21.3calibrated estimate ±20% | Q4_K_M | 20.5 / 134.4 GB | Unified memory | up to 128k |
|
| EXAONE 4.5 33B | Runs well | est. 17.1 tok/s13.7–20.5calibrated estimate ±20% | Q4_K_M | 21.2 / 134.4 GB | Unified memory | up to 128k |
|
| Qwen3 32B | Runs well | est. 16.1 tok/s12.9–19.3calibrated estimate ±20% | Q4_K_M | 22.5 / 134.4 GB | Unified memory | up to 32k |
|
| DeepSeek R1 Distill Qwen 32B | Runs well | est. 16.0 tok/s12.8–19.2calibrated estimate ±20% | Q4_K_M | 22.6 / 134.4 GB | Unified memory | up to 32k |
|
| Solar Open 2 250B | Runs slowly | est. 27.1 tok/s19.0–35.2theoretical estimate ±30% | IQ4_XS | 136.6 GB RAM | CPU only | up to 256k |
|
| Llama 3.3 70B | Runs slowly | est. 7.8 tok/s6.2–9.3calibrated estimate ±20% | Q4_K_M | 45.8 / 134.4 GB | Unified memory | up to 128k |
|
Reasoning models spend extra tokens thinking, so their speed thresholds are 1.5× stricter (30 / 12 / 3 tok/s).
Verdict by memory size
| Model | 64 GB | 128 GB | 192 GB |
|---|---|---|---|
| EXAONE 4.0 1.2B | Runs great | Runs great | Runs great |
| HyperCLOVA X SEED 1.5B | Runs great | Runs great | Runs great |
| Qwen3.5 35B-A3B | Runs great | Runs great | Runs great |
| gpt-oss-20b | Runs great | Runs great | Runs great |
| Qwen3 30B-A3B (2507) | Runs great | Runs great | Runs great |
| Gemma 4 26B-A4B | Runs great | Runs great | Runs great |
| gpt-oss-120b | Won't run | Runs great | Runs great |
| Kanana 1.5 15.7B-A3B | Runs great | Runs great | Runs great |
| DeepSeek R1 Distill Llama 8B | Runs great | Runs great | Runs great |
| Kanana 1.5 8B | Runs great | Runs great | Runs great |
| Llama 3.1 8B | Runs great | Runs great | Runs great |
| Qwen3.5 9B | Runs great | Runs great | Runs great |
| Qwen3 8B | Runs great | Runs great | Runs great |
| Gemma 4 12B | Runs great | Runs great | Runs great |
| Gemma 3 12B | Runs great | Runs great | Runs great |
| Qwen3.5 122B-A10B | Won't run | Runs great | Runs great |
| HyperCLOVA X SEED Think 14B | Runs great | Runs great | Runs great |
| Qwen3 14B | Runs great | Runs great | Runs great |
| Phi-4 | Runs great | Runs great | Runs great |
| Solar Open 100B | Won't run | Runs great | Runs great |
| Mistral Small 3.2 24B | Runs great | Runs great | Runs great |
| Gemma 3 27B | Runs great | Runs great | Runs great |
| Qwen3.5 27B | Runs well | Runs well | Runs well |
| EXAONE 4.0 32B | Runs well | Runs well | Runs well |
| EXAONE 4.5 33B | Runs well | Runs well | Runs well |
| Qwen3 32B | Runs well | Runs well | Runs well |
| DeepSeek R1 Distill Qwen 32B | Runs well | Runs well | Runs well |
| Solar Open 2 250B | Won't run | Won't run | Runs slowly |
| Llama 3.3 70B | Runs slowly | Runs slowly | Runs slowly |
Measured results on the Apple M2 Ultra
Public benchmarks we calibrate against. Their conditions (context, backend, flags) can differ from the estimates above.
| Model | Quant | Backend | Context | Prompt (tok/s) | Generation (tok/s) | Flags | Source | Measured |
|---|---|---|---|---|---|---|---|---|
| gpt-oss-120b | MXFP4 | llama.cpp | 2k | 1,245.0 | 79.7 | — | github.com | 2025-08-15 |
| llama-2-7b | Q4_0 | llama.cpp | 512 | 1,238.0 | 94.3 | Metal | github.com | 2023-11-15 |
Frequently asked questions
What is the largest model that runs entirely on the Apple M2 Ultra?
Qwen3.5 122B-A10B at Q4_K_M (a 78.26 GB file) fits entirely in 192 GB of unified memory with 8k context, at est. 35.9 tok/s (28.7–43.1, calibrated estimate ±20%).
How many local LLMs run well on the Apple M2 Ultra?
At Q4_K_M with 8k context, with 192 GB of memory, out of 29 tracked models: 22 run great, 5 run well, 2 run slowly and 0 won't run.
Can the Apple M2 Ultra run a 70B model like Llama 3.3 70B?
Runs slowly — Unified memory, Q4_K_M: est. 7.8 tok/s (6.2–9.3, calibrated estimate ±20%).
Speeds are estimates from memory bandwidth, calibrated against public benchmarks, and each one comes with an error band and a confidence label. Real results vary with drivers, backend, context length and thermals.