Which local LLMs can the NVIDIA RTX PRO 6000 Blackwell run?
Assumes 32 GB of DDR5-5600 system RAM, Windows with this GPU driving the display, 8k context and an f16 KV cache.
Specs
- VRAM
- 96 GB
- Memory bandwidth
- 1,792.0 GB/s
- FP16 compute
- 126.0 TFLOPS
- Launch year
- 2025
- Also written as
- RTX PRO 6000, RTX PRO 6000 Blackwell Workstation Edition
Verdicts at a glance
At Q4_K_M (or the closest available quant) with 8k context.
- Runs great56 modelsRuns well0 modelsRuns slowly0 modelsWon't run3 models2 more models run at a lower quant.
How to read the verdicts
- Runs great
- Fully on the GPU at 20 tok/s or more
- Runs well
- Fully on the GPU at 8–20 tok/s; MoE experts in system RAM at 20 tok/s or more; or at least 90% on the GPU at 8 tok/s or more
- Runs slowly
- 2–8 tok/s; CPU-only; less than 90% on the GPU; or MoE experts in system RAM below 20 tok/s
- Won't run
- Does not fit, or under 2 tok/s
Every model on the NVIDIA RTX PRO 6000 Blackwell
Scroll sideways to see every column.
| Model | Verdict | Speed | Quant | Memory | Runs as | Context | Notes |
|---|---|---|---|---|---|---|---|
| MiniCPM5 1B | Runs great | est. 1,407.3 tok/s1,238.5–1,576.2calibrated estimate ±12% | Q4_K_M | 1.9 / 96.0 GB | Full GPU | up to 128k |
|
| Llama 3.2 1B | Runs great | est. 1,163.2 tok/s1,023.6–1,302.7calibrated estimate ±12% | Q4_K_M | 2.2 / 96.0 GB | Full GPU | up to 128k | — |
| EXAONE 4.0 1.2B | Runs great | est. 955.1 tok/s840.5–1,069.8calibrated estimate ±12% | Q4_K_M | 2.5 / 96.0 GB | Full GPU | up to 64k |
|
| LFM2.5 2.6B | Runs great | est. 695.3 tok/s486.7–903.8theoretical estimate ±30% | Q4_K_M | 3.0 / 96.0 GB | Full GPU | up to 128k |
|
| HyperCLOVA X SEED 1.5B | Runs great | est. 691.0 tok/s608.1–773.9calibrated estimate ±12% | Q4_K_M | 3.0 / 96.0 GB | Full GPU | up to 16k | — |
| MiniCPM5 2B | Runs great | est. 656.0 tok/s577.2–734.7calibrated estimate ±12% | Q4_K_M | 2.9 / 96.0 GB | Full GPU | up to 128k |
|
| Ling-3.0 Tiny | Runs great | est. 485.1 tok/s339.6–630.7theoretical estimate ±30% | Q4_K_M | 5.9 / 96.0 GB | Full GPU | up to 128k |
|
| Spark-X2.5 4B | Runs great | est. 432.3 tok/s380.4–484.1calibrated estimate ±12% | Q4_K_M | 4.1 / 96.0 GB | Full GPU | up to 256k |
|
| Llama 3.2 3B | Runs great | est. 423.9 tok/s373.0–474.7calibrated estimate ±12% | Q4_K_M | 4.1 / 96.0 GB | Full GPU | up to 128k | — |
| Qwen3.5 4B | Runs great | est. 404.8 tok/s356.3–453.4calibrated estimate ±12% | Q4_K_M | 4.2 / 96.0 GB | Full GPU | up to 256k |
|
| Gemma 4 E2B | Runs great | est. 386.7 tok/s270.7–502.7theoretical estimate ±30% | Q4_K_M | 2.7 / 96.0 GB | Full GPU | up to 128k |
|
| gpt-oss-20b | Runs great | est. 338.9 tok/s271.1–406.7calibrated estimate ±20% | MXFP4 | 12.9 / 96.0 GB | Full GPU | up to 128k |
|
| Mellum2.1 12B-A2.5B Thinking | Runs great | est. 334.9 tok/s267.9–401.8calibrated estimate ±20% | Q4_K_M | 9.2 / 96.0 GB | Full GPU | up to 128k |
|
| Nemotron 3.5 Lightning 30B-A3B | Runs great | est. 305.7 tok/s214.0–397.4theoretical estimate ±30% | Q4_K_M | 25.7 / 96.0 GB | Full GPU | up to 256k |
|
| Gemma 4 E4B | Runs great | est. 257.7 tok/s180.4–335.0theoretical estimate ±30% | Q4_K_M | 4.3 / 96.0 GB | Full GPU | up to 128k |
|
| MiniCPM-V 4.7 35B-A3B | Runs great | est. 239.5 tok/s191.6–287.4calibrated estimate ±20% | Q4_K_M | 22.2 / 96.0 GB | Full GPU | up to 256k |
|
| gpt-oss-120b | Runs great | est. 237.1 tok/s189.7–284.5calibrated estimate ±20% | MXFP4 | 64.2 / 96.0 GB | Full GPU | up to 128k |
|
| Kanana 1.5 15.7B-A3B | Runs great | est. 233.4 tok/s186.7–280.1calibrated estimate ±20% | Q4_K_M | 12.5 / 96.0 GB | Full GPU | up to 32k | — |
| Qwen3.6 35B-A3B | Runs great | est. 229.8 tok/s183.9–275.8calibrated estimate ±20% | Q4_K_M | 22.4 / 96.0 GB | Full GPU | up to 256k |
|
| Qwen3.5 35B-A3B | Runs great | est. 213.4 tok/s170.7–256.0calibrated estimate ±20% | Q4_K_M | 22.9 / 96.0 GB | Full GPU | up to 256k |
|
| Gemma 4 26B-A4B | Runs great | est. 210.4 tok/s168.3–252.4calibrated estimate ±20% | Q4_K_M | 18.3 / 96.0 GB | Full GPU | up to 256k |
|
| DeepSeek R1 Distill Llama 8B | Runs great | est. 209.3 tok/s184.2–234.4calibrated estimate ±12% | Q4_K_M | 6.9 / 96.0 GB | Full GPU | up to 128k |
|
| Kanana 1.5 8B | Runs great | est. 209.3 tok/s184.2–234.4calibrated estimate ±12% | Q4_K_M | 6.9 / 96.0 GB | Full GPU | up to 32k | — |
| Llama 3.1 8B | Runs great | est. 209.3 tok/s184.2–234.4calibrated estimate ±12% | Q4_K_M | 6.9 / 96.0 GB | Full GPU | up to 128k | — |
| Nanbeige4.2 3B | Runs great | est. 204.8 tok/s143.3–266.2theoretical estimate ±30% | Q4_K_M | 5.0 / 96.0 GB | Full GPU | up to 128k |
|
| Qwen3.5 9B | Runs great | est. 204.4 tok/s179.8–228.9calibrated estimate ±12% | Q4_K_M | 6.6 / 96.0 GB | Full GPU | up to 256k |
|
| DeepSeek R1 0528 Qwen3 8B | Runs great | est. 201.1 tok/s177.0–225.2calibrated estimate ±12% | Q4_K_M | 7.1 / 96.0 GB | Full GPU | up to 128k |
|
| Qwen3 8B | Runs great | est. 201.1 tok/s177.0–225.2calibrated estimate ±12% | Q4_K_M | 7.1 / 96.0 GB | Full GPU | up to 32k |
|
| Kolibri 1 | Runs great | est. 197.0 tok/s157.6–236.4calibrated estimate ±20% | Q4_K_M | 50.8 / 96.0 GB | Full GPU | up to 256k |
|
| Qwen3 30B-A3B (2507) | Runs great | est. 195.0 tok/s156.0–234.1calibrated estimate ±20% | Q4_K_M | 20.4 / 96.0 GB | Full GPU | up to 256k | — |
| Ling-3.0 Flash | Runs great | est. 193.8 tok/s135.7–252.0theoretical estimate ±30% | Q4_K_M | 76.0 / 96.0 GB | Full GPU | up to 256k |
|
| K2-Horizon 7B | Runs great | est. 184.5 tok/s162.4–206.7calibrated estimate ±12% | Q4_K_M | 7.4 / 96.0 GB | Full GPU | up to 128k |
|
| Gemma 4 12B | Runs great | est. 172.9 tok/s152.2–193.7calibrated estimate ±12% | Q4_K_M | 8.4 / 96.0 GB | Full GPU | up to 256k |
|
| Gemma 3 12B | Runs great | est. 160.1 tok/s140.9–179.3calibrated estimate ±12% | Q4_K_M | 9.0 / 96.0 GB | Full GPU | up to 128k | — |
| K2-Horizon MoVA 36B-A4B | Runs great | est. 149.6 tok/s104.7–194.5theoretical estimate ±30% | Q4_K_M | 24.8 / 96.0 GB | Full GPU | up to 128k |
|
| HyperCLOVA X SEED Think 14B | Runs great | est. 123.0 tok/s86.1–160.0theoretical estimate ±30% | Q4_K_M | 11.0 / 96.0 GB | Full GPU | up to 128k |
|
| Qwen3 14B | Runs great | est. 121.3 tok/s106.7–135.8calibrated estimate ±12% | Q4_K_M | 11.1 / 96.0 GB | Full GPU | up to 32k |
|
| DeepSeek R1 Distill Qwen 14B | Runs great | est. 118.3 tok/s104.1–132.5calibrated estimate ±12% | Q4_K_M | 11.3 / 96.0 GB | Full GPU | up to 128k |
|
| Phi-4 | Runs great | est. 116.9 tok/s102.9–131.0calibrated estimate ±12% | Q4_K_M | 11.6 / 96.0 GB | Full GPU | up to 16k | — |
| Laguna S 2.1 | Runs great | est. 112.6 tok/s90.1–135.1calibrated estimate ±20% | Q4_K_M | 69.1 / 96.0 GB | Full GPU | up to 256k |
|
| Qwen3.5 122B-A10B | Runs great | est. 106.3 tok/s85.0–127.5calibrated estimate ±20% | Q4_K_M | 77.2 / 96.0 GB | Full GPU | up to 256k |
|
| Solar Open 100B | Runs great | est. 102.3 tok/s81.9–122.8calibrated estimate ±20% | Q4_K_M | 64.6 / 96.0 GB | Full GPU | up to 128k |
|
| Mistral Small 3.2 24B | Runs great | est. 80.0 tok/s70.4–89.6calibrated estimate ±12% | Q4_K_M | 16.5 / 96.0 GB | Full GPU | up to 128k | — |
| Gemma 3 27B | Runs great | est. 72.8 tok/s64.1–81.6calibrated estimate ±12% | Q4_K_M | 18.4 / 96.0 GB | Full GPU | up to 128k | — |
| Qwen3.6 27B | Runs great | est. 72.3 tok/s63.6–80.9calibrated estimate ±12% | Q4_K_M | 17.8 / 96.0 GB | Full GPU | up to 256k |
|
| Muse Glimmer 30B | Runs great | est. 72.0 tok/s63.4–80.7calibrated estimate ±12% | Q4_K_M | 17.8 / 96.0 GB | Full GPU | up to 128k |
|
| Qwen3.5 27B | Runs great | est. 71.1 tok/s62.6–79.6calibrated estimate ±12% | Q4_K_M | 17.8 / 96.0 GB | Full GPU | up to 256k |
|
| Qwen3.8 27B | Runs great | est. 69.8 tok/s61.4–78.2calibrated estimate ±12% | Q4_K_M | 18.2 / 96.0 GB | Full GPU | up to 256k |
|
| Gemma 4 31B | Runs great | est. 66.1 tok/s58.1–74.0calibrated estimate ±12% | Q4_K_M | 20.1 / 96.0 GB | Full GPU | up to 256k |
|
| Granite 4.2 30B | Runs great | est. 63.1 tok/s55.6–70.7calibrated estimate ±12% | Q4_K_M | 20.8 / 96.0 GB | Full GPU | up to 64k |
|
| EXAONE 4.0 32B | Runs great | est. 63.1 tok/s55.5–70.7calibrated estimate ±12% | Q4_K_M | 20.8 / 96.0 GB | Full GPU | up to 128k |
|
| EXAONE 4.5 33B | Runs great | est. 60.9 tok/s53.6–68.2calibrated estimate ±12% | Q4_K_M | 21.0 / 96.0 GB | Full GPU | up to 256k |
|
| Qwen3 32B | Runs great | est. 57.3 tok/s50.4–64.1calibrated estimate ±12% | Q4_K_M | 22.6 / 96.0 GB | Full GPU | up to 32k |
|
| DeepSeek R1 Distill Qwen 32B | Runs great | est. 57.0 tok/s50.2–63.9calibrated estimate ±12% | Q4_K_M | 22.7 / 96.0 GB | Full GPU | up to 64k |
|
| Nanbeige4.3 33B | Runs great | est. 55.7 tok/s49.0–62.3calibrated estimate ±12% | Q4_K_M | 23.0 / 96.0 GB | Full GPU | up to 64k |
|
| Llama 3.3 70B | Runs great | est. 27.7 tok/s24.4–31.1calibrated estimate ±12% | Q4_K_M | 45.8 / 96.0 GB | Full GPU | up to 32k | — |
| Solar Open 2 250B | Won't runTry Q2_K (heavy quality loss): Runs well | est. 83.0 tok/s58.1–107.9theoretical estimate ±30% | Q2_K | 92.7 / 96.0 GB + 4.4 GB RAM | MoE experts in RAM | up to 256k |
|
| DeepSeek V4 Flash 0731 | Won't runTry Q2_K (heavy quality loss): Runs well | est. 44.2 tok/s30.9–57.4theoretical estimate ±30% | Q2_K | 92.5 / 96.0 GB + 26.1 GB RAM | MoE experts in RAM | up to 64k |
|
| GLM-5.3 Flash | Won't run | — | Q4_K_M | needs 181.4 GB | — | — |
|
Models with a thinking mode spend extra tokens thinking, so their speed thresholds are 1.5× stricter (30 / 12 / 3 tok/s). With thinking turned off, a model may do one grade better.
Measured results on the NVIDIA RTX PRO 6000 Blackwell
Public benchmarks we calibrate against. Their conditions (context, backend, flags) can differ from the estimates above.
| Model | Quant | Backend | Context | Prompt (tok/s) | Generation (tok/s) | Flags | Source | Measured |
|---|---|---|---|---|---|---|---|---|
| llama-2-7b | Q4_0 | llama.cpp | 512 | 14,855.0 | 274.2 | — | github.com | 2025-09-05 |
Frequently asked questions
What is the largest model that runs entirely on the NVIDIA RTX PRO 6000 Blackwell?
Ling-3.0 Flash at Q4_K_M (a 77.01 GB file) fits entirely in 96 GB with 8k context, at est. 193.8 tok/s (135.7–252.0, theoretical estimate ±30%).
How many local LLMs run well on the NVIDIA RTX PRO 6000 Blackwell?
At Q4_K_M with 8k context, out of 59 tracked models: 56 run great, 0 run well, 0 run slowly and 3 won't run.
Can the NVIDIA RTX PRO 6000 Blackwell run a 70B model like Llama 3.3 70B?
Runs great — Full GPU, Q4_K_M: est. 27.7 tok/s (24.4–31.1, calibrated estimate ±12%).
Speeds are estimates from memory bandwidth, calibrated against public benchmarks, and each one comes with an error band and a confidence label. Real results vary with drivers, backend, context length and thermals.