Which local LLMs can the NVIDIA RTX PRO 6000 Blackwell Max-Q run?
Assumes 32 GB of DDR5-5600 system RAM, Windows with this GPU driving the display, 8k context and an f16 KV cache.
Specs
- VRAM
- 96 GB
- Memory bandwidth
- 1,792.0 GB/s
- FP16 compute
- 110.0 TFLOPS
- Launch year
- 2025
- Also written as
- RTX PRO 6000 Max-Q, RTX PRO 6000 Blackwell Max-Q Workstation Edition
Verdicts at a glance
At Q4_K_M (or the closest available quant) with 8k context.
- Runs great56 modelsRuns well0 modelsRuns slowly0 modelsWon't run3 models2 more models run at a lower quant.
How to read the verdicts
- Runs great
- Fully on the GPU at 20 tok/s or more
- Runs well
- Fully on the GPU at 8–20 tok/s; MoE experts in system RAM at 20 tok/s or more; or at least 90% on the GPU at 8 tok/s or more
- Runs slowly
- 2–8 tok/s; CPU-only; less than 90% on the GPU; or MoE experts in system RAM below 20 tok/s
- Won't run
- Does not fit, or under 2 tok/s
Every model on the NVIDIA RTX PRO 6000 Blackwell Max-Q
Scroll sideways to see every column.
| Model | Verdict | Speed | Quant | Memory | Runs as | Context | Notes |
|---|---|---|---|---|---|---|---|
| MiniCPM5 1B | Runs great | est. 1,407.3 tok/s1,238.5–1,576.2calibrated estimate ±12% | Q4_K_M | 1.9 / 96.0 GB | Full GPU | up to 128k |
|
| Llama 3.2 1B | Runs great | est. 1,163.2 tok/s1,023.6–1,302.7calibrated estimate ±12% | Q4_K_M | 2.2 / 96.0 GB | Full GPU | up to 128k | — |
| EXAONE 4.0 1.2B | Runs great | est. 955.1 tok/s840.5–1,069.8calibrated estimate ±12% | Q4_K_M | 2.5 / 96.0 GB | Full GPU | up to 64k |
|
| LFM2.5 2.6B | Runs great | est. 695.3 tok/s486.7–903.8theoretical estimate ±30% | Q4_K_M | 3.0 / 96.0 GB | Full GPU | up to 128k |
|
| HyperCLOVA X SEED 1.5B | Runs great | est. 691.0 tok/s608.1–773.9calibrated estimate ±12% | Q4_K_M | 3.0 / 96.0 GB | Full GPU | up to 16k | — |
| MiniCPM5 2B | Runs great | est. 656.0 tok/s577.2–734.7calibrated estimate ±12% | Q4_K_M | 2.9 / 96.0 GB | Full GPU | up to 128k |
|
| Ling-3.0 Tiny | Runs great | est. 485.1 tok/s339.6–630.7theoretical estimate ±30% | Q4_K_M | 5.9 / 96.0 GB | Full GPU | up to 128k |
|
| Spark-X2.5 4B | Runs great | est. 432.3 tok/s380.4–484.1calibrated estimate ±12% | Q4_K_M | 4.1 / 96.0 GB | Full GPU | up to 256k |
|
| Llama 3.2 3B | Runs great | est. 423.9 tok/s373.0–474.7calibrated estimate ±12% | Q4_K_M | 4.1 / 96.0 GB | Full GPU | up to 128k | — |
| Qwen3.5 4B | Runs great | est. 404.8 tok/s356.3–453.4calibrated estimate ±12% | Q4_K_M | 4.2 / 96.0 GB | Full GPU | up to 256k |
|
| Gemma 4 E2B | Runs great | est. 386.7 tok/s270.7–502.7theoretical estimate ±30% | Q4_K_M | 2.7 / 96.0 GB | Full GPU | up to 128k |
|
| gpt-oss-20b | Runs great | 373.9 tok/s8k estimate 338.9 tok/s (271.1–406.7, calibrated estimate ±20%)measured (1 run, 512 context) | MXFP4 | 12.9 / 96.0 GB | Full GPU | up to 128k |
|
| Mellum2.1 12B-A2.5B Thinking | Runs great | est. 334.9 tok/s267.9–401.8calibrated estimate ±20% | Q4_K_M | 9.2 / 96.0 GB | Full GPU | up to 128k |
|
| Nemotron 3.5 Lightning 30B-A3B | Runs great | est. 305.7 tok/s214.0–397.4theoretical estimate ±30% | Q4_K_M | 25.7 / 96.0 GB | Full GPU | up to 256k |
|
| Gemma 4 E4B | Runs great | est. 257.7 tok/s180.4–335.0theoretical estimate ±30% | Q4_K_M | 4.3 / 96.0 GB | Full GPU | up to 128k |
|
| MiniCPM-V 4.7 35B-A3B | Runs great | est. 239.5 tok/s191.6–287.4calibrated estimate ±20% | Q4_K_M | 22.2 / 96.0 GB | Full GPU | up to 256k |
|
| gpt-oss-120b | Runs great | est. 237.1 tok/s189.7–284.5calibrated estimate ±20% | MXFP4 | 64.2 / 96.0 GB | Full GPU | up to 128k |
|
| Kanana 1.5 15.7B-A3B | Runs great | est. 233.4 tok/s186.7–280.1calibrated estimate ±20% | Q4_K_M | 12.5 / 96.0 GB | Full GPU | up to 32k | — |
| Qwen3.6 35B-A3B | Runs great | est. 229.8 tok/s183.9–275.8calibrated estimate ±20% | Q4_K_M | 22.4 / 96.0 GB | Full GPU | up to 256k |
|
| Qwen3.5 35B-A3B | Runs great | est. 213.4 tok/s170.7–256.0calibrated estimate ±20% | Q4_K_M | 22.9 / 96.0 GB | Full GPU | up to 256k |
|
| Gemma 4 26B-A4B | Runs great | est. 210.4 tok/s168.3–252.4calibrated estimate ±20% | Q4_K_M | 18.3 / 96.0 GB | Full GPU | up to 256k |
|
| DeepSeek R1 Distill Llama 8B | Runs great | est. 209.3 tok/s184.2–234.4calibrated estimate ±12% | Q4_K_M | 6.9 / 96.0 GB | Full GPU | up to 128k |
|
| Kanana 1.5 8B | Runs great | est. 209.3 tok/s184.2–234.4calibrated estimate ±12% | Q4_K_M | 6.9 / 96.0 GB | Full GPU | up to 32k | — |
| Llama 3.1 8B | Runs great | est. 209.3 tok/s184.2–234.4calibrated estimate ±12% | Q4_K_M | 6.9 / 96.0 GB | Full GPU | up to 128k | — |
| Nanbeige4.2 3B | Runs great | est. 204.8 tok/s143.3–266.2theoretical estimate ±30% | Q4_K_M | 5.0 / 96.0 GB | Full GPU | up to 128k |
|
| Qwen3.5 9B | Runs great | est. 204.4 tok/s179.8–228.9calibrated estimate ±12% | Q4_K_M | 6.6 / 96.0 GB | Full GPU | up to 256k |
|
| DeepSeek R1 0528 Qwen3 8B | Runs great | est. 201.1 tok/s177.0–225.2calibrated estimate ±12% | Q4_K_M | 7.1 / 96.0 GB | Full GPU | up to 128k |
|
| Qwen3 8B | Runs great | est. 201.1 tok/s177.0–225.2calibrated estimate ±12% | Q4_K_M | 7.1 / 96.0 GB | Full GPU | up to 32k |
|
| Kolibri 1 | Runs great | est. 197.0 tok/s157.6–236.4calibrated estimate ±20% | Q4_K_M | 50.8 / 96.0 GB | Full GPU | up to 256k |
|
| Qwen3 30B-A3B (2507) | Runs great | est. 195.0 tok/s156.0–234.1calibrated estimate ±20% | Q4_K_M | 20.4 / 96.0 GB | Full GPU | up to 256k | — |
| Ling-3.0 Flash | Runs great | est. 193.8 tok/s135.7–252.0theoretical estimate ±30% | Q4_K_M | 76.0 / 96.0 GB | Full GPU | up to 256k |
|
| K2-Horizon 7B | Runs great | est. 184.5 tok/s162.4–206.7calibrated estimate ±12% | Q4_K_M | 7.4 / 96.0 GB | Full GPU | up to 128k |
|
| Gemma 4 12B | Runs great | est. 172.9 tok/s152.2–193.7calibrated estimate ±12% | Q4_K_M | 8.4 / 96.0 GB | Full GPU | up to 256k |
|
| Gemma 3 12B | Runs great | est. 160.1 tok/s140.9–179.3calibrated estimate ±12% | Q4_K_M | 9.0 / 96.0 GB | Full GPU | up to 128k | — |
| K2-Horizon MoVA 36B-A4B | Runs great | est. 149.6 tok/s104.7–194.5theoretical estimate ±30% | Q4_K_M | 24.8 / 96.0 GB | Full GPU | up to 128k |
|
| HyperCLOVA X SEED Think 14B | Runs great | est. 123.0 tok/s86.1–160.0theoretical estimate ±30% | Q4_K_M | 11.0 / 96.0 GB | Full GPU | up to 128k |
|
| Qwen3 14B | Runs great | est. 121.3 tok/s106.7–135.8calibrated estimate ±12% | Q4_K_M | 11.1 / 96.0 GB | Full GPU | up to 32k |
|
| DeepSeek R1 Distill Qwen 14B | Runs great | est. 118.3 tok/s104.1–132.5calibrated estimate ±12% | Q4_K_M | 11.3 / 96.0 GB | Full GPU | up to 128k |
|
| Phi-4 | Runs great | est. 116.9 tok/s102.9–131.0calibrated estimate ±12% | Q4_K_M | 11.6 / 96.0 GB | Full GPU | up to 16k | — |
| Laguna S 2.1 | Runs great | est. 112.6 tok/s90.1–135.1calibrated estimate ±20% | Q4_K_M | 69.1 / 96.0 GB | Full GPU | up to 256k |
|
| Qwen3.5 122B-A10B | Runs great | est. 106.3 tok/s85.0–127.5calibrated estimate ±20% | Q4_K_M | 77.2 / 96.0 GB | Full GPU | up to 256k |
|
| Solar Open 100B | Runs great | est. 102.3 tok/s81.9–122.8calibrated estimate ±20% | Q4_K_M | 64.6 / 96.0 GB | Full GPU | up to 128k |
|
| Mistral Small 3.2 24B | Runs great | est. 80.0 tok/s70.4–89.6calibrated estimate ±12% | Q4_K_M | 16.5 / 96.0 GB | Full GPU | up to 128k | — |
| Gemma 3 27B | Runs great | est. 72.8 tok/s64.1–81.6calibrated estimate ±12% | Q4_K_M | 18.4 / 96.0 GB | Full GPU | up to 128k | — |
| Qwen3.6 27B | Runs great | est. 72.3 tok/s63.6–80.9calibrated estimate ±12% | Q4_K_M | 17.8 / 96.0 GB | Full GPU | up to 256k |
|
| Muse Glimmer 30B | Runs great | est. 72.0 tok/s63.4–80.7calibrated estimate ±12% | Q4_K_M | 17.8 / 96.0 GB | Full GPU | up to 128k |
|
| Qwen3.5 27B | Runs great | est. 71.1 tok/s62.6–79.6calibrated estimate ±12% | Q4_K_M | 17.8 / 96.0 GB | Full GPU | up to 256k |
|
| Qwen3.8 27B | Runs great | est. 69.8 tok/s61.4–78.2calibrated estimate ±12% | Q4_K_M | 18.2 / 96.0 GB | Full GPU | up to 256k |
|
| Gemma 4 31B | Runs great | est. 66.1 tok/s58.1–74.0calibrated estimate ±12% | Q4_K_M | 20.1 / 96.0 GB | Full GPU | up to 256k |
|
| Granite 4.2 30B | Runs great | est. 63.1 tok/s55.6–70.7calibrated estimate ±12% | Q4_K_M | 20.8 / 96.0 GB | Full GPU | up to 64k |
|
| EXAONE 4.0 32B | Runs great | est. 63.1 tok/s55.5–70.7calibrated estimate ±12% | Q4_K_M | 20.8 / 96.0 GB | Full GPU | up to 128k |
|
| EXAONE 4.5 33B | Runs great | est. 60.9 tok/s53.6–68.2calibrated estimate ±12% | Q4_K_M | 21.0 / 96.0 GB | Full GPU | up to 256k |
|
| Qwen3 32B | Runs great | est. 57.3 tok/s50.4–64.1calibrated estimate ±12% | Q4_K_M | 22.6 / 96.0 GB | Full GPU | up to 32k |
|
| DeepSeek R1 Distill Qwen 32B | Runs great | est. 57.0 tok/s50.2–63.9calibrated estimate ±12% | Q4_K_M | 22.7 / 96.0 GB | Full GPU | up to 64k |
|
| Nanbeige4.3 33B | Runs great | est. 55.7 tok/s49.0–62.3calibrated estimate ±12% | Q4_K_M | 23.0 / 96.0 GB | Full GPU | up to 64k |
|
| Llama 3.3 70B | Runs great | est. 27.7 tok/s24.4–31.1calibrated estimate ±12% | Q4_K_M | 45.8 / 96.0 GB | Full GPU | up to 32k | — |
| Solar Open 2 250B | Won't runTry Q2_K (heavy quality loss): Runs well | est. 83.0 tok/s58.1–107.9theoretical estimate ±30% | Q2_K | 92.7 / 96.0 GB + 4.4 GB RAM | MoE experts in RAM | up to 256k |
|
| DeepSeek V4 Flash 0731 | Won't runTry Q2_K (heavy quality loss): Runs well | est. 44.2 tok/s30.9–57.4theoretical estimate ±30% | Q2_K | 92.5 / 96.0 GB + 26.1 GB RAM | MoE experts in RAM | up to 64k |
|
| GLM-5.3 Flash | Won't run | — | Q4_K_M | needs 181.4 GB | — | — |
|
Models with a thinking mode spend extra tokens thinking, so their speed thresholds are 1.5× stricter (30 / 12 / 3 tok/s). With thinking turned off, a model may do one grade better.
Measured results on the NVIDIA RTX PRO 6000 Blackwell Max-Q
Public benchmarks we calibrate against. Their conditions (context, backend, flags) can differ from the estimates above.
| Model | Quant | Backend | Context | Prompt (tok/s) | Generation (tok/s) | Flags | Source | Measured |
|---|---|---|---|---|---|---|---|---|
| llama-2-7b | Q4_0 | llama.cpp | 512 | 12,242.0 | 271.3 | — | github.com | 2026-05-08 |
| gpt-oss-20b | MXFP4 | llama.cpp | 512 | 17,385.0 | 373.9 | CUDA, -fa 1 | github.com | 2026-06-24 |
Frequently asked questions
What is the largest model that runs entirely on the NVIDIA RTX PRO 6000 Blackwell Max-Q?
Ling-3.0 Flash at Q4_K_M (a 77.01 GB file) fits entirely in 96 GB with 8k context, at est. 193.8 tok/s (135.7–252.0, theoretical estimate ±30%).
How many local LLMs run well on the NVIDIA RTX PRO 6000 Blackwell Max-Q?
At Q4_K_M with 8k context, out of 59 tracked models: 56 run great, 0 run well, 0 run slowly and 3 won't run.
Can the NVIDIA RTX PRO 6000 Blackwell Max-Q run a 70B model like Llama 3.3 70B?
Runs great — Full GPU, Q4_K_M: est. 27.7 tok/s (24.4–31.1, calibrated estimate ±12%).
Speeds are estimates from memory bandwidth, calibrated against public benchmarks, and each one comes with an error band and a confidence label. Real results vary with drivers, backend, context length and thermals.