DGX Spark 128GB vs RTX 5090 for local LLMs
- DGX Spark 128GB: Uses its largest memory option, 128 GB, with 8k context and an f16 KV cache.
- RTX 5090: Assumes 32 GB of DDR5-5600 system RAM, Windows with this GPU driving the display, 8k context and an f16 KV cache.
What to know before choosing
These are different kinds of machine. The RTX 5090 is a graphics card with 32GB of VRAM at 1,792 GB/s; this page pairs it with 32GB of system RAM. The DGX Spark here is the 128GB model, whose GPU can use 116.9 GB of unified memory in CanRun's count, at much lower bandwidth. A model that fits in the card's VRAM runs far faster on it: Gemma 4 31B runs great on the RTX 5090 and slowly on the DGX Spark, a public benchmark.
Past 32GB the order flips. gpt-oss-120b runs great on the DGX Spark, also a public benchmark, and does not fit the RTX 5090 with 32GB of system RAM. With 64GB of RAM it runs well there with its experts in RAM, a theoretical estimate that depends on RAM speed. At Q4_K_M, Qwen3.5 122B-A10B runs well on the DGX Spark and slowly on the RTX 5090 with 64GB of RAM, also theoretical. The 2-bit UD-Q2_K_XL file that the table suggests for the RTX 5090 fits even with 32GB of RAM and runs well there, a theoretical estimate, but with heavy quality loss.
On the DGX Spark's 64GB configuration gpt-oss-120b does not fit either. That size and linked units are on the NVIDIA DGX Spark page.
Speed
The RTX 5090 generates text about 5.9× faster than the DGX Spark 128GB (median of our estimates over the 49 models that fit entirely in memory on both at Q4_K_M or the closest available quant).
- Reference model
- K2-Horizon MoVA 36B-A4B at Q4_K_M, the largest model that fits entirely in memory on both at Q4_K_M or the closest available quant
- DGX Spark 128GB
- est. 37.7 tok/s (26.4–49.1, theoretical estimate ±30%)
- RTX 5090
- est. 149.6 tok/s (104.7–194.5, theoretical estimate ±30%)
- Prompt processing on K2-Horizon MoVA 36B-A4B
- DGX Spark 128GB: about 2,496 tok/s (1,497–3,494, rough estimate ±40%); RTX 5090: about 12,393 tok/s (7,436–17,350, rough estimate ±40%)
Specs side by side
| Spec | DGX Spark 128GB | RTX 5090 | Difference |
|---|---|---|---|
| Memory | 128 GB of unified memory | 32 GB | — |
| Memory bandwidth | 273.0 GB/s | 1,792.0 GB/s | 6.56× on the RTX 5090 |
| FP16 compute | 62.5 TFLOPS | 104.8 TFLOPS | 1.68× on the RTX 5090 |
| Launch year | 2025 | 2025 | — |
| Price | $3,999 launch MSRP (128 GB) | $4,200 street (as of 2026-08-09) | — |
| Public benchmark results | 14 results | 15 results | — |
Verdicts at a glance
Out of 59 models at Q4_K_M (or the closest available quant) with 8k context.
- DGX Spark 128GBRuns great34 modelsRuns well12 modelsRuns slowly10 modelsWon't run3 models
- RTX 5090Runs great49 modelsRuns well1 modelRuns slowly1 modelWon't run8 models
How to read the verdicts
- Runs great
- Fully on the GPU at 20 tok/s or more
- Runs well
- Fully on the GPU at 8–20 tok/s; MoE experts in system RAM at 20 tok/s or more; or at least 90% on the GPU at 8 tok/s or more
- Runs slowly
- 2–8 tok/s; CPU-only; less than 90% on the GPU; or MoE experts in system RAM below 20 tok/s
- Won't run
- Does not fit, or under 2 tok/s
Where the verdicts differ
24 models get a different verdict on the two.
Scroll sideways to see every column.
| Model | DGX Spark 128GB verdict | DGX Spark 128GB speed | RTX 5090 verdict | RTX 5090 speed |
|---|---|---|---|---|
| Ling-3.0 FlashQ4_K_M | Runs great | est. 52.8 tok/s36.9–68.6theoretical estimate ±30% | Won't run | — |
| gpt-oss-120bMXFP4 | Runs great | 58.7 tok/s8k estimate 50.8 tok/s (40.6–60.9, calibrated estimate ±20%)measured (1 run, 2k context) | Won't rungpt-oss-120b verdict on the RTX 5090 | — |
| Kolibri 1Q4_K_M | Runs great | est. 52.2 tok/s41.7–62.6calibrated estimate ±20% | Runs well | est. 83.3 tok/s58.3–108.3theoretical estimate ±30% |
| EXAONE 4.5 33BQ4_K_M | Runs slowly | est. 10.3 tok/s8.3–12.4calibrated estimate ±20% | Runs great | est. 60.9 tok/s53.6–68.2calibrated estimate ±12% |
| Nanbeige4.3 33BQ4_K_M | Runs slowly | est. 9.4 tok/s7.6–11.3calibrated estimate ±20% | Runs great | est. 55.7 tok/s49.0–62.3calibrated estimate ±12% |
| DeepSeek R1 Distill Qwen 32BQ4_K_M | Runs slowly | est. 9.7 tok/s7.7–11.6calibrated estimate ±20% | Runs great | est. 57.0 tok/s50.2–63.9calibrated estimate ±12% |
| Qwen3 32BQ4_K_M | Runs slowly | 10.7 tok/s8k estimate 9.7 tok/s (7.8–11.7, calibrated estimate ±20%)measured (2 runs, 512 context) | Runs greatQwen3 32B verdict on the RTX 5090 | 53.3 tok/s8k estimate 57.3 tok/s (50.4–64.1, calibrated estimate ±12%)measured (1 run, 512 context) |
| EXAONE 4.0 32BQ4_K_M | Runs slowly | est. 10.7 tok/s8.6–12.9calibrated estimate ±20% | Runs great | est. 63.1 tok/s55.5–70.7calibrated estimate ±12% |
| Gemma 4 31BQ4_K_M | Runs slowly | 10.1 tok/s8k estimate 11.2 tok/s (9.0–13.5, calibrated estimate ±20%)measured (1 run, 512 context) | Runs greatGemma 4 31B verdict on the RTX 5090 | est. 66.1 tok/s58.1–74.0calibrated estimate ±12% |
| Muse Glimmer 30BQ4_K_M | Runs well | est. 12.2 tok/s9.8–14.7calibrated estimate ±20% | Runs great | est. 72.0 tok/s63.4–80.7calibrated estimate ±12% |
| Granite 4.2 30BQ4_K_M | Runs slowly | est. 10.7 tok/s8.6–12.9calibrated estimate ±20% | Runs great | est. 63.1 tok/s55.6–70.7calibrated estimate ±12% |
| Qwen3.5 27BQ4_K_M | Runs slowly | 11.3 tok/s8k estimate 12.1 tok/s (9.7–14.5, calibrated estimate ±20%)measured (1 run, 4k context) | Runs greatQwen3.5 27B verdict on the RTX 5090 | est. 71.1 tok/s62.6–79.6calibrated estimate ±12% |
| Qwen3.6 27BQ4_K_M | Runs well | est. 12.3 tok/s9.8–14.7calibrated estimate ±20% | Runs great | est. 72.3 tok/s63.6–80.9calibrated estimate ±12% |
| Qwen3.8 27BQ4_K_M | Runs slowly | est. 11.8 tok/s9.5–14.2calibrated estimate ±20% | Runs greatQwen3.8 27B verdict on the RTX 5090 | 81.8 tok/s8k estimate 69.8 tok/s (61.4–78.2, calibrated estimate ±12%)measured (1 run, 1k context) |
| Gemma 3 27BQ4_K_M | Runs well | est. 12.4 tok/s9.9–14.8calibrated estimate ±20% | Runs great | est. 72.8 tok/s64.1–81.6calibrated estimate ±12% |
| Mistral Small 3.2 24BQ4_K_M | Runs well | est. 13.6 tok/s10.9–16.3calibrated estimate ±20% | Runs great | est. 80.0 tok/s70.4–89.6calibrated estimate ±12% |
| Qwen3 14BQ4_K_M | Runs well | est. 20.6 tok/s16.5–24.7calibrated estimate ±20% | Runs greatQwen3 14B verdict on the RTX 5090 | est. 121.3 tok/s106.7–135.8calibrated estimate ±12% |
| DeepSeek R1 Distill Qwen 14BQ4_K_M | Runs well | est. 20.1 tok/s16.1–24.1calibrated estimate ±20% | Runs great | est. 118.3 tok/s104.1–132.5calibrated estimate ±12% |
| HyperCLOVA X SEED Think 14BQ4_K_M | Runs well | est. 20.9 tok/s14.6–27.2theoretical estimate ±30% | Runs great | est. 123.0 tok/s86.1–160.0theoretical estimate ±30% |
| Phi-4Q4_K_M | Runs well | est. 19.8 tok/s15.9–23.8calibrated estimate ±20% | Runs great | est. 116.9 tok/s102.9–131.0calibrated estimate ±12% |
| Gemma 4 12BQ4_K_M | Runs well | est. 29.4 tok/s23.5–35.2calibrated estimate ±20% | Runs greatGemma 4 12B verdict on the RTX 5090 | est. 172.9 tok/s152.2–193.7calibrated estimate ±12% |
| Qwen3.5 122B-A10BQ4_K_M | Runs well | 29.0 tok/s8k estimate 24.0 tok/s (19.2–28.8, calibrated estimate ±20%)measured (1 run, 4k context) | est. 66.4 tok/s46.5–86.3theoretical estimate ±30% | |
| Laguna S 2.1Q4_K_M | Runs well | est. 28.4 tok/s22.7–34.1calibrated estimate ±20% | Won't run | — |
| Solar Open 100BQ4_K_M | Runs well | est. 22.8 tok/s18.3–27.4calibrated estimate ±20% | Won't runTry IQ4_XS: Runs slowly | est. 21.9 tok/s15.3–28.4theoretical estimate ±30% |
Every model on both
Scroll sideways to see every column.
| Model | DGX Spark 128GB verdict | DGX Spark 128GB speed | RTX 5090 verdict | RTX 5090 speed |
|---|---|---|---|---|
| Ling-3.0 FlashQ4_K_M | Runs great | est. 52.8 tok/s36.9–68.6theoretical estimate ±30% | Won't run | — |
| gpt-oss-120bMXFP4 | Runs great | 58.7 tok/s8k estimate 50.8 tok/s (40.6–60.9, calibrated estimate ±20%)measured (1 run, 2k context) | Won't rungpt-oss-120b verdict on the RTX 5090 | — |
| Kolibri 1Q4_K_M | Runs great | est. 52.2 tok/s41.7–62.6calibrated estimate ±20% | Runs well | est. 83.3 tok/s58.3–108.3theoretical estimate ±30% |
| K2-Horizon MoVA 36B-A4BQ4_K_M | Runs great | est. 37.7 tok/s26.4–49.1theoretical estimate ±30% | Runs great | est. 149.6 tok/s104.7–194.5theoretical estimate ±30% |
| Qwen3.5 35B-A3BQ4_K_M | Runs great | est. 61.9 tok/s49.5–74.2calibrated estimate ±20% | Runs greatQwen3.5 35B-A3B verdict on the RTX 5090 | est. 213.4 tok/s170.7–256.0calibrated estimate ±20% |
| Qwen3.6 35B-A3BQ4_K_M | Runs great | est. 70.5 tok/s56.4–84.6calibrated estimate ±20% | Runs greatQwen3.6 35B-A3B verdict on the RTX 5090 | est. 229.8 tok/s183.9–275.8calibrated estimate ±20% |
| MiniCPM-V 4.7 35B-A3BQ4_K_M | Runs great | est. 76.0 tok/s60.8–91.2calibrated estimate ±20% | Runs great | est. 239.5 tok/s191.6–287.4calibrated estimate ±20% |
| EXAONE 4.5 33BQ4_K_M | Runs slowly | est. 10.3 tok/s8.3–12.4calibrated estimate ±20% | Runs great | est. 60.9 tok/s53.6–68.2calibrated estimate ±12% |
| Nanbeige4.3 33BQ4_K_M | Runs slowly | est. 9.4 tok/s7.6–11.3calibrated estimate ±20% | Runs great | est. 55.7 tok/s49.0–62.3calibrated estimate ±12% |
| DeepSeek R1 Distill Qwen 32BQ4_K_M | Runs slowly | est. 9.7 tok/s7.7–11.6calibrated estimate ±20% | Runs great | est. 57.0 tok/s50.2–63.9calibrated estimate ±12% |
| Qwen3 32BQ4_K_M | Runs slowly | 10.7 tok/s8k estimate 9.7 tok/s (7.8–11.7, calibrated estimate ±20%)measured (2 runs, 512 context) | Runs greatQwen3 32B verdict on the RTX 5090 | 53.3 tok/s8k estimate 57.3 tok/s (50.4–64.1, calibrated estimate ±12%)measured (1 run, 512 context) |
| EXAONE 4.0 32BQ4_K_M | Runs slowly | est. 10.7 tok/s8.6–12.9calibrated estimate ±20% | Runs great | est. 63.1 tok/s55.5–70.7calibrated estimate ±12% |
| Nemotron 3.5 Lightning 30B-A3BQ4_K_M | Runs great | est. 69.8 tok/s48.8–90.7theoretical estimate ±30% | Runs great | est. 305.7 tok/s214.0–397.4theoretical estimate ±30% |
| Gemma 4 31BQ4_K_M | Runs slowly | 10.1 tok/s8k estimate 11.2 tok/s (9.0–13.5, calibrated estimate ±20%)measured (1 run, 512 context) | Runs greatGemma 4 31B verdict on the RTX 5090 | est. 66.1 tok/s58.1–74.0calibrated estimate ±12% |
| Qwen3 30B-A3B (2507)Q4_K_M | Runs great | est. 60.7 tok/s48.6–72.9calibrated estimate ±20% | est. 195.0 tok/s156.0–234.1calibrated estimate ±20% | |
| Muse Glimmer 30BQ4_K_M | Runs well | est. 12.2 tok/s9.8–14.7calibrated estimate ±20% | Runs great | est. 72.0 tok/s63.4–80.7calibrated estimate ±12% |
| Granite 4.2 30BQ4_K_M | Runs slowly | est. 10.7 tok/s8.6–12.9calibrated estimate ±20% | Runs great | est. 63.1 tok/s55.6–70.7calibrated estimate ±12% |
| Qwen3.5 27BQ4_K_M | Runs slowly | 11.3 tok/s8k estimate 12.1 tok/s (9.7–14.5, calibrated estimate ±20%)measured (1 run, 4k context) | Runs greatQwen3.5 27B verdict on the RTX 5090 | est. 71.1 tok/s62.6–79.6calibrated estimate ±12% |
| Qwen3.6 27BQ4_K_M | Runs well | est. 12.3 tok/s9.8–14.7calibrated estimate ±20% | Runs great | est. 72.3 tok/s63.6–80.9calibrated estimate ±12% |
| Qwen3.8 27BQ4_K_M | Runs slowly | est. 11.8 tok/s9.5–14.2calibrated estimate ±20% | Runs greatQwen3.8 27B verdict on the RTX 5090 | 81.8 tok/s8k estimate 69.8 tok/s (61.4–78.2, calibrated estimate ±12%)measured (1 run, 1k context) |
| Gemma 3 27BQ4_K_M | Runs well | est. 12.4 tok/s9.9–14.8calibrated estimate ±20% | Runs great | est. 72.8 tok/s64.1–81.6calibrated estimate ±12% |
| Gemma 4 26B-A4BQ4_K_M | Runs great | 47.0 tok/s8k estimate 51.5 tok/s (41.2–61.8, calibrated estimate ±20%)measured (1 run, 32k context) | Runs greatGemma 4 26B-A4B verdict on the RTX 5090 | 216.1 tok/s8k estimate 210.4 tok/s (168.3–252.4, calibrated estimate ±20%)measured (2 runs, 512/8k context) |
| Mistral Small 3.2 24BQ4_K_M | Runs well | est. 13.6 tok/s10.9–16.3calibrated estimate ±20% | Runs great | est. 80.0 tok/s70.4–89.6calibrated estimate ±12% |
| gpt-oss-20bMXFP4 | Runs great | est. 71.7 tok/s57.4–86.0calibrated estimate ±20% | Runs greatgpt-oss-20b verdict on the RTX 5090 | 282.3 tok/s8k estimate 338.9 tok/s (271.1–406.7, calibrated estimate ±20%)measured (1 run, 2k context) |
| Kanana 1.5 15.7B-A3BQ4_K_M | Runs great | est. 62.2 tok/s49.7–74.6calibrated estimate ±20% | Runs great | est. 233.4 tok/s186.7–280.1calibrated estimate ±20% |
| Qwen3 14BQ4_K_M | Runs well | est. 20.6 tok/s16.5–24.7calibrated estimate ±20% | Runs greatQwen3 14B verdict on the RTX 5090 | est. 121.3 tok/s106.7–135.8calibrated estimate ±12% |
| DeepSeek R1 Distill Qwen 14BQ4_K_M | Runs well | est. 20.1 tok/s16.1–24.1calibrated estimate ±20% | Runs great | est. 118.3 tok/s104.1–132.5calibrated estimate ±12% |
| HyperCLOVA X SEED Think 14BQ4_K_M | Runs well | est. 20.9 tok/s14.6–27.2theoretical estimate ±30% | Runs great | est. 123.0 tok/s86.1–160.0theoretical estimate ±30% |
| Phi-4Q4_K_M | Runs well | est. 19.8 tok/s15.9–23.8calibrated estimate ±20% | Runs great | est. 116.9 tok/s102.9–131.0calibrated estimate ±12% |
| Gemma 3 12BQ4_K_M | Runs great | est. 27.2 tok/s21.7–32.6calibrated estimate ±20% | Runs greatGemma 3 12B verdict on the RTX 5090 | est. 160.1 tok/s140.9–179.3calibrated estimate ±12% |
| Mellum2.1 12B-A2.5B ThinkingQ4_K_M | Runs great | est. 104.4 tok/s83.5–125.2calibrated estimate ±20% | Runs great | est. 334.9 tok/s267.9–401.8calibrated estimate ±20% |
| Gemma 4 12BQ4_K_M | Runs well | est. 29.4 tok/s23.5–35.2calibrated estimate ±20% | Runs greatGemma 4 12B verdict on the RTX 5090 | est. 172.9 tok/s152.2–193.7calibrated estimate ±12% |
| Qwen3.5 9BQ4_K_M | Runs great | 35.2 tok/s8k estimate 34.7 tok/s (27.8–41.6, calibrated estimate ±20%)measured (1 run, 4k context) | Runs greatQwen3.5 9B verdict on the RTX 5090 | est. 204.4 tok/s179.8–228.9calibrated estimate ±12% |
| K2-Horizon 7BQ4_K_M | Runs great | est. 31.3 tok/s25.1–37.6calibrated estimate ±20% | Runs great | est. 184.5 tok/s162.4–206.7calibrated estimate ±12% |
| Qwen3 8BQ4_K_M | Runs great | 43.9 tok/s8k estimate 34.1 tok/s (27.3–41.0, calibrated estimate ±20%)measured (1 run, 512 context) | Runs greatQwen3 8B verdict on the RTX 5090 | 234.3 tok/s8k estimate 201.1 tok/s (177.0–225.2, calibrated estimate ±12%)measured (1 run, 512 context) |
| DeepSeek R1 0528 Qwen3 8BQ4_K_M | Runs great | est. 34.1 tok/s27.3–41.0calibrated estimate ±20% | Runs great | est. 201.1 tok/s177.0–225.2calibrated estimate ±12% |
| DeepSeek R1 Distill Llama 8BQ4_K_M | Runs great | est. 35.5 tok/s28.4–42.6calibrated estimate ±20% | Runs great | est. 209.3 tok/s184.2–234.4calibrated estimate ±12% |
| Kanana 1.5 8BQ4_K_M | Runs great | est. 35.5 tok/s28.4–42.6calibrated estimate ±20% | Runs great | est. 209.3 tok/s184.2–234.4calibrated estimate ±12% |
| Llama 3.1 8BQ4_K_M | Runs great | est. 35.5 tok/s28.4–42.6calibrated estimate ±20% | Runs greatLlama 3.1 8B verdict on the RTX 5090 | est. 209.3 tok/s184.2–234.4calibrated estimate ±12% |
| Gemma 4 E4BQ4_K_M | Runs great | est. 61.5 tok/s43.1–80.0theoretical estimate ±30% | Runs great | 234.0 tok/s8k estimate 257.7 tok/s (180.4–335.0, theoretical estimate ±30%)measured (1 run, 512 context) |
| Ling-3.0 TinyQ4_K_M | Runs great | est. 179.8 tok/s125.8–233.7theoretical estimate ±30% | Runs great | est. 485.1 tok/s339.6–630.7theoretical estimate ±30% |
| Gemma 4 E2BQ4_K_M | Runs great | est. 115.9 tok/s81.1–150.6theoretical estimate ±30% | Runs great | 341.8 tok/s8k estimate 386.7 tok/s (270.7–502.7, theoretical estimate ±30%)measured (1 run, 512 context) |
| Qwen3.5 4BQ4_K_M | Runs great | est. 68.7 tok/s55.0–82.5calibrated estimate ±20% | Runs great | est. 404.8 tok/s356.3–453.4calibrated estimate ±12% |
| Nanbeige4.2 3BQ4_K_M | Runs great | est. 34.8 tok/s24.3–45.2theoretical estimate ±30% | Runs great | est. 204.8 tok/s143.3–266.2theoretical estimate ±30% |
| Spark-X2.5 4BQ4_K_M | Runs great | est. 73.4 tok/s58.7–88.1calibrated estimate ±20% | Runs great | est. 432.3 tok/s380.4–484.1calibrated estimate ±12% |
| Llama 3.2 3BQ4_K_M | Runs great | est. 72.0 tok/s57.6–86.3calibrated estimate ±20% | Runs great | est. 423.9 tok/s373.0–474.7calibrated estimate ±12% |
| LFM2.5 2.6BQ4_K_M | Runs great | est. 118.0 tok/s82.6–153.4theoretical estimate ±30% | Runs great | est. 695.3 tok/s486.7–903.8theoretical estimate ±30% |
| MiniCPM5 2BQ4_K_M | Runs great | est. 111.4 tok/s89.1–133.6calibrated estimate ±20% | Runs great | est. 656.0 tok/s577.2–734.7calibrated estimate ±12% |
| HyperCLOVA X SEED 1.5BQ4_K_M | Runs great | est. 117.3 tok/s93.8–140.8calibrated estimate ±20% | Runs great | est. 691.0 tok/s608.1–773.9calibrated estimate ±12% |
| EXAONE 4.0 1.2BQ4_K_M | Runs great | est. 162.1 tok/s129.7–194.6calibrated estimate ±20% | Runs great | est. 955.1 tok/s840.5–1,069.8calibrated estimate ±12% |
| Llama 3.2 1BQ4_K_M | Runs great | est. 197.5 tok/s158.0–236.9calibrated estimate ±20% | Runs great | est. 1,163.2 tok/s1,023.6–1,302.7calibrated estimate ±12% |
| MiniCPM5 1BQ4_K_M | Runs great | est. 238.9 tok/s191.1–286.7calibrated estimate ±20% | Runs great | est. 1,407.3 tok/s1,238.5–1,576.2calibrated estimate ±12% |
| Qwen3.5 122B-A10BQ4_K_M | Runs well | 29.0 tok/s8k estimate 24.0 tok/s (19.2–28.8, calibrated estimate ±20%)measured (1 run, 4k context) | est. 66.4 tok/s46.5–86.3theoretical estimate ±30% | |
| Laguna S 2.1Q4_K_M | Runs well | est. 28.4 tok/s22.7–34.1calibrated estimate ±20% | Won't run | — |
| Solar Open 100BQ4_K_M | Runs well | est. 22.8 tok/s18.3–27.4calibrated estimate ±20% | Won't runTry IQ4_XS: Runs slowly | est. 21.9 tok/s15.3–28.4theoretical estimate ±30% |
| Llama 3.3 70BQ4_K_M | Runs slowly | 4.7 tok/s8k estimate 4.7 tok/s (3.8–5.7, calibrated estimate ±20%)measured (2 runs, 512 context) | Runs slowlyLlama 3.3 70B verdict on the RTX 5090 | est. 2.7 tok/s2.2–3.3calibrated estimate ±20% |
| GLM-5.3 FlashQ4_K_M | Won't run | — | Won't run | — |
| DeepSeek V4 Flash 0731MXFP4 | Won't run | — | Won't run | — |
| Solar Open 2 250BIQ4_XS | Won't runTry Q2_K (heavy quality loss): Runs well | est. 22.0 tok/s17.6–26.3calibrated estimate ±20% | Won't run | — |
Models with a thinking mode spend extra tokens thinking, so their speed thresholds are 1.5× stricter (30 / 12 / 3 tok/s). With thinking turned off, a model may do one grade better.
Public benchmarks
Measured results on the DGX Spark 128GB
| Model | Quant | Backend | Context | Prompt (tok/s) | Generation (tok/s) | Flags | Source | Measured |
|---|---|---|---|---|---|---|---|---|
| gpt-oss-120b | MXFP4 | llama.cpp | 2k | 2,444.0 | 58.7 | CUDA, no mmap | github.com | 2026-02-05 |
| Gemma 4 31B | Q4_K_M | llama.cpp | 512 | 687.0 | 10.1 | CUDA | github.com | 2026-04-14 |
| Gemma 4 26B-A4B | Q4_K_M | llama.cpp | 32k | — | 47.0 | CUDA, depth 32768 | github.com | 2026-06-11 |
| Qwen3.5 35B-A3B | Q8_0 | llama.cpp | 2k | 2,572.0 | 49.2 | CUDA | github.com | 2026-03-12 |
| Qwen3.5 9B | Q4_K_M | llama.cpp | 4k | — | 35.2 | — | github.com | 2026-08-13 |
| Qwen3.5 27B | Q4_K_M | llama.cpp | 4k | — | 11.3 | — | github.com | 2026-08-13 |
| Qwen3.5 122B-A10B | Q4_K_M | llama.cpp | 4k | — | 29.0 | — | github.com | 2026-08-13 |
| Llama 3.3 70B | Q4_K_M | llama.cpp | 512 | 284.0 | 4.7 | CUDA, Vulkan | github.com | 2025-11-22 |
| Llama 3.3 70B | Q4_K_M | llama.cpp | 512 | — | 4.6 | CUDA | github.com | 2026-08-12 |
| Llama 3.3 70B | Q8_0 | llama.cpp | 512 | — | 2.9 | CUDA | github.com | 2026-08-12 |
| Llama 3.3 70B | Q2_K | llama.cpp | 512 | — | 7.2 | CUDA | github.com | 2026-08-12 |
| Qwen3 32B | Q4_K_M | llama.cpp | 512 | 630.0 | 10.5 | CUDA, Vulkan | github.com | 2025-11-22 |
| Qwen3 32B | Q4_K_M | llama.cpp | 512 | 762.0 | 10.8 | CUDA | github.com | 2025-12-05 |
| Qwen3 8B | Q4_K_M | llama.cpp | 512 | 3,168.0 | 43.9 | CUDA | github.com | 2025-12-05 |
Measured results on the RTX 5090
| Model | Quant | Backend | Context | Prompt (tok/s) | Generation (tok/s) | Flags | Source | Measured |
|---|---|---|---|---|---|---|---|---|
| llama-2-7b | Q4_0 | llama.cpp | 512 | 14,073.0 | 290.0 | — | github.com | 2025-10-23 |
| gpt-oss-20b | MXFP4 | llama.cpp | 2k | 9,841.0 | 282.3 | — | github.com | 2025-08-20 |
| Qwen3 8B | Q4_K_XL | llama.cpp | 16k | — | 145.3 | — | hardware-corner.net | 2026-08-09 |
| Qwen3 30B-A3B (2507) | Q4_K_XL | llama.cpp | 16k | — | 141.6 | — | hardware-corner.net | 2026-08-09 |
| gpt-oss-120b | MXFP4 | llama.cpp | 4k | — | 9.6 | --n-cpu-moe 21, DDR4 server | hardware-corner.net | 2026-08-09 |
| Gemma 4 26B-A4B | Q4_K_M | llama.cpp | 512 | 13,588.0 | 223.8 | CUDA, KV q8_0 | github.com | 2026-05-24 |
| Gemma 4 26B-A4B | Q4_K_M | ollama | 8k | — | 208.4 | CUDA, FA off | github.com | 2026-04-05 |
| Qwen3.8 27B | Q4_K_M | llama.cpp | 1k | 3,972.0 | 81.8 | CUDA | github.com | 2026-09-12 |
| Qwen3 8B | Q4_K_XL | llama.cpp | 4k | 11,933.0 | 200.4 | CUDA | hardware-corner.net | 2026-03-02 |
| Qwen3 32B | Q4_K_XL | llama.cpp | 4k | 2,931.0 | 61.4 | CUDA | hardware-corner.net | 2025-12-09 |
| Qwen3 8B | Q4_K_M | llama.cpp | 512 | — | 234.3 | CUDA, KV q8_0 | github.com | 2026-10-07 |
| Qwen3 8B | Q8_0 | llama.cpp | 512 | 10,787.0 | 136.2 | CUDA | github.com | 2025-10-16 |
| Qwen3 32B | Q4_K_M | llama.cpp | 512 | 3,446.0 | 53.3 | CUDA | github.com | 2026-09-15 |
| Gemma 4 E4B | Q4_K_M | llama.cpp | 512 | — | 234.0 | Vulkan | github.com | 2026-09-08 |
| Gemma 4 E2B | Q4_K_M | llama.cpp | 512 | — | 341.8 | Vulkan | github.com | 2026-09-08 |
Public benchmarks we calibrate against. Their conditions (context, backend, flags) can differ from the estimates above.
Frequently asked questions
Which is faster for local LLMs, the DGX Spark 128GB or the RTX 5090?
The RTX 5090 generates text about 5.9× faster than the DGX Spark 128GB (median of our estimates over the 49 models that fit entirely in memory on both at Q4_K_M or the closest available quant). Estimated speed on K2-Horizon MoVA 36B-A4B at Q4_K_M: the DGX Spark 128GB est. 37.7 tok/s (26.4–49.1, theoretical estimate ±30%); the RTX 5090 est. 149.6 tok/s (104.7–194.5, theoretical estimate ±30%).
What is the largest model each can run entirely in memory?
The DGX Spark 128GB fits Ling-3.0 Flash at Q4_K_M (77.01 GB file) entirely in 128 GB of unified memory, at est. 52.8 tok/s (36.9–68.6, theoretical estimate ±30%). The RTX 5090 fits K2-Horizon MoVA 36B-A4B at Q4_K_M (22.37 GB file) entirely in 32 GB, at est. 149.6 tok/s (104.7–194.5, theoretical estimate ±30%).
How many local LLMs run well on each?
At Q4_K_M with 8k context, out of 59 tracked models: the DGX Spark 128GB — 34 Runs great, 12 Runs well, 10 Runs slowly, 3 Won't run; the RTX 5090 — 49 Runs great, 1 Runs well, 1 Runs slowly, 8 Won't run. 24 models get a different verdict.
Speeds are estimates from memory bandwidth, calibrated against public benchmarks, and each one comes with an error band and a confidence label. Real results vary with drivers, backend, context length and thermals.