Ryzen AI Max+ 395 128GB vs RTX 5070 Ti for local LLMs
- Ryzen AI Max+ 395 128GB: Uses its largest memory option, 128 GB, with 8k context and an f16 KV cache.
- RTX 5070 Ti: Assumes 32 GB of DDR5-5600 system RAM, Windows with this GPU driving the display, 8k context and an f16 KV cache.
What to know before choosing
CanRun counts about 70% of the Strix Halo's 128GB as the GPU's share, 89.6 GB, against 15.4 GB free on the RTX 5070 Ti. gpt-oss-120b needs 64.3 GB, so it sits entirely in the Strix Halo's share and runs great there, per a public benchmark. With the page's 32GB of system RAM, the RTX 5070 Ti cannot load it. With 64GB, it runs slowly with experts in RAM, a theoretical estimate (±30%). For 100B-class MoE models, total memory counts: VRAM plus system RAM, or the unified pool.
Text generation follows memory bandwidth, where the card has far more. Qwen3 14B fits in 16GB and runs great on the RTX 5070 Ti, but only well on the Strix Halo. A verdict gap is not always a speed gap, though. Qwen3.5 35B-A3B runs great on the Strix Halo and runs well on the RTX 5070 Ti, a theoretical estimate (±30%). On the card some experts sit in system RAM, and CanRun reserves Runs great for models entirely on the GPU.
The GeForce RTX 5070 Ti Laptop is a different GPU with 12 GB of VRAM. gpt-oss-20b runs great on the desktop card, but the laptop keeps some experts in system RAM and runs it well, a theoretical estimate. The Ryzen AI Max+ 395 (Strix Halo) also comes in 32GB and 64GB systems; at 64GB gpt-oss-120b does not load.
Speed
The RTX 5070 Ti generates text about 3× faster than the Ryzen AI Max+ 395 128GB (median of our estimates over the 29 models that fit entirely in memory on both at Q4_K_M or the closest available quant).
- Reference model
- gpt-oss-20b at MXFP4, the largest model that fits entirely in memory on both at Q4_K_M or the closest available quant
- Ryzen AI Max+ 395 128GB
- est. 68.6 tok/s (54.8–82.3, calibrated estimate ±20%)
- RTX 5070 Ti
- 189.5 tok/s measured at 2k context (1 run; 8k estimate 193.7 tok/s, 155.0–232.5, calibrated estimate ±20%)
- Prompt processing on gpt-oss-20b
- Ryzen AI Max+ 395 128GB: about 2,021 tok/s (1,212–2,829, rough estimate ±40%); RTX 5070 Ti: about 6,697 tok/s (4,018–9,376, rough estimate ±40%)
Specs side by side
| Spec | Ryzen AI Max+ 395 128GB | RTX 5070 Ti | Difference |
|---|---|---|---|
| Memory | 128 GB of unified memory | 16 GB | — |
| Memory bandwidth | 256.0 GB/s | 896.0 GB/s | 3.5× on the RTX 5070 Ti |
| FP16 compute | 29.7 TFLOPS | 44.1 TFLOPS | 1.48× on the RTX 5070 Ti |
| Launch year | 2025 | 2025 | — |
| Price | $1,999 launch MSRP (128 GB) | $949 street (as of 2026-08-09) | — |
| Public benchmark results | 17 results | 3 results | — |
Verdicts at a glance
Out of 58 models at Q4_K_M (or the closest available quant) with 8k context.
- Ryzen AI Max+ 395 128GBRuns great34 modelsRuns well12 modelsRuns slowly9 modelsWon't run3 models
- RTX 5070 TiRuns great29 modelsRuns well8 modelsRuns slowly11 modelsWon't run10 models
How to read the verdicts
- Runs great
- Fully on the GPU at 20 tok/s or more
- Runs well
- Fully on the GPU at 8–20 tok/s; MoE experts in system RAM at 20 tok/s or more; or at least 90% on the GPU at 8 tok/s or more
- Runs slowly
- 2–8 tok/s; CPU-only; less than 90% on the GPU; or MoE experts in system RAM below 20 tok/s
- Won't run
- Does not fit, or under 2 tok/s
Where the verdicts differ
22 models get a different verdict on the two.
Scroll sideways to see every column.
| Model | Ryzen AI Max+ 395 128GB verdict | Ryzen AI Max+ 395 128GB speed | RTX 5070 Ti verdict | RTX 5070 Ti speed |
|---|---|---|---|---|
| Ling-3.0 FlashQ4_K_M | Runs great | est. 48.4 tok/s33.9–62.9theoretical estimate ±30% | Won't run | — |
| gpt-oss-120bMXFP4 | Runs great | 50.2 tok/s8k estimate 48.5 tok/s (38.8–58.1, calibrated estimate ±20%)measured (1 run, 512 context) | Won't run | — |
| Kolibri 1Q4_K_M | Runs great | est. 48.0 tok/s38.4–57.6calibrated estimate ±20% | Won't runTry Q2_K (heavy quality loss): Runs well | est. 84.6 tok/s59.2–110.0theoretical estimate ±30% |
| K2-Horizon MoVA 36B-A4BQ4_K_M | Runs great | est. 35.1 tok/s24.5–45.6theoretical estimate ±30% | Runs well | est. 38.6 tok/s27.0–50.2theoretical estimate ±30% |
| Qwen3.5 35B-A3BQ4_K_M | Runs great | 64.8 tok/s8k estimate 56.0 tok/s (44.8–67.2, calibrated estimate ±20%)measured (1 run, 512 context) | Runs well | est. 83.5 tok/s58.5–108.6theoretical estimate ±30% |
| Qwen3.6 35B-A3BQ4_K_M | Runs great | 63.8 tok/s8k estimate 63.1 tok/s (50.5–75.7, calibrated estimate ±20%)measured (1 run, 512 context) | Runs well | est. 91.1 tok/s63.8–118.5theoretical estimate ±30% |
| MiniCPM-V 4.7 35B-A3BQ4_K_M | Runs great | est. 67.5 tok/s54.0–81.1calibrated estimate ±20% | Runs well | est. 97.0 tok/s67.9–126.1theoretical estimate ±30% |
| Nemotron 3.5 Lightning 30B-A3BQ4_K_M | Runs great | est. 65.9 tok/s46.1–85.7theoretical estimate ±30% | Runs well | est. 62.2 tok/s43.5–80.8theoretical estimate ±30% |
| Qwen3 30B-A3B (2507)Q4_K_M | Runs great | est. 54.2 tok/s43.3–65.0calibrated estimate ±20% | Runs well | est. 74.3 tok/s52.0–96.6theoretical estimate ±30% |
| Gemma 4 26B-A4BQ4_K_M | Runs great | 54.3 tok/s8k estimate 48.1 tok/s (38.5–57.7, calibrated estimate ±20%)measured (2 runs, 512 context) | Runs well | est. 87.9 tok/s61.5–114.3theoretical estimate ±30% |
| Qwen3 14BQ4_K_M | Runs well | est. 20.3 tok/s16.2–24.4calibrated estimate ±20% | Runs great | est. 60.6 tok/s53.4–67.9calibrated estimate ±12% |
| DeepSeek R1 Distill Qwen 14BQ4_K_M | Runs well | est. 19.8 tok/s15.8–23.8calibrated estimate ±20% | Runs great | est. 59.2 tok/s52.1–66.3calibrated estimate ±12% |
| HyperCLOVA X SEED Think 14BQ4_K_M | Runs well | est. 20.6 tok/s14.4–26.8theoretical estimate ±30% | Runs great | est. 61.5 tok/s43.1–80.0theoretical estimate ±30% |
| Phi-4Q4_K_M | Runs well | est. 19.6 tok/s15.7–23.5calibrated estimate ±20% | Runs great | est. 58.5 tok/s51.4–65.5calibrated estimate ±12% |
| Gemma 4 12BQ4_K_M | Runs well | 24.4 tok/s8k estimate 28.9 tok/s (23.2–34.7, calibrated estimate ±20%)measured (1 run, 512 context) | Runs great | est. 86.5 tok/s76.1–96.8calibrated estimate ±12% |
| Qwen3.5 122B-A10BQ4_K_M | Runs well | 20.9 tok/s8k estimate 22.7 tok/s (18.2–27.2, calibrated estimate ±20%)measured (1 run, 512 context) | Won't run | — |
| Laguna S 2.1Q4_K_M | Runs well | est. 26.4 tok/s21.1–31.7calibrated estimate ±20% | Won't run | — |
| Solar Open 100BQ4_K_M | Runs well | est. 21.6 tok/s17.3–26.0calibrated estimate ±20% | Won't runTry Q2_K (heavy quality loss): Runs slowly | est. 21.8 tok/s15.2–28.3theoretical estimate ±30% |
| Muse Glimmer 30BQ4_K_M | Runs well | est. 12.1 tok/s9.6–14.5calibrated estimate ±20% | Runs slowly | est. 12.2 tok/s9.8–14.7calibrated estimate ±20% |
| Qwen3.6 27BQ4_K_M | Runs well | est. 12.1 tok/s9.7–14.5calibrated estimate ±20% | Runs slowly | est. 12.2 tok/s9.8–14.7calibrated estimate ±20% |
| Gemma 3 27BQ4_K_M | Runs well | est. 12.2 tok/s9.8–14.6calibrated estimate ±20% | Runs slowly | est. 12.6 tok/s10.1–15.2calibrated estimate ±20% |
| Llama 3.3 70BQ4_K_M | Runs slowly | est. 4.6 tok/s3.7–5.6calibrated estimate ±20% | Won't runTry Q2_K (heavy quality loss): Runs slowly | est. 2.7 tok/s2.2–3.2calibrated estimate ±20% |
Every model on both
Scroll sideways to see every column.
| Model | Ryzen AI Max+ 395 128GB verdict | Ryzen AI Max+ 395 128GB speed | RTX 5070 Ti verdict | RTX 5070 Ti speed |
|---|---|---|---|---|
| Ling-3.0 FlashQ4_K_M | Runs great | est. 48.4 tok/s33.9–62.9theoretical estimate ±30% | Won't run | — |
| gpt-oss-120bMXFP4 | Runs great | 50.2 tok/s8k estimate 48.5 tok/s (38.8–58.1, calibrated estimate ±20%)measured (1 run, 512 context) | Won't run | — |
| Kolibri 1Q4_K_M | Runs great | est. 48.0 tok/s38.4–57.6calibrated estimate ±20% | Won't runTry Q2_K (heavy quality loss): Runs well | est. 84.6 tok/s59.2–110.0theoretical estimate ±30% |
| K2-Horizon MoVA 36B-A4BQ4_K_M | Runs great | est. 35.1 tok/s24.5–45.6theoretical estimate ±30% | Runs well | est. 38.6 tok/s27.0–50.2theoretical estimate ±30% |
| Qwen3.5 35B-A3BQ4_K_M | Runs great | 64.8 tok/s8k estimate 56.0 tok/s (44.8–67.2, calibrated estimate ±20%)measured (1 run, 512 context) | Runs well | est. 83.5 tok/s58.5–108.6theoretical estimate ±30% |
| Qwen3.6 35B-A3BQ4_K_M | Runs great | 63.8 tok/s8k estimate 63.1 tok/s (50.5–75.7, calibrated estimate ±20%)measured (1 run, 512 context) | Runs well | est. 91.1 tok/s63.8–118.5theoretical estimate ±30% |
| MiniCPM-V 4.7 35B-A3BQ4_K_M | Runs great | est. 67.5 tok/s54.0–81.1calibrated estimate ±20% | Runs well | est. 97.0 tok/s67.9–126.1theoretical estimate ±30% |
| Nemotron 3.5 Lightning 30B-A3BQ4_K_M | Runs great | est. 65.9 tok/s46.1–85.7theoretical estimate ±30% | Runs well | est. 62.2 tok/s43.5–80.8theoretical estimate ±30% |
| Qwen3 30B-A3B (2507)Q4_K_M | Runs great | est. 54.2 tok/s43.3–65.0calibrated estimate ±20% | Runs well | est. 74.3 tok/s52.0–96.6theoretical estimate ±30% |
| Gemma 4 26B-A4BQ4_K_M | Runs great | 54.3 tok/s8k estimate 48.1 tok/s (38.5–57.7, calibrated estimate ±20%)measured (2 runs, 512 context) | Runs well | est. 87.9 tok/s61.5–114.3theoretical estimate ±30% |
| gpt-oss-20bMXFP4 | Runs great | est. 68.6 tok/s54.8–82.3calibrated estimate ±20% | Runs great | 189.5 tok/s8k estimate 193.7 tok/s (155.0–232.5, calibrated estimate ±20%)measured (1 run, 2k context) |
| Kanana 1.5 15.7B-A3BQ4_K_M | Runs great | est. 57.2 tok/s45.8–68.6calibrated estimate ±20% | Runs great | est. 151.6 tok/s121.3–181.9calibrated estimate ±20% |
| Qwen3 14BQ4_K_M | Runs well | est. 20.3 tok/s16.2–24.4calibrated estimate ±20% | Runs great | est. 60.6 tok/s53.4–67.9calibrated estimate ±12% |
| DeepSeek R1 Distill Qwen 14BQ4_K_M | Runs well | est. 19.8 tok/s15.8–23.8calibrated estimate ±20% | Runs great | est. 59.2 tok/s52.1–66.3calibrated estimate ±12% |
| HyperCLOVA X SEED Think 14BQ4_K_M | Runs well | est. 20.6 tok/s14.4–26.8theoretical estimate ±30% | Runs great | est. 61.5 tok/s43.1–80.0theoretical estimate ±30% |
| Phi-4Q4_K_M | Runs well | est. 19.6 tok/s15.7–23.5calibrated estimate ±20% | Runs great | est. 58.5 tok/s51.4–65.5calibrated estimate ±12% |
| Gemma 3 12BQ4_K_M | Runs great | est. 26.8 tok/s21.4–32.1calibrated estimate ±20% | Runs great | est. 80.0 tok/s70.4–89.6calibrated estimate ±12% |
| Mellum2.1 12B-A2.5B ThinkingQ4_K_M | Runs great | est. 93.1 tok/s74.5–111.7calibrated estimate ±20% | Runs great | est. 235.4 tok/s188.3–282.5calibrated estimate ±20% |
| Gemma 4 12BQ4_K_M | Runs well | 24.4 tok/s8k estimate 28.9 tok/s (23.2–34.7, calibrated estimate ±20%)measured (1 run, 512 context) | Runs great | est. 86.5 tok/s76.1–96.8calibrated estimate ±12% |
| Qwen3.5 9BQ4_K_M | Runs great | 37.2 tok/s8k estimate 34.2 tok/s (27.4–41.0, calibrated estimate ±20%)measured (1 run, 4k context) | Runs great | est. 102.2 tok/s89.9–114.4calibrated estimate ±12% |
| K2-Horizon 7BQ4_K_M | Runs great | est. 30.9 tok/s24.7–37.1calibrated estimate ±20% | Runs great | est. 92.3 tok/s81.2–103.3calibrated estimate ±12% |
| Qwen3 8BQ4_K_M | Runs great | 43.6 tok/s8k estimate 33.7 tok/s (26.9–40.4, calibrated estimate ±20%)measured (1 run, 512 context) | Runs great | est. 100.5 tok/s88.5–112.6calibrated estimate ±12% |
| DeepSeek R1 0528 Qwen3 8BQ4_K_M | Runs great | est. 33.7 tok/s26.9–40.4calibrated estimate ±20% | Runs great | est. 100.5 tok/s88.5–112.6calibrated estimate ±12% |
| DeepSeek R1 Distill Llama 8BQ4_K_M | Runs great | est. 35.0 tok/s28.0–42.0calibrated estimate ±20% | Runs great | est. 104.6 tok/s92.1–117.2calibrated estimate ±12% |
| Kanana 1.5 8BQ4_K_M | Runs great | est. 35.0 tok/s28.0–42.0calibrated estimate ±20% | Runs great | est. 104.6 tok/s92.1–117.2calibrated estimate ±12% |
| Llama 3.1 8BQ4_K_M | Runs great | est. 35.0 tok/s28.0–42.0calibrated estimate ±20% | Runs great | est. 104.6 tok/s92.1–117.2calibrated estimate ±12% |
| Gemma 4 E4BQ4_K_M | Runs great | est. 59.3 tok/s41.5–77.1theoretical estimate ±30% | Runs great | est. 156.0 tok/s109.2–202.8theoretical estimate ±30% |
| Ling-3.0 TinyQ4_K_M | Runs great | est. 154.3 tok/s108.0–200.5theoretical estimate ±30% | Runs great | est. 369.6 tok/s258.7–480.4theoretical estimate ±30% |
| Gemma 4 E2BQ4_K_M | Runs great | est. 109.5 tok/s76.6–142.3theoretical estimate ±30% | Runs great | est. 261.7 tok/s183.2–340.2theoretical estimate ±30% |
| Qwen3.5 4BQ4_K_M | Runs great | est. 67.8 tok/s54.2–81.3calibrated estimate ±20% | Runs great | est. 202.4 tok/s178.1–226.7calibrated estimate ±12% |
| Nanbeige4.2 3BQ4_K_M | Runs great | est. 34.3 tok/s24.0–44.5theoretical estimate ±30% | Runs great | est. 102.4 tok/s71.7–133.1theoretical estimate ±30% |
| Spark-X2.5 4BQ4_K_M | Runs great | est. 72.3 tok/s57.9–86.8calibrated estimate ±20% | Runs great | est. 216.1 tok/s190.2–242.1calibrated estimate ±12% |
| Llama 3.2 3BQ4_K_M | Runs great | est. 70.9 tok/s56.7–85.1calibrated estimate ±20% | Runs great | est. 211.9 tok/s186.5–237.4calibrated estimate ±12% |
| LFM2.5 2.6BQ4_K_M | Runs great | est. 116.3 tok/s81.4–151.3theoretical estimate ±30% | Runs great | est. 347.6 tok/s243.3–451.9theoretical estimate ±30% |
| MiniCPM5 2BQ4_K_M | Runs great | est. 109.8 tok/s87.8–131.7calibrated estimate ±20% | Runs great | est. 328.0 tok/s288.6–367.3calibrated estimate ±12% |
| HyperCLOVA X SEED 1.5BQ4_K_M | Runs great | est. 115.6 tok/s92.5–138.8calibrated estimate ±20% | Runs great | est. 345.5 tok/s304.0–387.0calibrated estimate ±12% |
| EXAONE 4.0 1.2BQ4_K_M | Runs great | est. 159.8 tok/s127.9–191.8calibrated estimate ±20% | Runs great | est. 477.6 tok/s420.3–534.9calibrated estimate ±12% |
| Llama 3.2 1BQ4_K_M | Runs great | est. 194.7 tok/s155.7–233.6calibrated estimate ±20% | Runs great | est. 581.6 tok/s511.8–651.4calibrated estimate ±12% |
| MiniCPM5 1BQ4_K_M | Runs great | est. 235.5 tok/s188.4–282.6calibrated estimate ±20% | Runs great | est. 703.7 tok/s619.2–788.1calibrated estimate ±12% |
| Qwen3.5 122B-A10BQ4_K_M | Runs well | 20.9 tok/s8k estimate 22.7 tok/s (18.2–27.2, calibrated estimate ±20%)measured (1 run, 512 context) | Won't run | — |
| Laguna S 2.1Q4_K_M | Runs well | est. 26.4 tok/s21.1–31.7calibrated estimate ±20% | Won't run | — |
| Solar Open 100BQ4_K_M | Runs well | est. 21.6 tok/s17.3–26.0calibrated estimate ±20% | Won't runTry Q2_K (heavy quality loss): Runs slowly | est. 21.8 tok/s15.2–28.3theoretical estimate ±30% |
| Muse Glimmer 30BQ4_K_M | Runs well | est. 12.1 tok/s9.6–14.5calibrated estimate ±20% | Runs slowly | est. 12.2 tok/s9.8–14.7calibrated estimate ±20% |
| Qwen3.6 27BQ4_K_M | Runs well | est. 12.1 tok/s9.7–14.5calibrated estimate ±20% | Runs slowly | est. 12.2 tok/s9.8–14.7calibrated estimate ±20% |
| Gemma 3 27BQ4_K_M | Runs well | est. 12.2 tok/s9.8–14.6calibrated estimate ±20% | Runs slowly | est. 12.6 tok/s10.1–15.2calibrated estimate ±20% |
| Mistral Small 3.2 24BQ4_K_M | Runs well | est. 13.4 tok/s10.7–16.1calibrated estimate ±20% | Runs well | est. 22.6 tok/s18.1–27.2calibrated estimate ±20% |
| Llama 3.3 70BQ4_K_M | Runs slowly | est. 4.6 tok/s3.7–5.6calibrated estimate ±20% | Won't runTry Q2_K (heavy quality loss): Runs slowly | est. 2.7 tok/s2.2–3.2calibrated estimate ±20% |
| EXAONE 4.5 33BQ4_K_M | Runs slowly | est. 10.2 tok/s8.2–12.2calibrated estimate ±20% | Runs slowly | est. 6.4 tok/s5.1–7.7calibrated estimate ±20% |
| DeepSeek R1 Distill Qwen 32BQ4_K_M | Runs slowly | est. 9.5 tok/s7.6–11.5calibrated estimate ±20% | Runs slowly | est. 5.0 tok/s4.0–6.0calibrated estimate ±20% |
| Qwen3 32BQ4_K_M | Runs slowly | 10.2 tok/s8k estimate 9.6 tok/s (7.7–11.5, calibrated estimate ±20%)measured (2 runs, 512 context) | Runs slowly | est. 5.1 tok/s4.0–6.1calibrated estimate ±20% |
| EXAONE 4.0 32BQ4_K_M | Runs slowly | est. 10.6 tok/s8.4–12.7calibrated estimate ±20% | Runs slowly | est. 7.2 tok/s5.7–8.6calibrated estimate ±20% |
| Gemma 4 31BQ4_K_M | Runs slowly | est. 11.1 tok/s8.8–13.3calibrated estimate ±20% | Runs slowly | est. 8.3 tok/s6.7–10.0calibrated estimate ±20% |
| Granite 4.2 30BQ4_K_M | Runs slowly | est. 10.6 tok/s8.5–12.7calibrated estimate ±20% | Runs slowly | est. 6.7 tok/s5.4–8.1calibrated estimate ±20% |
| Qwen3.5 27BQ4_K_M | Runs slowly | est. 11.9 tok/s9.5–14.3calibrated estimate ±20% | Runs slowly | est. 11.3 tok/s9.0–13.6calibrated estimate ±20% |
| Qwen3.8 27BQ4_K_M | Runs slowly | est. 11.7 tok/s9.3–14.0calibrated estimate ±20% | Runs slowly | est. 10.4 tok/s8.3–12.5calibrated estimate ±20% |
| GLM-5.3 FlashQ4_K_M | Won't run | — | Won't run | — |
| DeepSeek V4 Flash 0731MXFP4 | Won't runTry Q2_K (heavy quality loss): Runs slowly | est. 19.1 tok/s13.4–24.9theoretical estimate ±30% | Won't run | — |
| Solar Open 2 250BIQ4_XS | Won't runTry Q2_K (heavy quality loss): Runs slowly | est. 18.0 tok/s12.6–23.4theoretical estimate ±30% | Won't run | — |
Models with a thinking mode spend extra tokens thinking, so their speed thresholds are 1.5× stricter (30 / 12 / 3 tok/s). With thinking turned off, a model may do one grade better.
Public benchmarks
Measured results on the Ryzen AI Max+ 395 128GB
| Model | Quant | Backend | Context | Prompt (tok/s) | Generation (tok/s) | Flags | Source | Measured |
|---|---|---|---|---|---|---|---|---|
| gpt-oss-120b | MXFP4 | llama.cpp | 512 | 707.0 | 50.2 | ROCm 7.0.2, no mmap | quozul.dev | 2025-10-27 |
| Gemma 4 26B-A4B | Q4_K_M | llama.cpp | 512 | 1,327.0 | 55.5 | Vulkan | github.com | 2026-06-30 |
| Gemma 4 26B-A4B | Q4_K_M | llama.cpp | 512 | 1,209.0 | 53.0 | Vulkan | github.com | 2026-08-18 |
| Gemma 4 26B-A4B | UD-Q4_K_XL | llama.cpp | 512 | 1,325.0 | 54.7 | Vulkan | github.com | 2026-05-18 |
| Gemma 4 31B | UD-Q4_K_XL | llama.cpp | 512 | 302.0 | 11.5 | Vulkan | github.com | 2026-05-18 |
| Gemma 4 12B | IQ4_XS | llama.cpp | 512 | 680.0 | 25.7 | Vulkan | github.com | 2026-06-04 |
| Gemma 4 12B | Q4_K_M | llama.cpp | 512 | 685.0 | 24.4 | Vulkan | github.com | 2026-06-04 |
| Qwen3.6 35B-A3B | Q8_0 | llama.cpp | 512 | — | 50.6 | Vulkan, FA off | github.com | 2026-06-10 |
| Qwen3.6 35B-A3B | UD-Q8_K_XL | llama.cpp | 512 | 1,046.0 | 46.2 | Vulkan | github.com | 2026-04-26 |
| Qwen3.6 35B-A3B | Q4_K_M | llama.cpp | 512 | 1,064.0 | 63.8 | Vulkan | github.com | 2026-04-26 |
| Qwen3.5 35B-A3B | Q4_K_M | llama.cpp | 512 | 1,077.0 | 64.8 | Vulkan | github.com | 2026-04-26 |
| Qwen3.5 9B | Q4_K_M | llama.cpp | 4k | 969.0 | 37.2 | Vulkan | localmaxxing.com | 2026-04-26 |
| Qwen3.5 122B-A10B | Q4_K_M | llama.cpp | 512 | 315.0 | 20.9 | KV q8_0 | github.com | 2026-07-19 |
| Qwen3 8B | Q4_K_M | llama.cpp | 512 | 1,238.0 | 43.6 | Vulkan | github.com | 2026-09-08 |
| Qwen3 32B | Q4_K_M | llama.cpp | 512 | 256.0 | 9.9 | ROCm | blog.tosukui.xyz | 2025-07-13 |
| Qwen3 32B | Q4_K_M | llama.cpp | 512 | 140.0 | 10.4 | Vulkan | blog.tosukui.xyz | 2025-05-16 |
| Qwen3 32B | Q8_0 | llama.cpp | 512 | 77.0 | 6.4 | ROCm | llm-tracker.info | 2025-05-11 |
Measured results on the RTX 5070 Ti
| Model | Quant | Backend | Context | Prompt (tok/s) | Generation (tok/s) | Flags | Source | Measured |
|---|---|---|---|---|---|---|---|---|
| llama-2-7b | Q4_0 | llama.cpp | 512 | 6,952.0 | 176.9 | — | github.com | 2025-12-07 |
| gpt-oss-20b | MXFP4 | llama.cpp | 2k | 6,326.0 | 189.5 | — | github.com | 2025-08-20 |
| Qwen3 8B | Q4_K_XL | llama.cpp | 4k | 5,557.0 | 120.5 | CUDA | hardware-corner.net | 2025-12-09 |
Public benchmarks we calibrate against. Their conditions (context, backend, flags) can differ from the estimates above.
Frequently asked questions
Which is faster for local LLMs, the Ryzen AI Max+ 395 128GB or the RTX 5070 Ti?
The RTX 5070 Ti generates text about 3× faster than the Ryzen AI Max+ 395 128GB (median of our estimates over the 29 models that fit entirely in memory on both at Q4_K_M or the closest available quant). Estimated speed on gpt-oss-20b at MXFP4: the Ryzen AI Max+ 395 128GB est. 68.6 tok/s (54.8–82.3, calibrated estimate ±20%); the RTX 5070 Ti 189.5 tok/s measured at 2k context (1 run; 8k estimate 193.7 tok/s, 155.0–232.5, calibrated estimate ±20%).
What is the largest model each can run entirely in memory?
The Ryzen AI Max+ 395 128GB fits Ling-3.0 Flash at Q4_K_M (77.01 GB file) entirely in 128 GB of unified memory, at est. 48.4 tok/s (33.9–62.9, theoretical estimate ±30%). The RTX 5070 Ti fits Muse Glimmer 30B at IQ4_XS (15.44 GB file) entirely in 16 GB, at est. 40.3 tok/s (35.5–45.2, calibrated estimate ±12%).
How many local LLMs run well on each?
At Q4_K_M with 8k context, out of 58 tracked models: the Ryzen AI Max+ 395 128GB — 34 Runs great, 12 Runs well, 9 Runs slowly, 3 Won't run; the RTX 5070 Ti — 29 Runs great, 8 Runs well, 11 Runs slowly, 10 Won't run. 22 models get a different verdict.
Speeds are estimates from memory bandwidth, calibrated against public benchmarks, and each one comes with an error band and a confidence label. Real results vary with drivers, backend, context length and thermals.