Qwen3 8B hardware requirements
What is Qwen3 8B?
Qwen3 8B is a dense model that Alibaba Cloud's Qwen team released in April 2025. It thinks step by step before every answer by default, which helps with math, coding and logic, and thinking can be switched off for quick everyday chat. Its model card lists more than 100 languages and tool calling for agent tasks, and the Apache 2.0 license allows commercial use.
What to know before running Qwen3 8B
Qwen3 8B needs 6.5 GB at Q4_K_M with an 8k context, so it fits fully on 8GB cards. It runs great on the GeForce RTX 4060 and the GeForce RTX 3060 12GB. Its KV cache is what limits it on small cards. At its native 32k context it needs 10.3 GB with an f16 KV cache, and on an 8GB card even 16k spills into system RAM. On such a card, 32k stays fully on the GPU only with a q4_0 KV cache.
The model thinks before it answers by default. That costs tokens, so CanRun holds it to a speed bar 1.5 times stricter than for models without thinking; on slower hardware it ends up a step below a same-sized model like Llama 3.1 8B. On the GeForce RTX 3050 8GB, for example, it runs well while Llama runs great. The model card describes a switch to turn thinking off (enable_thinking=False, or /no_think in a message — Qwen3-8B on Hugging Face), and with thinking off a model may do one grade better than the table shows.
About the model
- Parameters
- 8.2B
- Attention
- Full attention
- Max context
- 32,768 tokens
- Released
- 2025-04-29
- Model card
- Qwen3 8B on Hugging Face
Other sizes in this series
| Model | Parameters | Baseline quant | Memory needed (8k) | GeForce RTX 4060 | GeForce RTX 5060 Ti 16GB | GeForce RTX 3090 |
|---|---|---|---|---|---|---|
| Qwen3 8Bthis model | 8.2B | Q4_K_M | 6.5 GB | Runs great | Runs great | Runs great |
| Qwen3 14B | 14.8B | Q4_K_M | 10.5 GB | Runs slowly | Runs great | Runs great |
| Qwen3 30B-A3B (2507) | 30.5B total · 3.3B active (MoE) | Q4_K_M | 19.8 GB | Runs well | Runs well | Runs great |
| Qwen3 32B | 32.8B | Q4_K_M | 22.0 GB | Won't run | Runs slowly | Runs well |
GGUF files
| Quant | File size | Quality | Published by |
|---|---|---|---|
| Q2_K | 3.28 GB | Heavy loss | Community GGUF · unsloth |
| IQ4_XS | 4.58 GB | Good balance | Community GGUF · unsloth |
| Q4_K_Mbaseline | 5.03 GB | Good balance | Community GGUF · unsloth |
| Q8_0 | 8.71 GB | Near-lossless | Community GGUF · unsloth |
Some repositories publish only Unsloth Dynamic (UD) variants; sizes are those of the file you would download.
Memory needed by quant and context
| Quant | 4k | 8k | 16k | 32k |
|---|---|---|---|---|
| Q2_K | 4.2 | 4.9 | 6.1 | 8.7 |
| IQ4_XS | 5.4 | 6.0 | 7.3 | 9.9 |
| Q4_K_M | 5.8 | 6.5 | 7.7 | 10.3 |
| Q8_0 | 9.2 | 9.8 | 11.1 | 13.7 |
Which GPUs can run Qwen3 8B?
How to read the verdicts
- Runs great
- Fully on the GPU at 20 tok/s or more
- Runs well
- Fully on the GPU at 8–20 tok/s; MoE experts in system RAM at 20 tok/s or more; or at least 90% on the GPU at 8 tok/s or more
- Runs slowly
- 2–8 tok/s; CPU-only; less than 90% on the GPU; or MoE experts in system RAM below 20 tok/s
- Won't run
- Does not fit, or under 2 tok/s
Filter this list by budget, type and context in the calculator
Scroll sideways to see every column.
| Hardware | Verdict | Speed | Quant | Memory | Runs as | Context | Notes | Price |
|---|---|---|---|---|---|---|---|---|
| GeForce RTX 5090 | Runs greatQwen3 8B verdict on the GeForce RTX 5090 | 234.3 tok/s8k estimate 201.1 tok/s (177.0–225.2, calibrated estimate ±12%)measured (1 run, 512 context) | Q4_K_M | 7.1 / 32.0 GB | Full GPU | up to 32k | — | $4,200 street (as of 2026-08-09) |
| NVIDIA RTX PRO 6000 Blackwell | Runs great | est. 201.1 tok/s177.0–225.2calibrated estimate ±12% | Q4_K_M | 7.1 / 96.0 GB | Full GPU | up to 32k | — | — |
| NVIDIA RTX PRO 6000 Blackwell Max-Q | Runs great | est. 201.1 tok/s177.0–225.2calibrated estimate ±12% | Q4_K_M | 7.1 / 96.0 GB | Full GPU | up to 32k | — | — |
| Radeon RX 7900 XTX | Runs great | est. 117.0 tok/s102.9–131.0calibrated estimate ±12% | Q4_K_M | 7.1 / 24.0 GB | Full GPU | up to 32k | — | $999 launch MSRP |
| GeForce RTX 3090 Ti | Runs great | est. 113.1 tok/s99.5–126.7calibrated estimate ±12% | Q4_K_M | 7.1 / 24.0 GB | Full GPU | up to 32k | — | $1,999 launch MSRP |
| GeForce RTX 4090 | Runs greatQwen3 8B verdict on the GeForce RTX 4090 | est. 113.1 tok/s99.5–126.7calibrated estimate ±12% | Q4_K_M | 7.1 / 24.0 GB | Full GPU | up to 32k | — | $2,755 street (as of 2026-08-09) |
| Apple M5 Ultra · 96 GB | Runs great | est. 112.2 tok/s89.8–134.7calibrated estimate ±20% | Q4_K_M | 6.8 / 67.2 GB | Unified memory | up to 32k | — | — |
| GeForce RTX 5080 | Runs great | est. 107.7 tok/s94.8–120.7calibrated estimate ±12% | Q4_K_M | 7.1 / 16.0 GB | Full GPU | up to 32k | — | $1,256 street (as of 2026-08-09) |
| GeForce RTX 3090 | Runs greatQwen3 8B verdict on the GeForce RTX 3090 | est. 105.0 tok/s92.4–117.6calibrated estimate ±12% | Q4_K_M | 7.1 / 24.0 GB | Full GPU | up to 32k | — | $1,050 street (as of 2026-08-09) |
| GeForce RTX 3080 12GB | Runs great | est. 102.3 tok/s90.1–114.6calibrated estimate ±12% | Q4_K_M | 7.1 / 12.0 GB | Full GPU | up to 32k | — | $799 launch MSRP |
| GeForce RTX 5070 Ti | Runs great | est. 100.5 tok/s88.5–112.6calibrated estimate ±12% | Q4_K_M | 7.1 / 16.0 GB | Full GPU | up to 32k | — | $949 street (as of 2026-08-09) |
| GeForce RTX 5080 Laptop | Runs great | est. 100.5 tok/s88.5–112.6calibrated estimate ±12% | Q4_K_M | 7.1 / 16.0 GB | Full GPU | up to 32k | — | — |
| GeForce RTX 5090 Laptop | Runs great | est. 100.5 tok/s88.5–112.6calibrated estimate ±12% | Q4_K_M | 7.1 / 24.0 GB | Full GPU | up to 32k | — | — |
| Radeon RX 9070 | Runs great | 99.3 tok/s8k estimate 78.6 tok/s (69.2–88.0, calibrated estimate ±12%)measured (2 runs, 512 context) | Q4_K_M | 7.1 / 16.0 GB | Full GPU | up to 32k | — | $639 street (as of 2026-08-09) |
| Radeon RX 7900 XT | Runs great | est. 97.5 tok/s85.8–109.2calibrated estimate ±12% | Q4_K_M | 7.1 / 20.0 GB | Full GPU | up to 32k | — | $899 launch MSRP |
| GeForce RTX 3080 10GB | Runs great | est. 85.3 tok/s75.1–95.5calibrated estimate ±12% | Q4_K_M | 7.1 / 10.0 GB | Full GPU | up to 16k | — | $699 launch MSRP |
| GeForce RTX 4080 Super | Runs great | est. 82.6 tok/s72.7–92.5calibrated estimate ±12% | Q4_K_M | 7.1 / 16.0 GB | Full GPU | up to 32k | — | $999 launch MSRP |
| GeForce RTX 4080 | Runs great | est. 80.5 tok/s70.8–90.1calibrated estimate ±12% | Q4_K_M | 7.1 / 16.0 GB | Full GPU | up to 32k | — | $1,199 launch MSRP |
| Radeon RX 9070 XT | Runs great | est. 78.6 tok/s69.2–88.0calibrated estimate ±12% | Q4_K_M | 7.1 / 16.0 GB | Full GPU | up to 32k | — | $689 street (as of 2026-08-09) |
| Radeon RX 7800 XT | Runs great | est. 76.0 tok/s66.9–85.1calibrated estimate ±12% | Q4_K_M | 7.1 / 16.0 GB | Full GPU | up to 32k | — | $499 launch MSRP |
| GeForce RTX 4070 Ti Super | Runs great | est. 75.4 tok/s66.4–84.5calibrated estimate ±12% | Q4_K_M | 7.1 / 16.0 GB | Full GPU | up to 32k | — | $799 launch MSRP |
| GeForce RTX 5070 | Runs greatQwen3 8B verdict on the GeForce RTX 5070 | est. 75.4 tok/s66.4–84.5calibrated estimate ±12% | Q4_K_M | 7.1 / 12.0 GB | Full GPU | up to 32k | — | $629 street (as of 2026-08-09) |
| GeForce RTX 5070 Ti Laptop | Runs great | est. 75.4 tok/s66.4–84.5calibrated estimate ±12% | Q4_K_M | 7.1 / 12.0 GB | Full GPU | up to 32k | — | — |
| Apple M3 Ultra · 96 GB | Runs great | est. 74.8 tok/s59.9–89.8calibrated estimate ±20% | Q4_K_M | 6.8 / 67.2 GB | Unified memory | up to 32k | — | — |
| Apple M5 Max (40-core GPU) · 48 GB | Runs great | est. 74.8 tok/s59.8–89.8calibrated estimate ±20% | Q4_K_M | 6.8 / 33.6 GB | Unified memory | up to 32k | — | — |
| Apple M2 Ultra · 64 GB | Runs great | est. 73.1 tok/s58.5–87.7calibrated estimate ±20% | Q4_K_M | 6.8 / 44.8 GB | Unified memory | up to 32k | — | — |
| GeForce RTX 2080 Ti | Runs great | est. 69.1 tok/s60.8–77.4calibrated estimate ±12% | Q4_K_M | 7.1 / 11.0 GB | Full GPU | up to 32k | — | $999 launch MSRP |
| Apple M4 Max (40-core GPU) · 48 GB | Runs great | est. 66.5 tok/s53.2–79.8calibrated estimate ±20% | Q4_K_M | 6.8 / 33.6 GB | Unified memory | up to 32k | — | — |
| GeForce RTX 4090 Laptop | Runs great | est. 64.6 tok/s56.9–72.4calibrated estimate ±12% | Q4_K_M | 7.1 / 16.0 GB | Full GPU | up to 32k | — | — |
| Intel Arc B580 | Runs great | 61.0 tok/s8k estimate 43.1 tok/s (34.5–51.8, calibrated estimate ±20%)measured (1 run, 8k context) | Q4_K_M | 7.1 / 12.0 GB | Full GPU | up to 16k | — | $290 street (as of 2026-08-09) |
| GeForce RTX 4070 | Runs greatQwen3 8B verdict on the GeForce RTX 4070 | est. 56.6 tok/s49.8–63.3calibrated estimate ±12% | Q4_K_M | 7.1 / 12.0 GB | Full GPU | up to 32k | — | $599 launch MSRP |
| GeForce RTX 4070 Super | Runs great | est. 56.6 tok/s49.8–63.3calibrated estimate ±12% | Q4_K_M | 7.1 / 12.0 GB | Full GPU | up to 32k | — | $599 launch MSRP |
| Apple M5 Max (32-core GPU) · 36 GB | Runs great | est. 56.0 tok/s44.8–67.3calibrated estimate ±20% | Q4_K_M | 6.8 / 25.2 GB | Unified memory | up to 32k | — | — |
| GeForce GTX 1080 Ti | Runs great | est. 54.3 tok/s38.0–70.6theoretical estimate ±30% | Q4_K_M | 7.1 / 11.0 GB | Full GPU | up to 32k | — | $699 launch MSRP |
| GeForce RTX 3070 | Runs great | est. 50.3 tok/s44.2–56.3calibrated estimate ±12% | Q4_K_M | 7.1 / 8.0 GB | Full GPU | up to 8k | — | $499 launch MSRP |
| GeForce RTX 5060 | Runs great | est. 50.3 tok/s44.2–56.3calibrated estimate ±12% | Q4_K_M | 7.1 / 8.0 GB | Full GPU | up to 8k | — | $339 street (as of 2026-08-09) |
| GeForce RTX 5060 Ti 16GB | est. 50.3 tok/s44.2–56.3calibrated estimate ±12% | Q4_K_M | 7.1 / 16.0 GB | Full GPU | up to 32k | — | $569 street (as of 2026-08-09) | |
| GeForce RTX 5060 Ti 8GB | Runs great | est. 50.3 tok/s44.2–56.3calibrated estimate ±12% | Q4_K_M | 7.1 / 8.0 GB | Full GPU | up to 8k | — | $429 street (as of 2026-08-09) |
| Apple M4 Max (32-core GPU) · 36 GB | Runs great | est. 50.0 tok/s40.0–59.9calibrated estimate ±20% | Q4_K_M | 6.8 / 25.2 GB | Unified memory | up to 32k | — | — |
| GeForce RTX 4080 Laptop | Runs great | est. 48.5 tok/s42.7–54.3calibrated estimate ±12% | Q4_K_M | 7.1 / 12.0 GB | Full GPU | up to 32k | — | — |
| Apple M5 Pro · 24 GB | Runs great | 45.4 tok/s8k estimate 37.4 tok/s (29.9–44.9, calibrated estimate ±20%)measured (1 run, 512 context) | Q4_K_M | 6.8 / 16.8 GB | Unified memory | up to 16k | — | — |
| NVIDIA DGX Spark · 128 GB | Runs great | 43.9 tok/s8k estimate 34.1 tok/s (27.3–41.0, calibrated estimate ±20%)measured (1 run, 512 context) | Q4_K_M | 6.8 / 116.9 GB | Unified memory | up to 8k | — | $3,999 launch MSRP (128 GB) |
| Ryzen AI Max+ 395 (Strix Halo) · 32 GB | Runs great | 43.6 tok/s8k estimate 33.7 tok/s (26.9–40.4, calibrated estimate ±20%)measured (1 run, 512 context) | Q4_K_M | 6.8 / 22.4 GB | Unified memory | up to 8k | — | $1,999 launch MSRP (128 GB) |
| GeForce RTX 3060 12GB | est. 40.4 tok/s35.6–45.2calibrated estimate ±12% | Q4_K_M | 7.1 / 12.0 GB | Full GPU | up to 16k | — | $250 street (as of 2026-08-09) | |
| Radeon RX 9060 XT 16GB | Runs great | est. 39.2 tok/s34.5–43.9calibrated estimate ±12% | Q4_K_M | 7.1 / 16.0 GB | Full GPU | up to 16k | — | $449 street (as of 2026-08-09) |
| Apple M4 Pro · 24 GB | Runs greatQwen3 8B verdict on the Apple M4 Pro | est. 33.3 tok/s26.6–39.9calibrated estimate ±20% | Q4_K_M | 6.8 / 16.8 GB | Unified memory | up to 8k | — | — |
| GeForce RTX 4060 Ti 16GB | Runs great | est. 32.3 tok/s28.4–36.2calibrated estimate ±12% | Q4_K_M | 7.1 / 16.0 GB | Full GPU | up to 8k | — | $499 launch MSRP |
| GeForce RTX 4060 Ti 8GB | Runs great | est. 32.3 tok/s28.4–36.2calibrated estimate ±12% | Q4_K_M | 7.1 / 8.0 GB | Full GPU | up to 8k | — | $399 launch MSRP |
| GeForce RTX 4060 | Runs greatQwen3 8B verdict on the GeForce RTX 4060 | est. 30.5 tok/s26.9–34.2calibrated estimate ±12% | Q4_K_M | 7.1 / 8.0 GB | Full GPU | up to 8k | — | $299 launch MSRP |
| GeForce RTX 4060 Laptop | Runs well | est. 28.7 tok/s25.3–32.2calibrated estimate ±12% | Q4_K_M | 7.1 / 8.0 GB | Full GPU | up to 8k |
| — |
| GeForce RTX 4070 Laptop | Runs well | est. 28.7 tok/s25.3–32.2calibrated estimate ±12% | Q4_K_M | 7.1 / 8.0 GB | Full GPU | up to 8k |
| — |
| GeForce RTX 3050 8GB | Runs well | est. 25.1 tok/s22.1–28.2calibrated estimate ±12% | Q4_K_M | 7.1 / 8.0 GB | Full GPU | up to 8k |
| $249 launch MSRP |
| Apple M5 · 16 GB | Runs well | est. 18.7 tok/s15.0–22.5calibrated estimate ±20% | Q4_K_M | 6.8 / 11.2 GB | Unified memory | up to 16k | — | — |
| Apple M6 · 16 GB | Runs well | est. 18.7 tok/s15.0–22.5calibrated estimate ±20% | Q4_K_M | 6.8 / 11.2 GB | Unified memory | up to 16k | — | — |
| Apple M4 · 16 GB | Runs wellQwen3 8B verdict on the Apple M4 | est. 14.6 tok/s11.7–17.5calibrated estimate ±20% | Q4_K_M | 6.8 / 11.2 GB | Unified memory | up to 16k | — | — |
| DDR5-6000 dual-channel CPU · 16 GB | Runs slowly | est. 7.7 tok/s6.2–9.2calibrated estimate ±20% | Q4_K_M | 6.2 GB RAM | CPU only | up to 32k | — | — |
| DDR5-5600 dual-channel CPU · 16 GB | Runs slowly | est. 7.2 tok/s5.7–8.6calibrated estimate ±20% | Q4_K_M | 6.2 GB RAM | CPU only | up to 32k | — | — |
| DDR4-3200 dual-channel CPU · 16 GB | Runs slowly | est. 4.1 tok/s3.3–4.9calibrated estimate ±20% | Q4_K_M | 6.2 GB RAM | CPU only | up to 16k | — | — |
Models with a thinking mode spend extra tokens thinking, so their speed thresholds are 1.5× stricter (30 / 12 / 3 tok/s). With thinking turned off, a model may do one grade better.
Cheapest GPUs that run it
- Runs greatGeForce RTX 3060 12GB$250 street (as of 2026-08-09), est. 40.4 tok/s (35.6–45.2, calibrated estimate ±12%).
Measured results for Qwen3 8B
Public benchmarks we calibrate against. Their conditions (context, backend, flags) can differ from the estimates above.
| Hardware | Quant | Backend | Context | Prompt (tok/s) | Generation (tok/s) | Flags | Source | Measured |
|---|---|---|---|---|---|---|---|---|
| GeForce RTX 5090 | Q4_K_XL | llama.cpp | 16k | — | 145.3 | — | hardware-corner.net | 2026-08-09 |
| GeForce RTX 4090 | Q4_K_XL | llama.cpp | 16k | — | 104.3 | — | hardware-corner.net | 2026-08-09 |
| GeForce RTX 3090 | Q4_K_XL | llama.cpp | 16k | — | 87.5 | — | hardware-corner.net | 2026-08-09 |
| GeForce RTX 3060 12GB | Q4_K_XL | llama.cpp | 16k | — | 42.0 | — | hardware-corner.net | 2026-08-09 |
| GeForce RTX 4090 | Q4_K_XL | llama.cpp | 4k | 9,251.0 | 141.3 | CUDA | hardware-corner.net | 2026-03-01 |
| GeForce RTX 5090 | Q4_K_XL | llama.cpp | 4k | 11,933.0 | 200.4 | CUDA | hardware-corner.net | 2026-03-02 |
| GeForce RTX 3090 | Q4_K_XL | llama.cpp | 4k | 4,050.0 | 115.3 | CUDA | hardware-corner.net | 2025-12-09 |
| GeForce RTX 3060 12GB | Q4_K_XL | llama.cpp | 4k | 1,697.0 | 55.2 | CUDA | hardware-corner.net | 2025-12-09 |
| GeForce RTX 3090 Ti | Q4_K_XL | llama.cpp | 4k | 4,468.0 | 123.7 | CUDA | hardware-corner.net | 2025-12-09 |
| GeForce RTX 4060 Ti 16GB | Q4_K_XL | llama.cpp | 4k | 2,675.0 | 45.8 | CUDA | hardware-corner.net | 2025-12-09 |
| GeForce RTX 4070 Super | Q4_K_XL | llama.cpp | 4k | 4,322.0 | 75.4 | CUDA | hardware-corner.net | 2025-12-09 |
| GeForce RTX 4070 Ti Super | Q4_K_XL | llama.cpp | 4k | 5,220.0 | 96.3 | CUDA | hardware-corner.net | 2025-12-09 |
| GeForce RTX 4070 | Q4_K_XL | llama.cpp | 4k | 3,564.0 | 71.2 | CUDA | hardware-corner.net | 2025-12-09 |
| GeForce RTX 4080 Super | Q4_K_XL | llama.cpp | 4k | 6,137.0 | 104.2 | CUDA | hardware-corner.net | 2025-12-09 |
| GeForce RTX 4080 | Q4_K_XL | llama.cpp | 4k | 6,178.0 | 102.7 | CUDA | hardware-corner.net | 2025-12-09 |
| GeForce RTX 5060 Ti 16GB | Q4_K_XL | llama.cpp | 4k | 2,965.0 | 69.2 | CUDA | hardware-corner.net | 2025-12-09 |
| GeForce RTX 5070 Ti | Q4_K_XL | llama.cpp | 4k | 5,557.0 | 120.5 | CUDA | hardware-corner.net | 2025-12-09 |
| GeForce RTX 5070 | Q4_K_XL | llama.cpp | 4k | 3,488.0 | 85.8 | CUDA | hardware-corner.net | 2025-12-09 |
| GeForce RTX 5080 | Q4_K_XL | llama.cpp | 4k | 6,410.0 | 129.1 | CUDA | hardware-corner.net | 2025-12-09 |
| GeForce RTX 3080 10GB | Q4_K_XL | llama.cpp | 16k | 2,288.0 | 74.2 | CUDA | hardware-corner.net | 2025-12-09 |
| GeForce RTX 5090 | Q4_K_M | llama.cpp | 512 | — | 234.3 | CUDA, KV q8_0 | github.com | 2026-10-07 |
| GeForce RTX 5090 | Q8_0 | llama.cpp | 512 | 10,787.0 | 136.2 | CUDA | github.com | 2025-10-16 |
| Radeon RX 9070 | Q4_K_M | llama.cpp | 512 | 2,635.0 | 109.9 | Vulkan | github.com | 2026-09-06 |
| Radeon RX 9070 | Q4_K_M | llama.cpp | 512 | 3,447.0 | 88.7 | ROCm | github.com | 2026-09-07 |
| Intel Arc B580 | Q4_K_M | ollama | 8k | — | 61.0 | Vulkan | github.com | 2026-08-27 |
| Ryzen AI Max+ 395 (Strix Halo) | Q4_K_M | llama.cpp | 512 | 1,238.0 | 43.6 | Vulkan | github.com | 2026-09-08 |
| NVIDIA DGX Spark | Q4_K_M | llama.cpp | 512 | 3,168.0 | 43.9 | CUDA | github.com | 2025-12-05 |
| Apple M5 Pro | Q4_K_M | llama.cpp | 512 | 1,223.0 | 45.4 | Metal | github.com | 2026-09-08 |
| Apple M5 | Q8_0 | llama.cpp | 512 | 712.0 | 16.1 | Metal, KV q8_0 | andoriyanishant.com | 2026-09-13 |
Frequently asked questions
How much VRAM does Qwen3 8B need?
At Q4_K_M the file is 5.03 GB; with 8k context a graphics card needs about 6.5 GB (weights 4.67 GB, KV cache 1.2 GB, compute buffer 0.6 GB). The smallest tracked file (Q2_K (heavy quality loss)) needs about 4.9 GB. Part of the file stays off the graphics card: llama.cpp keeps the input embedding table in system RAM. A Mac counts the whole file, so it needs a little more.
What is the cheapest GPU that runs Qwen3 8B well?
Cheapest that runs it great: GeForce RTX 3060 12GB at $250 (street price as of 2026-08-09), est. 40.4 tok/s (35.6–45.2, calibrated estimate ±12%).
Can Qwen3 8B run on an 8 GB, 16 GB or 24 GB GPU?
At Q4_K_M with 8k context: GeForce RTX 4060 — Runs great, est. 30.5 tok/s (26.9–34.2, calibrated estimate ±12%); GeForce RTX 5060 Ti 16GB — Runs great, est. 50.3 tok/s (44.2–56.3, calibrated estimate ±12%); GeForce RTX 3090 — Runs great, est. 105.0 tok/s (92.4–117.6, calibrated estimate ±12%).
Speeds are estimates from memory bandwidth, calibrated against public benchmarks, and each one comes with an error band and a confidence label. Real results vary with drivers, backend, context length and thermals.