Which local LLMs can the GeForce RTX 5090 run?

Assumes 32 GB of DDR5-5600 system RAM, Windows with this GPU driving the display, 8k context and an f16 KV cache.

What to expect from the GeForce RTX 5090

The RTX 5090 has 32GB of VRAM and 1,792 GB/s of memory bandwidth. Dense 32B-class models such as Qwen3 32B and Gemma 4 31B run great at Q4_K_M with room to spare. The extra 8GB over a 24GB card matters for models that just miss 24GB: Nemotron 3.5 Lightning 30B-A3B needs 25.1 GB, so it stays fully on this card but has to keep experts in system RAM on the GeForce RTX 4090. The headroom also buys context: Gemma 4 31B still fits fully at 64k here, which a 24GB card cannot do.

32GB is still not enough for 70B. Llama 3.3 70B at Q4_K_M needs 45.2 GB, and split with system RAM it runs slowly. gpt-oss-120b does not load with 32GB of system RAM. With 64GB it loads with its experts in RAM, but CanRun's speed for that setup is a theoretical estimate, and a public benchmark in the table further down came in well below it.

Our data lists it at $4,200 street (as of 2026-08-09). If gpt-oss-120b is the goal, look at machines with a lot of unified memory: the 128GB Ryzen AI Max+ 395 (Strix Halo), for example, holds it completely and runs it great.

Specs

VRAM: 32 GB
Memory bandwidth: 1,792.0 GB/s
FP16 compute: 104.8 TFLOPS
Launch year: 2025
Price: $4,200 street (as of 2026-08-09)
Also written as: 5090, RTX5090

Verdicts at a glance

At Q4_K_M (or the closest available quant) with 8k context.

Runs great49 modelsRuns well1 modelRuns slowly1 modelWon't run8 models2 more models run at a lower quant.

How to read the verdicts

Runs great: Fully on the GPU at 20 tok/s or more
Runs well: Fully on the GPU at 8–20 tok/s; MoE experts in system RAM at 20 tok/s or more; or at least 90% on the GPU at 8 tok/s or more
Runs slowly: 2–8 tok/s; CPU-only; less than 90% on the GPU; or MoE experts in system RAM below 20 tok/s
Won't run: Does not fit, or under 2 tok/s

Every model on the GeForce RTX 5090

Dense models split between GPU and CPU slow down sharply — the CPU side sets the pace. MoE models that keep only their experts in system RAM degrade far more gently.

Scroll sideways to see every column.

59 models sorted by verdict and speed
Model	Verdict	Speed	Quant	Memory	Runs as	Context	Notes
MiniCPM5 1B	Runs great	est. 1,407.3 tok/s1,238.5–1,576.2calibrated estimate ±12%	Q4_K_M	1.9 / 32.0 GB	Full GPU	up to 128k	thinking mode
Llama 3.2 1B	Runs great	est. 1,163.2 tok/s1,023.6–1,302.7calibrated estimate ±12%	Q4_K_M	2.2 / 32.0 GB	Full GPU	up to 128k	—
EXAONE 4.0 1.2B	Runs great	est. 955.1 tok/s840.5–1,069.8calibrated estimate ±12%	Q4_K_M	2.5 / 32.0 GB	Full GPU	up to 64k	thinking mode
LFM2.5 2.6B	Runs great	est. 695.3 tok/s486.7–903.8theoretical estimate ±30%	Q4_K_M	3.0 / 32.0 GB	Full GPU	up to 128k	thinking mode
HyperCLOVA X SEED 1.5B	Runs great	est. 691.0 tok/s608.1–773.9calibrated estimate ±12%	Q4_K_M	3.0 / 32.0 GB	Full GPU	up to 16k	—
MiniCPM5 2B	Runs great	est. 656.0 tok/s577.2–734.7calibrated estimate ±12%	Q4_K_M	2.9 / 32.0 GB	Full GPU	up to 128k	thinking mode
Ling-3.0 Tiny	Runs great	est. 485.1 tok/s339.6–630.7theoretical estimate ±30%	Q4_K_M	5.9 / 32.0 GB	Full GPU	up to 128k	thinking mode
Spark-X2.5 4B	Runs great	est. 432.3 tok/s380.4–484.1calibrated estimate ±12%	Q4_K_M	4.1 / 32.0 GB	Full GPU	up to 256k	thinking mode
Llama 3.2 3B	Runs great	est. 423.9 tok/s373.0–474.7calibrated estimate ±12%	Q4_K_M	4.1 / 32.0 GB	Full GPU	up to 128k	—
Qwen3.5 4B	Runs great	est. 404.8 tok/s356.3–453.4calibrated estimate ±12%	Q4_K_M	4.2 / 32.0 GB	Full GPU	up to 256k	thinking mode
Gemma 4 E2B	Runs great	341.8 tok/s8k estimate 386.7 tok/s (270.7–502.7, theoretical estimate ±30%)measured (1 run, 512 context)	Q4_K_M	2.7 / 32.0 GB	Full GPU	up to 128k	thinking mode
Mellum2.1 12B-A2.5B Thinking	Runs great	est. 334.9 tok/s267.9–401.8calibrated estimate ±20%	Q4_K_M	9.2 / 32.0 GB	Full GPU	up to 128k	thinking mode
Nemotron 3.5 Lightning 30B-A3B	Runs great	est. 305.7 tok/s214.0–397.4theoretical estimate ±30%	Q4_K_M	25.7 / 32.0 GB	Full GPU	up to 256k	thinking mode
gpt-oss-20b	Runs greatgpt-oss-20b verdict on the GeForce RTX 5090	282.3 tok/s8k estimate 338.9 tok/s (271.1–406.7, calibrated estimate ±20%)measured (1 run, 2k context)	MXFP4	12.9 / 32.0 GB	Full GPU	up to 128k	thinking mode
MiniCPM-V 4.7 35B-A3B	Runs great	est. 239.5 tok/s191.6–287.4calibrated estimate ±20%	Q4_K_M	22.2 / 32.0 GB	Full GPU	up to 256k	thinking mode
Qwen3 8B	Runs greatQwen3 8B verdict on the GeForce RTX 5090	234.3 tok/s8k estimate 201.1 tok/s (177.0–225.2, calibrated estimate ±12%)measured (1 run, 512 context)	Q4_K_M	7.1 / 32.0 GB	Full GPU	up to 32k	thinking mode
Gemma 4 E4B	Runs great	234.0 tok/s8k estimate 257.7 tok/s (180.4–335.0, theoretical estimate ±30%)measured (1 run, 512 context)	Q4_K_M	4.3 / 32.0 GB	Full GPU	up to 128k	thinking mode
Kanana 1.5 15.7B-A3B	Runs great	est. 233.4 tok/s186.7–280.1calibrated estimate ±20%	Q4_K_M	12.5 / 32.0 GB	Full GPU	up to 32k	—
Qwen3.6 35B-A3B	Runs greatQwen3.6 35B-A3B verdict on the GeForce RTX 5090	est. 229.8 tok/s183.9–275.8calibrated estimate ±20%	Q4_K_M	22.4 / 32.0 GB	Full GPU	up to 256k	thinking mode
Gemma 4 26B-A4B	Runs greatGemma 4 26B-A4B verdict on the GeForce RTX 5090	216.1 tok/s8k estimate 210.4 tok/s (168.3–252.4, calibrated estimate ±20%)measured (2 runs, 512/8k context)	Q4_K_M	18.3 / 32.0 GB	Full GPU	up to 256k	thinking mode
Qwen3.5 35B-A3B	Runs greatQwen3.5 35B-A3B verdict on the GeForce RTX 5090	est. 213.4 tok/s170.7–256.0calibrated estimate ±20%	Q4_K_M	22.9 / 32.0 GB	Full GPU	up to 256k	thinking mode
DeepSeek R1 Distill Llama 8B	Runs great	est. 209.3 tok/s184.2–234.4calibrated estimate ±12%	Q4_K_M	6.9 / 32.0 GB	Full GPU	up to 128k	thinking mode
Kanana 1.5 8B	Runs great	est. 209.3 tok/s184.2–234.4calibrated estimate ±12%	Q4_K_M	6.9 / 32.0 GB	Full GPU	up to 32k	—
Llama 3.1 8B	Runs greatLlama 3.1 8B verdict on the GeForce RTX 5090	est. 209.3 tok/s184.2–234.4calibrated estimate ±12%	Q4_K_M	6.9 / 32.0 GB	Full GPU	up to 128k	—
Nanbeige4.2 3B	Runs great	est. 204.8 tok/s143.3–266.2theoretical estimate ±30%	Q4_K_M	5.0 / 32.0 GB	Full GPU	up to 128k	thinking mode
Qwen3.5 9B	Runs greatQwen3.5 9B verdict on the GeForce RTX 5090	est. 204.4 tok/s179.8–228.9calibrated estimate ±12%	Q4_K_M	6.6 / 32.0 GB	Full GPU	up to 256k	thinking mode
DeepSeek R1 0528 Qwen3 8B	Runs great	est. 201.1 tok/s177.0–225.2calibrated estimate ±12%	Q4_K_M	7.1 / 32.0 GB	Full GPU	up to 128k	thinking mode
Qwen3 30B-A3B (2507)	Runs greatQwen3 30B-A3B (2507) verdict on the GeForce RTX 5090	est. 195.0 tok/s156.0–234.1calibrated estimate ±20%	Q4_K_M	20.4 / 32.0 GB	Full GPU	up to 64k	—
K2-Horizon 7B	Runs great	est. 184.5 tok/s162.4–206.7calibrated estimate ±12%	Q4_K_M	7.4 / 32.0 GB	Full GPU	up to 128k	thinking mode
Gemma 4 12B	Runs greatGemma 4 12B verdict on the GeForce RTX 5090	est. 172.9 tok/s152.2–193.7calibrated estimate ±12%	Q4_K_M	8.4 / 32.0 GB	Full GPU	up to 256k	thinking mode
Gemma 3 12B	Runs greatGemma 3 12B verdict on the GeForce RTX 5090	est. 160.1 tok/s140.9–179.3calibrated estimate ±12%	Q4_K_M	9.0 / 32.0 GB	Full GPU	up to 128k	—
K2-Horizon MoVA 36B-A4B	Runs great	est. 149.6 tok/s104.7–194.5theoretical estimate ±30%	Q4_K_M	24.8 / 32.0 GB	Full GPU	up to 32k	thinking mode
HyperCLOVA X SEED Think 14B	Runs great	est. 123.0 tok/s86.1–160.0theoretical estimate ±30%	Q4_K_M	11.0 / 32.0 GB	Full GPU	up to 128k	thinking mode
Qwen3 14B	Runs greatQwen3 14B verdict on the GeForce RTX 5090	est. 121.3 tok/s106.7–135.8calibrated estimate ±12%	Q4_K_M	11.1 / 32.0 GB	Full GPU	up to 32k	thinking mode
DeepSeek R1 Distill Qwen 14B	Runs great	est. 118.3 tok/s104.1–132.5calibrated estimate ±12%	Q4_K_M	11.3 / 32.0 GB	Full GPU	up to 64k	thinking mode
Phi-4	Runs great	est. 116.9 tok/s102.9–131.0calibrated estimate ±12%	Q4_K_M	11.6 / 32.0 GB	Full GPU	up to 16k	—
Qwen3.8 27B	Runs greatQwen3.8 27B verdict on the GeForce RTX 5090	81.8 tok/s8k estimate 69.8 tok/s (61.4–78.2, calibrated estimate ±12%)measured (1 run, 1k context)	Q4_K_M	18.2 / 32.0 GB	Full GPU	up to 128k	thinking mode
Mistral Small 3.2 24B	Runs great	est. 80.0 tok/s70.4–89.6calibrated estimate ±12%	Q4_K_M	16.5 / 32.0 GB	Full GPU	up to 64k	—
Gemma 3 27B	Runs great	est. 72.8 tok/s64.1–81.6calibrated estimate ±12%	Q4_K_M	18.4 / 32.0 GB	Full GPU	up to 128k	—
Qwen3.6 27B	Runs great	est. 72.3 tok/s63.6–80.9calibrated estimate ±12%	Q4_K_M	17.8 / 32.0 GB	Full GPU	up to 128k	thinking mode
Muse Glimmer 30B	Runs great	est. 72.0 tok/s63.4–80.7calibrated estimate ±12%	Q4_K_M	17.8 / 32.0 GB	Full GPU	up to 128k	thinking mode
Qwen3.5 27B	Runs greatQwen3.5 27B verdict on the GeForce RTX 5090	est. 71.1 tok/s62.6–79.6calibrated estimate ±12%	Q4_K_M	17.8 / 32.0 GB	Full GPU	up to 128k	thinking mode
Gemma 4 31B	Runs greatGemma 4 31B verdict on the GeForce RTX 5090	est. 66.1 tok/s58.1–74.0calibrated estimate ±12%	Q4_K_M	20.1 / 32.0 GB	Full GPU	up to 128k	thinking mode
Granite 4.2 30B	Runs great	est. 63.1 tok/s55.6–70.7calibrated estimate ±12%	Q4_K_M	20.8 / 32.0 GB	Full GPU	up to 32k	thinking mode
EXAONE 4.0 32B	Runs great	est. 63.1 tok/s55.5–70.7calibrated estimate ±12%	Q4_K_M	20.8 / 32.0 GB	Full GPU	up to 128k	thinking mode
EXAONE 4.5 33B	Runs great	est. 60.9 tok/s53.6–68.2calibrated estimate ±12%	Q4_K_M	21.0 / 32.0 GB	Full GPU	up to 128k	thinking mode
DeepSeek R1 Distill Qwen 32B	Runs great	est. 57.0 tok/s50.2–63.9calibrated estimate ±12%	Q4_K_M	22.7 / 32.0 GB	Full GPU	up to 32k	thinking mode
Nanbeige4.3 33B	Runs great	est. 55.7 tok/s49.0–62.3calibrated estimate ±12%	Q4_K_M	23.0 / 32.0 GB	Full GPU	up to 32k	thinking mode
Qwen3 32B	Runs greatQwen3 32B verdict on the GeForce RTX 5090	53.3 tok/s8k estimate 57.3 tok/s (50.4–64.1, calibrated estimate ±12%)measured (1 run, 512 context)	Q4_K_M	22.6 / 32.0 GB	Full GPU	up to 32k	thinking mode
Kolibri 1	Runs well	est. 83.3 tok/s58.3–108.3theoretical estimate ±30%	Q4_K_M	30.1 / 32.0 GB + 21.0 GB RAM	MoE experts in RAM	up to 128k	Not fully on the GPU — Runs great needs the whole model in GPU memory Q2_K (heavy quality loss): Runs great thinking mode
Llama 3.3 70B	Runs slowlyLlama 3.3 70B verdict on the GeForce RTX 5090	est. 2.7 tok/s2.2–3.3calibrated estimate ±20%	Q4_K_M	32.0 / 32.0 GB + 14.4 GB RAM	Partial offload	up to 16k	Q2_K (heavy quality loss): Runs great
Qwen3.5 122B-A10B	Won't runTry UD-Q2_K_XL (heavy quality loss): Runs wellQwen3.5 122B-A10B verdict on the GeForce RTX 5090	est. 66.4 tok/s46.5–86.3theoretical estimate ±30%	UD-Q2_K_XL	31.0 / 32.0 GB + 13.2 GB RAM	MoE experts in RAM	up to 256k	Not fully on the GPU — Runs great needs the whole model in GPU memory With 64 GB RAM: Runs slowly thinking mode
Solar Open 100B	Won't runTry IQ4_XS: Runs slowly	est. 21.9 tok/s15.3–28.4theoretical estimate ±30%	IQ4_XS	30.8 / 32.0 GB + 27.5 GB RAM	MoE experts in RAM	up to 8k	Experts run from system RAM — Runs well needs 30 tok/s or more on this path Q2_K (heavy quality loss): Runs well With 64 GB RAM: Runs slowly thinking mode
DeepSeek V4 Flash 0731	Won't run	—	MXFP4	needs 155.1 GB	—	—	Needs about 155.1 GB; 31.4 GB of VRAM and 28.0 GB of RAM are free thinking mode
GLM-5.3 Flash	Won't run	—	Q4_K_M	needs 181.4 GB	—	—	Needs about 181.4 GB; 31.4 GB of VRAM and 28.0 GB of RAM are free thinking mode
gpt-oss-120b	Won't rungpt-oss-120b verdict on the GeForce RTX 5090	—	MXFP4	needs 63.6 GB	—	—	Needs about 63.6 GB; 31.4 GB of VRAM and 28.0 GB of RAM are free With 64 GB RAM: Runs well thinking mode
Laguna S 2.1	Won't run	—	Q4_K_M	needs 68.5 GB	—	—	Needs about 68.5 GB; 31.4 GB of VRAM and 28.0 GB of RAM are free With 64 GB RAM: Runs slowly thinking mode
Ling-3.0 Flash	Won't run	—	Q4_K_M	needs 75.4 GB	—	—	Needs about 75.4 GB; 31.4 GB of VRAM and 28.0 GB of RAM are free With 64 GB RAM: Runs well thinking mode
Solar Open 2 250B	Won't run	—	IQ4_XS	needs 136.5 GB	—	—	Needs about 136.5 GB; 31.4 GB of VRAM and 28.0 GB of RAM are free With 128 GB RAM: Runs slowly thinking mode

Models with a thinking mode spend extra tokens thinking, so their speed thresholds are 1.5× stricter (30 / 12 / 3 tok/s). With thinking turned off, a model may do one grade better.

Measured results on the GeForce RTX 5090

Public benchmarks we calibrate against. Their conditions (context, backend, flags) can differ from the estimates above.

Measured results on the GeForce RTX 5090
Model	Quant	Backend	Context	Prompt (tok/s)	Generation (tok/s)	Flags	Source	Measured
llama-2-7b	Q4_0	llama.cpp	512	14,073.0	290.0	—	github.com	2025-10-23
gpt-oss-20b	MXFP4	llama.cpp	2k	9,841.0	282.3	—	github.com	2025-08-20
Qwen3 8B	Q4_K_XL	llama.cpp	16k	—	145.3	—	hardware-corner.net	2026-08-09
Qwen3 30B-A3B (2507)	Q4_K_XL	llama.cpp	16k	—	141.6	—	hardware-corner.net	2026-08-09
gpt-oss-120b	MXFP4	llama.cpp	4k	—	9.6	--n-cpu-moe 21, DDR4 server	hardware-corner.net	2026-08-09
Gemma 4 26B-A4B	Q4_K_M	llama.cpp	512	13,588.0	223.8	CUDA, KV q8_0	github.com	2026-05-24
Gemma 4 26B-A4B	Q4_K_M	ollama	8k	—	208.4	CUDA, FA off	github.com	2026-04-05
Qwen3.8 27B	Q4_K_M	llama.cpp	1k	3,972.0	81.8	CUDA	github.com	2026-09-12
Qwen3 8B	Q4_K_XL	llama.cpp	4k	11,933.0	200.4	CUDA	hardware-corner.net	2026-03-02
Qwen3 32B	Q4_K_XL	llama.cpp	4k	2,931.0	61.4	CUDA	hardware-corner.net	2025-12-09
Qwen3 8B	Q4_K_M	llama.cpp	512	—	234.3	CUDA, KV q8_0	github.com	2026-10-07
Qwen3 8B	Q8_0	llama.cpp	512	10,787.0	136.2	CUDA	github.com	2025-10-16
Qwen3 32B	Q4_K_M	llama.cpp	512	3,446.0	53.3	CUDA	github.com	2026-09-15
Gemma 4 E4B	Q4_K_M	llama.cpp	512	—	234.0	Vulkan	github.com	2026-09-08
Gemma 4 E2B	Q4_K_M	llama.cpp	512	—	341.8	Vulkan	github.com	2026-09-08

Frequently asked questions

What is the largest model that runs entirely on the GeForce RTX 5090?

K2-Horizon MoVA 36B-A4B at Q4_K_M (a 22.37 GB file) fits entirely in 32 GB with 8k context, at est. 149.6 tok/s (104.7–194.5, theoretical estimate ±30%).

How many local LLMs run well on the GeForce RTX 5090?

At Q4_K_M with 8k context, out of 59 tracked models: 49 run great, 1 runs well, 1 runs slowly and 8 won't run.

Can the GeForce RTX 5090 run a 70B model like Llama 3.3 70B?

Runs slowly — Partial offload, Q4_K_M: est. 2.7 tok/s (2.2–3.3, calibrated estimate ±20%).

Speeds are estimates from memory bandwidth, calibrated against public benchmarks, and each one comes with an error band and a confidence label. Real results vary with drivers, backend, context length and thermals.