Can I run Llama 3.3 70B on the GeForce RTX 4060?

Won't runQ4_K_M needs 45.2 GB, 37.8 GB more than the 7.4 GB of usable memory, and no tracked quant runs at a usable speed with 32 GB of RAM.

Custom license · Llama 3.3 — check terms before commercial use Community GGUF · bartowski

8 GB VRAM · 272.0 GB/s · FP16 15.1 TFLOPS. Assumes 32 GB of DDR5-5600 system RAM, Windows with this GPU driving the display, 8k context and an f16 KV cache.

Verdict by system RAM

This page assumes 32 GB of system RAM. The verdict stays the same, but which file fits in memory depends on how much system RAM you have.

16 GBWon't runQ4_K_M doesn't fit
32 GBWon't runQ4_K_M doesn't fit(used on this page)
64 GBWon't runQ4_K_M fits, but too slowest. 1.1 tok/s (0.9–1.3, calibrated estimate ±20%)
96 GBWon't runQ4_K_M fits, but too slowest. 1.1 tok/s (0.9–1.3, calibrated estimate ±20%)
128 GBWon't runQ4_K_M fits, but too slowest. 1.1 tok/s (0.9–1.3, calibrated estimate ±20%)

Every quant of Llama 3.3 70B on the GeForce RTX 4060

5 tracked GGUF files, smallest first, at 32 GB of system RAM and 8k context
Quant	File size	Verdict	Speed	Memory	Runs as	Notes
IQ2_M	24.12 GB	Won't run	est. 1.9 tok/s1.5–2.3calibrated estimate ±20%	8.0 / 8.0 GB + 20.0 GB RAM	Partial offload	Fits, but too slow to use
Q2_K	26.38 GB	Won't run	est. 1.8 tok/s1.4–2.1calibrated estimate ±20%	8.0 / 8.0 GB + 22.2 GB RAM	Partial offload	Fits, but too slow to use
IQ4_XS	37.90 GB	Won't run	—	Needs about 40.6 GB; 7.4 GB of VRAM and 28.0 GB of RAM are free	—	—
Q4_K_Mbaseline	42.52 GB	Won't run	—	Needs about 45.2 GB; 7.4 GB of VRAM and 28.0 GB of RAM are free	—	—
Q8_0	74.98 GB	Won't run	—	Needs about 77.1 GB; 7.4 GB of VRAM and 28.0 GB of RAM are free	—	—

Where the memory goes at Q4_K_M

Q4_K_M does not fit — it needs 45.2 GB against 7.4 GB of usable memory, so there is nothing to break down.

System RAM, context length and KV cache

Context: 8k

KV cachef16q8_0q4_0

Memory needed and verdict at Q4_K_M for each context length and KV cache type, with estimated tokens per second.
Context	KV f16	KV q8_0	KV q4_0
4k	43.8 GBdoes not fit	43.2 GBdoes not fit	42.8 GBdoes not fit
8k	45.2 GBdoes not fit	43.9 GBdoes not fit	43.3 GBdoes not fit
16k	47.9 GBdoes not fit	45.4 GBdoes not fit	44.1 GBdoes not fit
32k	53.5 GBdoes not fit	48.5 GBdoes not fit	45.8 GBdoes not fit
64k	64.5 GBdoes not fit	54.5 GBdoes not fit	49.1 GBdoes not fit
128k	86.6 GBdoes not fit	66.6 GBdoes not fit	55.9 GBdoes not fit

S = Runs great · A = Runs well · B = Runs slowly · F = Won't run

At Q4_K_M no context length fits: even 4k needs 43.8 GB against 7.4 GB.

Estimated speed

Generation: —
Prompt processing: Prompt-processing estimates only apply when the whole model runs on the GPU.

Measured on this exact combination

No public measurement for this exact combination yet; the numbers above are estimates.

If this is not enough

A smaller model that runs well on the GeForce RTX 4060
Runs well
Qwen3.5 35B-A3B: Runs well, est. 39.3 tok/s (27.5–51.1, theoretical estimate ±30%)

How to run it

No run command: this file does not fit in 7.4 GB of usable memory plus 28.0 GB of RAM. For a different amount of RAM, pick it under “System RAM, context length and KV cache” above to see the commands for it.

Guide: How much VRAM do local LLMs need? GPU requirements for 8–32 GB (2026)

Other models on the GeForce RTX 4060

Llama 3.3 70B on other hardware

Frequently asked questions