Recommended GPUs for local LLMs in 2026: what each budget runs

For local LLMs, choose the memory size first and the card second: VRAM decides which models fit, and memory bandwidth mostly sets the speed once they do. What we recommend is a memory size, not a product. This guide lists what each size costs in our data and which models it runs.

What each memory size costs

At an 8k context, Llama 3.1 8B Runs great on every card in this guide, down to the 8GB RTX 4060 and RTX 5060. 12GB adds Gemma 3 12B and Qwen3 14B on the GPU alone, 16GB adds gpt-oss-20b, and 24GB adds dense models around 30B such as Qwen3.8 27B and Gemma 4 31B.

The list groups by memory size the cards in our catalog that have a street price, plus the RTX 4060, which has only a launch MSRP.

• 8GB: the GeForce RTX 5060 at $339 street (as of 2026-08-09), the GeForce RTX 5060 Ti 8GB at $429 street (as of 2026-08-09), and the GeForce RTX 4060 at $299 launch MSRP.
• 12GB: the GeForce RTX 3060 12GB at $250 street (as of 2026-08-09), the Intel Arc B580 at $290 street (as of 2026-08-09) and the GeForce RTX 5070 at $629 street (as of 2026-08-09).
• 16GB: the Radeon RX 9060 XT 16GB at $449 street (as of 2026-08-09), the GeForce RTX 5060 Ti 16GB at $569 street (as of 2026-08-09), the Radeon RX 9070 at $639 street (as of 2026-08-09), the Radeon RX 9070 XT at $689 street (as of 2026-08-09), the GeForce RTX 5070 Ti at $949 street (as of 2026-08-09) and the GeForce RTX 5080 at $1,256 street (as of 2026-08-09).
• 24GB: the GeForce RTX 3090 at $1,050 street (as of 2026-08-09) and the GeForce RTX 4090 at $2,755 street (as of 2026-08-09).
• 32GB: the GeForce RTX 5090 at $4,200 street (as of 2026-08-09).

For each of these cards, the table counts how many models in CanRun's catalog get each verdict at an 8k context and names the largest model that fits on the GPU alone. New models join the catalog often, so read it for the pattern.

Within one memory size, CanRun's calculation fits the same models on every card, because fit depends only on the memory a card can use. That is why Qwen3.8 27B and Gemma 4 31B Run slowly even on the RTX 5080 and Run great on the RTX 3090, a card our price data listed lower when this guide was written.

Reading the prices

A street price always comes with the date we recorded it, and all prices are in US dollars. It is a snapshot from public web pages, not today's price; the Sources section names each page and says where the figure cannot be checked there. When we have no street price, the card shows its launch MSRP instead, as the RTX 4060 does, and a launch MSRP is not what a card sells for today.

The RTX 3090 launched in 2020. Its price comes from a price-history page that tracks both new and used RTX 3090 cards, and our record does not say which kind the figure is. The RTX 4090 price comes from the same kind of page. If you buy a card used, its condition and warranty are for you to check.

Every model page, such as the one for Qwen3.8 27B, lists the cheapest GPUs by street price that run it great or well, and the calculator's "Find GPUs for a model" mode has a budget filter.

Six popular models across the price range

The grid follows six popular models on eight cards, ordered by street price as our data listed it when this guide was written, lowest first. Every verdict assumes an 8k context, an f16 KV cache (the memory that holds the context), 32GB of DDR5 system RAM and Windows. Each cell shows an estimated speed band and a confidence label.

From 8GB to 12GB: 14B dense models

Qwen3 14B is the first model in the grid where memory splits the cards. At Q4_K_M, the usual 4-bit quant, it needs 10.5 GB at 8k, more than the 7.4 GB an 8GB card can use, so on the RTX 5060 part of it runs from system RAM and it Runs slowly. On the 12GB RTX 3060 it fits with less than a gigabyte to spare and Runs well; check ollama ps after loading it. On the 12GB RTX 5070, whose memory is faster, it Runs great. Qwen3 14B can think before it answers, which costs many more tokens per answer, so CanRun holds it to 1.5× stricter speed thresholds. The RTX 5070 clears the stricter Runs great line; the RTX 3060 12GB stays at Runs well.

From 12GB to 16GB: gpt-oss-20b on the GPU alone

gpt-oss-20b is a mixture-of-experts (MoE) model: each layer holds many expert blocks, and only a few run for each token. It needs 12.3 GB at 8k, more than a 12GB card can use. On the RTX 3060 12GB and the RTX 5070, CanRun's calculation keeps the expert weights that do not fit in system RAM, and both cells read Runs well as theoretical estimates. Every 16GB card in the grid holds it fully on the GPU, where it Runs great, in line with its model card, which says it runs within 16GB of memory.

The RTX 5070 has faster memory than the RTX 5060 Ti 16GB (672 GB/s against 448 GB/s), and our price data listed it higher when this guide was written, yet it is the card that has to split gpt-oss-20b. Our page for gpt-oss-20b on the GeForce RTX 5070 shows the split.

From 16GB to 24GB: dense models around 30B

In the grid, the dense Qwen3.8 27B is the model that separates the 16GB cards from the 24GB RTX 3090; the section on a new 16GB card or a 24GB RTX 3090 looks at it closely.

Qwen3.5 35B-A3B is another MoE model, one that uses about 3B of its parameters per token. On every card up to 16GB in the grid, it Runs well with part of its experts in system RAM, a theoretical estimate. On the RTX 3090 it needs 22.3 GB against 23.4 GB usable, so it fits fully with a little over a gigabyte to spare and Runs great; with that margin, confirm 100% GPU in ollama ps.

32GB: more room, and a few larger models

For these six models, the RTX 5090 gets the same 8k verdicts as the RTX 3090. Across the whole catalog, the summary table shows a few changes. Nemotron 3.5 Lightning 30B-A3B needs 25.1 GB, so it fits on the GPU alone only at 32GB. Most of the extra room goes to longer context. gpt-oss-120b fits on no card in the grid with 32GB of system RAM; the section on system RAM below covers it.

Capacity or speed: the RTX 3060 12GB against 8GB cards

The GeForce RTX 3060 12GB launched in 2021 and the GeForce RTX 5060 in 2025, yet our price data listed the older card lower when this guide was written. The older card has more memory: the 12GB card can use 11.4 GB, each 8GB card 7.4 GB.

That gap decides the 12B and 14B dense models. Gemma 3 12B Runs great on the RTX 3060 12GB but Runs slowly on the RTX 4060 and the RTX 5060, where part of it sits in system RAM. Qwen3 14B Runs well on the 12GB card and Runs slowly on both 8GB cards. Gemma 4 12B needs 7.8 GB, more than an 8GB card can use; with at least 90% of it still on the GPU, it Runs well on both 8GB cards and Runs great on the RTX 3060 12GB. At IQ4_XS it fits fully on the GPU of both 8GB cards with little margin, so check ollama ps; it then Runs great on the RTX 5060 and Runs well on the RTX 4060.

For a model that fits, speed follows memory bandwidth. The RTX 3060 12GB has 360 GB/s, the RTX 4060 272 GB/s and the RTX 5060 448 GB/s. Public benchmarks of Llama 3.1 8B put the older RTX 3060 12GB ahead of the RTX 4060. Both come from LocalScore, a Mozilla Builders benchmark project, at a 4k context; they are not CanRun runs.

With experts in system RAM, the RTX 5060's faster memory raises the estimate: Gemma 4 26B-A4B only just clears the stricter Runs well line on the RTX 4060 and clears it more comfortably on the RTX 5060, so both Run well as theoretical estimates. Gemma 4 gets the thinking-mode thresholds even though its thinking is off by default. If 12B and 14B dense models are on your list, aim for 12GB or more.

NVIDIA's specifications list the RTX 3060 in two memory versions, 12GB on a 192-bit bus and 8GB on a 128-bit bus. The 8GB version is not the card in these tables, so check the memory on the box. The RTX 5060 Ti has the same trap, with 8GB and 16GB versions in the list above.

A new 16GB card or a 24GB RTX 3090

This is the choice between the GeForce RTX 5060 Ti 16GB at $569 street (as of 2026-08-09) and the GeForce RTX 3090 at $1,050 street (as of 2026-08-09), which may be a used price, as the section on prices explains.

Up to gpt-oss-20b, the two cards get the same verdicts. gpt-oss-20b Runs great on both, and both cells are public benchmarks from llama.cpp's gpt-oss guide thread; Qwen3 14B Runs great on both too. The RTX 3090 generates tokens faster (936 GB/s of bandwidth against 448 GB/s), but both cards reach Runs great for these models; for Qwen3 14B the RTX 5060 Ti 16GB only just clears the stricter thinking-mode line.

The difference is dense models around 30B. Qwen3.8 27B needs 17.6 GB at 8k, against 15.4 GB usable on the 16GB card. The bar shows where the rest goes:

The part that does not fit runs from system RAM at system RAM speed, so Qwen3.8 27B Runs slowly on the 16GB card and Runs great on the RTX 3090. Gemma 4 31B splits the same way.

On a 16GB card, a smaller quant narrows the gap to the RTX 3090. The ladder shows each quant of Qwen3.8 27B on the RTX 5060 Ti 16GB; every cell is calibrated.

IQ4_XS sits in the same Good balance quality group as Q4_K_M. It still leaves a little of the model in system RAM, but with at least 90% on the GPU it Runs well. Q2_K fits fully on the GPU, but it belongs to the Heavy loss group; GGUF quantization: Q4_K_M vs IQ4_XS vs Q8_0 — which to download explains the groups.

24GB also leaves room for context. Ollama picks its default context length from the GPU memory it finds: 4k tokens below 24 GiB, 32k tokens from 24 to 48 GiB, and 256k tokens at 48 GiB or more (Ollama docs, checked 2026-09-29). A 24GB card therefore starts at 32k, and in CanRun's calculation Qwen3.8 27B still fits fully on the RTX 3090 there. With an f16 KV cache it stays on the GPU up to 64k, where it needs 21.9 GB. At 128k an f16 cache no longer fits on the GPU; the table shows what a q8_0 or q4_0 cache changes, and every cell is calibrated.

The two cards also differ in power. NVIDIA's reference specifications list the RTX 3090 at 350 W of graphics card power with a 750 W system power supply, and the RTX 5060 Ti at 180 W with a 600 W supply. Partner cards can differ, so check your power supply and case before choosing the larger card.

Same memory at different prices

Among the 16GB cards with a street price in our data, memory bandwidth runs from 322 GB/s on the RX 9060 XT 16GB to 960 GB/s on the RTX 5080, with the RTX 5060 Ti 16GB at 448 GB/s and the RTX 5070 Ti at 896 GB/s. Each of them can use 15.4 GB, so they fit the same models.

Speed changes the verdict only for models near a line. Qwen3 14B, with its stricter thinking-mode thresholds, Runs well on the RX 9060 XT 16GB and Runs great on the RTX 5060 Ti 16GB, the RX 9070 XT and the RTX 5070 Ti.

AMD and Intel cards

The same rule holds across brands: the Radeon RX 9060 XT 16GB and the Radeon RX 9070 XT fit what the other 16GB cards fit, and the Intel Arc B580 what the other 12GB cards fit.

CanRun's speed estimates for AMD cards carry the same ±12% band as NVIDIA cards for dense models fully on the GPU; for Intel Arc the band is ±20%. Software support is the bigger difference. Ollama's GPU documentation lists the RX 9070 XT and the RX 9060 XT among the Radeon cards it supports through ROCm on Linux, but not in its Windows ROCm list. For other cards, the same page says Ollama adds support through Vulkan, which is on by default (checked October 2026). So on Windows, Ollama runs these two cards through Vulkan. llama.cpp's Vulkan benchmark thread has public results for the RX 9070 XT and the Arc B580.

Upgrading a PC you already have: system RAM and MoE models

If you already own a card, more system RAM can open up MoE models before a new GPU does. CanRun's calculation keeps as many experts as fit in VRAM and the rest in system RAM, and reads only the experts each token uses at RAM speed. llama.cpp's --n-cpu-moe makes this split, and current llama.cpp and Ollama versions make it on their own; Running MoE models bigger than your GPU: gpt-oss-120b with system RAM works through the RAM each card needs.

With 32GB of system RAM, Qwen3.5 35B-A3B Runs well even on the 8GB RTX 5060, as a theoretical estimate. With 16GB of RAM it does not fit on that card. The RTX 5060 Ti 16GB holds more of the model in VRAM, so in CanRun's calculation it still Runs well with 16GB of RAM, again a theoretical estimate.

gpt-oss-120b is where RAM decides. Its file is 63.4 GB, so with 32GB of RAM it fits on no card in the grid. With 64GB, CanRun's calculation fits it on every card in the grid from 12GB up. It then Runs slowly on the RTX 3060 12GB and on the RTX 3090 alike, a theoretical estimate in both cases. Of the cards in the grid, only the RTX 5090 reaches Runs well with 64GB, also theoretical. On the 8GB RTX 5060 it falls just short of fitting with 64GB in CanRun's calculation, so CanRun lists 96GB, where it Runs slowly, again a theoretical estimate.

So if gpt-oss-120b is the goal, count your system RAM before you price a new card. RAM speed counts as well: CanRun assumes dual-channel DDR5-5600, and slower RAM reads the experts more slowly. Treat speeds with experts in system RAM as theoretical estimates (±30%): against 35 public benchmarks on NVIDIA cards this part of the calculation was off by about 10% on average, but by close to 40% in some setups.

Where to go next

• How much VRAM do local LLMs need? 8–32 GB tiers (2026): the memory formula and every VRAM tier in detail.
• GGUF quantization: Q4_K_M vs IQ4_XS vs Q8_0 — which to download: what IQ4_XS, Q4_K_M and Q8_0 cost and what they keep.
• Ollama slow or forgetting your prompt? Context length defaults: Ollama's default context and how to change it.
• Card pages such as the GeForce RTX 3060 12GB, the GeForce RTX 5060 Ti 16GB and the GeForce RTX 3090 list every model on each card.

How sure are these numbers?

Memory figures come by formula from real GGUF file sizes and each model's published configuration, so they carry no error band. Prices are the least stable numbers here: street prices change, and ours are a dated snapshot.

Speeds are estimates with a band and a label. Measured means a public benchmark for that card, model and quant. Calibrated means ±12% for dense models fully on an NVIDIA or AMD card, and ±20% for MoE models, partial offload and other hardware such as Intel Arc. Theoretical means ±30%, for setups whose real speed varies widely, such as MoE experts in system RAM.

All verdicts assume Windows with the card driving the display, 32GB of DDR5-5600 dual-channel system RAM, an 8k context, an f16 KV cache, one model and one request at a time. Ollama and llama.cpp make their own memory estimates, so a model that fits with little margin here can end up partly in system RAM; the PROCESSOR column of ollama ps shows what happened.

Models with a thinking mode face 1.5× stricter speed thresholds, which the legend below does not show: here gpt-oss-20b and gpt-oss-120b, which always think, and Qwen3 14B, Qwen3.8 27B, Qwen3.5 35B-A3B and Gemma 4, which can switch it on and off. Llama 3.1 8B and Gemma 3 12B have none. The thresholds:

FAQ

Which matters more for a local LLM GPU, VRAM or speed?

VRAM, up to the point where the models you want fit. The RTX 5070 has faster memory than the RTX 5060 Ti 16GB, yet gpt-oss-20b runs fully on the GPU only on the 16GB card. Once a model fits, speed takes over: Qwen3 14B fits on both the RX 9060 XT 16GB and the RTX 5070 Ti, but Runs well on the first and Runs great on the second.

Is 16GB enough, or should I aim for 24GB?

16GB is enough if your models stop at gpt-oss-20b and 14B dense models; aim for 24GB if you want dense models around 30B on the GPU alone. gpt-oss-20b Runs great on the RTX 5060 Ti 16GB. Qwen3.8 27B needs 17.6 GB, so it Runs slowly on 16GB cards and Runs great on the 24GB RTX 3090; on a 16GB card, IQ4_XS brings it up to Runs well.

Can a cheap GPU with plenty of system RAM run large models?

MoE models, often yes; large dense models, only slowly. With 32GB of RAM, CanRun's calculation fits Qwen3.5 35B-A3B on the 8GB RTX 5060 by keeping most of its experts in system RAM; with 16GB it does not fit. The dense Qwen3.8 27B Runs slowly on that card, with much of it in system RAM. For gpt-oss-120b, CanRun's calculation needs 64GB of RAM on a 12GB or larger card and lists 96GB on an 8GB card, where 64GB falls just short. Check what your runtime did with ollama ps or the llama.cpp log.

Do AMD and Intel cards run the same models as NVIDIA cards?

At the same memory size, they fit the same models in CanRun's calculation; speed and software setup differ. gpt-oss-20b Runs great on the RX 9060 XT 16GB, as on the RTX 5060 Ti 16GB, and Qwen3 14B Runs well on the Arc B580, as on the RTX 3060 12GB. Before buying, check that your runtime supports the card on your OS.

Sources

• NVIDIA: GeForce RTX 3060 family specifications: the RTX 3060 in 12GB (192-bit) and 8GB (128-bit) memory versions.
• NVIDIA: GeForce RTX 3090 and 3090 Ti specifications: 24GB, graphics card power and required system power for the reference design.
• NVIDIA: GeForce RTX 5060 family specifications: RTX 5060 Ti memory options, total graphics power and required system power for the reference design.
• Ollama docs: hardware support: Radeon cards supported through ROCm on Windows and on Linux, and Vulkan support on by default.
• Ollama docs: context length: the default context by GPU memory.
• Ollama FAQ: the PROCESSOR column of ollama ps.
• llama.cpp: server README: --n-cpu-moe N, which keeps the MoE weights of the first N layers on the CPU, and --fit (on by default), which places as much as fits on the GPU automatically.
• gpt-oss-20b model card: total and active parameters, runs within 16GB of memory.
• Qwen3.5-35B-A3B model card: total and activated parameters.
• llama.cpp: guide to running gpt-oss: llama-bench results on RTX cards, the source of the measured gpt-oss-20b cells.
• llama.cpp: Vulkan performance thread: public llama-bench results for Radeon and Arc cards through the Vulkan backend.
• LocalScore: GeForce RTX 3060 and LocalScore: GeForce RTX 4060: the measured Llama 3.1 8B results.
• Street prices in our GPU data, each recorded on the date shown with it: the RTX 50 series from an MLQ.ai news article on GPU prices; the RTX 3060 12GB and the Radeon and Arc cards from Tom's Hardware's GPU section, an index of reviews and buying guides that changes over time and does not show the recorded figures today; and the RTX 3090 and RTX 4090 from Best Value GPU's new and used price histories (RTX 3090, RTX 4090).