AMD Radeon and Intel Arc for local LLMs: ROCm, Vulkan and what runs

In CanRun's calculation, a Radeon or Arc card fits the same models as an NVIDIA card with the same VRAM. gpt-oss-20b runs fully on the GPU and Runs great on the RX 9060 XT 16GB, as on the RTX 5060 Ti 16GB. Qwen3 14B fits fully on the 12GB Arc B580, as on the RTX 3060 12GB. Speed and software support differ: on Windows, Ollama uses ROCm only for the RX 7000 cards on its list and runs RX 9000 and Arc cards through Vulkan.

Radeon and Arc next to NVIDIA cards of the same size

A model runs fully on the GPU when its weights, its KV cache (the memory that holds the context) and a compute buffer (the runtime's working memory) fit in the VRAM the card can use. The brand plays no part in that sum. CanRun keeps the same amount back for the Windows desktop on every graphics card, so a 16GB Radeon and a 16GB GeForce can each use 15.4 GB, and a 12GB Arc and a 12GB GeForce can each use 11.4 GB.

The grid pairs each Radeon or Arc card with an NVIDIA card of the same memory size, one row after the other. Every verdict assumes an 8k context, an f16 KV cache, 32GB of DDR5 system RAM and Windows. Each cell shows the verdict, an estimated speed band and a confidence label; cells labelled theoretical get an extra note under the grid.

12GB: Arc B580 and RTX 3060 12GB

Qwen3 14B at Q4_K_M, the 4-bit quant CanRun uses as its baseline, needs 10.5 GB. That fits on both cards with less than a gigabyte to spare, and Qwen3 14B Runs well on both. With so little margin, check ollama ps after loading it: PROCESSOR should read 100% GPU. gpt-oss-20b needs 12.3 GB, a little more than either card can use. In CanRun's calculation the expert weights that do not fit go to system RAM, and both cells read Runs well as theoretical estimates.

Our data lists the Intel Arc B580 at $290 street (as of 2026-08-09) and the GeForce RTX 3060 12GB at $250 street (as of 2026-08-09).

16GB: RX 9060 XT 16GB, RX 9070 XT, RTX 5060 Ti 16GB and RTX 5070 Ti

gpt-oss-20b fits fully and Runs great on all four, in line with its model card, which says it runs within 16GB of memory. Qwen3 30B-A3B needs 19.8 GB, so on each of these cards part of its experts sit in system RAM, and it Runs well as a theoretical estimate. Gemma 3 27B, a dense model, needs 17.8 GB. Part of its weights go to system RAM (partial offload), and it Runs slowly on the RX 9070 XT just as on the RTX 5070 Ti.

Two more Radeon cards in our catalog share this memory size. The Radeon RX 9070 has the same memory bandwidth as the Radeon RX 9070 XT, so CanRun gives it the same generation estimates; Qwen3 14B Runs great on it. The older Radeon RX 7800 XT has nearly the same bandwidth (624 GB/s against 645 GB/s).

Our data lists the Radeon RX 9060 XT 16GB at $449 street (as of 2026-08-09), the GeForce RTX 5060 Ti 16GB at $569 street (as of 2026-08-09), the Radeon RX 9070 XT at $689 street (as of 2026-08-09) and the GeForce RTX 5070 Ti at $949 street (as of 2026-08-09).

20GB: RX 7900 XT

The Radeon RX 7900 XT sits between the 16GB and 24GB cards, with 19.4 GB usable. Gemma 3 27B, which spills over on 16GB cards, fits fully here and Runs great. Qwen3 30B-A3B misses by less than half a gigabyte: it needs 19.8 GB at Q4_K_M, so in CanRun's calculation a small share of its experts goes to system RAM and it Runs well, a theoretical estimate. One step down, IQ4_XS needs 17.6 GB, fits fully and Runs great as a calibrated estimate. IQ4_XS stays in the same Good balance quality group as Q4_K_M, as the ladder shows.

The dense Qwen3 32B needs 22.0 GB, more than this card can use, so it runs partly from system RAM and Runs slowly.

24GB: RX 7900 XTX and RTX 3090

The Radeon RX 7900 XTX and the GeForce RTX 3090 can each use 23.4 GB. Qwen3 30B-A3B fits fully and Runs great on both. Qwen3 32B needs 22.0 GB at 8k and fits on both with a little over a gigabyte to spare. Its verdicts differ between the two cards, and the speed section below explains why. With 32GB of system RAM, gpt-oss-120b fits on neither; for how much RAM it needs, see Running MoE models bigger than your GPU: gpt-oss-120b with system RAM.

The whole catalog, card by card

The table counts every model in CanRun's catalog by verdict for each card (the caption gives the total) and names the largest model that fits on the GPU alone. Two cards with the same memory size fit the same models, so their counts can differ only where speed moves a model across a verdict line. The counts change as models join the catalog.

ROCm, Vulkan and SYCL: what runs on Windows and Linux

Ollama and LM Studio both run GGUF models through llama.cpp, and llama.cpp reaches AMD and Intel cards mainly through three backends. ROCm is AMD's compute platform, which llama.cpp uses through its HIP backend. Vulkan is a graphics and compute API that works across vendors; Ollama's documentation notes that most Windows graphics drivers already include it. SYCL is the backend llama.cpp's documentation describes as designed primarily for Intel GPUs. The weights and KV cache are the same size whichever backend runs them, so CanRun's memory figures do not change. The runtime's own working memory can differ a little by backend, so near the limit, check ollama ps.

Ollama

Ollama's hardware page lists two sets of Radeon cards for ROCm (checked October 2026):

• Linux, with AMD's ROCm v7 driver: the RX 9070 XT, 9070 GRE, 9070, 9060 XT, 9060 XT LP and 9060, the RX 7000 cards from the 7600 up, the RX 6800 and faster RX 6000 cards, plus Radeon PRO and Ryzen AI chips.
• Windows, with a driver that supports ROCm v7 (HIP 7): only the RX 7900 XTX, 7900 XT, 7900 GRE, 7800 XT, 7700 XT, 7600 XT and 7600, plus the Radeon PRO W7500 to W7900.

For other GPUs Ollama uses Vulkan, which the same page says is enabled by default once its backend is installed, on Windows and Linux. So on Windows, Ollama runs the Radeon RX 9070 XT and the Radeon RX 9060 XT 16GB through Vulkan. The page does not name Intel Arc cards; they fall under the same Vulkan support, and for Linux the page links Intel's driver instructions.

To see what Ollama picked, look for the inference compute line in its server log, which names the library (ROCm, Vulkan or CUDA) for each GPU it found. For where the log lives and what to try when the card is not found, see Ollama or LM Studio not using your GPU? Check the split and fix the offload.

llama.cpp

Each llama.cpp release on GitHub carries separate Windows builds for Vulkan, ROCm and SYCL, next to the CUDA, CPU and other builds (checked on release b11630, October 2026). The Windows ROCm build covers more Radeon generations than Ollama's Windows list. Its release workflow targets RDNA 2 (RX 6000), RDNA 3 and RDNA 4 chips, including gfx1030 and gfx1031, which LLVM's processor table maps to the RX 6800 to RX 6900 XT and to the RX 6700 XT, and gfx1200 and gfx1201, the RX 9060 and RX 9070 families. Ollama runs RX 6000 and RX 9000 cards through Vulkan on Windows, so llama.cpp's ROCm build is the other option for them, though in llama.cpp's public results Vulkan generated faster on the RX 9070 XT (see "Which backend is faster?" below).

For Intel, llama.cpp's SYCL page lists the Arc B580 among its verified devices and Windows 11 as supported. Its Windows package includes the oneAPI libraries it depends on; you install Intel's GPU driver yourself. The releases also carry an OpenVINO build for Intel hardware, a backend llama.cpp's README lists as in progress.

Unzip a build, open PowerShell in that folder, and check which devices the build can see before you load a model. The second line is the command our combo pages print for gpt-oss-20b on a 16GB card. On Linux, run ./llama-server from that folder instead.

.\llama-server.exe --list-devices
.\llama-server.exe -hf ggml-org/gpt-oss-20b-GGUF:MXFP4 -c 8192 -ngl all -fa on

If --list-devices shows no GPU, that build cannot use your card; try the Vulkan build instead. If it lists more than one device, for example Radeon graphics built into the processor as well as your card, add -dev with your card's name from that list (such as -dev Vulkan1).

LM Studio

LM Studio ships its llama.cpp engine as separate runtimes for the CPU, CUDA, Vulkan, ROCm and Metal. Press Ctrl+Shift+R to open the runtimes tab, then pick the Vulkan or ROCm runtime for a Radeon card, or the Vulkan runtime for an Arc card.

Which backend is faster?

CanRun does not model the backend; its dense-model efficiency figures for Radeon and Arc cards come from public llama-bench results run through Vulkan. llama.cpp's own scoreboards give a rough answer for Llama 2 7B: on the RX 9070 XT, Vulkan generated about 30% faster than ROCm, and on the RX 7900 XTX about a tenth faster, while prompt speed was about the same on both cards. The runs come from different builds and machines, so measure your own card: load the same GGUF with each build and compare the eval time line that llama-server prints after each reply. For which figures to compare, see Tokens per second for local LLMs: what's fast enough and what sets the speed. Public ROCm results for gpt-oss-20b are discussed under "How sure are these numbers?".

Same VRAM, different speed: RX 9070 XT vs RTX 5070 Ti, RX 7900 XTX vs RTX 3090

Once a model fits, generation speed follows memory bandwidth, because every new token reads the active weights and the KV cache from VRAM. CanRun multiplies each card's bandwidth by an efficiency figure per vendor, fitted to public llama-bench results. The figure for Radeon cards is nearly a tenth higher than for NVIDIA cards; the one for Intel Arc is about a sixth lower.

RX 9070 XT and RTX 5070 Ti

The RX 9070 XT has 645 GB/s against 896 GB/s on the RTX 5070 Ti, so CanRun expects it to generate about four-fifths as fast. Neither card sits near a verdict line for Qwen3 14B or gpt-oss-20b, and both models Run great on each.

llama.cpp's public benchmark results agree. In llama-bench runs of Llama 2 7B at Q4_0, a model outside our catalog, the RX 9070 XT on Vulkan reached about three-quarters of the RTX 5070 Ti's speed on CUDA, both in generation and in prompt processing. These scoreboards come from different threads, builds and backends, so here and below, read them as a rough sign rather than a head-to-head test.

RX 9060 XT 16GB and RTX 5060 Ti 16GB

The gap is wider in the cheaper pair: 322 GB/s on the RX 9060 XT 16GB against 448 GB/s on the RTX 5060 Ti 16GB. Qwen3 14B Runs well on the RX 9060 XT 16GB and Runs great on the RTX 5060 Ti 16GB. Qwen3 14B can think before it answers, which costs many more tokens per reply, so CanRun holds it, like every model with a thinking mode, to speed thresholds 1.5× higher than the legend at the end shows. The RTX 5060 Ti 16GB's estimate only just clears that line, and the RX 9060 XT 16GB's falls below it. Phi-4, a 14B model without a thinking mode, Runs great on both.

RX 7900 XTX and RTX 3090

These two have nearly the same bandwidth (960 GB/s and 936 GB/s), so the vendor efficiency figure decides the order. Qwen3 32B Runs great on the RX 7900 XTX and Runs well on the RTX 3090. Here too the thinking-mode line falls between the two estimates, and their ±12% bands overlap. Read the split as two cards on either side of a line, not as a measured difference.

The public results agree on generation and disagree on prompts. In the same Llama 2 7B runs, the RX 7900 XTX on Vulkan generated about a sixth faster than the RTX 3090 on CUDA, but processed the prompt more than a quarter slower. Results for Radeon cards are still thin on two points: prompt speed, covered two sections down, and MoE models, whose one Radeon result is shown under "How sure are these numbers?".

Intel Arc B580: 12GB and the limits of the MoE estimate

The Intel Arc B580 has 12GB like the GeForce RTX 3060 12GB, with more memory bandwidth (456 GB/s against 360 GB/s).

CanRun's Arc efficiency figure, fitted to the B580's public Vulkan result, is lower than the NVIDIA one. The two effects nearly cancel, so the B580's dense estimates land close to the RTX 3060 12GB's.

Phi-4 Runs great on the B580. It needs 11.0 GB at 8k, which fits with little margin, so check ollama ps. Qwen3 14B Runs well, held back by the same thinking-mode line. At 16k with an f16 KV cache it no longer fits fully, and part of it runs from system RAM; a q8_0 (8-bit) KV cache brings 16k back onto the GPU. That trade is covered in KV cache explained: how context length eats VRAM.

MoE models on Arc

CanRun has no public measurement of an MoE model on an Arc card. For MoE models it adds a fixed time per token for every expert in every layer, fitted to public benchmarks for NVIDIA, AMD and the other hardware classes. For Arc it borrows the largest of those values, so its MoE estimates on the B580 lean low. Every MoE cell on the B580 is therefore labelled theoretical (±30%), even when the whole model is on the GPU.

Kanana 1.5 15.7B-A3B, a Korean MoE model, shows the effect. At IQ4_XS it fits fully on the B580 and Runs great, a theoretical estimate. The same file on the RTX 3060 12GB also Runs great, as a calibrated estimate. CanRun's B580 estimate for it comes out about a tenth lower than the RTX 3060 12GB's, although for dense models the B580 comes out slightly ahead. That gap comes from the borrowed MoE value, not from anything measured.

gpt-oss-20b needs 12.3 GB, a little more than the B580 can use, so CanRun keeps the expert weights that do not fit in system RAM:

In that setup it Runs well, a theoretical estimate on two counts: experts in system RAM, and an MoE model on Arc.

Prompts on Arc

The B580's weaker side is reading the prompt. In llama.cpp's public Vulkan runs of Llama 2 7B, it processed the prompt at under a third of the speed the RTX 3060 12GB reached on CUDA, while generating only a little more slowly. llama.cpp's SYCL page reports that a build on oneAPI 2026.1 processed prompts about three times as fast on an Arc B570 as the earlier release build based on oneAPI 2025.3. If you work with long prompts on Arc, try the SYCL build as well as Vulkan.

Why prompt processing lags on Radeon and Arc

Generation reads memory one token at a time. Prompt processing reads the whole prompt in large batches, so the GPU's compute sets its pace instead (see Tokens per second for local LLMs: what's fast enough and what sets the speed). It matters most when the model reads a lot before it answers: a pasted document, a chat whose cache was cleared, or the instructions and tool definitions of a coding agent.

CanRun's combo pages show a rough prompt speed (±40%) when the whole model is on the GPU. It starts from the card's FP16 compute and is divided by the model's size, as Tokens per second for local LLMs: what's fast enough and what sets the speed explains. For Radeon cards it uses the FP16 matrix figure on AMD's specification pages, which for the RX 9000 series is twice the FP16 figure on our GPU pages and for the RX 7000 series the same, and the factor is fitted to public llama-bench results from 2026 llama.cpp builds on the RX 9060 XT, RX 9070 XT and RX 7900 XTX. In that rough estimate the RX 9070 XT reads prompts at about seven-tenths of the RTX 5070 Ti's speed, while it generates at about four-fifths, and the RX 7900 XTX at a little over half the RTX 3090's.

The Arc B580's prompt figure is the least certain. A 2025 Vulkan result for Llama 2 7B came in about a third below our estimate, and a 2026 result from a newer build and driver, on a model outside our catalog, was far faster. So treat the prompt figures on Radeon and Arc pages as a rough range. If long prompts matter to you, measure: ollama run with --verbose prints a prompt eval rate, and llama-server prints a prompt eval time line after each reply.

How sure are these numbers?

The memory figures are calculated by formula from real GGUF file sizes and headers and each model's published configuration. They are the same for every brand, so the fit results in this guide apply to Radeon and Arc cards exactly as to NVIDIA cards.

The speeds are estimates with a band and a label. Measured means a public benchmark for that card, model and quant. Calibrated means ±12% for dense models fully on an NVIDIA or AMD card and ±20% for MoE models, partial offload and other hardware, Intel Arc included. Theoretical means ±30%; in this guide it covers MoE experts in system RAM and every MoE model on the Arc B580.

The Radeon and Arc dense-model efficiency figures rest on public Vulkan results for Llama 2 7B, which is not in the catalog, and no Radeon or Arc cell in the grid is measured. For MoE models, llama.cpp's gpt-oss guide thread has two ROCm results for gpt-oss-20b (llama-bench at a 2k context; not CanRun runs). An RX 7900 GRE, which is not in our catalog, ran the same MXFP4 file CanRun uses, and CanRun's formula puts that card about a tenth faster than it measured. The result for the Radeon RX 7900 XT is much slower. Its poster converted the model themselves, into a file that stores the weights outside the experts at 16 bits, so each token reads more memory than with the MXFP4 file. CanRun therefore does not match it to the MXFP4 cell, and gpt-oss-20b on the RX 7900 XT is a calibrated estimate that Runs great. Even allowing for the larger file, that result is lower than CanRun's estimate, so until more results arrive, read CanRun's MoE speeds on Radeon cards as possibly high.

All of this assumes Windows with the card driving the display, 32GB of DDR5-5600 dual-channel system RAM, an 8k context, an f16 KV cache, one model loaded, one request slot and a llama.cpp-class runtime. The backend (ROCm, Vulkan or SYCL) is not part of the calculation. On your PC, the PROCESSOR column of ollama ps shows what actually happened: 100% GPU, or a CPU/GPU split.

Verdict names in the text refer to the 8k, f16 KV baseline. Models with a thinking mode, such as Qwen3 14B, Qwen3 32B and gpt-oss, face the 1.5× higher thresholds described in the speed section, which the legend does not show. The thresholds:

FAQ

Can a Radeon or Arc card run the same models as an NVIDIA card?

Yes, at the same memory size, in CanRun's calculation: fit depends only on the VRAM a card can use. gpt-oss-20b fits fully on the RX 9070 XT as on the RTX 5070 Ti, and Qwen3 30B-A3B fits fully on the 24GB RX 7900 XTX as on the RTX 3090. Speed and software support differ, as the sections above show.

Does Ollama use ROCm for Radeon RX 9000 cards on Windows?

No. As checked in October 2026, Ollama's Windows ROCm list covers only seven RX 7000 cards, from the RX 7600 to the RX 7900 XTX, and Radeon PRO W7000 cards. On Windows, Ollama runs RX 9000 cards such as the RX 9070 XT and RX 9060 XT through Vulkan, which is on by default; on Linux, they are on its ROCm list. llama.cpp's own Windows ROCm build does target RX 9000 chips.

Should I use ROCm or Vulkan?

Start with whatever your tool supports for your card, then measure if speed matters. In llama.cpp's public Llama 2 7B results, Vulkan generated faster than ROCm on the RX 9070 XT and the RX 7900 XTX, with about the same prompt speed. Ollama picks the backend for you; llama.cpp and LM Studio let you choose the build. CanRun's estimates are the same for both, so compare on your own card: load the same GGUF with each build and compare the speeds it prints. On Arc, add the SYCL build if your prompts are long.

Is a Radeon card faster than an NVIDIA card with the same memory bandwidth?

Slightly, for generation, in CanRun's calculation: its efficiency figure for Radeon cards is nearly a tenth higher, so Qwen3 32B Runs great on the RX 7900 XTX and Runs well on the RTX 3090, two estimates on either side of the thinking-mode line. In public Llama 2 7B results, the RX 7900 XTX generated about a sixth faster than the RTX 3090 but read prompts more than a quarter slower.

Is 12GB enough for 14B models on an Arc or Radeon card?

For the 14B models named here, yes at Q4_K_M with an 8k context, with little margin. On the Arc B580, Qwen3 14B, Phi-4 and HyperCLOVA X SEED Think 14B all fit fully on the GPU, Phi-4 with the least room, so check ollama ps. CanRun keeps the same amount back on any 12GB card, so a 12GB Radeon fits the same models. For 16k, a q8_0 KV cache keeps Qwen3 14B on the B580's GPU.

Sources

• Ollama docs: hardware support: the ROCm lists for Linux and Windows and their driver requirements, Vulkan enabled by default once installed on Windows and Linux, Vulkan included with most Windows drivers, and the link to Intel's Linux driver instructions.
• llama.cpp README: the supported backends, including HIP for AMD GPUs, SYCL for Intel GPUs and Vulkan.
• llama.cpp release b11630: Windows builds for Vulkan, ROCm, SYCL, CUDA and the CPU.
• llama.cpp release workflow at b11630: the GPU targets of the Windows ROCm build, including gfx1030, gfx1031, gfx1200 and gfx1201.
• LLVM: AMDGPU processors: gfx1030 for the RX 6800, RX 6800 XT and RX 6900 XT, gfx1031 for the RX 6700 XT, gfx1200 for the RX 9060 and RX 9060 XT, gfx1201 for the RX 9070, RX 9070 XT and RX 9070 GRE.
• llama.cpp: SYCL backend: designed primarily for Intel GPUs, the Arc B580 among verified devices, Windows 11 support, the Windows package with its oneAPI libraries, and the oneAPI 2026.1 prompt-speed note for the Arc B570.
• llama.cpp: server README: --list-devices and -dev (--device).
• llama.cpp source: server-context.cpp: the prompt eval time and eval time lines printed after each reply.
• LM Studio 0.3.9 release notes: the CPU, CUDA, Vulkan, ROCm and Metal llama.cpp runtimes and the runtimes tab.
• llama.cpp: performance with Vulkan: llama-bench results for Llama 2 7B at Q4_0 on the RX 7900 XTX, RX 9070 XT and Arc B580, the basis of CanRun's Radeon and Arc figures.
• llama.cpp: performance on AMD ROCm (HIP): the same benchmark on the RX 9070 XT and RX 7900 XTX through ROCm, compared here with the Vulkan results.
• llama.cpp: performance on NVIDIA CUDA: the same benchmark on the RTX 5070 Ti, RTX 3090 and RTX 3060 12GB.
• llama.cpp: guide to running gpt-oss: gpt-oss-20b results through ROCm on the RX 7900 XT, with a self-converted file, and on the RX 7900 GRE, with the MXFP4 file.
• gpt-oss-20b model card: total and active parameters, runs within 16GB of memory.
• AMD: Radeon RX 9070 XT specifications and Radeon RX 7900 XTX specifications: FP16 vector and FP16 matrix performance.
• llama.cpp issue #30009: a 2026 Vulkan prompt-processing result on the Arc B580, for Gemma 4 E4B at Q4_0.