Running LLMs without a GPU: CPU-only speed and how much RAM you need

Yes. Without a graphics card, llama.cpp and the tools built on it run a model on the CPU from system RAM. Llama 3.1 8B at Q4_K_M fits in 16GB of RAM and Runs slowly on dual-channel DDR5-5600, a calibrated estimate. gpt-oss-20b, an MoE model, needs 32GB, yet CanRun estimates it faster than Llama 3.1 8B, because each token reads only a small part of it. The speed depends mostly on how fast your RAM is.

Which models run on a CPU alone

The grid puts six models on CanRun's three CPU-only profiles, ordered from the least memory bandwidth to the most: dual-channel DDR4-3200, DDR5-5600 and DDR5-6000. Every cell assumes an 8k context, an f16 KV cache and 4GB of RAM kept for the system and other programs. The 32GB in the caption applies to graphics cards only. In each cell, the last line gives the RAM size the verdict was worked out for: the smallest of 16, 32, 64 and 96GB that reaches it, or the largest CanRun checks when nothing works.

Two rules shape this grid. First, CanRun caps every CPU-only setup at Runs slowly, however high its estimate; Runs great and Runs well need a GPU. The badge therefore says only whether a model runs, and the speed band tells the cells apart. Llama 3.2 3B on DDR5-5600 has an estimate well above the speed Runs well asks of a graphics card, and it is still rated Runs slowly. Second, a model is rated Won't run when it does not fit in RAM or when its estimate falls under the floor in the legend at the end of this guide. Models with a thinking mode face a floor 1.5× higher, because they write many more tokens per answer.

Small dense models: 3B to 12B

Llama 3.2 3B, Llama 3.1 8B and Gemma 3 12B fit in 16GB of RAM and run on all three profiles, rated Runs slowly. Their speed bands are what differ. A dense model reads its whole file for every token, so the Gemma 3 12B file, at 7.3 GB, makes it slower than Llama 3.1 8B at 4.9 GB, and Llama 3.2 3B, the smallest of the three, is the quickest.

Thinking models and DDR4

Qwen3 14B thinks before it answers by default, so its floor is 1.5× higher. On DDR4-3200 its estimate falls under that floor and it Won't run at Q4_K_M. For that setup CanRun points to Q2_K, a Heavy loss quant, which Runs slowly. On DDR5-5600 the faster RAM lifts Q4_K_M over the floor, and it Runs slowly.

Dense models from 27B up

Gemma 3 27B needs 32GB of RAM and only just clears the floor on DDR5-5600, where it Runs slowly. On DDR4-3200 it Won't run at Q4_K_M. Qwen3 32B thinks by default, so the 1.5× floor applies; at Q4_K_M its estimate falls under that floor on all three profiles, and it is rated Won't run. On DDR5, CanRun points to Q2_K instead, which Runs slowly. Llama 3.3 70B Won't run even with 64GB, which holds its file. Having the RAM to load a model is not the same as running it at a usable pace.

What to install

You do not need special software. The llama.cpp README lists AVX, AVX2, AVX512 and AMX support for x86 processors. LM Studio's system requirements say an x64 Windows PC needs a processor with AVX2. In Ollama, ollama ps shows 100% CPU under PROCESSOR, which the Ollama FAQ describes as a model loaded entirely in system memory. CanRun's page for each model on a CPU profile prints a llama.cpp command with -ngl 0, which keeps every layer off the graphics card.

RAM bandwidth sets the speed: DDR4, DDR5 and dual channel

To write one token, the processor reads every weight the model uses for that token, plus the KV cache, the memory that holds the conversation so far. Memory bandwidth, how many gigabytes per second the processor can read from RAM, limits how often it can do that. CanRun's estimate is the RAM's rated bandwidth times an efficiency factor, divided by the bytes read per token. For a dense model on a CPU, CanRun counts on half of the rated bandwidth, a factor fitted to public benchmarks for processors with eight or more cores.

Dual-channel DDR4-3200 reads 51 GB/s, DDR5-5600 reads 90 GB/s and DDR5-6000 reads 96 GB/s. Going from DDR4-3200 to DDR5-5600 adds about three-quarters to the bandwidth, and CanRun's estimates rise by the same ratio. Llama 3.1 8B Runs slowly on both, but its band on the DDR5 row sits well above the DDR4 one. DDR5-6000 adds less than a tenth over DDR5-5600. A graphics card reads its own memory much faster: the RTX 3060 12GB reads 360 GB/s, about four times dual-channel DDR5-5600.

One stick or two

All three profiles assume dual channel. Mainstream desktop processors have two memory channels; Intel's specification for the Core Ultra 9 285K, for example, lists two. Corsair's guide to dual-channel RAM describes a single stick as single-channel and dual channel as twice its speed. With one stick, the bandwidth halves, and in CanRun's formula so does the estimate for every model. If your PC has one stick, a second matching stick in the slot your motherboard manual names doubles the bandwidth in CanRun's calculation. Four sticks can lower the rated memory speed on some processors; Running MoE models bigger than your GPU: gpt-oss-120b with system RAM covers that case.

Smaller files are faster

Since a dense model reads its whole file for every token, the quant changes CPU speed directly. Llama 3.1 8B at Q8_0 is 8.5 GB against 4.9 GB at Q4_K_M. The ladder below shows each step on dual-channel DDR5-5600; every cell is calibrated.

At Q8_0 the estimate is more than a third lower than at Q4_K_M. IQ4_XS is a little quicker than Q4_K_M, and Q2_K is quicker still but belongs to the Heavy loss group. All four are rated Runs slowly: Q8_0 and Q4_K_M because their estimates are under the Runs well line, IQ4_XS and Q2_K because of the CPU-only cap. Q4_K_M, the usual default, sits in the Good balance group; for how the quality groups differ, see GGUF quantization: Q4_K_M vs IQ4_XS vs Q8_0 — which to download.

Dense and MoE models on a CPU

A mixture-of-experts (MoE) model splits most of each layer into many expert blocks, and a router picks a few of them for every token. The whole file has to sit in RAM, since the router can pick any expert, but each token reads only the active part. The table lists total and active parameters for the models in this section.

On dual-channel DDR5-5600, CanRun's estimate for gpt-oss-20b is more than twice the estimate for Llama 3.1 8B, although its file, 12.1 GB, is about two and a half times as large. Both are rated Runs slowly, gpt-oss-20b because of the CPU-only cap. The Qwen3 30B-A3B (2507) file, 18.6 GB, is nearly four times the size of Llama 3.1 8B's, and CanRun's estimate for it is close to double. A community figure for this profile, discussed in the last section, is higher still.

gpt-oss-120b with 96GB of RAM

The gpt-oss-120b file is 63.4 GB. With 96GB of DDR5-5600 it fits; the CPU-only cap keeps it at Runs slowly (a calibrated estimate), though CanRun's estimate for it is higher than for Llama 3.1 8B. With 64GB it does not fit after the 4GB CanRun keeps for the system. CanRun's DDR4-3200 profile stops at 64GB, so on that row it is rated Won't run. Both gpt-oss models always reason before they answer, so a reply costs more tokens than the answer you read, and at CPU speeds that wait adds up; see Thinking models on local hardware: the speed cost and how to turn thinking off.

An MoE model for 16GB

On a PC with 16GB, Kanana 1.5 15.7B-A3B, a Korean MoE model from Kakao, fits on DDR5-5600. The CPU-only cap holds it at Runs slowly, but its estimate is about two-thirds higher than Llama 3.1 8B's. Its file is 10.4 GB. The model supports 32k of context, but with 16GB of RAM CanRun fits only 8k with an f16 KV cache; 32k needs a q4_0 cache.

Ling-3.0 Tiny also fits in 16GB, and its estimate is well above Kanana's, but its layer type makes it a theoretical estimate (±30%).

Where MoE models lose ground

Every token also reads the KV cache, and for a model that reads only a small active part, the cache makes up a larger share of each read. At 32k with an f16 KV cache, CanRun's estimate for Qwen3 30B-A3B is a little over half of its 8k estimate, against about two-thirds for Llama 3.1 8B. The other limit is RAM size, which the next section covers.

How much RAM you need

On a CPU, the whole model file and its KV cache have to fit in RAM, next to the OS and whatever else is open. CanRun keeps 4GB for those and compares the rest with the file at your quant plus the KV cache at your context. There is nowhere else for the model to go: if the total does not fit, CanRun rates the model Won't run at that RAM size.

• Llama 3.1 8B: the file is 4.9 GB and the KV cache at 8k is 1.1 GB, so 16GB is enough.
• gpt-oss-20b: its model card says it runs within 16GB of memory, but a PC with 16GB of RAM also has to hold the OS. In CanRun's calculation it misses the 12GB left free by a small margin, so it needs 32GB.
• Qwen3 30B-A3B: the file alone is more than 16GB leaves free, so it needs 32GB.
• gpt-oss-120b: 96GB, as above.

CanRun lists no 8GB size. By the same rule, 8GB leaves 4GB free, enough for Llama 3.2 3B at Q4_K_M and 8k but not for Llama 3.1 8B.

Longer context needs more RAM

The KV cache grows with the context. The table shows Llama 3.1 8B on dual-channel DDR5-5600 with 16GB of RAM, the size its grid cell uses; every cell is calibrated.

At 32k with an f16 KV cache, the cache alone is 4.3 GB, and the model still fits in 16GB. At 64k it does not, unless you switch to a q8_0 (8-bit) KV cache, which takes about half the space and brings 64k back inside 16GB. The speed band falls as the context grows, because every token reads the whole cache: at 32k with an f16 cache, the estimate is about two-thirds of the 8k one. For how the cache grows on other models, see KV cache explained: how context length eats VRAM.

Coding agents need a long context

Coding agents send far longer prompts than a chat, and Ollama's documentation asks for at least 64k of context for them. On a CPU that costs RAM and speed. Llama 3.1 8B at 64k with an f16 KV cache no longer fits in 16GB, as the table shows. Qwen3 30B-A3B at 64k fits in 32GB, but CanRun's estimate there falls to about a third of its 8k estimate. Each agent turn also makes the model read instructions, files and command output before it writes, and that prompt reading is the step CanRun cannot estimate on a CPU. A CPU-only PC can run a coding agent, but expect long waits on every turn. For the context and memory that agents need, see Local models for OpenClaw and coding agents: context length and VRAM.

Integrated graphics, Macs and Strix Halo

Most desktop and laptop processors include integrated graphics. Microsoft's explanation of GPU memory in Task Manager says that on integrated GPUs, dedicated memory is system memory reserved for graphics, and that integrated and discrete GPUs alike can use shared system memory. So integrated graphics read the same dual-channel RAM as the processor. In a calculation driven by memory bandwidth, like CanRun's, they cannot generate faster than that RAM allows. CanRun has no profile for ordinary integrated graphics; for such a PC, pick the CPU profile that matches its RAM.

Some machines without a graphics card run models faster because their memory bus is wider. The Ryzen AI Max+ 395 (Strix Halo) reads 256 GB/s and the Apple M4 Pro reads 273 GB/s, about three times dual-channel DDR5-5600. Their GPUs use that memory, so CanRun rates them by GPU rules: Qwen3 30B-A3B Runs great on a 32GB Strix Halo, while on the DDR5-5600 CPU profile the CPU-only cap holds it at Runs slowly. These machines have their own guides: Running local LLMs on a Mac (M4, M5): how much unified memory? and Strix Halo vs DGX Spark vs Mac Studio for local LLMs: what 128 GB gets you.

Where to go next

• The CPU profile pages, DDR4-3200 dual-channel CPU, DDR5-5600 dual-channel CPU and DDR5-6000 dual-channel CPU, list every model in the catalog with the RAM size it needs.
• Running MoE models bigger than your GPU: gpt-oss-120b with system RAM: what an added graphics card changes for MoE models that live mostly in system RAM.
• How much VRAM do local LLMs need? 8–32 GB tiers (2026): what fits on graphics cards from 8GB to 32GB.
• Tokens per second for local LLMs: what's fast enough and what sets the speed: how to read and measure generation speed.

How sure are these numbers?

The memory figures are calculated by formula from real GGUF file sizes and each model's published configuration, so they carry no band. On a CPU, CanRun counts the whole file, the KV cache and the 4GB it keeps for the system.

The speeds are estimates with a band and a label. CPU-only cells are calibrated, ±20%, the band CanRun uses everywhere except dense models fully on NVIDIA and AMD graphics cards. Models with layer types CanRun's speed formula has not been checked against, such as LFM2.5 and Ling 3.0, are theoretical, ±30%. The CPU factor assumes a processor with eight or more cores and desktop DDR4 or DDR5 in dual channel. CanRun does not model core count, so a processor with fewer cores can land below the band.

CanRun's data has no CPU-only measurement with stated conditions for the models in this guide. The nearest public figure is a community result collected on hardware-corner.net: a range for a Q4 file of the original Qwen3 30B-A3B, which has the same architecture as the 2507 update, on dual-channel DDR5-5600. It names neither the processor, the runtime nor the context length, so CanRun does not count it as a measurement. Even the low end of that range is above CanRun's 8k estimate band for Qwen3 30B-A3B and above its 4k estimate, so for this model CanRun's CPU estimate may be on the low side.

CanRun gives no prompt-processing estimate for CPU-only setups. Reading the prompt depends on compute rather than bandwidth, so the rates here do not tell you how long a long prompt or a pasted document takes to read. llama-bench measures both: prompt processing with -p and generation with -n.

All of this assumes an 8k context filled to that length, an f16 KV cache, one model loaded, one request at a time and a llama.cpp-class runtime. Verdict names in the text refer to that baseline; for other contexts, read the tables. Models with a thinking mode face speed thresholds 1.5× higher, which the legend does not show. The thresholds:

FAQ

Can you run a local LLM without a GPU?

Yes. llama.cpp, LM Studio and Ollama all run models on the CPU from system RAM. Llama 3.1 8B at Q4_K_M fits in 16GB of RAM and Runs slowly on dual-channel DDR5-5600, a calibrated estimate; Runs slowly is the highest verdict CanRun gives a CPU-only setup. gpt-oss-20b, an MoE model with a small active part, gets a higher estimate than Llama 3.1 8B but needs 32GB of RAM.

How much RAM do I need to run a local LLM on the CPU?

Enough for the model file and its KV cache after about 4GB for the system. At 8k, 16GB covers Llama 3.1 8B and Qwen3 14B at Q4_K_M, 32GB covers gpt-oss-20b and Qwen3 30B-A3B, and gpt-oss-120b needs 96GB. A longer context adds to the KV cache; the context table above shows how much.

Do more CPU cores make generation faster?

Not in CanRun's calculation, once you have about eight cores: generation waits on RAM bandwidth, so CanRun ties the speed to the RAM. With fewer cores the speed can fall below the band; llama.cpp sets the thread count with -t. Reading the prompt depends on compute instead, so the processor matters more for that step, which CanRun does not estimate on a CPU.

Is DDR4 RAM enough for a local LLM?

For small models, yes, at a little over half the DDR5-5600 speed in CanRun's calculation. Llama 3.1 8B Runs slowly on dual-channel DDR4-3200. Qwen3 14B, which thinks by default, Won't run there at Q4_K_M but Runs slowly on DDR5-5600. CanRun's DDR4 profile goes up to 64GB, too little for gpt-oss-120b. If your DDR4 PC has 96GB or more, the same RAM rule lets it fit, but CanRun does not estimate that size.

Which models make sense on a CPU?

MoE models with a small active part, as long as they fit in your RAM: with 32GB, gpt-oss-20b and Qwen3 30B-A3B get higher estimates than Llama 3.1 8B; with 16GB, Kanana 1.5 15.7B-A3B fits. Small dense models such as Llama 3.2 3B also get high estimates. Larger dense models are slow: Gemma 3 27B only just clears the floor on DDR5-5600, and Qwen3 32B, which thinks by default, Won't run at Q4_K_M on any of the three profiles; on DDR5, CanRun points to Q2_K instead.

Sources

• llama.cpp README: plain C/C++ implementation with AVX, AVX2, AVX512 and AMX support for x86.
• llama.cpp: server README: -ngl/--n-gpu-layers (layers stored in VRAM) and -t/--threads.
• llama.cpp: llama-bench README: prompt processing (-p) and text generation (-n) tests.
• LM Studio: system requirements: AVX2 required on x64 Windows.
• Ollama docs: context length: at least 64k of context for agents and coding tools.
• Ollama FAQ: the PROCESSOR column of ollama ps, where 100% CPU means the model is loaded entirely in system memory.
• Corsair: what is dual-channel RAM: one stick runs single-channel; dual channel doubles the speed of single-channel.
• Intel Core Ultra 9 285K specifications: two memory channels.
• Microsoft DirectX blog: GPUs in the Task Manager: on integrated GPUs, dedicated memory is system memory reserved for graphics; shared memory is normal system memory.
• gpt-oss-20b model card: total and active parameters, MXFP4, runs within 16GB of memory.
• gpt-oss-120b model card: total and active parameters, MXFP4.
• Qwen3-30B-A3B-Instruct-2507 model card: total and activated parameters, experts per token.
• Qwen3-14B model card: thinking mode on by default.
• Kanana 1.5 15.7B-A3B Instruct model card: the first MoE model in Kakao's Kanana family, for Korean and English.
• hardware-corner.net: Qwen3 hardware requirements: community results for the original Qwen3 30B-A3B on CPU and system RAM, including the dual-channel DDR5-5600 range discussed above (no processor, runtime or context length given).