Local LLM guides
Practical guides to running LLMs on your own PC or Mac. Every number in them comes from the same data and estimates as our GPU, model and combo pages.
- How much VRAM do local LLMs need? GPU requirements for 8–32 GB (2026)
LLM VRAM requirements by card size: what fits on 8, 12, 16, 24 and 32 GB graphics cards, the memory formula, and how context length and quantization change it.
October 29, 2026
- Ollama slow or forgetting your prompt? Context length defaults
Why Ollama trims long conversations or slows down: its default context length by VRAM tier, how to raise it, and what a longer context costs in memory.
October 3, 2026
- GGUF quantization: Q4_K_M vs IQ4_XS vs Q8_0 — which to download
What each GGUF quant level costs in gigabytes, how we group them by quality, and when dropping one level turns a model that does not fit into one that runs.
October 3, 2026
- Korean open LLMs (EXAONE, Kanana, HyperCLOVA X SEED, Solar): hardware guide
Specs, licenses and what it takes to run LG EXAONE, Kakao Kanana, NAVER HyperCLOVA X SEED and Upstage Solar locally, with verdicts on common GPUs and Macs.
October 3, 2026
- Running local LLMs on a Mac (M4, M5): how much unified memory?
How much of a Mac’s unified memory a local model can use, which M4 and M5 configurations fit which models, and why memory bandwidth sets the speed.
October 29, 2026
- KV cache explained: how context length eats VRAM
Why a longer context needs more memory, how much each model’s KV cache costs per token, and what q8_0 and q4_0 KV cache quantization buy you.
October 25, 2026
- Recommended GPUs for local LLMs in 2026: what each budget runs
Which local LLMs each GPU memory size runs, 8GB to 32GB, with dated street prices: 12GB vs 8GB cards, a new 16GB card vs a 24GB RTX 3090, and when system RAM helps.
October 15, 2026
- Running MoE models bigger than your GPU: gpt-oss-120b with system RAM
How much system RAM gpt-oss-120b and other MoE models need on 8–32 GB cards, how llama.cpp and Ollama keep experts in RAM, and why RAM speed sets the pace.
October 15, 2026
- Local models for OpenClaw and coding agents: context length and VRAM
How much context OpenClaw and coding agents need on a local model, which coding models hold 64k on 16GB and 24GB GPUs, and why replies can start slowly.
October 25, 2026
- Ollama or LM Studio not using your GPU? Check the split and fix the offload
Find out where Ollama or LM Studio loaded your model, why it splits between CPU and GPU or ignores the GPU, and what to shrink or fix first.
October 15, 2026
- Strix Halo vs DGX Spark vs Mac Studio for local LLMs: what 128 GB gets you
What fits on a 128GB Strix Halo, DGX Spark or Mac Studio, how much memory each GPU can use, and why bandwidth sets the speed, with public gpt-oss-120b results.
October 23, 2026
- Ollama vs llama.cpp vs LM Studio: same engine, different defaults
Ollama and LM Studio run GGUF models on llama.cpp. How each tool’s default context, parallel slots and GPU split change memory and speed, and how to match them.
October 15, 2026
- Thinking models on local hardware: the speed cost and how to turn thinking off
Why thinking models such as Qwen3 and gpt-oss feel slower on the same GPU, why CanRun rates them 1.5× stricter, and how to turn thinking off in Ollama and llama.cpp.
October 22, 2026
- Tokens per second for local LLMs: what's fast enough and what sets the speed
What generation and prompt speed mean for local LLMs, the speed each CanRun verdict needs, why bandwidth and VRAM spills set the pace, and how to measure your own.
October 25, 2026
- AMD Radeon and Intel Arc for local LLMs: ROCm, Vulkan and what runs
Which local LLMs fit on Radeon RX 9000 and RX 7000 cards and the Intel Arc B580, how ROCm, Vulkan and SYCL differ on Windows and Linux, and why the speeds differ.
October 25, 2026
- Local LLMs on a gaming laptop: laptop GPUs, VRAM and speed
Which local LLMs run on RTX 40 and RTX 50 laptop GPUs, why a laptop chip can have less VRAM than its desktop namesake, and what power limits and system RAM change.
October 15, 2026
- Two GPUs for local LLMs: memory adds up, speed doesn't
A second GPU adds room for bigger local LLMs, and speed only when a model stops spilling into RAM. Two RTX 3060 12GB vs one RTX 3090, mixed cards, and setup.
October 15, 2026
- Running LLMs without a GPU: CPU-only speed and how much RAM you need
Which local LLMs run on a PC with no graphics card, how much system RAM each needs, and why RAM bandwidth and MoE models decide the speed.
October 15, 2026
- Gemma 4 12B vs 26B-A4B vs 31B: differences, VRAM and which size fits your GPU
How Gemma 4 12B, 26B-A4B and 31B differ in memory and speed on 8–32GB GPUs, why the MoE model can use system RAM, and what the Ollama gemma4 tags hold.
October 21, 2026
- Qwen3.6 vs Qwen3.8, 27B vs 35B-A3B: differences and the VRAM each needs
Qwen3.6 27B, Qwen3.8 27B and Qwen3.6 35B-A3B on 8–32GB GPUs: what changes between versions, dense vs MoE memory and speed, and which quant fits a 16GB card.
October 19, 2026
- Running DeepSeek locally: hardware requirements for the R1 distills and the full model
Which DeepSeek models run on a home PC: VRAM for the 8B, 14B and 32B distills, context memory for long reasoning, and the memory DeepSeek V4 Flash needs.
October 23, 2026
- Ollama and LM Studio system requirements: minimum specs, RAM and VRAM by model size
What Ollama and LM Studio need to install and run: OS, CPU and GPU requirements, how much RAM and VRAM each model size takes, and what to pull on 8–32GB cards.
October 21, 2026
- DGX Spark for local LLMs: what runs on 64GB, 128GB and two or four linked units
Which local LLMs run on a 64GB or 128GB DGX Spark, how much memory its GPU can use, and what linking two or four units with llama.cpp or vLLM changes.
October 23, 2026
- Local LLMs on an RTX 5060 Ti 16GB: what we measured with Ollama
Four 8B to 14B models measured with Ollama on an RTX 5060 Ti 16GB: how the test works, how the results compare with CanRun estimates, and what fits in 16GB.
October 25, 2026