Gemma 3 12B hardware requirements

Custom license · Gemma — check terms before commercial useChatVision

What is Gemma 3 12B?

Gemma 3 12B is an open model from Google DeepMind, released in March 2025 alongside 1B, 4B and 27B sizes. It reads text and images, handles a 128k context and was pretrained on over 140 languages, with over 35 supported out of the box. Its card lists chat, summarization and image data extraction as uses. Its license is the Gemma Terms of Use with a prohibited-use policy, and the Hugging Face weights are gated until you accept it.

Model card · Release post

About the model

Parameters: 12.2B
Attention: Sliding-window attention
Max context: 131,072 tokens
Released: 2025-03-12
Model card: Gemma 3 12B on Hugging Face

Other sizes in this series

Graphics card memory each model needs at its baseline quant with 8k context, and its verdict on common graphics cards, smallest model first
Model	Parameters	Baseline quant	Memory needed (8k)	GeForce RTX 4060	GeForce RTX 5060 Ti 16GB	GeForce RTX 3090
Gemma 3 12Bthis model	12.2B	Q4_K_M	8.4 GB	Runs slowly	Runs great	Runs great
Gemma 3 27B	27.4B	Q4_K_M	17.8 GB	Runs slowly	Runs slowly	Runs great

GGUF files

Tracked quantizations of Gemma 3 12B, smallest first
Quant	File size	Quality	Published by
Q2_K	4.77 GB	Heavy loss	Community GGUF · bartowski
IQ4_XS	6.55 GB	Good balance	Community GGUF · bartowski
Q4_K_Mbaseline	7.30 GB	Good balance	Community GGUF · bartowski
Q8_0	12.51 GB	Near-lossless	Community GGUF · bartowski

File sizes do not include the vision projector (mmproj) that image input needs.

Memory needed by quant and context

Graphics card memory in GB: weights loaded on the GPU + f16 KV cache + compute buffer. Add about 0.6 GB if the GPU also drives your display on Windows.
Quant	4k	8k	16k	32k	64k	128k
Q2_K	5.6	5.9	6.5	7.7	10.2	15.0
IQ4_XS	7.3	7.7	8.3	9.5	11.9	16.8
Q4_K_M	8.1	8.4	9.0	10.2	12.7	17.6
Q8_0	13.3	13.6	14.2	15.4	17.9	22.8

Which GPUs can run Gemma 3 12B?

How to read the verdicts

Runs great: Fully on the GPU at 20 tok/s or more
Runs well: Fully on the GPU at 8–20 tok/s; MoE experts in system RAM at 20 tok/s or more; or at least 90% on the GPU at 8 tok/s or more
Runs slowly: 2–8 tok/s; CPU-only; less than 90% on the GPU; or MoE experts in system RAM below 20 tok/s
Won't run: Does not fit, or under 2 tok/s

Dense models split between GPU and CPU slow down sharply — the CPU side sets the pace. MoE models that keep only their experts in system RAM degrade far more gently.

Sort

Filter this list by budget, type and context in the calculator

Scroll sideways to see every column.

58 GPUs, Macs and CPU setups at Q4_K_M with 8k context, sorted by verdict and speed
Hardware	Verdict	Speed	Quant	Memory	Runs as	Context	Notes	Price
GeForce RTX 5090	Runs greatGemma 3 12B verdict on the GeForce RTX 5090	est. 160.1 tok/s140.9–179.3calibrated estimate ±12%	Q4_K_M	9.0 / 32.0 GB	Full GPU	up to 128k	—	$4,200 street (as of 2026-08-09)
NVIDIA RTX PRO 6000 Blackwell	Runs great	est. 160.1 tok/s140.9–179.3calibrated estimate ±12%	Q4_K_M	9.0 / 96.0 GB	Full GPU	up to 128k	—	—
NVIDIA RTX PRO 6000 Blackwell Max-Q	Runs great	est. 160.1 tok/s140.9–179.3calibrated estimate ±12%	Q4_K_M	9.0 / 96.0 GB	Full GPU	up to 128k	—	—
Radeon RX 7900 XTX	Runs great	est. 93.1 tok/s81.9–104.3calibrated estimate ±12%	Q4_K_M	9.0 / 24.0 GB	Full GPU	up to 128k	—	$999 launch MSRP
GeForce RTX 3090 Ti	Runs great	est. 90.0 tok/s79.2–100.8calibrated estimate ±12%	Q4_K_M	9.0 / 24.0 GB	Full GPU	up to 128k	—	$1,999 launch MSRP
GeForce RTX 4090	Runs greatGemma 3 12B verdict on the GeForce RTX 4090	est. 90.0 tok/s79.2–100.8calibrated estimate ±12%	Q4_K_M	9.0 / 24.0 GB	Full GPU	up to 128k	—	$2,755 street (as of 2026-08-09)
Apple M5 Ultra · 96 GB	Runs great	est. 89.3 tok/s71.5–107.2calibrated estimate ±20%	Q4_K_M	8.4 / 67.2 GB	Unified memory	up to 128k	—	—
GeForce RTX 5080	Runs great	est. 85.7 tok/s75.5–96.0calibrated estimate ±12%	Q4_K_M	9.0 / 16.0 GB	Full GPU	up to 64k	—	$1,256 street (as of 2026-08-09)
GeForce RTX 3090	Runs greatGemma 3 12B verdict on the GeForce RTX 3090	est. 83.6 tok/s73.6–93.6calibrated estimate ±12%	Q4_K_M	9.0 / 24.0 GB	Full GPU	up to 128k	—	$1,050 street (as of 2026-08-09)
GeForce RTX 3080 12GB	Runs great	est. 81.5 tok/s71.7–91.2calibrated estimate ±12%	Q4_K_M	9.0 / 12.0 GB	Full GPU	up to 32k	—	$799 launch MSRP
GeForce RTX 5070 Ti	Runs great	est. 80.0 tok/s70.4–89.6calibrated estimate ±12%	Q4_K_M	9.0 / 16.0 GB	Full GPU	up to 64k	—	$949 street (as of 2026-08-09)
GeForce RTX 5080 Laptop	Runs great	est. 80.0 tok/s70.4–89.6calibrated estimate ±12%	Q4_K_M	9.0 / 16.0 GB	Full GPU	up to 64k	—	—
GeForce RTX 5090 Laptop	Runs great	est. 80.0 tok/s70.4–89.6calibrated estimate ±12%	Q4_K_M	9.0 / 24.0 GB	Full GPU	up to 128k	—	—
Radeon RX 7900 XT	Runs great	est. 77.6 tok/s68.3–86.9calibrated estimate ±12%	Q4_K_M	9.0 / 20.0 GB	Full GPU	up to 128k	—	$899 launch MSRP
GeForce RTX 3080 10GB	Runs great	est. 67.9 tok/s59.7–76.0calibrated estimate ±12%	Q4_K_M	9.0 / 10.0 GB	Full GPU	up to 16k	—	$699 launch MSRP
GeForce RTX 4080 Super	Runs great	est. 65.7 tok/s57.9–73.6calibrated estimate ±12%	Q4_K_M	9.0 / 16.0 GB	Full GPU	up to 64k	—	$999 launch MSRP
GeForce RTX 4080	Runs great	est. 64.0 tok/s56.4–71.7calibrated estimate ±12%	Q4_K_M	9.0 / 16.0 GB	Full GPU	up to 64k	—	$1,199 launch MSRP
Radeon RX 9070	Runs great	est. 62.6 tok/s55.0–70.1calibrated estimate ±12%	Q4_K_M	9.0 / 16.0 GB	Full GPU	up to 64k	—	$639 street (as of 2026-08-09)
Radeon RX 9070 XT	Runs great	est. 62.6 tok/s55.0–70.1calibrated estimate ±12%	Q4_K_M	9.0 / 16.0 GB	Full GPU	up to 64k	—	$689 street (as of 2026-08-09)
Radeon RX 7800 XT	Runs great	est. 60.5 tok/s53.3–67.8calibrated estimate ±12%	Q4_K_M	9.0 / 16.0 GB	Full GPU	up to 64k	—	$499 launch MSRP
GeForce RTX 4070 Ti Super	Runs great	est. 60.0 tok/s52.8–67.2calibrated estimate ±12%	Q4_K_M	9.0 / 16.0 GB	Full GPU	up to 64k	—	$799 launch MSRP
GeForce RTX 5070	Runs greatGemma 3 12B verdict on the GeForce RTX 5070	est. 60.0 tok/s52.8–67.2calibrated estimate ±12%	Q4_K_M	9.0 / 12.0 GB	Full GPU	up to 32k	—	$629 street (as of 2026-08-09)
GeForce RTX 5070 Ti Laptop	Runs great	est. 60.0 tok/s52.8–67.2calibrated estimate ±12%	Q4_K_M	9.0 / 12.0 GB	Full GPU	up to 32k	—	—
Apple M3 Ultra · 96 GB	Runs great	est. 59.6 tok/s47.7–71.5calibrated estimate ±20%	Q4_K_M	8.4 / 67.2 GB	Unified memory	up to 128k	—	—
Apple M5 Max (40-core GPU) · 48 GB	Runs great	est. 59.5 tok/s47.6–71.5calibrated estimate ±20%	Q4_K_M	8.4 / 33.6 GB	Unified memory	up to 128k	—	—
Apple M2 Ultra · 64 GB	Runs great	est. 58.2 tok/s46.5–69.8calibrated estimate ±20%	Q4_K_M	8.4 / 44.8 GB	Unified memory	up to 128k	—	—
GeForce RTX 2080 Ti	Runs great	est. 55.0 tok/s48.4–61.6calibrated estimate ±12%	Q4_K_M	9.0 / 11.0 GB	Full GPU	up to 32k	—	$999 launch MSRP
Apple M4 Max (40-core GPU) · 48 GB	Runs great	est. 52.9 tok/s42.4–63.5calibrated estimate ±20%	Q4_K_M	8.4 / 33.6 GB	Unified memory	up to 128k	—	—
GeForce RTX 4090 Laptop	Runs great	est. 51.4 tok/s45.3–57.6calibrated estimate ±12%	Q4_K_M	9.0 / 16.0 GB	Full GPU	up to 64k	—	—
GeForce RTX 5060 Ti 16GB	Runs greatGemma 3 12B verdict on the GeForce RTX 5060 Ti 16GB	50.3 tok/s8k estimate 40.0 tok/s (35.2–44.8, calibrated estimate ±12%)measured (1 run, 512 context)	Q4_K_M	9.0 / 16.0 GB	Full GPU	up to 64k	—	$569 street (as of 2026-08-09)
GeForce RTX 4070	Runs greatGemma 3 12B verdict on the GeForce RTX 4070	est. 45.0 tok/s39.6–50.4calibrated estimate ±12%	Q4_K_M	9.0 / 12.0 GB	Full GPU	up to 32k	—	$599 launch MSRP
GeForce RTX 4070 Super	Runs great	est. 45.0 tok/s39.6–50.4calibrated estimate ±12%	Q4_K_M	9.0 / 12.0 GB	Full GPU	up to 32k	—	$599 launch MSRP
Apple M5 Max (32-core GPU) · 36 GB	Runs great	est. 44.6 tok/s35.7–53.5calibrated estimate ±20%	Q4_K_M	8.4 / 25.2 GB	Unified memory	up to 128k	—	—
GeForce GTX 1080 Ti	Runs great	est. 43.2 tok/s30.3–56.2theoretical estimate ±30%	Q4_K_M	9.0 / 11.0 GB	Full GPU	up to 32k	—	$699 launch MSRP
Apple M4 Max (32-core GPU) · 36 GB	Runs great	est. 39.8 tok/s31.8–47.7calibrated estimate ±20%	Q4_K_M	8.4 / 25.2 GB	Unified memory	up to 64k	—	—
GeForce RTX 4080 Laptop	Runs great	est. 38.6 tok/s34.0–43.2calibrated estimate ±12%	Q4_K_M	9.0 / 12.0 GB	Full GPU	up to 32k	—	—
Intel Arc B580	Runs great	est. 34.3 tok/s27.5–41.2calibrated estimate ±20%	Q4_K_M	9.0 / 12.0 GB	Full GPU	up to 32k	—	$290 street (as of 2026-08-09)
GeForce RTX 3060 12GB	Runs greatGemma 3 12B verdict on the GeForce RTX 3060 12GB	est. 32.2 tok/s28.3–36.0calibrated estimate ±12%	Q4_K_M	9.0 / 12.0 GB	Full GPU	up to 32k	—	$250 street (as of 2026-08-09)
Radeon RX 9060 XT 16GB	Runs great	est. 31.2 tok/s27.5–35.0calibrated estimate ±12%	Q4_K_M	9.0 / 16.0 GB	Full GPU	up to 64k	—	$449 street (as of 2026-08-09)
Apple M5 Pro · 24 GB	Runs great	est. 29.8 tok/s23.8–35.7calibrated estimate ±20%	Q4_K_M	8.4 / 16.8 GB	Unified memory	up to 64k	—	—
NVIDIA DGX Spark · 128 GB	Runs great	est. 27.2 tok/s21.7–32.6calibrated estimate ±20%	Q4_K_M	8.4 / 116.9 GB	Unified memory	up to 32k	—	$3,999 launch MSRP (128 GB)
Ryzen AI Max+ 395 (Strix Halo) · 32 GB	Runs great	est. 26.8 tok/s21.4–32.1calibrated estimate ±20%	Q4_K_M	8.4 / 22.4 GB	Unified memory	up to 32k	—	$1,999 launch MSRP (128 GB)
Apple M4 Pro · 24 GB	Runs greatGemma 3 12B verdict on the Apple M4 Pro	est. 26.5 tok/s21.2–31.8calibrated estimate ±20%	Q4_K_M	8.4 / 16.8 GB	Unified memory	up to 32k	—	—
GeForce RTX 4060 Ti 16GB	Runs great	est. 25.7 tok/s22.6–28.8calibrated estimate ±12%	Q4_K_M	9.0 / 16.0 GB	Full GPU	up to 32k	—	$499 launch MSRP
Apple M5 · 16 GB	Runs well	est. 14.9 tok/s11.9–17.9calibrated estimate ±20%	Q4_K_M	8.4 / 11.2 GB	Unified memory	up to 32k	Q2_K (heavy quality loss): Runs great	—
Apple M6 · 16 GB	Runs well	est. 14.9 tok/s11.9–17.9calibrated estimate ±20%	Q4_K_M	8.4 / 11.2 GB	Unified memory	up to 32k	Q2_K (heavy quality loss): Runs great	—
Apple M4 · 16 GB	Runs wellGemma 3 12B verdict on the Apple M4	est. 11.6 tok/s9.3–14.0calibrated estimate ±20%	Q4_K_M	8.4 / 11.2 GB	Unified memory	up to 32k	—	—
GeForce RTX 3070	Runs slowly	est. 21.8 tok/s17.5–26.2calibrated estimate ±20%	Q4_K_M	8.0 / 8.0 GB + 1.0 GB RAM	Partial offload	up to 64k	Less than 90% on the GPU — a GPU/CPU split is capped at Runs slowly Try IQ4_XS: Runs well Q2_K (heavy quality loss): Runs great	$499 launch MSRP
GeForce RTX 5060	Runs slowly	est. 21.8 tok/s17.5–26.2calibrated estimate ±20%	Q4_K_M	8.0 / 8.0 GB + 1.0 GB RAM	Partial offload	up to 64k	Less than 90% on the GPU — a GPU/CPU split is capped at Runs slowly Try IQ4_XS: Runs well Q2_K (heavy quality loss): Runs great	$339 street (as of 2026-08-09)
GeForce RTX 5060 Ti 8GB	Runs slowly	est. 21.8 tok/s17.5–26.2calibrated estimate ±20%	Q4_K_M	8.0 / 8.0 GB + 1.0 GB RAM	Partial offload	up to 64k	Less than 90% on the GPU — a GPU/CPU split is capped at Runs slowly Try IQ4_XS: Runs well Q2_K (heavy quality loss): Runs great	$429 street (as of 2026-08-09)
GeForce RTX 4060 Ti 8GB	Runs slowly	est. 17.3 tok/s13.9–20.8calibrated estimate ±20%	Q4_K_M	8.0 / 8.0 GB + 1.0 GB RAM	Partial offload	up to 64k	Less than 90% on the GPU — a GPU/CPU split is capped at Runs slowly Try IQ4_XS: Runs well Q2_K (heavy quality loss): Runs great	$399 launch MSRP
GeForce RTX 4060	Runs slowlyGemma 3 12B verdict on the GeForce RTX 4060	est. 16.7 tok/s13.4–20.1calibrated estimate ±20%	Q4_K_M	8.0 / 8.0 GB + 1.0 GB RAM	Partial offload	up to 64k	Less than 90% on the GPU — a GPU/CPU split is capped at Runs slowly Try IQ4_XS: Runs well Q2_K (heavy quality loss): Runs great	$299 launch MSRP
GeForce RTX 4060 Laptop	Runs slowly	est. 16.2 tok/s12.9–19.4calibrated estimate ±20%	Q4_K_M	8.0 / 8.0 GB + 1.0 GB RAM	Partial offload	up to 64k	Less than 90% on the GPU — a GPU/CPU split is capped at Runs slowly Try IQ4_XS: Runs well Q2_K (heavy quality loss): Runs great	—
GeForce RTX 4070 Laptop	Runs slowly	est. 16.2 tok/s12.9–19.4calibrated estimate ±20%	Q4_K_M	8.0 / 8.0 GB + 1.0 GB RAM	Partial offload	up to 64k	Less than 90% on the GPU — a GPU/CPU split is capped at Runs slowly Try IQ4_XS: Runs well Q2_K (heavy quality loss): Runs great	—
GeForce RTX 3050 8GB	Runs slowly	est. 14.9 tok/s11.9–17.8calibrated estimate ±20%	Q4_K_M	8.0 / 8.0 GB + 1.0 GB RAM	Partial offload	up to 64k	Less than 90% on the GPU — a GPU/CPU split is capped at Runs slowly Try IQ4_XS: Runs well Q2_K (heavy quality loss): Runs great	$249 launch MSRP
DDR5-6000 dual-channel CPU · 16 GB	Runs slowly	est. 6.1 tok/s4.9–7.3calibrated estimate ±20%	Q4_K_M	7.8 GB RAM	CPU only	up to 64k	—	—
DDR5-5600 dual-channel CPU · 16 GB	Runs slowly	est. 5.7 tok/s4.6–6.9calibrated estimate ±20%	Q4_K_M	7.8 GB RAM	CPU only	up to 64k	—	—
DDR4-3200 dual-channel CPU · 16 GB	Runs slowly	est. 3.3 tok/s2.6–3.9calibrated estimate ±20%	Q4_K_M	7.8 GB RAM	CPU only	up to 64k	—	—

Cheapest GPUs that run it

Runs greatGeForce RTX 3060 12GB$250 street (as of 2026-08-09), est. 32.2 tok/s (28.3–36.0, calibrated estimate ±12%).

Measured results for Gemma 3 12B

Public benchmarks we calibrate against. Their conditions (context, backend, flags) can differ from the estimates above.

Measured results for Gemma 3 12B
Hardware	Quant	Backend	Context	Prompt (tok/s)	Generation (tok/s)	Flags	Source	Measured
GeForce RTX 5060 Ti 16GB	Q4_K_M	ollama	512	1,692.0	50.3	CUDA, Windows 11, num_ctx 8192	github.com	2026-10-25

Frequently asked questions

How much VRAM does Gemma 3 12B need?

At Q4_K_M the file is 7.30 GB; with 8k context a graphics card needs about 8.4 GB (weights 7.29 GB, KV cache 0.5 GB, compute buffer 0.6 GB). The smallest tracked file (Q2_K (heavy quality loss)) needs about 5.9 GB.

What is the cheapest GPU that runs Gemma 3 12B well?

Cheapest that runs it great: GeForce RTX 3060 12GB at $250 (street price as of 2026-08-09), est. 32.2 tok/s (28.3–36.0, calibrated estimate ±12%).

Can Gemma 3 12B run on an 8 GB, 16 GB or 24 GB GPU?

At Q4_K_M with 8k context: GeForce RTX 4060 — Runs slowly, est. 16.7 tok/s (13.4–20.1, calibrated estimate ±20%); GeForce RTX 5060 Ti 16GB — Runs great, 50.3 tok/s measured at 512 context (1 run; 8k estimate 40.0 tok/s, 35.2–44.8, calibrated estimate ±12%); GeForce RTX 3090 — Runs great, est. 83.6 tok/s (73.6–93.6, calibrated estimate ±12%).

Speeds are estimates from memory bandwidth, calibrated against public benchmarks, and each one comes with an error band and a confidence label. Real results vary with drivers, backend, context length and thermals.