Llama 3.2 3B hardware requirements
About the model
- Parameters
- 3.21B
- Attention
- Full attention
- Max context
- 131,072 tokens
- Released
- 2024-09-25
- Model card
- Llama 3.2 3B on Hugging Face
GGUF files
| Quant | File size | Quality | Published by |
|---|---|---|---|
| IQ4_XS | 1.83 GB | Good balance | Community GGUF · bartowski |
| Q4_K_Mbaseline | 2.02 GB | Good balance | Community GGUF · bartowski |
| Q8_0 | 3.42 GB | Near-lossless | Community GGUF · bartowski |
Memory needed by quant and context
| Quant | 4k | 8k | 16k | 32k | 64k | 128k |
|---|---|---|---|---|---|---|
| IQ4_XS | 2.8 | 3.3 | 4.4 | 6.4 | 10.4 | 18.6 |
| Q4_K_M | 3.0 | 3.5 | 4.5 | 6.6 | 10.6 | 18.8 |
| Q8_0 | 4.4 | 4.9 | 5.9 | 8.0 | 12.0 | 20.2 |
Which GPUs can run Llama 3.2 3B?
How to read the verdicts
- Runs great
- Fully on the GPU at 20 tok/s or more
- Runs well
- Fully on the GPU at 8–20 tok/s; MoE experts in system RAM at 20 tok/s or more; or at least 90% on the GPU at 8 tok/s or more
- Runs slowly
- 2–8 tok/s; CPU-only; less than 90% on the GPU; or MoE experts in system RAM below 20 tok/s
- Won't run
- Does not fit, or under 2 tok/s
Scroll sideways to see every column.
| Hardware | Verdict | Speed | Quant | Memory | Runs as | Context | Notes | Price |
|---|---|---|---|---|---|---|---|---|
| GeForce RTX 5090 | Runs great | est. 423.9 tok/s373.0–474.7calibrated estimate ±12% | Q4_K_M | 4.1 / 32.0 GB | Full GPU | up to 128k | — | $4,200 street (as of 2026-08-09) |
| Radeon RX 7900 XTX | Runs great | est. 253.0 tok/s222.7–283.4calibrated estimate ±12% | Q4_K_M | 4.1 / 24.0 GB | Full GPU | up to 128k | — | $999 launch MSRP |
| GeForce RTX 3090 Ti | Runs great | est. 238.4 tok/s209.8–267.0calibrated estimate ±12% | Q4_K_M | 4.1 / 24.0 GB | Full GPU | up to 128k | — | $1,999 launch MSRP |
| GeForce RTX 4090 | Runs great | est. 238.4 tok/s209.8–267.0calibrated estimate ±12% | Q4_K_M | 4.1 / 24.0 GB | Full GPU | up to 128k | — | $2,755 street (as of 2026-08-09) |
| GeForce RTX 5080 | Runs great | est. 227.1 tok/s199.8–254.3calibrated estimate ±12% | Q4_K_M | 4.1 / 16.0 GB | Full GPU | up to 64k | — | $1,256 street (as of 2026-08-09) |
| GeForce RTX 3090 | Runs great | est. 221.4 tok/s194.8–248.0calibrated estimate ±12% | Q4_K_M | 4.1 / 24.0 GB | Full GPU | up to 128k | — | $1,050 street (as of 2026-08-09) |
| GeForce RTX 3080 12GB | Runs great | est. 215.7 tok/s189.8–241.6calibrated estimate ±12% | Q4_K_M | 4.1 / 12.0 GB | Full GPU | up to 64k | — | $799 launch MSRP |
| GeForce RTX 5070 Ti | Runs great | est. 211.9 tok/s186.5–237.4calibrated estimate ±12% | Q4_K_M | 4.1 / 16.0 GB | Full GPU | up to 64k | — | $949 street (as of 2026-08-09) |
| GeForce RTX 5080 Laptop | Runs great | est. 211.9 tok/s186.5–237.4calibrated estimate ±12% | Q4_K_M | 4.1 / 16.0 GB | Full GPU | up to 64k | — | — |
| GeForce RTX 5090 Laptop | Runs great | est. 211.9 tok/s186.5–237.4calibrated estimate ±12% | Q4_K_M | 4.1 / 24.0 GB | Full GPU | up to 128k | — | — |
| Radeon RX 7900 XT | Runs great | est. 210.8 tok/s185.5–236.1calibrated estimate ±12% | Q4_K_M | 4.1 / 20.0 GB | Full GPU | up to 128k | — | $899 launch MSRP |
| GeForce RTX 3080 10GB | Runs great | est. 179.8 tok/s158.2–201.3calibrated estimate ±12% | Q4_K_M | 4.1 / 10.0 GB | Full GPU | up to 32k | — | $699 launch MSRP |
| GeForce RTX 4080 Super | Runs great | est. 174.1 tok/s153.2–195.0calibrated estimate ±12% | Q4_K_M | 4.1 / 16.0 GB | Full GPU | up to 64k | — | $999 launch MSRP |
| Radeon RX 9070 | Runs great | est. 170.0 tok/s149.6–190.4calibrated estimate ±12% | Q4_K_M | 4.1 / 16.0 GB | Full GPU | up to 64k | — | $639 street (as of 2026-08-09) |
| Radeon RX 9070 XT | Runs great | est. 170.0 tok/s149.6–190.4calibrated estimate ±12% | Q4_K_M | 4.1 / 16.0 GB | Full GPU | up to 64k | — | $689 street (as of 2026-08-09) |
| GeForce RTX 4080 | Runs great | est. 169.6 tok/s149.2–189.9calibrated estimate ±12% | Q4_K_M | 4.1 / 16.0 GB | Full GPU | up to 64k | — | $1,199 launch MSRP |
| Radeon RX 7800 XT | Runs great | est. 164.5 tok/s144.7–184.2calibrated estimate ±12% | Q4_K_M | 4.1 / 16.0 GB | Full GPU | up to 64k | — | $499 launch MSRP |
| GeForce RTX 4070 Ti Super | Runs great | est. 158.9 tok/s139.9–178.0calibrated estimate ±12% | Q4_K_M | 4.1 / 16.0 GB | Full GPU | up to 64k | — | $799 launch MSRP |
| GeForce RTX 5070 | Runs great | est. 158.9 tok/s139.9–178.0calibrated estimate ±12% | Q4_K_M | 4.1 / 12.0 GB | Full GPU | up to 64k | — | $629 street (as of 2026-08-09) |
| GeForce RTX 5070 Ti Laptop | Runs great | est. 158.9 tok/s139.9–178.0calibrated estimate ±12% | Q4_K_M | 4.1 / 12.0 GB | Full GPU | up to 64k | — | — |
| GeForce RTX 2080 Ti | Runs great | est. 145.7 tok/s128.2–163.2calibrated estimate ±12% | Q4_K_M | 4.1 / 11.0 GB | Full GPU | up to 32k | — | $999 launch MSRP |
| GeForce RTX 4090 Laptop | Runs great | est. 136.2 tok/s119.9–152.6calibrated estimate ±12% | Q4_K_M | 4.1 / 16.0 GB | Full GPU | up to 64k | — | — |
| Apple M5 Max (40-core GPU) · 48 GB | Runs great | est. 124.5 tok/s99.6–149.4calibrated estimate ±20% | Q4_K_M | 3.5 / 33.6 GB | Unified memory | up to 128k | — | — |
| Apple M3 Ultra · 96 GB | Runs great | est. 121.8 tok/s97.4–146.1calibrated estimate ±20% | Q4_K_M | 3.5 / 67.2 GB | Unified memory | up to 128k | — | — |
| GeForce RTX 4070 | Runs great | est. 119.2 tok/s104.9–133.5calibrated estimate ±12% | Q4_K_M | 4.1 / 12.0 GB | Full GPU | up to 64k | — | $599 launch MSRP |
| GeForce RTX 4070 Super | Runs great | est. 119.2 tok/s104.9–133.5calibrated estimate ±12% | Q4_K_M | 4.1 / 12.0 GB | Full GPU | up to 64k | — | $599 launch MSRP |
| Apple M2 Ultra · 64 GB | Runs great | est. 118.9 tok/s95.2–142.7calibrated estimate ±20% | Q4_K_M | 3.5 / 44.8 GB | Unified memory | up to 128k | — | — |
| GeForce GTX 1080 Ti | Runs great | est. 114.5 tok/s80.1–148.8theoretical estimate ±30% | Q4_K_M | 4.1 / 11.0 GB | Full GPU | up to 32k | — | $699 launch MSRP |
| Apple M4 Max (40-core GPU) · 48 GB | Runs great | est. 110.7 tok/s88.6–132.8calibrated estimate ±20% | Q4_K_M | 3.5 / 33.6 GB | Unified memory | up to 64k | — | — |
| GeForce RTX 3070 | Runs great | est. 106.0 tok/s93.2–118.7calibrated estimate ±12% | Q4_K_M | 4.1 / 8.0 GB | Full GPU | up to 32k | — | $499 launch MSRP |
| GeForce RTX 5060 | Runs great | est. 106.0 tok/s93.2–118.7calibrated estimate ±12% | Q4_K_M | 4.1 / 8.0 GB | Full GPU | up to 32k | — | $339 street (as of 2026-08-09) |
| GeForce RTX 5060 Ti 16GB | Runs great | est. 106.0 tok/s93.2–118.7calibrated estimate ±12% | Q4_K_M | 4.1 / 16.0 GB | Full GPU | up to 64k | — | $569 street (as of 2026-08-09) |
| GeForce RTX 5060 Ti 8GB | Runs great | est. 106.0 tok/s93.2–118.7calibrated estimate ±12% | Q4_K_M | 4.1 / 8.0 GB | Full GPU | up to 32k | — | $429 street (as of 2026-08-09) |
| GeForce RTX 4080 Laptop | Runs great | est. 102.2 tok/s89.9–114.4calibrated estimate ±12% | Q4_K_M | 4.1 / 12.0 GB | Full GPU | up to 64k | — | — |
| Apple M5 Max (32-core GPU) · 36 GB | Runs great | est. 93.3 tok/s74.6–111.9calibrated estimate ±20% | Q4_K_M | 3.5 / 25.2 GB | Unified memory | up to 64k | — | — |
| Intel Arc B580 | Runs great | est. 90.9 tok/s72.7–109.1calibrated estimate ±20% | Q4_K_M | 4.1 / 12.0 GB | Full GPU | up to 64k | — | $290 street (as of 2026-08-09) |
| GeForce RTX 3060 12GB | Runs great | est. 85.1 tok/s74.9–95.4calibrated estimate ±12% | Q4_K_M | 4.1 / 12.0 GB | Full GPU | up to 64k | — | $250 street (as of 2026-08-09) |
| Radeon RX 9060 XT 16GB | Runs great | est. 84.9 tok/s74.7–95.0calibrated estimate ±12% | Q4_K_M | 4.1 / 16.0 GB | Full GPU | up to 64k | — | $449 street (as of 2026-08-09) |
| Apple M4 Max (32-core GPU) · 36 GB | Runs great | est. 83.1 tok/s66.5–99.7calibrated estimate ±20% | Q4_K_M | 3.5 / 25.2 GB | Unified memory | up to 64k | — | — |
| GeForce RTX 4060 Ti 16GB | Runs great | est. 68.1 tok/s59.9–76.3calibrated estimate ±12% | Q4_K_M | 4.1 / 16.0 GB | Full GPU | up to 64k | — | $499 launch MSRP |
| GeForce RTX 4060 Ti 8GB | Runs great | est. 68.1 tok/s59.9–76.3calibrated estimate ±12% | Q4_K_M | 4.1 / 8.0 GB | Full GPU | up to 32k | — | $399 launch MSRP |
| GeForce RTX 4060 | Runs great | est. 64.3 tok/s56.6–72.1calibrated estimate ±12% | Q4_K_M | 4.1 / 8.0 GB | Full GPU | up to 32k | — | $299 launch MSRP |
| Apple M5 Pro · 24 GB | Runs great | est. 62.2 tok/s49.8–74.7calibrated estimate ±20% | Q4_K_M | 3.5 / 16.8 GB | Unified memory | up to 32k | — | — |
| GeForce RTX 4060 Laptop | Runs great | est. 60.6 tok/s53.3–67.8calibrated estimate ±12% | Q4_K_M | 4.1 / 8.0 GB | Full GPU | up to 32k | — | — |
| GeForce RTX 4070 Laptop | Runs great | est. 60.6 tok/s53.3–67.8calibrated estimate ±12% | Q4_K_M | 4.1 / 8.0 GB | Full GPU | up to 32k | — | — |
| Ryzen AI Max+ 395 (Strix Halo) · 32 GB | Runs great | est. 56.2 tok/s45.0–67.5calibrated estimate ±20% | Q4_K_M | 3.5 / 22.4 GB | Unified memory | up to 32k | — | $1,999 launch MSRP |
| Apple M4 Pro · 24 GB | Runs great | est. 55.3 tok/s44.3–66.4calibrated estimate ±20% | Q4_K_M | 3.5 / 16.8 GB | Unified memory | up to 32k | — | — |
| GeForce RTX 3050 8GB | Runs great | est. 53.0 tok/s46.6–59.3calibrated estimate ±12% | Q4_K_M | 4.1 / 8.0 GB | Full GPU | up to 32k | — | $249 launch MSRP |
| NVIDIA DGX Spark · 128 GB | Runs great | est. 36.9 tok/s29.5–44.3calibrated estimate ±20% | Q4_K_M | 3.5 / 89.6 GB | Unified memory | up to 16k | — | $3,999 launch MSRP |
| Apple M5 · 16 GB | Runs great | est. 31.1 tok/s24.9–37.4calibrated estimate ±20% | Q4_K_M | 3.5 / 11.2 GB | Unified memory | up to 16k | — | — |
| Apple M4 · 16 GB | Runs great | est. 24.3 tok/s19.5–29.2calibrated estimate ±20% | Q4_K_M | 3.5 / 11.2 GB | Unified memory | up to 8k | — | — |
| DDR5-6000 dual-channel CPU · 16 GB | Runs slowly | est. 16.2 tok/s13.0–19.5calibrated estimate ±20% | Q4_K_M | 3.0 GB RAM | CPU only | up to 64k |
| — |
| DDR5-5600 dual-channel CPU · 16 GB | Runs slowly | est. 15.1 tok/s12.1–18.2calibrated estimate ±20% | Q4_K_M | 3.0 GB RAM | CPU only | up to 64k |
| — |
| DDR4-3200 dual-channel CPU · 16 GB | Runs slowly | est. 8.7 tok/s6.9–10.4calibrated estimate ±20% | Q4_K_M | 3.0 GB RAM | CPU only | up to 64k |
| — |
Cheapest GPUs that run it
- Runs greatGeForce RTX 3060 12GB$250 street (as of 2026-08-09), est. 85.1 tok/s (74.9–95.4, calibrated estimate ±12%).
Measured results for Llama 3.2 3B
No public measurements for this model yet.
Frequently asked questions
How much VRAM does Llama 3.2 3B need?
At Q4_K_M the weights are 2.02 GB; with 8k context the total is about 3.5 GB (KV cache 0.9 GB, compute buffer 0.6 GB). The smallest tracked file (IQ4_XS) needs about 3.3 GB.
What is the cheapest GPU that runs Llama 3.2 3B well?
Cheapest that runs it great: GeForce RTX 3060 12GB at $250 (street price as of 2026-08-09), est. 85.1 tok/s (74.9–95.4, calibrated estimate ±12%).
Can Llama 3.2 3B run on an 8 GB, 16 GB or 24 GB GPU?
At Q4_K_M with 8k context: GeForce RTX 4060 — Runs great, est. 64.3 tok/s (56.6–72.1, calibrated estimate ±12%); GeForce RTX 5060 Ti 16GB — Runs great, est. 106.0 tok/s (93.2–118.7, calibrated estimate ±12%); GeForce RTX 3090 — Runs great, est. 221.4 tok/s (194.8–248.0, calibrated estimate ±12%).
Speeds are estimates from memory bandwidth, calibrated against public benchmarks, and each one comes with an error band and a confidence label. Real results vary with drivers, backend, context length and thermals.