Mistral Small 3.2 24B hardware requirements
About the model
- Parameters
- 23.6B
- Attention
- Full attention
- Max context
- 131,072 tokens
- Released
- 2025-06-20
- Model card
- Mistral Small 3.2 24B on Hugging Face
GGUF files
| Quant | File size | Quality | Published by |
|---|---|---|---|
| Q2_K | 8.89 GB | Heavy loss | Community GGUF · bartowski |
| IQ4_XS | 12.76 GB | Good balance | Community GGUF · bartowski |
| Q4_K_Mbaseline | 14.33 GB | Good balance | Community GGUF · bartowski |
| Q8_0 | 25.05 GB | Near-lossless | Community GGUF · bartowski |
File sizes do not include the vision projector (mmproj) that image input needs.
Memory needed by quant and context
| Quant | 4k | 8k | 16k | 32k | 64k | 128k |
|---|---|---|---|---|---|---|
| Q2_K | 10.1 | 10.8 | 12.2 | 15.1 | 20.7 | 32.1 |
| IQ4_XS | 14.0 | 14.7 | 16.1 | 18.9 | 24.6 | 35.9 |
| Q4_K_M | 15.5 | 16.2 | 17.7 | 20.5 | 26.2 | 37.5 |
| Q8_0 | 26.3 | 27.0 | 28.4 | 31.2 | 36.9 | 48.2 |
Which GPUs can run Mistral Small 3.2 24B?
How to read the verdicts
- Runs great
- Fully on the GPU at 20 tok/s or more
- Runs well
- Fully on the GPU at 8–20 tok/s; MoE experts in system RAM at 20 tok/s or more; or at least 90% on the GPU at 8 tok/s or more
- Runs slowly
- 2–8 tok/s; CPU-only; less than 90% on the GPU; or MoE experts in system RAM below 20 tok/s
- Won't run
- Does not fit, or under 2 tok/s
Dense models split between GPU and CPU slow down sharply — the CPU side sets the pace. MoE models that keep only their experts in system RAM degrade far more gently.
Scroll sideways to see every column.
| Hardware | Verdict | Speed | Quant | Memory | Runs as | Context | Notes | Price |
|---|---|---|---|---|---|---|---|---|
| GeForce RTX 5090 | Runs great | est. 80.0 tok/s70.4–89.6calibrated estimate ±12% | Q4_K_M | 16.8 / 32.0 GB | Full GPU | up to 64k | — | $4,200 street (as of 2026-08-09) |
| Radeon RX 7900 XTX | Runs great | est. 47.8 tok/s42.0–53.5calibrated estimate ±12% | Q4_K_M | 16.8 / 24.0 GB | Full GPU | up to 32k | — | $999 launch MSRP |
| GeForce RTX 3090 Ti | Runs great | est. 45.0 tok/s39.6–50.4calibrated estimate ±12% | Q4_K_M | 16.8 / 24.0 GB | Full GPU | up to 32k | — | $1,999 launch MSRP |
| GeForce RTX 4090 | Runs great | est. 45.0 tok/s39.6–50.4calibrated estimate ±12% | Q4_K_M | 16.8 / 24.0 GB | Full GPU | up to 32k | — | $2,755 street (as of 2026-08-09) |
| GeForce RTX 3090 | Runs great | est. 41.8 tok/s36.8–46.8calibrated estimate ±12% | Q4_K_M | 16.8 / 24.0 GB | Full GPU | up to 32k | — | $1,050 street (as of 2026-08-09) |
| GeForce RTX 5090 Laptop | Runs great | est. 40.0 tok/s35.2–44.8calibrated estimate ±12% | Q4_K_M | 16.8 / 24.0 GB | Full GPU | up to 32k | — | — |
| Radeon RX 7900 XT | Runs great | est. 39.8 tok/s35.0–44.6calibrated estimate ±12% | Q4_K_M | 16.8 / 20.0 GB | Full GPU | up to 16k | — | $899 launch MSRP |
| Apple M5 Max (40-core GPU) · 48 GB | Runs great | est. 23.5 tok/s18.8–28.2calibrated estimate ±20% | Q4_K_M | 16.2 / 33.6 GB | Unified memory | up to 16k | — | — |
| Apple M3 Ultra · 96 GB | Runs great | est. 23.0 tok/s18.4–27.6calibrated estimate ±20% | Q4_K_M | 16.2 / 67.2 GB | Unified memory | up to 16k | — | — |
| Apple M2 Ultra · 64 GB | Runs great | est. 22.5 tok/s18.0–27.0calibrated estimate ±20% | Q4_K_M | 16.2 / 44.8 GB | Unified memory | up to 16k | — | — |
| Apple M4 Max (40-core GPU) · 48 GB | Runs great | est. 20.9 tok/s16.7–25.1calibrated estimate ±20% | Q4_K_M | 16.2 / 33.6 GB | Unified memory | up to 8k | — | — |
| GeForce RTX 5080 | Runs well | est. 23.5 tok/s18.8–28.2calibrated estimate ±20% | Q4_K_M | 16.0 / 16.0 GB + 0.8 GB RAM | Partial offload | up to 8k |
| $1,256 street (as of 2026-08-09) |
| GeForce RTX 5070 Ti | Runs well | est. 22.6 tok/s18.1–27.2calibrated estimate ±20% | Q4_K_M | 16.0 / 16.0 GB + 0.8 GB RAM | Partial offload | up to 8k |
| $949 street (as of 2026-08-09) |
| GeForce RTX 5080 Laptop | Runs well | est. 22.6 tok/s18.1–27.2calibrated estimate ±20% | Q4_K_M | 16.0 / 16.0 GB + 0.8 GB RAM | Partial offload | up to 8k |
| — |
| GeForce RTX 4080 Super | Runs well | est. 20.3 tok/s16.2–24.3calibrated estimate ±20% | Q4_K_M | 16.0 / 16.0 GB + 0.8 GB RAM | Partial offload | up to 8k |
| $999 launch MSRP |
| Radeon RX 9070 | Runs well | est. 20.0 tok/s16.0–24.0calibrated estimate ±20% | Q4_K_M | 16.0 / 16.0 GB + 0.8 GB RAM | Partial offload | up to 8k |
| $639 street (as of 2026-08-09) |
| Radeon RX 9070 XT | Runs well | est. 20.0 tok/s16.0–24.0calibrated estimate ±20% | Q4_K_M | 16.0 / 16.0 GB + 0.8 GB RAM | Partial offload | up to 8k |
| $689 street (as of 2026-08-09) |
| GeForce RTX 4080 | Runs well | est. 20.0 tok/s16.0–24.0calibrated estimate ±20% | Q4_K_M | 16.0 / 16.0 GB + 0.8 GB RAM | Partial offload | up to 8k |
| $1,199 launch MSRP |
| Radeon RX 7800 XT | Runs well | est. 19.6 tok/s15.7–23.5calibrated estimate ±20% | Q4_K_M | 16.0 / 16.0 GB + 0.8 GB RAM | Partial offload | up to 8k |
| $499 launch MSRP |
| GeForce RTX 4070 Ti Super | Runs well | est. 19.2 tok/s15.4–23.1calibrated estimate ±20% | Q4_K_M | 16.0 / 16.0 GB + 0.8 GB RAM | Partial offload | up to 8k |
| $799 launch MSRP |
| Apple M5 Max (32-core GPU) · 36 GB | Runs well | est. 17.6 tok/s14.1–21.1calibrated estimate ±20% | Q4_K_M | 16.2 / 25.2 GB | Unified memory | up to 32k |
| — |
| GeForce RTX 4090 Laptop | Runs well | est. 17.5 tok/s14.0–21.0calibrated estimate ±20% | Q4_K_M | 16.0 / 16.0 GB + 0.8 GB RAM | Partial offload | up to 8k |
| — |
| Apple M4 Max (32-core GPU) · 36 GB | Runs well | est. 15.7 tok/s12.6–18.8calibrated estimate ±20% | Q4_K_M | 16.2 / 25.2 GB | Unified memory | up to 32k |
| — |
| GeForce RTX 5060 Ti 16GB | Runs well | est. 14.8 tok/s11.8–17.7calibrated estimate ±20% | Q4_K_M | 16.0 / 16.0 GB + 0.8 GB RAM | Partial offload | up to 8k |
| $569 street (as of 2026-08-09) |
| Radeon RX 9060 XT 16GB | Runs well | est. 12.6 tok/s10.1–15.1calibrated estimate ±20% | Q4_K_M | 16.0 / 16.0 GB + 0.8 GB RAM | Partial offload | up to 8k |
| $449 street (as of 2026-08-09) |
| Apple M5 Pro · 24 GB | Runs well | est. 11.8 tok/s9.4–14.1calibrated estimate ±20% | Q4_K_M | 16.2 / 16.8 GB | Unified memory | up to 8k | — | — |
| GeForce RTX 4060 Ti 16GB | Runs well | est. 10.7 tok/s8.5–12.8calibrated estimate ±20% | Q4_K_M | 16.0 / 16.0 GB + 0.8 GB RAM | Partial offload | up to 8k | — | $499 launch MSRP |
| Ryzen AI Max+ 395 (Strix Halo) · 32 GB | Runs well | est. 10.6 tok/s8.5–12.7calibrated estimate ±20% | Q4_K_M | 16.2 / 22.4 GB | Unified memory | up to 32k | — | $1,999 launch MSRP |
| Apple M4 Pro · 24 GB | Runs well | est. 10.5 tok/s8.4–12.5calibrated estimate ±20% | Q4_K_M | 16.2 / 16.8 GB | Unified memory | up to 8k | — | — |
| GeForce RTX 3080 12GB | Runs slowly | est. 7.4 tok/s5.9–8.9calibrated estimate ±20% | Q4_K_M | 12.0 / 12.0 GB + 4.8 GB RAM | Partial offload | up to 32k |
| $799 launch MSRP |
| GeForce RTX 5070 | Runs slowly | est. 7.1 tok/s5.7–8.5calibrated estimate ±20% | Q4_K_M | 12.0 / 12.0 GB + 4.8 GB RAM | Partial offload | up to 32k |
| $629 street (as of 2026-08-09) |
| GeForce RTX 5070 Ti Laptop | Runs slowly | est. 7.1 tok/s5.7–8.5calibrated estimate ±20% | Q4_K_M | 12.0 / 12.0 GB + 4.8 GB RAM | Partial offload | up to 32k |
| — |
| NVIDIA DGX Spark · 128 GB | Runs slowly | est. 7.0 tok/s5.6–8.4calibrated estimate ±20% | Q4_K_M | 16.2 / 89.6 GB | Unified memory | up to 128k |
| $3,999 launch MSRP |
| GeForce RTX 4070 | Runs slowly | est. 6.8 tok/s5.4–8.1calibrated estimate ±20% | Q4_K_M | 12.0 / 12.0 GB + 4.8 GB RAM | Partial offload | up to 32k |
| $599 launch MSRP |
| GeForce RTX 4070 Super | Runs slowly | est. 6.8 tok/s5.4–8.1calibrated estimate ±20% | Q4_K_M | 12.0 / 12.0 GB + 4.8 GB RAM | Partial offload | up to 32k |
| $599 launch MSRP |
| GeForce RTX 4080 Laptop | Runs slowly | est. 6.6 tok/s5.2–7.9calibrated estimate ±20% | Q4_K_M | 12.0 / 12.0 GB + 4.8 GB RAM | Partial offload | up to 32k |
| — |
| Intel Arc B580 | Runs slowly | est. 6.4 tok/s5.1–7.6calibrated estimate ±20% | Q4_K_M | 12.0 / 12.0 GB + 4.8 GB RAM | Partial offload | up to 32k |
| $290 street (as of 2026-08-09) |
| GeForce RTX 3060 12GB | Runs slowly | est. 6.3 tok/s5.0–7.5calibrated estimate ±20% | Q4_K_M | 12.0 / 12.0 GB + 4.8 GB RAM | Partial offload | up to 32k |
| $250 street (as of 2026-08-09) |
| GeForce RTX 2080 Ti | Runs slowly | est. 6.1 tok/s4.9–7.3calibrated estimate ±20% | Q4_K_M | 11.0 / 11.0 GB + 5.8 GB RAM | Partial offload | up to 32k |
| $999 launch MSRP |
| Apple M5 · 24 GB | Runs slowly | est. 5.9 tok/s4.7–7.1calibrated estimate ±20% | Q4_K_M | 16.2 / 16.8 GB | Unified memory | up to 8k |
| — |
| GeForce GTX 1080 Ti | Runs slowly | est. 5.9 tok/s4.1–7.6theoretical estimate ±30% | Q4_K_M | 11.0 / 11.0 GB + 5.8 GB RAM | Partial offload | up to 32k |
| $699 launch MSRP |
| GeForce RTX 3080 10GB | Runs slowly | est. 5.5 tok/s4.4–6.6calibrated estimate ±20% | Q4_K_M | 10.0 / 10.0 GB + 6.8 GB RAM | Partial offload | up to 32k | — | $699 launch MSRP |
| Apple M4 · 24 GB | Runs slowly | est. 4.6 tok/s3.7–5.5calibrated estimate ±20% | Q4_K_M | 16.2 / 16.8 GB | Unified memory | up to 8k | — | — |
| GeForce RTX 3070 | Runs slowly | est. 4.3 tok/s3.4–5.1calibrated estimate ±20% | Q4_K_M | 8.0 / 8.0 GB + 8.8 GB RAM | Partial offload | up to 32k | — | $499 launch MSRP |
| GeForce RTX 5060 | Runs slowly | est. 4.3 tok/s3.4–5.1calibrated estimate ±20% | Q4_K_M | 8.0 / 8.0 GB + 8.8 GB RAM | Partial offload | up to 32k | — | $339 street (as of 2026-08-09) |
| GeForce RTX 5060 Ti 8GB | Runs slowly | est. 4.3 tok/s3.4–5.1calibrated estimate ±20% | Q4_K_M | 8.0 / 8.0 GB + 8.8 GB RAM | Partial offload | up to 32k | — | $429 street (as of 2026-08-09) |
| GeForce RTX 4060 Ti 8GB | Runs slowly | est. 4.1 tok/s3.3–4.9calibrated estimate ±20% | Q4_K_M | 8.0 / 8.0 GB + 8.8 GB RAM | Partial offload | up to 32k | — | $399 launch MSRP |
| GeForce RTX 4060 | Runs slowly | est. 4.0 tok/s3.2–4.8calibrated estimate ±20% | Q4_K_M | 8.0 / 8.0 GB + 8.8 GB RAM | Partial offload | up to 32k | — | $299 launch MSRP |
| GeForce RTX 4060 Laptop | Runs slowly | est. 4.0 tok/s3.2–4.8calibrated estimate ±20% | Q4_K_M | 8.0 / 8.0 GB + 8.8 GB RAM | Partial offload | up to 32k | — | — |
| GeForce RTX 4070 Laptop | Runs slowly | est. 4.0 tok/s3.2–4.8calibrated estimate ±20% | Q4_K_M | 8.0 / 8.0 GB + 8.8 GB RAM | Partial offload | up to 32k | — | — |
| GeForce RTX 3050 8GB | Runs slowly | est. 3.9 tok/s3.1–4.7calibrated estimate ±20% | Q4_K_M | 8.0 / 8.0 GB + 8.8 GB RAM | Partial offload | up to 32k | — | $249 launch MSRP |
| DDR5-6000 dual-channel CPU · 32 GB | Runs slowly | est. 3.1 tok/s2.5–3.7calibrated estimate ±20% | Q4_K_M | 15.7 GB RAM | CPU only | up to 32k | — | — |
| DDR5-5600 dual-channel CPU · 32 GB | Runs slowly | est. 2.9 tok/s2.3–3.4calibrated estimate ±20% | Q4_K_M | 15.7 GB RAM | CPU only | up to 32k | — | — |
| DDR4-3200 dual-channel CPU · 64 GB | Won't runTry Q2_K (heavy quality loss): Runs slowly | est. 2.5 tok/s2.0–3.0calibrated estimate ±20% | Q2_K | 10.2 GB RAM | CPU only | up to 16k | — | — |
Cheapest GPUs that run it
- Runs greatGeForce RTX 3090$1,050 street (as of 2026-08-09), est. 41.8 tok/s (36.8–46.8, calibrated estimate ±12%).
- Runs wellRadeon RX 9060 XT 16GB$449 street (as of 2026-08-09), est. 12.6 tok/s (10.1–15.1, calibrated estimate ±20%).
Measured results for Mistral Small 3.2 24B
No public measurements for this model yet.
Frequently asked questions
How much VRAM does Mistral Small 3.2 24B need?
At Q4_K_M the weights are 14.33 GB; with 8k context the total is about 16.2 GB (KV cache 1.3 GB, compute buffer 0.6 GB). The smallest tracked file (Q2_K (heavy quality loss)) needs about 10.8 GB.
What is the cheapest GPU that runs Mistral Small 3.2 24B well?
Cheapest that runs it great: GeForce RTX 3090 at $1,050 (street price as of 2026-08-09), est. 41.8 tok/s (36.8–46.8, calibrated estimate ±12%). Cheaper and still Runs well: Radeon RX 9060 XT 16GB at $449 (street price as of 2026-08-09), est. 12.6 tok/s (10.1–15.1, calibrated estimate ±20%).
Can Mistral Small 3.2 24B run on an 8 GB, 16 GB or 24 GB GPU?
At Q4_K_M with 8k context: GeForce RTX 4060 — Runs slowly, est. 4.0 tok/s (3.2–4.8, calibrated estimate ±20%); GeForce RTX 5060 Ti 16GB — Runs well, est. 14.8 tok/s (11.8–17.7, calibrated estimate ±20%); GeForce RTX 3090 — Runs great, est. 41.8 tok/s (36.8–46.8, calibrated estimate ±12%).
Speeds are estimates from memory bandwidth, calibrated against public benchmarks, and each one comes with an error band and a confidence label. Real results vary with drivers, backend, context length and thermals.