Can I run Qwen3 32B on the Apple M4?
Won't runQ4_K_M fits in memory (21.9 GB) but is too slow to use: est. 2.7 tok/s (1.9–3.6, theoretical estimate ±30%). IQ4_XS works instead: Runs slowly, est. 4.6 tok/s (3.7–5.5, calibrated estimate ±20%).
32 GB unified memory · 120.0 GB/s · FP16 3.8 TFLOPS. Assumes 8k context and an f16 KV cache. Verdicts are shown for each memory size.
Verdict by memory size
The page uses the 32 GB configuration; here is every size.
- 16 GBWon't runQ4_K_M doesn't fit
- 24 GBWon't runTry IQ4_XS: Runs slowlyest. 3.0 tok/s (2.1–3.9, theoretical estimate ±30%)
- 32 GBWon't runTry IQ4_XS: Runs slowlyest. 4.6 tok/s (3.7–5.5, calibrated estimate ±20%)(used on this page)
Every quant of Qwen3 32B on the Apple M4
| Quant | File size | Verdict | Speed | Memory | Runs as | Notes |
|---|---|---|---|---|---|---|
| Q2_K | 12.34 GB | Runs slowly | est. 6.3 tok/s5.0–7.6calibrated estimate ±20% | 15.1 / 22.4 GB | Unified memory | — |
| IQ4_XS | 17.69 GB | Runs slowly | est. 4.6 tok/s3.7–5.5calibrated estimate ±20% | 20.4 / 22.4 GB | Unified memory | — |
| Q4_K_Mbaseline | 19.76 GB | Won't run | est. 2.7 tok/s1.9–3.6theoretical estimate ±30% | 21.9 GB RAM | CPU only | Fits, but too slow to use |
| Q8_0 | 34.82 GB | Won't run | — | Needs about 37.5 GB; 28.0 GB of memory is free (22.4 GB usable by the GPU) | — | — |
Models with a thinking mode spend extra tokens thinking, so their speed thresholds are 1.5× stricter (30 / 12 / 3 tok/s). With thinking turned off, a model may do one grade better.
Where the memory goes at IQ4_XS
- Weights
- 17.7 GB
- KV cache
- 2.1 GB
- Compute buffer
- 0.6 GB
- Free
- 2.0 GB
- Not usable by the GPU
- 9.6 GB
Context length vs. KV cache
| Context | KV f16 | KV q8_0 | KV q4_0 |
|---|---|---|---|
| 4k | 19.3 GBRuns slowly est. 4.9 tok/s3.9–5.8calibrated estimate ±20% | 18.8 GBRuns slowly est. 5.0 tok/s4.0–6.0calibrated estimate ±20% | 18.5 GBRuns slowly est. 5.1 tok/s4.1–6.1calibrated estimate ±20% |
| 8k | 20.4 GBRuns slowly est. 4.6 tok/s3.7–5.5calibrated estimate ±20% | 19.4 GBRuns slowly est. 4.8 tok/s3.9–5.8calibrated estimate ±20% | 18.9 GBRuns slowly est. 5.0 tok/s4.0–6.0calibrated estimate ±20% |
| 16k | 22.0 GBWon't run est. 2.7 tok/s1.9–3.5theoretical estimate ±30% | 20.6 GBRuns slowly est. 4.6 tok/s3.7–5.5calibrated estimate ±20% | 19.6 GBRuns slowly est. 4.8 tok/s3.9–5.8calibrated estimate ±20% |
| 32k | 26.3 GBWon't run est. 2.3 tok/s1.6–3.0theoretical estimate ±30% | 22.3 GBWon't run est. 2.7 tok/s1.9–3.5theoretical estimate ±30% | 20.9 GBRuns slowly est. 4.5 tok/s3.6–5.4calibrated estimate ±20% |
S = Runs great · A = Runs well · B = Runs slowly · F = Won't run
At IQ4_XS the verdict stays Runs slowly up to 8k context with an f16 KV cache, and up to 16k with q8_0.
Estimated speed
- Generation
- est. 4.6 tok/s (3.7–5.5, calibrated estimate ±20%)
- Prompt processing
- about 46 tok/s (28–64, rough estimate ±40%)
Measured on this exact combination
No public measurement for this exact combination yet; the numbers above are estimates.
If this is not enough
A smaller model that runs well on the Apple M4
Runs greatQwen3 30B-A3B (2507): Runs great, est. 29.8 tok/s (23.8–35.7, calibrated estimate ±20%)
A lower quant of Qwen3 32B
Runs slowlyIQ4_XS (17.69 GB): Runs slowly, est. 4.6 tok/s (3.7–5.5, calibrated estimate ±20%)
Qwen3 32B on the Apple M5
Runs slowlyQ4_K_M: Runs slowly, est. 3.5 tok/s (2.5–4.6, theoretical estimate ±30%)
How to run it
Commands for the IQ4_XS file. Both tools download from Hugging Face on first run.
File: Qwen_Qwen3-32B-IQ4_XS.gguf (17.69 GB) from bartowski/Qwen_Qwen3-32B-GGUF.
llama-server -hf bartowski/Qwen_Qwen3-32B-GGUF:IQ4_XS -c 8192 -ngl all -fa on- -ngl all loads every layer on the GPU.
- -fa on enables flash attention (needed for KV cache quantization).
- --cache-type-k q8_0 --cache-type-v q8_0 shrinks the KV cache to 1.15 GB at 8k context.
Terminal:
OLLAMA_CONTEXT_LENGTH=8192 ollama serve
ollama run hf.co/bartowski/Qwen_Qwen3-32B-GGUF:IQ4_XS- Ollama defaults to 4k context on GPUs under 24 GB. Set OLLAMA_CONTEXT_LENGTH when starting the server, or /set parameter num_ctx inside the chat.
/set parameter num_ctx 8192
Related pages
Guide: Running local LLMs on a Mac (M4, M5): how much unified memory?
Other models on the Apple M4
Qwen3 32B on other hardware
Frequently asked questions
Can I run Qwen3 32B on the Apple M4?
Q4_K_M fits in memory (21.9 GB) but is too slow to use: est. 2.7 tok/s (1.9–3.6, theoretical estimate ±30%). IQ4_XS works instead: Runs slowly, est. 4.6 tok/s (3.7–5.5, calibrated estimate ±20%). IQ4_XS needs 20.4 GB at 8k context; the GPU can use 22.4 GB of the unified memory.
How much context can Qwen3 32B use on the Apple M4?
At IQ4_XS the verdict stays Runs slowly up to 8k context with an f16 KV cache (2.1 GB of KV) and up to 16k with a q8_0 KV cache (2.3 GB). The model supports up to 32k.
Which quant should I use, and how fast is it?
IQ4_XS (17.69 GB) is the recommended balance: Runs slowly, est. 4.6 tok/s (3.7–5.5, calibrated estimate ±20%). It is also the largest tracked file that gets this verdict.
Speeds are estimates from memory bandwidth, calibrated against public benchmarks, and each one comes with an error band and a confidence label. Real results vary with drivers, backend, context length and thermals.