Can I run Qwen3 30B-A3B (2507) on the Apple M4?
Runs greatQ4_K_M fits in unified memory (19.9 of 22.4 GB usable by the GPU): Runs great, est. 29.8 tok/s (23.8–35.7, calibrated estimate ±20%).
32 GB unified memory · 120.0 GB/s · FP16 3.8 TFLOPS. Assumes 8k context and an f16 KV cache. Verdicts are shown for each memory size.
Verdict by memory size
The page uses the 32 GB configuration; here is every size.
- 16 GBWon't runQ4_K_M doesn't fit
- 24 GBRuns slowlyest. 18.3 tok/s (12.8–23.8, theoretical estimate ±30%)Q2_K (heavy quality loss): Runs great
- 32 GBRuns greatest. 29.8 tok/s (23.8–35.7, calibrated estimate ±20%)(used on this page)
Every quant of Qwen3 30B-A3B (2507) on the Apple M4
| Quant | File size | Verdict | Speed | Memory | Runs as | Notes |
|---|---|---|---|---|---|---|
| Q2_K | 11.26 GB | Runs great | est. 37.0 tok/s29.6–44.4calibrated estimate ±20% | 12.6 / 22.4 GB | Unified memory | — |
| IQ4_XS | 16.38 GB | Runs great | est. 31.3 tok/s25.1–37.6calibrated estimate ±20% | 17.8 / 22.4 GB | Unified memory | — |
| Q4_K_Mbaseline | 18.56 GB | Runs great | est. 29.8 tok/s23.8–35.7calibrated estimate ±20% | 19.9 / 22.4 GB | Unified memory | — |
| Q8_0 | 32.48 GB | Won't run | — | Needs about 33.9 GB; 28.0 GB of memory is free (22.4 GB usable by the GPU) | — | — |
Where the memory goes at Q4_K_M
- Weights
- 18.6 GB
- KV cache
- 0.8 GB
- Compute buffer
- 0.6 GB
- Free
- 2.5 GB
- Not usable by the GPU
- 9.6 GB
Context length vs. KV cache
| Context | KV f16 | KV q8_0 | KV q4_0 |
|---|---|---|---|
| 4k | 19.5 GBRuns great est. 34.3 tok/s27.4–41.1calibrated estimate ±20% | 19.3 GBRuns great est. 36.8 tok/s29.5–44.2calibrated estimate ±20% | 19.2 GBRuns great est. 38.4 tok/s30.7–46.1calibrated estimate ±20% |
| 8k | 19.9 GBRuns great est. 29.8 tok/s23.8–35.7calibrated estimate ±20% | 19.6 GBRuns great est. 33.9 tok/s27.1–40.7calibrated estimate ±20% | 19.4 GBRuns great est. 36.6 tok/s29.3–44.0calibrated estimate ±20% |
| 16k | 20.8 GBRuns great est. 23.6 tok/s18.8–28.3calibrated estimate ±20% | 20.1 GBRuns great est. 29.2 tok/s23.4–35.1calibrated estimate ±20% | 19.7 GBRuns great est. 33.5 tok/s26.8–40.2calibrated estimate ±20% |
| 32k | 21.8 GBRuns slowly est. 9.9 tok/s6.9–12.8theoretical estimate ±30% | 21.1 GBRuns great est. 22.9 tok/s18.3–27.5calibrated estimate ±20% | 20.3 GBRuns great est. 28.7 tok/s23.0–34.4calibrated estimate ±20% |
| 64k | 25.0 GBRuns slowly est. 6.1 tok/s4.3–7.9theoretical estimate ±30% | 22.0 GBRuns slowly est. 9.5 tok/s6.6–12.3theoretical estimate ±30% | 21.5 GBRuns great est. 22.3 tok/s17.8–26.7calibrated estimate ±20% |
| 128k | 33.1 GBdoes not fit | 25.5 GBRuns slowly est. 5.8 tok/s4.1–7.5theoretical estimate ±30% | 22.2 GBRuns slowly est. 9.1 tok/s6.4–11.8theoretical estimate ±30% |
| 256k | 47.2 GBdoes not fit | 35.2 GBdoes not fit | 25.9 GBRuns slowly est. 5.5 tok/s3.9–7.2theoretical estimate ±30% |
S = Runs great · A = Runs well · B = Runs slowly · F = Won't run
At Q4_K_M the verdict stays Runs great up to 16k context with an f16 KV cache, and up to 32k with q8_0.
Estimated speed
- Generation
- est. 29.8 tok/s (23.8–35.7, calibrated estimate ±20%)
- Prompt processing
- about 182 tok/s (109–255, rough estimate ±40%)
Measured on this exact combination
No public measurement for this exact combination yet; the numbers above are estimates.
If this is not enough
A larger model that also runs well on the Apple M4
Runs greatMiniCPM-V 4.7 35B-A3B: Runs great, est. 37.5 tok/s (30.0–45.0, calibrated estimate ±20%)
How to run it
Commands for the Q4_K_M file. Both tools download from Hugging Face on first run.
File: Qwen3-30B-A3B-Instruct-2507-Q4_K_M.gguf (18.56 GB) from unsloth/Qwen3-30B-A3B-Instruct-2507-GGUF.
llama-server -hf unsloth/Qwen3-30B-A3B-Instruct-2507-GGUF:Q4_K_M -c 8192 -ngl all -fa on- -ngl all loads every layer on the GPU.
- -fa on enables flash attention (needed for KV cache quantization).
- --cache-type-k q8_0 --cache-type-v q8_0 shrinks the KV cache to 0.43 GB at 8k context.
Terminal:
OLLAMA_CONTEXT_LENGTH=8192 ollama serve
ollama run hf.co/unsloth/Qwen3-30B-A3B-Instruct-2507-GGUF:Q4_K_M- Ollama defaults to 4k context on GPUs under 24 GB. Set OLLAMA_CONTEXT_LENGTH when starting the server, or /set parameter num_ctx inside the chat.
/set parameter num_ctx 8192
Related pages
Guide: Running local LLMs on a Mac (M4, M5): how much unified memory?
Other models on the Apple M4
Qwen3 30B-A3B (2507) on other hardware
Frequently asked questions
Can I run Qwen3 30B-A3B (2507) on the Apple M4?
Q4_K_M fits in unified memory (19.9 of 22.4 GB usable by the GPU): Runs great, est. 29.8 tok/s (23.8–35.7, calibrated estimate ±20%). Q4_K_M needs 19.9 GB at 8k context; the GPU can use 22.4 GB of the unified memory.
How much context can Qwen3 30B-A3B (2507) use on the Apple M4?
At Q4_K_M the verdict stays Runs great up to 16k context with an f16 KV cache (1.6 GB of KV) and up to 32k with a q8_0 KV cache (1.7 GB). The model supports up to 256k.
Which quant should I use, and how fast is it?
Q4_K_M (18.56 GB) is the recommended balance: Runs great, est. 29.8 tok/s (23.8–35.7, calibrated estimate ±20%). It is also the largest tracked file that gets this verdict.
Speeds are estimates from memory bandwidth, calibrated against public benchmarks, and each one comes with an error band and a confidence label. Real results vary with drivers, backend, context length and thermals.