Can I run Qwen3.5 35B-A3B on the Apple M4 Pro?
Runs greatQ4_K_M fits in unified memory (23.4 of 33.6 GB usable by the GPU): Runs great, est. 60.3 tok/s (48.3–72.4, calibrated estimate ±20%).
48 GB unified memory · 273.0 GB/s · FP16 8.1 TFLOPS. Assumes 8k context and an f16 KV cache. Verdicts are shown for each memory size.
Verdict by memory size
The page uses the 48 GB configuration; here is every size.
- 24 GBWon't runTry IQ4_XS: Runs slowlyest. 62.8 tok/s (44.0–81.7, theoretical estimate ±30%)UD-Q2_K_XL (heavy quality loss): Runs great
- 48 GBRuns greatest. 60.3 tok/s (48.3–72.4, calibrated estimate ±20%)(used on this page)
- 64 GBRuns greatest. 60.3 tok/s (48.3–72.4, calibrated estimate ±20%)
Every quant of Qwen3.5 35B-A3B on the Apple M4 Pro
| Quant | File size | Verdict | Speed | Memory | Runs as | Notes |
|---|---|---|---|---|---|---|
| UD-Q2_K_XL | 12.57 GB | Runs great | est. 81.5 tok/s65.2–97.8calibrated estimate ±20% | 13.3 / 33.6 GB | Unified memory | — |
| IQ4_XS | 18.17 GB | Runs great | est. 62.8 tok/s50.3–75.4calibrated estimate ±20% | 18.9 / 33.6 GB | Unified memory | — |
| Q4_K_Mbaseline | 22.63 GB | Runs great | est. 60.3 tok/s48.3–72.4calibrated estimate ±20% | 23.4 / 33.6 GB | Unified memory | — |
| Q8_0 | 37.80 GB | Runs slowly | est. 35.3 tok/s24.7–45.9theoretical estimate ±30% | 38.0 GB RAM | CPU only | CPU inference — capped at Runs slowly |
Models with a thinking mode spend extra tokens thinking, so their speed thresholds are 1.5× stricter (30 / 12 / 3 tok/s). With thinking turned off, a model may do one grade better.
Where the memory goes at Q4_K_M
- Weights
- 22.6 GB
- KV cache
- 0.2 GB
- Compute buffer
- 0.6 GB
- Free
- 10.2 GB
- Not usable by the GPU
- 14.4 GB
Context length vs. KV cache
| Context | KV f16 | KV q8_0 | KV q4_0 |
|---|---|---|---|
| 4k | 23.3 GBRuns great est. 61.8 tok/s49.5–74.2calibrated estimate ±20% | 23.2 GBRuns great est. 62.6 tok/s50.1–75.1calibrated estimate ±20% | 23.2 GBRuns great est. 63.0 tok/s50.4–75.6calibrated estimate ±20% |
| 8k | 23.4 GBRuns great est. 60.3 tok/s48.3–72.4calibrated estimate ±20% | 23.3 GBRuns great est. 61.7 tok/s49.4–74.1calibrated estimate ±20% | 23.3 GBRuns great est. 62.5 tok/s50.0–75.0calibrated estimate ±20% |
| 16k | 23.6 GBRuns great est. 57.5 tok/s46.0–69.0calibrated estimate ±20% | 23.5 GBRuns great est. 60.1 tok/s48.1–72.2calibrated estimate ±20% | 23.4 GBRuns great est. 61.6 tok/s49.3–74.0calibrated estimate ±20% |
| 32k | 24.1 GBRuns great est. 52.6 tok/s42.1–63.2calibrated estimate ±20% | 23.8 GBRuns great est. 57.2 tok/s45.7–68.6calibrated estimate ±20% | 23.6 GBRuns great est. 59.9 tok/s47.9–71.9calibrated estimate ±20% |
| 64k | 25.1 GBRuns great est. 45.0 tok/s36.0–54.0calibrated estimate ±20% | 24.4 GBRuns great est. 52.0 tok/s41.6–62.4calibrated estimate ±20% | 24.1 GBRuns great est. 56.8 tok/s45.4–68.1calibrated estimate ±20% |
| 128k | 27.0 GBRuns great est. 34.8 tok/s27.9–41.8calibrated estimate ±20% | 25.8 GBRuns great est. 44.1 tok/s35.3–52.9calibrated estimate ±20% | 25.1 GBRuns great est. 51.4 tok/s41.1–61.7calibrated estimate ±20% |
| 256k | 30.9 GBRuns well est. 24.0 tok/s19.2–28.8calibrated estimate ±20% | 28.4 GBRuns great est. 33.8 tok/s27.0–40.5calibrated estimate ±20% | 27.1 GBRuns great est. 43.2 tok/s34.6–51.9calibrated estimate ±20% |
S = Runs great · A = Runs well · B = Runs slowly · F = Won't run
At Q4_K_M the verdict stays Runs great up to 128k context with an f16 KV cache, and up to 256k with q8_0.
Estimated speed
- Generation
- est. 60.3 tok/s (48.3–72.4, calibrated estimate ±20%)
- Prompt processing
- about 340 tok/s (204–476, rough estimate ±40%)
Measured on this exact combination
No public measurement for this exact combination yet; the numbers above are estimates.
If this is not enough
A larger model that also runs well on the Apple M4 Pro
Runs greatK2-Horizon MoVA 36B-A4B: Runs great, est. 36.8 tok/s (25.8–47.8, theoretical estimate ±30%)
How to run it
Commands for the UD-Q4_K_M file. Both tools download from Hugging Face on first run.
File: Qwen3.5-35B-A3B-UD-Q4_K_M.gguf (22.63 GB) from unsloth/Qwen3.5-35B-A3B-MTP-GGUF.
llama-server -hf unsloth/Qwen3.5-35B-A3B-MTP-GGUF:UD-Q4_K_M -c 8192 -ngl all -fa on- -ngl all loads every layer on the GPU.
- -fa on enables flash attention (needed for KV cache quantization).
- --cache-type-k q8_0 --cache-type-v q8_0 shrinks the KV cache to 0.09 GB at 8k context.
Terminal:
OLLAMA_CONTEXT_LENGTH=8192 ollama serve
ollama run hf.co/unsloth/Qwen3.5-35B-A3B-MTP-GGUF:UD-Q4_K_M- Ollama defaults to 4k context on GPUs under 24 GB. Set OLLAMA_CONTEXT_LENGTH when starting the server, or /set parameter num_ctx inside the chat.
/set parameter num_ctx 8192
Related pages
Guide: Running local LLMs on a Mac (M4, M5): how much unified memory?
Other models on the Apple M4 Pro
Qwen3.5 35B-A3B on other hardware
Frequently asked questions
Can I run Qwen3.5 35B-A3B on the Apple M4 Pro?
Q4_K_M fits in unified memory (23.4 of 33.6 GB usable by the GPU): Runs great, est. 60.3 tok/s (48.3–72.4, calibrated estimate ±20%). Q4_K_M needs 23.4 GB at 8k context; the GPU can use 33.6 GB of the unified memory.
How much context can Qwen3.5 35B-A3B use on the Apple M4 Pro?
At Q4_K_M the verdict stays Runs great up to 128k context with an f16 KV cache (2.7 GB of KV) and up to 256k with a q8_0 KV cache (2.9 GB). The model supports up to 256k.
Which quant should I use, and how fast is it?
Q4_K_M (22.63 GB) is the recommended balance: Runs great, est. 60.3 tok/s (48.3–72.4, calibrated estimate ±20%). It is also the largest tracked file that gets this verdict.
Speeds are estimates from memory bandwidth, calibrated against public benchmarks, and each one comes with an error band and a confidence label. Real results vary with drivers, backend, context length and thermals.