Can I run Qwen3.5 122B-A10B on the Apple M4 Pro?
Won't runQ4_K_M needs 79.0 GB, 34.2 GB more than the 44.8 GB of usable memory. UD-Q2_K_XL (heavy quality loss) works instead: Runs great, est. 34.1 tok/s (27.2–40.9, calibrated estimate ±20%).
64 GB unified memory · 273.0 GB/s · FP16 8.1 TFLOPS. Assumes 8k context and an f16 KV cache. Verdicts are shown for each memory size.
Verdict by memory size
The page uses the 64 GB configuration; here is every size.
- 24 GBWon't runQ4_K_M doesn't fit
- 48 GBWon't runTry UD-Q2_K_XL (heavy quality loss): Runs slowlyest. 32.4 tok/s (22.7–42.1, theoretical estimate ±30%)
- 64 GBWon't runTry UD-Q2_K_XL (heavy quality loss): Runs greatest. 34.1 tok/s (27.2–40.9, calibrated estimate ±20%)(used on this page)
Every quant of Qwen3.5 122B-A10B on the Apple M4 Pro
| Quant | File size | Verdict | Speed | Memory | Runs as | Notes |
|---|---|---|---|---|---|---|
| UD-Q2_K_XL | 42.85 GB | Runs great | est. 34.1 tok/s27.2–40.9calibrated estimate ±20% | 43.6 / 44.8 GB | Unified memory | — |
| IQ4_XS | 61.93 GB | Won't run | — | Needs about 62.7 GB; 60.0 GB of memory is free (44.8 GB usable by the GPU) | — | — |
| Q4_K_Mbaseline | 78.26 GB | Won't run | — | Needs about 79.0 GB; 60.0 GB of memory is free (44.8 GB usable by the GPU) | — | — |
| Q8_0 | 132.55 GB | Won't run | — | Needs about 133.3 GB; 60.0 GB of memory is free (44.8 GB usable by the GPU) | — | — |
Models with a thinking mode spend extra tokens thinking, so their speed thresholds are 1.5× stricter (30 / 12 / 3 tok/s). With thinking turned off, a model may do one grade better.
Where the memory goes at UD-Q2_K_XL
- Weights
- 42.9 GB
- KV cache
- 0.2 GB
- Compute buffer
- 0.6 GB
- Free
- 1.2 GB
- Not usable by the GPU
- 19.2 GB
Context length vs. KV cache
| Context | KV f16 | KV q8_0 | KV q4_0 |
|---|---|---|---|
| 4k | 43.5 GBRuns great est. 34.6 tok/s27.7–41.6calibrated estimate ±20% | 43.4 GBRuns great est. 34.9 tok/s27.9–41.9calibrated estimate ±20% | 43.4 GBRuns great est. 35.1 tok/s28.0–42.1calibrated estimate ±20% |
| 8k | 43.6 GBRuns great est. 34.1 tok/s27.2–40.9calibrated estimate ±20% | 43.5 GBRuns great est. 34.6 tok/s27.7–41.5calibrated estimate ±20% | 43.5 GBRuns great est. 34.9 tok/s27.9–41.9calibrated estimate ±20% |
| 16k | 43.9 GBRuns great est. 33.0 tok/s26.4–39.6calibrated estimate ±20% | 43.7 GBRuns great est. 34.0 tok/s27.2–40.8calibrated estimate ±20% | 43.6 GBRuns great est. 34.6 tok/s27.6–41.5calibrated estimate ±20% |
| 32k | 44.5 GBRuns great est. 31.0 tok/s24.8–37.2calibrated estimate ±20% | 44.1 GBRuns great est. 32.8 tok/s26.3–39.4calibrated estimate ±20% | 43.9 GBRuns great est. 33.9 tok/s27.1–40.7calibrated estimate ±20% |
| 64k | 44.5 GBRuns slowly est. 23.3 tok/s16.3–30.3theoretical estimate ±30% | 43.7 GBRuns slowly est. 27.4 tok/s19.2–35.6theoretical estimate ±30% | 44.4 GBRuns great est. 32.7 tok/s26.1–39.2calibrated estimate ±20% |
| 128k | 46.1 GBRuns slowly est. 17.7 tok/s12.4–23.0theoretical estimate ±30% | 44.6 GBRuns slowly est. 22.8 tok/s16.0–29.6theoretical estimate ±30% | 43.8 GBRuns slowly est. 27.0 tok/s18.9–35.1theoretical estimate ±30% |
| 256k | 49.3 GBRuns slowly est. 11.9 tok/s8.3–15.5theoretical estimate ±30% | 46.3 GBRuns slowly est. 17.1 tok/s12.0–22.2theoretical estimate ±30% | 44.7 GBRuns slowly est. 22.3 tok/s15.6–29.0theoretical estimate ±30% |
S = Runs great · A = Runs well · B = Runs slowly · F = Won't run
At UD-Q2_K_XL the verdict stays Runs great up to 32k context with an f16 KV cache, and up to 32k with q8_0.
Estimated speed
- Generation
- est. 34.1 tok/s (27.2–40.9, calibrated estimate ±20%)
- Prompt processing
- about 105 tok/s (63–147, rough estimate ±40%)
Measured on this exact combination
No public measurement for this exact combination yet; the numbers above are estimates.
If this is not enough
A smaller model that runs well on the Apple M4 Pro
Runs greatK2-Horizon MoVA 36B-A4B: Runs great, est. 36.8 tok/s (25.8–47.8, theoretical estimate ±30%)
A lower quant of Qwen3.5 122B-A10B
Runs greatUD-Q2_K_XL (42.85 GB, heavy quality loss): Runs great, est. 34.1 tok/s (27.2–40.9, calibrated estimate ±20%)
Qwen3.5 122B-A10B on the Apple M4 Max (40-core GPU)
Runs greatQ4_K_M: Runs great, est. 43.0 tok/s (34.4–51.6, calibrated estimate ±20%)
How to run it
Commands for the UD-Q2_K_XL file. Both tools download from Hugging Face on first run.
File: Qwen3.5-122B-A10B-UD-Q2_K_XL.gguf (42.85 GB) from unsloth/Qwen3.5-122B-A10B-MTP-GGUF.
llama-server -hf unsloth/Qwen3.5-122B-A10B-MTP-GGUF:UD-Q2_K_XL -c 8192 -ngl all -fa on- -ngl all loads every layer on the GPU.
- -fa on enables flash attention (needed for KV cache quantization).
- --cache-type-k q8_0 --cache-type-v q8_0 shrinks the KV cache to 0.11 GB at 8k context.
Terminal:
OLLAMA_CONTEXT_LENGTH=8192 ollama serve
ollama run hf.co/unsloth/Qwen3.5-122B-A10B-MTP-GGUF:UD-Q2_K_XL- Ollama defaults to 4k context on GPUs under 24 GB. Set OLLAMA_CONTEXT_LENGTH when starting the server, or /set parameter num_ctx inside the chat.
/set parameter num_ctx 8192
Related pages
Guide: Running local LLMs on a Mac (M4, M5): how much unified memory?
Other models on the Apple M4 Pro
Qwen3.5 122B-A10B on other hardware
Frequently asked questions
Can I run Qwen3.5 122B-A10B on the Apple M4 Pro?
Q4_K_M needs 79.0 GB, 34.2 GB more than the 44.8 GB of usable memory. UD-Q2_K_XL (heavy quality loss) works instead: Runs great, est. 34.1 tok/s (27.2–40.9, calibrated estimate ±20%). UD-Q2_K_XL needs 43.6 GB at 8k context; the GPU can use 44.8 GB of the unified memory.
How much context can Qwen3.5 122B-A10B use on the Apple M4 Pro?
At UD-Q2_K_XL the verdict stays Runs great up to 32k context with an f16 KV cache (0.8 GB of KV) and up to 32k with a q8_0 KV cache (0.4 GB). The model supports up to 256k.
Which quant should I use, and how fast is it?
UD-Q2_K_XL (heavy quality loss) (42.85 GB) is the recommended balance: Runs great, est. 34.1 tok/s (27.2–40.9, calibrated estimate ±20%). It is also the largest tracked file that gets this verdict.
Speeds are estimates from memory bandwidth, calibrated against public benchmarks, and each one comes with an error band and a confidence label. Real results vary with drivers, backend, context length and thermals.