Can I run Qwen3.5 35B-A3B on the Apple M4?
Runs slowlyQ4_K_M runs entirely from 22.8 GB of system RAM: Runs slowly, est. 25.1 tok/s (17.6–32.6, theoretical estimate ±30%).
32 GB unified memory · 120.0 GB/s · FP16 3.8 TFLOPS. Assumes 8k context and an f16 KV cache. Verdicts are shown for each memory size.
Verdict by memory size
The page uses the 32 GB configuration; here is every size.
- 16 GBWon't runQ4_K_M doesn't fit
- 24 GBWon't runTry IQ4_XS: Runs slowlyest. 30.6 tok/s (21.4–39.8, theoretical estimate ±30%)UD-Q2_K_XL (heavy quality loss): Runs great
- 32 GBRuns slowlyest. 25.1 tok/s (17.6–32.6, theoretical estimate ±30%)Try IQ4_XS: Runs great(used on this page)
Every quant of Qwen3.5 35B-A3B on the Apple M4
| Quant | File size | Verdict | Speed | Memory | Runs as | Notes |
|---|---|---|---|---|---|---|
| UD-Q2_K_XL | 12.57 GB | Runs great | est. 41.8 tok/s33.4–50.2calibrated estimate ±20% | 13.3 / 22.4 GB | Unified memory | — |
| IQ4_XS | 18.17 GB | Runs great | est. 31.0 tok/s24.8–37.2calibrated estimate ±20% | 18.9 / 22.4 GB | Unified memory | — |
| Q4_K_Mbaseline | 22.63 GB | Runs slowly | est. 25.1 tok/s17.6–32.6theoretical estimate ±30% | 22.8 GB RAM | CPU only | CPU inference — capped at Runs slowly |
| Q8_0 | 37.80 GB | Won't run | — | Needs about 38.5 GB; 28.0 GB of memory is free (22.4 GB usable by the GPU) | — | — |
Models with a thinking mode spend extra tokens thinking, so their speed thresholds are 1.5× stricter (30 / 12 / 3 tok/s). With thinking turned off, a model may do one grade better.
Where the memory goes at Q4_K_M
- Weights in system RAM
- 22.8 GB
Context length vs. KV cache
| Context | KV f16 | KV q8_0 | KV q4_0 |
|---|---|---|---|
| 4k | 22.7 GBRuns slowly est. 26.2 tok/s18.3–34.0theoretical estimate ±30% | 22.7 GBRuns slowly est. 26.7 tok/s18.7–34.7theoretical estimate ±30% | 22.7 GBRuns slowly est. 27.0 tok/s18.9–35.1theoretical estimate ±30% |
| 8k | 22.8 GBRuns slowly est. 25.1 tok/s17.6–32.6theoretical estimate ±30% | 22.7 GBRuns slowly est. 26.1 tok/s18.3–33.9theoretical estimate ±30% | 22.7 GBRuns slowly est. 26.6 tok/s18.7–34.6theoretical estimate ±30% |
| 16k | 23.0 GBRuns slowly est. 23.2 tok/s16.2–30.2theoretical estimate ±30% | 22.8 GBRuns slowly est. 25.0 tok/s17.5–32.4theoretical estimate ±30% | 22.7 GBRuns slowly est. 26.0 tok/s18.2–33.8theoretical estimate ±30% |
| 32k | 23.3 GBRuns slowly est. 20.2 tok/s14.1–26.2theoretical estimate ±30% | 23.0 GBRuns slowly est. 23.0 tok/s16.1–29.8theoretical estimate ±30% | 22.8 GBRuns slowly est. 24.8 tok/s17.4–32.3theoretical estimate ±30% |
| 64k | 24.0 GBRuns slowly est. 16.0 tok/s11.2–20.8theoretical estimate ±30% | 23.3 GBRuns slowly est. 19.8 tok/s13.9–25.7theoretical estimate ±30% | 23.0 GBRuns slowly est. 22.7 tok/s15.9–29.5theoretical estimate ±30% |
| 128k | 25.3 GBRuns slowly est. 11.3 tok/s7.9–14.7theoretical estimate ±30% | 24.1 GBRuns slowly est. 15.5 tok/s10.9–20.2theoretical estimate ±30% | 23.4 GBRuns slowly est. 19.4 tok/s13.6–25.3theoretical estimate ±30% |
| 256k | 28.0 GBRuns slowly est. 7.1 tok/s5.0–9.2theoretical estimate ±30% | 25.5 GBRuns slowly est. 10.8 tok/s7.6–14.1theoretical estimate ±30% | 24.2 GBRuns slowly est. 15.1 tok/s10.6–19.6theoretical estimate ±30% |
S = Runs great · A = Runs well · B = Runs slowly · F = Won't run
At Q4_K_M the verdict stays Runs slowly up to 256k context with an f16 KV cache, and up to 256k with q8_0.
Estimated speed
- Generation
- est. 25.1 tok/s (17.6–32.6, theoretical estimate ±30%)
- Prompt processing
- Prompt-processing estimates only apply when the whole model runs on the GPU.
Why this verdict: CPU inference — capped at Runs slowly
Measured on this exact combination
No public measurement for this exact combination yet; the numbers above are estimates.
If this is not enough
A smaller model that runs well on the Apple M4
Runs greatMiniCPM-V 4.7 35B-A3B: Runs great, est. 37.5 tok/s (30.0–45.0, calibrated estimate ±20%)
A lower quant of Qwen3.5 35B-A3B
Runs greatIQ4_XS (18.17 GB): Runs great, est. 31.0 tok/s (24.8–37.2, calibrated estimate ±20%)
Qwen3.5 35B-A3B on the Apple M4 Pro
Runs greatQ4_K_M: Runs great, est. 60.3 tok/s (48.3–72.4, calibrated estimate ±20%)
How to run it
Commands for the UD-Q4_K_M file. Both tools download from Hugging Face on first run.
File: Qwen3.5-35B-A3B-UD-Q4_K_M.gguf (22.63 GB) from unsloth/Qwen3.5-35B-A3B-MTP-GGUF.
llama-server -hf unsloth/Qwen3.5-35B-A3B-MTP-GGUF:UD-Q4_K_M -c 8192 -ngl 0 -fa on- -ngl 0 keeps everything on the CPU.
- -fa on enables flash attention (needed for KV cache quantization).
- --cache-type-k q8_0 --cache-type-v q8_0 shrinks the KV cache to 0.09 GB at 8k context.
Terminal:
OLLAMA_CONTEXT_LENGTH=8192 ollama serve
ollama run hf.co/unsloth/Qwen3.5-35B-A3B-MTP-GGUF:UD-Q4_K_M- Ollama defaults to 4k context on GPUs under 24 GB. Set OLLAMA_CONTEXT_LENGTH when starting the server, or /set parameter num_ctx inside the chat.
/set parameter num_ctx 8192
Related pages
Guide: Running local LLMs on a Mac (M4, M5): how much unified memory?
Other models on the Apple M4
Qwen3.5 35B-A3B on other hardware
Frequently asked questions
Can I run Qwen3.5 35B-A3B on the Apple M4?
Q4_K_M runs entirely from 22.8 GB of system RAM: Runs slowly, est. 25.1 tok/s (17.6–32.6, theoretical estimate ±30%). Q4_K_M needs 22.8 GB at 8k context; the GPU can use 22.4 GB of the unified memory.
How much context can Qwen3.5 35B-A3B use on the Apple M4?
At Q4_K_M the verdict stays Runs slowly up to 256k context with an f16 KV cache (5.4 GB of KV) and up to 256k with a q8_0 KV cache (2.9 GB). The model supports up to 256k.
Which quant should I use, and how fast is it?
Q4_K_M (22.63 GB) is the recommended balance: Runs slowly, est. 25.1 tok/s (17.6–32.6, theoretical estimate ±30%). It is also the largest tracked file that gets this verdict.
Speeds are estimates from memory bandwidth, calibrated against public benchmarks, and each one comes with an error band and a confidence label. Real results vary with drivers, backend, context length and thermals.