Can I run Qwen3.5 9B on the Apple M4 Pro?
Runs greatQ4_K_M fits in unified memory (6.7 of 16.8 GB usable by the GPU): Runs great, 37.0 tok/s measured at 4k context (1 run; 8k estimate 33.8 tok/s, 27.0–40.6, calibrated estimate ±20%).
24 GB unified memory · 273.0 GB/s · FP16 8.1 TFLOPS. Assumes 8k context and an f16 KV cache. Verdicts are shown for each memory size.
Verdict by memory size
The page uses the 24 GB configuration; here is every size.
- 24 GBRuns great37.0 tok/s measured at 4k context (1 run; 8k estimate 33.8 tok/s, 27.0–40.6, calibrated estimate ±20%)(used on this page)
- 48 GBRuns great37.0 tok/s measured at 4k context (1 run; 8k estimate 33.8 tok/s, 27.0–40.6, calibrated estimate ±20%)
- 64 GBRuns great37.0 tok/s measured at 4k context (1 run; 8k estimate 33.8 tok/s, 27.0–40.6, calibrated estimate ±20%)
Every quant of Qwen3.5 9B on the Apple M4 Pro
| Quant | File size | Verdict | Speed | Memory | Runs as | Notes |
|---|---|---|---|---|---|---|
| IQ4_XS | 5.47 GB | Runs great | est. 36.2 tok/s28.9–43.4calibrated estimate ±20% | 6.3 / 16.8 GB | Unified memory | — |
| Q4_K_Mbaseline | 5.87 GB | Runs great | 37.0 tok/s8k estimate 33.8 tok/s (27.0–40.6, calibrated estimate ±20%)measured (1 run, 4k context) | 6.7 / 16.8 GB | Unified memory | — |
| Q8_0 | 9.79 GB | Runs well | est. 20.6 tok/s16.5–24.8calibrated estimate ±20% | 10.6 / 16.8 GB | Unified memory | — |
Models with a thinking mode spend extra tokens thinking, so their speed thresholds are 1.5× stricter (30 / 12 / 3 tok/s). With thinking turned off, a model may do one grade better.
Where the memory goes at Q4_K_M
- Weights
- 5.9 GB
- KV cache
- 0.3 GB
- Compute buffer
- 0.6 GB
- Free
- 10.1 GB
- Not usable by the GPU
- 7.2 GB
Context length vs. KV cache
| Context | KV f16 | KV q8_0 | KV q4_0 |
|---|---|---|---|
| 4k | 6.5 GBRuns great est. 34.6 tok/s27.6–41.5calibrated estimate ±20% | 6.5 GBRuns great est. 34.9 tok/s27.9–41.9calibrated estimate ±20% | 6.4 GBRuns great est. 35.1 tok/s28.1–42.1calibrated estimate ±20% |
| 8k | 6.7 GBRuns great 37.0 tok/s8k estimate 33.8 tok/s (27.0–40.6, calibrated estimate ±20%)measured (1 run, 4k context) | 6.6 GBRuns great est. 34.5 tok/s27.6–41.4calibrated estimate ±20% | 6.5 GBRuns great est. 34.9 tok/s27.9–41.9calibrated estimate ±20% |
| 16k | 7.1 GBRuns great est. 32.4 tok/s25.9–38.9calibrated estimate ±20% | 6.8 GBRuns great est. 33.7 tok/s27.0–40.4calibrated estimate ±20% | 6.7 GBRuns great est. 34.4 tok/s27.6–41.3calibrated estimate ±20% |
| 32k | 7.7 GBRuns well est. 29.9 tok/s23.9–35.9calibrated estimate ±20% | 7.2 GBRuns great est. 32.2 tok/s25.8–38.6calibrated estimate ±20% | 7.0 GBRuns great est. 33.6 tok/s26.9–40.3calibrated estimate ±20% |
| 64k | 9.1 GBRuns well est. 25.9 tok/s20.7–31.1calibrated estimate ±20% | 8.1 GBRuns well est. 29.6 tok/s23.6–35.5calibrated estimate ±20% | 7.6 GBRuns great est. 32.0 tok/s25.6–38.4calibrated estimate ±20% |
| 128k | 11.9 GBRuns well est. 20.4 tok/s16.3–24.5calibrated estimate ±20% | 9.9 GBRuns well est. 25.4 tok/s20.3–30.5calibrated estimate ±20% | 8.8 GBRuns well est. 29.2 tok/s23.4–35.1calibrated estimate ±20% |
| 256k | 14.5 GBRuns slowly est. 9.4 tok/s6.6–12.3theoretical estimate ±30% | 13.4 GBRuns well est. 19.8 tok/s15.9–23.8calibrated estimate ±20% | 11.2 GBRuns well est. 24.9 tok/s20.0–29.9calibrated estimate ±20% |
S = Runs great · A = Runs well · B = Runs slowly · F = Won't run
At Q4_K_M the verdict stays Runs great up to 16k context with an f16 KV cache, and up to 32k with q8_0.
Estimated speed
- Generation
- 37.0 tok/s measured at 4k context (1 run; 8k estimate 33.8 tok/s, 27.0–40.6, calibrated estimate ±20%)
- Prompt processing
- about 327 tok/s (196–457, rough estimate ±40%)
Measured on this exact combination
Public benchmarks we calibrate against. Their conditions (context, backend, flags) can differ from the estimates above.
| Hardware | Quant | Backend | Context | Prompt (tok/s) | Generation (tok/s) | Flags | Source | Measured |
|---|---|---|---|---|---|---|---|---|
| Apple M4 Pro | Q4_K_M | llama.cpp | 4k | — | 37.0 | — | github.com | 2026-07-29 |
If this is not enough
A larger model that also runs well on the Apple M4 Pro
Runs wellMistral Small 3.2 24B: Runs well, est. 13.2 tok/s (10.6–15.9, calibrated estimate ±20%)
How to run it
Commands for the Q4_K_M file. Both tools download from Hugging Face on first run.
File: Qwen3.5-9B-Q4_K_M.gguf (5.87 GB) from unsloth/Qwen3.5-9B-MTP-GGUF.
llama-server -hf unsloth/Qwen3.5-9B-MTP-GGUF:Q4_K_M -c 8192 -ngl all -fa on- -ngl all loads every layer on the GPU.
- -fa on enables flash attention (needed for KV cache quantization).
- --cache-type-k q8_0 --cache-type-v q8_0 shrinks the KV cache to 0.14 GB at 8k context.
Terminal:
OLLAMA_CONTEXT_LENGTH=8192 ollama serve
ollama run hf.co/unsloth/Qwen3.5-9B-MTP-GGUF:Q4_K_M- Ollama defaults to 4k context on GPUs under 24 GB. Set OLLAMA_CONTEXT_LENGTH when starting the server, or /set parameter num_ctx inside the chat.
/set parameter num_ctx 8192
Related pages
Guide: Running local LLMs on a Mac (M4, M5): how much unified memory?
Other models on the Apple M4 Pro
Qwen3.5 9B on other hardware
Frequently asked questions
Can I run Qwen3.5 9B on the Apple M4 Pro?
Q4_K_M fits in unified memory (6.7 of 16.8 GB usable by the GPU): Runs great, 37.0 tok/s measured at 4k context (1 run; 8k estimate 33.8 tok/s, 27.0–40.6, calibrated estimate ±20%). Q4_K_M needs 6.7 GB at 8k context; the GPU can use 16.8 GB of the unified memory.
How much context can Qwen3.5 9B use on the Apple M4 Pro?
At Q4_K_M the verdict stays Runs great up to 16k context with an f16 KV cache (0.5 GB of KV) and up to 32k with a q8_0 KV cache (0.6 GB). The model supports up to 256k.
Which quant should I use, and how fast is it?
Q4_K_M (5.87 GB) is the recommended balance: Runs great, 37.0 tok/s measured at 4k context (1 run; 8k estimate 33.8 tok/s, 27.0–40.6, calibrated estimate ±20%). It is also the largest tracked file that gets this verdict.
Speeds are estimates from memory bandwidth, calibrated against public benchmarks, and each one comes with an error band and a confidence label. Real results vary with drivers, backend, context length and thermals.