Can I run Qwen3.5 27B on the Apple M4 Pro?
Runs slowlyQ4_K_M runs entirely from 17.6 GB of system RAM: Runs slowly, 9.9 tok/s measured at 512 context (1 run; 8k estimate 7.7 tok/s, 5.4–10.1, theoretical estimate ±30%). UD-Q2_K_XL (heavy quality loss) would reach Runs well, est. 16.5 tok/s (13.2–19.8, calibrated estimate ±20%).
24 GB unified memory · 273.0 GB/s · FP16 8.1 TFLOPS. Assumes 8k context and an f16 KV cache. Verdicts are shown for each memory size.
Verdict by memory size
The page uses the 24 GB configuration; here is every size.
- 24 GBRuns slowly9.9 tok/s measured at 512 context (1 run; 8k estimate 7.7 tok/s, 5.4–10.1, theoretical estimate ±30%)UD-Q2_K_XL (heavy quality loss): Runs well(used on this page)
- 48 GBRuns slowly9.9 tok/s measured at 512 context (1 run; 8k estimate 11.8 tok/s, 9.4–14.1, calibrated estimate ±20%)Try IQ4_XS: Runs well
- 64 GBRuns slowly9.9 tok/s measured at 512 context (1 run; 8k estimate 11.8 tok/s, 9.4–14.1, calibrated estimate ±20%)Try IQ4_XS: Runs well
Every quant of Qwen3.5 27B on the Apple M4 Pro
| Quant | File size | Verdict | Speed | Memory | Runs as | Notes |
|---|---|---|---|---|---|---|
| UD-Q2_K_XL | 12.04 GB | Runs well | est. 16.5 tok/s13.2–19.8calibrated estimate ±20% | 13.2 / 16.8 GB | Unified memory | — |
| IQ4_XS | 15.71 GB | Runs slowly | est. 8.4 tok/s5.9–10.9theoretical estimate ±30% | 16.2 GB RAM | CPU only | — |
| Q4_K_Mbaseline | 17.11 GB | Runs slowly | 9.9 tok/s8k estimate 7.7 tok/s (5.4–10.1, theoretical estimate ±30%)measured (1 run, 512 context) | 17.6 GB RAM | CPU only | — |
| Q8_0 | 29.05 GB | Won't run | — | Needs about 30.2 GB; 20.0 GB of memory is free (16.8 GB usable by the GPU) | — | — |
Models with a thinking mode spend extra tokens thinking, so their speed thresholds are 1.5× stricter (30 / 12 / 3 tok/s). With thinking turned off, a model may do one grade better.
Where the memory goes at Q4_K_M
- Weights in system RAM
- 17.6 GB
Context length vs. KV cache
| Context | KV f16 | KV q8_0 | KV q4_0 |
|---|---|---|---|
| 4k | 17.4 GBRuns slowly est. 7.9 tok/s5.5–10.2theoretical estimate ±30% | 17.3 GBRuns slowly est. 7.9 tok/s5.5–10.3theoretical estimate ±30% | 17.2 GBRuns slowly est. 7.9 tok/s5.6–10.3theoretical estimate ±30% |
| 8k | 17.6 GBRuns slowly 9.9 tok/s8k estimate 7.7 tok/s (5.4–10.1, theoretical estimate ±30%)measured (1 run, 512 context) | 17.4 GBRuns slowly est. 7.8 tok/s5.5–10.2theoretical estimate ±30% | 17.3 GBRuns slowly est. 7.9 tok/s5.5–10.3theoretical estimate ±30% |
| 16k | 18.2 GBRuns slowly est. 7.5 tok/s5.3–9.8theoretical estimate ±30% | 17.7 GBRuns slowly est. 7.7 tok/s5.4–10.0theoretical estimate ±30% | 17.4 GBRuns slowly est. 7.8 tok/s5.5–10.2theoretical estimate ±30% |
| 32k | 19.3 GBRuns slowly est. 7.1 tok/s5.0–9.2theoretical estimate ±30% | 18.3 GBRuns slowly est. 7.5 tok/s5.2–9.7theoretical estimate ±30% | 17.7 GBRuns slowly est. 7.7 tok/s5.4–10.0theoretical estimate ±30% |
| 64k | 22.5 GBdoes not fit | 19.4 GBRuns slowly est. 7.0 tok/s4.9–9.1theoretical estimate ±30% | 18.3 GBRuns slowly est. 7.4 tok/s5.2–9.7theoretical estimate ±30% |
| 128k | 27.4 GBdoes not fit | 23.4 GBdoes not fit | 19.6 GBRuns slowly est. 7.0 tok/s4.9–9.1theoretical estimate ±30% |
| 256k | 37.2 GBdoes not fit | 29.2 GBdoes not fit | 24.9 GBdoes not fit |
S = Runs great · A = Runs well · B = Runs slowly · F = Won't run
At Q4_K_M the verdict stays Runs slowly up to 32k context with an f16 KV cache, and up to 64k with q8_0.
Estimated speed
- Generation
- 9.9 tok/s measured at 512 context (1 run; 8k estimate 7.7 tok/s, 5.4–10.1, theoretical estimate ±30%)
- Prompt processing
- Prompt-processing estimates only apply when the whole model runs on the GPU.
Measured on this exact combination
Public benchmarks we calibrate against. Their conditions (context, backend, flags) can differ from the estimates above.
| Hardware | Quant | Backend | Context | Prompt (tok/s) | Generation (tok/s) | Flags | Source | Measured |
|---|---|---|---|---|---|---|---|---|
| Apple M4 Pro | Q4_K_M | llama.cpp | 512 | 115.0 | 9.9 | Metal | github.com | 2026-07-18 |
If this is not enough
A smaller model that runs well on the Apple M4 Pro
Runs wellMistral Small 3.2 24B: Runs well, est. 13.2 tok/s (10.6–15.9, calibrated estimate ±20%)
A lower quant of Qwen3.5 27B
Runs wellUD-Q2_K_XL (12.04 GB, heavy quality loss): Runs well, est. 16.5 tok/s (13.2–19.8, calibrated estimate ±20%)
Qwen3.5 27B on the Apple M5 Pro
Runs wellQ4_K_M: Runs well, est. 13.2 tok/s (10.6–15.9, calibrated estimate ±20%)
How to run it
Commands for the Q4_K_M file. Both tools download from Hugging Face on first run.
File: Qwen3.5-27B-Q4_K_M.gguf (17.11 GB) from unsloth/Qwen3.5-27B-MTP-GGUF.
llama-server -hf unsloth/Qwen3.5-27B-MTP-GGUF:Q4_K_M -c 8192 -ngl 0 -fa on- -ngl 0 keeps everything on the CPU.
- -fa on enables flash attention (needed for KV cache quantization).
- --cache-type-k q8_0 --cache-type-v q8_0 shrinks the KV cache to 0.29 GB at 8k context.
Terminal:
OLLAMA_CONTEXT_LENGTH=8192 ollama serve
ollama run hf.co/unsloth/Qwen3.5-27B-MTP-GGUF:Q4_K_M- Ollama defaults to 4k context on GPUs under 24 GB. Set OLLAMA_CONTEXT_LENGTH when starting the server, or /set parameter num_ctx inside the chat.
/set parameter num_ctx 8192
Related pages
Guide: Running local LLMs on a Mac (M4, M5): how much unified memory?
Other models on the Apple M4 Pro
Qwen3.5 27B on other hardware
Frequently asked questions
Can I run Qwen3.5 27B on the Apple M4 Pro?
Q4_K_M runs entirely from 17.6 GB of system RAM: Runs slowly, 9.9 tok/s measured at 512 context (1 run; 8k estimate 7.7 tok/s, 5.4–10.1, theoretical estimate ±30%). UD-Q2_K_XL (heavy quality loss) would reach Runs well, est. 16.5 tok/s (13.2–19.8, calibrated estimate ±20%). Q4_K_M needs 17.6 GB at 8k context; the GPU can use 16.8 GB of the unified memory.
How much context can Qwen3.5 27B use on the Apple M4 Pro?
At Q4_K_M the verdict stays Runs slowly up to 32k context with an f16 KV cache (2.1 GB of KV) and up to 64k with a q8_0 KV cache (2.3 GB). The model supports up to 256k.
Which quant should I use, and how fast is it?
Q4_K_M (17.11 GB) is the recommended balance: Runs slowly, 9.9 tok/s measured at 512 context (1 run; 8k estimate 7.7 tok/s, 5.4–10.1, theoretical estimate ±30%). It is also the largest tracked file that gets this verdict.
Speeds are estimates from memory bandwidth, calibrated against public benchmarks, and each one comes with an error band and a confidence label. Real results vary with drivers, backend, context length and thermals.