Can I run Gemma 4 31B on the Apple M4 Pro?
Runs slowlyQ4_K_M runs entirely from 19.0 GB of system RAM: Runs slowly, est. 7.2 tok/s (5.0–9.3, theoretical estimate ±30%). UD-Q2_K_XL (heavy quality loss) would reach Runs well, est. 16.7 tok/s (13.3–20.0, calibrated estimate ±20%).
24 GB unified memory · 273.0 GB/s · FP16 8.1 TFLOPS. Assumes 8k context and an f16 KV cache. Verdicts are shown for each memory size.
Verdict by memory size
The page uses the 24 GB configuration; here is every size.
- 24 GBRuns slowlyest. 7.2 tok/s (5.0–9.3, theoretical estimate ±30%)UD-Q2_K_XL (heavy quality loss): Runs well(used on this page)
- 48 GBRuns slowlyest. 10.9 tok/s (8.7–13.1, calibrated estimate ±20%)Try IQ4_XS: Runs well
- 64 GBRuns slowlyest. 10.9 tok/s (8.7–13.1, calibrated estimate ±20%)Try IQ4_XS: Runs well
Every quant of Gemma 4 31B on the Apple M4 Pro
| Quant | File size | Verdict | Speed | Memory | Runs as | Notes |
|---|---|---|---|---|---|---|
| UD-Q2_K_XL | 11.77 GB | Runs well | est. 16.7 tok/s13.3–20.0calibrated estimate ±20% | 13.0 / 16.8 GB | Unified memory | — |
| IQ4_XS | 16.37 GB | Runs slowly | est. 8.0 tok/s5.6–10.4theoretical estimate ±30% | 17.0 GB RAM | CPU only | — |
| Q4_K_Mbaseline | 18.32 GB | Runs slowly | est. 7.2 tok/s5.0–9.3theoretical estimate ±30% | 19.0 GB RAM | CPU only | — |
| Q8_0 | 32.64 GB | Won't run | — | Needs about 33.9 GB; 20.0 GB of memory is free (16.8 GB usable by the GPU) | — | — |
Models with a thinking mode spend extra tokens thinking, so their speed thresholds are 1.5× stricter (30 / 12 / 3 tok/s). With thinking turned off, a model may do one grade better.
Where the memory goes at Q4_K_M
- Weights in system RAM
- 19.0 GB
Context length vs. KV cache
| Context | KV f16 | KV q8_0 | KV q4_0 |
|---|---|---|---|
| 4k | 18.7 GBRuns slowly est. 7.3 tok/s5.1–9.5theoretical estimate ±30% | 18.5 GBRuns slowly est. 7.4 tok/s5.2–9.6theoretical estimate ±30% | 18.4 GBRuns slowly est. 7.4 tok/s5.2–9.6theoretical estimate ±30% |
| 8k | 19.0 GBRuns slowly est. 7.2 tok/s5.0–9.3theoretical estimate ±30% | 18.7 GBRuns slowly est. 7.3 tok/s5.1–9.5theoretical estimate ±30% | 18.5 GBRuns slowly est. 7.4 tok/s5.2–9.6theoretical estimate ±30% |
| 16k | 19.7 GBRuns slowly est. 6.9 tok/s4.9–9.0theoretical estimate ±30% | 19.0 GBRuns slowly est. 7.2 tok/s5.0–9.3theoretical estimate ±30% | 18.7 GBRuns slowly est. 7.3 tok/s5.1–9.5theoretical estimate ±30% |
| 32k | 21.8 GBdoes not fit | 19.8 GBRuns slowly est. 6.9 tok/s4.8–9.0theoretical estimate ±30% | 19.1 GBRuns slowly est. 7.2 tok/s5.0–9.3theoretical estimate ±30% |
| 64k | 24.8 GBdoes not fit | 22.3 GBdoes not fit | 19.9 GBRuns slowly est. 6.9 tok/s4.8–8.9theoretical estimate ±30% |
| 128k | 30.8 GBdoes not fit | 25.8 GBdoes not fit | 23.1 GBdoes not fit |
| 256k | 42.7 GBdoes not fit | 32.7 GBdoes not fit | 27.3 GBdoes not fit |
S = Runs great · A = Runs well · B = Runs slowly · F = Won't run
At Q4_K_M the verdict stays Runs slowly up to 16k context with an f16 KV cache, and up to 32k with q8_0.
Estimated speed
- Generation
- est. 7.2 tok/s (5.0–9.3, theoretical estimate ±30%)
- Prompt processing
- Prompt-processing estimates only apply when the whole model runs on the GPU.
Measured on this exact combination
No public measurement for this exact combination yet; the numbers above are estimates.
If this is not enough
A smaller model that runs well on the Apple M4 Pro
Runs wellMistral Small 3.2 24B: Runs well, est. 13.2 tok/s (10.6–15.9, calibrated estimate ±20%)
A lower quant of Gemma 4 31B
Runs wellUD-Q2_K_XL (11.77 GB, heavy quality loss): Runs well, est. 16.7 tok/s (13.3–20.0, calibrated estimate ±20%)
Gemma 4 31B on the Apple M5 Pro
Runs wellQ4_K_M: Runs well, est. 12.3 tok/s (9.8–14.7, calibrated estimate ±20%)
How to run it
Commands for the Q4_K_M file. Both tools download from Hugging Face on first run.
File: gemma-4-31B-it-Q4_K_M.gguf (18.32 GB) from unsloth/gemma-4-31B-it-GGUF.
llama-server -hf unsloth/gemma-4-31B-it-GGUF:Q4_K_M -c 8192 -ngl 0 -fa on- -ngl 0 keeps everything on the CPU.
- -fa on enables flash attention (needed for KV cache quantization).
- --cache-type-k q8_0 --cache-type-v q8_0 shrinks the KV cache to 0.36 GB at 8k context.
Terminal:
OLLAMA_CONTEXT_LENGTH=8192 ollama serve
ollama run hf.co/unsloth/gemma-4-31B-it-GGUF:Q4_K_M- Ollama defaults to 4k context on GPUs under 24 GB. Set OLLAMA_CONTEXT_LENGTH when starting the server, or /set parameter num_ctx inside the chat.
/set parameter num_ctx 8192
Related pages
Guide: Running local LLMs on a Mac (M4, M5): how much unified memory?
Other models on the Apple M4 Pro
Gemma 4 31B on other hardware
Frequently asked questions
Can I run Gemma 4 31B on the Apple M4 Pro?
Q4_K_M runs entirely from 19.0 GB of system RAM: Runs slowly, est. 7.2 tok/s (5.0–9.3, theoretical estimate ±30%). UD-Q2_K_XL (heavy quality loss) would reach Runs well, est. 16.7 tok/s (13.3–20.0, calibrated estimate ±20%). Q4_K_M needs 19.0 GB at 8k context; the GPU can use 16.8 GB of the unified memory.
How much context can Gemma 4 31B use on the Apple M4 Pro?
At Q4_K_M the verdict stays Runs slowly up to 16k context with an f16 KV cache (1.3 GB of KV) and up to 32k with a q8_0 KV cache (1.4 GB). The model supports up to 256k.
Which quant should I use, and how fast is it?
Q4_K_M (18.32 GB) is the recommended balance: Runs slowly, est. 7.2 tok/s (5.0–9.3, theoretical estimate ±30%). It is also the largest tracked file that gets this verdict.
Speeds are estimates from memory bandwidth, calibrated against public benchmarks, and each one comes with an error band and a confidence label. Real results vary with drivers, backend, context length and thermals.