Can I run Gemma 4 26B-A4B on the Apple M4?
Runs wellQ4_K_M fits in unified memory (17.7 of 22.4 GB usable by the GPU): Runs well, est. 23.6 tok/s (18.9–28.4, calibrated estimate ±20%). UD-Q2_K_XL (heavy quality loss) would reach Runs great, est. 33.4 tok/s (26.7–40.1, calibrated estimate ±20%).
32 GB unified memory · 120.0 GB/s · FP16 3.8 TFLOPS. Assumes 8k context and an f16 KV cache. Verdicts are shown for each memory size.
Verdict by memory size
The page uses the 32 GB configuration; here is every size.
- 16 GBWon't runTry UD-Q2_K_XL (heavy quality loss): Runs slowlyest. 29.4 tok/s (20.5–38.2, theoretical estimate ±30%)
- 24 GBRuns slowlyest. 19.0 tok/s (13.3–24.6, theoretical estimate ±30%)Try IQ4_XS: Runs wellUD-Q2_K_XL (heavy quality loss): Runs great
- 32 GBRuns wellest. 23.6 tok/s (18.9–28.4, calibrated estimate ±20%)UD-Q2_K_XL (heavy quality loss): Runs great(used on this page)
Every quant of Gemma 4 26B-A4B on the Apple M4
| Quant | File size | Verdict | Speed | Memory | Runs as | Notes |
|---|---|---|---|---|---|---|
| UD-Q2_K_XL | 10.55 GB | Runs great | est. 33.4 tok/s26.7–40.1calibrated estimate ±20% | 11.3 / 22.4 GB | Unified memory | — |
| IQ4_XS | 13.60 GB | Runs well | est. 25.0 tok/s20.0–30.0calibrated estimate ±20% | 14.3 / 22.4 GB | Unified memory | — |
| Q4_K_Mbaseline | 16.95 GB | Runs well | est. 23.6 tok/s18.9–28.4calibrated estimate ±20% | 17.7 / 22.4 GB | Unified memory | — |
| Q8_0 | 26.86 GB | Runs slowly | est. 12.2 tok/s8.6–15.9theoretical estimate ±30% | 27.0 GB RAM | CPU only | CPU inference — capped at Runs slowly |
Models with a thinking mode spend extra tokens thinking, so their speed thresholds are 1.5× stricter (30 / 12 / 3 tok/s). With thinking turned off, a model may do one grade better.
Where the memory goes at Q4_K_M
- Weights
- 17.0 GB
- KV cache
- 0.2 GB
- Compute buffer
- 0.6 GB
- Free
- 4.7 GB
- Not usable by the GPU
- 9.6 GB
Context length vs. KV cache
| Context | KV f16 | KV q8_0 | KV q4_0 |
|---|---|---|---|
| 4k | 17.6 GBRuns well est. 24.2 tok/s19.3–29.0calibrated estimate ±20% | 17.5 GBRuns well est. 24.4 tok/s19.5–29.3calibrated estimate ±20% | 17.5 GBRuns well est. 24.5 tok/s19.6–29.5calibrated estimate ±20% |
| 8k | 17.7 GBRuns well est. 23.6 tok/s18.9–28.4calibrated estimate ±20% | 17.6 GBRuns well est. 24.1 tok/s19.3–28.9calibrated estimate ±20% | 17.6 GBRuns well est. 24.4 tok/s19.5–29.3calibrated estimate ±20% |
| 16k | 17.9 GBRuns well est. 22.6 tok/s18.1–27.2calibrated estimate ±20% | 17.8 GBRuns well est. 23.6 tok/s18.8–28.3calibrated estimate ±20% | 17.7 GBRuns well est. 24.1 tok/s19.3–28.9calibrated estimate ±20% |
| 32k | 18.4 GBRuns well est. 20.9 tok/s16.7–25.1calibrated estimate ±20% | 18.1 GBRuns well est. 22.5 tok/s18.0–27.0calibrated estimate ±20% | 17.9 GBRuns well est. 23.5 tok/s18.8–28.2calibrated estimate ±20% |
| 64k | 19.4 GBRuns well est. 18.1 tok/s14.5–21.7calibrated estimate ±20% | 18.8 GBRuns well est. 20.7 tok/s16.5–24.8calibrated estimate ±20% | 18.4 GBRuns well est. 22.4 tok/s17.9–26.9calibrated estimate ±20% |
| 128k | 21.3 GBRuns well est. 14.3 tok/s11.4–17.2calibrated estimate ±20% | 20.1 GBRuns well est. 17.8 tok/s14.2–21.3calibrated estimate ±20% | 19.4 GBRuns well est. 20.5 tok/s16.4–24.6calibrated estimate ±20% |
| 256k | 22.3 GBRuns slowly est. 6.5 tok/s4.6–8.5theoretical estimate ±30% | 19.8 GBRuns slowly est. 9.5 tok/s6.7–12.4theoretical estimate ±30% | 21.4 GBRuns well est. 17.5 tok/s14.0–21.0calibrated estimate ±20% |
S = Runs great · A = Runs well · B = Runs slowly · F = Won't run
At Q4_K_M the verdict stays Runs well up to 128k context with an f16 KV cache, and up to 128k with q8_0.
Estimated speed
- Generation
- est. 23.6 tok/s (18.9–28.4, calibrated estimate ±20%)
- Prompt processing
- about 151 tok/s (91–212, rough estimate ±40%)
Measured on this exact combination
No public measurement for this exact combination yet; the numbers above are estimates.
If this is not enough
A larger model that also runs well on the Apple M4
Runs greatMiniCPM-V 4.7 35B-A3B: Runs great, est. 37.5 tok/s (30.0–45.0, calibrated estimate ±20%)
A lower quant of Gemma 4 26B-A4B
Runs greatUD-Q2_K_XL (10.55 GB, heavy quality loss): Runs great, est. 33.4 tok/s (26.7–40.1, calibrated estimate ±20%)
Gemma 4 26B-A4B on the Apple M6
Runs greatQ4_K_M: Runs great, est. 32.8 tok/s (26.3–39.4, calibrated estimate ±20%)
How to run it
Commands for the UD-Q4_K_M file. Both tools download from Hugging Face on first run.
File: gemma-4-26B-A4B-it-UD-Q4_K_M.gguf (16.95 GB) from unsloth/gemma-4-26B-A4B-it-GGUF.
llama-server -hf unsloth/gemma-4-26B-A4B-it-GGUF:UD-Q4_K_M -c 8192 -ngl all -fa on- -ngl all loads every layer on the GPU.
- -fa on enables flash attention (needed for KV cache quantization).
- --cache-type-k q8_0 --cache-type-v q8_0 shrinks the KV cache to 0.09 GB at 8k context.
Terminal:
OLLAMA_CONTEXT_LENGTH=8192 ollama serve
ollama run hf.co/unsloth/gemma-4-26B-A4B-it-GGUF:UD-Q4_K_M- Ollama defaults to 4k context on GPUs under 24 GB. Set OLLAMA_CONTEXT_LENGTH when starting the server, or /set parameter num_ctx inside the chat.
/set parameter num_ctx 8192
Related pages
Guide: Running local LLMs on a Mac (M4, M5): how much unified memory?
Other models on the Apple M4
Gemma 4 26B-A4B on other hardware
Frequently asked questions
Can I run Gemma 4 26B-A4B on the Apple M4?
Q4_K_M fits in unified memory (17.7 of 22.4 GB usable by the GPU): Runs well, est. 23.6 tok/s (18.9–28.4, calibrated estimate ±20%). UD-Q2_K_XL (heavy quality loss) would reach Runs great, est. 33.4 tok/s (26.7–40.1, calibrated estimate ±20%). Q4_K_M needs 17.7 GB at 8k context; the GPU can use 22.4 GB of the unified memory.
How much context can Gemma 4 26B-A4B use on the Apple M4?
At Q4_K_M the verdict stays Runs well up to 128k context with an f16 KV cache (2.7 GB of KV) and up to 128k with a q8_0 KV cache (1.4 GB). The model supports up to 256k.
Which quant should I use, and how fast is it?
Q4_K_M (16.95 GB) is the recommended balance: Runs well, est. 23.6 tok/s (18.9–28.4, calibrated estimate ±20%). It is also the largest tracked file that gets this verdict.
Speeds are estimates from memory bandwidth, calibrated against public benchmarks, and each one comes with an error band and a confidence label. Real results vary with drivers, backend, context length and thermals.