Can I run gpt-oss-20b on the Apple M4?
Runs greatMXFP4 fits in unified memory (12.9 of 16.8 GB usable by the GPU): Runs great, est. 31.9 tok/s (25.5–38.3, calibrated estimate ±20%).
24 GB unified memory · 120.0 GB/s · FP16 3.8 TFLOPS. Assumes 8k context and an f16 KV cache. Verdicts are shown for each memory size.
Verdict by memory size
The page uses the 24 GB configuration; here is every size.
- 16 GBWon't runMXFP4 doesn't fit
- 24 GBRuns greatest. 31.9 tok/s (25.5–38.3, calibrated estimate ±20%)(used on this page)
- 32 GBRuns greatest. 31.9 tok/s (25.5–38.3, calibrated estimate ±20%)
Every quant of gpt-oss-20b on the Apple M4
| Quant | File size | Verdict | Speed | Memory | Runs as | Notes |
|---|---|---|---|---|---|---|
| MXFP4baseline | 12.11 GB | Runs great | est. 31.9 tok/s25.5–38.3calibrated estimate ±20% | 12.9 / 16.8 GB | Unified memory | — |
Models with a thinking mode spend extra tokens thinking, so their speed thresholds are 1.5× stricter (30 / 12 / 3 tok/s). With thinking turned off, a model may do one grade better.
Where the memory goes at MXFP4
- Weights
- 12.1 GB
- KV cache
- 0.2 GB
- Compute buffer
- 0.6 GB
- Free
- 3.9 GB
- Not usable by the GPU
- 7.2 GB
Context length vs. KV cache
| Context | KV f16 | KV q8_0 | KV q4_0 |
|---|---|---|---|
| 4k | 12.7 GBRuns great est. 33.1 tok/s26.4–39.7calibrated estimate ±20% | 12.7 GBRuns great est. 33.6 tok/s26.9–40.4calibrated estimate ±20% | 12.7 GBRuns great est. 33.9 tok/s27.2–40.7calibrated estimate ±20% |
| 8k | 12.9 GBRuns great est. 31.9 tok/s25.5–38.3calibrated estimate ±20% | 12.8 GBRuns great est. 33.0 tok/s26.4–39.6calibrated estimate ±20% | 12.7 GBRuns great est. 33.6 tok/s26.9–40.3calibrated estimate ±20% |
| 16k | 13.2 GBRuns well est. 29.8 tok/s23.8–35.8calibrated estimate ±20% | 13.0 GBRuns great est. 31.7 tok/s25.4–38.1calibrated estimate ±20% | 12.9 GBRuns great est. 32.9 tok/s26.3–39.5calibrated estimate ±20% |
| 32k | 13.7 GBRuns well est. 26.3 tok/s21.1–31.6calibrated estimate ±20% | 13.3 GBRuns well est. 29.5 tok/s23.6–35.4calibrated estimate ±20% | 13.1 GBRuns great est. 31.6 tok/s25.3–37.9calibrated estimate ±20% |
| 64k | 14.8 GBRuns well est. 21.4 tok/s17.1–25.6calibrated estimate ±20% | 14.1 GBRuns well est. 25.9 tok/s20.7–31.1calibrated estimate ±20% | 13.7 GBRuns well est. 29.3 tok/s23.4–35.1calibrated estimate ±20% |
| 128k | 15.3 GBRuns slowly est. 9.7 tok/s6.8–12.6theoretical estimate ±30% | 15.5 GBRuns well est. 20.8 tok/s16.7–25.0calibrated estimate ±20% | 14.7 GBRuns well est. 25.5 tok/s20.4–30.6calibrated estimate ±20% |
S = Runs great · A = Runs well · B = Runs slowly · F = Won't run
At MXFP4 the verdict stays Runs great up to 8k context with an f16 KV cache, and up to 16k with q8_0.
Estimated speed
- Generation
- est. 31.9 tok/s (25.5–38.3, calibrated estimate ±20%)
- Prompt processing
- about 211 tok/s (127–296, rough estimate ±40%)
Measured on this exact combination
No public measurement for this exact combination yet; the numbers above are estimates.
How to run it
Commands for the MXFP4 file. Both tools download from Hugging Face on first run.
File: gpt-oss-20b-MXFP4.gguf (12.11 GB) from ggml-org/gpt-oss-20b-GGUF.
llama-server -hf ggml-org/gpt-oss-20b-GGUF:MXFP4 -c 8192 -ngl all -fa on- -ngl all loads every layer on the GPU.
- -fa on enables flash attention (needed for KV cache quantization).
- --cache-type-k q8_0 --cache-type-v q8_0 shrinks the KV cache to 0.11 GB at 8k context.
Terminal:
OLLAMA_CONTEXT_LENGTH=8192 ollama serve
ollama run hf.co/ggml-org/gpt-oss-20b-GGUF:MXFP4- Ollama defaults to 4k context on GPUs under 24 GB. Set OLLAMA_CONTEXT_LENGTH when starting the server, or /set parameter num_ctx inside the chat.
/set parameter num_ctx 8192
Related pages
Guide: Running local LLMs on a Mac (M4, M5): how much unified memory?
Other models on the Apple M4
gpt-oss-20b on other hardware
Frequently asked questions
Can I run gpt-oss-20b on the Apple M4?
MXFP4 fits in unified memory (12.9 of 16.8 GB usable by the GPU): Runs great, est. 31.9 tok/s (25.5–38.3, calibrated estimate ±20%). MXFP4 needs 12.9 GB at 8k context; the GPU can use 16.8 GB of the unified memory.
How much context can gpt-oss-20b use on the Apple M4?
At MXFP4 the verdict stays Runs great up to 8k context with an f16 KV cache (0.2 GB of KV) and up to 16k with a q8_0 KV cache (0.2 GB). The model supports up to 128k.
Which quant should I use, and how fast is it?
MXFP4 (12.11 GB) is the only tracked file: Runs great, est. 31.9 tok/s (25.5–38.3, calibrated estimate ±20%).
Speeds are estimates from memory bandwidth, calibrated against public benchmarks, and each one comes with an error band and a confidence label. Real results vary with drivers, backend, context length and thermals.