slotstream
Run a 105 GB AI model on a Mac that can't hold it.
Slotstream runs Qwen3.8-Flash-Next, a 125-billion-parameter open model, on Macs with 16 to 64 GB of memory. It keeps most of the model on the SSD and loads the parts it needs as it writes. Our 48 GB M5 Pro measured 15.86 tokens per second at a 22 GB memory target (how it was measured).
Chat with it, ask it about pictures, or code with it: slotstream launch claude
starts Claude Code on the local model, and Codex, Pi, opencode and Hermes work
the same way. Developers can connect their own apps through its Ollama-,
OpenAI- and Anthropic-compatible APIs or its Swift library.
After a one-time download it works offline, with no Python and no cloud account. The whole engine is one native Swift program on Apple's MLX and Metal; see Built native. Every published number has a recorded method, and the experiments that failed stay in the measurements.
Get started · Speed · Guides · Get help
I'm building Sevra on Slotstream: private, personal AI optimized for your computer. Sevra will choose a tested model for your hardware, keep that choice current as models improve, and let you control what it remembers. The Mac app is in development and runs Slotstream in process; see how it is built and join the waitlist. Slotstream's command-line tool, APIs and Swift library remain independently usable.
Who it's for
Slotstream is built for Macs that cannot hold the model in memory: 16 to 64 GB. That is where the engineering, the measurements and the defaults go, so that frontier-class intelligence runs on the Macs most people already own. It also runs on 96 GB and larger Macs, where the model fits in memory, but it is not optimized for them: engines that keep the whole model in memory report faster replies there. See related projects if that is your Mac.
Will it run on my Mac?
You need an Apple Silicon Mac with at least 16 GB of memory, macOS 14 or later, and about 110 GB of free SSD space. Open About This Mac from the Apple menu to check your chip and memory. On an 8 GB Mac even the smallest memory plan doesn't fit, so Slotstream refuses to start instead of swapping. Windows, Linux and Intel Macs are not supported. The hardware guide has the tested macOS versions.
Speed
tok/s means tokens per second; a token is a small piece of text, often part
of a word. Reply speeds below describe generation after the model has warmed up.
Our development Mac, a 48 GB M5 Pro, measured 15.86 tok/s with 0.2.19 at a 22 GB memory target, in a controlled benchmark on eight prompts the engine was never tuned on. The engine predicts which experts the next layers will need and reads them from the SSD before they are asked for, which changes speed and never the output. The expert lookahead guide has the measurements behind each release. This historical test used smaller prompt passes and disabled prefix caching, leaving more memory for experts. It is not a measurement of today's automatic configuration. A qualified full-answer baseline on 0.2.23 is still pending.
What to expect by memory
Rough planning ranges for warm replies, from community reports and our own measurements, rounded outward. Faster chips and SSDs sit at the top of each range; other apps and memory pressure pull results down.
| Installed RAM | Estimated warm reply speed | Example automatic context window |
|---|---|---|
| 8 GB | Support coming soon. The current model doesn't fit yet. | Not available yet |
| 16–<24 GB | ~1–6 tok/s | 32,768 tokens |
| 24–<48 GB | ~6–16 tok/s | 32,768 through 32 GB; 65,536 at 36 GB |
| 48–<96 GB | ~15–27 tok/s | 32,768 at 48 GB; 131,072 at 64 GB |
| 96 GB+, the model fits in memory | ~20–32 tok/s | 262,144 tokens, the model's full window |
Context examples use decimal-GB memory simulations. A Mac's marketed capacity,
Metal limits and available memory can produce a different plan; slotstream doctor shows the actual choice.
The middle rows are anchored on our M5 Pro's measurement; the top ends of the last two rows come from a 128 GB M5 Max with a larger, manually chosen memory target, and the last row is outside Slotstream's target range. These are estimates, not limits. The hardware guide has the basis of each range, every result measured on real Macs with credits and test conditions, and every automatic memory plan.
Recent prompt-processing results
The changes shipped in 0.2.23 shorten prompt processing and repeated-history work. These measurements use the same 48 GB M5 Pro at a 10 GB target, with three clean pairs per comparison:
| Workload | Matched control | Median times, control → enabled | Median paired time reduction |
|---|---|---|---|
| 16K inventory prompt, MTP on | Larger-read workspace policy off | 155.22 s → 53.94 s prefill | 65.40% |
| 2K prose follow-up, MTP off | Prefix checkpoints disabled | 30.73 s → 4.42 s request | 85.62% |
Both comparisons switch a feature off in the same tested binary. They measure prompt processing or a cached follow-up, not an increase in reply tok/s or a whole-release speedup. Times are arm medians; reductions are medians of paired changes. The hardware guide explains the fixtures and the latest audit's exclusions.
A fresh installed-release study used ordinary caching at the same memory target. These are observed first-read ranges and median exact-repeat request times across the prescribed request order, with capped replies:
| Prompt | Eligible first reads / repeats | First-read prefill range | Median repeated request |
|---|---|---|---|
| 2K code | 4 / 3 | 13.86–28.11 s | 2.79 s |
| 2K prose | 3 / 3 | 14.91–26.51 s | 3.22 s |
The desktop load screen passed for the included observations, but several runs had system swap-ins. Request history changed read batching, so these results do not replace the general speed estimates or decode headline. See the measurement and its limits.
Memory and context
Auto mode picks the memory target, cache size, speculative decoding and
context window for your Mac. It takes the largest window in the table above
that still leaves room for speculative decoding and a complete conversation,
given the memory free at startup, without an unmeasured loss of useful
expert cache. slotstream doctor shows the choice and
why, and --max-context 65536 sets a window yourself, up to 262,144 tokens.
Starting a reply takes time. Slotstream first reads your question and the
conversation history, which can take minutes for a long prompt. Follow-up
turns reuse unchanged history, a new conversation reuses the system prompt
earlier ones started with, and serve --prefix-cache-dir keeps long
conversations and shared system prompts on disk so that they survive a
restart. The hardware guide
has the prompt-reading estimates
for each memory size.
Install
Open Terminal and paste this command:
curl -fsSL https://raw.githubusercontent.com/carloslfu/slotstream/main/install.sh | sh
Run the same command to update. If slotstream isn't found afterward, open
a new terminal window.
Use it
Check your Mac, then ask for a first reply:
slotstream doctor
slotstream run --prompt "Why is the sky blue?"
doctor checks memory and disk space without loading the model. The first
run asks to download it, then prints a reply. This download can take hours,
but you only need to do it once. Interrupted downloads resume when you try
again. Follow the step-by-step setup for more help.
Guides
| What would you like to do? | Guide |
|---|---|
| Chat in Open WebUI or another app | Connect a chat app |
| Ask about a picture | Use an image |
| Start a coding agent on the model, in one command | Use coding agents |
| Code with Claude Code | Use Claude Code |
| Code with Codex | Use Codex |
| Code with Pi or opencode | Use Pi or opencode |
| Work with files and tools through Hermes | Use Hermes |
| Code with fx, Vercel Labs' coding agent | Use fx |
| Fix a problem, move the model, or uninstall | Troubleshooting |
Install chat apps and agents separately. They provide the interface and tools;
Slotstream runs the model. Keep its server running while a connected app uses it.
slotstream launch claude (or codex, pi, opencode, hermes) starts
that agent already connected, and starts the server in the background first
when none is running.
For developers, the engineering guide links to the OpenAI- and Ollama-compatible API references, Swift li