slotstream
Runs a 125B mixture-of-experts model on an Apple Silicon Mac by streaming experts from SSD.
Content & Media16d ago
- Category
- Content & Media
- Website
- github.com
- Language
- English
- Listed
- 16d ago
- Stars
- 373
- Forks
- 23
carloslfu/slotstream ↗, Swift, MIT, Last commit 1d ago
slotstream keeps the 3.8 GB trunk of Qwen3.8-Flash-Next resident and streams its experts through a fixed pool of slots sized to the memory the machine actually has, so a 105 GB model runs on a 48 GB MacBook Pro at around 12 tokens per second and starts in about 2 seconds. It ships as one Swift binary with no Python and answers the Ollama and OpenAI chat APIs, so existing tools connect unchanged. Needs Apple Silicon, macOS 14 or later, and roughly 110 GB of free disk. MIT licensed.
Like this product?