Running a 2.8-trillion-parameter model on a laptop by streaming from SSDs

A project called Deltafin runs the full, uncompressed Kimi K3 model, 2.8 trillion parameters, on a single MacBook. The trick is that it does not try to fit the model in memory. Kimi K3 is a mixture-of-experts model, so only a few experts fire for any given token, and Deltafin streams those expert weights on demand from four SSDs, spreading the reads across the drives so the disk I/O runs in parallel. The resident core is quantized to int8 while the experts stay in BF16, which keeps memory pressure down without touching the parts that carry most of the model's quality.

The measured result on an M5 Max with 128GB of RAM is about 1.0 to 1.1 tokens per second with speculative decoding turned on, where small draft models propose tokens that K3 then checks. Throughput scales with drives: one SSD gets roughly half the speed of four, three gets about 90 percent. The honest caveats are in the repository: prefill takes around 375 seconds for a 512-token prompt, the full download is 1.7TB, and this is a research demo, not a product. Every reported number is a single cold run with the stated prompt.

Why it matters

This will not replace an API for interactive use, but it shows that the ceiling on local inference is storage bandwidth, not model size. If you care about running frontier-scale open models privately, the path is now fast SSDs and smart streaming rather than waiting for enough RAM to hold the whole thing.