A 17 MB speech-to-text model that runs on the CPU

Cactus released Whistle, an open speech-to-text model that is 16.9 MB and runs on the CPU with no dependencies, shipped as a single file. For comparison, Whisper base is 145.3 MB and Moonshine tiny v2 is 41.9 MB. Whistle loads into the same C++ engine as the company's text model, so one binary can turn a spoken clip into tool calls and return both the transcript and the JSON calls.

The design is worth reading if you work on on-device audio. Thirty seconds of 16 kHz audio becomes 80 log-mel bins, and a convolutional stem compresses 3,000 frames down to 375, one frame per 80 milliseconds. The encoder is eight attention blocks with non-causal attention, so each frame can see the whole clip, and its keys and values are computed once per clip and reused across beams during a five-beam search. Keyword biasing runs through an Aho-Corasick automaton, and silent clips return empty output early.

The speed numbers are the point. On an Apple M4 Pro CPU, time to first token is 11.1 ms for ten seconds of audio, and decoding runs at 1,319 tokens per second against 266 for Whisper base. Whistle reports lower word error rates than both comparison models on LibriSpeech, SPGISpeech, Earnings-22 and the FLEURS average, measured over 86,174 utterances, though Whisper base still wins on TED-LIUM, AMI and the MLS average. It covers seven languages.

Why it matters

If you build for phones, wearables, cars or microcontrollers, a model this small lets you run speech recognition locally, so audio never leaves the device, and wire it straight to tool calls instead of maintaining a separate speech pipeline.