Mistral's Leanstral 1.5 saturates a math-proof benchmark and finds real bugs

Mistral released Leanstral 1.5, an open model built to write formal proofs in Lean 4. The headline number is that it saturates miniF2F, reaching 100% on both the validation and test sets, a benchmark that until recently no system could clear. It also solves 587 of 672 problems on PutnamBench and reaches 87% on FATE-H for abstract algebra. The model is a mixture-of-experts with 119B total parameters but only 6B active, released under Apache-2.0 and available now as a free API endpoint and on Hugging Face.

The more useful part is what happens when you give it room to think. On PutnamBench it solved 44 problems at a 50k-token budget and 587 at 4 million tokens, a clean case of test-time scaling. And it is not confined to contest math. Pointed at real open-source repositories, it found flaws including 5 that had never been reported on GitHub. One was a zigzag decoding function where the expression (value + 1) overflowed, crashing in debug mode and silently corrupting data in release mode. Running the full PutnamBench set costs about $4 per problem.

Why it matters

If you write code where correctness can be specified, such as cryptography, parsers, or low-level libraries, a proof engine that finds real overflow bugs and ships with open weights is something you can point at your own code today, not a demo. The bug-finding results are the reason to try it before the benchmark scores.