A new benchmark shows agents rebuilding software that would take humans weeks

Most coding benchmarks ask a model to fix a bug or add a feature in an existing codebase, work measured in minutes or hours. MirrorCode, built by Epoch AI together with METR, asks for something much larger. The agent gets only a detailed specification and the ability to run the original program, never its source code, and has to reimplement the whole thing from scratch so that it passes a large suite of end-to-end tests. The benchmark covers 24 target programs across Unix utilities, bioinformatics, interpreters, cryptography, compression, and more.

The standout result is gotree, a phylogenetics toolkit of roughly 16,905 lines of Go. Claude Opus 4.6 rebuilt it as 7,644 lines of Rust that passed about 99.95% of the tests, 2,000 of 2,001. Epoch estimates the same job would take a skilled engineer somewhere between two and seventeen weeks. The model got there by spending around 280 million tokens, near $550 of compute. Not everything fell. Pkl, a configuration language, stayed unsolved even after a run that stretched to roughly 9,300 messages and 27 context compactions before the agent gave up at 35% of tests passing.

The claim in Epoch's writeup is that models can now carry a coding task that runs for the equivalent of weeks of human work, as long as the goal is precise and checkable. The catch is in those two conditions: a clear spec and an automatic way to verify the result.

Why it matters

If you build with coding agents, this points to where they are already useful: large, well-specified rebuilds with tests to grade them, not vague open-ended work. It also sets a price. Weeks of engineering for a few hundred dollars of tokens is worth modeling into your own build-versus-buy decisions, with verification as the part you still own.