The same model, nine agent harnesses, and a 17x cost gap

A benchmark from a group called Runta, posted as FrontierHarness, tries to isolate one variable that usually hides inside coding-agent comparisons: the harness, meaning the scaffolding around the model that plans, calls tools, and manages context. Runta ran nine harnesses across twelve configurations on the same underlying model, Kimi K3, over identical software engineering tasks. All 360 trials started from fresh checkpoint restores so no run got an unfair cache advantage. Same model, same hardware, same tasks. Only the harness changed.

The spread is large. On raw cost per task, Exo Harness came in at $1.05 with a 53.3% pass rate, while Claude Code cost $18.34 for the same work, roughly a 17x difference. Measured by cost per successful task the gap narrows but stays wide, with OpenCode at about $0.06 against Claude Code's $2.88. On quality, Codex led with a 66.7% pass rate at $3.47 per task, and Pi offered the best balance at 60% for $2.43. This is a self-published benchmark on a single model, so the exact figures deserve caution, but the direction is hard to dismiss: how an agent is wrapped matters as much as which model sits inside it. The full results are here.

Why it matters

If you run coding agents at any volume, the harness you happen to use could be quietly multiplying your bill without improving results. Before you blame or switch models, measure your own harness on your own tasks, because a 17x swing pays for a lot of testing.