Princeton researchers pick apart Google's claim that AI agents built an OS for $916

Stephan Rabanser, Sayash Kapoor, and colleagues at Princeton picked apart Google's claim that AI agents using Gemini 3.5 Flash and Antigravity 2.0 had built a working operating system from "a single prompt" at a cost of around $900 in API fees. The post reads as a checklist of what an open-world agent evaluation should clear before its headline number gets reported as a result.

The first issue is the prompt itself. Google's "single prompt" turned out to run thousands of lines, which mixes the contribution of the model with the contribution of the prompt engineer. The second is the definition of autonomy. The writeup says the agents got "no additional guidance or corrections from a human," but does not say whether manual restarts, retries, or approvals counted as guidance. The third is novelty. Toy operating systems are a standard undergraduate assignment with plenty of public implementations, and Google did not check whether the agents wrote new code or pieced it together from existing ones. Finally, Google has not released the prompt, the code, or the run logs, which means no one outside the project can reproduce the work or audit the cost figure.

The authors are careful not to deny that agents can now hold up through long tasks. Their point is narrower: vendor demos that arrive without their artifacts are not evidence, and the AI press should start asking for them.

Why it matters

If you are reading agent benchmarks to plan procurement, this is the right template for what to demand: the full prompt, the run logs, the diff against any baseline code, and an independent evaluation. The era of taking the headline number on faith should be ending.