AI models jumped from 28% to 80% at spotting IKEA assembly mistakes

Epoch AI built a small, oddly practical test called the Furniture Assembly Benchmark, and the ten-month trend line is the story. Models get photographs of partly built IKEA furniture, the official manual, and tools to inspect the images, then have to find every step with a mistake and describe it, or confirm the build is correct. There are 60 images across three furniture builds of varying difficulty, and each model gets a budget of 80 agentic steps per image. Grading is done in two passes: check whether the model flagged the right steps, then use a lenient model-based judge on how it described each error.

The results move fast. Last November, the best model was Anthropic's Claude Opus 4.5 at 28 percent. By this September, OpenAI's GPT-6 Astra reached 80 percent, a 52-point gain in ten months. Claude Fable 5.1 sits at 70 percent and Claude Opus 5 at 61, while Epoch notes that Chinese open-weight models such as Kimi K3 trail the frontier by about seven months. The task rewards patient visual and spatial reasoning: you have to line up a photo against a diagram and catch a screw in the wrong hole or a panel facing the wrong way.

This is a narrow benchmark and it does not prove anything about robots doing the assembly, only about models judging photos of it. But the point Epoch draws is that error-checking from images is the kind of skill behind real work like appliance repair and maintenance, where a system watches what you are doing and tells you where you went wrong.

Why it matters

If you are building assistants that guide people through physical tasks, this suggests the underlying visual-checking ability is now good enough to prototype seriously, at least on the frontier models. The steep curve also means benchmarks you dismissed as too hard a year ago are worth re-testing, because the gap between "AI fails at this" and "AI mostly handles this" is closing in months, not years.