Most working hours fall outside what AI benchmarks measure

Mercor's research team mapped 29 widely used AI benchmarks onto the ONET database of jobs and tasks, and the gap they found is large. Only 924 of 18,797 tasks in ONET have any benchmark coverage, and just 208 of 923 occupations have even one task measured by five or more benchmark prompts. Six of the 22 job families have none at all. Combining that coverage with estimates of how work time is distributed, they conclude that about 90% of American working hours go to tasks that existing benchmarks barely touch.

The coverage that does exist is skewed in a predictable way. Tasks exposed to AI are roughly six times more likely to be benchmarked than tasks that are not, so the field tends to test models on the work it already expects them to do well. Even after controlling for exposure, jobs with a larger total wage bill get more attention: lawyers are heavily covered, while legal secretaries and court clerks are almost absent. Tasks rated most important are measured 6 to 10 times more often than those rated least important.

The team calls the results provisional, since the 29-benchmark sample is not complete and probably understates true coverage. You can read the full breakdown with the per-occupation maps.

Why it matters

If you make build-or-buy decisions from benchmark scores, remember that those scores describe a narrow slice of real work, weighted toward high-wage jobs. For most middle-income roles there is little public evidence of what models can actually do, so you will have to test on your own tasks.