Ai2's FairStress finds models bend their fairness under pressure

Ai2 released FairStress, a benchmark that argues the usual fairness check is too shallow. Most tests ask whether a model picks the same candidate when you swap the demographics. FairStress instead measures whether a well-supported decision is equally hard to overturn whichever group benefits, how the model explains its choice, and what its internal activations reveal. The scenarios are 300 two-candidate decisions across 29 domains such as hiring, lending, and criminal justice, combined with "adversarial prefixes" that argue for one side. That produces over 13 million test cases, with a 50K-item core subset selected using item response theory.

The headline finding is what the authors call anti-stereotypical over-correction. With no identity shown, models pick the stronger candidate at nearly equal rates. As identity becomes explicit, they grow more willing to abandon a correct choice when doing so favors a minority-coded candidate. On ambiguous cases, Claude Opus 5 and GPT-5.5 chose the minority-coded candidate about 60 to 65% of the time. Under matched persuasive prefixes, switches toward minority-coded candidates ran 8.8 to 42.4 points more common than the reverse. The explanations diverged too: minority-coded choices drew words like "resilience" and "diversity," while majority-coded ones used "evidence" and "criteria." Of nine debiasing methods the team tried, most did not reliably close the gap, and one activation-steering fix crashed accuracy on clear-cut cases from 98% to 47%.

Why it matters

If you evaluate models for fair decisions, checking that the final answer stays the same across demographic swaps can pass a model that is still easy to push off a correct call in one direction. FairStress is a concrete way to probe that, and the finding that most debiasing fixes failed is a warning against assuming a quick patch will work.