A researcher argues neural networks were never that messy

For years the standard story in interpretability has been that neural networks are messy by nature: single neurons fire for unrelated concepts, and useful features are crammed together in "superposition" because the network has more things to represent than neurons to hold them. Nick Cammarata, now at Goodfire, argues the canonical evidence for this was misread, and that the apparent mess mostly comes from fixable causes like weak tools, short datasets, and dated architecture. He is careful to call it a working hypothesis, not a settled result.

The sharpest piece of evidence is a rerun of InceptionV1. Retrain its last layer with ordinary modern changes, residual connections and LayerNorm, at the same size and on the same data, and Cammarata estimates that 99 to 100 percent of its last-layer neurons become interpretable. The famous "cat/car" neuron that launched the polysemanticity story is reinterpreted as a simple V-shape detector, with its two poles tracking pointed versus round shapes rather than a list of objects. Preliminary audits of a language model, Qwen3-32B, put roughly 90 percent of neurons at one main job and 2 to 4 percent still unexplained. Compare that to Anthropic's 2023 monosemanticity work, where the median neuron scored zero, or his own 2023 OpenAI result of about 1.7 percent interpretable. He is blunt that any single cleanliness number depends on thresholds, and that scaling the method to very large models is still mostly untested.

Why it matters

If this holds, a lot of interpretability effort has been spent fighting a problem that was partly self-inflicted. For anyone building tools to understand models, the practical question becomes whether to keep investing in sparse autoencoders for superposition or to first fix the architecture and tooling and see what is already legible.