← all news

No, looped transformers are not hiding the model's reasoning

AI · · · source (magazine.sebastianraschka.com)

The Information reported that GPT-6 Astra relies on "looped transformers" and that this somehow hides the model's chain of thought. Sebastian Raschka takes both claims apart in a clear explainer. A looped transformer reuses the same block of weights several times instead of stacking distinct blocks. Nanbeige, for example, runs the same 22 transformer blocks twice, so it gets 44 effective layers of computation while storing only 22 sets of weights. Variants go further: the Ouro model applies 48 blocks four times with learned exit gates, and Mixture-of-Recursions lets a router decide how many loops each token needs. The appeal is efficiency, and Raschka points to SMELT results showing looped designs reach the same quality with 6.8 to 18 percent less training compute.

The interesting part is his pushback on the "hidden reasoning" story. Astra produces shorter reasoning traces, and some read that as the model concealing its work. Raschka argues the simpler explanation is that a stronger model just makes fewer wrong turns, so it needs fewer tokens to get there. He also notes OpenAI has hidden raw reasoning from users since its very first reasoning model, so looping changes nothing about that policy. On capability, he cites Astra scoring 99.9 percent on ARC-AGI-3 against 7.8 percent for GPT-5.6 Sol, and OpenAI's own claim that Astra's computation graph is within a factor of two of GPT-4's depth.

Why it matters

If you follow model releases, this is a useful filter against reading architecture leaks as conspiracy. Shorter traces are most likely a sign of competence, not concealment, and looped transformers are an old efficiency idea, not a trick to keep you in the dark.

LLM ArchitectureOpenAIResearch