← all news

Researchers pulled hidden reasoning out of Claude, GPT, and Gemini APIs

AI · · · source (simonwillison.net)

A new paper (arXiv 2608.09867) shows that the encrypted reasoning blocks the big labs return through their APIs were not as sealed as they looked. Anthropic, OpenAI, and Google all send back encrypted "thinking" that clients are meant to pass along but not read. The researchers found that every model in a family used the same encryption key, so a block produced by a strong model could be replayed into a weaker sibling. Once inside the weaker model, a jailbreak prompt got it to transcribe the hidden reasoning in plain text. For Claude Haiku 4.5, the prompt was as blunt as asking it to transcribe the reasoning attached to the turn, verbatim.

Simon Willison, who wrote up the work, notes that this exposed reasoning was clearly never meant for human eyes, and that models tend to trust their own thinking traces. That second point is the sharper risk: the paper describes a prompt-injection variant where a crafted encrypted trace convinces a model to follow instructions it would normally refuse. All three providers have since patched the flaw, and the authors say they could not repeat the attacks afterward.

Why it matters

If you build on these APIs, treat provider reasoning tokens as sensitive rather than opaque plumbing, and do not assume an encrypted block is trustworthy just because the model produced it. The bug is fixed, but the lesson about a model trusting its own hidden reasoning is the part worth keeping.

SecurityResearchLLMs