Claude Sonnet 5's new tokenizer quietly raises the real price

Anthropic pitches Claude Sonnet 5 as performance close to Opus 4.8 at a lower price, with a 1 million token context window and up to 128,000 output tokens. The list price looks unchanged from Sonnet 4.6 at $3 and $15 per million input and output tokens, with introductory rates of $2 and $10 running through August 31. Simon Willison looked past the headline and found the catch: the model ships a new tokenizer that turns the same text into roughly 30% more tokens, so the bill you actually pay goes up even though the per-token rate did not.

He measured it on real documents instead of trusting the round number. English prose came out about 1.42 times more expensive than on Sonnet 4.6, Spanish about 1.33 times, and Python code about 1.27 times. Simplified Mandarin was the exception and stayed roughly the same. The effect varies by content, so the only honest way to know your own number is to run your own text through it. Willison also flags a quieter API change: temperature, top_p, and top_k are no longer supported, which will break code that tuned sampling.

The practical takeaway is that a like-for-like price comparison between models is now misleading. Two models can advertise the same dollar figure per million tokens and still cost very different amounts for the same job, because tokenization decides how many tokens your input becomes. You can read Willison's full measurements before you switch.

Why it matters

If you run Sonnet at any volume, budget for a real cost increase of roughly a quarter to a half on English and code, not the flat price the pricing page implies. Measure your own workload before migrating, and stop comparing models on advertised per-token rates alone.