Why price per million tokens tells you almost nothing

Jan Ilowski makes a simple case that the number everyone quotes for AI models, dollars per million tokens, is close to useless for deciding what to run. The problem is that a token is not a fixed unit of work. Each lab uses its own tokenizer, so the same text becomes a different number of tokens depending on the model. He gives a plain example: a passage that is 160 tokens under one tokenizer becomes 200 under another, and Anthropic's recent tokenizer change raised counts by about 30 percent with no matching change in the sticker price. Compare two models on their per-token rate and you are comparing two different rulers.

The second gap is reasoning you pay for but never see. Chain-of-thought work is billed the same as visible output, and how much of it a model spends varies widely, so two models at the same rate can bill very differently for the same job. His benchmark table drives the point home. DeepSeek V4 Pro lists at $0.435 and $0.87 per million tokens yet finishes a task for around four or five cents. GPT-5.5 looks expensive at $5 and $30 but comes in near $0.99 per task, below Claude Opus on the same work. Sonnet 5 costs more per finished task than Opus 4.8 while doing worse.

His conclusion is to stop shopping on the rate card and measure cost per completed task on your own workload. You can read the full argument and table on his blog.

Why it matters

If you pick models by the price-per-token line in a pricing page, you can end up paying more for worse results. Run your real prompts through the candidates, divide total spend by tasks completed, and choose on that number instead.