OpenRouter's convenience comes with inconsistent results

OpenRouter lets you call many models through one API and quietly picks a backend provider for you, often the cheapest or fastest available. Simon Willison, pointing to a writeup by Mohamed Moustafa, explains why that convenience can bite you: the same model ID can behave differently depending on which provider handles the request. Providers run different serving software with different optimizations and settings, so the output you get is not fully determined by the model you asked for.

The specifics matter for anyone shipping on top of this. Some providers behind a vision model do not actually support image input, so a request that works one day can fail the next when routing shifts. The reasoning-effort setting is handled differently from one provider to another, which means a control you rely on may not do the same thing everywhere. Moustafa's fix is to stop leaving it to chance: use the provider.only option to pin requests to a specific provider, and call the /endpoints method to see which providers are actually available for a given model ID before you commit. The takeaway is not to avoid OpenRouter, but to treat automatic routing as something you opt into deliberately rather than accept by default.

Why it matters

If you have built anything on OpenRouter and assumed a model ID gives you consistent behavior, that assumption is wrong, and it can surface as intermittent failures you struggle to reproduce. Pinning the provider turns a hidden variable into one you control, which is worth doing before your next debugging session over a bug that only appears sometimes.