How Databricks cut its AI coding bill without losing quality

Databricks published a detailed look at how it keeps the cost of AI coding tools under control as usage grows across its engineering teams. The most effective move was also the least glamorous: switching to cheaper models where they are good enough. The team benchmarked GLM models, found their price to performance highly competitive, and rolled them out internally. It also declined to adopt Opus 4.7 after testing showed it did not meaningfully beat 4.6 on their work, a reminder that the newest model is not always worth its price.

Routing did much of the rest. Databricks built smart routing into its Unity AI Gateway, sending each request to request-level, task-level, or escalation-based handling, and reports more than a 30% cut in average task cost while keeping quality close to the most expensive option. Separately, tuning the harness and caching settings cut generated tokens by almost half with no drop in quality. Instead of hard spending cutoffs, developers see live spend dashboards and hit self-clearing gates that downshift them to cheaper models. Databricks open-sourced two of the tools behind this: Unity AI Gateway for cost management and Omnigent, a meta-harness for choosing models.

Why it matters

If your team's coding-agent spend is climbing, most of the savings here need no model retraining: pick cheaper models that pass your own benchmarks, route by task, and fix caching. The open-sourced gateway gives you a place to start measuring before you cut.