Gemini learns to watch video like a person, not at a fixed frame rate

Google added agentic video understanding to Gemini, a change in how the model reads video. Until now, models like these process video at a fixed frame rate, usually one frame per second, whether or not those frames matter. The new approach lets Gemini decide what to watch: it can search, scan, and inspect specific segments across frames, audio, and transcripts, choosing the speed and the modality for each part rather than grinding through everything at one rate.

The efficiency gains are the reason to pay attention. Google reports up to 88% fewer tokens, up to 66% lower cost, and up to 7% better accuracy, with the largest gains on long videos that run from ten minutes to several hours. Instead of loading a whole multi-hour file at full resolution, the model calls an internal tool to pull only the relevant part, which is where most of the waste in long-video processing comes from.

The practical uses follow from that control. Google points to sub-second moment retrieval for automated editing, needle-in-a-haystack search across long footage, anomaly detection with dynamic resampling, and accurate counting of repeated actions or distinct objects. The feature is available through the Gemini API on Gemini 3.7 Flash, 3.6 Flash, and 3.5 Flash-Lite.

Why it matters

If you build anything on top of video, from editing tools to security review to media search, the old fixed-frame approach made long clips expensive and slow. Letting the model choose what to inspect changes the cost math, so it is worth re-checking whether tasks you previously ruled out as too costly now fit your budget.