DiffusionGemma trades quality for speed in open text generation

Google released DiffusionGemma, an experimental open-weight model that generates text by diffusion rather than autoregressive decoding. It is a 26B mixture-of-experts model with 3.8B parameters active per step, shipped under Apache 2.0 on Hugging Face. Instead of producing one token at a time, the model emits blocks of 256 tokens in parallel and refines them across multiple passes, the same general approach used in image diffusion.

The case Google makes is speed. They report over 1,000 tokens per second on an NVIDIA H100 and around 700 on a consumer RTX 5090, with the quantized version fitting in 18 GB of VRAM. The catch: output quality is lower than the standard Gemma 4 at comparable scale. Google frames this as a deliberate tradeoff. Diffusion moves the inference bottleneck from memory bandwidth to raw compute, which suits code infilling, in-line editing, and other interactive workloads where a fast first draft is worth more than the best possible answer.

The release also signals that diffusion text models are moving out of research into something a developer can pick up and try. Whether the quality gap shrinks in future releases is the open question for the technique itself.

Why it matters

If you build tools where latency is the user complaint, DiffusionGemma is the first open weight you can profile against autoregressive baselines on your own workload. The right test is not whether it is as smart as Gemma 4, but whether the speed bump actually moves the product metric you care about.