NVIDIA's open diffusion LLMs run six times faster than autoregressive baselines

NVIDIA released a family of diffusion language models under its Nemotron-Labs research program: 3B, 8B, and 14B text models plus an 8B vision-language model, all in base and chat variants with weights on Hugging Face. The interesting part is not the size. Each checkpoint can switch between three generation modes with the same weights, running autoregressive decoding for compatibility, block-wise diffusion for parallel generation, or self-speculation, where a diffusion draft is verified by an AR pass.

The numbers favor diffusion. NVIDIA says the 8B model beats Qwen3-8B by 1.2 points on average accuracy after joint AR-and-diffusion training on 1.3 trillion pretraining tokens and 45 billion SFT tokens. In self-speculation mode on a B200 GPU, it runs at about 865 tokens per second, roughly four times the autoregressive baseline, and reports up to 6.4× speedups on a hardware-agnostic tokens-per-forward-pass metric. Block-wise attention keeps the KV cache compatible with existing serving stacks, which is the bottleneck that has kept earlier diffusion-LM work academic.

The training recipe, technical report, and SGLang integration are all open. That matters because, until now, diffusion language models have lived in papers and small demos. Putting weights and a working serving path on Hugging Face turns the AR versus diffusion debate into something developers can try on their own workloads.

Why it matters

If you serve LLMs at high concurrency, this is the first credible diffusion LLM you can actually deploy. The speedup is real but assumes self-speculation, so test on your latency profile before swapping out an AR model; diffusion buys throughput at the cost of more complex inference scheduling.