NVIDIA releases Cosmos 3, an open omni-model for physical AI

NVIDIA released Cosmos 3 on Hugging Face, a single foundation model that absorbs what used to be four separate Cosmos models for physical AI: world generation, reasoning, dynamics, and policy. The model takes inputs from text, image, video, and action signals, and outputs any combination of those. That means one checkpoint can act as a vision-language model, a video generator, a forward dynamics model that predicts the next frame from an action, or an inverse dynamics model that infers the action from a video.

There are two open-weight sizes. Cosmos 3 Nano is 16 billion parameters split into an 8B reasoner and 8B generator, sized for a workstation GPU like the RTX PRO 6000. Cosmos 3 Super is 64B (32B plus 32B), aimed at Hopper or Blackwell datacenter cards for large-scale synthetic data generation. The architecture is a Mixture-of-Transformers backbone with modality-specific encoders, an autoregressive subsequence for reasoning, and a diffusion subsequence for generation. The two can be swapped without changing the architecture, which is what lets the same model switch modes.

The release includes Diffusers integration, post-training scripts, and synthetic-data datasets covering robot simulation, physics, spatial reasoning, human motion, autonomous driving, and warehouse operations. License terms are on the Hugging Face model cards.

Why it matters

For teams working on robotics, AV stacks, or warehouse perception, training-data scarcity has been a real constraint, and gluing four specialized models together added complexity. A single open omni-model that runs on one workstation lowers the floor for serious experiments. The 16B Nano is the one to try first.