AI2 releases MolmoMotion, an open model for 3D motion forecasting

The Allen Institute for AI has released MolmoMotion, an open model that predicts how an object will move in three dimensions before it moves. You give it a single video frame, mark a few points on an object, and write an instruction like move the bowl. The model forecasts where those marked points travel over the next few seconds. It comes in two versions built on the Molmo 2 vision-language backbone: an autoregressive model that writes future coordinates out as text, which is most accurate when the path is well defined, and a flow-matching model that works in continuous 3D space and handles cases where an instruction could play out several ways.

The release is more than a model. AI2 also published MolmoMotion-1M, a dataset of 1.16 million videos covering 736 motion types and 5,600 objects, which they describe as the largest collection of action-described, object-grounded 3D point trajectories built so far, plus PointMotionBench, a 2,700-clip benchmark spanning indoor manipulation, hand-object interaction, and outdoor scenes. On a robotics pick-and-place task the model reached 76.3% success against 56.0% for the Molmo 2 baseline, and the gap is wider early in training: 51% versus 19% at 10,000 steps. Used to guide video generation, it beat a larger model on four of five quality measures.

Weights, the dataset, the benchmark, and the technical report are all open, the usual AI2 pattern. You can read the announcement on the AI2 blog.

Why it matters

If you work on robotics or video generation, motion forecasting has been the missing piece between recognizing an object and acting on it. An open model plus a million-example dataset lowers the cost of trying this on your own task instead of waiting for a closed lab to ship it.