NVIDIA Cosmos World Foundation Model Updates

NVIDIA has announced significant updates to its Cosmos world foundation models platform, bringing three new versions that advance synthetic data generation and physical AI reasoning capabilities. These updates address a key challenge in training AI-driven robots and autonomous vehicles: the need for diverse, representative, and physics-aware training datasets that are expensive and time-consuming to collect from real-world sources.

Key Updates

Cosmos Transfer 2.5 provides faster and more scalable data augmentation from simulation and 3D spatial inputs. Using a ControlNet architecture, it generates photorealistic video sequences with precise spatial alignment and controlled scene composition. It accepts structured inputs including segmentation maps, depth maps, edge maps, motion keypoints, LiDAR scans, and 3D bounding boxes, transforming them into physics-grounded photorealistic videos.

Cosmos Predict 2.5 enhances long-tail scenario generation for sequences up to 30 seconds, delivering up to 10x higher accuracy when post-trained on proprietary or domain-specific data. It now supports multiview outputs, custom camera layouts, and alternate policy outputs such as action simulation.

Cosmos Reason 2 introduces advanced physical AI reasoning with improved spatiotemporal understanding and enhanced timestamp precision. New features include object detection with 2D/3D point localization and bounding box coordinates, reasoning explanations and labels, and expanded long-context support up to 256K input tokens.

Developer Impact

Developers can leverage these models through the NVIDIA Cosmos platform, which integrates with NVIDIA Omniverse for creating realistic 3D simulations. The models enable developers to generate scalable synthetic training data grounded in physics, reducing the need for expensive real-world data collection while improving model generalization across edge cases and real-world variations.