NVIDIA Cosmos Foundation Models Updated
NVIDIA has released the latest versions of its Cosmos world foundation models (WFMs), a suite of AI systems designed to accelerate synthetic data generation and physical AI development for robotics, autonomous vehicles, and other physical AI applications.
Three Key Model Updates
Cosmos Transfer 2.5 enables faster and more scalable data augmentation from simulation and 3D spatial inputs. It generates photorealistic video sequences from structured inputs like segmentation maps, depth maps, edge maps, LiDAR scans, and 3D bounding boxes. The ControlNet-based architecture preserves pretrained knowledge while enabling fine-grained control over scene composition, object placement, and motion dynamics across diverse environments and lighting conditions.
Cosmos Predict 2.5 enhances long-tail scenario generation for sequences up to 30 seconds, delivering up to 10x higher accuracy when post-trained on proprietary or domain-specific data. It supports multiview outputs, custom camera layouts, and alternate policy outputs such as action simulation, making it useful for generating diverse training scenarios.
Cosmos Reason 2 introduces advanced physical AI reasoning with improved spatiotemporal understanding and timestamp precision. It adds object detection with 2D/3D point localization and bounding box coordinates, along with reasoning explanations and labels. The model supports expanded long-context support up to 256K input tokens for complex reasoning tasks.
Why This Matters
Training physical AI systems like humanoid robots and autonomous vehicles requires massive amounts of diverse, representative training data. Collecting real-world datasets is expensive, time-intensive, and often limited in scope. NVIDIA's Cosmos WFMs address this by enabling scalable generation of high-fidelity, physics-grounded synthetic data that can be used for training and testing. These models integrate with NVIDIA Omniverse to create 3D scenes that serve as ground truth inputs for Cosmos Transfer.
Developer Resources
NVIDIA provides the NVIDIA Cosmos Cookbook with step-by-step workflows, technical recipes, and concrete examples for building, adapting, and deploying Cosmos WFMs. All models are available on GitHub for developer access.