NVIDIA Cosmos World Foundation Models Update

NVIDIA has advanced its Cosmos platform with three major model releases aimed at addressing critical challenges in robotics and autonomous vehicle development. These models tackle the expensive and time-intensive process of collecting diverse, representative training datasets for physical AI systems.

Key Model Updates

Cosmos Transfer 2.5 generates photorealistic synthetic videos from structured inputs like segmentation maps, depth maps, and 3D bounding boxes. Using ControlNet architecture, it preserves spatial alignment while adding photorealism through diverse environment and lighting variations—enabling developers to scale synthetic data generation without collecting expensive real-world footage.

Cosmos Predict 2.5 enhances long-tail scenario generation for sequences up to 30 seconds, delivering up to 10x higher accuracy when post-trained on domain-specific data. It now supports multiview outputs, custom camera layouts, and alternate policy outputs such as action simulation—critical for capturing edge cases in autonomous systems.

Cosmos Reason 2 introduces advanced physical AI reasoning with improved spatiotemporal understanding and timestamp precision. New capabilities include object detection with 2D/3D point localization, bounding box coordinates, reasoning explanations, and expanded long-context support up to 256K input tokens—enabling complex reasoning tasks like motion prediction and context-aware decision-making.

Developer Integration

These models integrate with NVIDIA Omniverse (built on OpenUSD), allowing developers to create physics-accurate 3D simulations that serve as ground truth inputs for Cosmos Transfer. The models work downstream as foundations for post-training domain or task-specific physical AI models, accelerating development cycles for robotics and autonomous vehicles.

Developers can explore step-by-step workflows in the NVIDIA Cosmos Cookbook.