NVIDIA said on Nov. 5 that it added four open-weight Cosmos 3 Nano checkpoints1 for interactive world simulation, covering multi-view driving scenes, simulator rendering and bimanual robot video prediction.

According to NVIDIA's Hugging Face post, the checkpoints are meant for robotics and autonomous-driving researchers who want to vary scenes and study how a policy responds. The company's accompanying performance figures are its own comparisons, and the page that carries them gives no metric values for one of them.

From offline generation to streaming rollouts

NVIDIA's Cosmos platform page2 describes Cosmos 3 as a family of world foundation models in three sizes: Super (64B parameters), Nano (16B) and Edge (4B). It says the platform can run closed-loop simulation to improve behavior "without real-world risk." That page does not mention the four new checkpoints.

The blog post says the new checkpoints cover two modes. Transfer-Auto is a bidirectional model for offline synthetic driving data. Sim-Auto is its causally distilled streaming variant, which generates short chunks of camera video and LiDAR from generated history and updated controls. The post says Sim-Transfer and Sim-Bimanual likewise generate observations as controls arrive.

The blog attributes the streaming behavior to a four-stage post-training recipe1: bidirectional adaptation, teacher forcing, consistency distillation and self-forcing. The final stage trains the model on its own generated history. NVIDIA's stated reason is that a slightly mispredicted object becomes context for later frames, so errors can compound over a long rollout.

All four checkpoints are post-trained examples of Cosmos 3 Nano with 16B trainable parameters. They are released under OpenMDW 1.1, which the post says permits commercial and non-commercial use under its terms.

What each checkpoint takes in and produces

According to the blog post:

  • Cosmos3-Nano-Transfer-Auto generates synchronized video across up to 11 camera views, including fisheyes, with optional joint RGB–LiDAR output. Inputs are per-camera World Scenario Map (WSM) controls, text prompts, optional initial RGB images and, for LiDAR, HD-map range controls.
  • Cosmos3-Nano-Sim-Auto streams synchronized camera and LiDAR observations in a few denoising steps per chunk. The post says closed-loop use requires a simulation runtime and a WSM renderer to update the controls from driving inputs.
  • Cosmos3-Nano-Sim-Transfer turns a simulator's spatial controls (depth, segmentation, blur) and a text prompt into RGB video. Physics and ground-truth state stay with the simulator.
  • Cosmos3-Nano-Sim-Bimanual predicts the next observation directly from a bimanual robot's actions, for YAM and AgiBot embodiments, driven by a policy or human-operated joysticks.

The post links an inference skill for each checkpoint and says the post-training recipes start from Cosmos3-Nano or compatible base weights and cover all four stages.

The Transfer-Auto skill file on GitHub4 says the released model is optimized for the reference camera rig. It warns of reduced quality as camera placement, viewing direction, field of view and lens projection diverge, and says a custom rig may require post-training. Its distributed example requires four suitable GPUs, with memory fit and speed depending on the request and hardware.

The hardware notes in the blog are specific to Sim-Bimanual: Linux, an NVIDIA GPU, Docker and NVIDIA Container Toolkit. The post says the published vLLM-Omni setup was tested on one GB200 and that this is not a minimum-memory specification. The post gives no equivalent requirements for the other three checkpoints.

NVIDIA's performance claims and what they leave out

The quantitative claims appear on NVIDIA's Cosmos Lab research page3, not in the blog post. The research page links a technical report. That report is not reviewed here, so the page is the only support for the figures below, and the page includes no independent test results.

In its section on simulating from actions, the page says the model has "higher video quality and reference-frame PSNR than DreamDojo 2B and 14B on AgiBot-World and AgiBot-GEAR." It gives no PSNR or video-quality values, no evaluation configuration and no model version. It does not name the checkpoint compared, though the section concerns camera and robot actions, the task the blog assigns to Sim-Bimanual.

In its section on spatial controls, the page claims "stronger spatial control across all four evaluated modalities, including 59.2% higher edge F1 and 26.5% lower depth error than the Cosmos 3 GA reference." The two percentages are for edge and depth only. The page does not say what the "Cosmos 3 GA reference" is, does not give absolute scores, and does not name the checkpoint tested. It also does not say which modalities make up the four. The sample size and evaluation data are not stated either.

Without those details, a reader cannot tell whether the gains reflect the post-training recipe, a weaker reference configuration or the choice of test data. The DreamDojo comparison is limited to two benchmarks, AgiBot-World and AgiBot-GEAR. The page does not say whether either dataset was used in training, so contamination cannot be assessed from it.

The blog post itself is more cautious than the research page. It says "these problems remain hard" and that the work ahead is to measure where the models faithfully support policy learning and evaluation, understand their limits, and improve where they fall short. It also advises evaluating quality on held-out episodes, noting that a successful short training run verifies execution, not convergence.