Skip to content

World Models, Robotics, and the Future of 3D AI

a16z Podcast

Hosted by Erik Torenberg

23m
Transcript

Original podcast

Summary

Written by a model from the transcript; not yet reviewed.

This episode of the A16Z Podcast explores the release of Atlas, the first multimodal world model from World Labs, featuring co-founder Justin Johnson in conversation with the hosts. They examine how world models combine visual and physical understanding to generate, reconstruct, and simulate environments, distinguishing them from traditional video generation models. The discussion covers applications across gaming, VFX, and robotics, specifically how sparse photo inputs can create simulations for training robots. The conversation also addresses the technical evolution from Gaussian splats to native frame generation, providing listeners with a clear look at spatial AI infrastructure.

  • Atlas uses grounded 3D reference images and precise camera control to prevent long video generations from degrading or hallucinating. 4:19
  • World models can reconstruct real-world spaces from just a few casual photos to simulate robot behavior in specific environments. 12:39
  • Gaussian splats and explicit 3D representations remain useful for client-side rendering on mobile and VR devices. 8:47

Chapters

  1. 0:00
    Introduction to World Models

    The hosts introduce the concept of world models as visual and physical AI engines distinct from language models.

  2. 0:35
    Atlas Launch Announcement

    Justin Johnson announces Atlas, the first multimodal world model that can generate, reconstruct, and simulate environments.

  3. 1:37
    Core Capabilities of Atlas

    Atlas’s three main functions—generation, reconstruction, and simulation—are explained with examples across VFX, gaming, and robotics.

  4. 3:46
    Robotics Applications and Real‑to‑Sim

    Atlas enables rapid real‑to‑simulation workflows for robotics, allowing fine‑tuning of foundation models in specific environments.

  5. 4:20
    Spatial Control and Directorial Power

    Atlas uses spatially grounded references and precise camera control to maintain coherence over long video generations.

  6. 7:04
    Comparison to Previous Work and Gaussians

    The discussion contrasts Atlas with World Labs’ earlier product Marble and explores the role of Gaussian splats in 3D representations.

  7. 15:57
    Future of AI‑Generated Games and Worlds

    The panel speculates on AI‑driven game creation, VR experiences, and the integration of world models into creative pipelines.

  8. 18:08
    Physics, Edge Cases, and Limitations

    Limitations in simulating complex physics like black holes are discussed, along with potential future improvements.

  9. 22:27
    Closing Remarks and Next Steps

    The hosts wrap up, thank Justin, and provide podcast outro and call‑to‑action instructions.

Highlights

  • 1:45

    Three Core Capabilities of Atlasinsight

    The guest explains that Atlas operates as a foundational world model focusing on three core pillars: generation, reconstruction, and simulation. This allows users to create novel worlds, model existing spaces, and test how robots behave within them.

  • 4:19

    Preventing Hallucination with Spatial Controlstrategy

    The guest explains how Atlas avoids the common video generation problem where outputs get jumbled after several seconds. By grounding reference images in 3D space and using precise camera steering, the model maintains directorial control over long sequences.

  • 10:53

    Real-to-Sim for Robotics Trainingstrategy

    The guest explains how simulation serves as the crucial last mile in robotics training and evaluation, ensuring foundation models work correctly in exact factory or workspace conditions.

  • 11:47

    Breaking Free from Gaussian Bottlenecksinsight

    The guest details how Atlas differs from their previous product, Marble, by eliminating the dependency on Gaussian splatting as a central bottleneck. The new architecture handles 2D and 3D modalities more natively and scales more easily.

  • 18:41

    Bullet Time Shots from Three iPhonesstory

    The guest shares a surprising and successful application of Atlas where three synchronized iPhones capture an event, such as a strawberry dropping into oat milk, enabling complex bullet time reframing and freeze-frame flyarounds.

  • 19:49

    The Underappreciated Value of 3D Inputinsight

    The guest argues that beyond final 3D outputs, 3D control surfaces provide immense value on the input side by letting human directors precisely dictate camera movements instead of relying solely on text prompts.

Quotes

  • “I can take just a few photos of a space, as few as one, up to 100 or more. And the more photos you give it, the more you can reconstruct that existing real world space accurately.”
    Unnamed speaker · 2:29
  • “When you generate a video from Atlas, you're not just giving it a text prompt. You also go in and steer the camera and say, I want the camera to fly this way, this way, this way, this way, with exact precise pixel control.”
    Unnamed speaker · 4:37
  • “One is for evaluation, right? Like, you definitely want to evaluate robotics in the final situation, in the final environment where they're actually going to end up.”
    Unnamed speaker · 10:57
  • “If you want something that's sort of non‑Newtonian, right? Like, black hole physics, or like, you know, microscopic stuff, and once you're down in the quantum mechanics realm, I think then you'd have to rethink a bit the architecture and the data side.”
    Unnamed speaker · 20:59
  • “We never wanted to be building these generative slot machines, where you just, like, pull the thing and hope you get out a good generation, or you don't feel like that generation was yours, or that you controlled it, or that you knew what you were getting out.”
    Unnamed speaker · 17:43
  • “So we can do this thing, or you can take, like, a couple regular iPhones and stick them on tripods, just like, three iPhones, take synchronized capture of some event happening, and then you can use Atlas to do these bullet time reframing shots, like, freeze frame fly around.”
    Unnamed speaker · 18:48

About this episode

World Labs co-founder Justin Johnson joins MTS hosts Theo Jaffee and Sophia Puccini to discuss Atlas, World Labs’ latest world model, and the broader case for AI systems that understand and interact with the physical world. Justin explains how Atlas approaches three core tasks: generating new worlds, reconstructing real environments from images, and simulating how objects or robots might behave within them. Underlying it is a bigger thesis: just as language models became general-purpose engines for working with text, world models could become a horizontal layer for visual and physical intelligence across industries from entertainment and gaming to construction and robotics. They also explore how world models could change video games and creative tools, why precise spatial control matters, and the potential for “real-to-sim-to-real” robotics, where a few photos of a physical environment could eventually be enough to build a simulation and adapt a robot to that specific space.

Transcript

0:00
23:30

5,023 words · 4 speakers

Speaker names marked are inferred from speaking time and the show’s known hosts — they are not voice-verified.

Topics

  • Atlas
  • World Models
  • Marble
  • VR
  • World Labs
  • VFX
  • Sora
  • OpenAI
  • Alice
  • LLM

Page updated · transcript and summary are machine-generated