Atlas and the question of world models

On September 1, World Labs, founded by Fei-Fei Li, released a new model called Atlas.

The company calls it a world model for spatial intelligence. It can handle text, images, video and 3D information together, to generate, reconstruct and simulate spatial worlds.

After seeing the news, I also went and tried Marble, which World Labs has already made public. Strictly speaking, what I tried wasn’t the just-released Atlas; Atlas is still in early access and will come to a new version of Marble later.

Marble can generate a 3D world you can walk into and explore from a single photo. It really is quite interesting the first time you see it, but once you actually walk around inside, the flaws are obvious too: places the original photo didn’t capture have to be filled in by the model, some objects start warping when you change angle, and the spatial relationships don’t always match the real environment.

So the experience didn’t make me feel “the world is about to change.” It was more that it helped me concretely understand a concept that had been somewhat abstract before:

what a world model is trying to solve, and why researchers like Fei-Fei Li and Yann LeCun have long been focused on this direction.

LLMs already know a lot about the world, so why isn’t that enough?

Imagine a glass sitting at the edge of a table.

Today’s LLMs of course know that glass may break if it falls to the floor, that a cup too close to the edge is easily knocked off, and that to pick up a cup you can grab the body or the handle.

That’s all knowledge about the world. But if a robot were really standing in front of this table, it would also need to handle more specific questions: where exactly is the cup right now, how far is it from the edge, would the robot arm hit something nearby reaching over from this angle; if it pushes the cup, which way will it move; from the other side of the table, where should the same cup appear?

What needs understanding here is no longer just “what cups are usually like,” but what state this world in front of it is in right now, and how it might change next.

A world model deals roughly with questions like these.

You can think of a world model as an internal simulation of the world

There isn’t yet a definition of world model that all researchers fully agree on, but the idea behind it is easy to understand.

Before doing many things, people roughly predict the consequences first. Seeing a cup about to fall off a table, we don’t need to wait for it to shatter to know we’d better reach out and steady it; crossing the street and seeing a car approaching, we naturally judge its direction, speed and distance, then decide whether to keep walking.

If AI is to act in the physical world in the future, it needs similar abilities: understanding the current state, predicting what might happen next, and judging how its own actions will change the world.

So at the heart of a world model is one question:

if I do this, what will happen?

That’s also why it’s often described as a kind of internal simulator. Before acting, AI can first predict a few possible futures, then choose a more suitable path.

What exactly does Atlas do?

World Labs describes world-model capabilities with three very clear words: Generate, Reconstruct, Simulate.

Generate

means continuing from inputs like text and images to generate a world, including space not seen in the original input.

Reconstruct

means using one or more real photos to recreate a real-world 3D scene. The more input information, the less the model has to guess on its own.

Simulate

adds time and change. Atlas doesn’t only handle a static scene; it also tries to understand how the world changes over time, and how to turn real environments into simulations usable for training and testing robots.

Technically, Atlas organizes different kinds of information — text, images, camera pose, 3D depth — into one spatial context. Each image isn’t just some pixels; it also corresponds to a specific 3D viewing position and angle. Based on the spatial information it has already seen, the model goes on to predict new viewpoints, images and 3D structure.

For me, understanding this far is enough. How a multimodal autoregressive diffusion transformer is actually implemented can wait until I really need to study model architectures.

How is it different from a video model like Sora?

The two concepts really are easy to mix up, because a good video generation model must itself learn a great deal about how the world works.

If you ask a model to generate a video of a ball rolling off a table, it at least needs to make the motion look like common sense. As video models get stronger and stronger, the line between them and world models will get blurrier and blurrier.

But the two still focus on somewhat different problems.

Video generation first needs to solve: what should the next frames look like?

A world model cares more about whether the world itself stays consistent. Is the table still there when you look from another angle? Does an object still exist once it’s hidden from view? Looking back from the other side of the room, have the spatial relationships between objects changed in odd ways? If an agent takes an action, how will the state of the whole world change next?

So generating a visually plausible video and maintaining a world you can keep observing, predicting and acting in aren’t entirely the same problem.

When I tried Marble myself, this was the gap that was easiest to see. One photo can quickly grow into a beautiful 3D scene, but as soon as you start exploring from different directions, you find there’s still a good distance between “looks plausible” and “really understands this space.”

Yann LeCun has also been betting on world models for years

While trying to understand Atlas, the other person I thought of was Yann LeCun.

For years he has argued that if AI is ultimately to understand the physical world, plan and act, relying on language alone isn’t enough. Meta’s later V-JEPA 2 is called a world model outright: it learns motion and change in the world from large amounts of video, then adds a small amount of robot action data, so the model can predict how the world might change after a given action.

So Fei-Fei Li and LeCun really are focused on a very similar broad direction:

AI needs to form some kind of internal model of how the physical world works.

The two sides’ technical paths differ.

LeCun’s JEPA strongly emphasizes that you don’t need to predict every pixel of reality. If I want to judge whether a car is coming toward me, I don’t need to also predict where every leaf will be in the next second, the creases in passers-by’s clothes or the shape of the clouds. What really affects action may only be: the car is approaching, roughly how fast, and how much distance is left.

So JEPA mainly makes predictions in an abstract representation.

World Labs’ Atlas leans more generative. It directly generates images, video, depth and 3D worlds, and uses these different forms together to build spatial context.

I don’t think there’s any need right now to judge which path is “more correct.” Both are trying to answer a question that’s still very open:

what kind of internal representation would let AI really understand a world that changes and can be acted in?

Why do conversations about world models so often end up talking about robots?

Back to the cup on the table.

Suppose the task is for a robot to put the cup in the sink. It first has to see its surroundings, know where the cup, the table and itself are, then consider which direction to approach from, how to grasp the cup, and whether it will bump into other things while moving.

A world model is closer to helping it predict: what will happen if it does this.

There’s also a concept that’s come up a lot lately, called VLA (Vision-Language-Action). It connects visual input, language goals and robot actions, and is closer to solving: to achieve this goal, what action should I take now?

Real systems won’t always split so neatly into two parts, of course, and today’s VLAs don’t necessarily all need a separate world model. But distinguishing them this way helps a lot in understanding the map of the technology:

a world model helps predict the world; a policy / VLA helps choose actions; the robot actually carries the actions out, then keeps adjusting based on new observations.

So AI’s work in the physical world gradually forms a loop: observe the environment, predict the result, take action, then observe the new result.

That’s also why world models, spatial intelligence, embodied AI, VLA and robotics so often appear together lately. They aren’t the same concept; they’re gradually piecing together different parts of “how AI acts in the real world.”

So how significant is this Atlas release, really?

I think it’s still hard to draw any especially big conclusion right now.

Atlas has only just been released and is currently open only to a small number of partners. What ordinary users can try directly is still Marble, and my impression from using it is very clear: it can already do things that were hard to imagine before, but there’s still an obvious distance to go before it accurately and reliably understands and simulates real space.

So if you ask whether it will change most people’s daily lives today, I think not for now.

But from the point of view of how the technology is developing, this path is worth keeping an eye on.

Over the past few years, generative AI’s most obvious progress happened first in language, then gradually extended to images, sound and video; agents then started moving AI from generating content to using tools and completing tasks. World models focus on a different set of questions: can AI build a space that persists, understand the relationships between objects and change over time, and predict what consequences its own actions will have?

If these abilities gradually mature, the range of applications will be broad. Games and film could more easily create complete explorable spaces, architecture and design could turn real environments into simulations, AR / VR could more easily get 3D worlds, and robots could first learn and be tested in large numbers of simulated environments before entering reality.

Some of these already show early applications; others are still far off.

Atlas right now is more like a node in this process of development. It hasn’t solved the world-model problem, but it has put generation, spatial reconstruction, temporal simulation and robotics simulation into a more unified model framework.

Having tried it myself, my understanding of world models became a bit more concrete

If you only look at the official demos, it’s easy to focus on whether the generated results are stunning enough.

When you try it yourself with a photo of a space you know well, you start watching for other things: how did the model fill in the places the original photo didn’t capture? From another angle, can the original objects stay consistent? Do the proportions of the space slowly drift? If you add a few more photos of the same place, does the result get closer to the real environment?

These questions all point to the same distinction:

generating a world that looks plausible and building a sufficiently reliable world model are still a long way apart.

That’s probably also why I think Atlas is worth watching but not worth getting overexcited about.

The direction it represents is easy to understand: if AI in the future doesn’t only handle text, images and web pages, but really enters our physical world, it needs to gradually learn space, time, objects, change, and the consequences of action.

This isn’t done well enough yet.

But where the world-model path is trying to go is becoming clearer and clearer.

Btw, the most common meaning of Atlas in English is “a book of maps.”

The word comes from Atlas in Greek mythology, a Titan condemned to hold up the sky. World Labs hasn’t publicly explained why the model got this name, so there’s no need to read an official meaning into it.

Still, a model for generating, reconstructing and simulating worlds being called Atlas is quite fitting.