From Words to Worlds: Spatial Intelligence is AI’s Next Frontier

Key ideas

Li argues language models lack grounding in the physical world, defines world models by three capabilities, and sets out why building them is harder and where they apply.

1. 1

   Name what the leading technology cannot do

   Li credits large language models with beginning to change how people work with abstract knowledge, then lists their limits. She writes that top multimodal models rarely beat chance on estimating distance, orientation and size, and that generated video often loses coherence after seconds. Her conclusion is that AI without spatial understanding stays disconnected from the physical reality it tries to understand.

2. 2

   Ground the thesis in how cognition works

   Li calls spatial intelligence the scaffolding of human cognition. She draws on everyday acts such as parking a car, on firefighters communicating through gesture, and on historical cases including Eratosthenes measuring the Earth and Watson and Crick building 3D models. She says many scientists have conjectured that perception and action became the core loop driving the evolution of intelligence.

3. 3

   Define the category through capabilities

   Because the field is nascent, Li writes, what matters most is establishing guiding principles. She defines world models by three capabilities: generative, producing geometrically and physically consistent worlds; multimodal, taking inputs from images to gestures; and interactive, returning the next state after an action.

4. 4

   Spell out the open research problems

   Li names what remains unsolved at World Labs: a universal task function as simple as next-token prediction, ways to extract spatial information from abundant 2D images and video, and architectures beyond 1D or 2D tokenization. She calls these barriers formidable and says World Labs' research teams are devoted to them.

5. 5

   Sequence applications and keep people central

   Li sorts applications by timeline: creative tools now, robotics as an ambitious mid-term goal, and science, healthcare and education over a longer horizon. She rejects both techno-utopia and apocalypse in favor of what she calls a pragmatic view. Her stated motivation is that AI must augment human capability rather than replace it.

In Fei-Fei’s words

What’s available here

• Original source

  Linked. Opens on the publisher’s site.

• Player on this page

  Not applicable. Written piece; nothing to play.

• Full text or transcript

  Not reviewed. No license to show a text version here.

• Our summary

  Draft. Written by us with AI from the full public text and checked against it; quotes are word for word. Not yet rights-reviewed.

• Ask questions with AI

  Not reviewed. Off until there is licensed text to work from.