Ilya Sutskever (OpenAI Chief Scientist) — Why next-token prediction could surpass human intelligence

Key ideas

Sutskever argues next-token prediction might surpass human performance, names reliability as his pick if AI's economic value disappoints, and puts understanding results ahead of new ideas.

From a machine transcript of the audio. Quotes link to the moment they are said.

1. 1

   Reliability may limit economic value

   Sutskever says he really does not think disappointing real-world impact is likely, but if he had to pick one cause it would be reliability, with users still checking every answer. He grounds his thinking in self-driving, saying his Tesla looks as if it does everything while still having a long way to go on reliability.

2. 2

   Compete on usefulness, not only cost

   Asked whether inference will become cost-prohibitive, Sutskever reframes it: better models will cost more, and the question is how useful they are. Someone paying a lawyer $500 an hour, he says, would pay $400 for reliable advice from a neural net. On commoditization he accepts the force is real and says the answer is to keep improving models, while adding that serving costs can differ and some companies may specialize.

3. 3

   OpenAI left robotics for lack of data

   Sutskever defends OpenAI's decision to leave robotics, saying progress came from combining compute and data and robotics then offered no path to data. He says a path can be created now, but only by those committed to building many thousands of robots and solving physical and logistical problems that are not like software.

4. 4

   Make understanding the main activity

   Sutskever says coming up with whole new ideas is a modest part of the work, and that much of the time goes to understanding results and what could have caused an unexpected one. Asked about ImageNet, he calls it a new understanding of very old things. He credits his own record to trying really hard, which he adds is necessary but not sufficient.

5. 5

   Prediction can go beyond imitation

   Sutskever challenges the claim that next-token prediction cannot surpass people. He argues that predicting tokens well means understanding the reality that produced them, including people's thoughts and feelings. So a smart enough base network might, to a pretty decent degree, extrapolate how a hypothetical person of great insight would behave.

In Ilya’s words

What’s available here

• Original source

  Linked. Opens on YouTube.

• Player on this page

  Available. YouTube’s own player, loaded only when you press play.

• Full text or transcript

  Not reviewed. No license to show a text version here.

• Our summary

  Draft. Written by us with AI from a machine transcript of the video’s audio (Whisper) and checked against it. Machine transcripts can mishear; each quote links to the moment so you can hear it. Not yet rights-reviewed.

• Ask questions with AI

  Not reviewed. Off until there is licensed text to work from.