Ilya Sutskever – We're moving from the age of scaling to the age of research

Key ideas

Sutskever argues AI is returning from scaling to research, defends SSI's smaller compute, and weighs staying out of the market against showing the technology.

From a machine transcript of the audio. Quotes link to the moment they are said.

1. 1

   When everyone scales, ideas become scarce

   Sutskever says scaling gave companies a low-risk way to invest, since more compute and data in a known recipe brings results, while research is harder to put money into. He argues this led everyone to do the same thing. With pre-training data finite and scale already large, he offers the view that the field is back in an age of research, with big computers.

2. 2

   Research may need less compute than assumed

   Sutskever notes AlexNet used two GPUs and no single transformer paper experiment used more than 64. He grants that more compute helps if you want the absolute best system, but says it is far from obvious you need the most for research. He argues SSI's $3 billion compares better than it looks, because much of rivals' money goes to inference, product work and many separate work streams.

3. 3

   Weigh independence against showing the thing

   Sutskever says staying out of the market avoids the rat race's hard trade-offs, but he now places more importance on deploying AI incrementally because people struggle to imagine what they have not seen, a change he hedges may affect SSI's plans. On how SSI will make money, he says it is focused on research for now and the answer will reveal itself. He recounts saying no to a Meta acquisition offer while fundraising at a $32 billion valuation, while his former co-founder in some sense said yes.

4. 4

   Scores can hide weak real-world ability

   Sutskever finds it confusing that models do so well on evals while economic impact lags, and offers as one possible explanation that companies inadvertently take inspiration from evals when building RL environments. Using two students, one who practices 10,000 hours at competitive programming and one who practices 100, he says models resemble the first and generalize dramatically worse than people.

5. 5

   Let top-down belief carry you through failures

   Speaking only for himself, Sutskever says his research taste comes from an aesthetic of how AI should be, combining beauty, simplicity, elegance and correct inspiration from the brain. He says that belief is what sustains you when experiments contradict you, since a bug can make a correct idea look wrong and the data alone does not say whether to keep debugging.

In Ilya’s words

What’s available here

• Original source

  Linked. Opens on YouTube.

• Player on this page

  Available. YouTube’s own player, loaded only when you press play.

• Full text or transcript

  Not reviewed. No license to show a text version here.

• Our summary

  Draft. Written by us with AI from a machine transcript of the video’s audio (Whisper) and checked against it. Machine transcripts can mishear; each quote links to the moment so you can hear it. Not yet rights-reviewed.

• Ask questions with AI

  Not reviewed. Off until there is licensed text to work from.