Aligning AI with human intent: A discussion with Mira Murati, OpenAI | Cutting Edge

Key ideas

Murati describes how OpenAI chooses what to build and aligns models to human intent, and why fixing bias or safety problems takes repeated auditing after each intervention.

From a machine transcript of the audio. Quotes link to the moment they are said.

1. 1

   How OpenAI chose language and code

   Murati says OpenAI started with language because so much data exists, because some consider mastering it a mark of intelligence, and because instructions in natural language make alignment easier. Code came partly because many people at OpenAI program, which gives quick feedback on whether a tool is useful.

2. 2

   Aligned GPT-3 proved more useful

   Murati says GPT-3 was first trained only to predict the next word, then aligned to human intent using reinforcement learning from human feedback. OpenAI found the aligned version safer and more reliable, and businesses preferred it, which she describes as safety research gaining a practical application.

3. 3

   Every fix needs an audit afterward

   When OpenAI removed violent and sexual images from DALL-E 2's training data, Murati says an audit showed bias had grown: a prompt for a CEO went from mostly white men to only white men. Other techniques were then needed to restore balance, and she calls the problem hard both sociologically and technically.

4. 4

   Open use cases one at a time

   Murati says OpenAI goes use case by use case and so far has not applied its models to military uses, because it does not feel able to mitigate all the risks. She treats military or medical settings, and maybe legal ones, without a human in control as high stakes, and says the models are not there yet.

5. 5

   Measure AGI by the generalization gap

   Murati suggests judging distance from AGI by what humans do well and machines do not, especially applying existing heuristics in new settings with little instruction. She notes that a system trained on one data set may not work as expected when deployed somewhere completely different.

In Mira’s words

What’s available here

• Original source

  Linked. Opens on YouTube.

• Player on this page

  Available. YouTube’s own player, loaded only when you press play.

• Full text or transcript

  Not reviewed. No license to show a text version here.

• Our summary

  Draft. Written by us with AI from a machine transcript of the video’s audio (Whisper) and checked against it. Machine transcripts can mishear; each quote links to the moment so you can hear it. Not yet rights-reviewed.

• Ask questions with AI

  Not reviewed. Off until there is licensed text to work from.