This week, we’ll consider: What are some of the various claims about how AI poses an existential risk? What are the different ways of getting involved to reduce these risks?

For context on the field’s current perspectives on these questions, a 2020 survey of AI safety and governance researchers (Clarke et al., 2021) found that, on average, researchers currently guess there is:

• A 10% chance of existential catastrophe from misaligned, influence-seeking AI
• A 6% chance of existential catastrophe from AI-exacerbated war or AI misuse
• A 7% chance of existential catastrophe from “other scenarios”

Note that there were high levels of uncertainty and disagreement in the above survey’s results. These imply that many researchers must be wrong about important questions, which arguably makes skeptical and questioning mindsets.

Required Materials (~60 mins.)

Readings on AI safety:

• Risks from Advanced AI Video (ONLY FIRST 23 MINUTES) OR Transcript (35 mins.)
  • Alternatively, you could read The case for taking AI seriously as a threat to humanity - Vox (20 mins.)
• Choose one of the following:
  • Introducing Pathways: A next-generation AI architecture (5 mins.)
  • Yann LeCun on a vision to make AI systems learn and reason like animals and humans (10 mins.)
• AI Instrumental Goals - Section VI (10 mins.)
• Why AI alignment could be hard with modern deep learning (20 mins.)

Readings on Bayes’ rule and evidence:

• Bayes’ rule: Guide

More to explore