Risks from power-seeking AI systems (article narration by Zershaaneh Qureshi)

Hundreds of prominent AI scientists and other notable figures signed a statement in 2023 saying that mitigating the risk of extinction from AI should be a global priority. At 80,000 Hours, we’ve considered risks from AI to be the world’s most pressing problem since 2016.

But what led us to this conclusion? Could AI really cause human extinction? We’re not certain, but we think the risk is worth taking very seriously.

In particular, as companies create increasingly powerful AI systems, there’s a concerning chance that:

These AI systems may develop dangerous long-term goals we don’t want.
To pursue these goals, they may seek power and undermine the safeguards meant to contain them.
They may even aim to disempower humanity and potentially cause our extinction.

This article is written by Cody Fenwick and Zershaaneh Qureshi, and narrated by Zershaaneh Qureshi. It discusses why future AI systems could disempower humanity, what current AI research reveals about behaviours like power-seeking and deception, and how you can help mitigate the dangers.

You can see the original article — packed with graphs, images, footnotes, and further resources — on the 80,000 Hours website:

https://80000hours.org/problem-profiles/risks-from-power-seeking-ai/

Chapters:

Risks from power-seeking AI systems (00:01:00)
Introduction (00:01:17)
Summary (00:03:09)
Why are the risks from power-seeking AI a pressing world problem? (00:04:04)
Section 1: Humans will likely build advanced AI systems with long-term goals (00:05:43)
Section 2: AIs with long-term goals may be inclined to seek power (00:11:32)
Section 3: These power-seeking AI systems could successfully disempower humanity (00:26:26)
Section 4. People might create power-seeking AI systems without enough safeguards, despite the risks (00:38:34)
Section 5: Work on this problem is neglected and tractable (00:47:37)
Section 6: What are the arguments against working on this problem? (00:59:20)
Section 7: How you can help (01:25:07)
Thank you for listening (01:28:56)

Audio editing: Dominic Armstrong
Production: Zershaaneh Qureshi, Elizabeth Cox, and Katy Moore

Denne episoden er hentet fra en åpen RSS-feed og er ikke publisert av Podme. Den kan derfor inneholde annonser.

Episoder(342)

How middle powers avoid losing everything in a post-AI world | Anton Leicht

In a post-AGI world, can a country without access to frontier AI even be considered sovereign anymore?Anton Leicht says once frontier AI becomes a core economic input, the countries that own it will p...

14 Jul 1h 33min

#246 – Sneha Revanur on how a small team of activists helped pass America's landmark AI safety laws

Six years ago, aged just 15, Sneha Revanur founded the AI advocacy nonprofit Encode AI — back when AI felt like a niche issue. Now the world’s caught up with her, and she’s ready to share everything s...

8 Jul 52min

We can guess what intergalactic war would look like. And strangely, it matters.

Intergalactic war is probably billions of years away — yet physics can already tell us how it ends. And strangely that conclusion is relevant to decisions people have to make today.In this video, Rob ...

18 Jun 15min

How AI could create the world’s biggest problems (article by Zershaaneh Qureshi)

Imagine you’re living 15,000 years ago. Your people are hunter-gatherers and you sleep under the stars. If someone told you humans would one day build cities with millions of people, fly through the a...

11 Jun 1h 29min

#245 – Rohin Shah on what it's really like to run AGI safety at Google DeepMind (and where I disagree with 'doomers')

Most people working on AI safety think without a massive effort AI systems will probably end up with goals catastrophically different from humanity’s. Today’s guest, Rohin Shah — head of AGI Safety an...

2 Jun 2h 48min

What makes for a dream job? | Benjamin Todd

What actually makes a job fulfilling? It's not what most career advice tells you. "Follow your passion" sounds inspiring, but it's misleading — and the research backs that up.Drawing on hundreds of st...

28 Mai 28min

#244 – Benjamin Todd on how we’re updating our career advice for the strangest time in history

The average career is 80,000 hours long. With AI advancing so rapidly, the hours you have left in your career matter more than ever.Some leading AI researchers think there’s a 10% chance that AI syste...

26 Mai 1h 6min

Can AIs already start 'rogue deployments' inside AI companies? (Landmark new METR report)

A red-teamer was embedded inside Anthropic for three weeks, told to imagine he was an evil Claude, and asked to figure out how to launch a ‘rogue AI deployment’ without getting caught. It’s one part o...

20 Mai 20min