80,000 Hours Podcast16 Huhti 2025

#215 – Tom Davidson on how AI-enabled coups could allow a tiny group to seize power

Throughout history, technological revolutions have fundamentally shifted the balance of power in society. The Industrial Revolution created conditions where democracies could flourish for the first time — as nations needed educated, informed, and empowered citizens to deploy advanced technologies and remain competitive.

Unfortunately there’s every reason to think artificial general intelligence (AGI) will reverse that trend.

Today’s guest — Tom Davidson of the Forethought Centre for AI Strategy — claims in a new paper published today that advanced AI enables power grabs by small groups, by removing the need for widespread human participation.

Links to learn more, video, highlights, and full transcript. https://80k.info/td

Also: come work with us on the 80,000 Hours podcast team! https://80k.info/work

There are a few routes by which small groups might seize power:

Military coups: Though rare in established democracies due to citizen/soldier resistance, future AI-controlled militaries may lack such constraints.
Self-built hard power: History suggests maybe only 10,000 obedient military drones could seize power.
Autocratisation: Leaders using millions of loyal AI workers, while denying others access, could remove democratic checks and balances.

Tom explains several reasons why AI systems might follow a tyrant’s orders:

They might be programmed to obey the top of the chain of command, with no checks on that power.
Systems could contain "secret loyalties" inserted during development.
Superior cyber capabilities could allow small groups to control AI-operated military infrastructure.

Host Rob Wiblin and Tom discuss all this plus potential countermeasures.

Chapters:
• Cold open (00:00:00)
• How AI enables tiny groups to seize power (00:00:50)
• The 3 different threats (00:02:14)
• Is this common sense or far-fetched? (00:03:24)
• “No person rules alone.” Except now they might. (00:06:27)
• Underpinning all 3 threats: Secret AI loyalties (00:12:31)
• Are secret AI loyalties possible right now? (00:16:59)
• Key risk factors (00:20:30)
• Preventing secret loyalties in a nutshell (00:22:07)
• Are human power grabs more plausible than 'rogue AI'? (00:24:32)
• If you took over the US, could you take over the whole world? (00:33:22)
• Will this make it impossible to escape autocracy? (00:37:31)
• Threat 1: AI-enabled military coups (00:41:34)
• Will we sleepwalk into an AI military coup? (00:51:47)
• Could AIs be more coup-resistant than humans? (00:57:53)
• Threat 2: Autocratisation (01:00:48)
• Will AGI be super-persuasive? (01:11:06)
• Threat 3: Self-built hard power (01:13:31)
• Can you stage a coup with 10,000 drones? (01:21:23)
• That sounds a lot like sci-fi... is it credible? (01:23:33)
• Will we foresee and prevent all this? (01:27:54)
• Are people psychologically willing to do coups? (01:29:22)
• Will a balance of power between AIs prevent this? (01:33:31)
• Will whistleblowers or internal mistrust prevent coups? (01:35:48)
• Will rogue AI preempt a human power grab? (01:44:31)
• The best reasons not to worry (01:47:09)
• How likely is this in the US? (01:49:28)
• Is a small group seizing power really so bad? (01:56:58)
• Countermeasure 1: Block internal misuse (02:00:33)
• Countermeasure 2: Cybersecurity (02:10:27)
• Countermeasure 3: Model spec transparency (02:12:36)
• Countermeasure 4: Sharing AI access broadly (02:21:55)
• Is it more dangerous to concentrate or share AGI? (02:26:45)
• Is it important to have more than one powerful AI country? (02:29:31)
• In defence of open sourcing AI models (02:32:38)
• 2 ways to stop secret AI loyalties (02:40:19)
• Preventing AI-enabled military coups in particular (02:53:18)
• How listeners can help (02:59:06)
• How to help if you work at an AI company (03:03:00)
• The power ML researchers still have, for now (03:07:09)
• How to help if you're an elected leader (03:10:29)

This episode was originally recorded on January 20, 2025.

Video editing: Simon Monsour
Audio engineering: Ben Cordell, Milo McGuire, Simon Monsour, and Dominic Armstrong
Camera operator: Jeremy Chevillotte
Transcriptions and web: Katy Moore

Tämä jakso on lisätty Podme-palveluun avoimen RSS-syötteen kautta eikä se ole Podmen omaa tuotantoa. Siksi jakso saattaa sisältää mainontaa.

Jaksot(341)

How a small team of activists helped pass America's landmark AI safety laws | Sneha Revanur, Encode AI

Six years ago, aged just 15, Sneha Revanur founded the AI advocacy nonprofit Encode AI — back when AI felt like a niche issue. Now the world’s caught up with her, and she’s ready to share everything s...

8 Heinä 52min

We can guess what intergalactic war would look like. And strangely, it matters.

Intergalactic war is probably billions of years away — yet physics can already tell us how it ends. And strangely that conclusion is relevant to decisions people have to make today.In this video, Rob ...

18 Kesä 15min

How AI could create the world’s biggest problems (article by Zershaaneh Qureshi)

Imagine you’re living 15,000 years ago. Your people are hunter-gatherers and you sleep under the stars. If someone told you humans would one day build cities with millions of people, fly through the a...

11 Kesä 1h 29min

#245 – Rohin Shah on what it's really like to run AGI safety at Google DeepMind (and where I disagree with 'doomers')

Most people working on AI safety think without a massive effort AI systems will probably end up with goals catastrophically different from humanity’s. Today’s guest, Rohin Shah — head of AGI Safety an...

2 Kesä 2h 48min

What makes for a dream job? | Benjamin Todd

What actually makes a job fulfilling? It's not what most career advice tells you. "Follow your passion" sounds inspiring, but it's misleading — and the research backs that up.Drawing on hundreds of st...

28 Touko 28min

#244 – Benjamin Todd on how we’re updating our career advice for the strangest time in history

The average career is 80,000 hours long. With AI advancing so rapidly, the hours you have left in your career matter more than ever.Some leading AI researchers think there’s a 10% chance that AI syste...

26 Touko 1h 6min

Can AIs already start 'rogue deployments' inside AI companies? (Landmark new METR report)

A red-teamer was embedded inside Anthropic for three weeks, told to imagine he was an evil Claude, and asked to figure out how to launch a ‘rogue AI deployment’ without getting caught. It’s one part o...

20 Touko 20min

#243 – 'Godfather of AI' Yoshua Bengio: "I now see a path" to safe superintelligent AI

The co-inventor of modern AI and the most cited living scientist believes he's figured out how to ensure AI is honest, incapable of deception, and never goes rogue. Yoshua Bengio – Turing Award Winner...

7 Touko 2h 35min