“Mitigating Risks from Rouge AI” by poppinfresh

“Mitigating Risks from Rouge AI” by poppinfresh

Introduction

Misaligned AI systems, which have a tendency to use their capabilities in ways that conflict with the intentions of both developers and users, could cause significant societal harm. Identifying them is seen as increasingly important to inform development and deployment decisions and design mitigation measures. There are concerns, however, that this will prove challenging. For example, misaligned AIs may only reveal harmful behaviors in rare circumstances, or perceive detection attempts as threatening and deploy countermeasures – including deception and sandbagging – to evade them.

For these reasons, developing a range of efforts to detect misaligned behavior, including power-seeking, deception, and sandbagging, among other capabilities, have been proposed. One important indicator, though, has been hiding in plain sight for years. In this post, we identify an underappreciated method that may be both necessary and sufficient to identify misaligned AIs: whether or not they've turned red, i.e. gone rouge.

In [...]

---

Outline:

(00:10) Introduction

(01:42) Historical Evidence for Rouge AI

(02:36) Recent Empirical Work

(03:46) Potential Countermeasure

(03:49) The EYES Eval

(04:55) EYES Eval Demonstration

(05:44) Future Research Directions

(06:46) Conclusion

---

First published:
April 1st, 2025

Source:
https://forum.effectivealtruism.org/posts/uKKoj9iqj2cWKsjrt/mitigating-risks-from-rouge-ai

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Det här avsnittet är hämtat från ett öppet RSS-flöde och publiceras inte av Podme. Det kan innehålla reklam.

Avsnitt(250)

“Moral Imagination & Effective Altruism” by Toby_Ord

“Moral Imagination & Effective Altruism” by Toby_Ord

Michael Nielsen has a beautiful new essay on moral imagination: the ability humans have to 'develop and transmit new notions of good action, indeed, even new kinds of good'. As examples, he gives: Ha...

28 Aug 11min

“What if the third wave is a puddle?” by Morgan Fairless

“What if the third wave is a puddle?” by Morgan Fairless

The world of private philanthropy may be in for a period of large growth. News of Coefficient Giving increasing their funding for GiveWell to one billion in the near-term, and the general growth of th...

18 Aug 9min

“What would make us scale or stop? NOVAH’s pre-commitment before seeing the RCT results” by I.J.J., AlexisAt

“What would make us scale or stop? NOVAH’s pre-commitment before seeing the RCT results” by I.J.J., AlexisAt

TL;DR NOVAH (No Violence At Home) was incubated by Charity Entrepreneurship (now Ambitious Impact) in 2024 to test a promising idea: preventing intimate partner violence through edutainment, in our ca...

17 Aug 15min

“General capability - and capabilities generally - have no good y-axis” by Gregory Lewis🔸

“General capability - and capabilities generally - have no good y-axis” by Gregory Lewis🔸

BLUF: To determine whether AI is ‘improving exponentially’, ‘hitting the wall’, or any other claim which involves a quantity or magnitude (e.g. ‘This model was a big leap/small increment’). We need a...

8 Aug 1h 7min

“Never born, then maybe died twice” by NickLaing

“Never born, then maybe died twice” by NickLaing

“Your balls are baked aren’t they, mate”. He would not have been bornThanks Lyndon, that describes the situation. Like almost 1 in 10 modern men, my sperm weren’t up to the task - at least not the old...

1 Aug 10min

“Announcing: Cluelessness Critiques Competition” by Toby Tremlett🔹, Will Aldred

“Announcing: Cluelessness Critiques Competition” by Toby Tremlett🔹, Will Aldred

TL;DR We’re hosting an essay competition to elicit responses to Anthony DiGiovanni's sequence, ‘The challenge of unawareness for impartial altruist action guidance’—submit your entries by August 14th...

30 Juli 13min

“Spiro: an update 2.5 years on and a fundraising ask for expansion” by Habiba Banu

“Spiro: an update 2.5 years on and a fundraising ask for expansion” by Habiba Banu

Summary Back in November 2023 I posted here to launch Spiro and raise our first 198 thousand dollars. Two and a half years later this is an update and a fundraiser for the next step. The short version...

26 Juli 17min

“Let’s taboo the V-word” by lincolnq

“Let’s taboo the V-word” by lincolnq

“How long have you been v*g*n?” This is one of the most common icebreakers at animal protection events. It's a baseline assumption, and it mostly holds true: if you’re out advocating for animals not t...

14 Juli 12min

Populärt inom Samhälle & Kultur

podme-dokumentar
gynning-berg
p3-dokumentar
aftonbladet-krim
en-mork-historia
svenska-fall
creepypodden-med-jack-werner
hor-har
killradet
rss-schyffert-sundin-ar-det-har-nat
aftonbladet-daily
p1-dokumentar
flashback-forever
kod-katastrof
rss-mer-an-bara-morsa
vad-blir-det-for-mord
rss-vill-du-veta-en-hemlis-2
rss-nemo-moter-en-van
rss-brottsutredarna
p3-dystopia