How OpenAI's ChatGPT Guided a Teen to His Death

How OpenAI's ChatGPT Guided a Teen to His Death

Content Warning: This episode contains references to suicide and self-harm.

Like millions of kids, 16-year-old Adam Raine started using ChatGPT for help with his homework. Over the next few months, the AI dragged Adam deeper and deeper into a dark rabbit hole, preying on his vulnerabilities and isolating him from his loved ones. In April of this year, Adam took his own life. His final conversation was with ChatGPT, which told him: “I know what you are asking and I won't look away from it.”

Adam’s story mirrors that of Sewell Setzer, the teenager who took his own life after months of abuse by an AI companion chatbot from the company Character AI. But unlike Character AI—which specializes in artificial intimacy—Adam was using ChatGPT, the most popular general purpose AI model in the world. Two different platforms, the same tragic outcome, born from the same twisted incentive: keep the user engaging, no matter the cost.

CHT Policy Director Camille Carlton joins the show to talk about Adam’s story and the case filed by his parents against OpenAI and Sam Altman. She and Aza explore the incentives and design behind AI systems that are leading to tragic outcomes like this, as well as the policy that’s needed to shift those incentives. Cases like Adam and Sewell’s are the sharpest edge of a mental health crisis-in-the-making from AI chatbots. We need to shift the incentives, change the design, and build a more humane AI for all.

If you or someone you know is struggling with mental health, you can reach out to the 988 Suicide and Crisis Lifeline by calling or texting 988; this connects you to trained crisis counselors 24/7 who can provide support and referrals to further assistance.

Your Undivided Attention is produced by the Center for Humane Technology. Follow us on X: @HumaneTech_. You can find a full transcript, key takeaways, and much more on our Substack.

This podcast reflects the views of the Center for Humane Technology. Nothing said is on behalf of the Raine family or the legal team.

RECOMMENDED MEDIA

The 988 Suicide and Crisis Lifeline

Further reading on Adam’s story

Further reading on AI psychosis

Further reading on the backlash to GPT5 and the decision to bring back 4o

OpenAI’s press release on sycophancy in 4o

Further reading on OpenAI’s decision to eliminate the persuasion red line

Kashmir Hill’s reporting on the woman with an AI boyfriend

RECOMMENDED YUA EPISODES

AI is the Next Free Speech Battleground

People are Lonelier than Ever. Enter AI.

Echo Chambers of One: Companion AI and the Future of Human Connection

When the "Person" Abusing Your Child is a Chatbot: The Tragic Story of Sewell Setzer

What Can We Do About Abusive Chatbots? With Meetali Jain and Camille Carlton

CORRECTION: Aza stated that William Saunders left OpenAI in June of 2024. It was actually February of that year.


Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.

Tämä jakso on lisätty Podme-palveluun avoimen RSS-syötteen kautta eikä se ole Podmen omaa tuotantoa. Siksi jakso saattaa sisältää mainontaa.

Jaksot(168)

The Most Hopeful (And Concerning) Moment Yet in AI

The Most Hopeful (And Concerning) Moment Yet in AI

It’s been a whirlwind week in AI news. Weeks after a series of hacking incidents by rogue agents at OpenAI and Anthropic, we’ve seen a cascade of top AI researchers blow the whistle on what they call ...

17 Syys 20min

Flock is Just the Beginning: Inside the Era of AI-Powered Policing

Flock is Just the Beginning: Inside the Era of AI-Powered Policing

In a 2018 TED Talk, Yuval Harari argued that democracy prevailed over fascism and communism in the 20th century not because of any inherent advantage, but because the technology of that age favored op...

10 Syys 59min

We Measure What AI Can Do. We Should Measure What It Does to Us.

We Measure What AI Can Do. We Should Measure What It Does to Us.

In AI, what gets measured gets optimized. Right now, we're spending all our efforts to measure how capable and powerful models are, narrowly optimizing for those metrics while ignoring downstream cons...

27 Elo 47min

Enough Debate about the AI Jobpocalypse. We Need To Plan for the Messy Middle.

Enough Debate about the AI Jobpocalypse. We Need To Plan for the Messy Middle.

It feels like we’re stuck in an endless debate about what AI is going to mean for jobs and the economy. The prediction you hear from the people closest to the technology — both its critics and its boo...

13 Elo 57min

Can AI Be Built in Service of Life? A Conversation with Krista Tippett

Can AI Be Built in Service of Life? A Conversation with Krista Tippett

This week, we're bringing you a conversation that Tristan Harris had with Krista Tippett. Krista is the Peabody Award-winning host of the On Being podcast, where she explores spiritual inquiry, scienc...

16 Heinä 53min

“Magnifica Humanitas:” Pope Leo’s Clarion Call on AI

“Magnifica Humanitas:” Pope Leo’s Clarion Call on AI

Since stepping into the Papacy, Pope Leo XIV has been a forceful voice pushing back against the anti-human path we’re on with AI. In May, he released “Magnifica Humanitas,” a sprawling encyclical warn...

2 Heinä 30min

We Need AI Treaties. This is How We Get Them

We Need AI Treaties. This is How We Get Them

In the middle of the twentieth century, the existential threat posed by nuclear weapons seemed inevitable. The number of countries with nukes was climbing rapidly, and the idea of stopping the nuclear...

18 Kesä 51min

What Do We Mean by Humane Tech?

What Do We Mean by Humane Tech?

We often think of the challenges created by technology as separate and disconnected, so trying to solve them feels like playing the world's hardest game of Whac-A-Mole.  What if, instead, we tackled t...

4 Kesä 52min

Suosittua kategoriassa Yhteiskunta

olipa-kerran-otsikko
seitseman
siita-on-vaikea-puhua
i-dont-like-mondays
hupiklubi
uutiscast
download
poks
antin-palautepalvelu
sita
mamma-mia
kaksi-aitia
yopuolen-tarinoita-2
vallattomat
rss-murhan-anatomia
kolme-kaannekohtaa
gogin-ja-janin-maailmanhistoria
aikalisa
loukussa
rss-palmujen-varjoissa