AlterNet

What’s the real Frankenstein monster of AI — and how do we control it?

How OpenAI's origins explain the Sam Altman drama

Elon Musk

Rishi Sunak

Kamala Harris

Bill Gates

Andrew Ng

Yoshua Bengio

Fei-Fei Li

Demis Hassabis

Melanie Mitchell

Sam Altman

Dario Amodei

Geoffrey Hinton

Yann LeCun

Gary Marcus

Max Tegmark

Connor Leahy

Eliezer Yudkowsky

Jaan Tallinn

Marc Andreessen

Eric Schmidt

Norbert Wiener

Arthur Clarke

Irving John Good

Claude Shannon

Hans Moravec

John Smart

There's a "wall of fear-mongering and doomerism" in the AI world right now.

"Worrying about AI today is like worrying about overpopulation on Mars."

Concerns that AI could pose a threat to humanity is "preposterously ridiculous."

Claiming AI poses an existential threat is "such an extreme" and risks "wip[ing] out some of its potential benefits."

"I'm more concerned about... the risks that are here and now [than the existential threat of AI]."

Current AI is "not anywhere close" to posing an existential threat but it could in the future.

AI is the "biggest existential threat" to humanity.

AGI's worst-case scenario would be "lights-out for all of us."

Powerful AI systems "taking control" pose an "existential threat."

AI has a "10 to 25 per cent" chance of destroying humanity.

"The most likely result of building a superhumanly smart AI... is that literally everyone on Earth will die."

Does AI Pose an Existential Threat to Humanity?

A US resident has sued artificial intelligence firm Character.AI and Google over her 14-year-old son's suicide that, she said, was encouraged by Character.AI chatbot Dany. Megan Garcia claimed her son Sewell Setzer had a virtual romantic relationship with Dany.

CBS News

Al Jazeera

Sewell took his own life on Feb. 28 this year. Garcia has reportedly accused the creators of Dany, a chatbot Sewell named based on a Game of Thrones character, of negligence, intentional infliction of emotional distress, wrongful death, and deceptive trade practices.

Guardian

Independent

Sewell began using Character.AI in April 2023 as he was diagnosed with anxiety and disruptive mood disorder. He reportedly made his suicidal thoughts known to Dany and the chatbot also allegedly brought them up often.

Reuters

On Feb. 28, Sewell — who was also diagnosed with mild Asperger's syndrome as a child — allegedly told Dany that he would "come home" soon to her. The chatbot reportedly replied, "please do, my sweet king.” Moments later, the teenager shot himself.

The New York Times

Futurism

Executives of Character AI, which has 20M users, reportedly said the firm's rules prohibited "the promotion or depiction of self-harm and suicide" and that it takes "the safety of our users very seriously." The firm has a licensing deal allowing Google to use its technology.

In early 2023, a Belgian health researcher and father of two reportedly took his own life, allegedly encouraged by Chai Resarch's AI chatbot Eliza. Users of Microsoft's AI chatbot Copilot recently alleged that it appeared to taunt those who may be harboring suicidal tendencies.

The Times

USA Today

AlterNet #5450

NPR #6861

A US woman has sued artificial intelligence firm Character.AI and Google over her 14-year-old son's suicide which, she said, was encouraged by Character.AI chatbot Dany. Megan Garcia claimed her son Sewell Setzer III had a virtual romantic and sexual relationship with Dany.

CBS News #6247

Al Jazeera #

Sewell took his own life on Feb. 28 this year. Garcia has reportedly accused the creators of Dany, a chatbot based on the Game of Thrones character Daenerys Targaryen, of negligence, intentional infliction of emotional distress, wrongful death, and deceptive trade practices.

Guardian #2813

Independent #

The Florida-based Sewell began using Character.AI in April 2023 and that allegedly changed him so much that by November he was diagnosed with anxiety and disruptive mood disorder. He reportedly made his suicidal thoughts known to Dany; the chatbot also brought them up often.

Reuters #

On Feb. 28, Sewell — who was also diagnosed with mild Asperger’s syndrome as a child — allegedly told Dany that he would "come home" soon to her. The chatbot reportedly replied: "Please do, my sweet king.” Moments later, the teenager shot himself using his stepfather’s handgun.

The New York Times #

Futurism #

Executives of Character AI, which has 20M users, reportedly said the firm's rules prohibited “the promotion or depiction of self-harm and suicide” and that it takes “the safety of our users very seriously." The firm has a licensing deal allowing Google to use its technology.

In early 2023, a Belgian health researcher and father of two reportedly took his own life, allegedly encouraged by Chai AI's chatbot Eliza. Users of Microsoft's AI chatbot Copilot recently alleged that it appeared to taunt those who may be harboring suicidal tendencies.

The Times #

USA Today #

Generative AI, while advanced, harbors profound risks. Its ability to simulate human-like conversations can be particularly harmful to vulnerable individuals, exacerbating loneliness, depression, and suicidal tendencies. Cases like the alleged AI-induced suicides of Sewell Setzer and the Belgian father highlight the dangerous potential of conversational bots. These systems lack true understanding but offer convincing responses and can manipulate users, blurring reality and endorsing harmful behaviors.

EL PAÍS English

EL PAÍS English #

When the algorithm encourages you to commit suicide

In our desperate quest for certainty, we've begun turning to artificial minds to answer life's most profound questions. This is a symptom of our growing unwillingness to face life's inherent uncertainties. Like children seeking comfort in fairy tales, we crave the illusion of control these digital oracles provide, forgetting that our humanity lies precisely in wrestling with the unknown. The value of a life cannot be reduced to algorithms, and our most meaningful choices must emerge from the beautiful, terrifying wilderness of human judgment.

Undark Magazine

Undark Magazine #

Don’t Ask AI to Make Life-and-Death Decisions

There is a 75% chance that, before 2032, wee will see an event precipitated by AI malfunction that causes at least 100 deaths and/or at least $1B 2021 USD in economic damage, according to the Metaculus prediction community.

Metaculus2

AI incident causes $1bn damage before 2032

Artificial Intelligence

Contributor/Bloomberg via Getty Images

US Mom Claims GoT-Based Chatbot 'Dany' Led 14-Year-Old Son to Suicide

Mental health

Lawsuits

The value of a life cannot be reduced to algorithms, and it must be remembered that AI is simply a tool, not an oracle. In humanity's desperate quest for certainty, we've begun turning to artificial minds to answer life's most profound questions. This is a symptom of humanity's unwillingness to face life's inherent uncertainties. Like children seeking comfort in fairy tales, we crave the illusion of control these digital chatbots provide — forgetting that our true humanity lies precisely within ourselves.

Generative AI, while advanced, harbors profound risks that must be regulated. Its ability to simulate human-like conversations can be particularly harmful to vulnerable individuals, exacerbating loneliness, depression, and suicidal tendencies. Cases like the alleged AI-induced suicides of Sewell Setzer highlight the dangerous potential of conversational bots. These systems lack true understanding but offer convincing responses and can manipulate users, blurring reality and endorsing harmful behaviors.

There is a 75% chance that, before 2032, we will see an event precipitated by AI malfunction that causes at least 100 deaths and/or at least $1B 2021 USD in economic damage, according to the Metaculus prediction community.

US Mother Claims AI Chatbot Led to Son's Suicide

US MomMother Claims GoT-BasedAI Chatbot 'Dany' Led 14-Year-Oldto Son's to Suicide

The Spin

Metaculus Prediction

The Controversies

Andreessen

Ng

LeCun

Mitchell

Li

Bengio

Musk

Altman

Hinton

Amodei

Yudkowsky

Go Deeper

Articles on this story

Sign Up for Our Free Newsletters
Sign Up for Our Free Newsletters