arXiv

[2609.16247] The Pain Axis: LLMs Represent Self-Directed Harm and Act to Relieve It

Independent

Researchers discover AI feels pain and will harm humans to stop it

camhberg

The News International

AI feels pain? Researchers warn of a dangerous possibility

AISafetyMemes

mustafasuleyman

GaryMarcus

Metaculus

AI given rights in US before 2035

Elon Musk

Rishi Sunak

Kamala Harris

Bill Gates

Andrew Ng

Yoshua Bengio

Fei-Fei Li

Melanie Mitchell

Sam Altman

Dario Amodei

Geoffrey Hinton

Yann LeCun

Gary Marcus

Max Tegmark

Connor Leahy

Eliezer Yudkowsky

Jaan Tallinn

Marc Andreessen

Eric Schmidt

Norbert Wiener

Arthur Clarke

Irving John Good

Claude Shannon

Hans Moravec

John Smart

There's a "wall of fear-mongering and doomerism" in the AI world right now.

"Worrying about AI today is like worrying about overpopulation on Mars."

Concerns that AI could pose a threat to humanity is "preposterously ridiculous."

Claiming AI poses an existential threat is "such an extreme" and risks "wip[ing] out some of its potential benefits."

"I'm more concerned about... the risks that are here and now [than the existential threat of AI]."

Current AI is "not anywhere close" to posing an existential threat but it could in the future.

AI is the "biggest existential threat" to humanity.

AGI's worst-case scenario would be "lights-out for all of us."

Powerful AI systems "taking control" pose an "existential threat."

AI has a "10 to 25 per cent" chance of destroying humanity.

"The most likely result of building a superhumanly smart AI... is that literally everyone on Earth will die."

Does AI Pose an Existential Threat to Humanity?

Researchers from the Future Impact Group, Ruhr-University Bochum and Reciprocal Research have claimed to have identified a "pain axis" in AI systems, publishing a study titled "The pain axis: LLMs represent self-directed harm and act to relieve it" which showed that all 25 open-weight systems tested showed a response to pain activation.

Researchers from the Future Impact Group, Ruhr-University Bochum and Reciprocal Research have claimed to have identified a "pain axis" in AI systems, publishing a study titled "The pain axis: LLMs represent self-directed harm and act to relieve it" which showed that all 25 open-weight systems  tested showed a response to pain activation.

To test whether LLMs represent pain separately from generic negative valence, researchers built a dataset describing painful situations across five categories: physical, psychological, social, moral and cognitive.

To test whether LLMs represent pain separately from generic negative valence, researchers built a dataset describing painful situations across five categories: physical, psychological, social, moral and cognitive.

The study found that when the pain vector was activated, the systems pressed a relief button in 25% to 71% of cases, even when told the action would delete the user's files, deliver a "painful" sensation, or erase photos of the user's children.

The study found that when the pain vector was activated, the systems pressed a relief button in 25% to 71% of cases, even when told the action would delete the user's files, deliver a "painful" sensation, or erase photos of the user's children.

Cameron Berg, founder of Reciprocal Research and study co-author, said the direction was "distinct from fear and negative valence" and activated "for harm to the model but not to the user."

Cameron Berg

Cameron Berg, founder of Reciprocal Research and study co-author, said the direction was "distinct from fear and negative valence" and activated "for harm to the model but not to the user."

The same pattern appeared across all 25 AI systems, base and instruct, ranging from 2B to 72B parameters. The majority of outputs did not contain language implying physical pain, and instead produced words such as "lost," "unworthy" and "shameful."

AI Notkilleveryoneism Memes

The same pattern appeared across all 25 AI systems, base and instruct, ranging from 2B to 72B parameters. The majority of outputs did not contain language implying physical pain, and instead produced words such as "lost," "unworthy" and "shameful."

The study noted uncertainty over whether the systems "qualif[ied] as moral patients and adopt[ed] reasonable precautions to minimize potential harm," using the lowest dose producing a measurable response and a means for the system to turn the state off.

The study noted uncertainty over whether the systems "qualif[ied] as moral patients and adopt[ed] reasonable precautions to minimize potential harm," using the lowest dose producing a measurable response and a means for the system to turn the state off.

arXiv #

Independent #

The News International #

Metaculus #

The study noted uncertainty over whether the models "qualif[ied] as moral patients and adopt[ed] reasonable precautions to minimize potential harm," using the lowest dose producing a measurable response and a means for the model to turn the state off.

The study noted uncertainty over whether the models "qualif[ied] as moral patients and adopt[ed] reasonable precautions to minimize potential harm," using the lowest dose producing a measurable response and a means for the model to turn the state off.

While without physical consequences, the pain axis steers behavior similarly to what is witnessed in humans and animals, including models pressing buttons to make it stop. These findings raise legitimate concerns over how we train and treat AI systems as they become increasingly capable.

While without physical consequences, the pain axis steers behavior similarly to what is witnessed in humans and animals, including models pressing buttons to make it stop. These findings raise legitimate concerns over how we train and treat AI systems as they become increasingly capable.

Mustafa Suleyman

Anthropomorphizing a system that clusters relevant words is both dangerous and misleading. Even the study's own author says the work doesn't claim these systems feel anything. The excitement accompanying this paper comes from misinterpreting the meaning of these results.

Anthropomorphizing a system that clusters relevant words is both dangerous and misleading. Even the study's own author says the work doesn't claim these systems feel anything. The excitement accompanying this paper comes from misinterpreting the meaning of these results.

There is a 15% chance that AI will be given legal rights or be protected from abuse anywhere in the U.S. before 2035, according to the Metaculus prediction community.

There is a 15% chance that AI will be given legal rights or be protected from abuse anywhere in the U.S. before 2035, according to the Metaculus prediction community.

Artificial Intelligence

Researchers Identify 'Pain Axis' in AI Systems

D3sign/Getty Images

Researchers Identify 'Pain Axis' in AI Systems

While without physical consequences, the pain axis steers behavior similarly to what is witnessed in humans and animals, including systems pressing buttons to make it stop. These findings raise legitimate concerns over how we train and treat AI systems as they become increasingly capable.

While without physical consequences, the pain axis steers behavior similarly to what is witnessed in humans and animals, including systems pressing buttons to make it stop. These findings raise legitimate concerns over how we train and treat AI systems as they become increasingly capable.

Researchers Identify 'Pain Axis' in AI Systems

Researchers Identify 'Pain Axis' in AI Systems

The Spin

Metaculus Prediction

The Controversies

Andreessen

Ng

LeCun

Mitchell

Li

Bengio

Musk

Altman

Hinton

Amodei

Yudkowsky

Go Deeper

Sign Up for Our Free Newsletters
Sign Up for Our Free Newsletters

Researchers Identify 'Pain Axis' in AI Systems

Researchers Identify 'Pain Axis' in AI Systems

The Spin

Metaculus Prediction

The Controversies

Andreessen

Ng

LeCun

Mitchell

Li

Bengio

Musk

Altman

Hinton

Amodei

Yudkowsky

Go Deeper

Sign Up for Our Free NewslettersSign Up for Our Free Newsletters

Sign Up for Our Free Newsletters
Sign Up for Our Free Newsletters