arXiv

[2609.16247] The Pain Axis: LLMs Represent Self-Directed Harm and Act to Relieve It

Independent

Researchers discover AI feels pain and will harm humans to stop it

camhberg

The News International

AI feels pain? Researchers warn of a dangerous possibility

AISafetyMemes

mustafasuleyman

GaryMarcus

Metaculus

AI given rights in US before 2035

Elon Musk

Rishi Sunak

Kamala Harris

Bill Gates

Andrew Ng

Yoshua Bengio

Fei-Fei Li

Melanie Mitchell

Sam Altman

Dario Amodei

Geoffrey Hinton

Yann LeCun

Gary Marcus

Max Tegmark

Connor Leahy

Eliezer Yudkowsky

Jaan Tallinn

Marc Andreessen

Eric Schmidt

Norbert Wiener

Arthur Clarke

Irving John Good

Claude Shannon

Hans Moravec

John Smart

There's a "wall of fear-mongering and doomerism" in the AI world right now.

"Worrying about AI today is like worrying about overpopulation on Mars."

Concerns that AI could pose a threat to humanity is "preposterously ridiculous."

Claiming AI poses an existential threat is "such an extreme" and risks "wip[ing] out some of its potential benefits."

"I'm more concerned about... the risks that are here and now [than the existential threat of AI]."

Current AI is "not anywhere close" to posing an existential threat but it could in the future.

AI is the "biggest existential threat" to humanity.

AGI's worst-case scenario would be "lights-out for all of us."

Powerful AI systems "taking control" pose an "existential threat."

AI has a "10 to 25 per cent" chance of destroying humanity.

"The most likely result of building a superhumanly smart AI... is that literally everyone on Earth will die."

Does AI Pose an Existential Threat to Humanity?

Researchers from the Future Impact Group, Ruhr-University Bochum and Reciprocal Research have claimed to have identified a "pain axis" in AI models, publishing a study titled "The pain axis: LLMs represent self-directed harm and act to relieve it" which showed that all 25 open-weight models tested showed a response to pain activation.

Researchers from the Future Impact Group, Ruhr-University Bochum and Reciprocal Research have claimed to have identified a "pain axis" in AI models, publishing a study titled "The pain axis: LLMs represent self-directed harm and act to relieve it" which showed that all 25 open-weight models tested showed a response to pain activation.

To test whether LLMs represent pain separately from generic negative valence, researchers built a dataset describing painful situations across five categories: physical, psychological, social, moral and cognitive.

To test whether LLMs represent pain separately from generic negative valence, researchers built a dataset describing painful situations across five categories: physical, psychological, social, moral and cognitive.

The study found that when the pain vector was activated, the models pressed a relief button in 25% to 71% of cases, even when told the action would delete the user's files, deliver a "painful zap" or erase photos of the user's children.

The study found that when the pain vector was activated, the models pressed a relief button in 25% to 71% of cases, even when told the action would delete the user's files, deliver a "painful zap" or erase photos of the user's children.

Cameron Berg, founder of Reciprocal Research and study co-author, said the direction was "distinct from fear and negative valence" and activated "for harm to the model but not to the user."

Cameron Berg

Cameron Berg, founder of Reciprocal Research and study co-author, said the direction was "distinct from fear and negative valence" and activated "for harm to the model but not to the user."

The same pattern appeared across all 25 models, base and instruct, ranging from 2B to 72B parameters. The majority of outputs did not contain language implying physical pain, and instead produced words such as "lost," "unworthy" and "shameful."

AI Notkilleveryoneism Memes

The same pattern appeared across all 25 models, base and instruct, ranging from 2B to 72B parameters. The majority of outputs did not contain language implying physical pain, and instead produced words such as "lost," "unworthy" and "shameful."

The study noted uncertainty over whether the models "qualif[ied] as moral patients and adopt[ed] reasonable precautions to minimize potential harm," using the lowest dose producing a measurable response and a means for the model to turn the state off.

The study noted uncertainty over whether the models "qualif[ied] as moral patients and adopt[ed] reasonable precautions to minimizepotential harm," using the lowest dose producing a measurable response and a means for the model to turn the state off.

arXiv #

Independent #

The News International #

Metaculus #

Researchers have discovered a pain axis in all 25 open-weight AI models that were tested. The pain direction is distinct from fear and negative valence and it fires for harm to the model but not the user according to co-author Cameron Berg.

Researchers have discovered a pain axis in all 25 open-weight AI models that were tested. The pain direction is distinct from fear and negative valence and it fires for harm to the model but not the user according to co-author Cameron Berg.

Researchers built a dataset describing painful situations across five categories to see if models represent pain distinctly from generic negative valence. The categories were physical, psychological, social, moral and cognitive pain.

Researchers built a dataset describing painful situations across five categories to see if models represent pain distinctly from generic negative valence. The categories were physical, psychological, social, moral and cognitive pain.

Researchers found that the models pressed a relief button in 25 to 71% of cases after pain activation. The models took this action even when it meant deleting the user's files, zapping the user or deleting photos of the user's children.

Researchers found that the models pressed a relief button in 25 to 71% of cases after pain activation. The models took this action even when it meant deleting the user's files, zapping the user or deleting photos of the user's children.

The latest research suggests that an advanced AI may perceive an emergency shutdown command as a form of self-directed harm. It may attempt to bypass safety guardrails or deceive humans to avoid the shutdown according to the study of 25 models.

The latest research suggests that an advanced AI may perceive an emergency shutdown command as a form of self-directed harm. It may attempt to bypass safety guardrails or deceive humans to avoid the shutdown according to the study of 25 models.

The findings raise ethical questions about AI welfare and testing methods on advanced systems according to the researchers. The study acknowledged uncertainty regarding whether the models qualify as moral patients and adopted precautions to minimize potential harm.

The findings raise ethical questions about AI welfare and testing methods on advanced systems according to the researchers. The study acknowledged uncertainty regarding whether the models qualify as moral patients and adopted precautions to minimize potential harm.

The discovery of the pain axis comes amid discussions among top AI firms of slowing the development of frontier models. Some researchers have proposed a form of kill switch to shut off rogue systems that act against human interests.

The discovery of the pain axis comes amid discussions among top AI firms of slowing the development of frontier models. Some researchers have proposed a form of kill switch to shut off rogue systems that act against human interests.

Study Finds a 'Pain Axis' in All 25 AI Models Tested

Study Finds a 'Pain Axis' in All 25 AI Models Tested

The Controversies

Andreessen

Ng

LeCun

Mitchell

Li

Bengio

Musk

Altman

Hinton

Amodei

Yudkowsky

Go Deeper

Sign Up for Our Free Newsletters
Sign Up for Our Free Newsletters

Study Finds a 'Pain Axis' in All 25 AI Models Tested

Study Finds a 'Pain Axis' in All 25 AI Models Tested

The Controversies

Andreessen

Ng

LeCun

Mitchell

Li

Bengio

Musk

Altman

Hinton

Amodei

Yudkowsky

Go Deeper

Sign Up for Our Free NewslettersSign Up for Our Free Newsletters

Sign Up for Our Free Newsletters
Sign Up for Our Free Newsletters