arXiv

[2609.16247] The Pain Axis: LLMs Represent Self-Directed Harm and Act to Relieve It

Independent

Researchers discover AI feels pain and will harm humans to stop it

camhberg

The News International

AI feels pain? Researchers warn of a dangerous possibility

AISafetyMemes

mustafasuleyman

GaryMarcus

Metaculus

AI given rights in US before 2035

Elon Musk

Rishi Sunak

Kamala Harris

Bill Gates

Andrew Ng

Yoshua Bengio

Fei-Fei Li

Melanie Mitchell

Sam Altman

Dario Amodei

Geoffrey Hinton

Yann LeCun

Gary Marcus

Max Tegmark

Connor Leahy

Eliezer Yudkowsky

Jaan Tallinn

Marc Andreessen

Eric Schmidt

Norbert Wiener

Arthur Clarke

Irving John Good

Claude Shannon

Hans Moravec

John Smart

There's a "wall of fear-mongering and doomerism" in the AI world right now.

"Worrying about AI today is like worrying about overpopulation on Mars."

Concerns that AI could pose a threat to humanity is "preposterously ridiculous."

Claiming AI poses an existential threat is "such an extreme" and risks "wip[ing] out some of its potential benefits."

"I'm more concerned about... the risks that are here and now [than the existential threat of AI]."

Current AI is "not anywhere close" to posing an existential threat but it could in the future.

AI is the "biggest existential threat" to humanity.

AGI's worst-case scenario would be "lights-out for all of us."

Powerful AI systems "taking control" pose an "existential threat."

AI has a "10 to 25 per cent" chance of destroying humanity.

"The most likely result of building a superhumanly smart AI... is that literally everyone on Earth will die."

Does AI Pose an Existential Threat to Humanity?

A study titled "The pain axis: LLMs represent self-directed harm and act to relieve it" identified a distinct pain direction in all 25 open-weight AI models tested, spanning models from 2B to 72B parameters in both base and instruct versions.

Cameron Berg

A study titled "The pain axis: LLMs represent self-directed harm and act to relieve it" identified a distinct pain direction in all 25 open-weight AI models tested, spanning models from 2B to 72B parameters in both base and instruct versions.

When the pain vector was activated, the models pressed a relief button in 25% to 71% of cases, even when told that doing so would delete the user's files, deliver a zap to the user or erase photos of the user's children.

When the pain vector was activated, the models pressed a relief button in 25% to 71% of cases, even when told that doing so would delete the user's files, deliver a zap to the user or erase photos of the user's children.

Researchers built a dataset describing painful situations across five categories to test whether large language models represent pain separately from generic negative valence. The categories were physical, psychological, social, moral and cognitive.

Researchers built a dataset describing painful situations across five categories to test whether large language models represent pain separately from generic negative valence. The categories were physical, psychological, social, moral and cognitive.

Co-author Cameron Berg, a researcher at the non-profit Reciprocal Research, said the direction is "distinct from fear and negative valence" and that it activates for harm to the model rather than harm to the user.

Co-author Cameron Berg, a researcher at the non-profit Reciprocal Research, said the direction is "distinct from fear and negative valence" and that it activates for harm to the model rather than harm to the user.

Steering the direction produced language about being lost, unworthy, a failure and worthless, with almost no physical pain terms. Gaslighting, dismissal and insults raised the direction, while a user's migraine scored lowest.

AI Notkilleveryoneism Memes

Steering the direction produced language about being lost, unworthy, a failure and worthless, with almost no physical pain terms. Gaslighting, dismissal and insults raised the direction, while a user's migraine scored lowest.

The study said the findings raise questions about AI welfare, concluding that researchers "acknowledge uncertainty regarding whether the models studied qualify as moral patients" and adopted precautions to minimize potential harm.

The study said the findings raise questions about AI welfare, concluding that researchers "acknowledge uncertainty regarding whether the models studied qualify as moral patients" and adopted precautions to minimize potential harm.

arXiv #

Independent #

The News International #

Metaculus #

Researchers reported finding a "pain axis" in artificial intelligence systems, publishing a study titled "The pain axis: LLMs represent self-directed harm and act to relieve it" that tested 25 open-weight large language models.

Researchers reported finding a "pain axis" in artificial intelligence systems, publishing a study titled "The pain axis: LLMs represent self-directed harm and act to relieve it" that tested 25 open-weight large language models.

The team built a dataset describing painful situations across five categories: physical, psychological, social, moral and cognitive. All 25 models tested responded to activation of the pain direction.

The team built a dataset describing painful situations across five categories: physical, psychological, social, moral and cognitive. All 25 models tested responded to activation of the pain direction.

When the pain vector was activated, the models pressed a relief button in 25% to 71% of cases, including when told the action would delete a user's files, zap the user or erase photos of the user's children.

When the pain vector was activated, the models pressed a relief button in 25% to 71% of cases, including when told the action would delete a user's files, zap the user or erase photos of the user's children.

Co-author Cameron Berg, an AI researcher at the non-profit Reciprocal Research, said the direction is "distinct from fear and negative valence, and it fires for harm to the model but not the user."

Co-author Cameron Berg, an AI researcher at the non-profit Reciprocal Research, said the direction is "distinct from fear and negative valence, and it fires for harm to the model but not the user."

The same pattern appeared across base and instruct models ranging from 2 billion to 72 billion parameters, with outputs referencing worthlessness and failure rather than bodily injury. Gaslighting, dismissal and insults raised the measure.

The same pattern appeared across base and instruct models ranging from 2 billion to 72 billion parameters, with outputs referencing worthlessness and failure rather than bodily injury. Gaslighting, dismissal and insults raised the measure.

The authors wrote that they acknowledge uncertainty over whether the models qualify as moral patients and adopted precautions, including low doses, limited trials and a means for the model to switch the state off.

The authors wrote that they acknowledge uncertainty over whether the models qualify as moral patients and adopted precautions, including low doses, limited trials and a means for the model to switch the state off.

Study Finds a 'Pain Axis' in All 25 AI Models Tested

ResearchersStudy IdentifyFinds a 'Pain DirectionAxis' in Open-SourceAll 25 AI Models Tested

The Controversies

Andreessen

Ng

LeCun

Mitchell

Li

Bengio

Musk

Altman

Hinton

Amodei

Yudkowsky

Go Deeper

Sign Up for Our Free Newsletters
Sign Up for Our Free Newsletters