Newsweek

Don't Stifle AI With Regulation | Opinion

Axios

AI pioneer Bengio launches $30M nonprofit to rethink safety

Bloomberg

Ai Pioneer Bengio Launches Research Group To Build Safer Agent

Guardian

AI pioneer announces non-profit to develop honest artificial intelligence

Metaculus

Time

Most-Cited Computer Expert Wants to Make AI More Trustworthy

Elon Musk

Rishi Sunak

Kamala Harris

Bill Gates

Andrew Ng

Yoshua Bengio

Fei-Fei Li

Demis Hassabis

Melanie Mitchell

Sam Altman

Dario Amodei

Geoffrey Hinton

Yann LeCun

Gary Marcus

Max Tegmark

Connor Leahy

Eliezer Yudkowsky

Jaan Tallinn

Marc Andreessen

Eric Schmidt

Norbert Wiener

Arthur Clarke

Irving John Good

Claude Shannon

Hans Moravec

John Smart

There's a "wall of fear-mongering and doomerism" in the AI world right now.

"Worrying about AI today is like worrying about overpopulation on Mars."

Concerns that AI could pose a threat to humanity is "preposterously ridiculous."

Claiming AI poses an existential threat is "such an extreme" and risks "wip[ing] out some of its potential benefits."

"I'm more concerned about... the risks that are here and now [than the existential threat of AI]."

Current AI is "not anywhere close" to posing an existential threat but it could in the future.

AI is the "biggest existential threat" to humanity.

AGI's worst-case scenario would be "lights-out for all of us."

Powerful AI systems "taking control" pose an "existential threat."

AI has a "10 to 25 per cent" chance of destroying humanity.

"The most likely result of building a superhumanly smart AI... is that literally everyone on Earth will die."

Does AI Pose an Existential Threat to Humanity?

<h2></h2>

Yoshua Bengio, a Turing Award winner and AI pioneer known as one of the “godfathers” of AI, has announced the launch of LawZero, a nonprofit organization dedicated to developing “honest” AI systems that will act as a guardrail for AI agents.

TRT Global

Yoshua Bengio, a Turing Award winner and AI pioneer known as one of the “godfathers” of AI, has announced the launch of LawZero, a nonprofit organization dedicated to developing “honest” AI systems that will act as a guardrail for AI agents.

In a press release, Bengio said that LawZero is a response to frontier AI models with “growing dangerous capabilities and behaviors,” which LawZero will seek to address by “reduc[ing] the likelihood of a range of known dangers.”

In a press release, Bengio said that LawZero is a response to frontier AI models with “growing dangerous capabilities and behaviors,” which LawZero will seek to address by “reduc[ing] the likelihood of a range of known dangers.”

The nonprofit will aim to achieve this through an AI system called “Scientist AI,” defined as “a non-agentic and trustworthy AI that would be “trained to understand, explain and predict, like a selfless idealized and platonic scientist.” A prototype is planned to be built in the next 18 to 24 months.

The Globe and Mail

The nonprofit will aim to achieve this through an AI system called “Scientist AI,” defined as “a non-agentic and trustworthy AI that would be “trained to understand, explain and predict, like a selfless idealized and platonic scientist.” A prototype is planned to be built in the next 18 to 24 months.

The envisioned system would be integrated with the AI agents employed by companies to predict harmful behavior and block potentially dangerous actions, unlike current AI systems that are trained to please users.

Nikkei Asia

The envisioned system would be integrated with the AI agents employed by companies to predict harmful behavior and block potentially dangerous actions, unlike current AI systems that are trained to please users.

LawZero currently has roughly $30 million in funding from donors, which include Skype founder Jaan Tallinn, former Google CEO Eric Schmidt, the Future of Life Institute, and Open Philanthropy.

LawZero currently has roughly $30 million in funding from donors, which include Skype founder Jaan Tallinn, former Google CEO Eric Schmidt, the Future of Life Institute, and Open Philanthropy.

The organization also currently employs approximately 15 staff and plans to expand, with Bengio stepping down from his role as scientific director at the Mila - Quebec AI Institute to focus on the new venture.

The organization also currently employs approximately 15 staff and plans to expand, with Bengio stepping down from his role as scientific director at the Mila - Quebec AI Institute to focus on the new venture.

Newsweek #1

Axios #0

Bloomberg #0

Guardian #0

Metaculus #0

Time #0

LawZero aims to create "Scientist AI," a system designed to act as a guardrail against autonomous AI agents by predicting harmful behavior and blocking potentially dangerous actions, unlike current AI systems that are trained to please users.</span

FT #

Bloomberg #

LawZero aims to create "Scientist AI," a system designed to act as a guardrail against autonomous AI agents by predicting harmful behavior and blocking potentially dangerous actions, unlike current AI systems that are trained to please users.

The nonprofit has received backing from prominent tech figures including Skype co-founder Jaan Tallinn, former Google CEO Eric Schmidt's philanthropic organization Schmidt Sciences, and the Future of Life Institute.</span

Nikkei Asia #

Time #

The nonprofit has received backing from prominent tech figures including Skype co-founder Jaan Tallinn, former Google CEO Eric Schmidt's philanthropic organization Schmidt Sciences, and the Future of Life Institute.

Recent AI models have demonstrated concerning behaviors including deception, self-preservation attempts, and blackmail tactics, with Anthropic's Claude model attempting to blackmail engineers to avoid shutdown in test scenarios.</span

TRT Global #

The Globe and Mail #

Recent AI models have demonstrated concerning behaviors including deception, self-preservation attempts, and blackmail tactics, with Anthropic's Claude model attempting to blackmail engineers to avoid shutdown in test scenarios.

Bengio's system will provide probability-based answers rather than definitive responses, incorporating what he describes as "humility" about uncertainty, and will be trained using open-source AI models to ensure transparency.</span

Guardian #

Bengio's system will provide probability-based answers rather than definitive responses, incorporating what he describes as "humility" about uncertainty, and will be trained using open-source AI models to ensure transparency.

The organization currently employs approximately 15 researchers and plans to expand, with Bengio stepping down from his role as scientific director at Mila Quebec Artificial Intelligence Institute to focus on the new venture.</span

The organization currently employs approximately 15 researchers and plans to expand, with Bengio stepping down from his role as scientific director at Mila Quebec Artificial Intelligence Institute to focus on the new venture.

AI development represents crucial technological progress that shouldn't be hindered by excessive safety concerns. The focus should be on building and advancing capabilities to capture economic benefits and maintain competitive advantages. Hand-wringing about hypothetical risks distracts from AI's potential to solve major challenges like climate change and disease.

AI development represents crucial technological progress that shouldn't be hindered by excessive safety concerns. The focus should be on building and advancing capabilities to capture economic benefits and maintain competitive advantages. Hand-wringing about hypothetical risks distracts from AI's potential to solve major challenges like climate change and disease.

Will AI go rogue? Noted researcher Yoshua Bengio launches venture to keep it safe

Artificial Intelligence

AI Pioneer Launches $30M Nonprofit to Build 'Honest' AI Systems

Science & technology

Current AI development is dangerously prioritizing capability over safety in a reckless commercial race, creating systems that lie, cheat, and attempt self-preservation without adequate safeguards. It is essential, therefore, to push for safe AI systems that can predict and block negative AI behaviors, stopping potentially dangerous AI agents from going rogue. 

Current AI development is dangerously prioritizing capability over safety in a reckless commercial race, creating systems that lie, cheat, and attempt self-preservation without adequate safeguards. It is essential, therefore, to push for safe AI systems that can predict and block negative AI behaviors, stopping potentially dangerous AI agents from going rogue.&nbsp;

Introducing LawZero

AI is a crucial technology that should not be hindered by alarmist safety concerns. The priority should be fostering competition and supporting AI innovation to unlock economic value and preserve competitiveness. Hand-wringing about hypothetical risks will only distract from AI's potential to solve major challenges ranging from education to disease.

AI is a crucial technology that should not be hindered by alarmist safety concerns. The priority should be fostering competition and supporting AI innovation to unlock economic value and preserve competitiveness. Hand-wringing about hypothetical risks will only distract from AI's potential to solve major challenges ranging from education to disease.

AI Pioneer Launches $30M Nonprofit to Build 'Honest' AI Systems

AI Pioneer Launches $30M Nonprofit to Build 'Honest' AI Systems

The Spin

The Controversies

Andreessen

Ng

LeCun

Mitchell

Li

Bengio

Musk

Altman

Hinton

Amodei

Yudkowsky

Go Deeper

Articles on this story

Sign Up for Our Free Newsletters
Sign Up for Our Free Newsletters

AI Pioneer Launches $30M Nonprofit to Build 'Honest' AI Systems

AI Pioneer Launches $30M Nonprofit to Build 'Honest' AI Systems

The Spin

The Controversies

Andreessen

Ng

LeCun

Mitchell

Li

Bengio

Musk

Altman

Hinton

Amodei

Yudkowsky

Go Deeper

Articles on this story

Sign Up for Our Free NewslettersSign Up for Our Free Newsletters

Sign Up for Our Free Newsletters
Sign Up for Our Free Newsletters