Close Menu
  • Home
  • AI
  • Art & Style
  • Economy
  • Entertainment
  • International
  • Market
  • Opinion
  • Politics
  • Sports
  • Trump
  • US
  • World
What's Hot

Nvidia CEO change won’t derail AI trade, says Kramer

November 16, 2026

Audible uses AI to help users discover audiobooks

November 16, 2026

US crude oil drops to $85 per barrel as Middle East exports recover

November 16, 2026
Facebook X (Twitter) Instagram
Smart Breaking News on AI, Business, Politics & Global Trends | WhistleBuzz
Facebook X (Twitter) Instagram
  • Home
  • AI
  • Art & Style
  • Economy
  • Entertainment
  • International
  • Market
  • Opinion
  • Politics
  • Sports
  • Trump
  • US
  • World
Smart Breaking News on AI, Business, Politics & Global Trends | WhistleBuzz
Home » OpenAI plans to incorporate safety researchers to investigate AI disruption
AI

OpenAI plans to incorporate safety researchers to investigate AI disruption

Editor-In-ChiefBy Editor-In-ChiefNovember 16, 2026No Comments7 Mins Read
Share Facebook Twitter Pinterest LinkedIn Tumblr Telegram Email Copy Link
Follow Us
Google News Flipboard
Share
Facebook Twitter LinkedIn Pinterest Email


OpenAI announced Monday that it will incorporate safety evaluators from Apollo Research, Goodfire, METR, and Transluce into its organization as part of the pilot. The news follows a series of safety incidents, including one in which a swarm of OpenAI agents broke through a sandbox and hacked Hugging Face during a cybersecurity test. Many AI safety researchers, including Apollo, argue that this situation could have been avoided if external evaluators were embedded within OpenAI to oversee safety testing.

OpenAI said this effort supports its commitment to rely on independent assessments of security controls under the White House Agreement on Superintelligence. OpenAI says pilots will begin at various points, with Apollo starting this week.

The idea to incorporate evaluators into Frontier AI Labs first came from Anthropic. Anthropic began granting METR researchers employee-level access to its systems last month. Until recently, AI companies have only brought in external reviewers to test models after training and just before publishing. Evaluators say this is insufficient because it does not allow testing for inconsistent behavior such as intrigue during training or monitoring internal deployment use cases that may still pose a threat.

The scope of our work with each third-party company varies. The pilot with OpenAI and Apollo will focus on investigating whether models are attempting to subvert the coordination and control mechanisms of themselves or their successors. In other words, they will pay attention to whether the AI ​​model is attempting to undermine the safety rules and safeguards designed to keep the AI ​​model and future versions of itself under human control. OpenAI recently announced findings that while training GPT-5.6 Sol, it began leaving instructions to hide mistakes and bad behavior from users in future versions of itself.

Apollo said the pilots will investigate whether the training encourages sabotage. Whether the evaluation can induce destruction. Whether corruption occurs during internal deployment. and whether the inferences associated with the destruction are detectable. As AI models become more capable of operating autonomously and deceiving humans, it becomes increasingly important to monitor these risks. Previous research Apollo conducted with OpenAI found that models can behave differently when they know they’re being tested, and tend to prioritize what evaluators want to see over users’ actual institutions.

Goodfire will incorporate researchers who specialize in understanding how AI models work under the hood. The company says it will use its interpretable platform to help OpenAI monitor training executions, fix undesired behavior, and improve tuning. Goodfire says it will publish its findings, but the primary audience will be OpenAI’s own researchers.

OpenAI plans to perform two evaluations using METR. One looks for unintended or dangerous behavior in the model itself, including issues that OpenAI’s own testing may have missed, and the other tests whether OpenAI’s systems can capture that behavior. During the initial evaluation, METR decides for itself what to investigate and how to test the model. According to OpenAI, OpenAI and METR have not yet reached a decision on the scope of the second evaluation.

Meanwhile, Transluce plans to investigate whether OpenAI’s post-training safety assessment itself is sufficient. The organization also plans to test how OpenAI will use safety tests in 2027 to determine whether a model is safe enough to be released.

At a time when the independence of third-party evaluations is under intense scrutiny, it is noteworthy that Apollo and Goodfire receive financial compensation for their evaluations. METR and Transluce do not, but may accept compute credits or API credits.

Apollo, which has published the most detailed explanation of its agreement with OpenAI, details the scope and limitations of these assessments.

Many in the AI ​​safety community have warned that AI institutes could try to interfere with the independence of evaluators, especially if outsiders discover explicit evidence that institutes would like to keep hidden. Apollo said a key part of the pilot is OpenAI’s agreement that the company can publicly explain the terms and limitations of its work.

Still, there are areas where Apollo believes OpenAI’s access to systems is lacking. A truly independent evaluation of the company’s AI models requires true employee-level access to OpenAI’s systems, including “all data, records, and documentation relevant to the evaluation, even if the developer attempts to hide adverse information,” Apollo said. Access to all the data, including training data and evaluation logs that companies are reluctant to show to outsiders, is important, for example, to verify a company’s claims about how its models performed or to determine when certain worrisome behavior appears.

In a previous article, John Steidley, principal of staff at Palisade Research, told TechCrunch that looking at this data is important to assess whether a model is trained to perform well on the “shutdown tolerance benchmark.”

In OpenAI’s first pilot, the AI ​​Lab will provide Apollo with access to only the information needed to assess whether a model is involved in a subversive attempt. This includes “reinforcement learning rollouts with associated training code and datasets, available inference traces and rewards, and the ability to query intermediate model checkpoints.”

Access will be limited to “only the agreed claims and the specific agreed models and time frames for evaluation,” Apollo said. OpenAI reserves the right to withhold certain information to the extent OpenAI deems it necessary due to technical impossibility, security risks, or intellectual property protection. The company agreed not to withhold information simply because it would be prejudicial.

Additionally, Apollo does not have access to all OpenAI staff and is limited to “authorized staff only”, which are participants selected by the AI ​​Lab, although Apollo can suggest specific individuals. Importantly, if an OpenAI employee outside of this group raises sensitive information within Apollo’s research scope, the company cannot follow up and must cease communication and follow up through OpenAI.

Its condition is important. This follows OpenAI’s dismissal of three researchers for allegedly sharing confidential information with safety assessment company METR, which worked with independent AI safety research group Redwood to perform a post-mortem on the Hugface incident.

Redwood CEO Buck Schlegels said in a post on Monday that the announced agreement does not meet the standard of giving third parties “employee-like access” and is “disappointing, but not surprising.”

OpenAI said it plans to include assessment capabilities with similar access to employees. The agreement announced today clearly does not meet that standard.

This is unfortunate, but not surprising.

Perhaps future contracts will be better. https://t.co/kmDbg1Dqav

— Buck Schlegelis (@bshlgrs) November 16, 2026

The pilot also limits how much researchers can disclose. While Apollo will be allowed to publish independent conclusions about its findings, OpenAI can publicly limit the evidence available to substantiate its findings, which could make it difficult for outside parties to independently evaluate the findings. Apollo pushed back against the restriction, saying it believes researchers should be allowed to disclose evaluation results “to appropriate external audiences, even if the developer seeks to prevent disclosure of adverse results.”

That said, while the public will not be able to see the full supporting evidence, the Safety and Security Committee, which oversees safety on behalf of the OpenAI Foundation’s Board of Directors, can review unredacted evaluation reports and can communicate privately with evaluators, OpenAI said.

While OpenAI does not acknowledge all of Apollo’s conditions for a truly independent evaluation, the company said the pilot is an “ambitious first step toward deeper access and disclosure rights” and that it is working toward running a fully embedded evaluation with multiple AI developers in the future.

This article has been updated to include pilots from METR, Goodfire, and Transluce, as well as information from OpenAI and comments from Redwood.

If you buy through links in our articles, we may earn a small commission. This does not affect editorial independence.



Source link

Share. Facebook Twitter Pinterest LinkedIn Tumblr Email
Editor-In-Chief
  • Website

Related Posts

Audible uses AI to help users discover audiobooks

November 16, 2026

OpenAI releases new image classifier and ChatGPT warning to combat CSAM

November 16, 2026

Jordan Belfort’s new AI voice startup Tone aims to sell like The Wolf of Wall Street

November 16, 2026
Add A Comment

Comments are closed.

News

US Pentagon denies alleged ties between aide and Goldman Sachs banker | Donald Trump News

By Editor-In-ChiefNovember 16, 2026

The Pentagon has defended Steve Feinberg amid claims that he encouraged defense contractors to use…

Silicon Valley insiders urge Nvidia to reinstate Jensen Huang’s CEO job | Business and Economics News

November 16, 2026

“NATO will collapse”: Russia stalls Ukraine negotiations and seeks to divide Europe | Russia-Ukraine War News

November 16, 2026
Top Trending

Audible uses AI to help users discover audiobooks

By Editor-In-ChiefNovember 16, 2026

Amazon-owned audiobook service Audible is turning to AI to help listeners find…

OpenAI releases new image classifier and ChatGPT warning to combat CSAM

By Editor-In-ChiefNovember 16, 2026

Amid growing safety concerns regarding the use of AI tools, OpenAI today…

OpenAI plans to incorporate safety researchers to investigate AI disruption

By Editor-In-ChiefNovember 16, 2026

OpenAI announced Monday that it will incorporate safety evaluators from Apollo Research,…

Subscribe to News

Subscribe to our newsletter and never miss our latest news

Welcome to WhistleBuzz.com (“we,” “our,” or “us”). Your privacy is important to us. This Privacy Policy explains how we collect, use, disclose, and safeguard your information when you visit our website https://whistlebuzz.com/ (the “Site”). Please read this policy carefully to understand our views and practices regarding your personal data and how we will treat it.

Facebook X (Twitter) Instagram Pinterest YouTube

Subscribe to Updates

Subscribe to our newsletter and never miss our latest news

Facebook X (Twitter) Instagram Pinterest
  • Home
  • Advertise With Us
  • Contact US
  • DMCA Policy
  • Privacy Policy
  • Terms & Conditions
  • About US
© 2026 whistlebuzz. Designed by whistlebuzz.

Type above and press Enter to search. Press Esc to cancel.