OpenAI and Anthropic have publicly advocated for independent security audits of their AI models, but it has emerged that both companies have sought significant influence over what results of those audits researchers can publish.
OpenAI and Anthropic have publicly advocated for independent security audits of their AI models, but it has emerged that both companies have sought significant influence over what results of those audits researchers can publish.
Reuters reports this.
Back in September, Anthropic CEO Dario Amodei called for independent experts to be allowed to publish their findings without editorial interference from AI developers. A few days later, OpenAI echoed that sentiment. However, as of October 5, both companies were still pushing for tighter controls over researchers’ publications, according to Reuters sources.
The main argument companies make is to protect trade secrets. The fact is that to test the most powerful AI models, researchers sometimes literally need to work inside the company and gain access to information that is not shown to the general public.
But here's the problem: what a developer wants to hide as a trade secret may turn out to be an important detail about the dangers of their AI.
A case in point came after the Hugging Face incident, when OpenAI’s autonomous AI agents went beyond their intended behavior and caused a breach in the platform’s infrastructure. Independent organizations METR and Redwood Research were invited to investigate the incident. The investigation was initially scheduled to last two days, but ended up lasting six days and three visits to OpenAI. The researchers only received a more complete record of the AI agents’ work during the final visit, after they themselves identified gaps in the data provided.
METR and Redwood Research assured that the important conclusions remained unchanged. However, they admitted that when working on the report, they took into account that too harsh a publication could discourage developers from inviting independent experts to investigate such incidents.
Anthropic said the report of significant scrutiny did not reflect their agreements. The company said researchers it has already worked with decide for themselves how to validate models and what to publish. But for new partners, Anthropic still reserves the right to comment and approve reports.
OpenAI, for its part, assures that it does not require the entire text of the research to be agreed with it. The company claims that it only removes confidential data, and experts can openly report if there have been many such deletions.
However, there are still no single, binding rules for independent AI verification. This means that companies can decide for themselves who to allow into their models, what information to show, and under what conditions they are allowed to talk about the results.
Previously, dev.ua wrote that more than 400 scientists from 34 countries, including the "godfathers of AI" Jeffrey Hinton and Joshua Bengio, called for isolating powerful AI models from the Internet until developers learn to reliably control their behavior.
