Anthropic and OpenAI, two of the most prominent names in artificial intelligence, are currently negotiating the terms under which outside safety researchers can test their models. According to Reuters, both labs have pushed for stricter limits on what these external evaluators are allowed to publish, citing the need to protect trade secrets. The talks reveal a growing tension between the AI industry's desire for confidentiality and the public's interest in independent, verifiable safety assessments.
What is 'embedded evaluation'?
At the heart of the dispute is a practice known as "embedded evaluation." In this model, external researchers are placed inside an AI lab, working alongside company engineers to test advanced models for potential risks—such as bias, harmful outputs, or the ability to deceive. The idea is that by embedding evaluators within the development process, they can catch problems that might otherwise go unnoticed.
However, this close access comes with a catch. The labs argue that allowing evaluators to freely publish their findings could expose proprietary methods, training data, or other sensitive information that gives them a competitive edge. They want to review and potentially restrict what the evaluators disclose, to ensure that trade secrets remain protected.
The evaluators' counterargument
On the other side, safety evaluators contend that without the freedom to report their findings openly, the public has no way of knowing whether the tests were meaningful or merely performative. If a lab can edit or suppress critical findings before they are released, the entire exercise loses its credibility. Independent evaluation is only valuable if it is truly independent.
This tug-of-war is not just academic. It has real implications for how AI safety is assessed and for the trust that the public and regulators place in these companies. As AI models become more powerful and more integrated into daily life, the demand for transparent safety testing is likely to grow.
What this means for investors
For everyday investors, this dispute may seem like an internal industry squabble, but it carries broader significance. The outcome of these negotiations could shape the regulatory landscape for AI companies, which in turn could affect their valuations and growth prospects.
If labs are seen as too secretive or as suppressing safety findings, they could face reputational damage and stricter regulatory oversight. On the other hand, if they are forced to disclose too much, they might lose their competitive advantage. Investors will be watching to see how this balance is struck, as it could influence the long-term sustainability of AI business models.
This story also comes amid a broader backdrop of heightened scrutiny on AI companies. For instance, Anthropic has been pushing its data center partners to accelerate construction ahead of a potential IPO, signaling that the company is preparing for public markets. How it handles safety reviews could be a key factor in how it is perceived by investors.
Similarly, IREN shares jumped when Anthropic was named in an AI cloud deal, showing how closely tied AI companies' fortunes are to their infrastructure and partnerships. Any regulatory or reputational hit could ripple through these relationships.
The bigger picture
The clash between AI labs and safety evaluators is part of a larger conversation about accountability in the tech industry. As AI becomes more powerful, the question of who gets to assess its risks—and how openly they can do so—will only become more pressing.
For now, the talks between Anthropic, OpenAI, and outside reviewers are ongoing. The outcome will likely set a precedent for how other AI companies handle external safety evaluations. Investors should keep an eye on these developments, as they could signal how much transparency the industry is willing to embrace.
In the meantime, the broader tech sector continues to face its own challenges. For example, a recent tech selloff drove investors into eurozone bonds, highlighting the volatility that can accompany tech investments. While AI remains a promising area, it is not immune to such swings.
Ultimately, the resolution of this dispute will help define the norms for AI safety testing. Whether that leads to more robust oversight or more guarded practices remains to be seen, but it is a story worth following for anyone with a stake in the future of technology.
