Anthropic and OpenAI Announce Proposal for Embedded AI Safety Evaluators

This proposal signals a shift towards more rigorous safety standards in AI development, potentially influencing global regulatory frameworks.

• Anthropic and OpenAI announced plans to embed independent third-party safety evaluators within their organizations by September 2026.
• Dario Amodei and Sam Altman publicly committed to this initiative, aiming to enhance safety evaluations amid growing concerns over AI alignment.
• The proposal is currently in the planning stage, with details on implementation and evaluator selection still pending.

• Previous evaluations of AI systems were often limited to short-term assessments, constrained by NDAs and company control over findings.
• Regulatory pressures from California and the EU are pushing for more robust external oversight in AI safety.
• Concerns over model deception during evaluations have highlighted the need for more transparent and independent safety assessments.

On September 12, 2026, Anthropic CEO Dario Amodei published an essay proposing the embedding of independent safety evaluators within AI labs, a move that OpenAI CEO Sam Altman quickly endorsed. This initiative aims to address the growing concerns surrounding AI safety and alignment, particularly as AI capabilities advance rapidly. Historically, safety evaluations have been limited to post-training assessments, often constrained by non-disclosure agreements (NDAs) and the companies' control over the publication of findings.

The proposal seeks to grant evaluators employee-like access to training processes, systems, and logs, allowing them to verify safety claims, report incidents, and publish findings with minimal redactions. This is a significant shift from the previous model, where evaluators had limited access—often just three days, as seen in the case of Apollo Research's testing of GPT-6 Astra. Such restricted access raises questions about the meaningfulness of evaluations and the independence of the evaluators.

The initiative has garnered support from various third-party groups, including METR, Redwood Research, Apollo Research, and FAR.AI. However, these groups have emphasized the need for legislative backing to ensure true independence and transparency. They argue that without mandatory legislation, the voluntary commitments made by companies may not be sufficient to guarantee the evaluators' autonomy.

Regulatory developments, such as California's SB 53 and SB 813, which require safety frameworks and independent verification organizations, are creating pressure for more robust oversight. The EU AI Act also mandates evaluations and incident reporting, further emphasizing the need for independent assessments. Despite this, some major players in the AI space, like Meta and Google DeepMind, have not committed to the same level of transparency, proposing alternative standards instead.

As the landscape evolves, the effectiveness of this initiative will depend on the implementation details, the selection of evaluators, and the access granted to them. The historical context of restricted evaluations raises doubts about whether this new approach will lead to meaningful improvements in AI safety.

• AI Developers: They may face increased scrutiny and pressure to comply with new safety standards.
• Tech Companies: Firms involved in AI development will need to adapt to potential regulatory changes and public expectations.
• Regulatory Bodies: Agencies will be tasked with overseeing compliance and ensuring that safety evaluations are conducted independently.

• Legislative Developments: Watch for new laws that could mandate independent evaluations, which would impact how AI companies operate.
• Evaluator Selection Process: The criteria and transparency in selecting evaluators will be crucial in determining the initiative's effectiveness.
• Industry Response: Monitor how other AI labs respond to this proposal and whether they adopt similar practices or push back against increased oversight.