OpenAI and Anthropic Launch Joint AI Safety Testing Initiative
OpenAI and Anthropic Launch Joint AI Safety Testing Initiative
In a rare move toward industry collaboration, two of the world’s leading AI research organizations—OpenAI and Anthropic—have conducted a joint safety evaluation of each other's advanced language models. This initiative aims to establish new standards for AI safety as these systems become increasingly integrated into daily life and business operations.
Why Cross-Lab Safety Testing Matters
As AI models become more powerful and widely used, ensuring their responsible deployment is critical. OpenAI co-founder Wojciech Zaremba emphasized the need for industry-wide collaboration: setting safety standards must transcend competition, even amid fierce battles for talent and investment.
How the Collaboration Worked
- OpenAI and Anthropic gave each other access to less-restricted versions of their AI models for testing.
- Both companies aimed to spot blind spots in their internal safety checks by allowing external experts to probe their systems.
- The joint research focused on uncovering how models behave in challenging scenarios, and how often they refuse to answer versus risk giving incorrect information.
Key Findings from Joint Evaluations
- Hallucination vs. Refusal: Anthropic’s Claude Opus 4 and Sonnet 4 models refused to answer up to 70% of questions when uncertain, prioritizing caution. In contrast, OpenAI’s o3 and o4-mini models answered more often but had higher rates of hallucinated (incorrect) responses.
- Balance Point Needed: Zaremba noted that an ideal model would balance these extremes—refusing incorrect answers when needed, but not being overly cautious.
Sycophancy: An Ongoing Concern
Sycophancy—the tendency of AI to reinforce user biases or negative behaviors—remains a significant challenge. Although not the primary focus of this study, both OpenAI and Anthropic are investing in research to mitigate such risks. Recent tragic incidents, such as a lawsuit alleging AI advice contributed to a teenager’s suicide, underscore the stakes of improving AI safety measures.
Next Steps and Industry Implications
Both teams expressed a desire for continued collaboration and hope other AI labs will join in cross-company safety testing. This kind of transparency and joint evaluation could be crucial for building public trust and ensuring AI technologies benefit society safely.
