Former UK Prime Minister
The endorsement follows an essay published by Amodei, which outlines a three-part strategy to manage the risks associated with increasingly powerful AI models. Both Sunak and Hassabis argued that the industry has reached a critical juncture where voluntary measures are no longer sufficient to ensure public safety and prevent the potential loss of human control over autonomous systems.
Industry Leaders Advocate for Stricter Safety Standards
Demis Hassabis, who leads Google DeepMind, described Amodei’s proposal as a necessary path forward for the sector. While noting that the specific implementation details require further refinement, Hassabis emphasized that the overall direction aligns with his own recent advocacy for an industry-wide standards body. He maintained that establishing common benchmarks for frontier AI is essential for navigating the current technological landscape.
Rishi Sunak, who currently serves as an advisor to Anthropic, echoed these sentiments while reflecting on his previous efforts to address AI risks during his tenure as Prime Minister. Sunak noted that the prospect of losing control over AI systems, which he had previously discussed at the 2023 Bletchley summit, has transitioned from a theoretical concern into a tangible reality. He warned that the industry may be approaching a point of recursive self-improvement faster than previously anticipated.
Government Oversight and Independent Verification
In his call for reform, Sunak urged governments, particularly in the United States, to adopt a more rigorous regulatory approach. He suggested that advanced AI should be managed with the same level of scrutiny applied to high-stakes fields like nuclear energy and advanced biology. According to the former Prime Minister, the era of relying solely on voluntary testing is over, and the current environment necessitates mandatory, independent checks to verify that research is conducted safely.
Sunak’s recommendations include the implementation of stronger safeguards to prevent the theft of advanced models and a more transparent assessment of how close companies are to achieving autonomous self-improvement. He concluded that when industry leaders themselves advocate for the introduction of speed limits, it is incumbent upon policymakers to heed those warnings.
Anthropic has already begun taking steps to address these concerns. The company announced it will provide third-party evaluators with permanent, employee-level access to its systems. This initiative is designed to allow external experts to monitor safety compliance, report potential incidents, and assess the alignment of AI models during the training process.
