‘Pacing’ won’t eliminate the risk of AI doom. Here’s what could | David Krueger

THE NEWS

What happened

AI safety expert David Krueger argues that the proposed 'pacing' strategy by Anthropic CEO Dario Amodei is insufficient to prevent existential risks. Instead of a temporary slowdown, Krueger advocates for an immediate, indefinite international moratorium on frontier AI development, suggesting the decommissioning of advanced chips and supply chains as a verifiable method to halt the race between the US and China.

CONTEXT

Why it matters

The debate over AI safety has reached a critical point. While industry leaders like Dario Amodei propose a temporary 'pacing' slowdown, experts like David Krueger argue this is not enough. The core issue remains: we do not know how to control systems smarter than us or prevent them from deceiving us. Krueger proposes a radical solution: an indefinite international moratorium on frontier AI development. This would involve decommissioning advanced chips and dismantling the lithography supply chain needed to build more powerful models. With geopolitical tensions high between the US and China, a verified pause could be the only way to ensure humanity's survival while we solve these fundamental safety problems.

AT A GLANCE

Key facts

• Anthropic CEO Dario Amodei has proposed a one-to-two-year slowdown in AI development endorsed by Sam Altman and Elon Musk.
• David Krueger, an AI professor at the University of Montreal and founder of Evitable, criticizes the 'pacing' plan as inadequate for ensuring safety.
• Krueger argues that current AI systems cannot be trusted due to unknown risks regarding control, misbehavior, and deceptive capabilities.
• The proposed alternative is an indefinite international moratorium involving the decommissioning of advanced computer chips and data centers.
• Verification could be achieved by dismantling the supply chain for EUV and DUV lithography required for frontier AI models.
• Current safety standards are described as 'safe-until-proven dangerous,' falling short of industries like aviation.
• The article suggests a potential diplomatic opportunity involving US President Trump and Chinese President Xi Jinping to negotiate a halt to the AI arms race.