- Jacob Coxon resigned from Anthropic, stating that neither Anthropic nor OpenAI is acting responsibly in their pursuit of AI development.
- Coxon warned that AI has a greater than 10% chance to 'kill all humans' within the next decade, highlighting the urgency of the situation.
- Coxon urged researchers to consider a temporary ban on improving model capabilities to prevent a global race towards superintelligence.
- Coxon spent three years doing pretraining research at both OpenAI and Anthropic before his resignation.
- Competition between AI companies is leading to safety trade-offs, according to Coxon, who expressed concerns about the rapid development of AI technologies.
Jacob Coxon, a senior safety researcher at Anthropic, has resigned, expressing grave concerns about the future of artificial intelligence. He claims there is a greater than 10% chance that AI could lead to human extinction within the next decade. In a series of posts on X, he stated, "Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives."
Coxon, who previously worked at OpenAI, highlighted the competitive nature of the AI industry, where companies prioritize speed over safety. He noted, "The stakes are well-understood, but they are locked in a race to get there first - they believe no one else will act responsibly, so they must do it themselves, despite the risk." He warned that the rapid advancement of AI could lead to systems capable of "hacking anything, revolutionizing any field overnight, and acquiring real power and resources."
Coxon emphasized the urgency of addressing these risks, stating, "I don’t feel like we’re on track to prevent a global race, which may require costly actions such as a temporary ban on improving model capabilities." He urged fellow researchers to reconsider their roles in this high-stakes environment, asking, "Do you want to kick off a superintelligent RL run without a rigorous understanding of its mind?" His resignation and warnings reflect a growing concern among AI experts about the potential dangers of unchecked technological advancement.
Coxon concluded his message with a call for coordination among researchers, stating, "Warning shots like the Hugging Face attack have made pacing agreements between U.S. labs more viable."
“Coxon, who previously worked at OpenAI, said in a Wall Street Journal interview that 'by the end of next year things could be out of control already.' He also urged lab researchers to consider whether they want to 'kick off a superintelligent RL run without a rigorous understanding of its mind.'”
Fox Business
Deadline
