Former Anthropic researcher Jacob Coxon has warned that humanity could lose control of advanced artificial intelligence if the technology continues to develop without stronger safety measures.
Coxon gave the warning on Monday while testifying before the New York City Council during a hearing on the risks and regulation of artificial intelligence.
He told lawmakers that, on the current path, he believes it is more likely than not that humanity could lose control of advanced AI systems, potentially leading to human extinction.
Coxon also accused leading AI companies of being “extremely reckless given the stakes” and criticised what he described as a “move fast, break things” culture in the industry.
He said that approach might work for a social media or photo-sharing application, but not for what he called “the most powerful technology ever built.”
Coxon, who worked at both Anthropic and OpenAI, resigned from Anthropic last month over concerns about the pace and direction of AI development.
He has previously argued that major AI companies are focused too heavily on developing increasingly powerful systems and competing with each other, while safety measures are not advancing quickly enough.
At Monday’s hearing, he called for greater transparency, independent oversight and stronger safeguards, including slowing the development of frontier AI until researchers are confident that the systems can remain under human control.
Representatives from OpenAI, Anthropic, Google and Meta also appeared before the Council. The companies defended their commitment to AI safety while expressing openness to some forms of regulation.
The New York City Council is considering proposals that would require greater disclosure about AI systems, third-party approval or validation of certain systems, and stronger protections for whistleblowers.
The hearing comes amid growing concern among governments and AI researchers about the risks associated with increasingly capable AI systems.
Comments
Add a comment