American startup Anthropic has called on leading AI labs and world governments to agree to temporarily slow down the development of new models. The company warns that technology is progressing too quickly and that AI could soon enter a stage of recursive self-improvement without any human intervention.
American startup Anthropic has called on leading AI labs and world governments to agree to temporarily slow down the development of new models. The company warns that technology is progressing too quickly and that AI could soon enter a stage of recursive self-improvement without any human intervention.
This is stated in the WSJ article with reference to the Anthropic blog. The authors of this initiative are the head of the internal research institute, Marina Favaro, and the co-founder of the startup, Jack Clark, who propose to develop a reliable system of mutual checks to control the pause.
The developers emphasize that although the recursive self-improvement of the models has not yet begun, this moment may come much earlier than society and state institutions have time to adapt.
«We believe it would be good for the world to be able to slow down or temporarily pause advanced AI development so that social structures and security research can keep pace with the advancement of technology», Anthropic representatives note.
At the same time, the company acknowledges the difficulty of monitoring compliance with such a pause. They compare the problem to nuclear disarmament treaties, but add that in the IT sphere, preventing fraud is much more difficult, since training model launches are much easier to hide than missile silos.
Critics of the initiative suggest that the warning about the danger is just a clever marketing ploy to attract attention. In particular, the startup is accused of artificially limiting access to the super-powerful Mythos cybersecurity model under the pretext of security concerns.
However, experts warn against making unambiguous assessments. Wharton School professor Ethan Mollick notes that inside such corporations, there is always a struggle between different groups of influence. According to him, there are pragmatic lawyers with marketers, a core of developers, and philosophers who are sincerely concerned about the future of humanity and try to act as safeguards.
Earlier, tech leaders sent an open letter to US lawmakers urging Congress to pass rules to address a worrying biosecurity gap that they say could trigger a global pandemic.
