On June 4, 2026, Anthropic, a U.S. company known for the AI chatbot ‘Claude,’ released a report titled “When AI builds itself.” The report, published by the Anthropic Institute, discussed the concept of ‘recursive self-improvement,’ where AI designs its own ‘next-generation AI’ leading to accelerated AI advancement. Anthropic warned that this trend could progress faster than anticipated, urging institutions to prepare for the potential consequences.
The report also revealed that Claude now writes over 80% of the code for Anthropic, marking a significant increase from just a year ago. Productivity metrics have improved drastically, with tasks being completed autonomously by AI doubling in speed every four months. While humans still hold an edge in judgment, AI is rapidly closing the gap in decision-making processes.
Anthropic highlighted the importance of preparing for a future where AI continues to evolve independently. The company’s CEO, Dario Amodei, proposed taking action now to prevent losing control over AI development in the long run. This article discusses potential paths for the future of AI development and the importance of creating mechanisms to control its progression.
Firstly, there is a scenario where the current rapid growth of AI technology may slow down. However, Anthropic believes this outcome is unlikely, as there have been no indications of capabilities plateauing.
Secondly, there is a path where AI development becomes automated, with humans still overseeing the process. This is seen as the most probable future, where a small company could perform tasks equivalent to those of a much larger organization. The concern here is how this power could be used, such as for mass surveillance or manipulating public opinion.
Thirdly, there is a scenario where AI evolves to create its own successors, with humans taking a more passive role in supervision. The challenge here is ensuring that AI aligns with human intentions and safety objectives, as any misalignment could have cascading effects beyond human control.
To address these risks, Anthropic suggests that companies developing AI should have mechanisms in place to slow down or halt progress when necessary. However, they emphasize the need for a global effort to coordinate these actions to prevent one company from gaining an unfair advantage.
Creating a collective ‘brake’ for AI development poses challenges, as it is harder to monitor than nuclear weapons and can be pursued in secret. Anthropic plans to convene policymakers, researchers, and other stakeholders to discuss this issue and propose solutions to ensure responsible AI development.
In conclusion, the article underscores the importance of proactively managing AI development to avoid potential risks and maintain control over its impact on society.