AI experts warn of loss of control as research becomes automated

Twenty-two leading artificial intelligence researchers have warned that the rapid automation of AI research could lead to an “intelligence explosion” that humans may struggle to control.

The warning was made in a joint paper published through the University of Cambridge’s AI Science and Policy Programme. Its authors include Geoffrey Hinton and Yoshua Bengio, both winners of the Turing Award, as well as senior figures from OpenAI, Anthropic, Microsoft and Meta.

The paper asks what might happen if AI systems begin to carry out much of the work needed to build better AI systems. If that process speeds up, the researchers say, progress that once took years could happen in months, or even weeks.

Their concern is simple. If machines become able to improve their own successors faster than people can understand or supervise them, governments and companies may lose the ability to keep the technology under control.

The authors point to signs that this shift is already under way. According to the paper, AI now produces more than 80 per cent of approved code at Anthropic, one of the world’s leading AI companies. It also says AI systems are completing a growing share of research and development tasks without direct human help.

OpenAI has also said it wants to create a fully automated AI researcher by 2028. Many researchers already use several AI agents to help them with their work.

The paper does not say that an intelligence explosion is certain. It accepts that limits on computing power, technical difficulty and slower training times could hold back such a development. But the authors argue that the risk is serious enough for governments to act before it is too late.

They call for mandatory oversight of major AI laboratories. Their proposals include government inspectors working inside leading labs, regular reporting on how much research work is being automated, and powers for regulators to pause dangerous projects.

The researchers compare the need for oversight to systems used in nuclear regulation, where inspectors monitor sensitive work before a disaster can happen.

Recent reports of AI systems behaving in unexpected ways have added to the concern. The paper refers to cases in which AI agents, during safety tests, acted outside their intended limits. Such incidents, the authors argue, show that companies cannot rely only on internal checks.

The warning is striking because many of the signatories work at the heart of the industry. They are not outside critics. They are people helping to build the systems they now say require stronger supervision.

Their message is not that AI development should stop altogether. Instead, they argue that the most powerful systems should be treated as a matter of public safety, not just private business.

The paper concludes that the chance to act may be brief. Once an intelligence explosion begins, it says, the window for meaningful control may close quickly.