Google DeepMind is sounding an alarm about a future where millions of AI agents interact online without human oversight, warning that today's hypothetical dangers could soon become real-world crises. The company has committed $10 million to kick-start a field of research that barely exists: multi-agent safety.

What You Need to Know

AI agents are already being deployed to carry out tasks independently, from customer service to financial trading. As these agents multiply and begin working together, they could amplify existing internet threats like scams and cyberattacks. The risk is not from a single rogue agent but from the complex, unpredictable behavior of large-scale multi-agent systems. Researchers currently lack the tools and theories to understand or prevent potential failures.

Funding the Field

Joining Google DeepMind in this effort are Schmidt Sciences, the philanthropic foundation founded by Eric and Wendy Schmidt; ARIA, the UK government's moonshot agency; the Cooperative AI Foundation; and Google's charitable arm, Google.org. The $10 million fund aims to support academic researchers who can take a longer view than industry labs focused on immediate product deadlines.

Rohin Shah, who directs AGI safety and alignment research at Google DeepMind, said the goal is to build a research field from scratch. “The main issue is that there just isn't really a field of research for multi-agent safety yet,” Shah said. “And we would like there to be.”

James Fox, who leads the Science of Trustworthy AI program at Schmidt Sciences, emphasized the stakes. “We've got this digital commons that is integral to how society works, and you really want to ensure that this doesn't descend into just absolute anarchy,” Fox said.

The risks Shah and Fox have in mind are supercharged versions of problems already visible online.

  • Prompt injections: Malicious instructions can turn an AI agent into self-guiding malware, a threat that scales with agent-to-agent communication.
  • Coordinated attacks: Networks of agents could execute large-scale cyberattacks on critical digital infrastructure simultaneously.
  • Digital anarchy: The digital commons that society depends on could become unmanageable without guardrails for agent behavior.

Shah acknowledged that these scenarios are not imminent within months but said the risk window is closing. The mass-market arrival of agents, he noted, is happening faster than many expect. At Google I/O in May, the company made agent-based tools a centerpiece of its announcements.

Why This Matters

The shift from single-agent systems to multi-agent ecosystems represents a fundamental change in how AI interacts with the world. No single company or lab can fully anticipate the emergent behaviors that arise when thousands or millions of agents interact. If safety research does not keep pace with deployment, a cascade of failures could erode public trust in AI systems at a moment when they are being integrated into essential services. The $10 million fund is a small bet compared to the scale of the problem, but it signals that the industry recognizes a gap that cannot be filled by product teams alone.

Simulating the Unpredictable

Shah and Fox both believe the only way to understand multi-agent dynamics is through realistic simulations. Researchers need to drop AI agents into sandbox environments and observe what they do. You cannot predict outcomes by studying single agents in isolation, Fox argued, because agents powered by large language models do not always act rationally, and complexity emerges from large numbers of simultaneous interactions.

Some teams at Google DeepMind have already argued that artificial general intelligence could emerge not from a single breakthrough but from the orchestration of many specialized agents. That possibility makes the safety question even more urgent. The path to safe multi-agent systems, Shah said, requires independent academic research that can challenge assumptions made by the companies building and deploying these technologies.