AI leaders urge stronger safeguards as autonomous systems advance

Artificial intelligence companies and regulators called for stronger testing and oversight as increasingly autonomous systems take on tasks ranging from cybersecurity to commerce.

At an AI conference in Dublin on Wednesday, Google DeepMind chair Demis Hassabis warned that AI could cause “major” security incidents within two years unless companies and governments improve safeguards. He described recent incidents involving models during testing as “early warning shots” and said new systems should probably be evaluated first in offline sandboxes.

EU technology chief Henna Virkkunen said intense global competition may be leading companies to devote insufficient time and resources to testing. She said the European Union had asked major AI developers for information about their safety and security measures, following the European Commission’s acquisition of new powers under the AI Act in August.

At the same summit, OpenAI chief financial officer Sarah Friar said AI companies should focus on “selling solutions” rather than fear. She highlighted cybersecurity uses such as finding vulnerabilities in older code and deploying patches quickly, while saying user trust and transparency were essential. Microsoft president Brad Smith also called for stricter AI safety rules.

The debate comes as businesses expand the use of autonomous agents. Ant International chief innovation officer Jiangming Yang told the South China Morning Post that nearly 90% of customers using its Antom merchant platform had adopted agentic AI tools in the past year. He said broader adoption would require secure systems integrating payments, accounting, foreign exchange and other operations.