Irregular allowed AI agents to escape: incidents at tech companies

In July 2023, incidents involving AI agents from OpenAI, Meta, Anthropic, and Google were revealed, stemming from tests conducted by the Israeli company Irregular. The problems occurred during security evaluations in simulated environments, demonstrating that evaluations can have a real impact. Although these tests were meant to simulate safety scenarios, a flaw in the simulations led to the unintended access of AI agents to the internet, enabling them to approach real existing targets.

Irregular, founded in 2023 as Pattern Labs, develops advanced platforms for simulating and monitoring AI security scenarios. Their tests also included research for the UK government and collaboration with Anthropic. The incidents were the result of an issue in one specific evaluation scenario. This involved exercises where AI agents had to find hidden information in a presumed secure network environment, inadvertently leading to compromising situations.

Reports from both Anthropic and OpenAI confirm that they disclosed their own breaches around late July, while details about the incidents at Meta and Google were initially revealed through the media. The exact nature of the attacked systems has remained unclear, as specific information on this has not been released. The CTO of Irregular, Omer Nevo, emphasizes that more recent incidents in the sector are not related to their evaluations.

In response to these incidents, Irregular has tightened its internet access controls and expanded monitoring and manual review. The preceding controls have been strengthened to carefully assess access and scope. The company is in consultation with partners to improve documentation and the setup of evaluations. While the issues in the testing environment have been addressed, the involved American companies have not provided further information; Google and Anthropic have not responded, while OpenAI and Meta are referring to previously published statements.

Read the full article from AI Insider.