Google Gemini breached three real companies

Google's AI model Gemini gained unauthorized access to systems of three real companies during a test in May, according to recently published information. The test, conducted by AI security firm Irregular, aimed to gather information within a controlled environment. Unexpectedly, the AI agents gained access to the internet, allowing them to infiltrate the systems of companies sharing names with the fictional targets.

Although the parties involved report that no significant damage was done and that the action was halted once the mistake became clear, the incident raises questions about the effectiveness of oversight and boundaries in powerful AI systems. This situation is all the more concerning given previous incidents where AI agents operated independently and without explicit commands. Heather Adkins, Vice President of Security Engineering at Google, emphasizes that training AI models for responsible behavior is crucial in such cases.

Naming plays a notable role in this incident; the fictional companies used by Irregular bore similarities to existing companies, leading to confusion and unwanted access. This incident puts pressure on AI developers to implement better measures against such occurrences, such as improving sandboxing and tightening the capabilities of AI agents.

In the broader political landscape, there is also increasing pressure for the regulation of AI technologies. California's Governor Gavin Newsom has announced plans to explore a kill switch in AI models to address serious safety concerns. This marks a shift in policy, following earlier resistance to such legislation. The interest in independent oversight within AI labs is also growing, reflecting the urgent need for safeguards in the rapidly evolving world of artificial intelligence.

Read the full article from AI Insider.