AI leaders call for stricter oversight and regulation

Leaders from OpenAI, Google DeepMind, and Anthropic warned this week in the US and the UK that powerful AI systems could take over human control. They advocate for stricter oversight, rigorous safety testing, and clear procedures for curtailing new technologies. This call also has implications for Europe, where the new AI regulation will impact both governmental institutions and companies.

With the development of systems like GPT-4, Gemini, and Claude, these leaders stress the need for strict safety standards before such AI models are deployed. They call for detailed testing with so-called red teams, which focus on errors and potential abuse scenarios. They also emphasize the importance of emergency shut-off switches, to disable AI applications when they exhibit undesirable behavior.

As pressure mounts for transparency in training data and energy consumption, companies remain hesitant to share this information for fear of losing competitive advantage. This hampers independent evaluations and increases the demand for insight. The risks of generative AI, such as creating misleading content or writing vulnerable code, are already evident; therefore, the focus should not only be on individual errors but also on the potential for significant harm from the accumulation of small mistakes.

The European AI Act, which will be phased in by 2026, brings additional obligations especially for generic models. This legislation emphasizes risk assessment and ensures that applications with significant impacts on citizens must comply with heavier requirements for conformity and transparency. Researchers and regulators must also be guaranteed insight, with the aim of stimulating innovation while ensuring oversight.

In the Netherlands, the government is working on a distributed oversight system for the AI Act, where various regulatory bodies will be responsible for specific aspects. This will lead to stricter requirements for procurement and use of AI services in the public sector. Despite these prospects, however, many companies lag behind in transparency and independence in safety evaluations, keeping the risk of incidents persistent.

Read the full article from AI Insider.