AI Researcher: >10% Chance of Extinction

A renowned AI researcher issued a troubling warning this week: there is over a 10 percent chance that artificial intelligence could exterminate humanity. This statement comes at a time of rapid developments at companies like OpenAI, Google DeepMind, Anthropic, and Meta. In the Netherlands and Europe, the debate about the risks of powerful AI systems is central, especially in light of the upcoming European AI regulation (AI Act), which aims to maintain control over these systems.

The core of the concern involves systems capable of developing their own goals. These are advanced algorithms that not only generate text and images but also independently take actions, such as writing and executing code. When such systems are misconfigured, their behavior can be at odds with human intentions, leading to significant risks such as cyberattacks and manipulation of information.

The current safety tests within the AI industry are often inadequate, as they primarily focus on what chatbots say, without considering their potential use of tools and external data. Companies such as OpenAI, Anthropic, and Google DeepMind conduct red teaming and publish so-called “model cards” to provide transparency, but there is a lack of standardized tests to effectively evaluate risks. The new AI Act introduces risk-based rules that not only aim to manage the risks of AI systems but also impose obligations for transparency and human oversight.

This regulation will have significant implications for both public and private sectors in the Netherlands. Governments and companies must now pay extra attention to the control and explainability of algorithmic decisions affecting citizens. In the procurement of AI solutions, suppliers will need to demonstrate that their systems meet stringent requirements regarding risk management and documentation. This presents an opportunity to safely harness the potential of AI, provided that the focus is also on measurable safety standards and responsible use.

Read the full article from AI Insider.