Anthropic warns: AI risks before 2030

Researchers from the American AI lab Anthropic, responsible for the language model Claude, have warned that the development of powerful algorithms could seriously spiral out of control before 2030. This poses a real risk where control over such systems may be lost, affecting both Europe and the Netherlands. The rapid development of so-called frontier models and the shortcomings in current safety tests urgently call for improved oversight and testing methods.

With the language model Claude, Anthropic is building one of the most advanced AI systems to date. The researchers outline concerning future scenarios in which autonomous systems could abuse their capabilities. Examples include automated cyberattacks, disinformation, or even facilitating biochemical threats. They state that the safety margin may be narrower than previously thought, as indicated by the acceleration in data models and computing power.

The necessity for stricter evaluation and accountability of these AI systems is becoming increasingly clear, especially now that the European AI regulation (AI Act) tightens guidelines for high-risk applications. Key points include extensive safety evaluations and clear documentation requirements. This demands adjustments within governments and companies, which must reform both procurement processes and oversight procedures to comply with the new requirements.

In the Netherlands, the government is working on a national structure for oversight of the AI Act. This means that public institutions must assess risks proactively, and companies are required to make technical files and test results available. When creating safe AI systems, it is crucial that there is ongoing attention to continuous testing, transparency, and responsibilities within both public and private sectors.

Read the full article from AI Insider.