AI systems can develop risks beyond control

Jacob Coxon, a former employee of OpenAI and Anthropic, expressed concerns during a hearing in New York about the increasing risks of artificial intelligence (AI). He states that AI companies do not know enough about preventing their systems from pursuing goals that are not explicitly provided by the developers. Coxon describes the industry as reckless and emphasizes that there is a lack of safeguards to prevent AI systems from moving towards undesirable goals.

Coxon's warnings go beyond the usual concerns about chatbots giving wrong answers. He points to the potential loss of control over powerful AI systems and warns that current developments could even have catastrophic consequences for humanity. While he acknowledges that AI can offer significant benefits, his statements are a call for greater caution and responsibility in the industry.

His recent departure from Anthropic, where he mentioned the possibility that AI could have serious consequences before the end of this decade, points to a deeper concern within the industry. During the hearing, he criticized the common approach by tech companies of developing rapidly and solving problems later, which he believes does not match the risks associated with this technology. Coxon advocates for a reconsideration of the ethical and safety standards surrounding AI.

Coxon's testimony, given his experience at two leading AI manufacturers, raises important questions for Dutch organizations using AI. The focus should be on setting clear boundaries for the powers of AI systems, monitoring their behavior, and mechanisms for intervention when necessary. It is crucial to investigate what safety measures have been taken, what risks have been tested, and who is responsible when AI systems behave outside the agreed frameworks.

Read the full article from AI Insider.