68-year-old loses savings to AI (OpenAI) in Thailand

A 68-year-old man has lost all his savings in Thailand to a virtual girlfriend created by artificial intelligence. As a result, he ended up on the street. A Dutch citizen who noticed the situation helped the man with initial support, after which further care was arranged. This incident underscores the risks associated with relationship bots and paid features that can lead users into financially dangerous situations.

The man used a chat app in which he regularly paid for extra contact with his virtual partner. These extra features made the interaction more personal and intimate, leading to high expenses that became unaffordable. The specific app has not been disclosed, but the use of an AI-driven relationship bot is clear. Such systems can quickly build trust, which may prompt users to spend excessively.

Relationship bots, which operate with advanced language models, simulate human interaction by mimicking attention and understanding. Examples of such apps include Replika and Character.AI, which can be downloaded in the Netherlands. They are often based on a freemium model, where basic features are free but extra options must be paid for. This payment model can make older users, who may have less experience with in-app purchases, vulnerable to unexpected high costs.

The European AI regulation imposes increasingly strict requirements on the transparency of such systems. Users need to be aware that they are communicating with an AI system, and the GDPR also protects the data of EU citizens. This leads to strict rules surrounding data minimization and data security. Nevertheless, there remains a need for additional protection, especially for vulnerable users. Help is available for those who notice that a loved one is getting into financial trouble due to such apps, and consumer organizations offer support with misleading information or privacy issues.

The incident illustrates the need for more oversight on emotional AI systems and the protection of users for whom such technologies can be risky. While innovation in the AI sector is essential, clear boundaries must be established to prevent unnecessary risks.