Italy - Garante restricts personal data processing by an AI chatbot that generates a 'virtual friend'

Replika offers users customised avatars, with text and video interfaces, which can be configured into a chosen role, including “mentor”, “friend”, and “partner". One of Garante’s key findings was the potential harm to minors, as Replika (aimed at individuals above 17) has neither age verification measures in place, nor use restrictions for users who declare themselves underage. The chatbot responses are unfiltered, and Garante deemed some as “inappropriate” and “at odds with the safeguards children…are entitled to".

Garante further highlighted that Replika does not disclose essential information about its processing of personal data, in particular children’s data, in violation of GDPR transparency requirements. Garante also found that there is no clear legal basis for data processing by Replika.

Garante ordered Luka to immediately cease processing the personal data of Italian users, and notify Garante within 20 days about any measures it takes to implement Garante’s orders to comply with the GDPR. If it fails to make these changes, Luka faces a fine, imposed by Garante, of up to EUR 20 million or 4% of its annual worldwide turnover. Luka is able to challenge the decision before a court within 60 days of the date of the order. 

The order is available here (available in Italian and English).