Skip to content

Italy - Garante restricts personal data processing by an AI chatbot that generates a 'virtual friend'

The Italian supervisory authority (Garante) issued an urgent order against Luka Inc. (Luka), a US-based developer and operator of the online app “Replika” (Replika), an artificial intelligence (AI) chatbot on 2 February 2023. Garante placed a temporary restriction on Luka’s processing of the personal data of Replika’s users in Italy with immediate effect. Garante’s intervention followed a series of press reports, which showed the app to pose a risk to minors and emotionally vulnerable individuals. 

Replika offers users customised avatars, with text and video interfaces, which can be configured into a chosen role, including “mentor”, “friend”, and “partner". One of Garante’s key findings was the potential harm to minors, as Replika (aimed at individuals above 17) has neither age verification measures in place, nor use restrictions for users who declare themselves underage. The chatbot responses are unfiltered, and Garante deemed some as “inappropriate” and “at odds with the safeguards children…are entitled to".

Garante further highlighted that Replika does not disclose essential information about its processing of personal data, in particular children’s data, in violation of GDPR transparency requirements. Garante also found that there is no clear legal basis for data processing by Replika.

Garante ordered Luka to immediately cease processing the personal data of Italian users, and notify Garante within 20 days about any measures it takes to implement Garante’s orders to comply with the GDPR. If it fails to make these changes, Luka faces a fine, imposed by Garante, of up to EUR 20 million or 4% of its annual worldwide turnover. Luka is able to challenge the decision before a court within 60 days of the date of the order. 

The order is available here (available in Italian and English).

Related expertise