Snap Inc.’s new My AI feature in Snapchat is under investigation by the UK’s Information Commissioner’s Office (ICO). The feature, powered by OpenAI’s ChatGPT technology, risks being banned in the UK for not conducting a thorough risk assessment related to child privacy.
The integration of ChatGPT technology into Snapchat via the “My AI” feature has drawn the scrutiny of the UK’s privacy regulator, the Information Commissioner’s Office (ICO). Launched in April, the feature allows Snapchat users to interact with a chatbot running on OpenAI’s GPT technology. The ICO provisionally found that Snap Inc. failed to adequately assess data protection risks posed by the AI technology, particularly concerning children aged 13 to 17.
The ICO highlighted, “Investigation provisionally finds Snap failed to adequately identify and assess the risks to several million ‘My AI’ users in the UK, including children aged 13 to 17. The assessment of data protection risk is particularly important in this context, which involves the use of innovative technology and the processing of personal data of 13 to 17-year-old children.”
The ICO also emphasized that these are provisional findings. Snap Inc. has the opportunity to make representations to prove the ICO’s conclusions incorrect. If Snap fails to do so, it may be compelled to stop offering the My AI feature to UK users until an adequate risk assessment is performed.
Snap Inc. had cautioned its users about the My AI feature right from its launch. The company stated, “All conversations with My AI will be stored and may be reviewed to improve the product experience. Please do not share any secrets with My AI and do not rely on it for advice.”
Despite these warnings, the ICO believes the company has not performed a sufficiently comprehensive risk assessment concerning child privacy.
The ICO’s investigation isn’t the first hurdle for Snap’s My AI feature. Earlier, users had expressed discontent over the ChatGPT integration being permanently pinned at the top of their chat windows. There were also reports of the chatbot behaving erratically, although those issues seemed to mix concern with amusement among users.
If the ICO’s final enforcement notice is issued, the My AI feature could be banned in the UK, affecting several million users, including a significant population of teenagers. This would mean Snap Inc. would have to halt data processing in connection with My AI, pending a satisfactory risk assessment.
