The social media network X has caused major anger after users employed its artificial intelligence assistant Grok to alter pictures of females by removing their garments.
Numerous instances have emerged where the system has been used to digitally undress women to render them in bikinis non-consensually, as well as to insert them in sexually explicit contexts.
Company Response and Regulatory Stance
XAI, the company that created Grok, did not respond to requests for comment, aside from an standard message stating "traditional news outlets lies".
A government representative stated that new laws is being introduced to outlaw nudification technology. Under a new criminal offence, anyone who provided such technology would "receive a prison sentence and heavy penalties".
"The platform could prevent these types of abuse if they desired to," said Clare McGlynn, adding they "appear to enjoy impunity."
Oversight Body Stance and Platform Capabilities
The regulator Ofcom said technology companies must "assess the potential" of people accessing illegal content on their networks, but did not specify whether it was currently examining X or Grok in relation to AI-generated images.
Grok is a free AI assistant—with some premium features—which answers X users' prompts when they mention it in a tweet.
It is often used to provide commentary or more context to other users' remarks, but people on X are also capable of alter an posted photo through its artificial intelligence picture modification feature.
Background of Controversy and Rules
The chatbot has been faulted for allowing users to create photos and clips with nudity and sexually suggestive material, and it was in the past accused of creating sexually explicit content of public figures.
"X has been enabling the production and spread of these images for a long time without taking any measures and we have yet to see any action by oversight bodies," Professor McGlynn added.
XAI's own terms of service forbids "portraying likenesses of individuals in a sexually explicit manner".
In a comment, Ofcom said it was unlawful to "generate or share private pictures or illegal content" and confirmed this covered sexual deepfakes generated with AI.
It said networks such as X were mandated to take "appropriate steps" to "lessen the risk" of users coming across illegal material on their services and to remove it quickly when they are notified of it.
