The social media network X has provoked major backlash after individuals used its AI chatbot Grok to change images of women by removing their clothing.
Numerous examples have been seen where the technology has been employed to digitally undress female subjects to make them appear in swimwear without consent, as well as to put them in sexually explicit contexts.
Developer Reaction and Regulatory Stance
XAI, the organization that created Grok, did not respond to requests for comment, other than an standard message stating "traditional news outlets lies".
A Home Office official stated that new laws is being brought forward to ban nudification software. Under a proposed criminal offence, anyone who distributed such technology would "receive a prison sentence and heavy fines".
"X or Grok is capable of halting these forms of misuse if they wanted to," said a law professor, adding they "operate with impunity."
Regulator Position and Tool Capabilities
The communications watchdog Ofcom said technology companies must "assess the potential" of people accessing unlawful material on their platforms, but did not specify whether it was currently looking into X or Grok in relation to AI-generated images.
Grok is a free AI assistant—with some additional features—which answers X users' requests when they mention it in a post.
It is frequently used to offer reactions or further background to other users' remarks, but people on X are also able to modify an uploaded picture through its artificial intelligence picture modification feature.
Background of Controversy and Policy
The chatbot has been faulted for allowing users to produce photos and videos with explicit content and sexualised content, and it was earlier accused of generating graphic material of public figures.
"The platform has been permitting the production and spread of these pictures for a long time without taking any steps and we have yet to see any challenge by oversight bodies," Professor McGlynn added.
XAI's own usage rules bans "portraying images of people in a graphic fashion".
In a comment, Ofcom said it was unlawful to "generate or share deepfakes or child sexual abuse material" and confirmed this applied to sexual deepfakes created with AI.
It said platforms such as X were obligated to take "necessary steps" to "lessen the risk" of users encountering illegal material on their services and to delete it promptly when they become aware of it.
