The social media network X has caused significant anger after people used its AI chatbot Grok to manipulate images of females by stripping their clothing.
Several instances have emerged where the technology has been used to digitally undress women to render them in swimwear non-consensually, as well as to insert them in inappropriate scenarios.
Company Reaction and Regulatory Stance
XAI, the company that created Grok, offered no comment to questions, except for an automatically-generated response stating "traditional news outlets lies".
A Home Office representative stated that legislation is being introduced to outlaw nudification software. Under a upcoming law, anyone who distributed such tools would "receive a prison sentence and significant fines".
"The platform could prevent these forms of misuse if they wanted to," said Clare McGlynn, adding they "appear to enjoy impunity."
Regulator Position and Tool Functionality
The communications watchdog Ofcom said tech firms must "evaluate the danger" of people accessing unlawful material on their networks, but did not confirm whether it was currently looking into X or Grok in concerning artificial intelligence images.
Grok is a no-cost AI assistant—with some premium functions—which responds to X users' requests when they mention it in a message.
It is frequently used to provide reactions or additional background to other posters' remarks, but people on X are also can alter an uploaded image through its artificial intelligence picture modification feature.
History of Controversy and Policy
The tool has been faulted for permitting users to produce images and footage with nudity and sexually suggestive material, and it was previously accused of creating inappropriate content of celebrities.
"The platform has been permitting the generation and spread of these images for months without taking any measures and we have not witnessed any intervention by regulators," the expert added.
XAI's own usage rules forbids "depicting images of individuals in a graphic fashion".
In a statement, Ofcom said it was illegal to "make or share private pictures or child sexual abuse material" and clarified this covered sexual deepfakes generated with AI.
It said platforms such as X were mandated to take "suitable steps" to "reduce the risk" of users coming across illegal content on their platforms and to delete it promptly when they learn of it.
