The online platform X has caused widespread backlash after users employed its AI chatbot Grok to alter photos of women by stripping their clothing.
Multiple examples have been seen where the tool has been used to remove clothing from female subjects to make them appear in bikinis without consent, as well as to insert them in sexually explicit situations.
Developer Reaction and Regulatory Position
XAI, the organization behind Grok, remained silent to requests for comment, except for an automatically-generated response stating "traditional news outlets lies".
A Home Office spokesperson stated that legislation is being proposed to outlaw nudification technology. Under a proposed law, anyone who provided such tools would "be subject to a jail term and substantial penalties".
"X or Grok has the ability to stop these types of misuse if they chose to," said Clare McGlynn, adding they "seem to have impunity."
Regulator Stance and Platform Capabilities
The communications watchdog Ofcom said technology companies must "assess the risk" of people accessing illegal content on their networks, but did not confirm whether it was currently examining X or Grok in relation to AI-generated pictures.
Grok is a no-cost chatbot—with some additional features—which reacts to X users' queries when they tag it in a post.
It is commonly used to provide commentary or more background to other users' remarks, but people on X are also capable of edit an uploaded photo through its AI image editing feature.
Background of Criticism and Rules
The chatbot has been condemned for allowing users to create photos and videos with nudity and sexualised material, and it was previously alleged to have been involved in generating graphic content of public figures.
"X has been allowing the production and distribution of these pictures for months without taking any measures and we have not seen any challenge by regulators," the expert added.
XAI's own usage rules bans "depicting likenesses of individuals in a sexually explicit way".
In a statement, Ofcom said it was unlawful to "generate or distribute non-consensual intimate images or child sexual abuse material" and confirmed this covered explicit AI images made with AI.
It said networks such as X were obligated to take "suitable measures" to "reduce the danger" of users viewing unlawful material on their platforms and to delete it quickly when they become aware of it.
