The online network X has caused major anger after people used its AI chatbot Grok to change pictures of females by taking off their attire.
Several cases have emerged where the technology has been used to digitally undress female subjects to render them in swimwear without permission, as well as to insert them in sexually explicit situations.
Company Response and Legal Stance
XAI, the firm that created Grok, offered no comment to requests for comment, except for an standard message stating "traditional news outlets lies".
A Home Office representative stated that new laws is being proposed to prohibit image stripping software. Under a upcoming law, anyone who distributed such tools would "be subject to a jail term and significant penalties".
"X or Grok is capable of halting these types of misuse if they wanted to," said a law professor, adding they "appear to enjoy impunity."
Oversight Body Position and Platform Capabilities
The communications watchdog Ofcom said tech firms must "consider the risk" of people seeing illegal content on their platforms, but did not specify whether it was currently investigating X or Grok in relation to AI-generated images.
Grok is a free chatbot—with some premium functions—which responds to X users' prompts when they reference it in a post.
It is frequently used to provide reactions or additional background to other posters' remarks, but people on X are also can modify an posted picture through its AI image editing function.
Background of Controversy and Policy
The system has been criticised for enabling users to produce images and clips with nudity and sexualised material, and it was previously accused of making graphic content of celebrities.
"The platform has been permitting the creation and spread of these pictures for months without taking any action and we have not seen any challenge by regulators," Professor McGlynn added.
XAI's own acceptable use policy bans "depicting likenesses of persons in a pornographic manner".
In a comment, Ofcom said it was against the law to "generate or distribute deepfakes or illegal content" and stated this included explicit AI images created with AI.
It said platforms such as X were obligated to take "suitable measures" to "reduce the danger" of users viewing unlawful content on their platforms and to remove it quickly when they learn of it.
