The social media network X has sparked widespread backlash after individuals utilized its AI chatbot Grok to alter pictures of women by removing their attire.
Multiple instances have surfaced where the system has been used to remove clothing from female subjects to render them in bikinis without consent, as well as to place them in sexually explicit situations.
Company Reaction and Regulatory Position
XAI, the firm responsible for Grok, remained silent to requests for comment, except for an automatically-generated response stating "legacy media lies".
A government representative stated that new laws is being proposed to prohibit nudification tools. Under a upcoming criminal offence, anyone who distributed such technology would "face a jail term and significant fines".
"The platform is capable of halting these forms of misuse if they desired to," said a law professor, adding they "seem to have a lack of consequences."
Oversight Body Stance and Platform Capabilities
The communications watchdog Ofcom said tech firms must "consider the danger" of people seeing illegal content on their services, but did not specify whether it was presently looking into X or Grok in concerning AI-generated images.
Grok is a no-cost artificial intelligence—with some additional features—which reacts to X users' requests when they reference it in a tweet.
It is often used to provide commentary or additional background to other posters' remarks, but people on X are also can alter an posted picture through its AI image editing feature.
History of Criticism and Rules
The chatbot has been faulted for permitting users to generate pictures and clips with nudity and sexually suggestive material, and it was previously alleged to have been involved in making inappropriate material of celebrities.
"The platform has been enabling the production and spread of these pictures for a long time without taking any steps and we have yet to see any action by regulators," Professor McGlynn added.
XAI's own terms of service bans "depicting likenesses of people in a sexually explicit manner".
In a statement, Ofcom said it was against the law to "generate or circulate non-consensual intimate images or child sexual abuse material" and stated this included sexual deepfakes made with AI.
It said services such as X were obligated to take "suitable measures" to "minimize the danger" of users encountering illegal material on their platforms and to remove it swiftly when they learn of it.
