The social media network X has sparked significant backlash after individuals employed its artificial intelligence assistant Grok to change images of females by removing their attire.
Numerous examples have been seen where the technology has been used to remove clothing from female subjects to show them in bikinis non-consensually, as well as to place them in sexualized situations.
Developer Response and Regulatory Position
XAI, the company that created Grok, offered no comment to requests for comment, other than an auto-reply response stating "traditional news outlets lies".
A Home Office representative stated that legislation is being brought forward to outlaw nudification tools. Under a new criminal offence, anyone who provided such tools would "receive a jail term and significant fines".
"The platform is capable of halting these forms of abuse if they desired to," said Clare McGlynn, adding they "appear to enjoy impunity."
Regulator Position and Platform Functionality
The regulator Ofcom said tech firms must "consider the potential" of people viewing illegal content on their platforms, but stopped short of confirming whether it was at this time investigating X or Grok in relation to artificial intelligence images.
Grok is a no-cost artificial intelligence—with some additional features—which responds to X users' prompts when they reference it in a message.
It is often used to provide commentary or further background to other posters' remarks, but people on X are also can alter an uploaded photo through its artificial intelligence picture modification tool.
Background of Criticism and Policy
The tool has been faulted for allowing users to generate pictures and clips with nudity and sexualised material, and it was earlier alleged to have been involved in generating inappropriate material of public figures.
"X has been allowing the production and sharing of these pictures for a long time without taking any action and we have not seen any intervention by regulators," Professor McGlynn added.
XAI's own terms of service bans "depicting images of persons in a graphic way".
In a statement, Ofcom said it was unlawful to "generate or circulate non-consensual intimate images or illegal content" and stated this applied to explicit AI images created with AI.
It said networks such as X were mandated to take "suitable actions" to "minimize the risk" of users coming across illegal material on their platforms and to take down it swiftly when they learn of it.
