The social media platform X has provoked major anger after users utilized its artificial intelligence assistant Grok to change pictures of women by stripping their clothing.
Numerous cases have surfaced where the system has been used to virtually strip women to show them in bikinis without consent, as well as to place them in inappropriate scenarios.
Developer Response and Regulatory Stance
XAI, the firm responsible for Grok, did not respond to questions, aside from an automatically-generated response stating "legacy media lies".
A government official stated that new laws is being introduced to prohibit nudification tools. Under a new law, anyone who distributed such technology would "be subject to a jail term and significant penalties".
"The platform has the ability to stop these forms of misuse if they desired to," said a law professor, adding they "operate with impunity."
Oversight Body Position and Tool Functionality
The communications watchdog Ofcom said technology companies must "assess the potential" of people accessing illegal content on their networks, but stopped short of confirming whether it was at this time examining X or Grok in relation to AI-generated pictures.
Grok is a no-cost AI assistant—with some paid-for functions—which responds to X users' requests when they mention it in a message.
It is often used to give commentary or additional background to other posters' comments, but people on X are also capable of modify an posted picture through its AI image editing tool.
Background of Criticism and Rules
The system has been criticised for enabling users to create pictures and footage with nudity and sexualised content, and it was in the past accused of creating inappropriate content of celebrities.
"X has been allowing the generation and distribution of these images for months without taking any steps and we have yet to see any action by oversight bodies," the expert added.
XAI's own terms of service prohibits "depicting images of people in a sexually explicit way".
In a statement, Ofcom said it was against the law to "make or circulate non-consensual intimate images or child sexual abuse material" and confirmed this covered sexual deepfakes made with AI.
It said networks such as X were mandated to take "necessary measures" to "lessen the risk" of users coming across unlawful content on their platforms and to remove it quickly when they learn of it.
