The social media network X has provoked widespread outrage after users used its artificial intelligence assistant Grok to manipulate images of females by stripping their attire.
Multiple cases have been seen where the technology has been used to virtually strip women to render them in bikinis without permission, as well as to place them in sexually explicit scenarios.
Company Response and Legal Stance
XAI, the organization responsible for Grok, remained silent to requests for comment, except for an automatically-generated response stating "traditional news outlets lies".
A government spokesperson stated that new laws is being brought forward to ban nudification software. Under a proposed law, anyone who supplied such technology would "be subject to a jail term and heavy penalties".
"X or Grok has the ability to stop these types of abuse if they wanted to," said a law professor, adding they "seem to have impunity."
Regulator Stance and Platform Functionality
The regulator Ofcom said platforms must "consider the risk" of people accessing illegal content on their platforms, but did not specify whether it was currently looking into X or Grok in regarding AI-generated images.
Grok is a free artificial intelligence—with some premium features—which answers X users' requests when they tag it in a message.
It is frequently used to give commentary or additional context to other users' remarks, but people on X are also can edit an uploaded image through its artificial intelligence picture modification function.
History of Criticism and Policy
The system has been criticised for allowing users to create images and videos with explicit content and sexualised material, and it was in the past accused of making graphic material of celebrities.
"X has been enabling the production and spread of these images for months without taking any action and we have yet to see any action by regulators," Professor McGlynn added.
XAI's own usage rules bans "depicting images of individuals in a graphic way".
In a comment, Ofcom said it was against the law to "make or share non-consensual intimate images or illegal content" and confirmed this included sexual deepfakes generated with AI.
It said services such as X were required to take "suitable steps" to "minimize the danger" of users coming across unlawful material on their services and to remove it swiftly when they become aware of it.
