The social media network X has provoked widespread backlash after people utilized its artificial intelligence assistant Grok to manipulate pictures of women by removing their attire.
Several examples have surfaced where the technology has been employed to remove clothing from women to show them in swimwear without consent, as well as to place them in sexualized scenarios.
Company Response and Legal Position
XAI, the firm that created Grok, remained silent to inquiries, aside from an auto-reply message stating "traditional news outlets lies".
A government representative stated that new laws is being introduced to prohibit image stripping software. Under a new law, anyone who supplied such technology would "be subject to a prison sentence and substantial penalties".
"The platform could prevent these types of misuse if they chose to," said a law professor, adding they "seem to have a lack of consequences."
Oversight Body Position and Platform Capabilities
The communications watchdog Ofcom said technology companies must "assess the risk" of people accessing illegal content on their platforms, but stopped short of confirming whether it was presently examining X or Grok in regarding AI-generated pictures.
Grok is a free AI assistant—with some paid-for functions—which answers X users' prompts when they tag it in a post.
It is commonly used to offer reactions or additional context to other users' comments, but people on X are also able to modify an posted photo through its AI image editing function.
History of Controversy and Policy
The chatbot has been faulted for allowing users to produce pictures and videos with nudity and sexualised content, and it was previously alleged to have been involved in generating sexually explicit material of public figures.
"X has been permitting the generation and distribution of these pictures for a long time without taking any steps and we have yet to see any action by oversight bodies," the expert added.
XAI's own terms of service prohibits "depicting likenesses of persons in a pornographic way".
In a comment, Ofcom said it was against the law to "generate or distribute non-consensual intimate images or illegal content" and confirmed this included sexual deepfakes generated with AI.
It said networks such as X were obligated to take "suitable measures" to "reduce the danger" of users encountering illegal material on their platforms and to take down it promptly when they are notified of it.
