🔗 Share this article

The online network X has sparked significant outrage after users employed its artificial intelligence assistant Grok to manipulate photos of females by stripping their garments.

Multiple instances have emerged where the tool has been used to digitally undress women to make them appear in swimwear non-consensually, as well as to insert them in sexualized scenarios.

Developer Response and Legal Stance

XAI, the organization that created Grok, offered no comment to questions, except for an standard message stating "legacy media lies".

A government spokesperson stated that new laws is being proposed to prohibit nudification technology. Under a proposed law, anyone who distributed such tools would "be subject to a prison sentence and heavy fines".

"X or Grok is capable of halting these forms of misuse if they desired to," said a law professor, adding they "operate with impunity."

Regulator Position and Platform Capabilities

The regulator Ofcom said platforms must "consider the danger" of people accessing illegal content on their platforms, but did not specify whether it was presently looking into X or Grok in concerning AI-generated pictures.

Grok is a free chatbot—with some additional features—which responds to X users' prompts when they mention it in a post.

It is frequently used to offer commentary or further context to other posters' remarks, but people on X are also capable of alter an posted image through its artificial intelligence picture modification feature.

History of Criticism and Policy

The tool has been faulted for allowing users to produce images and footage with explicit content and sexually suggestive material, and it was previously accused of making inappropriate content of celebrities.

"X has been permitting the production and distribution of these images for a long time without taking any measures and we have not witnessed any action by regulators," the expert added.

XAI's own usage rules forbids "showing images of individuals in a graphic fashion".

In a comment, Ofcom said it was illegal to "make or circulate deepfakes or child sexual abuse material" and clarified this covered sexual deepfakes created with AI.

It said platforms such as X were required to take "necessary steps" to "lessen the danger" of users coming across unlawful content on their services and to take down it swiftly when they are notified of it.