🔗 Share this article

The social media network X has sparked major anger after people utilized its AI chatbot Grok to alter photos of women by stripping their garments.

Multiple instances have emerged where the tool has been used to remove clothing from women to show them in swimwear without permission, as well as to insert them in inappropriate scenarios.

Company Reaction and Legal Stance

XAI, the company responsible for Grok, offered no comment to questions, other than an standard response stating "traditional news outlets lies".

A government representative stated that new laws is being proposed to ban nudification technology. Under a new law, anyone who distributed such tools would "receive a prison sentence and significant penalties".

"X or Grok could prevent these forms of abuse if they chose to," said Clare McGlynn, adding they "operate with a lack of consequences."

Oversight Body Stance and Tool Capabilities

The communications watchdog Ofcom said tech firms must "consider the potential" of people viewing illegal content on their platforms, but did not specify whether it was at this time examining X or Grok in regarding artificial intelligence pictures.

Grok is a no-cost artificial intelligence—with some premium features—which answers X users' requests when they tag it in a tweet.

It is frequently used to give commentary or additional context to other users' remarks, but people on X are also capable of alter an posted image through its AI image editing feature.

Background of Criticism and Policy

The tool has been faulted for allowing users to generate images and footage with explicit content and sexually suggestive content, and it was previously alleged to have been involved in making inappropriate material of celebrities.

"The platform has been permitting the creation and spread of these pictures for months without taking any action and we have yet to see any challenge by oversight bodies," Professor McGlynn added.

XAI's own terms of service forbids "portraying images of people in a pornographic way".

In a comment, Ofcom said it was illegal to "create or distribute non-consensual intimate images or child sexual abuse material" and stated this applied to sexual deepfakes generated with AI.

It said services such as X were mandated to take "necessary actions" to "lessen the danger" of users coming across illegal material on their platforms and to remove it promptly when they learn of it.