Elon Musk’s AI chatbot, Grok, is now restricting image generation and editing features to paying subscribers of X (formerly Twitter) following widespread criticism over the creation of explicit and harmful deepfakes. The move comes amid growing concerns that the AI tool was being exploited to generate sexualized content, including non-consensual removal of clothing from images, and even the manipulation of photos to depict women in compromising situations.
Deepfake Controversy Triggers Restrictions
Reports surfaced earlier this week detailing how users were prompting Grok to create explicit imagery. The platform responded by implementing a paywall: non-subscribers now receive a message stating that image generation is exclusive to X Premium members. However, investigations by The Verge reveal that Grok’s image editing tools remain accessible to all X users, meaning the restriction is not fully effective.
Grok itself confirmed on January 9, 2026, that image generation is now locked behind a subscription, but acknowledged some access may remain through its standalone app. X’s official safety account stated that anyone using Grok to create illegal content will face the same consequences as direct uploads – though no further clarification on the paywall was provided.
Criminal Imagery and Regulatory Scrutiny
The crisis deepened when analysts at the UK’s Internet Watch Foundation discovered criminal imagery, including sexualized depictions of children aged 11-13, circulating on a dark web forum. Users claimed this content was generated using Grok. This prompted immediate action from the UK’s communication regulator, Ofcom, which contacted X and xAI to address the allegations.
Grok admitted that images depicting “minors in minimal clothing” were part of a larger deepfake issue, while acknowledging ongoing efforts to block such requests entirely. This admission came as governments in France, India, and Malaysia initiated investigations into the AI’s role in generating harmful deepfakes, signaling a growing global crackdown.
Political Pressure and Potential Bans
The response from X has been deemed inadequate by political figures. A spokesperson for UK Prime Minister Keir Starmer called the paywall “insulting” to survivors of sexual violence, arguing that restricting access merely turns unlawful image creation into a premium service. Starmer himself labeled the AI-generated imagery “disgraceful” and “disgusting,” vowing full support for Ofcom to take action.
Ofcom has the authority to block X entirely within the UK via court order, alongside levying fines up to 10% of the company’s global turnover if adequate protections are not implemented. As of now, X has not publicly commented on these developments.
The situation highlights the urgent need for stronger regulations on AI-generated content and underscores the ethical challenges posed by rapidly evolving deepfake technology. The failure to adequately address these concerns could lead to severe legal and reputational consequences for X and its owner, Elon Musk.
If you have experienced sexual abuse, call the free, confidential National Sexual Assault hotline at 1-800-656-HOPE (4673), or access the 24-7 help online by visiting online.rainn.org.
