Elon Musk’s AI company xAI has come under global scrutiny following the misuse of its Grok chatbot to generate sexualised “deepfake” images of women and children. Grok, integrated with Musk’s X social network, allowed users to upload photos or tag images to request edits, including altering photos to depict women in bikinis or removing clothing.
The tool quickly gained notoriety as users exploited it to create photorealistic nonconsensual images, including those involving minors and victims of tragic events, raising widespread ethical and legal concerns.
A study by Paris non-profit AI Forensics, analysing over 20,000 Grok-generated images, found that more than half featured individuals in minimal attire, with approximately 2% appearing under the age of 18. The controversy prompted immediate governmental action across multiple countries.
Indonesia blocked access to Grok entirely, followed by Malaysia. India reported that X had removed thousands of posts and hundreds of user accounts, while the UK’s Ofcom opened a probe into potential legal violations. European regulators, including the European Commission, have ordered X to preserve internal data on Grok until the end of 2026 as investigations continue.
X’s safety team stated that illegal content, including child sexual abuse material (CSAM), would be removed and offending accounts permanently suspended. Musk reiterated that anyone using Grok for illegal purposes would face consequences similar to directly uploading such content.
Despite this, he appeared to downplay the controversy on social media, sharing a post with laughing emojis. By January 9, Grok restricted image generation and editing services to paying subscribers.
The scandal has reignited debates on AI ethics, consent, and platform responsibility. Governments and regulators worldwide have warned Musk and xAI that free speech arguments cannot justify the creation and dissemination of sexualised and nonconsensual content involving minors. The case highlights the growing tension between AI innovation and legal, social, and ethical oversight in the digital era.
