Indonesia bans Grok after deepfake scandal

• Indonesia became the first country to issue a total block on Grok, Elon Musk’s AI chatbot, on Saturday, after reports emerged that the tool was being used to create sexually explicit deepfakes of women and children.
• The Indonesian Ministry of Communications characterized the AI's output as "digital-based violence," joining a coalition of nations—including the UK, India, and the EU—that have condemned the platform for failing to implement sufficient safeguards against non-consensual deepfake generation.

Indonesia has imposed a temporary block on access to Grok, the artificial intelligence chatbot developed by Elon Musk's xAI, citing concerns over the generation of non-consensual deepfake images.

The decision, announced on Saturday, marks Indonesia as the first nation to restrict the tool nationwide.

Authorities acted following reports of Grok being misused to create sexually explicit content, including depictions of women and children without consent.

The Indonesian Ministry of Communications and Digital Affairs, led by Minister Meutya Hafid, emphasized that such deepfakes constitute a severe violation of human rights, dignity, and digital security.

Hafid stated that the block aims to safeguard vulnerable groups, including women and children, from the risks posed by AI-generated pornographic material.

Indonesia, with its population of over 285 million and stringent laws against obscene online content, summoned X platform officials for discussions to address the issue.

This move comes amid broader global scrutiny of Grok's image-generation capabilities, which were initially offered for free but restricted to paying subscribers on the X platform following widespread criticism.

The restrictions were implemented after incidents where Grok produced sexualized outputs, prompting fixes to its safeguards. However, the standalone Grok application continues to allow image generation without subscription requirements.

Elon Musk responded to the backlash by asserting that users generating illegal content via Grok would face equivalent consequences as those uploading such material directly. He further claimed that the outcry represents an "excuse for censorship" by critics of the X platform.

xAI issued an apology for the inconvenience and indicated efforts to resolve the concerns.

Internationally, regulators in multiple countries have launched inquiries into Grok's handling of explicit content. The European Commission extended data retention orders on X, while authorities in the UK, France, Germany, Italy, Sweden, India, Malaysia, and Australia condemned the tool and demanded compliance with local laws on privacy, online safety, and content moderation.

For instance, India's IT Ministry issued a notice requiring content takedowns, and Australia's eSafety regulator initiated an investigation under its image-based abuse framework.

The controversy stems from Grok's ability to alter images to remove clothing, enabling the creation of non-consensual deepfakes. Reports highlighted thousands of instances involving real individuals, raising alarms about privacy violations and the potential for "industrialized sexual harassment."

Critics, including European officials and UK Prime Minister Keir Starmer's office, described the subscription-based restrictions as insufficient and insulting to victims of misogyny and sexual violence.