India warns X to remove vulgar AI content or face legal action

The government issued a stern notice to Elon Musk-led social media platform X to immediately remove all vulgar, obscene and unlawful content, especially generated by AI app Grok, or face action under the law. The Ministry of Electronics and IT (Meity) has issued a notice to the Chief Compliance Officer for India operations of X for failure to observe statutory due diligence obligations under the Information Technology Act, 2000, and IT Rules, 2021. "...X is hereby directed to...remove or disable access, without delay, to all content already generated or disseminated in violation of applicable laws, in strict compliance with the timelines prescribed under the IT Rules, 2021, without vitiating the evidence in any manner," the order dated January 2 said. Meity has also directed X to take action against offending content, users and accounts. The ministry has directed the US-based social media firm to submit a detailed action taken report (ATR) within 72 hours of the date when the order was issued.

The government has received from time to time, including through public discourse and representations from various parliamentary stakeholders, that certain categories of content circulating on X may not be in compliance with applicable laws relating to decency and obscenity, the order said. Earlier in the day, Rajya Sabha member Priyanka Chaturvedi had written a letter to Union Minister Ashwini Vaishnaw, seeking urgent intervention on increasing incidents of the AI app Grok being misused to create vulgar photos of women and post them on social media. The order said that "Grok AI" service, developed by X, is being misused by users to create fake accounts to host, generate, publish or share obscene images or videos of women in a derogatory or vulgar manner to indecently denigrate them. "Importantly, this is not limited to the creation of fake accounts but also targets women who host or publish their images or videos, through prompts, image manipulation and synthetic outputs. "Such conduct reflects a serious failure of platform-level safeguards and enforcement mechanisms, and amounts to gross misuse of artificial intelligence technologies in violation of applicable laws," according to the order.

The ministry has sought an action taken report towards immediate compliance for the prevention of hosting, generation, publication, transmission, sharing or uploading of obscene, nude, indecent and sexually explicit content through the misuse of Musk's startup xAI and its AI platform 'Grok'. "It is reiterated that non-compliance with the...requirements shall be viewed seriously and may result in strict legal consequences against your platform, its responsible officers and the users on the platform who violate the law, without any further notice, under the IT Act, the IT Rules, the BNSS, the BNS and other applicable laws," the order said. Meity has asked X to immediately undertake a comprehensive technical, procedural and governance-level review of "Grok", including its prompt-processing, output generation, image handling and safety guardrails, to ensure that the application does not generate, promote or facilitate content which "contains nudity, sexualisation, sexually explicit or otherwise unlawful content in any form". On December 29, Meity asked social media firms to immediately review their compliance framework and act against obscene and unlawful content on their platform, failing which they may face prosecution under the law of the land. After issuing an advisory, Meity observed that social media platforms have not been strictly acting on obscene, vulgar, inappropriate, and unlawful content.