India has directed Elon Musk’s X to immediately make technical and procedural changes to its chatbot Grok AI after the Ai tool generated “obscene” content, including AI-altered p*rn*graphic images of women, according to social media users’ complaints and public comments by lawmakers.
India’s IT ministry released an order Friday directing Musk’s X to remedy the situation with Grok AI by limiting creation of content featuring “n*dity, s*xualization, s*xually explicit or otherwise unlawful” material. The ministry also asked the social media service’s operator to provide, within 72 hours, an action-taken report for removal of such content and blockage of accounts that were uploading it or giving it wide circulation in India, according to a letter sent to Twitter.
The order, seen by TechCrunch, cautioned that not doing so can risk X’s “safe harbor” protections — a provision under Indian law that offers immunity from liability for user-generated content.
India’s action comes after users expressed concerns about Grok AI promoting the transformation of images of people — usually women — to look like they were wearing bikinis, in response to which an Indian parliamentarian Priyanka Chaturvedi lodged a formal complaint.
In a related development, new reports identified cases in which the AI chatbot was processing images of minors in s*xualized manners and X admitted earlier Friday that such incidents had occurred due to safety lapses. Those images were later removed.
But those created using Grok AI that showed women wearing bikinis through AI manipulation still appear to be accessible on X at the time of publication, according to TechCrunch.
The new order comes days after the Indian IT ministry on Monday issued a wider notice to social media firms, which was also reviewed by TechCrunch, reminding them that they are required to comply with local laws and that there won’t be any “compromise” when it comes to adherence of law, citing obscene and s*xually explicit content as a category that challenges water on existing regulations. The advisory recommended that companies reinforce internal protection mechanisms, while also warning that failure to do so could result in legal action under India’s IT and criminal laws.
“Once again, it is emphasized that failure to follow the above shall be met with serious action taking skills and consequences in respect of your platform’ its responsible officers as well as the users of such a platform which are violating the law, without another opportunity,” cautioned the order.
Failure to comply could result in action against X under the IT law and various provisions of the Indian Penal Code, said the Indian government.
India, one of the largest digital markets in the world, has become a key test case for how far governments are willing to go in imposing liability on platforms for A.I.-generated content. Any increase in law enforcement pressure within the country could have ripple effects for global technology companies that operate under the canopy of various jurisdictions.
The order comes as Musk’s X continues to mount a challenge in courts on multiple aspects of India’s content regulation rules, arguing that federal government takedown powers risk overreach — even though the platform has complied with most blocking orders. At the same time, over the past year Grok has also been employed by users 10 of X to fact-check and to offer real-time commentary on news events — that is, its outputs have become with some regularity more visible than those of stand-alone A.I.
X and xAI did not immediately respond to requests for comment on the Indian government’s directive.
