The UK communications regulator Ofcom launched a proper investigation into Elon Musk’s social media platform X concerning its AI chatbot, Grok following studies that Grok has been used to generate nonconsensual sexual deepfakes.
Leon Neal/Getty Pictures
disguise caption
toggle caption
Leon Neal/Getty Pictures
Indonesia and Malaysia briefly blocked X’s chatbot, Grok, over the weekend after it made scores of pretend photos publicly sexualizing largely girls and, in some situations, youngsters late final yr.
Governments all over the world are additionally launching investigations. The newest got here on Monday because the UK media regulator, Ofcom, launched a probe into the social media platform, which may end in a ban.
Grok had been producing sexually specific photos of individuals for a while. However the concern obtained widespread consideration in late December as folks used the chatbot to edit a excessive quantity of present photos by tagging the bot in feedback and giving it prompts corresponding to “put her in a bikini.” Whereas Grok didn’t reply to the entire requests, it obliged in lots of instances. In some instances, Bellingcat senior investigator and researcher Kolina Koltai famous, customers can get Grok to generate frontal nudes.
Untold numbers of girls and in some instances, youngsters, as Reuters first reported, have had their likenesses sexualized on-line by Grok with out their permission, together with one of many moms of X proprietor Elon Musk’s youngsters.
It is uncommon for thus many governments to take motion in opposition to a social media firm however this case is totally different, stated Riana Pfefferkorn, a coverage fellow at Stanford College. “Making child sexual abuse [material] is flagrantly illegal, pretty much everywhere on Earth.”
By final Friday, X had restricted Grok’s AI picture era function to make it solely obtainable to paying subscribers. Non-paying customers can nonetheless put folks in bikinis publicly with just some clicks, however they will solely put in a number of such requests earlier than being prompted to join a premium membership, which prices $8 a month.
NPR reviewed Grok’s publicly obtainable photos generated earlier this month and located it had stopped making photos of scantily clad girls a number of days into 2026. Nevertheless, it generally nonetheless gives up bikini-clad males.
xAI, the mother or father firm of X, has been pushing grownup content material with Grok since final yr. In Might, Koltai first famous that the chatbot would generate sexually specific photos in response to requests on X like “take off her clothes.” This previous summer time, Grok launched “spicy mode” in its standalone app, which allowed customers to place bikinis on AI-generated characters.
Ben Winters, director of AI and privateness on the advocacy group Client Federation of America, stated that Grok not solely permits modifying photos of actual folks, however it additionally gives a straightforward distribution platform by way of X. “It’s a further and significant escalation,” he stated.
Governments are outraged over X’s transfer to limit entry to the picture era perform to subscribers. British Know-how Secretary Liz Kendall advised Sky Information “it is insulting to say that you can still access this service if you pay for it.”
The Indonesian authorities discovered that Grok lacked efficient guardrails to cease customers from making nonconsensual pornographic content material primarily based on actual Indonesian residents, the Related Press first reported.
“The government sees nonconsensual sexual deepfakes as a serious violation of human rights, dignity and the safety of citizens in the digital space,” Indonesian Communication and Digital Affairs Minister Meutya Hafid stated in an announcement.
The AP additionally reported Grok will keep blocked in Malaysia till efficient guardrails in opposition to misuse are put in place.
In response to NPR’s questions, X spokesperson Victoria Gillespie pointed to a assertion posted on January 3 that stated “anyone using or prompting Grok to make illegal content will suffer the same consequences as if they upload illegal content.” The assertion echoed Musk’s personal publish earlier that day.
Such an strategy is an “attempt to abdicate responsibility, ” stated Winters.
“It certainly is not just the user that is prompting it alone,” he stated. “It is the fact that the image would not be created if not for … the tool they made.”
Earlier than the wave of Grok-generated nonconsensual specific deepfakes hit X, different AI makers had added related capabilities to their chatbots. In November, Google launched a brand new picture producing mannequin, Nano Banana Professional. In December, OpenAI up to date its mannequin ChatGPT Pictures. Each can even edit photos to place folks in bikinis, Wired reported final month. A thread on Reddit distributing such photos has been taken down.
In one other X publish, Musk steered that since instruments from different AI firms even have this undressing perform, the stress from governments is a type of censorship directed at his platform.
Koltai famous the development to make AI-generated nonconsensual intimate media has been within the making for the previous yr or two. “You’ll see these ads in Instagram even – it’s like, upload a photo of you and your crush and you guys can kiss,” she stated. “So that’s kissing and hugging, and then you see there’s that more extreme spectrum.”
“There is obviously a huge issue within the tech industry because we’ve seen this across multiple platforms, and [there is] not always great guidelines or boundaries or regulation,” stated Koltai.
The criticism within the U.S. has been way more muted than in different international locations. Sen. Ted Cruz, R-Tex., posted on X Wednesday that the photographs “should be taken down and guardrails should be put in place.” He additionally stated that he’s “encouraged that X has announced that they’re taking these violations seriously and working to remove any unlawful images and offending users from their platform.”
Grok has generated frequent controversy prior to now yr. Final summer time, the chatbot referred to itself as “MechaHitler” and spewed antisemitic conspiracy theories.
Apps with so-called nudifying capabilities have existed for years, largely within the shadows of the web. Winters stated officers have to do extra to police a few of X’s options. “There are misrepresentations about the safety of their products. There are violations of their terms of service,” he stated.
“We haven’t seen really any significant action from any U.S. agencies, whether it’s state or federal, that have the authority to enforce the law,” Winters stated.
