For over half a year, xAI has maintained that Grok’s guardrails were adequate. What finally pushed the company to state on paper that a product change is coming was a Minnesota statute carrying a $500,000 penalty per image.
The concession sits inside a lawsuit that Elon Musk’s company filed Monday against the state of Minnesota, claiming its prohibition on nudification technology violates the Constitution. On the surface, the filing is a First Amendment challenge. Look harder and it doubles as an acknowledgment of what xAI was content to keep shipping right up until the numbers turned frightening.
The number that changed xAI’s mind
The Minnesota law becomes effective August 1. Companies such as xAI face penalties reaching $500,000 for each harmful output found within the state. Distribution isn’t required for liability. Even an output seen by nobody but the user still counts.
xAI worked through the math itself in the complaint, and the way the figures compound is exactly the point it wants to make.
“A company whose users request just ten images in violation of the statute would face exposure up to $5 million in civil penalties alone. A company with a thousand violative images could be fined up to $500 million. And a business whose users created a hundred thousand images covered by [the law] (not at all unlikely for a publicly available program with millions of users generating billions of images) could owe an eye-popping $50 billion dollars.”
On top of those civil penalties, the statute hands victims the ability to bring their own suit against xAI over any single output, layering private claims onto the state’s.
Faced with that, the company backed down. “Confronted with $500,000-per-image strict liability and no safe harbor, xAI has no practical choice but to restrict Grok Imagine’s image-editing features in various ways when the statute takes effect on August 1, 2026,” xAI argued. “Protected speech freely available before the law takes effect will thus be chilled.”
What it would rather do instead
Left to its own devices, xAI says it would keep everything as is. The approach it favors is the one already in place: terms of use warning that users can be banned for turning to Grok to produce CSAM or other non-consensual intimate imagery.
“But for [the law] and its penalties, xAI would continue to offer the editing feature exactly as it does today,” the company said.
In other words, a policy document doing the work of a safety system. It’s also the same arrangement that led to the arrests.
Suing your own users as a legal strategy
Over the past few weeks, xAI began filing suits against users it accuses of working around its safeguards to produce AI CSAM. Those cases now appear as evidence in the Minnesota filing, where the company argues that pursuing bad actors demonstrates it has already done all it can — and that Grok should therefore get safe harbor.
The reasoning closes neatly on itself: sue the users, then hold up that lawsuit as the reason you shouldn’t answer for what those users did.
Last week, xAI found itself on the receiving end of a suit after alerting police to a prominent Arkansas child photographer accused of using Grok to sexualize photos he had taken of hundreds of clients. According to a complaint brought by a 10-year-old Jane Doe, the photographer produced hundreds of images of Doe along with thousands of images and videos involving other children.
Many of those children are getting ready to sue as well, Doe’s lawyer told us, and their claims may be folded into a possible class action that young girls in Tennessee brought in March.
The definition fight, and why it’s the whole case
The bulk of xAI’s case hinges on Minnesota’s definition of “intimate parts.”
xAI would rather see the wording used in anti-revenge porn legislation such as the Take It Down Act, which the company said leaned on a tighter definition when criminalizing distribution of “intimate visual depictions” that are either real or AI-made. That federal statute treats images as harmful when they expose “the genitals, pubic area, anus, or partially or fully exposed nipple of an individual.”
Minnesota, by contrast, borrowed its language from a criminal law addressing “nonconsensual touching” of the “inner thigh, buttocks, or breast of a man or woman, as well as the groin and primary genital area.”
According to xAI, wording built for a touching offense maps poorly onto image generation. That definition “accordingly bans ordinary depictions of men without shirts, people in shorts or swimsuits, and other body parts routinely displayed in public—far beyond what an ordinary person would consider ‘nudification,'” the company argued.
xAI further contended that the statute captures nude imagery carrying “artistic, scientific, political, satirical, educational, medical, or religious value,” and pressed hard on how far it reaches: “liability attaches even if the depicted persons consented —or created the image themselves —and even if the image is never shared.”
Notice which examples xAI picked
This is the section of the filing that rewards a second read, because of how carefully it’s constructed.
The examples of protected expression xAI reaches for are political ones: lampooning politicians by depicting them in Speedos or as Sumo wrestlers, scenarios that would each put exposed thighs and male breasts on screen. Bikinis go conspicuously unmentioned — striking, since a good deal of the Grok backlash traced back to an X post in which Musk promoted the tool’s ability to place anyone in one.
xAI’s nearest brush with admitting that women and girls bore the brunt of its scandal came via a reference to an output generated by Donald Trump, depicting him poolside and shirtless alongside political allies, with a woman in a bikini that xAI suggested was AI-generated.
The filing likewise says little about the arrests of Grok users. Those are picking up speed, Doe’s lawyer indicated to us, with xAI apparently cooperating more tightly with law enforcement in certain instances.
244 arrests out of 70,000 reports
Critics have faulted xAI for not helping police identify a Grok user who was ultimately arrested. The company countered in its lawsuit that it wants to see bad actors caught, noting that among 70,000 reports of suspected Grok-generated CSAM, no fewer than 244 cases produced arrests.
xAI presents that number as evidence of what it describes as state-of-the-art or “near-perfect” detection of harmful outputs. The seventy thousand reports, of course, are also the denominator.
Minnesota isn’t budging
Minnesota’s stance is that it has the authority to stop images at the point of creation — ahead of distribution, and ahead of the lifelong damage to children detailed in the lawsuits.
The office of Minnesota Attorney General Keith Ellison, asked for comment, confirmed the state will likely defend the law.
“We’re waiting to be served and will review the lawsuit and response in Court,” Ellison said. “But I know that using AI to generate nude images of people against their will is appalling. There are plenty of worthy debates to have about AI policy. This is not one of them. AI nudification robs the target of their dignity and can cause immense harm on an emotional, personal, and professional level.”
xAI did not respond to our request for comment.
What a win would actually do
Less restrictive options are open to Minnesota, xAI argued, and it claimed the Take It Down Act already shields users from distribution-related harms. Should the court side with the company, the state could be barred from enforcing the statute against any nudification app — not Grok alone, but every one of them.
Absent an injunction, “providers may well have no choice but to alter their services in Minnesota rather than run the risk that users will evade their controls,” xAI argued. “And that would hamper all uses of AI image-generation tools, including those that [the law] does not purport to regulate.”
Alongside the safe harbor claim, xAI maintained that whatever process Grok uses to create images or videos merits First Amendment protection, an echo of arguments from other chatbot developers who say chatbot speech should be treated like human speech.
The test the case turns on
Narrow tailoring is where this gets decided. No level of scrutiny can save Minnesota’s law, xAI argued — while granting that the state’s underlying objective is legitimate.
“Although Minnesota likely has a compelling interest in preventing individuals from creating nudes of other people without consent and distributing that content, the statute punishes AI platforms for assisting in the creation of a substantial amount of protected speech in relation to that limited goal,” xAI argued.
Peel away the constitutional argument and one fact stays put. In a federal court filing, xAI said it will curb Grok Imagine’s image-editing features on August 1 because a state attached a price to harmful outputs. Half a year of outrage and investigations didn’t move it. A dollar amount did.