AI model distillation, explained: the cheap way to inherit an expensive model’s brain

Building a capable AI model on the cheap comes down to one trick: get somebody else’s costly system to do the teaching for you. That single sentence contains the entire dispute.

The argument went public in January of last year, when the Chinese AI startup DeepSeek shipped R1, its first model. On several key benchmarks, R1 came close to frontier-level performance. The bigger talking point was the rumor attached to it — that it had cost a fraction of what the largest U.S. labs had poured into their flagship systems.

Soon after that launch, OpenAI alleged that DeepSeek had distilled the model powering ChatGPT in order to train its own. Just over a year later, Anthropic leveled the same charge at DeepSeek, plus the Chinese labs Moonshot and MiniMax, this time involving Claude. Then in June, Anthropic said Chinese tech giant Alibaba had used Claude for distillation “illicitly.”

Note the choice of word. Illicitly — not illegally.

No legal framework yet — plenty of politics

Distillation sits in a legal grey zone, which describes most of the current AI industry. Even so, the Trump administration has pledged to clamp down on Chinese labs distilling American-made models, casting the U.S.–China AI contest as a geopolitical matter of the highest order.

“AI distillation, when legitimately used to produce smaller, lighter-weight models from more advanced systems, is a vital part of [the AI] ecosystem,” Michael Kratsios, who directs the White House Office of Science and Technology Policy, wrote in a memo in April. “Industrial distillation activities that aim to systematically undermine American research and development and access proprietary information, however, are unacceptable.”

In other words, the White House isn’t condemning the method. It is drawing boundaries around who uses it, on what scale, and against whom.

Where the term comes from

In its original sense, distillation is an alchemical procedure for extracting a small quantity of something valuable from a much larger mass of material that isn’t. Heat a fermented liquid, catch the ethanol vapor, let it cool. What is left is pure alcohol, separated out, extracted and sold.

Model distillation — also called knowledge distillation — works along much the same lines. Rather than laboriously generating knowledge from nothing, it pulls valuable training data out of a bigger, already-pretrained model.

That laborious part is where the money goes. Normally, a model must work through a lengthy trial-and-error process to accumulate facts that a human child simply takes for granted: distinguishing a cat from a dog, knowing that 2+2 always comes out to four, understanding that “tree” names one of those tall leafy things outdoors. Carried out the usual way, that learning stage devours enormous amounts of compute.

Firms have been prepared to spend the GDP of a small nation on it, on the widespread assumption that this is the most reliable route to artificial general intelligence. Whether that assumption survives contact with reality remains a multibillion-dollar open question.

Teacher, student, and the bit that usually gets lost

Distillation puts a smaller “student” model alongside a flagship “teacher.” Over the course of many exchanges, the student learns to anticipate how the teacher will respond, shaping itself in the teacher’s image. Learning still happens — it is simply a far cheaper, more streamlined version of what the teacher went through.

Here is the nuance most coverage flattens. A good teacher model doesn’t reply in yes-or-no fashion. It exposes its reasoning.

Imagine a teacher model that learned to recognize motorcycles by working through millions of images scraped from social platforms and other online databases, assisted by humans completing millions of CAPTCHAs that train it to distinguish motorcycles from lookalike objects. The student isn’t merely observing the teacher stamp “Yes” or “No” onto pictures. It observes the teacher reason its way there.

The teacher may land on an 85% likelihood that the object in an image is a motorcycle, a 12% chance it’s an e-bike, 2.7% that it’s a donkey and 0.3% that it’s Mr. Snuffleupagus. Those figures are known as soft labels, and they are what develops the student’s own reasoning and statistical prediction abilities.

Two comparisons worth keeping

Picture a flagship model as a born-and-raised New Yorker who learned the city through endless wandering and wrong turns. The distilled model is the friend in town for the first time, tagging along behind her and scribbling a rough map as they go.

Or consider the Wright Brothers, whose first flying machine got off the ground in 1903 following numerous failures and a long line of earlier innovators who attempted the same feat and fell short. The moment they pulled it off, everyone else had a model to build from. The difficult learning had already happened.

That is why most experts will say there is nothing inherently immoral about model distillation. Scientific progress rests on building upon what others have discovered. “If I have seen further,” in Isaac Newton’s phrasing, “it is by standing on the shoulders of giants.” There’s even an argument that distillation does net good, because it hands the capacity to build powerful AI to people who would never be able to finance a flagship of their own.

The safety layer often stays behind

Geopolitics is not the only concern in play, and arguably not the gravest one.

AI remains an inexact science. Train a flagship model on something close to the whole of the internet and it absorbs harmless pattern-recognition rules — sentence structure, say, or how to identify a cat. It also absorbs deeply embedded human biases. Given sufficient compute, models can gradually be trained to recognize those biases and avoid them. They can likewise be fine-tuned to detect when a user is after something dangerous — the textbook case being a bioweapon recipe — and decline.

Those protections don’t reliably carry over through distillation. A student model typically absorbs foundational behaviors while skipping the deeper stratum of safety knowledge the teacher draws on to judge when a request should be refused. Alignment goes missing in transit.

Bad habits stack up

Bias leaks the same way. Even attentive parents transmit habits they never intended to teach, and a student model can absorb distorted views of the world from its teacher with nobody spotting it.

That student is then put to work training yet another model. The original bias passes down a further generation, and another after that — drifting ever further from the flagship that created it, and from anyone in a position to audit its origin.

So when you’re sizing up a small, inexpensive model that outperforms its weight class, the question worth asking isn’t how it scores on benchmarks. It’s whose knowledge it inherited — and what it quietly failed to inherit alongside it.