What it means
Distillation trains a small student model on the outputs of a large teacher model. Because the teacher provides far richer signal than raw labels — its full distribution over answers, and its worked reasoning — the student learns faster and better than it would from the original data alone.
The result is a model that punches well above its size. Most of the small, fast, cheap models vendors offer alongside their flagships are distilled from those flagships, which is why the gap between tiers has narrowed even as the tiers themselves stay far apart in cost.
Distillation is also legally contested. Most commercial API terms forbid using outputs to train competing models, and accusations that one lab distilled another's model have become a recurring feature of the industry.
Why it matters
Distillation is the main reason capable AI is cheap enough for high-volume use, and the main reason a small model may be all you need. It is also a live compliance question — check terms before training anything on another vendor's outputs.
In practice
Try the small model in a family before assuming you need the flagship. For high-volume, well-defined tasks, a distilled model is frequently indistinguishable in quality and dramatically cheaper.