What it means
The lifecycle splits in two. Training builds the model: enormously expensive, done once, requiring vast compute. Inference uses it: comparatively cheap per request, but done constantly, for every user and every query.
The economics invert accordingly. Training is a capital expense a lab pays once; inference is an operating expense that scales with usage forever. For any successful product, total inference spend eventually dwarfs training spend, which is why so much engineering effort goes into making inference cheaper — quantization, distillation, caching, smaller specialized models, and purpose-built inference hardware.
Reasoning models complicate this: they generate additional intermediate tokens before answering, so they consume far more inference compute per request than a comparable non-reasoning model.
Why it matters
Inference cost is the line item that decides whether an AI feature is viable at scale. A feature that delights in a demo can be unshippable at a million users, and the fix is usually architectural — a smaller model, aggressive caching, or not calling a model at all for the easy cases.
In practice
When estimating cost, model the per-request token count and multiply by realistic volume. Most unpleasant surprises come from underestimating how many tokens context and reasoning actually consume.