Small Language Model

Small language models sit deliberately below the frontier. They are built to run on a laptop, a phone, or a modest server rather than a cluster, and they achieve that through smaller parameter counts, careful training data, and often distillation from a larger model.

The boundary is a moving one — what counts as small this year would have been large a few years ago, and the useful definition is practical rather than numeric: a model you can run where you need it, at a cost that permits high volume.

What changed the calculus is how good they became. A small model tuned for a specific task now routinely matches a frontier model on that task while costing a fraction as much per call and returning answers considerably faster.

Cost and latency, and they compound at volume. A feature that is uneconomic on a frontier model is often obviously viable on a small one, and the difference decides whether an AI feature ships. Local execution also resolves data-residency constraints outright — nothing leaves the device — which is why these models dominate on-device assistants.

Try the small model in a family before assuming you need the flagship, and route by difficulty rather than sending everything to the largest model available. Classification, extraction, routing and summarization are the jobs where small models most often prove sufficient.