Low-Rank Adaptation

Full fine-tuning updates every parameter, which means the compute and memory of training a whole model and a complete copy of it per task. LoRA avoids that by freezing the original model and training a small set of additional parameters alongside it — typically a fraction of a percent of the total.

The results are close enough to full fine-tuning for most purposes, at a small fraction of the cost, and the artifact is small enough to be distributed easily. It is also composable: one base model can be served with many swappable adapters, which is how platforms offer per-customer customization without hosting a separate model each.

This is the standard approach to fine-tuning open-weight models today, and the reason a hobbyist with one consumer GPU can fine-tune at all.

LoRA is what made fine-tuning accessible outside well-funded labs. If you evaluated fine-tuning years ago and concluded it was out of reach, the economics have changed by orders of magnitude.

For adapting an open-weight model to a house style or a narrow task, LoRA is the default starting point. It still teaches behavior rather than facts — for facts, use retrieval.