What it means
At each step a model produces a probability distribution over possible next tokens. Temperature controls how that distribution is sampled. Low temperature makes the model favor the most likely token heavily, producing focused, repeatable, somewhat plain output. Higher temperature flattens the distribution, making less likely tokens more available — more varied and more surprising, and also more likely to go off the rails.
A temperature of zero is effectively deterministic: the model takes the top choice every time. Even then, output may not be bit-identical across runs, because floating-point arithmetic on parallel hardware is not perfectly reproducible.
Crucially, temperature does not control accuracy. Lowering it makes a model more consistent, not more correct — it will state a wrong thing more reliably.
Why it matters
Temperature is widely misunderstood as a truthfulness dial. It isn't. If a model doesn't know something, low temperature produces a consistently confident wrong answer, which is arguably worse than an inconsistent one, since consistency reads as reliability.
In practice
Low for extraction, classification, and anything you need repeatable. Higher for brainstorming and creative drafting. To reduce errors, add grounding, not a lower temperature.