Context Window

Everything a model uses to produce an answer has to fit in its context window: your question, the conversation so far, any documents you pasted, the system instructions, and the answer being generated. It is measured in tokens rather than words. When a conversation exceeds the window, something has to be dropped or summarized, which is why long chats sometimes seem to forget earlier details.

Context windows have grown enormously, and vendors compete on the number. But capacity is not the same as attention — models reliably use information at the beginning and end of a long context better than material buried in the middle, an effect often called "lost in the middle". A large window makes something possible; it does not guarantee the model will use all of it well.

Context is the main lever most people have over output quality, and the main constraint. It sets how much of a document you can analyze in one pass, how long an agent can run before losing the thread, and how much a request costs — since you pay per token, a large context is a real expense on every call.

If quality degrades on a long document, the fix is usually not a bigger model but less context: extract the relevant sections and pass those. Putting the most important material at the start or end of the prompt measurably helps.