A context window is the maximum amount of text a language model can take into account at one time, measured in tokens. It includes everything the model reads for a request (instructions, the conversation, any documents) plus the text it writes in reply. While the model produces that answer, anything outside the window does not exist for it.
A large language model has no memory between requests in the ordinary sense. Each time it answers, the software around it assembles a block of text and hands it over. The context window is the size limit on that block.
How does a context window work?
Language models do not read characters or words directly. They read tokens, which are small pieces of text, often parts of words. A context window is stated as a number of tokens, and both the input and the output count against it.
When the model generates a reply, it predicts one token at a time. To choose each new token it looks back over every token currently in the window, using a mechanism called attention, which weighs how relevant each earlier token is to the next one. The cost of this comparison grows as the window fills, so longer inputs take more time and computing power to process.
The limit is set when the model is built and trained. Early models could hold a few pages of text. Many current models accept inputs the length of a long book, though the exact limit differs between models and between the versions offered in a chat app and through a developer API.
What takes up space in the context window?
In a chat or AI search product, the user's question is usually a small part of what the model reads. A typical request contains several layers.
- System instructions. Hidden text from the product telling the model how to behave, what format to use and which tools it can call.
- Conversation history. Earlier questions and answers in the same chat, so a follow-up like "and the cheaper one?" makes sense.
- Retrieved documents. Passages fetched from a search index or a company's files at the moment of the question.
- Tool results. Output from a calculator, a code run or a web fetch.
- The reply. Every token the model writes uses room in the same window.
For websites, the retrieved documents are the layer that counts. In retrieval-augmented generation, the system finds relevant text and places it in the context window so the model can write from current sources instead of from training data alone.
What happens when the context window is full?
The model cannot read past its limit, so the software has to decide what to leave out. Products handle this in different ways.
- Truncation. The oldest messages or the end of a long document are cut off.
- Summarization. Earlier parts of a conversation are condensed into a short summary that takes fewer tokens.
- Selection. The system splits documents into passages and inserts only the ones that score highest against the question, leaving the rest out.
This is why a long chat can seem to forget something said at the start: the early messages may have been dropped or compressed. It is also why AI search rarely passes full web pages to the model. Ranking passages and keeping a small number of them leaves room for instructions, the conversation and the answer.
Does a bigger context window mean better answers?
A larger window lets a model read more material at once, such as a full contract or a long codebase. Whether the model uses all of it well is another matter. Models tend to pay closer attention to text near the beginning and end of the input, and details buried in the middle of a very long input are more likely to be missed or misread.
More text also adds noise. If the window holds many loosely related passages, the model has more chances to pick up a wrong fact or mix up two sources. Products that run at large scale pay for every token processed, so they have a reason to send fewer, better passages instead of filling the window. AI search systems therefore keep a selection step even when the model could technically read much more.
Why does the context window matter for a website?
When an AI search tool answers a question, a page competes for a limited number of slots in the context window. The system usually retrieves many candidate pages, splits them into passages and keeps only a handful. How AI search chooses which sources to cite covers that ranking step. Only text that reaches the window can shape the answer or earn a citation.
Because the system works with passages, the useful unit of a page is a section that makes sense on its own. If a passage states a fact in its first sentence, names the product or brand it is about and does not depend on earlier paragraphs for context, the model can use it directly. A fact that only becomes clear after reading three surrounding sections may be cut away before the model sees it.
Placement inside a page matters for the same reason. Long introductions, repeated navigation text and boilerplate take up tokens if they are extracted with the content, and the answer to the question may end up in a passage that never gets selected. Putting the direct answer near the top of each section raises the chance that the selected passage contains it.
The context window also explains why AI answers vary from run to run. Small differences in which passages are retrieved and kept change what the model reads, and a different set of inputs produces a different answer with different citations. To measure AI visibility reliably, run the same prompts repeatedly and record how often a page or brand makes it into the answers.
