How does a context window work?#
When you give an AI a prompt, like a question or a document, it breaks the text down into smaller pieces called "tokens" (a token is roughly a word or part of a word). The context window is the maximum number of these tokens the model can hold in its active memory for a single task. All the information within this window—your questions, its previous answers, and any documents you've provided—is considered simultaneously to generate the next response.
If the amount of text exceeds the context window's limit, the model effectively forgets the earliest information provided in that specific session. This is why models with larger context windows are better at tasks involving long documents or extended conversations, as they can maintain a complete picture of the task without losing critical details.