In short. A long context window allows more input, while measured use of that input varies with passage position and task.
Definition: No. A context window describes the input span a language model can accept. In controlled long-context tasks, answer accuracy varies with the position of relevant information and with the work required to use it.
Key facts
- Liu and colleagues' Lost in the Middle: How Language Models Use Long Contexts, published in 2024, finds that performance is often highest with relevant information near the beginning or end of the input.
- The 2024 RULER study evaluates 17 models on 13 tasks. Almost all show substantial performance declines as context length increases, despite nearly perfect accuracy on a simple retrieval test.
- LongBench, published in 2024, includes 21 datasets across six task categories, including question answering, summarization and code completion.
What changes when a passage moves?
In Lost in the Middle, the question-answering task supplies a question and documents, with one document containing the annotated answer. The researchers move that document within the input and vary the number of distracting documents. Reordering leaves the expected answer unchanged.
The resulting accuracy often follows a U-shaped curve: higher at the ends and lower in the middle. The paper calls preference for the beginning primacy bias and preference for the end recency bias. These are measured patterns in answer performance.
Why is finding one fact an incomplete test?
A needle-in-a-haystack test asks a model to retrieve a specified piece of information embedded in distracting text. RULER extends this setup with multiple targets and additional tasks. Its variable-tracking task requires following connections between variables; its word-frequency tasks require aggregating information across the input.
The authors describe these synthetic tasks as behavioral checks. Their published evaluation reports a score for each input length rather than a breakdown by evidence position. A length comparison and a position comparison therefore examine different aspects of performance.
The Lost in the Middle key-value retrieval task asks for the value associated with a specified key. Placing the query before and after the data produces near-perfect retrieval, while minimally changing the question-answering performance trends.
How can an assistant's use of documents be checked?
A practical test adapted from Lost in the Middle keeps the question and answer-bearing document fixed, moves that document between positions, and checks answer correctness. Varying the number of distractors separately tests sensitivity to input length.
Task selection also matters. LongBench separates single-document and multi-document question answering from summarization and code completion. For an assistant answering questions about a website's documents, the practical implication is to evaluate those questions directly and report results by task and input length.
These studies do not measure how a current answer engine orders website passages or chooses citations.
Related terms
FAQ
Does a middle passage always get ignored?
No. Lost in the Middle reports reduced accuracy in tested settings, with performance often higher at the ends. Its finding concerns the ability to use relevant information when answering.
Is a successful retrieval test enough for document summarization?
RULER treats retrieval and aggregation as separate behaviors. For summarization, you can include summarization tasks, which LongBench evaluates as a distinct category.
Should I put every important website fact at the beginning?
Lost in the Middle moves documents inside a model's supplied input. For a website assistant, you can use that method to test document ordering before adopting a placement rule.
Sources
- Liu et al., Transactions of the Association for Computational Linguistics, "Lost in the Middle: How Language Models Use Long Contexts", 2024
- Hsieh et al., arXiv, "RULER: What's the Real Context Size of Your Long-Context Language Models?", 6 August 2024
- Bai et al., ACL Anthology, "LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding", August 2024
Check how a page reads to answer engines. Open the GEO score analyzer