Pre-launch · Training and the free Foundations exam are open now. Levels 1–5 open soon. Get notified on any level page.
25 min read
Current tools accept very long inputs: whole reports, contracts, months of tickets. Accepting a long input and using all of it well are different things. This module covers what research shows about how models handle long contexts, and the working patterns that follow from it: send less, place what matters where it gets used, split long documents into passes you can check, and check whether something in the middle was dropped.
Position matters. Liu and colleagues (2024) gave models a question and a set of documents, one of which held the answer, and moved that document around. Models did best when the relevant document was at the beginning or the end of the input and noticeably worse when it sat in the middle. They also found that performance fell as the input grew longer, including for models built for long contexts.
The models in that study have since been replaced, and the size of the effect varies by model. The working assumption still holds: a long input is not read with even attention, and you should test for misses instead of assuming there are none.
What follows from that.
Split and combine. When a document is too long to trust in one pass, process it in parts:
Splitting can reduce long-context omission. It does not guarantee attention, coverage or a correct combination, so extraction and merge failures still have to be checked. Each record can be checked against a few pages.
Always work from the source. Summarizing a summary drops qualifiers such as "pending" and "in most cases." If you need a shorter version, go back to the records or the source, not to the last summary.
Check coverage. A long-document summary fails quietly: it reads well and leaves something out. Three cheap checks:
When one pass is fine. A short document, or a task that needs the gist and not every item, doesn't need this. Splitting adds calls, cost, and a combine step that can fail. Use it when missing one item has a consequence.
Ingrid Halvorsen is a safety coordinator at a food packaging plant. An outside auditor delivered a 94-page report in 11 sections. She must give the plant manager a list of every open finding with its deadline.
Before:
Here is our safety audit report. Summarize the key findings and
list anything we need to fix.
[94 pages]
"Key findings" invites the model to choose. Nothing ties a finding to a page, so Ingrid can't check the list without rereading the report. A finding in section 6 could be missing and the summary would read just as well.
After (run once per section):
Extract open findings from one section of a safety audit report.
The section is between the <section> tags.
For each finding the auditor marks as open or unresolved, return:
- section: the section number
- finding_id: the auditor's ID, or "none given"
- finding: one sentence, in the auditor's terms
- deadline: the date stated, or "not stated"
- quote: the sentence that shows it is open
Include every open finding, including minor ones. If the section
has no open findings, return exactly: NO OPEN FINDINGS.
Do not include findings marked closed.
<section number="6">
[section text]
</section>
After (run once, on the 11 records):
Below are records of open findings from 11 sections of a safety
audit. Produce one list for the plant manager, sorted by
deadline, earliest first. Put "not stated" deadlines at the end.
Keep every finding. Do not merge findings, and do not drop any.
Keep the section number and finding_id on every line.
End with a line: TOTAL FINDINGS: N.
[11 records]
Ingrid counts 11 records. She adds up the findings in the records herself and gets 23, which matches the total line. She compares sections 5 and 7 to the report by hand. Section 7's record missed a finding that the auditor described as "remains outstanding," so she adds that phrase to the extraction prompt and reruns the section. She then reruns the combine step: the total line now reads 24, and her hand count agrees.
Why this is stronger. Each pass reads a few pages, which can reduce long-context omission. Extraction and the merge still need checking. Every item has a section and a quote, so checking takes minutes. "NO OPEN FINDINGS" makes an empty section different from a skipped one. The total line and the hand count catch a dropped item, and the middle spot-check found a real miss.
Setup. Take a document of at least 15 pages with clear sections: a handbook, a long report, meeting minutes from several months. Pick one specific detail from a middle section and write it down.
Task. Ask for a list of all items of one kind (action items, deadlines, requirements) in a single pass. Then do it by section with a fixed record and a combine step. Compare the two lists, and check whether each one found your planted detail.
What a good result looks like. You can say how many items each approach found and which ones differed. The by-section list carries a section reference on every item. You ran the count check and one middle spot-check.