Long-Context Prompts That Don't Get Lost

Intermediate

A million tokens in, one relevant needle out

01 The Concept

Modern models accept enormous inputs — whole codebases, hundred-page contracts, a year of meeting notes. But 'fits in context' ≠ 'gets used well'. Attention over long inputs is uneven: material in the middle of a giant prompt gets less weight than the start and end, and models skim just like tired humans.

Three habits fix most of it. Structure the haystack: label and delimit each document so sections are addressable ('in <contract-B>…'). Position deliberately: long material first, your question and instructions last, restated. Force evidence: require verbatim quotes with locations before conclusions — quote-then-answer converts skimming into lookup, and makes hallucinated 'findings' immediately visible.

02 Weak vs. Strong

EX 01Comparing two contracts

Two contracts follow, tagged <contract-A> (our current vendor) and <contract-B> (proposed replacement). TASK PREVIEW (details after the documents): compare liability & indemnification. <contract-A> [80 pages] </contract-A> <contract-B> [75 pages] </contract-B> TASK: 1. From each contract, quote verbatim every clause about liability caps, indemnification, and exclusions — with its section number. 2. Build a side-by-side table from those quotes only. 3. Flag anything present in one contract but absent in the other — absence is a finding. 4. End with: which sections of either document you did not read closely. If a topic isn't covered in the quotes you found, say 'not found' — do not summarize from memory of typical contracts.

→ Why it works

Tagged documents, task restated after the material, quotes-with-locations before conclusions, absence treated as signal, and a coverage confession. Each habit closes a specific long-context failure.

EX 02Asking about a big codebase dump

40 source files follow, each preceded by === FILE: path ===. [files] TASK: hypothesize why checkout latency doubled. Rules: - Cite evidence as file:line for every claim ('checkout.go:112 acquires the lock inside the loop') - Rank your top 3 hypotheses by likelihood, each with its evidence and the experiment that would confirm it - List the files you treated as irrelevant, in one line - If the cause likely lives in a file NOT provided (config, infra, DB schema), say exactly what you'd need

→ Why it works

File-line citations make claims checkable; ranked falsifiable hypotheses beat one confident guess; and the 'what's missing' clause stops the model pinning the crime on whatever code happens to be visible.

03 Key Points

01Big context ≠ attentive context: middles get skimmed — structure and position accordingly

02Label every document: <report-2024>, <email-3> — so both you and the model can address sections

03Instructions go AFTER the long material (and can be previewed before it); the end of the prompt is prime attention

04Quote-then-answer: 'first give verbatim quotes with section references, then conclusions based only on those quotes'

05Ask for a coverage note: 'state which sections you did not use' — reveals silent skimming

06If it's really a search task, retrieve-then-read (RAG) still beats dumping everything in

04 Model-Specific Notes

Claude is trained for long-document work with XML tags and quote-first workflows — 'extract relevant quotes into <quotes> before answering' is a documented, measurable win. Put instructions after the documents.

05 For Your Role

Pasting a 30-page insurance policy? Ask: 'quote every clause about water damage with its section number, then explain in plain English using only those quotes.'

Next: Deep-Research Prompts →