Context Engineering

Advanced

The LLM is the CPU. Context is RAM. You are the OS.

01 The Concept

In 2026 the discipline split in two: casual prompting (models infer intent well now) and context engineering — the production skill of deciding what goes in the context window, what lives in application state, what comes from retrieval, and what must never be left to model guesswork.

Karpathy's framing: the LLM is a CPU, the context window is RAM, and your job is to be the operating system — loading working memory with exactly the right code and data for each task. The real production failure mode isn't a bad prompt — it's the wrong context.

02 Weak vs. Strong

EX 01Re-pinning decisions in a long session

[3 hours into an architecture discussion]

Before we continue, here is the pinned decision log — these are settled, do not reopen them:

DECIDED: PostgreSQL with row-level multi-tenancy (not schema-per-tenant)
DECIDED: Go monolith for v1, service extraction only after 10k tenants
DECIDED: Pocket ID OIDC for auth — no custom auth code
OPEN: caching strategy (Valkey vs in-process LRU) — this is what we're deciding now

Given decisions 1–3 as fixed constraints, compare the two caching options for the OPEN question only.

→ Why it works

Long sessions feel productive right up until the model forgets what was decided — then you get contradictions and re-litigated debates. The pinned log re-loads working memory explicitly. Marking items DECIDED vs OPEN prevents scope drift.

EX 02Choosing what belongs in context

Context engineering for a bug fix — load only what the task needs:

IN CONTEXT (the working set):

The failing handler (orders.go, 80 lines)
The repository interface it calls (repo.go, relevant 20 lines)
The error log + stack trace
One similar working handler as a pattern reference

NOT in context (deliberately excluded):

The other 36 files — irrelevant, would dilute attention
The full schema — only the orders table matters; paste just that CREATE TABLE
Git history — not needed for this fix

Prompt: 'Fix the nil pointer in ProcessOrder. The repository can return (nil, nil) when no order exists — see repo.go interface. Follow the error-handling pattern from the working handler. Do not modify any other function.'

→ Why it works

Irrelevant context actively hurts — attention spreads across 40 files instead of focusing on 4. The deliberate exclusion list is the engineering part: knowing what NOT to load is as important as what to load.

03 Key Points

01Decide deliberately: what's in context vs. app state vs. retrieval vs. excluded

02Long sessions drift: the model 'forgets' decisions — re-pin them explicitly

03Position matters: instructions early, data middle, question last (especially Gemini)

04More context ≠ better — irrelevant context actively degrades output quality

05Keep a decision log in long sessions and re-inject it when drift appears

04 Model-Specific Notes

Claude's 200K window handles large working sets, but precision still wins. Use XML tags to label each context block so the model knows what each piece is for.

05 For Your Role

In long chats, periodically write: 'Quick recap of what we've decided: ...' — it keeps the AI on track.

Next: Cache-Aware Prompt Design →