Cache-Aware Prompt Design

Advanced

Order your prompt like a cache, not an essay

01 The Concept

Prompt caching changed the economics of production prompts: providers cache the prefix of your prompt, so repeated calls that share an identical opening pay a fraction of the cost (up to ~90% cheaper on cached tokens) and respond much faster. But caching only works on exact prefix matches — one changed character invalidates everything after it.

The design rule: static first, volatile last. System instructions, policies, few-shot examples, tool definitions — all identical across calls — go at the top. Anything that changes per call (the user's message, retrieved documents, timestamps) goes at the bottom. The classic cache-killer is an innocent-looking dynamic value near the top: a timestamp in the system prompt, a randomly-ordered list, a per-user greeting. That one line turns every call into a cold cache miss.

02 Weak vs. Strong

EX 01Restructuring a support-bot prompt

System prompt (identical every call — cacheable): "You are the Vitae support assistant. [4,000 tokens of product docs, policies, escalation rules, response format...]" User message (volatile tail): "[context] date: 2026-07-10 | customer: {{user_name}} | plan: {{plan}} [message] {{message}}"

→ Why it works

Moving the timestamp and username out of the system prompt makes the 4,000-token prefix byte-identical across every call — cached after the first request. The volatile facts still reach the model, just at the tail where they don't poison the cache.

EX 02RAG prompt with cache-friendly layout

Layout every call as: 1. System instructions + answer format + citation rules (static, cached) 2. Few-shot examples of good grounded answers (static, cached) 3. Retrieved chunks, each tagged <doc id="..."> — sorted by document ID, not by similarity score 4. User question (last) Sorting note: similarity order changes per query even when the same chunks are retrieved; ID order means identical retrieval sets produce identical prompt sections — partial cache hits on repeated topics.

→ Why it works

Instructions and examples become a permanent cached prefix, and deterministic chunk ordering stops equivalent retrievals from looking like different prompts.

03 Key Points

01Caches match exact prefixes: static content first, per-call content last

02One dynamic token near the top (timestamp, session ID, shuffled list) invalidates everything below it

03Stabilize ordering: sort tool lists and example lists deterministically — 'same content, different order' is a different prefix

04Long shared context (policies, schemas, docs) is exactly what caching is for — stop trimming it for cost, start ordering it

05Design prompt updates around cache lifetimes: batch your system-prompt edits; every edit is a fleet-wide cache flush

06Measure: providers report cached vs. uncached tokens — watch that ratio like you watch latency

04 Model-Specific Notes

The Claude API uses explicit cache_control breakpoints — you mark where cacheable prefixes end. Default cache lifetime is short (minutes); design chat UIs so follow-up turns land inside it, and consider the extended-TTL option for slow-cadence workloads.

05 For Your Role

This is a builder's lesson, but the ordering habit helps anywhere: put the stable framing first and the changing details last — your prompts get easier to reuse as templates too.

Next: Computer-Use & GUI Agent Prompting →