Prompt caching changed the economics of production prompts: providers cache the prefix of your prompt, so repeated calls that share an identical opening pay a fraction of the cost (up to ~90% cheaper on cached tokens) and respond much faster. But caching only works on exact prefix matches — one changed character invalidates everything after it.
The design rule: static first, volatile last. System instructions, policies, few-shot examples, tool definitions — all identical across calls — go at the top. Anything that changes per call (the user's message, retrieved documents, timestamps) goes at the bottom. The classic cache-killer is an innocent-looking dynamic value near the top: a timestamp in the system prompt, a randomly-ordered list, a per-user greeting. That one line turns every call into a cold cache miss.
System prompt (identical every call — cacheable): "You are the Vitae support assistant. [4,000 tokens of product docs, policies, escalation rules, response format...]"
User message (volatile tail): "[context] date: 2026-07-10 | customer: {{user_name}} | plan: {{plan}} [message] {{message}}"
Layout every call as:
Sorting note: similarity order changes per query even when the same chunks are retrieved; ID order means identical retrieval sets produce identical prompt sections — partial cache hits on repeated topics.
The Claude API uses explicit cache_control breakpoints — you mark where cacheable prefixes end. Default cache lifetime is short (minutes); design chat UIs so follow-up turns land inside it, and consider the extended-TTL option for slow-cadence workloads.