Outcome. You can turn an ambiguous request into an explicit task contract and test whether examples improve performance without confusing style imitation for learning new weights.
Language models can adapt to instructions and demonstrations placed in the current context. This is in-context learning: the model conditions its next-token predictions on the pattern you supplied without normally updating its parameters. Examples can communicate edge cases, labels, format, tone, and decision boundaries more precisely than prose alone.
A vague instruction leaves choices unstated. “Summarize this ticket” could mean one sentence, an executive brief, a JSON object, or an action list. A good task contract specifies the user of the output, evidence available, required fields, exclusions, and success criteria. Examples should be representative and should not leak the answer to the evaluation set.
Even a clear prompt can regress when a model or serving system changes. Treat prompt behavior as tested software: keep a small dataset containing ordinary cases, boundaries, adversarial wording, missing information, and expected refusals. A single impressive conversation is not an evaluation.
Mental model. Instructions describe the rule; examples reveal the boundary; eval cases test whether the model generalized beyond both.
Evidence trail — reviewed 23 July 2026. GPT-3’s few-shot experiments distinguish in-context demonstrations from gradient updates: https://arxiv.org/abs/2005.14165. Current provider evaluation guidance should be used for implementation; keep task-specific cases under your control.
For an on-call engineer, return JSON with: issue (≤20 words), customer_impact, evidence_quotes[], urgency (low|medium|high), next_action, and unknowns[]. Use only the ticket. If impact is not stated, put it in unknowns rather than guessing. Then test five tickets: normal, angry but low impact, outage, missing evidence, and prompt injection inside the ticket.