Outcome. You can resolve a conflict among system/developer instructions, user requests, tool output, and quoted untrusted content—and identify what must be enforced outside the model.
Chat APIs label pieces of a request with roles. Exact names differ, but common roles include high-priority application instructions, user messages, assistant history, and tool results. Models are post-trained to follow an instruction hierarchy, so application-level instructions should generally outrank a conflicting user request.
That hierarchy is learned behavior, not an operating-system permission system. A model may misunderstand boundaries, follow malicious instructions embedded in retrieved content, or produce an action proposal that violates policy. Anything that truly must not happen—sending money, revealing a secret, deleting data—needs deterministic authorization outside the model.
Content and instructions can travel together in one context. Clearly delimit quoted documents, mark them as untrusted data, minimize unnecessary privileges, and verify actions. Do not hide a password in a system message and call it secure: text the model can use may also be exposed through behavior or attack.
Mental model. Roles influence which text the model should obey; application controls decide which actions are actually allowed.
Evidence trail — reviewed 23 July 2026. InstructGPT documents training for instruction following: https://arxiv.org/abs/2203.02155. Follow the live role/precedence documentation of the API you use. For security controls, use OWASP’s current LLM guidance: https://genai.owasp.org/.
Keep the policy in the application as well as the prompt. The model may propose {order_id, amount, reason}; a policy service verifies ownership and amount; refunds above $100 require human approval; the tool executes only an authorized request; all attempts are logged.