Confidentiality, Integrity, Availability — for AI Systems

Beginner

The CIA triad still applies, with new failure modes

01 The Concept

The classic security triad maps onto AI systems with new twists. Confidentiality now includes the model leaking training data, system prompts, or other users' context through its outputs — not just a database breach. Integrity means the model's behavior stays as intended: no injected instructions silently changing what it does. Availability covers both classic denial-of-service and a new variant — resource-exhaustion attacks that run up your inference bill or degrade quality for everyone else.

The useful shift is realizing that 'confidential data' now includes things baked into the model itself: a system prompt is effectively a secret the model is asked to keep, and it's much easier to accidentally leak than a database credential.

02 Case Study

CASEThe National Eating Disorders Association's "Tessa" chatbot incident, widely reported (NPR and others, 2023)

NEDA replaced its human-staffed helpline with an AI chatbot named Tessa shortly before the incident came to light: the chatbot gave callers seeking help for eating disorders concrete advice about calorie restriction and weight loss — the exact opposite of appropriate guidance for that population, and a direct integrity failure, since the bot's actual behavior diverged sharply from its intended purpose. NEDA suspended the chatbot once the pattern was publicly reported, and the case became a widely cited example in AI-safety discussions of what happens when a system's behavior isn't verified to match its stated purpose before deployment in a sensitive context.

→ Why it matters

This wasn't a hack, a leak, or a server outage — it's an integrity failure in the CIA-triad sense: the system's actual behavior no longer matched what it was supposed to do, and nobody caught the gap before real, vulnerable users were exposed to it. That's precisely the kind of failure mode classic confidentiality/availability-focused security reviews are least equipped to catch.

03 Key Points

01Confidentiality risk includes system-prompt and training-data leakage via model output

02Integrity risk includes injected instructions silently changing model behavior

03Availability risk includes both DoS and inference-cost exhaustion attacks

04A system prompt is a secret the model has to actively keep — treat it that way

Next: What Is AI Security, Really? →