LLMs can leak sensitive data through two very different mechanisms, and conflating them leads to the wrong fix. First: memorization — a model trained or fine-tuned on data containing PII or secrets can sometimes regurgitate exact snippets verbatim under the right prompting. Second: context leakage — in multi-tenant systems, a bug in session handling or prompt construction lets one user's data bleed into another user's context window. A widely-reported 2023 incident at Samsung illustrated a related, simpler risk: employees pasted confidential source code and meeting notes into ChatGPT to get help with their work, and because that input could potentially be used for future model improvements, the company banned employee use of public generative AI tools shortly after — not a technical breach, but the same underlying lesson that anything typed into a third-party AI service should be treated as potentially retained.
The fix for memorization is data hygiene before training (scrub PII, dedupe near-duplicates that make memorization more likely) plus differential-privacy-style training when the stakes justify it. The fix for context leakage is standard application security: strict session isolation, careful prompt-construction code, and treating 'what goes into this user's context' as sensitive as any other user-scoped data.
Within about 20 days of Samsung's semiconductor division allowing engineers to use ChatGPT, employees pasted in confidential source code to get debugging help and meeting notes to have them summarized — real, specific pieces of company IP typed into a public third-party service with no company policy addressing whether that was safe. There was no breach, no hack, and no malicious actor; the risk was simply that once that text left the company's systems, Samsung had no way to control how it might be retained or used going forward. Samsung responded by banning employee use of public generative AI tools on company devices shortly after the incident became public.