Sensitive Information Disclosure
This happens when an AI accidentally spills secrets it shouldn't, like sharing someone else's private information, passwords, or company trade secrets in its answers.
A security vulnerability where an LLM inadvertently exposes sensitive data—such as PII, credentials, or proprietary information—by regurgitating training data or leaking details from its input context during inference.
An AI security failure mode characterized by the unauthorized exfiltration of confidential data, including PII, credentials, or proprietary intellectual property, resulting from the model's failure to maintain data boundaries during generation, whether through the memorization and subsequent regurgitation of training corpus artifacts or the improper inclusion of sensitive context in the output stream.
evolution
- 2020-05 · historyGPT-3 Training Data Memorization
Researchers demonstrated that large language models could inadvertently memorize and regurgitate verbatim training data, including PII.
- 2022-11 · historyChatGPT Public Launch
The widespread adoption of consumer-facing LLMs highlighted the risk of users inadvertently inputting sensitive corporate data into public models.
- 2023-03 · historyChatGPT History Leak
A bug in the Redis client library caused ChatGPT users to see the titles of other users' chat histories, marking a major real-world PII disclosure incident.
- 2023-05 · historySamsung Data Leak
Samsung employees accidentally leaked proprietary source code and internal meeting notes by using ChatGPT to assist with work tasks.
- 2023-12 · historyTraining Data Extraction Attacks
Researchers proved that simple prompt injection techniques could force models like ChatGPT to output large volumes of their underlying training data.