Membership inference
A technique used to figure out if a specific piece of information, like a person's private medical record, was used to teach an AI model, which can reveal sensitive data that was supposed to remain secret.
A privacy attack where an adversary queries a target model to determine if a specific data point was included in its training set, typically by exploiting differences in the model's confidence scores or loss values for seen versus unseen data.
A class of inference attacks aimed at quantifying the membership status of a record in a target model's training distribution by analyzing the model's output distribution, specifically leveraging the model's tendency to exhibit lower generalization error or higher confidence on training samples compared to non-members.
evolution
- 2016 · historyFoundational Discovery
Shokri et al. introduced the first formal membership inference attack against machine learning models using shadow models.
- 2017 · historyExploiting Overfitting
Salem et al. demonstrated that membership inference is highly effective when models overfit to their training data.
- 2019 · historyGenerative Model Vulnerability
Hayes et al. extended membership inference techniques to generative models like GANs.
- 2021 · historyLarge Language Model Context
Carlini et al. proved that membership inference attacks can extract training data from large language models via prompt-based queries.