Model inversion
Model inversion is a privacy breach where someone uses a computer program to 'trick' an AI into revealing the private information, like photos or personal records, that it was originally taught with.
Model inversion is an inference-time attack where an adversary queries a target model to reconstruct sensitive training data samples, effectively exploiting the model's memorization of its training set to compromise data privacy.
Model inversion is a class of reconstruction attacks wherein an adversary leverages black-box or white-box access to a model's output distribution to perform gradient-based or optimization-based inversion, mapping model parameters back to the underlying training data manifold to recover sensitive input features or records.
evolution
- 2007 · historyInitial formalization
Fredrikson et al. introduced the concept of model inversion attacks against linear classifiers in the context of health informatics.
- 2015 · historyDeep learning vulnerability
Fredrikson et al. demonstrated that model inversion could effectively reconstruct recognizable human faces from deep neural network outputs.
- 2019 · historyGenerative model inversion
Researchers showed that GANs could be used to perform more sophisticated model inversion attacks, significantly improving the quality of reconstructed data.
- 2021 · historyLarge-scale privacy risk
Studies highlighted that large-scale pre-trained models are susceptible to inversion attacks that extract training data, leading to widespread privacy concerns.