Poison Training Data
Poisoning training data is like a prankster sneaking fake flashcards into a student's study deck. Because the student learns from these incorrect cards, they end up making specific mistakes later on whenever they see a secret 'trigger' symbol, even though they seem to be learning normally otherwise.
Data poisoning is an adversarial attack where an actor injects malicious samples or modifies labels within a training set to compromise a model's integrity. This process embeds latent vulnerabilities—often backdoors—that remain dormant until activated by specific input patterns, allowing the attacker to manipulate model outputs post-deployment.
Adversaries may attempt to poison datasets used by an AI model by modifying the underlying data or its labels. This allows the adversary to embed vulnerabilities in AI models trained on the data that may not be easily detectable. Data poisoning attacks may or may not require modifying the labels; the embedded vulnerability is activated at a later time by data samples containing a specific backdoor trigger. Poisoned data can be introduced via AI supply chain compromise or after the adversary gains initial access to the system, effectively subverting the model's objective function during the training phase.