Impact
Impact is when a bad actor tries to break, mess with, or ruin your AI's work, data, or reputation, like someone trying to sabotage a machine so it stops working or starts giving wrong answers.
Impact refers to adversarial techniques aimed at disrupting the availability, integrity, or reliability of AI systems. This includes sabotaging data, corrupting model outputs, or manipulating operational processes to erode trust or achieve malicious objectives.
The adversary is trying to manipulate, interrupt, erode confidence in, or destroy your AI systems and data. Impact consists of techniques that adversaries use to disrupt availability or compromise integrity by manipulating business and operational processes. Techniques used for impact can include destroying or tampering with data. In some cases, business processes can look fine, but may have been altered to benefit the adversaries' goals. These techniques might be used by adversaries to follow through on their end goal or to provide cover for a confidentiality breach.
evolution
- 2016 · historyAdversarial Machine Learning Formalization
Researchers formally categorized evasion and poisoning attacks as primary threats to the integrity and availability of machine learning models.
- 2018 · historyModel Denial of Service (MDoS) Identification
Security researchers identified that computationally expensive inputs could be used to exhaust AI system resources, effectively creating a new class of availability impact.
- 2021 · historyMITRE ATLAS Framework Launch
The MITRE Corporation released the Adversarial Threat Landscape for Artificial-Intelligence Systems (ATLAS) to standardize the classification of AI-specific impact techniques.
- 2023 · historyOWASP Top 10 for LLMs
The release of the OWASP Top 10 for Large Language Model Applications formally codified 'Insecure Output Handling' and 'Model Denial of Service' as critical impact vectors.