Reconnaissance
Reconnaissance is like a scout casing a building before a break-in; the attacker is quietly looking around to learn how the AI system works so they can figure out the best way to attack it later.
Reconnaissance involves the systematic collection of data regarding an AI system's architecture, training data, or deployment environment. This intelligence is used to identify vulnerabilities and map out the attack surface, allowing the adversary to refine their strategy and increase the probability of success in subsequent stages of the kill chain.
The adversary is trying to gather information about the AI system they can use to plan future operations. Reconnaissance consists of techniques that involve adversaries actively or passively gathering information that can be used to support targeting. Such information may include details of the victim organizations' AI capabilities and research efforts. This information can be leveraged by the adversary to aid in other phases of the adversary lifecycle, such as using gathered information to obtain relevant AI artifacts, targeting AI capabilities used by the victim, tailoring attacks to the particular models used by the victim, or to drive and lead further Reconnaissance efforts.
evolution
- 2017 · historyAdversarial Machine Learning Formalization
The publication of 'Adversarial Examples in the Physical World' established the framework for probing model vulnerabilities, a prerequisite for reconnaissance.
- 2020 · historyModel Inversion Attacks
Researchers demonstrated that adversaries could reconstruct training data through API queries, formalizing the concept of data-focused reconnaissance.
- 2022 · historyPrompt Injection Discovery
The emergence of LLM-specific prompt injection techniques introduced a new reconnaissance vector for mapping system instructions and internal logic.
- 2023 · historyMITRE ATLAS Framework
MITRE officially codified 'Reconnaissance' as a distinct tactic in the ATLAS knowledge base for adversarial tactics against AI systems.