Collection
Collection is the phase where an attacker acts like a spy, scouting out and gathering specific pieces of information or digital files related to an AI system to help them achieve their goal.
Collection involves the techniques and sources an adversary uses to identify and acquire AI-related artifacts, such as model weights, training datasets, or configuration files, which are necessary to advance their operational objectives or prepare for exfiltration.
The adversary is trying to gather AI artifacts and other related information relevant to their goal. Collection consists of techniques adversaries may use to gather information and the sources information is collected from that are relevant to following through on the adversary's objectives. Frequently, the next goal after collecting data is to steal (exfiltrate) the AI artifacts, or use the collected information to stage future operations. Common target sources include software repositories, container registries, model repositories, and object stores.
evolution
- 2016 · historyModel Inversion Attacks
Researchers demonstrated that adversaries could reconstruct training data or sensitive attributes by querying machine learning models.
- 2017 · historyMembership Inference Attacks
Shokri et al. formalized techniques to determine if a specific data record was part of a model's training set, marking a shift toward data exfiltration.
- 2021 · historyModel Extraction Formalization
The industry began categorizing the theft of model parameters and architecture as a distinct phase of the AI attack lifecycle.
- 2023 · historyMITRE ATLAS Framework
The MITRE ATLAS knowledge base officially codified 'Collection' as a distinct tactic for AI-specific adversary behavior.