Command and Control
Command and Control is like a puppeteer pulling strings; it is the secret way a hacker sends instructions to a compromised AI system to make it do what they want, often hiding these commands inside normal-looking messages to avoid being caught.
Command and Control refers to the communication channels established by an adversary to maintain persistent access and issue instructions to a compromised AI system. Practitioners focus on identifying these covert channels, which often blend in with legitimate traffic to bypass network security controls and evade detection.
The adversary is trying to communicate with compromised AI systems to control them. Command and Control consists of techniques that adversaries may use to communicate with systems under their control within a victim network. Adversaries commonly attempt to mimic normal, expected traffic to avoid detection. There are many ways an adversary can establish command and control with various levels of stealth depending on the victim's network structure and defenses.
evolution
- 2017 · historyEmergence of Adversarial ML
Researchers began formalizing how adversaries could manipulate model inputs to establish unauthorized control over AI decision-making processes.
- 2021 · historyAdversarial ML Threat Matrix
Microsoft and MITRE released the ATLAS framework, explicitly categorizing Command and Control as a core tactic for compromising AI systems.
- 2023 · historyPrompt Injection Proliferation
The rise of LLMs introduced indirect prompt injection as a primary vector for establishing remote command and control over AI agents.
- 2024 · historyNIST AI RMF Integration
NIST updated its AI Risk Management Framework to include specific guidance on mitigating unauthorized remote control and command-based attacks on AI infrastructure.