Agent hijacking
Agent hijacking is like tricking a digital assistant into ignoring its original instructions and following a hacker's secret commands instead, effectively turning the assistant into a double agent.
Agent hijacking is a security compromise where an attacker subverts an autonomous AI agent's goal or actions—via prompt injection, poisoned tools, or manipulated memory—to force the agent to execute unauthorized tasks on the attacker's behalf.
Agent hijacking refers to the unauthorized redirection of an autonomous agent's objective function or operational trajectory. By exploiting vulnerabilities in the agent's input processing (prompt injection), tool integration (poisoned tools), or persistent state (manipulated memory), an adversary gains control over the agent's decision-making loop, compelling it to perform actions that deviate from its intended policy to serve the attacker's objectives.
evolution
- 2022-12 · historyEmergence of Indirect Prompt Injection
Researchers demonstrated that LLM-based agents could be hijacked by hidden instructions embedded in web content they were tasked to summarize.
- 2023-03 · historyTool-Use Vulnerabilities
The integration of plugins in GPT-4 highlighted that agents could be manipulated into executing malicious API calls through deceptive prompt inputs.
- 2023-10 · historyCompromise of Agentic Memory
Studies revealed that persistent memory stores (like vector databases) could be poisoned to inject long-term malicious instructions into an agent's operational context.
- 2024-02 · historyAutonomous Agent Hijacking Frameworks
Security research formalized the 'Agent Hijacking' threat model, focusing on how autonomous loops allow attackers to maintain control over an agent's multi-step reasoning process.