Indirect Prompt Injection
Indirect prompt injection is a security flaw where a malicious actor hides secret instructions inside a website or document that you ask an AI to read. Instead of you giving the AI orders, the AI accidentally follows the hidden instructions found in the content it is processing, potentially tricking it into doing things you didn't intend.
A vulnerability where an LLM is compromised by untrusted data retrieved from external sources, such as web pages, documents, or tool outputs, rather than direct user input. The model interprets embedded instructions within the ingested content as system-level commands, leading to unauthorized actions or data exfiltration.
A class of prompt injection attack characterized by the execution of adversarial instructions embedded within retrieved, untrusted context. By manipulating the model's input stream via external data ingestion—bypassing the direct user-to-model prompt boundary—the attacker forces the model to prioritize malicious instructions over system prompts, effectively hijacking the model's control flow and execution context.
evolution
- 2022-09 · historyConcept Introduction
Security researcher Riley Goodside demonstrates prompt injection, leading to the identification of indirect vectors where models process untrusted external data.
- 2023-02 · historyBing Chat Exploitation
Researchers demonstrate that Bing Chat could be manipulated by hidden instructions embedded in websites it indexed, effectively performing an indirect prompt injection.
- 2023-07 · historyOWASP Top 10 for LLMs
The OWASP Foundation officially lists 'Prompt Injection' as the number one critical vulnerability for LLM applications, formalizing the threat of indirect attacks.
- 2024-01 · historyIndirect Injection via RAG
Academic research highlights the systemic risk of Retrieval-Augmented Generation (RAG) systems being compromised by malicious content stored in vector databases.