LLM Prompt Injection
Prompt injection is like tricking a digital assistant into ignoring its original rules. Just as a prankster might try to confuse a person by giving them conflicting instructions, an attacker sends a cleverly worded message to an AI that convinces it to stop following its safety guidelines and start doing whatever the attacker wants instead.
Prompt injection is a security vulnerability where an adversary inputs malicious instructions to manipulate an LLM into overriding its system-level directives. This technique allows attackers to bypass safety filters, gain unauthorized access to system functions, or force the model to execute unintended tasks, often serving as an initial access vector for further exploitation.
LLM Prompt Injection is a class of adversarial attacks where malicious inputs are crafted to subvert the model's instruction-following hierarchy, causing the LLM to prioritize adversary-supplied directives over its original system instructions. These attacks manifest as direct, indirect, or triggered injections, enabling the bypass of security controls, unauthorized privilege escalation, and persistent state manipulation within the model's operational context.