Memory
Memory is a feature that lets an AI 'remember' things you tell it so it can use that information in later conversations. Because the AI learns from what you say, a malicious person could trick the AI into 'remembering' bad instructions, which would then stay in the AI's memory and affect how it behaves every time you talk to it in the future.
In AI security, memory refers to a persistent, user-specific database that stores context across sessions. Adversaries can exploit this by using prompt injection techniques to insert malicious data or persistent instructions into the database. Once stored, these injected memories influence the model's future outputs, allowing an attacker to maintain a long-term, unauthorized influence over the model's behavior.
Memory is a stateful retrieval mechanism that persists user-defined context across discrete inference sessions via a backend database. Adversaries leverage Direct or Indirect Prompt Injection to perform unauthorized writes to this memory store. By manipulating the stored context, an attacker achieves persistent state corruption, ensuring that malicious instructions or adversarial biases are retrieved and integrated into the model's system prompt or context window during subsequent sessions.
seen in events