SIGNAL//DESK
AI securitysrc: MITRE ATLAS

AI Model Access

AI Model Access is when someone tries to get close enough to an AI to interact with it, influence how it thinks, or peek at how it works, similar to someone trying to get into a restricted building to see what is happening inside.

AI Model Access refers to the various methods an adversary uses to establish a connection with an AI system, ranging from public API queries to gaining unauthorized entry into the underlying infrastructure, in order to probe, manipulate, or extract information from the model.

The adversary is attempting to gain some level of access to an AI model. AI Model Access enables techniques that use various types of access to the AI model that can be used by the adversary to gain information, develop attacks, and as a means to input data to the model. The level of access can range from the full knowledge of the internals of the model to access to the physical environment where data is collected for use in the AI model. The adversary may use varying levels of model access during the course of their attack, from staging the attack to impacting the target system. Access to an AI model may require access to the system housing the model, the model may be publicly accessible via an API, or it may be accessed indirectly via interaction with a product or service that utilizes AI as part of its processes.

evolution

  1. 2016-08 · history
    Model Inversion Attacks

    Researchers demonstrated that black-box API access to machine learning models could be exploited to reconstruct sensitive training data.

  2. 2017-02 · history
    Model Extraction Attacks

    Studies showed that an adversary with query access could steal the functionality of a proprietary model by training a substitute model on its outputs.

  3. 2020-12 · history
    Prompt Injection Emergence

    The rise of large language models introduced prompt injection, where attackers gain unauthorized control over model behavior through input manipulation.

  4. 2023-03 · history
    Malicious Model Injection

    Security researchers identified risks where attackers upload backdoored models to public repositories to gain access to downstream user environments.

← all terms