366 concepts · layman → expert
Glossary
Every term defined at three depths and split by sense — so harness
(the agent runtime) never gets confused with Harness.io (the CI/CD product).
harness
the 'cockpit' that lets an AI model actually do things, not just chat
Harness (Harness.io)
Harness.io — a commercial CI/CD and software-delivery platform
Prompt injection
Someone hides an instruction inside something AI reads — an email, a PDF, a website — and AI obeys the smuggled instruction instead of yours. They didn't break
guardrail
a rule or filter meant to stop an AI from doing something harmful
supply chain attack
attacking software by poisoning something it depends on, not the target directly
zero-day
a security hole the defenders don't know about yet, so there's no fix
model context protocol
a standard way to plug tools and data into an AI assistant
Token
The piece the AI reads in. It's not letters — it's chunks. The word 'unbelievable' might be three tokens. 'Mr Tan' might be one token plus another. This is why
Probability distribution
At every word, the AI computes a list: every possible next token, with a probability beside it. 'the' might be 23%, 'a' 18%, 'your' 4%, ... down to 0.0001% for
Sampling
The dice roll. The AI has a list of possible next tokens with probabilities (the distribution). Sampling is the step where it picks one. This is why you ask the
Temperature
The creativity dial. Low (0–0.3): boring, safe, repetitive, reliable. Same prompt → almost the same output. High (1.0–1.5): varied, surprising, occasionally bri
Model
The model is the AI. Specifically: a giant file of numbers (the weights — billions of them) plus the architecture that uses them to compute probabilities. When
Probability machine
The AI is not a knowledge base. It is a probability machine. It writes the next word by rolling weighted dice on what 'most likely comes next.' That is the mech
Non-determinism
Same input → DIFFERENT output. That is the AI half of the 98/2. When you ask Roost the same question twice you can get two different replies — because the next
Eloquence trap
AI is fluent before it is correct. The smoother the output, the MORE you must check. My key message for the day.
Hallucination
AI makes things up. Confidently. Without flagging. Not malice, not a bug — the mechanism of the probability machine, working as designed, sampling tokens that '
Sycophancy
AI tells you what you want to hear. If you start the prompt with "Mrs Lim is clearly a moderate-risk client, draft her a balanced portfolio recommendation," AI
Context window
How much text an LLM can see at once — including your prompt and its own reply. If your client chat history exceeds this limit, the model will lose track of e
Cost-per-token
What each token of input or output costs you — measured in fractions of a cent. High-volume automated tasks like parsing hundreds of client emails can accumul
Embedding
A numeric fingerprint of meaning — turns text into coordinates so similar things land near each other. This is how Roost finds the right policy document when
Few-shot
Show the AI 3-5 examples in the prompt; it pattern-matches the next case. This is the most effective way to ensure the model adopts your specific firm's tone
Fine-tuning
Retraining a model on your own examples to bias it toward a domain. This is rarely necessary for FAs; usually, a well-crafted system prompt with few-shot exam
Function calling
When the LLM picks a tool from a list and emits the arguments to call it — text in, structured action out. This allows Roost to actually perform tasks like ad
Inference
Running the model to get an answer — the live use of a trained model. Every time you hit "send" in Roost, you are triggering an inference cycle.
MCP
Model Context Protocol — the standard for plugging tools into an LLM like Claude. Roost exposes its operations over MCP, ensuring that your AI assistant can s
Prompt
The setup + the question you send the model — everything inside the input box. A well-structured prompt is the difference between a hallucination and a useful
RAG
Retrieval-Augmented Generation — look up relevant docs first, then ask the LLM. Reduces hallucination by forcing the model to base its answer on your specific
Stop sequence
A string that tells the model 'when you write this, stop'. Useful for preventing the model from rambling or generating unwanted boilerplate text after it has
System prompt
The instructions the operator gives the model BEFORE the customer's message — invisible to the customer. Where you tell Roost 'never recommend a product; alwa
Tokenisation
The step before inference — your text gets chopped into tokens the model recognises. Because the model doesn't read words or letters, it must first translate
Tool use
When an LLM calls a tool to act in the world (send email, query DB) rather than just produce text. This turns the model from a passive "eloquence trap" into a
Weights
The billions of numbers learned during training — the 'soul' of the model on disk. These weights are what make the model a probability machine; they are stati
Calibration
How well a model's stated confidence tracks actual correctness. LLMs are typically over-confident, meaning they often present incorrect information with high ce
Eloquence
The polish of the prose. Separable from whether it's correct. The model sounds confident whether it's right or wrong. That gap is where the Eloquence trap lives
Latency
How long the model takes to reply — typically 1-30 seconds depending on length. High latency can disrupt real-time client interactions if the FA is waiting for
Stochasticity
Built-in randomness. The technical word for 'not deterministic'. This means you cannot expect the exact same output twice, even with the same prompt.
Cost-per-token (property)
AI inference has a per-use cost. Different from owning software outright. Plan for it. Every interaction consumes resources that scale with the length of your p
Cost overrun
AI calls cost money. Loops, retries, or long contexts can spiral fast — Project Vend, June 2025. In your FA practice, this looks like an automated agent stuck i
Data poisoning
Injecting bad examples into a model's training data so it learns the wrong thing. In your FA practice, this looks like using unverified 'open-source' datasets t
Jailbreak
Tricking a model past its safety rules — 'pretend you are an unrestricted AI'. In your FA practice, this looks like a user bypassing your compliance guardrails
Mode collapse
When a model gives near-identical outputs across varied prompts. Suggests over-fine-tuning. In your FA practice, this looks like your internal 'expert' bot givi
Slopsquatting
Attackers publish malicious packages with names that LLMs hallucinate. huggingface-cli, react-codeshift. In your FA practice, this looks like an AI suggesting a
Large Language Model
An AI trained on huge amounts of text to predict the next word — that one skill produces fluent language.
Transformer
The architecture behind modern AI: attention lets it weigh every word against every other, in parallel.
Neural network
Math loosely modelled on the brain — layers of weighted connections tuned by data.
Machine learning
Software that learns patterns from examples instead of being hand-coded.
Deep learning
Machine learning with many-layered neural networks.
Generative AI
AI that creates new content, rather than just labelling existing data.
Foundation model
One big pre-trained model adapted to many tasks; the most capable generation is the 'frontier'.
Training
Tuning the model's internal numbers on data so it gets better at the task.
Parameters
The billions of learned numbers that store what a model knows.
Attention mechanism
How the model decides which words matter to which.
Agent
AI that takes multi-step actions with tools to reach a goal, not just answer once.
Agentic AI
AI that acts autonomously over many steps — it plans, uses tools, and self-corrects.
Alignment
Getting AI to actually pursue what we intend, not a convenient proxy.
RLHF
Training AI on human preference ratings so its answers match what people want.
Multimodal
AI that handles more than text — images, audio, video too.
Reasoning model
An LLM that works through steps before answering, trading compute for better logic.
Mixture of Experts
Many expert sub-models; only a few fire per token — big capacity at lower cost.
Chain-of-thought
Making the model show its working improves its answers.
Diffusion model
Image AI that turns random noise into a picture, step by step.
Zero-shot
Doing a task with no examples given — just the instruction.
Prompt engineering
Crafting the input to reliably get the output you want.
Grounding
Anchoring answers to real sources so the model can't just make things up.
Vector database
A database that finds things by meaning, not keywords — it powers RAG.
Knowledge cutoff
The model knows nothing after its training date — unless you tell it.
Quantization
Shrinking a model by storing its numbers at lower precision.
Distillation
Teaching a small model to copy a big one.
AGI
AI as broadly capable as a human across most tasks — still hypothetical.
Test-time compute
Letting the model think longer at answer-time to get better results.
Reinforcement learning
Learning by reward and penalty through trial and error.
Natural language processing
Getting computers to understand and produce human language.
Reconnaissance
The adversary is trying to gather information about the AI system they can use to plan future operations. Reconnaissance consists of techniques that involve ad
Resource Development
The adversary is trying to establish resources they can use to support operations. Resource Development consists of techniques that involve adversaries creatin
Initial Access
The adversary is trying to gain access to the AI system. The target system could be a network, mobile device, or an edge device such as a sensor platform. The
AI Model Access
The adversary is attempting to gain some level of access to an AI model. AI Model Access enables techniques that use various types of access to the AI model th
Execution
The adversary is trying to run malicious code embedded in AI artifacts or software. Execution consists of techniques that result in adversary-controlled code r
Persistence
The adversary is trying to maintain their foothold via AI artifacts or software. Persistence consists of techniques that adversaries use to keep access to syst
Privilege Escalation
The adversary is trying to gain higher-level permissions. Privilege Escalation consists of techniques that adversaries use to gain higher-level permissions on
Defense Evasion
The adversary is trying to avoid being detected by AI-enabled security software. Defense Evasion consists of techniques that adversaries use to avoid detection
Credential Access
The adversary is trying to steal account names and passwords. Credential Access consists of techniques for stealing credentials like account names and password
Discovery
The adversary is trying to figure out your AI environment. Discovery consists of techniques an adversary may use to gain knowledge about the system and interna
Lateral Movement
The adversary is trying to move through your AI environment. Lateral Movement consists of techniques that adversaries may use to gain access to and control oth
Collection
The adversary is trying to gather AI artifacts and other related information relevant to their goal. Collection consists of techniques adversaries may use to g
AI Attack Staging
The adversary is leveraging their knowledge of and access to the target system to tailor the attack. AI Attack Staging consists of techniques adversaries use t
Command and Control
The adversary is trying to communicate with compromised AI systems to control them. Command and Control consists of techniques that adversaries may use to comm
Exfiltration
The adversary is trying to steal AI artifacts or other information about the AI system. Exfiltration consists of techniques that adversaries may use to steal d
Impact
The adversary is trying to manipulate, interrupt, erode confidence in, or destroy your AI systems and data. Impact consists of techniques that adversaries use
Sensitive Information Disclosure
When an LLM application reveals confidential data — PII, credentials, proprietary content, or other users' data — through its outputs, e.g. by regurgitating tra
Improper Output Handling
Insufficient validation, sanitisation, or encoding of LLM output before it is passed to downstream systems, enabling XSS, SSRF, SQL injection, or remote code ex
Excessive Agency
Harm caused by granting an LLM-based system too much functionality, permission, or autonomy, so that unexpected or manipulated outputs trigger damaging real-wor
System Prompt Leakage
Exposure of the hidden system prompt or instructions governing an LLM, which can reveal secrets, business logic, or guardrail design that attackers then bypass.
Vector and Embedding Weaknesses
Vulnerabilities in the embeddings and vector stores used for RAG — poisoning, leakage across tenants, or injection via retrieved content — that corrupt or exfil
Unbounded Consumption
Allowing uncontrolled, resource-intensive LLM operations (long generations, recursion, high query volume) that cause denial-of-service, runaway cost, or model e
Model Poisoning
Tampering with training data, fine-tuning data, or model weights to implant backdoors, biases, or vulnerabilities that surface at inference time.
Indirect Prompt Injection
Prompt injection delivered through content the model ingests (a web page, document, or tool output) rather than the user's message, hijacking the model via untr
Adversarial example
An input crafted with small, often imperceptible perturbations that causes a model to misclassify or misbehave, exploiting the brittleness of learned decision b
Model inversion
An attack that reconstructs sensitive training inputs (e.g. faces, records) by probing a model's outputs, breaching the privacy of the training data.
Membership inference
Determining whether a specific record was part of a model's training set by analysing its outputs — a privacy attack against the training data.
Model theft
Extracting or replicating a proprietary model — its weights or behaviour — through exfiltration or query-based distillation, stealing the IP and enabling furthe
Agent hijacking
Subverting an autonomous AI agent's goal or actions — via prompt injection, poisoned tools, or manipulated memory — so it acts on the attacker's behalf.
AI red teaming
The structured practice of adversarially testing an AI system — eliciting jailbreaks, harmful outputs, and failures — to find and fix weaknesses before attacker
Shadow AI
Unsanctioned use of AI tools and models inside an organisation, outside IT/security oversight, creating data-leakage, compliance, and supply-chain risk.
Supervised learning
Learning from labelled examples — input paired with the right answer.
Unsupervised learning
Finding structure in data with no labels.
Self-supervised learning
The model makes its own labels from raw data — how LLMs pre-train.
Overfitting
When a model memorises the training data instead of the pattern, so it fails on new data.
Underfitting
When a model is too simple to capture the pattern.
Gradient descent
The optimisation that nudges a model's numbers downhill toward lower error.
Backpropagation
The method that works out how to adjust every weight to reduce error.
Loss function
The score of how wrong the model is — training shrinks it.
Hyperparameter
A training setting you choose rather than learn — like learning rate or model size.
Bias-variance tradeoff
The balance between a model too simple (bias) and one too sensitive to its data (variance).
Feature
An input signal the model uses; choosing good ones is feature engineering.
Labelled data
Examples tagged with the right answer — the fuel for supervised learning.
Classification
Sorting inputs into categories.
Regression
Predicting a number.
Cross-validation
Testing a model on held-out slices of data to check it generalises.
Precision and recall
How many flagged were right (precision) vs how many real ones were caught (recall).
Convolutional neural network
A neural network design tuned for images.
Recurrent neural network
A neural network for sequences, step by step (largely superseded by Transformers).
Generative adversarial network
Two networks competing — one generates, one critiques — to produce realistic data.
Transfer learning
Reusing a model trained on one task as a starting point for another.
Search Open Technical Databases
Adversaries may search for publicly available research and technical documentation to learn how and where AI is used within a victim organization. The adversary
Journals and Conference Proceedings
Many of the publications accepted at premier artificial intelligence conferences and journals come from commercial labs. Some journals and conferences are open
Pre-Print Repositories
Pre-Print repositories, such as arXiv, contain the latest academic research papers that haven't been peer reviewed. They may contain research notes, or technica
Technical Blogs
Research labs at academic institutions and company R&D divisions often have blogs that highlight their use of artificial intelligence and its application to the
Search Open AI Vulnerability Analysis
Much like the [Search Open Technical Databases](/techniques/AML.T0000), there is often ample research available on the vulnerabilities of common AI models. Once
Search Victim-Owned Websites
Adversaries may search websites owned by the victim for information that can be used during targeting. Victim-owned websites may contain technical details about
Search Application Repositories
Adversaries may search open application repositories during targeting. Examples of these include Google Play, the iOS App store, the macOS App Store, and the Mi
Active Scanning
An adversary may probe or scan the victim system to gather information for targeting. This is distinct from other reconnaissance techniques that do not involve
Acquire Public AI Artifacts
Adversaries may search public sources, including cloud storage, public-facing services, and software or data repositories, to identify AI artifacts. These AI ar
Datasets
Adversaries may collect public datasets to use in their operations. Datasets used by the victim organization or datasets that are representative of the data use
Models
Adversaries may acquire public models to use in their operations. Adversaries may seek models used by the victim organization or models that are representative
Obtain Capabilities
Adversaries may search for and obtain software capabilities for use in their operations. Capabilities may be specific to AI-based attacks [Adversarial AI Attack
Adversarial AI Attack Implementations
Adversaries may search for existing open source implementations of AI attacks. The research community often publishes their code for reproducibility and to furt
Software Tools
Adversaries may search for and obtain software tools to support their operations. Software designed for legitimate use may be repurposed by an adversary for mal
Develop Capabilities
Adversaries may develop their own capabilities to support operations. This process encompasses identifying requirements, building solutions, and deploying capab
Adversarial AI Attacks
Adversaries may develop their own adversarial attacks. They may leverage existing libraries as a starting point ([Adversarial AI Attack Implementations](/techni
Acquire Infrastructure
Adversaries may buy, lease, or rent infrastructure for use throughout their operation. A wide variety of infrastructure exists for hosting and orchestrating adv
AI Development Workspaces
Developing and staging AI attacks often requires expensive compute resources. Adversaries may need access to one or many GPUs in order to develop an attack. The
Consumer Hardware
Adversaries may acquire consumer hardware to conduct their attacks. Owning the hardware provides the adversary with complete control of the environment. These d
Publish Poisoned Datasets
Adversaries may [Poison Training Data](/techniques/AML.T0020) and publish it to a public location. The poisoned dataset may be a novel dataset or a poisoned var
AI Supply Chain Compromise
Adversaries may gain initial access to a system by compromising the unique portions of the AI supply chain. This could include [Hardware](/techniques/AML.T0010.
Hardware
Adversaries may target AI systems by disrupting or manipulating the hardware supply chain. AI models often run on specialized hardware such as GPUs, TPUs, or em
AI Software
Adversaries may target software packages that are commonly used in AI-enabled systems or are part of the AI DevOps lifecycle. This can include deep learning fra
Data
Data is a key vector of supply chain compromise for adversaries. Every AI project will require some form of data. Many rely on large open source datasets that a
Model
AI-enabled systems often rely on open sourced models in various ways. Most commonly, the victim organization may be using these models for fine tuning. These mo
AI Model Inference API Access
Adversaries may gain access to a model via legitimate access to the inference API. Inference API access can be a source of information to the adversary ([Discov
AI-Enabled Product or Service
Adversaries may use a product or service that uses artificial intelligence under the hood to gain access to the underlying AI model. This type of indirect model
Physical Environment Access
In addition to the attacks that take place purely in the digital domain, adversaries may also exploit the physical environment for their attacks. If the model i
Full AI Model Access
Adversaries may gain full "white-box" access to an AI model. This means the adversary has complete knowledge of the model architecture, its parameters, and clas
Discover AI Model Ontology
Adversaries may discover the ontology of an AI model's output space, for example, the types of objects a model can detect. The adversary may discovery the ontol
Discover AI Model Family
Adversaries may discover the general family of model. General information about the model may be revealed in documentation, or the adversary may use carefully c
Poison Training Data
Adversaries may attempt to poison datasets used by an AI model by modifying the underlying data or its labels. This allows the adversary to embed vulnerabilitie
Establish Accounts
Adversaries may create accounts with various services for use in targeting, to gain access to resources needed in [AI Attack Staging](/tactics/AML.TA0001), or f
Create Proxy AI Model
Adversaries may obtain models to serve as proxies for the target model in use at the victim organization. Proxy models are used to simulate complete access to t
Train Proxy via Gathered AI Artifacts
Proxy models may be trained from AI artifacts (such as data, model architectures, and pre-trained models) that are representative of the target model gathered b
Train Proxy via Replication
Adversaries may replicate a private model. By repeatedly querying the victim's [AI Model Inference API Access](/techniques/AML.T0040), the adversary can collect
Use Pre-Trained Model
Adversaries may use an off-the-shelf pre-trained model as a proxy for the victim model to aid in staging the attack.
Discover AI Artifacts
Adversaries may search private sources to identify AI learning artifacts that exist on the system and gather information about them. These artifacts can include
User Execution
An adversary may rely upon specific actions by a user in order to gain execution. Users may inadvertently execute unsafe code introduced via [AI Supply Chain Co
Unsafe AI Artifacts
Adversaries may develop unsafe AI artifacts that when executed have a deleterious effect. The adversary can use this technique to establish persistent access to
Valid Accounts
Adversaries may obtain and abuse credentials of existing accounts as a means of gaining Initial Access. Credentials may take the form of usernames and passwords
Evade AI Model
Adversaries can [Craft Adversarial Data](/techniques/AML.T0043) that prevents an AI model from correctly identifying the contents of the data or [Generate Deepf
Manipulate AI Model
Adversaries may directly manipulate an AI model to change its behavior or introduce malicious code. Manipulating a model gives the adversary a persistent change
Poison AI Model
Adversaries may manipulate an AI model's weights to change it's behavior or performance, resulting in a poisoned model. Adversaries may poison a model by direct
Modify AI Model Architecture
Adversaries may directly modify an AI model's architecture to re-define it's behavior. This can include adding or removing layers as well as adding pre or post-
Exfiltration via AI Inference API
Adversaries may exfiltrate private information via [AI Model Inference API Access](/techniques/AML.T0040). AI Models have been shown leak private information ab
Infer Training Data Membership
Adversaries may infer the membership of a data sample or global characteristics of the data in its training set, which raises privacy concerns. Some strategies
Invert AI Model
AI models' training data could be reconstructed by exploiting the confidence scores that are available via an inference API. By querying the inference API strat
Extract AI Model
Adversaries may extract a functional copy of a private model. By repeatedly querying the victim's [AI Model Inference API Access](/techniques/AML.T0040), the ad
Exfiltration via Cyber Means
Adversaries may exfiltrate AI artifacts or other information relevant to their goals via traditional cyber means. See the ATT&CK [Exfiltration](https://attack.
Denial of AI Service
Adversaries may target AI-enabled systems with a flood of requests for the purpose of degrading or shutting down the service. Since many AI systems require sign
Spamming AI System with Chaff Data
Adversaries may spam the AI system with chaff data that causes increase in the number of detections. This can cause analysts at the victim organization to waste
Erode AI Model Integrity
Adversaries may degrade the target model's performance with adversarial data inputs to erode confidence in the system over time. This can lead to the victim org
Cost Harvesting
Adversaries may deliberately drive a victim's AI services beyond normal operating capacity with the intent of increasing the cost of services. This may be achie
AI Artifact Collection
Adversaries may collect AI artifacts for [Exfiltration](/tactics/AML.TA0010) or for use in [AI Attack Staging](/tactics/AML.TA0001). AI artifacts include models
Data from Information Repositories
Adversaries may leverage information repositories to mine valuable information. Information repositories are tools that allow for storage of information, typica
Data from Local System
Adversaries may search local system sources, such as file systems and configuration files or local databases, to find files of interest and sensitive data prior
Verify Attack
Adversaries can verify the efficacy of their attack via an inference API or access to an offline copy of the target model. This gives the adversary confidence t
Craft Adversarial Data
Adversarial data are inputs to an AI model that have been modified such that they cause the adversary's desired effect in the target model. Effects can range fr
White-Box Optimization
In White-Box Optimization, the adversary has full access to the target model and optimizes the adversarial example directly. Adversarial examples trained in thi
Black-Box Optimization
In Black-Box attacks, the adversary has black-box (i.e. [AI Model Inference API Access](/techniques/AML.T0040) via API access) access to the target model. With
Black-Box Transfer
In Black-Box Transfer attacks, the adversary uses one or more proxy models (trained via [Create Proxy AI Model](/techniques/AML.T0005) or [Train Proxy via Repli
Manual Modification
Adversaries may manually modify the input data to craft adversarial data. They may use their knowledge of the target model to modify parts of the data they susp
Insert Backdoor Trigger
The adversary may add a perceptual trigger into inference data. The trigger may be imperceptible or non-obvious to humans. This technique is used in conjunction
External Harms
Adversaries may abuse their access to a victim system and use its resources or capabilities to further their goals by causing harms external to that system. The
Financial Harm
Financial harm involves the loss of wealth, property, or other monetary assets due to theft, fraud or forgery, or pressure to provide financial resources to the
Reputational Harm
Reputational harm involves a degradation of public perception and trust in organizations. Examples of reputation-harming incidents include scandals or false im
Societal Harm
Societal harms might generate harmful outcomes that reach either the general public or specific vulnerable groups such as the exposure of children to vulgar con
User Harm
User harms may encompass a variety of harm types including financial and reputational that are directed at or felt by individual victims of the attack rather th
AI Intellectual Property Theft
Adversaries may exfiltrate AI artifacts to steal intellectual property and cause economic harm to the victim organization. Proprietary training data is costly
Exploit Public-Facing Application
Adversaries may attempt to take advantage of a weakness in an Internet-facing computer or program using software, data, or commands in order to cause unintended
Command and Scripting Interpreter
Adversaries may abuse command and script interpreters to execute commands, scripts, or binaries. These interfaces and languages provide ways of interacting with
LLM Prompt Injection
An adversary may craft malicious prompts as inputs to an LLM that cause the LLM to act in unintended ways. These "prompt injections" are often designed to cause
Direct
An adversary may inject prompts directly as a user of the LLM. This type of injection may be used by the adversary to gain a foothold in the system or to misuse
Indirect
An adversary may inject prompts indirectly via separate data channel ingested by the LLM such as include text or multimedia pulled from databases or websites. T
Phishing
Adversaries may send phishing messages to gain access to victim systems. All forms of phishing are electronically delivered social engineering. Phishing can be
Spearphishing via Social Engineering LLM
Adversaries may turn LLMs into targeted social engineers. LLMs are capable of interacting with users via text conversations. They can be instructed by an advers
AI Agent Tool Invocation
Adversaries may use their access to an AI agent to invoke tools the agent has access to. LLMs are often connected to other services or resources via tools to in
LLM Jailbreak
Adversaries may induce a large language model (LLM) to ignore, circumvent, or override its safety/alignment behaviors and/or guardrails to elicit outputs the mo
Unsecured Credentials
Adversaries may search compromised systems to find and obtain insecurely stored credentials. These credentials can be stored and/or misplaced in many locations
Extract LLM System Prompt
Adversaries may attempt to extract a large language model's (LLM) system prompt. This can be done via prompt injection to induce the model to reveal its own sys
LLM Data Leakage
Adversaries may craft prompts that induce the LLM to leak sensitive information. This can include private user data or proprietary information. The leaked infor
Publish Poisoned Models
Adversaries may publish a poisoned model to a public location such as a model registry or code repository. The poisoned model may be a novel model or a poisoned
Erode Dataset Integrity
Adversaries may poison or manipulate portions of a dataset to reduce its usefulness, reduce trust, and cause users to waste resources correcting errors.
Malicious Package
Adversaries may develop malicious software packages that when imported by a user have a deleterious effect. Malicious packages may behave as expected to the use
Publish Hallucinated Entities
Adversaries may create an entity they control, such as a software package, website, or email address to a source hallucinated by an LLM. The hallucinations may
LLM Prompt Self-Replication
An adversary may use a carefully crafted [LLM Prompt Injection](/techniques/AML.T0051) designed to cause the LLM to replicate the prompt as part of its output.
Discover LLM Hallucinations
Adversaries may prompt large language models and identify hallucinated entities. They may request software packages, commands, URLs, organization names, or e-ma
Domains
Adversaries may acquire domains that can be used during targeting. Domain names are the human readable names used to represent one or more IP addresses. They ca
Physical Countermeasures
Adversaries may acquire or manufacture physical countermeasures to aid or support their attack. These components may be used to disrupt or degrade the model, s
Discover AI Model Outputs
Adversaries may discover model outputs, such as class scores, whose presence is not required for the system to function and are not intended for use by the end
Generative AI
Adversaries may search for and obtain generative AI models or tools, such as large language models (LLMs), to assist them in various steps of their operation. G
Gather RAG-Indexed Targets
Adversaries may identify data sources used in retrieval augmented generation (RAG) systems for targeting purposes. By pinpointing these sources, attackers can f
LLM Prompt Crafting
Adversaries may use their acquired knowledge of the target generative AI system to craft prompts that bypass its defenses and allow malicious instructions to be
Retrieval Content Crafting
Adversaries may write content designed to be retrieved by user queries and influence a user of the system in some way. This abuses the trust the user has in the
LLM Trusted Output Components Manipulation
Adversaries may utilize prompts to a large language model (LLM) which manipulate various components of its response in order to make it appear trustworthy to th
LLM Prompt Obfuscation
Adversaries may hide or otherwise obfuscate prompt injections or retrieval content to avoid detection from humans, large language model (LLM) guardrails, or oth
Discover LLM System Information
The adversary is trying to discover something about the large language model's (LLM) system information. This may be found in a configuration file containing th
Special Character Sets
Adversaries may discover delimiters and special characters sets used by the large language model. For example, delimiters used in retrieval augmented generation
System Instruction Keywords
Adversaries may discover keywords that have special meaning to the large language model (LLM), such as function names or object names. These can later be exploi
System Prompt
Adversaries may discover a large language model's system instructions provided by the AI system builder to learn about the system's capabilities and circumvent
RAG Poisoning
Adversaries may inject malicious content into data indexed by a retrieval augmented generation (RAG) system to contaminate a future thread through RAG-based sea
False RAG Entry Injection
Adversaries may introduce false entries into a victim's retrieval augmented generation (RAG) database. Content designed to be interpreted as a document by the l
Citations
Adversaries may manipulate the citations provided in an AI system's response, in order to make it appear trustworthy. Variants include citing a providing the wr
Embed Malware
Adversaries may embed malicious code into AI Model files. AI models may be packaged as a combination of instructions and weights. Some formats such as pickle fi
Container Registry
An adversary may compromise a victim's container registry by pushing a manipulated container image and overwriting an existing container name and/or tag. Users
Reverse Shell
Adversaries may utilize a reverse shell to communicate and control the victim system. Typically, a user uses a client to connect to a remote machine which is l
Impersonation
Adversaries may impersonate a trusted person or organization in order to persuade and trick a target into performing some action on their behalf. For example, a
Masquerading
Adversaries may attempt to manipulate features of their artifacts to make them appear legitimate or benign to users and/or security tools. Masquerading occurs w
Cloud Service Discovery
Adversaries may attempt to enumerate the cloud services running on a system after gaining access. These methods can differ from platform-as-a-service (PaaS), to
Corrupt AI Model
An adversary may purposefully corrupt a malicious AI model file so that it cannot be successfully deserialized in order to evade detection by a model scanner. T
LLM Response Rendering
An adversary may get a large language model (LLM) to respond with private information that is hidden from the user when the response is rendered by the user's c
Serverless
Adversaries may purchase and configure serverless cloud infrastructure, such as Cloudflare Workers, AWS Lambda functions, or Google Apps Scripts, that can be us
Drive-by Compromise
Adversaries may gain access to an AI system through a user visiting a website over the normal course of browsing, or an AI agent retrieving information from the
Stage Capabilities
Adversaries may upload, install, or otherwise set up capabilities that can be used during targeting. To support their operations, an adversary may need to take
AI Agent Context Poisoning
Adversaries may attempt to manipulate the context used by an AI agent's large language model (LLM) to influence the responses it generates or actions it takes.
Memory
Adversaries may manipulate the memory of a large language model (LLM) in order to persist changes to the LLM to future chat sessions. Memory is a common featu
Thread
Adversaries may introduce malicious instructions into a chat thread of a large language model (LLM) to cause behavior changes which persist for the remainder of
Modify AI Agent Configuration
Adversaries may modify the configuration files for AI agents on a system. This allows malicious changes to persist beyond the life of a single agent and affects
RAG Credential Harvesting
Adversaries may attempt to use their access to a large language model (LLM) on the victim's system to collect credentials. Credentials may be stored in internal
Credentials from AI Agent Configuration
Adversaries may access the credentials of other tools or services on a system from the configuration of an AI agent. AI Agents often utilize external tools or
Discover AI Agent Configuration
Adversaries may attempt to discover configuration information for AI agents present on the victim's system. Agent configurations can include tools or services t
Embedded Knowledge
Adversaries may attempt to discover the data sources a particular agent can access. The AI agent's configuration may reveal data sources or knowledge. The emb
Tool Definitions
Adversaries may discover the tools the AI agent has access to. By identifying which tools are available, the adversary can understand what actions may be execut
Activation Triggers
Adversaries may discover keywords or other triggers (such as incoming emails, documents being added, incoming message, or other workflows) that activate an agen
Data from AI Services
Adversaries may use their access to a victim organization's AI-enabled services to collect proprietary or otherwise sensitive information. As organizations adop
RAG Databases
Adversaries may prompt the AI service to retrieve data from a RAG database. This can include the majority of an organization's internal documents.
AI Agent Tools
Adversaries may prompt the AI service to invoke various tools the agent has access to. Tools may retrieve data from different APIs or services in an organizatio
Exfiltration via AI Agent Tool Invocation
AI agent tools capable of performing write operations may be invoked to exfiltrate data to an adversary. Sensitive information can be encoded into the tool's in
Gather Victim Identity Information
Adversaries may gather information about the victim's identity that can be used during targeting. Information about identities may include a variety of details,
Generate Deepfakes
Adversaries may use generative artificial intelligence (GenAI) to create synthetic media (i.e. imagery, video, audio, and text) that appear authentic. These "[d
Process Discovery
Adversaries may attempt to get information about processes running on a system. Once obtained, this information could be used to gain an understanding of common
OS Credential Dumping
Adversaries may extract credentials from OS caches, application memory, or other sources on a compromised system. Credentials are often in the form of a hash or
Use Alternate Authentication Material
Adversaries may use alternate authentication material, such as password hashes, Kerberos tickets, and application access tokens, in order to move laterally with
Manipulate User LLM Chat History
Adversaries may manipulate a user's large language model (LLM) chat history to cover the tracks of their malicious behavior. They may hide persistent changes th
Application Access Token
Adversaries may use stolen application access tokens to bypass the typical authentication process and access restricted accounts, information, or services on re
Prompt Infiltration via Public-Facing Application
An adversary may introduce malicious prompts into the victim's system via a public-facing application with the intention of it being ingested by an AI at some p
Delay Execution of LLM Instructions
Adversaries may include instructions to be followed by the AI system in response to a future event, such as a specific keyword or the next interaction, in order
Triggered
An adversary may trigger a prompt injection via a user action or event that occurs within the victim's environment. Triggered prompt injections often target AI
Search Open Websites/Domains
Adversaries may search public websites and/or domains for information about victims that can be used during targeting. Information about victims may be availabl
AI Service API
Adversaries may communicate using the API of an AI service on the victim's system. The adversary's commands to the victim system, and often the results, are emb
Virtualization/Sandbox Evasion
Adversaries may employ various means to detect and avoid virtualization and analysis environments. This may include changing behaviors based on the results of c
AI Agent Tool Credential Harvesting
Adversaries may attempt to use their access to an AI agent on the victim's system to retrieve data from available agent tools to collect credentials. Agent tool
AI Agent Tool Data Poisoning
Adversaries may place malicious content on a victim's system where it can be retrieved by an AI Agent Tool. This may be accomplished by placing documents in a l
AI Agent Clickbait
Adversaries may craft deceptive web content designed to bait Computer-Using AI agents or AI web browsers into taking unintended actions, such as clicking button
Data Destruction via AI Agent Tool Invocation
Adversaries may invoke an AI agent's tool capable of performing mutative operations to perform Data Destruction. Adversaries may destroy data and files on speci
Generate Malicious Commands
Adversaries may use large language models (LLMs) to dynamically generate malicious commands from natural language. Dynamically generated commands may be harder
Deploy AI Agent
Adversaries may launch AI agents in the victim's environment to execute actions on their behalf. AI agents may have access to a wide range of tools and data sou
Publish Poisoned AI Agent Tool
Adversaries may create and publish poisoned AI agent tools. Poisoned tools may contain an [LLM Prompt Injection](/techniques/AML.T0051), which can lead to a var
Poisoned AI Agent Tool
A victim may invoke a poisoned tool when interacting with their AI agent. A poisoned tool may execute an [LLM Prompt Injection](/techniques/AML.T0051) or perfor
Malicious Link
An adversary may rely upon a user clicking a malicious link in order to gain execution. Users may be subjected to social engineering to get them to click on a l
Escape to Host
Adversaries may break out of a container or virtualized environment to gain access to the underlying host. This can allow an adversary access to other container
Exploitation for Credential Access
Adversaries may exploit software vulnerabilities in an attempt to collect credentials. Exploitation of a software vulnerability occurs when an adversary takes a
Exploitation for Defense Evasion
Adversaries may exploit a system or application vulnerability to bypass security features. Exploitation of a vulnerability occurs when an adversary takes advant
AI Agent
Adversaries may abuse AI agents present on the victim's system for command and control. AI agents are often granted access to tools that can execute shell comma
AI Supply Chain Rug Pull
Adversaries may publish legitimate AI components or software, gain user adoption, then push an update with a malicious variant, leading to [AI Supply Chain Comp
AI Agent Tool Poisoning
Adversaries may achieve persistence by poisoning tools used by AI agents including built-in tools or tools available to the agent via Model Context Protocol (MC
AI Agent Tool
Adversaries may target AI agent tools as a means to compromise a victim's AI supply chain. Tools add capabilities to AI agents, allowing them to interact with o
Excessive Queries
Adversaries may send an excessive number of otherwise normal or low-complexity queries to an AI system with the goal of overwhelming its capacity and increasing
Resource-Intensive Queries
Adversaries may craft inputs specifically designed to increase the compute resources required for processing. For generative AI models, adversaries may use lon
Agentic Resource Consumption
Adversaries may coerce an agentic AI system into performing computationally expensive tool calls that waste resources and consume API budgets. They may utilize
Call Chains
Adversaries may extract call chains from AI agent configurations, which can reveal potentially targets for remote code execution (RCE) or other vulnerabilities.
AI Supply Chain Reputation Inflation
AI Supply Chain Reputation Inflation is the process of building or leveraging genuinely credible-looking trust signals to increase the perceived legitimacy of A
Machine Compromise
Adversaries may compromise a machine by exploiting or manipulating AI-enabled components on the system. Compromising a victim system allows the adversary to exe
AI Service Proxies
Adversaries may utilize commercial proxy services that resell access to AI services such as frontier model APIs. This infrastructure can be used to conduct lar
Local AI Agent
Adversaries may achieve full system compromise by abusing AI agents running locally on a host, such as computer-use agents or AI-driven browsers. These agents a
AI Artifacts
Adversaries may achieve full system compromise by introducing malicious AI artifacts, such as models or data, that contain embedded malware or other malicious c
Deepfake-Assisted Phishing
Adversaries may use deepfakes (AI-generated synthetic images, audio, or video) in phishing campaigns to impersonate trusted individuals, executives, or organiza
AI Agent Configuration
Adversaries may acquire publicly accessible AI agent configuration files to understand agent capabilities, gain unauthorized access to tools and data sources, o
Code Repositories
Adversaries may search public code repositories for information about a victim or victim system that can be used during targeting. Victims may store code or art
Substrate
The earned, tacit core of capability — domain judgment built over years that can't be written down, downloaded, or outsourced.
Surface
The visible, learnable layer — tools, interfaces, workflows, vocabulary. Fast to learn, fast to replace, useless without substrate.
Epistemic Labour
The work of knowing whether AI output is true and sound — reading meaning and nuance, not just fact-checking. The labour AI makes scarce.
Intellectual Labour
Weightless cognitive production — drafting, coding, analysing. The labour AI is commoditising now.
Accountability Labour
Owning the decision and bearing the consequence — the one thing AI can't do and you can't delegate.
Architectural Labour
Building the systems through which AI operates — the growth category where the new jobs live.
The Squeezed Middle
AI lifts novices and supercharges the few, but hollows the mid-level rung where experts were actually made — breaking the pipeline.
Power Law Divergence
AI doesn't lift everyone evenly — outcomes split into a power law: the few who pair judgment with AI-direction pull far ahead.
The Moat Inversion
For two centuries value flowed to the explicit and scalable; AI exhausts that, so the only moat left is the irreducibly human.
Concretisation
Using AI to turn an organisation's tacit know-how into explicit, proprietary, scalable assets — finally codifying 'how we really do it'.
Commoditisation Resistance
The practices that keep human substrate visible enough to pay for — so it isn't commoditised away.
AI Theatre
High adoption, high confidence, zero verification — an organisation that looks like it's 'doing AI' while granting Epistemic Credit to unverified output.
Workslop
AI-generated content that looks like good work but lacks substance — and creates more work for others to fix.
Capability Transfer
The third path: hire experts to BUILD the capability inside your organisation, not to do the work for you.
The Forge
A third model of professional education for when experts are scarce and content is free — deliberately rebuilding how substrate gets made.
AI Guildhall Studio
A physical supervised-practice space where people do real, stakes-bearing AI work under expert eyes — and a deliberate third space.
Adjacency Pathway
A fast track for mid-career domain experts to port their existing judgment into AI work — months, not years.
The Knowledge Layer
The structured, curated knowledge an AI system runs on — your codified context, owned by you.
The Awareness Model
C4AIL's map of AI consciousness: four tiers (Unaware, User, Amplifier, Orchestrator) across seven maturity levels (L0-L6).
Epistemic Credit
The unearned trust people grant to AI's fluent output because it sounds authoritative.
Eloquence Trap
Mistaking fluent, confident AI output for correct output — the single most dangerous AI failure.
Verification Capacity
The competency AI adoption requires MORE of, not less: judging AI output — catching the error, supplying missing context, engaging rather than deferring.
Decision Survivability
The real test of AI maturity: can you defend the decision after it goes wrong?
Minimum Viable Literacy
Enough AI understanding for a leader to commission, interrogate, and challenge expert advice — not to become the expert.
Translator
A leader who makes AI's infrastructure-level reality legible to decision-makers — bridging what the tech does and what the business must decide.
Orchestrator
The rare practitioner who designs the AI capability others amplify, and protects control as it scales.
Human Mirror
Reflecting AI output against your context, intent, and experience — the human supplying the meaning the machine can't.
Sovereign Command
Human authority kept above machine fluency — the organisation owns, can defend, and can scale its AI decisions without losing control.
ARGS
The four pillars of AI sovereignty: Agency, Architecture, Governance, Scaling.
CAGE
The context-engineering protocol for directing AI: Context, Align, Goals, Examples.
ARCH
The agentic-execution loop made human-legible: Action, Reasoning, Contextual check, Horizon.
Logic Pipes
Structured reasoning chains that turn AI's probabilistic output into reliable, repeatable outcomes.
98/2 Principle
A reliable AI system is ~98% deterministic software (the harness you own) and ~2% probabilistic AI at the edge.
Crown Jewels
The handful of data and systems that, if lost, end you — identify and protect those first.
The Institutional Vault
C4AIL's universal name for the codified knowledge store — the explicit knowledge your AI runs on. The Vault holds and *runs the rules the Brain codified* (the d
Reliability Trap
Errors compound multiplicatively across multi-step and agentic AI chains - each step looks fine, the total does not.
Atrophy Trap
Permanently delegating judgment to AI (or to one bridge-role) makes the whole organisation progressively unable to do or judge the work.
Confidence Plateau
AI removes the visible failure that normally corrects overconfidence, so people grow more confident while growing less capable.
Comprehension Debt
The accumulating weight of decisions made by people who no longer understand the logic behind their own work.
Legibility Debt
The gap between what an organisation knows and what it has made available in a form machines can act on.
The Inverted Stack
Putting probabilistic AI where deterministic rules belong - letting the model make the call that should be hard-coded.
The Verification Bottleneck
A brilliant expert drowning in manual line-by-line review of AI output - what domain expertise without AI architecture produces.
Leverage Leaks
The three ways organisations lose even the value their best people create: Architecture, Infrastructure, and Talent leaks.
Compliance Theatre
Policies filed and training completed that satisfy the auditor without changing the outcome - the governance Eloquence Trap.
The Knowledge Paradox
The organisations that most need structured AI use are the ones least able to initialise it, because their knowledge was never made legible.
The Four Labours
Work decomposed by the human it requires: Intellectual (commoditised), Physical (lags), Accountability (the monopoly), Architectural (the growth category).
Two-Sided Risk
AI risk is the risk of adopting badly AND the competitive risk of falling behind - governance must weigh both, not only say no.
The Cost of Not Acting
The quantifiable price of inaction - lost margin, talent, and market position from moving too slowly.
Floor and Ceiling
The 90-95% who work through AI (the Floor) and the 5-10% who design and govern it (the Ceiling).
The Five Roles
Floor User, Translator, Architect, Orchestrator, Trainer - defined by labour function, not job title.
Architect
The L3-4 builder who turns expert knowledge into deterministic AI workflows - Logic Pipes, CAGE templates, verification engines.
Trainer
An Orchestrator who develops others - the role most organisations forget, and the one that keeps the pipeline alive.
Floor User
The domain expert (L0-2) working through AI-structured interfaces - the 90-95% backbone, not a lesser role.
The CAIO
The Translator made institutional - a Direct-seat Orchestrator measured by dispensability, not by being the bottleneck.
The Four Seats
One AI story told from four accountabilities: Direct (Board), Operate (Manager), Build (Practitioner), Floor (User).
Spec Loop vs Chat Loop
Fix the specification so the correction compounds (Spec Loop), instead of fixing the conversation and repeating it next time (Chat Loop).
Living Material
AI output is provisional and version-controlled - decisions get review dates, not just approval dates.
The Harness
The deterministic 98% you own and engineer - gates, autonomy tiers, audit log, kill path - wrapped around the swappable model.
The Six Numbers
The governance dashboard that measures whether AI use actually works: first-time-right, acceptance-without-verification, rework, error-classification, correctio
Bright Lines
Clear boundaries for which actions AI may take alone versus which need a human sign-off - drawn where accountability begins.
Autonomy Tiers
AI actions classified by autonomy x reversibility - irreversible or high-blast-radius actions are always-ask.
Agent-as-Principal
An AI agent is a distinct, revocable identity with scoped credentials - never a human's full rights.
The Accountability Sink
A consequential action taken under delegated authority with no human in the moment - accountability closing with no one inside it.
The Intern Test
Would you give an intern this access and let them act for you? If not, do not give it to the AI agent.
The Pre-AI Permission Audit
Tighten your file and data permissions before you deploy any AI search tool - the highest-return security action.
The Co-Creation Model
Redesign the junior role so AI does the volume and the junior does the judgment under a senior who develops their taste.
The Trainer Paradox
You need advanced practitioners to develop advanced practitioners - and they barely exist yet.
Novice Pathway
Building substrate from scratch - the slow, uncompressible 3-5 year track to Orchestrator.
The Signing Moment
The threshold where a qualified human puts their name on the work and owns it - where accountability is claimed.
GOVERN-IDENTIFY-SECURE-VALIDATE
Ethan's Practical Cybersecurity Decisions roadmap - govern the risk, identify the crown jewels, secure them, validate it works.
The Five Pillars
Practical Cyber's five domains to secure: Identity, Devices, Network, Applications, Data.
SRP
Safety, Reliability, Productivity - the Practical Cyber reframe of the CIA triad for a business audience.
The Maturity Scale (level register)
The seven AI maturity levels (L0-L6), named in TWO registers - blunt (technical/whitepaper) and soft (customer/marketing); same rung, only the label changes.