Discover LLM Hallucinations
Sometimes AI makes things up, like fake websites or software names. Bad actors can trick the AI into creating these fake items to see what it invents, which helps them figure out how to use those fake things to trick other people later.
This is the process where an adversary probes an LLM to elicit non-existent entities such as packages, URLs, or credentials. By identifying these hallucinations, an attacker can map out potential targets for follow-on exploitation, such as registering a hallucinated domain or publishing malicious packages that the model previously hallucinated.
The systematic elicitation of non-existent entities—including software artifacts, network locations, and organizational identifiers—from an LLM to identify exploitable gaps in the model's grounding. Adversaries leverage these hallucinations to facilitate downstream attacks, such as dependency confusion or typosquatting, by weaponizing the model's propensity to generate plausible but non-existent references. Because these hallucinations often exhibit cross-model consistency, they represent a systemic vulnerability that can be weaponized across multiple LLM implementations.