Generative AI
Generative AI is a type of computer program that can create new text, images, or code on its own. Just as a person might use these tools to help with work, bad actors can use them as a digital assistant to help plan or carry out cyberattacks more easily.
Generative AI refers to machine learning models, such as LLMs, capable of producing synthetic content. In an adversarial context, these tools are leveraged to automate or scale malicious activities, including phishing, malware development, and command generation. Adversaries obtain these models through public APIs, cloud hosting, or local deployment of open-source variants to bypass developer-imposed safety guardrails.
Generative AI encompasses probabilistic models, primarily transformer-based LLMs, utilized by adversaries to augment the operational lifecycle of cyberattacks. These models facilitate techniques such as content crafting, code generation, and deepfake creation. Adversarial exploitation involves the acquisition of models via cloud services or local inference frameworks, often necessitating the circumvention of alignment protocols through jailbreaking, fine-tuning for guardrail removal, or the deployment of intentionally 'uncensored' models to bypass safety filters and terms of service.