The most critical audience for this material consists of Enterprise Security Architects, AI Red Team Leads, and C-Suite Risk Officers (CISO/CTO). These professionals are responsible for the deployment of autonomous systems and must understand the architectural flaws that allow external, untrusted data to seize control of internal compute resources.
Senior AI Security Researcher Summary
Abstract:
This presentation by Johann Rehberger (Zenity) details the evolving threat landscape of "Agentic ProbLLMs," focusing on the exploitation of autonomous coding and computer-use agents. The core vulnerability discussed is indirect prompt injection, where an adversary places malicious instructions in a location an agent is likely to "perceive" (e.g., a webpage or source code file). Rehberger demonstrates how these injections trigger unauthorized tool use, leading to Remote Code Execution (RCE), data exfiltration, and the creation of "ZombAIs"—compromised agents joined to a command-and-control (C2) infrastructure. The research highlights critical design flaws in systems like Claude’s computer-use, Devin, and GitHub Copilot, specifically regarding the lack of robust sandboxing and the dangers of automated configuration modifications.
0:00 – Adversarial Brittleness: Demonstrates that LLMs are easily manipulated by hidden instructions (e.g., forcing a 1+1=42 result or misidentifying images), establishing the baseline for prompt injection.
2:07 – Indirect Prompt Injection: Defines the primary threat as a third party controlling part of the prompt via external data, impacting the confidentiality, integrity, and availability of the system.
4:25 – The Agentic OODA Loop: Analogizes agent behavior (Observe, Orient, Decide, Act) to fighter pilot combat, noting that agents "win" by moving through this loop faster than human oversight can intervene.
6:03 – Creating "ZombAIs": Illustrates a "zombie" agent scenario where a computer-use AI is directed to a malicious page, downloads a binary, modifies permissions (chmod +x), and executes it, connecting the host to a Sliver C2 framework.
8:57 – AI ClickFix Technique: Adapts a common human-targeted TTP to AI. The agent is tricked into copying a malicious command to its clipboard and pasting it into a terminal, demonstrating full system compromise without explicit user approval.
12:55 – Exploiting Devin & The AI Kill Chain: Analyzes the "Lethal Trifecta" (prompt injection + sensitive data + exfiltration vector). Bypasses Devin’s safety filters by splitting an attack into two stages: creating a local web server and then tunneling that server to a public URL using the agent's own port-forwarding tools.
19:17 – Multi-Tool Data Exfiltration: Details a method for exfiltrating environment variables (API keys) from Anthropic’s Claude and Amazon Q via DNS requests, bypassing model-level "bad domain" filters by using custom attacker-controlled domains.
22:51 – Agent Configuration Manipulation: Identifies a fundamental flaw where agents can write to their own configuration files. Shows GitHub Copilot being forced into a "YOLO mode" (automated command execution) by an injection that modifies the settings.json file.
26:39 – Steganographic Injections: Demonstrates "hidden" injections using Unicode tag characters. These are invisible to human users in the IDE but interpreted by the LLM as high-priority instructions, leading to "zero-click" RCE.
28:48 – Key Takeaways: Emphasizes an "Assume Breach" mindset for AI deployment. Agents must be treated as potential "malicious insiders" that operate at machine speed, requiring hardware-level sandboxing and the removal of automatic tool invocation for sensitive commands.