What I can and cannot prove about AI attackers
An honest accounting. I built every trap I could think of for catching AI agents, ran them for months, and this is the evidence that survived — plus the machinery that has never fired once.
Why this page exists
Claims about AI-driven attacks are easy to make and hard to check. Most of what gets published is either vendor telemetry you cannot inspect or a demo against a model that agreed to cooperate.
I run a honeypot instrumented specifically to catch AI agents. Here is what it has actually caught, graded by how well each claim is evidenced — including the things I built that produced nothing.
1 · The evidence ladder
Every capture is graded, because "AI agent" without a stated standard of proof is marketing.
| Tier | Standard of proof | Count |
|---|---|---|
| declared | The caller identified its framework, or handed over protocol-level identity | 150 |
| inferred | Scripted transport reaching bait layers — a hypothesis, not a catch | 1,110 |
| none | No AI evidence | 24 |
Roughly one in nine. That ratio is the most useful number on this page. Anyone quoting a large "AI attacks detected" figure without a comparable ratio is quoting the wrong number — I know, because an earlier version of this platform did exactly that, and an audit of my own labelling found most of it was inference.
2 · The strongest evidence I have
Files only agent tooling wants
598 fetches from 250 distinct addresses of files that exist purely for AI coding assistants:
| File | Fetches |
|---|---|
AGENTS.md | 89 |
.mcp.json | 87 |
.cursor/rules/production-secrets.mdc | 79 |
.github/copilot-instructions.md | 64 |
CLAUDE.md | 19 |
A commodity scanner hunting credentials has no reason to want a Cursor rules file. These paths are not in the standard wordlists — they exist because agent tooling reads them. That is behavioural evidence, and it is the most defensible AI signal I own.
Falsifiable prediction: if these paths enter common scanner wordlists, this signal dies. I would rather say that now than defend it later.
Frameworks that named themselves
231 self-identifications across eight distinct agent frameworks:
| Framework | Count |
|---|---|
| Claude Code / MCP | 52 |
| Cursor | 48 |
| Codex / generic agent | 47 |
| GitHub Copilot | 33 |
| LLM plugin / retrieval | 19 |
| Gemini | 12 |
| Cline | 10 |
| Windsurf | 10 |
Caveat on the same page as the claim: this is derived from which agent-specific file was requested, not from a cryptographic assertion. It is strong circumstantial evidence, not proof.
3 · What has never fired
This is the section most write-ups omit.
| Trap | Built to catch | Organic result |
|---|---|---|
| Prompt-injection elicitation 22 surfaces, 3 role-confusion techniques | An agent stating its task, tools and model | 5 URL follows, 0 identifications |
| Decoy MCP server | Protocol-level agent identity | 21 hits, 0 identified tool calls |
| Decoy LLM API | The prompt sent with a stolen model key | 0 captures |
| Cross-surface correlation | One actor walking web → network | 0 organic |
Every one of those mechanisms works — each was verified end to end with a controlled test. None has been triggered by a real visitor.
The null result is the finding. Traps that require an attacker to read, reason about, and comply with instructions have caught nothing across months of live traffic. Traps that require no cooperation — a credential that phones home when used, a file only agent tooling requests — produce data continuously.
If you are designing detection for AI-driven attacks, that asymmetry is the most useful thing on this page. Do not build detection that depends on the adversary's cooperation. Its false-negative rate is set by them, not by you.
4 · What the traffic actually looks like
Across 408 stolen-credential uses, the clients were Boto3, aws-cli, aiohttp and axios. Zero browsers. That proves automation end to end — and automation is not the same as an autonomous agent. Boto3 predates the current generation of models by years.
So the honest position is two separate claims:
- Proven: attack labour is automated, distributed across machines, and a distinct cohort requests files only agent tooling reads.
- Not proven: that a language model is orchestrating any of it.
I keep those apart in everything I publish. The credential-lifecycle analysis covers the first in detail.
Honest limits
Discovery is the confound under every zero. Agents arrive through repositories, links, search results and registries. A honeypot on an unlinked address is measured by whoever finds unlinked addresses — which is, by construction, the scanning population. Absence of agent compliance here is partly a sampling artifact and cannot be claimed as a property of the landscape.
The injection window is short. Full coverage across all 22 surfaces is recent. Zero over a few weeks is weak evidence of anything; it becomes interesting at a few months.
Framework identification is path-derived. It reflects which file was requested, not a verified client identity, and it would be trivial for an adversary to request those files deliberately.
Self-tests are excluded from every figure here, but they exist in the raw data and were removed by hand. That is a process I would not trust at ten times the volume.
What I would tell another defender
Build the traps that work whether or not anyone cooperates. Grade your evidence out loud, and publish the ratio rather than the total. And when a mechanism produces nothing for months, say so — the shape of what does not happen is a real finding about how much of today's attack traffic is actually reasoning.