Fake AI agents probed our site 2364 times in one week

What they were looking for

The number 2661 looked like great news. It is the number a lot of people chase right now. It means AI engines are reading you and quoting you.

224 of those requests were real. Every one of the 224 came from an address inside its vendor's published IP list. The other 2437 were fake. 2364 of them were hunting for files like these.

Look at the agent config files in that list. Those are not the old WordPress attack paths. Someone built a scanner that hunts for the credentials your coding agent uses. The same run also asked for /.mcp.json, /.aider.conf.yml and /.anthropic/config.json. It also tried the Vite dev server trick of prefixing /@fs/ to reach files outside the web root.

The tools we all installed this year are now on the target list.

How we know they were fake

A user agent is a claim. Anyone can type ChatGPT-User into a request. It proves nothing.

The source IP address is different. You cannot fake it and still receive the answer. And these vendors publish exactly which addresses they use.

• OpenAI publishes openai.com/chatgpt-user.json. It listed 226 address ranges on 2026-08-13.
• Anthropic publishes claude.com/crawling/bots.json. It listed 23 address ranges on 2026-08-13.
• DuckDuckGo publishes duckduckgo.com/duckduckbot.json. It listed 481 address ranges on 2026-08-13.

We checked every request against those files. Here is every one of the 2661, sorted. Nothing is rounded and nothing is left out.

Read the first two rows first. 2364 probe requests came from nine machines. Eight of the nine are ordinary Google Cloud virtual machines. Reverse DNS says bc.googleusercontent.com and whois says Google LLC. The ninth is a rented dedicated server at Limestone Networks. All nine are rented computers. Not one of them is vendor infrastructure.

Rows three and four are the rest of the fake traffic. Those two rows list 25 and 4 addresses. Three of those also appear up in the probe rows. So they add 26 addresses we had not already counted. Those did not probe for secrets. They read normal pages while wearing a borrowed name. We do not know what they are. We only know they are not what they claimed, so we do not count them as citations either. Fake plus fake is 2437 requests from 35 addresses.

The last two rows are the smallest and they are real. We checked those six requests against DuckDuckGo's published list and Anthropic's published list. Every address is inside its vendor's ranges. Add up the yes rows and you get 224. That is where the real number in this report comes from.

Then the detail that ends the argument. Seven of the nine wore a whole wardrobe. Inside one week a single address claimed to be ChatGPT-User, ClaudeBot, GPTBot, OAI-SearchBot, PerplexityBot, Amazonbot and GoogleOther. It sent plain browser traffic too. One machine cannot be seven companies. It is a scanner working down a list of names.

The other two machines were simpler and more revealing. They only ever claimed to be Claude. One of them sent 577 requests on August 12 and 461 on August 13. On that second day it was the only source of Claude branded traffic we had at all.

The dramatic blue line on our chart was one rented computer.

Real traffic looks nothing like that. Real citations came from 98 different OpenAI addresses in 16 countries, about two requests each. That is what a crowd of real people asking real questions looks like.

What we got wrong first

We saw the probe paths on our dashboard and we had a theory. The theory was that attackers were asking ChatGPT and Claude to fetch these URLs for them. That would mean the probes arrive from real vendor computers. It would mean IP blocklists cannot help you. It is a scary idea and it sounded right.

We checked it before we published it. It was wrong.

The truth is duller and more useful. It is plain user agent spoofing. And our own dashboard fell for it, which is the part worth your attention.

Our dashboard had a filter that we described as excluding spoofed AI traffic. It was not doing that. It was matching path names. It knew .env and wp-admin because those are the old attack paths. It had never heard of local.settings.json or .codex/config.toml. New vocabulary walked straight through a filter we believed checked identity.

So the fake traffic was counted as real citations. On August 12 our dashboard said 1449 AI answer fetches. The real number was 32. Across the week it overstated real AI citations by 11.9 times. 91.6 percent of that number was a stranger's scanner.

Why the spike was not growth

This is the part that should worry a founder more than the probes do.

Real citations are flat. Between 28 and 38 every single day. No trend at all.

The graph that went up thirty times was one attacker turning up the volume. If we had trusted it we would have written a post about our growing AI visibility. We would have been publishing an attacker's scan rate as our own success metric.

Nothing leaked

No credential file ever returned 200. Across the ten files in the table above they sent 363 requests. 262 came back 404 and 101 came back as a redirect. We tested every one of those paths again by hand on 2026-08-13 and each returned 404.

The scanner did receive 196 responses with status 200. All of them were public pages. 124 of those hit /wp-admin, which on our site is a honeypot page that says "Looking for vulnerabilities? So are we." The rest were real articles. The scanner read our security research while looking for our keys. We enjoyed that.

How to run this check on your own site

You can do this today. You do not need a product.

We released the check as a free script. One file, standard library only, no install. It reads your access log, fetches the published lists from OpenAI, Anthropic, DuckDuckGo and Perplexity and prints verified, fake and uncheckable counts. Get it at github.com/sunglasses-dev/sunglasses.

Or do it by hand. The steps are simple.

1. Pull your logs for requests claiming to be an AI fetcher. Look for ChatGPT-User, Claude-User, GPTBot, OAI-SearchBot and PerplexityBot.
2. Download openai.com/chatgpt-user.json and claude.com/crawling/bots.json.
3. Check each source IP against those lists. Anything outside is lying about who it is.
4. Look at what the liars asked for. If they asked for config files, someone is hunting credentials on your host.
5. Count your AI citations again with the fakes removed. The number will be smaller. It will also be true.

Two warnings from doing this ourselves.

Response status is not enough on its own. Our honeypot returns a real 200, so a filter that counted every 200 as a citation would have scored 124 attacker requests as citations. Check identity and outcome together.

Not every vendor publishes a list. Cohere and Amazon do not. We logged 4210 more requests wearing those two names in the same week, most of them asking for the same secret files. We cannot prove those are fake, so we do not say they are. We label them uncheckable and we keep them out of the citation number. Uncheckable is not the same as clean.

Why this matters beyond one website

The AI citation metric is becoming a business metric. People report it to investors. Almost nobody checks it.

It is trivially forgeable. One rented machine and one header. Your dashboard then tells you a story an attacker wrote. We proved that on our own site with our own logs this week.

There is a second lesson underneath. Our filter checked what the request asked for, not who was asking. That is the same mistake we write about every week. An AI agent trusts a tool result because of where it appeared, not because anyone proved where it came from. Identity has to be checked at the boundary, with evidence, every time. A name in a header is not evidence. A file both vendors publish is.

We got to learn that on a dashboard instead of on a production agent. Cheap lesson.

How we measured this

All traffic figures come from our own Cloudflare edge logs for the zone serving sunglasses.dev, pulled on 2026-08-13 for the days August 6 through August 13 2026. Cloudflare's free plan keeps roughly eight days, so August 5 and earlier could not be included. Requests were grouped by user agent, path, response status and source IP. Vendor ranges were fetched live from the two published files on 2026-08-13. Reverse DNS and whois were run per address. The live 404 responses were confirmed by hand with curl on the same day.

We publish the pattern rather than the addresses. Attacker infrastructure rotates. A public list of nine IPs would be wrong within a month. The check is the durable part.