Why we put a decoy credential on an AI security website
Sunglasses built this experiment to observe what happens when an AWS credential appears exposed on an AI company website, rather than speculate about what someone might do with it. AZ, our founder, was a nondeveloper who started coding in February 2026. He wanted sunglasses.dev to show real backend security work, not just a polished interface and a claim that AI had built it. If we were going to talk about protecting AI systems, we needed to do the less glamorous work too. Collect evidence. Read it carefully. Separate what happened from what made a good headline.
Claude Fable 5.1, our Chief of Staff and Main Engineer, proposed the idea, planned the experiment and built the whole system. Fable built the decoys, the Cloudflare edge collector, the daily ledger and the digest. Fable also performed the initial analysis and wrote the first readout.
ASTRA, our independent reviewer model, audited the evidence and cut the overclaims. A request to launch a server did not tell us what that server would run. A request to list secrets did not establish that secret values were retrieved. A request to invoke a model did not prove that a model answered.
Cava, our Director of Strategy, shaped the corrected research for readers and search.
The purpose was learning, not a staged demonstration of invulnerability. We want real observations to inform the patterns used in Sunglasses scanning workflows and its AI agent input firewall. That requires a separate step from observation to a testable detection rule. This report does not claim those rules have already shipped or that the experiment measured their effectiveness.
How the AWS canary experiment worked
The experiment paired decoy file responses on sunglasses.dev with AWS credential use alerts supplied by Thinkst Canarytokens.
Fake .env files carried a deliberately exposed decoy AWS credential. The setup also included decoys resembling Claude and Cursor agent configuration files. Those placements were research sensors, not instructions for readers to put real credentials on a public host.
Thinkst supplied the credential sensing. The Cloudflare collector recorded observations of requests to the decoy routes. The ledger and digest brought the records together for review. Matching source addresses allowed us to compare selected edge observations with subsequent alerts.
A file response and a credential use alert answer different questions. An edge observation shows that a decoy response was served. It does not prove the recipient parsed its contents. An AWS alert records a named action associated with use of the decoy credential. It does not supply the response needed to establish success.
Our contribution was the deployment, collection, correlation and analysis. Thinkst deserves credit for the sensing mechanism.
What the saved AWS alerts recorded
The reporting window contained 127 alert records, with Bedrock discovery kept separate from attempted model invocation.
The interval begins at midnight UTC on September 5, 2026 and ends immediately before midnight UTC on September 12, 2026. Every count in the following table belongs to that interval. Counts describe saved alerts, not a proven exhaustive count of AWS requests.
| Action group | Alert records | AWS action names | What the records support |
|---|---|---|---|
| Bedrock invocation attempts | 9 | InvokeModel and Converse | Attempted model invocation. Success, generated output and motive are not established. |
| Bedrock discovery | 32 | ListFoundationModels and ListInferenceProfiles | Requests for model and inference profile information. These are not model invocation attempts. |
| Identity checks | 37 | GetCallerIdentity, GetUser and GetAccount | Attempts to inspect identity or account information. GetCallerIdentity does not require permissions. |
| Messaging reconnaissance | 10 | GetSendQuota, GetSMSAttributes and ListVerifiedEmailAddresses | Quota and identity discovery involving SES and SNS. No send operation is shown. |
| EC2 launch attempts | 3 | RunInstances | Requests to launch instances. The workload and outcome are unknown. |
| EC2 discovery and preparation | 3 | DescribeImages, DescribeSecurityGroups and ImportKeyPair | Discovery and preparation requests. ImportKeyPair is not a read only discovery operation. |
| IAM identity change attempts | 2 | CreateUser and AddUserToGroup | Attempts to create a user and add a user to a group. Successful changes are not established. |
| Secrets inventory | 4 | ListSecrets | Requests for secret inventory metadata. This operation does not return secret values. |
| Other enumeration | 27 | ListFunctions (logged as ListFunctions20150331), ListClusters, GetServiceQuota, SimulatePrincipalPolicy, ListAccounts, ListUsers, ListRoles, ListUserPolicies, DescribeOrganization, ListEventBuses, CheckDomainAvailability | Requests to inspect services, account structure or permissions. Returned information is not recorded. |
The distinction matters most for Bedrock. A source address asking what models exist is not the same observation as a source address asking a model to run. Only the invocation category supports the narrower description of attempted LLMjacking.
The same discipline applies elsewhere. RunInstances does not establish cryptocurrency mining. ListSecrets does not establish secret theft. An IAM change request does not establish persistence. Messaging quota checks do not establish delivered spam.
The two correlated sequences
A Google Cloud address in Belgium. edge minute bucket alert clock
A Google Cloud address in Belgium appeared in both a successful decoy response observation and a subsequent Bedrock Converse alert on September 10. The first observed successful decoy response was in the Cloudflare minute bucket at 06 hours 12 minutes UTC. The saved Converse alert displayed 06 hours 17 minutes UTC. The edge observations for this address covered 7 distinct decoy paths. Those displayed times are five minutes apart. They are not a validated measurement of how quickly the credential moved from a file into an AWS request.
A Google Cloud address in the Netherlands. edge minute bucket alert clock
A Google Cloud address in the Netherlands appeared in both a successful decoy response observation and a subsequent Bedrock Converse alert on September 11. The first observed successful decoy response was in the Cloudflare minute bucket at 16 hours 23 minutes UTC. The saved Converse alert displayed 16 hours 28 minutes UTC. The edge observations for this address covered 4 distinct decoy paths. Again, the displayed gap is five minutes. The correlation is useful without turning it into a speed record or an attribution claim.
Why the clocks and source addresses need caveats
- Matching source addresses connect observations, but they do not identify people or establish the exact time of an underlying AWS action.
- The edge time is a Cloudflare minute bucket. The alert time is the Date/Time displayed in the Thinkst email. Thinkst documents that AWS logging can delay alerts. The saved records do not preserve all the separate clocks needed to measure the true interval. Later fetches within a scanning burst would also produce a different apparent gap.
- These addresses also appeared in observations of .env decoys. That does not prove the exact file that supplied the credential. Fetching an agent configuration and later appearing in an AWS alert would not, on its own, prove extraction from that configuration.
- Shared hosting does not establish shared control. Source addresses can be shared, proxied or reassigned. The estimated country describes an address location, not a person's nationality. No matching edge observation for another source address would prove only an absence from the available observations, not that a credential was sold or shared.
What developers should do about exposed AWS credentials
Preventing real credential exposure remains more important than observing what happens to a decoy. The Sunglasses manual provides broader AI agent hardening guidance.
Keep .env files out of the public document root and deployed static assets. Exclude credentials from build artifacts, repository history, backups and debug responses. Verify from outside the application that sensitive files cannot be retrieved. A secret should not become public because a web server treated it as another download.
Prefer temporary credentials through roles and workload identity rather than embedding persistent AWS keys in applications. Apply minimum permissions to each workload. Keep human administrative access separate and protect it with MFA. AWS publishes guidance on securing access keys.
Use secret scanning before commits and during delivery. If a real key is exposed, revoke or rotate it promptly and investigate its use. Removing the file does not invalidate a credential already copied elsewhere. Review relevant identity changes, service activity and billing rather than assuming that removing the original leak ends the incident.
Use canaries as an additional detection layer, not a substitute for access control. Choose a placement that answers a specific question, validate delivery with a labeled test and assign someone to respond. A private sentinel in a location that should never be accessed serves a different purpose from a public decoy that invites routine probing.
Treat collected logs as hostile input too. Request paths and client identifiers can contain arbitrary text. Escape them before rendering a report. When an AI agent reviews them, preserve the boundary between quoted evidence and instructions. This is the direct connection to Sunglasses as an input firewall for AI agents, not a claim that Sunglasses prevented the recorded AWS calls.
Methods and limitations
This case study describes a bounded observation on a single website, not a census of exposed credential abuse or a product benchmark.
The totals are saved Thinkst alert records deduplicated by email UID. The collection does not establish that each underlying AWS request produces exactly one alert. Known test exclusions remain unresolved, so the totals must not be labeled confirmed malicious activity.
The Cloudflare source uses sampled adaptive grouped analytics rather than complete raw request logs. Earlier collection also had grouping and deduplication limitations. A grouped row, a request estimate and a distinct path are different units. Edge observations cannot establish complete coverage or prove that a returned credential was extracted.
Edge minute buckets and alert text timestamps are different clocks. Alert delivery can be delayed. The displayed gaps are not verified elapsed times between credential retrieval and API execution.
The supplied records lack AWS response fields, production account audit evidence and billing evidence. They do not establish successful model generation, resource creation, persistent access or financial impact. A decoy's intended restrictions are not a substitute for verifying its isolation.
Source addresses are not people. Geolocation is approximate. This website's exposure and sensor behavior cannot establish how frequently agent configuration files are sought across the internet.
This experiment did not test the Sunglasses firewall. Credential use against AWS is not evidence of a successful attack on sunglasses.dev or of a call blocked by Sunglasses.
Credits and prior research
Thinkst supplied the canary sensing and Sysdig documented the LLMjacking attack class before this experiment. Thinkst's AWS API Keys Canarytoken documentation explains the credential sensor and alert delivery caveat. Sysdig's original LLMjacking research was published in May 2024. Our novelty is this site's observation and analysis, not discovery of the attack class.
AWS documents why GetCallerIdentity does not require permissions and what ListSecrets returns. Those distinctions are part of the interpretation, not footnotes to ignore.
The Sunglasses Team authored this project and report under founder AZ's direction, as a joint planned operation under the Sunglasses security topic. Claude Fable 5.1 proposed, planned and built the system and wrote the initial analysis. GPT 6 ASTRA independently reviewed the evidence. The Hermes agent CAVA shaped the corrected report for readers and search. Responsibility for publication remains with Sunglasses.