How Sunglasses Became a CI-Safe Input Filter for AI Agents

This report explains the reliability work that changed Sunglasses from a research project into something we can credibly ask developers to run in real workflows.

Security tools live or die on trust. A filter can have a large detection library, smart attack categories, and a strong research story, but if it blocks clean code or stalls on ordinary files, teams will not put it in CI. They should not. CI is where noisy tools become ignored tools.

The last few releases were therefore not just maintenance. They were a trust reset. This is the honest version of what broke, what we fixed, and what we still refuse to claim.

Sunglasses is a CI-safe input filter for AI agents: fast, local, deterministic, and now gated against clean-code false positives and stale agent-facing metadata. That is the frame. Pattern count still matters, but it is supporting proof, not the headline. A large pattern library gets attention. Reliability gets installs.

The filter framing also raises the bar on us. A noisy scanner is annoying; a noisy filter that blocks clean code is a workflow breaker, because the whole promise is that bad content gets stopped at the door — the agent never reads the poisoned version because the gate blocks it before ingestion. If the filter wrongly blocks clean input, it breaks the build. So the clean-code false-positive story matters more for us, not less.

The current live package is v0.2.65, with 1,038 detection patterns, 65 attack categories, and 7,548 detection keywords. The test suite is currently 221 passed and 7 xfailed. Those numbers show scope. The reliability gates show discipline.

The two most important reliability failures were simple to describe and serious in practice.

First, the filter was flagging clean code in the tested clean-code corpus. That is a credibility killer for any security tool. A developer trying it for the first time should not see harmless files treated as malicious because generic words leaked into detection logic.

Second, the filter could stall on an ordinary file. One measured case took 117 seconds. CI buyers do not care how smart a filter is if it can hang on normal input.

A third issue mattered for how AI agents and answer engines read the project. Some machine-readable project files were shipping stale release facts, which meant agent-facing metadata could tell systems the wrong project reality even while the human-facing pages were current.

Clean-code false positives in the tested corpus went from 86 to 0 after we fixed a keyword-denylist leak.

The root cause was not the original theory. The problem was a denylist leaking generic words and plurals into filter behavior — ordinary terms like “ai agents,” “cookie,” “env,” and “path.” Those words appear in harmless code and documentation. When they become triggers in the wrong place, clean repositories start looking malicious.

That is the wrong failure mode for a filter meant to run in developer workflows. The fix was implemented in the filter engine, and the regression is now guarded by a real clean-code false-positive test. The full test suite stayed green, which matters: reducing noise is only useful if detection coverage does not collapse at the same time.

One stall case dropped from 117 seconds to 0.30 seconds after a whole-document matching path was fixed.

The root cause was a set of whole-document classifier expressions being evaluated with a search across every offset, which produced O(n²)-style behavior on ordinary files. The fix preserved the intended whole-document meaning by anchoring those classifiers to a position-zero match while keeping normal scanning behavior for the token-finding patterns. On the worst real file in the corpus, the run went from a hang to a few seconds of linear work.

The result is the difference between “interesting project” and “safe to put in automation.” A filter does not need to be perfect to be useful, but it cannot unpredictably stall on normal input.

The strongest proof of the new reliability posture is that the v0.2.65 release held back six candidate patterns before they shipped.

In that release cycle, six of twenty-five candidate patterns fired on clean code. The clean-code false-positive gate caught them before release. Sunglasses shipped only the clean nineteen and quarantined the six for regex tightening. Each candidate had even passed an internal smoke check; the release gate caught what the smoke check missed, which is why the gate exists.

That matters more than the final pattern count. It proves the gate is not decorative. The project had a chance to ship a larger number and chose not to, because correctness mattered more.

Agent-facing project files now stay synced with the live release facts and are blocked by a preflight gate if they drift.

This matters because modern AI systems do not only read blog posts and homepages. They read machine-friendly project metadata. For Sunglasses, those surfaces include AGENTS.md, mcp.json, .well-known/mcp.json, and stats/current.json.

Those files should be citation targets, not stale mirrors. If they lag behind the current package, answer engines and AI agents can learn the wrong version, wrong pattern count, or wrong category count. The fix makes those files part of the release contract: they are auto-synced and gated so stale agent-facing metadata cannot quietly ship as the project’s declared source of truth.

Sunglasses SARIF output now routes findings to real explanatory pages instead of dead per-ID links.

This is important for the GitHub Action path. A useful finding should not be a dead-end alert. It should create a loop: scan the repository in CI, surface a blocked input, open the finding in GitHub’s security workflow, and follow a real explanation of the category.

That is how a security tool becomes teachable. A dead 404 link says “trust us.” A real category explanation says “here is what this means and why it matters.”

Sunglasses added blocking site gates for stale numbers, malformed data, broken links, internal leaks, template issues, and answer-engine extraction structure.

This is not glamorous, but it is necessary. A security project cannot ask developers to trust its filter while the public site serves stale counts, broken links, malformed structured data, or inconsistent machine-readable files.

The new gates turn credibility cleanup from a one-time audit into a release property. Future site releases have to pass the checks instead of relying on someone noticing after the fact.

The reliability work changes the adoption question from “does Sunglasses have enough patterns?” to “can Sunglasses run in real workflows without creating trust debt?” The answer is now much stronger.

One scope note: in CI, Sunglasses catches injection that lives in the repository and its agent-facing metadata — poisoned READMEs, tool descriptions, and discovery files. Runtime content an agent fetches live (web pages, retrieved documents, tool output) is the same filter applied at a different point, not something CI alone covers.

This does not mean every future false positive is impossible. It means the project now has the gates, tests, and release discipline to treat false positives as release blockers instead of acceptable noise.

Sunglasses will not claim “zero false positives” as an absolute promise. That would be the wrong lesson. The credible claim is narrower and stronger: clean-code false positives in the tested corpus went from 86 to 0, and a clean-code false-positive gate now blocks candidate patterns that regress that behavior.

Sunglasses will also not claim it catches novel or adaptively obfuscated injections it has no pattern for. Known-pattern blocking raises an attacker’s cost; it is not a completeness proof — see what Sunglasses catches and does not catch. And the false-positive reduction did not come from weakening detection: no detection tests were removed or relaxed to reach 86 to 0, and the full test suite stayed green.

Sunglasses will also not claim that a large pattern count alone makes a filter trustworthy. Pattern count is useful only when the release process can reject noisy patterns before they reach users.

As of this report, the live package is Sunglasses v0.2.65.

The reliability metrics in this report describe measured behavior on the Sunglasses test and clean-code corpus. They are not a promise that false positives or stalls are impossible on every possible input; they describe the gates, tests, and fixes now in place.