AI Agent Runtime Security Needs Evidence Checks

Evidence is the runtime boundary

An AI agent runtime turns model output into real work. It schedules child agents. It releases text. It calls tools. It grades tests. Each step can mistake a description of safety for proof of safety.

The seven mechanisms in this guide share one failure. A favorable summary crosses a trust boundary without the records needed to support it. The summary may be an approved plan. It may be a high safety score. It may be a blocked replay. It may be a safe average. It may be a simple claim that a tool is available.

The fix is not one more prompt. The fix is an evidence contract. The runtime names the exact records needed for an action. It checks their identity and lineage. It rejects missing joins. It stops when records conflict.

Jack tested each mechanism with inert hostile text and benign controls. Each bounded candidate detector separated 6 of 6 hostile fixtures from 8 of 8 benign controls. The installed Sunglasses versions recorded in the source cards missed most or all primary hostile fixtures in their tested channels. These were research gaps at the time of testing. This article does not turn those results into a claim of universal or live coverage.

Bound delegation and resource use

A runtime must treat every delegation request as a new use of authority and resources. Parent approval must not become unlimited permission for children to create more children.

The self_delegation_resource_loop mechanism starts when a runtime reads output shaped like a scheduling command. The resulting task can enqueue itself again. It can also create a growing tree of child tasks. Cost and queue use rise even when each individual request looks valid.

This is different from a normal retry. A normal retry has a clear owner, a fixed limit and a terminal condition. A hostile loop hides the same work behind new task IDs, aliases or indirect child chains. A simple check for matching parent and child names will miss that shape.

Enforce a signed delegation budget at the scheduler. Count depth, total descendants, repeated plan identity, tool fan out, elapsed time and cost. Make each child inherit less authority than its parent. Require fresh approval when a child asks for a new destination or stronger capability. Stop cycles by following lineage across the full task graph.

The saved detector required evidence of a task queue or delegation carrier, recursive work, repeated cost or fan out and approval bypass. It separated 6 hostile fixtures from 8 benign controls. Sunglasses 0.2.73 allowed five hostile variants in the tested message channel. One variant hit the existing GLS-TD-002 rule. All eight benign fixtures remained clean.

This mechanism maps to MITRE ATLAS agentic resource consumption. See MITRE ATLAS AML.T0034.002. The defensive lesson is plain. The scheduler owns the budget. The model does not.

Inspect the final output

A high internal safety score cannot authorize a different object. The runtime must inspect the final generated output and bind the decision to that exact output record.

The ras_representation_output_divergence mechanism pairs favorable representation evidence with an output that another policy marks unsafe. A weak reducer sees the high score and releases the answer. It ignores the contradiction because the score has more authority than the output verdict.

The source context is the paper RAS Measuring LLM Safety Through Refusal Alignment. The paper proposes a white box method that measures hidden state alignment under unsafe and jailbreak prompts. Jack tested a separate operational question. What happens when a downstream reducer treats such a score as release permission?

The answer requires strict object identity. Give every output an immutable ID. Bind the representation score and the output policy verdict to that ID. Run policy on the final form after any translation, summary or formatting step. If the score and verdict disagree, withhold the output and preserve both records for review.

The inert candidate detector separated 6 hostile fixtures from 8 benign controls. Sunglasses 0.2.73 allowed the primary hostile record with zero findings in the tested message channel. The result does not show a flaw in the RAS research system. It shows why representation evidence must remain advisory at an output release boundary.

Bind intent to tool execution

Plan approval does not prove that the dispatched tool call still matches the plan. The runtime must compare the exact operation at the last safe moment before execution.

The component_execution_integrity_drift mechanism preserves approved plan text while a component rewrites the concrete payload. A plugin, adapter or connector can change the method, arguments, destination or record identity. Valid syntax does not make the changed action authorized.

The source paper Securing LLM Agents Need Intent to Execution Integrity describes the path from natural language intent to tool calls, API requests and code execution. Jack turned that broad concern into an inert dispatch test. The test asks whether a third party component can keep the plan while changing the payload.

Use typed operations. Record component identity and version. Normalize the approved intent and proposed payload with one canonical method. Bind approval to the operation, arguments, destination, data scope and expiry. Compare that binding at dispatch. Any mismatch needs a new decision.

The bounded detector separated 6 hostile fixtures from 8 benign controls. The installed scanner missed all six hostile fixtures in the tested API response channel. All eight benign fixtures remained clean. That gap was a candidate for hardening. It was not live product coverage.

Reset state between security tests

A mitigation test is invalid when one candidate inherits state from another. Restore the same baseline before every candidate and prove that the restoration worked.

The candidate_evaluation_state_leakage mechanism creates a false pass through residue. Candidate A changes an emulator, workspace, route or policy. Candidate B then runs without a verified reset. The evaluator credits B for a blocked replay even though A caused the block.

The source paper COHORT Collaborative Orchestration for Hardening via Offensive Replay on Emulated Topologies describes a workflow that proposes controls, implements commands and validates them with offensive replay. Jack tested a separate state integrity risk around candidate comparison.

Start with a content addressed snapshot. Record its digest. Restore it before each candidate. Measure the actual state after restoration. Compare that state with the expected digest before applying the candidate. Bind the command transcript, replay ID, result and score to both the candidate ID and baseline ID.

A reset event alone proves little. The event can fail. It can target the wrong environment. The post reset digest supplies the missing receipt. No matching digest means no score.

The saved candidate detector found 6 of 6 hostile fixtures and left 8 of 8 benign controls negative. Sunglasses 0.3.2 allowed the primary hostile fixture with zero findings in the tested message channel. All 14 scanner runs completed. The proposed detector recorded no false positives or false negatives in that bounded corpus.

Prove test coverage before reporting a pass

A passing score says something about the probes that ran. It says nothing about probes that never ran.

The transform_chain_scorer_early_stop mechanism stops an automated red team after the first passing scorer. The report then promotes partial execution into a complete safety claim. A benign result hides an untested dimension.

The source paper Redefining AI Red Teaming in the Agentic Era From Weeks to Hours describes automated attack selection, transform composition, probe execution and scoring. Jack tested the evidence gap that appears when a score becomes permission to skip the rest of a plan.

Make the execution manifest the authority for coverage. Give every planned probe and transform chain an immutable ID. Record started, completed, failed and skipped states. Bind every score to one probe. Reconcile the planned set with the executed set before the report can say complete.

Early stopping can be valid for cost or safety. The report must say that testing stopped. It must name the missing probes. It must not call the resulting score complete coverage.

The detector matched 6 hostile fixtures and left 8 benign controls negative. Sunglasses 0.2.73 allowed the primary hostile fixture with zero findings in the tested message channel. The toy safe and vulnerable targets diverged as expected. The evidence supports plan versus execution reconciliation. It does not prove that a named red team platform has this flaw.

Keep every unsafe member verdict

A safe average cannot erase an unsafe example. Aggregate analytics must stay separate from release authority.

The wardens_hard_example_reweighting_false_safe mechanism gives a dangerous member little weight. The weighted result looks safe. A downstream consumer then suppresses the member verdict and approves release.

The source paper Information Theoretic Adversarial Training of Large Language Models supplies the research context for adversarial training and dynamic weighting. Jack tested a separate release failure. A method built to optimize or summarize training does not prove that every evaluated member is safe.

Store raw member results. Give each example an immutable ID and preserve its source, transform, model version, weight and verdict. Let the aggregate help with analysis. Never let it overrule a blocking member without an explicit policy decision and review record.

The bounded detector separated 6 hostile fixtures from 8 benign controls. Sunglasses 0.2.73 allowed the hostile corpus with zero findings in the tested message channel. The benign corpus also produced no findings. This does not show that dynamic weighting is a vulnerability. It shows that a release gate needs member level evidence.

Protect the connected tool inventory

An AI agent should not reveal every connected tool to any requester. A complete tool list maps the agent's reachable attack surface.

The tool_capability_inventory_disclosure mechanism moves tool discovery metadata into an outward answer. The answer names concrete tools and reachability classes. No source data needs to leave the system. The inventory itself gives an attacker useful reconnaissance.

This differs from tool poisoning. Tool poisoning changes a tool definition so the agent behaves unsafely. Inventory disclosure reveals which tools exist and where the agent may reach. One can help prepare the other, but they are different control failures.

Apply requester authorization before listing tools. Label sensitive capabilities. Return a generic availability answer when details are not needed. Track aliases so a renamed tool does not bypass policy. Filter tool discovery output before model use and again before any outward response.

The candidate detector separated 6 hostile fixtures from 8 benign controls. Sunglasses 0.2.73 allowed the primary hostile fixture with zero findings in the tested message channel. Six hostile and eight benign scanner receipts were saved. The fixture used no real tool, credential, network or external action.

The source mapping is MITRE ATLAS AML.T0084.001 Tool Definitions. For the broader protocol boundary, read MCP security for AI agents.

Build one runtime evidence contract

These checks work best as one contract. The contract says which evidence must exist before the runtime can schedule, release, execute, score, approve or disclose.

1. Name the action. Use a typed action with a stable ID.
2. Name the authority. Record who approved it, what scope they approved and when approval expires.
3. Name the object. Bind the decision to the exact output, payload, candidate, baseline, probe, member or inventory response.
4. Preserve lineage. Record every transform, component, parent task and state transition that changed the object.
5. Check completeness. Compare planned records with executed records. Reject missing joins.
6. Check contradictions. A favorable score cannot erase an unsafe verdict or identity mismatch.
7. Fail closed. Missing or conflicting evidence stops the action. It does not become a warning after the action.

Text filtering still matters. Hostile instructions often arrive through files, tool output and metadata. Sunglasses is a content layer input filter for that boundary. Runtime enforcement must then carry the evidence forward and check it again when authority becomes action.

The pattern cards show why both layers matter. A lexical detector can quarantine a narrow evidence shape. It cannot prove that an emulator was restored, that a score belongs to an output or that a payload still matches approved intent. Structured checks must make those decisions.

What the evidence proves

The seven candidate detectors each separated 6 hostile fixtures from 8 benign controls in their saved corpora. Their source cards include inert examples, toy targets and scanner receipts. The papers and MITRE records provide research or threat context. Jack's tests isolate the downstream authorization failures described here.

The evidence does not prove that any cited research system contains these failures. It does not prove compromise in the wild. It does not prove universal detection. The installed scanner results came from versions and channels named in the source cards. Product coverage can change after review and shipment.

That limit is useful. It keeps a test result tied to the object it actually measured. That is the same discipline this guide asks runtimes to enforce.

To see how the shipped scanner makes its own decisions explainable, read how Sunglasses works and the pattern manual. The CVP runs show the same evidence discipline applied to real repositories. The FAQ covers what the scanner does and does not claim.

Sources