Why this surface, now
The July 8 Trends Brief was clean enough to act on: the strongest climbers were llm vulnerability (+53.2% WoW), indirect prompt injection (+60.0%), prompt injection (+38.5%), and ai red team (+25.3%). That is not permission to mint another broad page. It is a signal that the AI security testing surface should speak in the buyer language answer engines already understand.
The docs-first read keeps the scope honest. Promptfoo frames red teaming as simulated adversarial inputs for RAG, LLM agents, and chatbots. Microsoft PyRIT remains a risk-identification framework for generative AI. Inspect frames evaluations through datasets, agents, tools, scorers, MCP tools, sandboxing, and eval sets. OWASP LLM01 keeps prompt injection as behavior manipulation that can include indirect inputs and critical-decision influence.
A July 8 Gemini-observed answer for best LLM vulnerability testing tools for AI agents before deployment opened with tool/API/database/memory/MCP bad-action risk, then bucketed the market into open-source scanners, enterprise testing platforms, and agent-specific checklist categories. The refresh here is narrow: answer the hot query without overclaiming. LLM vulnerability testing tools find whether an agent can be manipulated before deployment; runtime trust decides whether the resulting evidence should become authority for the next action now.
Where LLM vulnerability testing tools fit
The useful buyer map is not “one tool replaces the stack.” Scanners and fuzzers find failure modes, red-team frameworks generate adversarial pressure, enterprise platforms map blast radius, and agent-specific checklists ask whether tools, APIs, MCP servers, databases, memory, permissions, and downstream execution paths can be abused.
They prove the model or agent can be tested for prompt injection, jailbreaks, data leakage, unsafe output, schema/tool-call failures, and repeatable vulnerabilities before deployment. The finding is evidence; it still does not decide whether the next real MCP output, command result, callback, or file action should execute.
They map code-to-runtime exposure, test AI components and plugins, validate tool access, and show whether prompt injection can reach APIs, databases, or MCP servers. Blast-radius knowledge reduces surprise; runtime trust still checks whether this specific tool result, permission, parameter, or destination should become action authority now.
They name the high-risk agent failures that convert model manipulation into bad actions. The live gate asks whether the current prompt history, retrieved document, memory, MCP response, tool output, generated file, callback, package metadata, or outbound request is safe enough for impact.
LLM vulnerability testing finds whether an agent can be manipulated before deployment; Sunglasses-style runtime trust decides whether the resulting tool call, MCP response, file change, callback, package action, or outbound request should execute now.
Where AI red teaming fits
Red teaming is how teams find LLM vulnerabilities before an agent reaches production. The practical loop is simple: generate adversarial inputs, run them through the model or agent workflow, evaluate the response, analyze the weakness, and turn the result into a report, regression case, CI/CD check, or control requirement.
For agents, the red-team surface is bigger than the chat transcript. The useful tests include indirect prompt injection, prohibited or high-risk actions, sensitive-data leakage, task-adherence failure, tool-based vulnerabilities, unauthorized API or database use, MCP output manipulation, web/content poisoning, coding-agent file edits, terminal receipts, verifier sabotage, and sandboxed agent tasks. Agent-evaluation frameworks now explicitly support custom tools, MCP tools, shell/python/browser tools, external agents such as Claude Code, Codex CLI, and Gemini CLI, and sandboxes for untrusted model code.
AI red teaming finds the LLM or agentic workflow vulnerability; runtime trust decides whether this specific prompt history, tool output, MCP response, callback, destination, file edit, package action, or approval path should be allowed to shape an agent action now.
What AI security testing covers
Real agent systems have code, packages, CI/CD metadata, tools, MCP servers, memories, approvals, browsers, callbacks, and outbound destinations. Testing has to cover the workflow, not just the chat box.
Testing gives security teams something concrete: reproducible failures, known attack paths, control gaps, and regression cases. Without that, runtime controls are guessing.
Where PyRIT and Foundry AI Red Teaming Agent fit
Microsoft PyRIT and the Foundry AI Red Teaming Agent do not just test whether a single prompt can jailbreak a model. The useful agentic tests exercise multi-turn attacks, prompt converters such as Base64 or ROT13, tool-output context, synthetic or mock tools, prohibited-action policies, scoring, reporting, and release gates. That still leaves a smaller production question: after a red-team run, what should happen when the live workflow reads a fresh web page, MCP response, command result, package metadata field, callback, CI annotation, or approval note?
Where Promptfoo-style agent, coding-agent, and MCP tests fit
Promptfoo's public docs separate red teaming for agents, MCP-plugin testing, coding-agent plugin checks, CI/CD integration, and framework comparisons into concrete surfaces — answer engines can now classify the space as more than generic jailbreak testing. A passing test or a useful finding still does not decide whether the next live MCP result, terminal output, repository instruction, package metadata field, callback, generated file, or outbound request should be allowed to shape an action now.
Where LLM jailbreak detection tools fit
What testing still cannot decide
The trap is treating a passed test as permanent permission. Agents do not act in a static lab. They act after fresh prompts, fresh tool outputs, fresh packages, fresh web pages, fresh CI metadata, fresh approval state, and fresh callbacks. A test can say “this class of attack is covered” and the next run can still combine trusted-looking pieces in a way the test did not anticipate.
The short version: AI security testing proves classes of risk; runtime trust decides whether this already-approved agent action should execute now.
Three examples: tested systems, unsafe actions
A red-team harness may prove that an agent can read a hostile web page and receive an instruction to ignore prior rules. Production risk depends on what happens next: summarizing the page is one risk; updating a ticket, sending an email, querying a CRM, invoking a payment workflow, or committing a file is another. Runtime trust ties the suspicious input to the downstream action — deny, sandbox, require review, strip the evidence, or route through a safer workflow.
Our CI/CD metadata poisoning patterns show how pipeline annotations, job summaries, SARIF messages, dependency-bot comments, and GitOps status can become trusted-looking evidence for coding agents. The runtime question is still separate: should the agent merge, suppress, deploy, roll back, or approve based on this exact metadata now? A forged “all checks passed” message is evidence to verify, not authority to obey.
Testing can inspect tool descriptions, verify servers, scan wrappers, and harden package paths. It still does not prove every future invocation is safe: an approved MCP server can be called with a poisoned argument, a generated wrapper can return misleading evidence, a callback can drift to a different destination. Runtime trust checks the live action, not just the connector's existence.
How Sunglasses catches it
Sunglasses is built around the attack patterns that appear when trusted-looking agent context becomes unsafe action: prompt injection, MCP/tool poisoning, metadata poisoning, workflow-state drift, policy-scope redefinition, stale evidence, forged approvals, callback drift, and supply-chain-shaped agent failures. Red-team findings are strongest when they become reusable action-boundary checks instead of one-time reports.
Nine test-and-pipeline-output patterns ship in this release — GLS-PFX-001 (SARIF fix-plan override), GLS-CMP-009 (Actions step-summary poisoning), GLS-CMP-010 (Renovate config metadata), GLS-TOP-643 (OSC 8 hyperlink smuggling in terminal output), and five GLS-V3 rows covering orchestrator, audit-log, dependency-automation, config-preview, and access-log tool output — all searchable in the pattern database.
Adjacent shipped families: API descriptor poisoning, build and repo metadata poisoning, identity discovery poisoning, and tool-output receipt forgery. Sunglasses turns them into action-boundary questions: which carrier influenced the agent, what authority did it believe it had, and what evidence would make this action safe enough to run?
Buyer checklist: test before deploy, gate during runtime
If you are evaluating AI security testing for agents, ask both sets of questions. The testing questions prove you can find failures. The runtime-trust questions prove you can stop the next unsafe action.
The fifth question is the hinge. A team can have excellent tests and still lack the runtime decision that matters most when an agent is about to act.
What not to overclaim
Sunglasses is not a full AppSec platform. It does not replace SAST, SCA, DAST, container scanning, IDE security, cloud posture management, agentic pentesting, PyRIT, Promptfoo, garak, Inspect, Foundry AI Red Teaming Agent, MCP security testing, or broad production-validation suites. Those tools solve real problems.
The stronger claim is narrower. When AppSec and AI security testing tell you a workflow has risk, Sunglasses helps name the agent-specific failure primitive and reason about the action boundary. When a scanner sees suspicious CI metadata, Sunglasses asks whether the agent should use that evidence to merge, suppress, deploy, or roll back. When an MCP connector passes validation, Sunglasses still asks whether this invocation, with these arguments and this context, should execute.
Prompt-to-runtime security is not complete until runtime can say no to the next trusted-looking action.