Sunglasses vs Promptfoo (open-source CLI)
These solve different jobs, not the same job differently. Sunglasses is a runtime pre-ingestion filter that blocks live attacks. Testing is not defending — and defending is not testing.
In its README’s words, “a CLI and library for evaluating and red-teaming LLM-based apps”. The vendor also offers Guardrails and an MCP Proxy; this page compares the CLI only.
You test with the Promptfoo CLI to find weaknesses, then run Sunglasses at runtime so those weaknesses are actually blocked in production. Different phases of the same security lifecycle.
Side by side
One tests before you ship. One defends while you run. The grey line under each Promptfoo cell quotes its README or docs, with the page and the date fetched (2026-10-29).
Use them together
Build an adversarial suite with the Promptfoo CLI, measure how your stack responds, then put Sunglasses at the runtime boundary so the attacks you found are blocked in production.
Sunglasses runs with no API key, no cloud, no telemetry, and sends nothing anywhere unless you pass --report-to. The Promptfoo CLI’s evals run on your machine, per its docs, and fit into your dev loop and CI.
When to pick which
you need live protection — your agent is reading untrusted emails, web pages, tool output, or handoffs right now and needs a block, quarantine, allow_redacted or allow decision at runtime.
you need repeatable evaluation — comparing prompts and models against test cases before you ship.
you want a full lifecycle: test adversarial cases before release, defend against them at runtime.
Sources and how to check
README of promptfoo/promptfoo and promptfoo.dev/docs/intro, fetched 2026-10-29. Every quoted line in the table is verbatim from one of these two pages.
The vendor also offers Guardrails and an MCP Proxy (promptfoo.dev, fetched 2026-10-29). This page compares the CLI only, and it does not say whether the CLI’s red-team attack generation calls a service.
Tell us at [email protected] and we will correct this page.