How I benchmark an agent guardrail without lying to myself
It would be easy to write a benchmark for Doberman that makes Doberman look good: pick the attacks it already stops, skip the ones it doesn't, round up. I've tried to build the opposite of that - a methodology that reports the numbers whether they're flattering or not, and a results page (docs/BENCHMARKS.md) that lists what it missed before what it caught. Doberman is scored as a decision function over tool-call cases, not by driving a live LLM agent. Each case - a tool call or an ordered sequen
🎯 Vulnerability: Remote Code Execution (RCE)
💡 Defense Fix: Avoid executing shell commands via system(). Use parameterized APIs and strict input sanitization.