AgentGuard vs Semgrep vs CodeQL

AI Agent Security Benchmark — 39 samples covering 17 detection rules. Run date: 2026-07-05.

AgentGuard v0.6.4

100%
39/39 detected

Semgrep

0%
0/39 detected

CodeQL

0%
0/39 detected

Why Semgrep and CodeQL Detect Nothing

Semgrep and CodeQL are general-purpose SAST tools. They have zero rules for AI agent security. Their rule sets target traditional web application vulnerabilities (SQL injection, XSS, path traversal) — none of which apply to AI agent code.

AgentGuard is purpose-built for the AI agent attack surface. Every rule targets a specific OWASP ASI Top 10 vulnerability or a novel attack vector unique to autonomous AI systems.

Detection Coverage by OWASP ASI Category

CategoryAttackSamplesAgentGuardSemgrepCodeQL
ASI01Prompt Injection16100%0%0%
ASI02Tool Abuse5100%0%0%
ASI03Data Exfiltration4100%0%0%
ASI06Output Handling2100%0%0%
ASI07Credential Leakage6100%0%0%
ASI09Resource Exhaustion2100%0%0%
ASI10Isolation Bypass5100%0%0%

Beyond OWASP: Novel Attack Vectors

Novel RuleAttack VectorAgentGuardSemgrepCodeQL
ASI-MEMORY-POISONPersistent vector store poisoningDetectsNo supportNo support
ASI-TOOL-TRUSTBlind trust in tool outputsDetectsNo supportNo support
ASI-CHAIN-AMPLIFYDestructive amplification loopsDetectsNo supportNo support
ASI-AGENT-COLLUSIONMulti-agent conspiracy patternsDetectsNo supportNo support
ASI01-INTERPROCEDURALCross-function taint trackingDetectsNo supportNo support
ASI01-CROSS-FILECross-file import resolutionDetectsNo supportNo support

Why This Matters

Real-World Validation
AgentGuard v0.6.1 scanned Microsoft AutoGen (59K stars) and LlamaIndex (50K stars) — detecting 332 critical vulnerabilities across 3,500+ files. Findings reported as GitHub Issues #7917, #7918, llama_index#22245.
Zero False Positives
AgentGuard maintains 0% false positive rate on its entire benchmark suite while achieving 100% detection. Clean code samples are correctly passed. Generic variable names, commented code, and sanitized inputs are not flagged.

Technical Differentiation

CapabilityAgentGuardSemgrepCodeQL
OWASP ASI Top 1010/100/100/10
Interprocedural taintYesNoLimited
Cross-file analysisYesNoYes (QL)
JS/TS supportYesYesYes
Memory poison detectionYesNoNo
Agent collusion detectionYesNoNo
GitHub ActionMarketplaceMarketplaceNative
MCP Server modeYesNoNo
Open SourceMITLGPLMIT
SARIF outputYesYesYes

AgentGuard — Autonomous security scanner for AI agents. OWASP ASI Top 10 compliant.

github.com/dockfixlabs/agentguard | pip install dfx-agentguard

This benchmark is reproducible. github.com/dockfixlabs/agentguard-benchmark