measure(g0): fair-target deep scan -- detection 3/5 on app-shaped+sound vs ~0 library-heavy

Browse Source

Ran ousast scan --mode deep on the 5 app-shaped + sound pool targets (the fair change-scoped
eval set from the shape classification). Detection 3/5: graphiti (hunter, Cypher injection),
mcp-remote (hunter, url-scheme injection), dcmcp (ENGINE overlay -- 'outbound.request reached
from parameter' on readFileFromUrl; parameter-as-source fired because an entry point is present).
2/5 missed: malla (DOM/template XSS) and dms (Jinja SSTI), both in families families.toml
documents as limited.

This validates the strategic pivot: the near-zero catch was substantially a TARGET MISMATCH --
on app-shaped repos the engine itself detects (0 -> 3/5). But all hits stop at suspicion/
static_corroboration; release-grade BLOCK still needs proof-arm elevation + family coverage +
the DSPy hint loop. Cost ~$0.007/repo. So bet #1 (fix eval target) is necessary and material
but not sufficient.

Next: recompute a floor on an app-shaped sound subset / real app corpus (VAmPI, NodeGoat,
DjanGoat, vibe-py); measure proof-arm suspicion->proven lift.

Co-Authored-By: Claude Opus 4.8 <[email protected]>

...

This commit is contained in:

Marius Ciepluch

2026-10-12 11:16:35 +02:00

co-authored by Claude Opus 4.8

parent e15ed680a9

commit 04667c815e

6 changed files with 2861 additions and 0 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

benchmarks/measurements/2026-10-12-hunter-fair-targets/malla-findings.json

+1293

File diff suppressed because it is too large Load Diff