mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-16 19:59:26 +00:00
Marius CiepluchandClaude Opus 5
20edd9354b
Spec feedback after group 2, recorded in the brief where the goal is stated rather than in a task where it would be lost. The standing test is that the tool must find bugs or it is SAST noise, and that most developers already have dataflow in CodeQL and Semgrep and never use it. That settles what the differentiator is: not the engine, which they already have, but the RUNG -- a claim the graph decided, reported without configuration. Everything below model_entailed is on trial. The scoreboard, kept honest: vampi finds 3 of 3 known vulnerabilities at model_entailed, including a second BOLA the engine found before it was in the recipe, with 2 false positives that are endpoints the OpenAPI spec declares public. libpng produced 198 rows and zero valid ones, for a structural reason rather than a tuning one -- the library never calls a sink the c/memory family models. Nine defects this session, each found by a measurement and none by review; three were one bug class in three matchers, two were a capability present but unreachable. The teachable finding is not any one of them, it is that a corpus of function-level pairs cannot see any of them, and every one appeared within minutes of pointing the tool at a real repository. The gates stayed byte-identical through all nine, which says the same thing from the other side. Two priorities change. 2.11, the suspicion band, becomes the critical path rather than cleanup: libpng's 163 "API is present" rows are exactly the noise the quote is about, and no arbiter accuracy survives shipping them. And a family that cannot decide its class has to say so, because a silent C scan reading as a clean bill of health is worse than noise -- it is quiet. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG