mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-16 03:49:26 +00:00
Marius CiepluchandClaude Opus 5
00e178d69e
contributor-scan 2.11. libpng emitted 198 sections, 163 of them asserting that `memcpy` and `strcpy` are PRESENT in a library that uses them correctly throughout. That is the noise developers have learned to skip in every other tool, and shipping it costs the findings beside it their credibility. **Filtering on rung was the obvious rule and it was wrong.** The obligations checker emits at `suspicion` too, so hiding that rung would have hidden an entire detector -- six tests failed and they were right to. The discriminator is evidence_level: on libpng 163 of 198 rows are `static_corroboration`, on vampi 3 of 60 are, and the model layer and the obligations checker carry `suspicion` there instead. That is exactly the distinction the ladder exists to make -- an undecided CLAIM about a site is not an observation that an API exists. So a pattern-only rule is grouped once it has more than three sites, and keeps its full sections under that. One `python-flask-debug` is a finding a contributor can act on and must not be summarised away; fifty-four `c-unsafe-copy` are a property of the codebase. libpng 198 findings, 198 sections -> 39, 161 grouped into three lines vampi 60 findings, 61 sections -> 61, nothing grouped Counted, never dropped: findings.json stays complete because it is the record a baseline is compared against, and the grouped section says the model reached none of these and that its silence means no fact table covered the site rather than that the site is safe. What it does not fix, recorded rather than glossed: vampi still emits 61 sections for a 520-line application, 51 of them reasoned claims this rule does not touch. It fixes pattern noise, not volume. Also records what group 2 established about the three corpus asks, so none is started on a wrong assumption. Joern ships php2cpg but our fact tables have zero PHP families, so WordPress would parse and produce nothing -- fact tables first, corpus second. An older vulnerable libpng will not help for a reason no version can fix. Vibe-code repositories are the best fit and the cheapest route to a second true-positive checkout, because Python and JavaScript are the two languages whose fact tables are complete. And Group 8, for per-repository state a developer owns: a committed dismissal file the tool reads and never writes, dismissals as a split evaluation set, and GEPA/SIMBA-style evolution scoped to the QUESTION -- the judge and residual prompts -- never to the fact tables. That boundary is the whole design: a prompt is how we ask, a fact table is what we claim, and the closed-loop leak measured on 2026-09-05 is why the distinction is not negotiable. Gates byte-identical. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
25 lines
2.0 KiB
JSON
25 lines
2.0 KiB
JSON
{
|
|
"counted_never_dropped": "findings.json stays complete -- it is the record a baseline is compared against, and this is a view of it. The grouped section states the count, the rules, the top files, and that the model reaching none of them means no fact table covered the site rather than that the site is safe.",
|
|
"generated": "2026-09-09",
|
|
"measured": {
|
|
"libpng": {
|
|
"findings": 198,
|
|
"grouped": 161,
|
|
"report_lines": 679,
|
|
"sections_after": 39,
|
|
"sections_before": 198
|
|
},
|
|
"vampi": {
|
|
"findings": 60,
|
|
"grouped": 0,
|
|
"note": "its three pattern rows are under the limit and keep their sections",
|
|
"sections_after": 61,
|
|
"sections_before": 61
|
|
}
|
|
},
|
|
"rule": "A report is proportionate to what the engine can say. A finding whose only evidence is a text pattern -- evidence_level `static_corroboration` at rung `suspicion`, nothing corroborating the match -- is grouped by rule once that rule has more than three sites. Under three it keeps the sections it always had. Anything a proposer reasoned about keeps its section whatever its rung, because an undecided CLAIM about a site is not an observation that an API exists.",
|
|
"spec": "contributor-scan 2.11 (Req 5.1, 5.2)",
|
|
"what_it_does_not_fix": "vampi still emits 61 sections for a 520-line application: 38 model suspicions and 13 obligation findings, all reasoned claims and none of them collapsed by this rule. Whether fifty-one undecided claims is itself too many for a contributor is a separate question this task does not answer, and the honest position is that the rule fixes pattern noise rather than volume.",
|
|
"why_evidence_level_and_not_rung": "Filtering on rung was the obvious rule and it was wrong: the obligations checker emits at `suspicion` too, so hiding that rung would have hidden an entire detector. The discriminator is evidence_level. On libpng 163 of 198 rows are static_corroboration; on vampi 3 of 60 are. The model layer and the obligations checker carry `suspicion` there instead, which is the distinction the ladder was built to make."
|
|
}
|