mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-16 03:49:26 +00:00
Marius CiepluchandClaude Fable 5.1
c0b9e41000
Tasks 1.2, 2.1-2.5 and 6.2 (pair-side filters). - pairs.py: coverage with a source counts as detection and as leak; function-scoped matching by line-range containment from the vulnerable side's parse (unresolvable labels are counted loss, never silent misses); exact or alias sink matching; weak CWE-only labels never match; side-by-side inventory/overlay/hunter scorers; per_profile, per_mechanism, achievable, known_limit and loss payload; loader rejects unknown mechanism/provenance/split and reserved or missing function labels on harvested slices; candidate rulesets thread through for the improve loop. - overlay.py: OverlayRecord.function (additive) filled from the shared naming helper in one parse; benchmark.py: ExpectedFinding.mechanism. - cli pairs: --slice over every slice, --profile, --split, --hunter, --hunter-model. - catalogs migrated to the new schema (provenance, split, function, mechanism, known_limit for owasp-java-hash); baseline recorded in the spec notes. Co-Authored-By: Claude Fable 5.1 <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
SAST bake-off slice (OWASP + Juliet + Python)
Labeled true/false cases from the suites SAST papers actually quote. Not cheat-sheet fixtures and not the 90% smoke gate.
| Suite | Upstream | License | Protocol |
|---|---|---|---|
| OWASP Benchmark Java v1.2 | OWASP-Benchmark/BenchmarkJava | GPL-2.0 | true file = TP if we fire; false file = FP if we fire. Score = TPR − FPR. |
| OWASP Benchmark Python v0.1 | OWASP-Benchmark/BenchmarkPython | GPL-3.0 | Same CSV protocol (expectedresults-0.1.csv). |
| Juliet C 1.3 | arichardson/juliet-test-suite-c | CC0 | bad() vs goodB2G() / goodG2B(). |
| Juliet Java 1.3 | UnitTestBot/juliet-java-test-suite | CC0 | bad() vs goodG2B() (same sink, hardcoded source). |
This directory vendors a representative slice, not the full 2,740 / 180k cases.
uv run ousast pairs --slice sast
slice=sast is an honesty dashboard: CI still only fails the local fixture pairs.
Juliet goodG2B often keeps the dangerous sink (strcpy, Runtime.exec); a
regex harness is supposed to keep firing there.