mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-16 03:49:26 +00:00
feat(pre-push-safety-net): establish shared push and engine contracts
This commit is contained in:
@@ -1,17 +1,23 @@
|
||||
# Overview — what this is, and what is actually true of it
|
||||
|
||||
**Current direction, 2026-09-12:** an actionable pre-push security safety net for AI-accelerated
|
||||
development across supported ecosystems. See `safety-net.md` and the proposed `pre-push-safety-net`
|
||||
development across supported ecosystems. See `safety-net.md` and the approved `pre-push-safety-net`
|
||||
spec. The ranker remains the scope mechanism; third-party vendor code stays outside targets and
|
||||
graphs. Advisory must meet the same actionability bar as blocking.
|
||||
|
||||
**Historical scoreboard below:** it was written September 9. Later commits record a completing
|
||||
whole-PMPro scan and Joern 4.0.625 ranking measurements (82/91/54 on the three plugins), while
|
||||
default installation pins remain 4.0.623. The pre-push workflow, changed-code latency and actionable
|
||||
default installation pins are now aligned to 4.0.625 with verified PHP/JavaScript image smoke.
|
||||
The pre-push workflow, changed-code latency and actionable
|
||||
precision are not demonstrated by those results. In particular, the manually selected two-file
|
||||
slice below is a useful experiment, not evidence that an automatic push scope would select the
|
||||
same context. Current scope/status corrections live in `flow-aware-ranking/brief.md` and the roadmap.
|
||||
|
||||
The new scoring instrument retains missing targets and unanswered queries. Historical rank positions
|
||||
do not establish detection recall; VAmPI's old budget of 16 omitted its unmatched SQLI target. The
|
||||
corrected positional run keeps that target unresolved. Labeled PMPro/Node inputs and a frozen
|
||||
diagnostic profile are committed; independent capability evaluation is still required.
|
||||
|
||||
**Written 2026-09-09.** Fourteen specs have accumulated and the roadmap still described a pre-Joern world, so
|
||||
this is the synthesis above them: what the system does, what has been measured, and what is not true yet.
|
||||
Every number here comes from a committed artifact in `benchmarks/measurements/`.
|
||||
|
||||
@@ -21,8 +21,9 @@ engine are out of scope.
|
||||
|
||||
- Whole-PMPro analysis has completed with its known CVE; the earlier "does not finish" diagnosis
|
||||
is superseded. The recorded 72-minute scan is not a pre-push performance result.
|
||||
- The latest Joern 4.0.625 + layout ranking budgets are PMPro 82, WP Statistics 91, MW WP Form 54,
|
||||
VAmPI 16. `<50` remains unmet; VAmPI's transitive SQLI is not measured by this position metric.
|
||||
- Historical Joern 4.0.625 + layout position budgets are PMPro 82, WP Statistics 91, MW WP Form 54.
|
||||
`<50` remains unmet. VAmPI's old 16 omitted SQLI; the corrected instrument retains the unmatched
|
||||
target and reports no full-target budget or detection recall without the required evidence.
|
||||
- Equal-score tie-breaking alone cannot bring PMPro or WP Statistics under 50. The existing
|
||||
ranker is useful, but its measured ordering is not enabled by the ordinary CLI caller.
|
||||
- Docker/Compose/ops now default to 4.0.625; task 1.1 passed local PHP/JavaScript image smoke and independent review. No host deployment or hook latency claim follows from that runtime evidence.
|
||||
@@ -45,14 +46,14 @@ engine are out of scope.
|
||||
- [ ] pre-push-safety-net — Immutable pushed tips, existing-ranker integration, compatible artifact
|
||||
reuse, total deadline, change attribution, actionability admission and compact hook behavior.
|
||||
Dependencies: contributor-scan. Requirements/design approved 2026-09-12; tasks generated
|
||||
(8 groups, 27 executable subtasks), independently reviewed and approved; group 1 implementation started.
|
||||
(8 groups, 27 executable subtasks), independently reviewed and approved; group 1 is complete (4/27 tasks).
|
||||
|
||||
### Next concrete step
|
||||
|
||||
The maintainer approved `pre-push-safety-net/tasks.md` and starting group 1. Requirements and design
|
||||
were accepted when the maintainer requested the next Kiro phase; the 27-subtask plan passed independent
|
||||
sanity review. Complete group 1 (reproducible runtime, readable PMPro/Node inputs, honest scoring
|
||||
and shared contracts). The first complete experiment is 5.2, before persistent caching and hook
|
||||
Group 1 has passed independent review: reproducible PHP/JavaScript runtime, pinned PMPro/Node inputs,
|
||||
honest scoring and shared contracts. Continue with group 2's immutable push snapshots under the
|
||||
approved task plan; see `pre-push-safety-net/implementation-group-1.md` for evidence and limits.
|
||||
The first complete experiment is 5.2, before persistent caching and hook
|
||||
installation work. Keep PMPro as the primary core-optimization example, then freeze the core and
|
||||
prove a changed security flow and its fixed twin on a JavaScript/Express Node API workload through
|
||||
the same ranker, with a real deadline and vendor-free graph. Include benign pushes and retain Python
|
||||
|
||||
@@ -78,5 +78,6 @@ updates as a shortcut to an unmeasured latency promise.
|
||||
|
||||
The product direction above is user-requested. The amended requirements, design and initial
|
||||
evaluation profile in `pre-push-safety-net` were approved for task generation on 2026-09-12
|
||||
("ok, next kiro phase"). Task approval remains pending. This is not authorization to implement
|
||||
or a claim that hook performance or capability admission has been demonstrated.
|
||||
("ok, next kiro phase"). The maintainer then approved the task plan and starting group 1
|
||||
("approve."). Implementation is authorized under that plan; task completion still requires
|
||||
independent review and fresh evidence. Hook performance and capability admission remain unproven.
|
||||
|
||||
Reference in New Issue
Block a user