mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-18 12:39:28 +00:00
docs(pre-push-safety-net): record approved design and implementation plan
This commit is contained in:
@@ -1,5 +1,29 @@
|
||||
# Design Document: contributor-scan
|
||||
|
||||
## Integration amendment — 2026-09-12 (draft, unapproved)
|
||||
|
||||
The maintainer has clarified the product as an actionable pre-push safety net across supported
|
||||
ecosystems. The proposed successor `../pre-push-safety-net/design.md` consumes this driver/backend
|
||||
without replacing its ranker: the ranker remains the scope mechanism, and vendor code stays out
|
||||
of both targets and graphs. Existing approvals below apply to the prior design, not this amendment.
|
||||
|
||||
Proposed shared seams needing requirements/design review before implementation:
|
||||
|
||||
- Accept an absolute deadline across graph build, overlays, retries and queries, not only region/call caps.
|
||||
- Accept an optional validated reusable graph lease while preserving ordinary fresh-build scans;
|
||||
backend remains authoritative for artifact identity, census, compatibility and cleanup.
|
||||
- Return exact selected/deferred question IDs and context boundaries from ranker decisions;
|
||||
callers must not reconstruct exclusions by slicing an earlier order.
|
||||
- Record engine, fact, query, layout and ranking provenance. Align Docker/Compose/ops pins with
|
||||
the evaluated baseline before packaging parity claims.
|
||||
- Resolve task 5.6's family-specific sanitizer semantics before admitting affected capabilities to
|
||||
a low-noise hook; ranker improvements do not repair detector semantics.
|
||||
|
||||
The push spec owns immutable Git inputs, comparison/admission policy, persistence and presentation.
|
||||
This spec owns reusable engine contracts and family correctness. No hook imports belong inside
|
||||
the detector, no second heuristic scope selector is introduced, and no task is marked complete by
|
||||
this documentation amendment. Revalidate normal scans and repository benchmarks when seams change.
|
||||
|
||||
## Overview
|
||||
|
||||
`model-grounded-detection` built an arbiter and measured it at 90.0% pair-correct / 6.7% leak on vibe-py. It
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"feature_name": "contributor-scan",
|
||||
"created_at": "2026-09-08T20:00:00Z",
|
||||
"updated_at": "2026-09-09T12:30:00Z",
|
||||
"updated_at": "2026-09-12T15:35:00Z",
|
||||
"language": "en",
|
||||
"phase": "implementation",
|
||||
"approvals": {
|
||||
@@ -22,5 +22,14 @@
|
||||
"depends_on": [
|
||||
"model-grounded-detection"
|
||||
],
|
||||
"approval_note": "2026-09-08 brief drafted after the maintainer asked what the spec state was and whether anything was forgotten. It was: model-grounded-detection is 16/16 complete and its arbiter is measured at 86.7% pair-correct / 6.7% leak, but the model layer has ZERO production callers -- ousast scan does not use it, so the feature is complete and unshipped at once. That is a scope gap in the predecessor (it had no task saying 'make this reachable', and its byte-identical-gates requirement is satisfied precisely because nothing is wired). This spec covers scan integration, whole-repo scanning against known vulnerable repositories, the deferred memory-family decision, and records the pipeline, ops and corpus work delivered outside any spec. Not approved; requirements not generated. ; 2026-09-08 requirements generated at the maintainer's direction ('we follow the spec and build integration afterwards'). ; 2026-09-09 requirements approved and design generated ('let us continue with these'): the model layer joins as one stage inside MAP beside the obligations checker, one CPG per scan, regions from entry points, a total budget rather than a per-region cap. Phase 2 (the first run on a pinned real repository) can invalidate the rest and comes before shipping. ; 2026-09-09 tasks approved by the maintainer ('approve the tasks and start group 1'); implementing group 1 (regions, driver, MAP stage). ; 2026-09-09 group 2 complete except 2.8/2.9/2.13-2.16, and it reshaped the phase as intended. Nine engine defects found by measurement and none by review, gates byte-identical through every fix -- which says the pair corpus could not have caught any of them. Three requirements amendments are DRAFTED AND UNAPPROVED at the end of requirements.md: (A) Req 5 should state what a report may repeat, because libpng emitted 198 findings with nothing in the spec forbidding it, and the model layer's own suspicion band is still 38 of vampi's 60; (B) a family that cannot decide a class must say so, because c/memory scored zero true positives on 89k lines of C for a structural reason and a silent scan reads as a clean bill of health; (C) Req 4.1 is satisfied by a checkout that cannot demonstrate detection at all, so it should require one that can. Separately, two task groups appended to tasks.md without passing requirements or design have been LIFTED OUT to the new finding-feedback-loop spec, where they belong. ; 2026-09-09 the three group-2 amendments APPROVED by the maintainer and folded into the numbered requirements: Req 5.5 (a report proportionate to the evidence behind a finding), Req 5.6 (a family that cannot decide a class states the limit, and no scan presents absence of findings as evidence of absence), Req 4.6 (at least one checkout carrying a vulnerability a family can decide in principle). Req 5.5 is half implemented by task 2.11 -- pattern matches -- and task 2.17 was added for the model layer's own suspicion band, which is 38 of vampi's 60 findings and is an enumerated candidate the graph could not decide with a judge's opinion attached rather than a reasoned claim."
|
||||
"approval_note": "2026-09-08 brief drafted after the maintainer asked what the spec state was and whether anything was forgotten. It was: model-grounded-detection is 16/16 complete and its arbiter is measured at 86.7% pair-correct / 6.7% leak, but the model layer has ZERO production callers -- ousast scan does not use it, so the feature is complete and unshipped at once. That is a scope gap in the predecessor (it had no task saying 'make this reachable', and its byte-identical-gates requirement is satisfied precisely because nothing is wired). This spec covers scan integration, whole-repo scanning against known vulnerable repositories, the deferred memory-family decision, and records the pipeline, ops and corpus work delivered outside any spec. Not approved; requirements not generated. ; 2026-09-08 requirements generated at the maintainer's direction ('we follow the spec and build integration afterwards'). ; 2026-09-09 requirements approved and design generated ('let us continue with these'): the model layer joins as one stage inside MAP beside the obligations checker, one CPG per scan, regions from entry points, a total budget rather than a per-region cap. Phase 2 (the first run on a pinned real repository) can invalidate the rest and comes before shipping. ; 2026-09-09 tasks approved by the maintainer ('approve the tasks and start group 1'); implementing group 1 (regions, driver, MAP stage). ; 2026-09-09 group 2 complete except 2.8/2.9/2.13-2.16, and it reshaped the phase as intended. Nine engine defects found by measurement and none by review, gates byte-identical through every fix -- which says the pair corpus could not have caught any of them. Three requirements amendments are DRAFTED AND UNAPPROVED at the end of requirements.md: (A) Req 5 should state what a report may repeat, because libpng emitted 198 findings with nothing in the spec forbidding it, and the model layer's own suspicion band is still 38 of vampi's 60; (B) a family that cannot decide a class must say so, because c/memory scored zero true positives on 89k lines of C for a structural reason and a silent scan reads as a clean bill of health; (C) Req 4.1 is satisfied by a checkout that cannot demonstrate detection at all, so it should require one that can. Separately, two task groups appended to tasks.md without passing requirements or design have been LIFTED OUT to the new finding-feedback-loop spec, where they belong. ; 2026-09-09 the three group-2 amendments APPROVED by the maintainer and folded into the numbered requirements: Req 5.5 (a report proportionate to the evidence behind a finding), Req 5.6 (a family that cannot decide a class states the limit, and no scan presents absence of findings as evidence of absence), Req 4.6 (at least one checkout carrying a vulnerability a family can decide in principle). Req 5.5 is half implemented by task 2.11 -- pattern matches -- and task 2.17 was added for the model layer's own suspicion band, which is 38 of vampi's 60 findings and is an enumerated candidate the graph could not decide with a judge's opinion attached rather than a reasoned claim.",
|
||||
"pending_amendments": [
|
||||
{
|
||||
"date": "2026-09-12",
|
||||
"document": "design.md",
|
||||
"topic": "Pre-push integration: shared deadline, reusable graph lease, exact ranker scope and provenance; preserve vendor exclusion.",
|
||||
"approved": false,
|
||||
"ready_for_implementation": false
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user