mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-17 12:09:27 +00:00
feat(pre-push-safety-net): compare targeted base and head evidence
This commit is contained in:
@@ -0,0 +1,41 @@
|
||||
# Group 4 implementation and verification
|
||||
|
||||
Date: 2026-09-13. The maintainer authorized group 4 as a whole. Scope is tasks
|
||||
4.1–4.3: targeted novelty comparison, actionability admission and compact reporting.
|
||||
Groups 1–3 are verified at `6ba2b06`; full transaction replay remains task 5.1.
|
||||
|
||||
Status: implementation in progress. Task completion requires independent review
|
||||
and fresh evidence. No production capability is enabled for normal hook alerts;
|
||||
capability admission still requires task 8.3 evaluation.
|
||||
|
||||
## Verification boundary
|
||||
|
||||
Exercise production policy functions with immutable Git snapshots and actual engine
|
||||
answers, plus deterministic error-path and equivalent-origin controls. Prove source
|
||||
bytes were read before interpreting empty answers. Synthetic evaluated capabilities
|
||||
exercise admission mechanics only and never establish real capability eligibility.
|
||||
The established gates protect detector regressions; these controls do not establish
|
||||
production hook latency, broad alert precision or feature-level readiness.
|
||||
|
||||
## Task 4.1 verification
|
||||
|
||||
VERIFIED (TASK scope), independent review APPROVED. Full suite: 1,167 passed,
|
||||
nine skipped in 122.44 seconds; global Ruff/format (233 files), mypy (110 source
|
||||
files), all three regression gates and diff checks passed. The first OFF control
|
||||
failed on novelty behavior. Review regressions reproduced false NEW classifications
|
||||
from wrong-mechanism base rows and non-string operations; both now stay unknown.
|
||||
|
||||
`benchmarks/measurements/2026-09-13-delta-comparison-smoke.json` records actual
|
||||
JavaScript guard removal as worsened, unchanged backlog moved by comments as
|
||||
unchanged, and a PHP source newly connected to a mapped unchanged sink as new.
|
||||
All inputs were read and both sides' questions completed. Current reviewed policy
|
||||
was independently rechecked against captured actual answers after the schema-only
|
||||
repair; the artifact preserves exact source hashes and that narrower provenance.
|
||||
Concurrent lab costs were 214.81, 214.52 and 291.01 seconds respectively under
|
||||
900-second budgets. These are not hook latency measurements.
|
||||
|
||||
Automatic dominance-only change-context projection remains an explicit upstream
|
||||
limitation for the later replay integration. These controls use declared questions
|
||||
and Git context directly; no missing-coverage result was erased. Ambiguous source
|
||||
and identifier changes remain unknown. Task 4.1 brings progress to 12/27 tasks;
|
||||
admission and rendering remain pending within this authorized group.
|
||||
@@ -178,3 +178,22 @@ incomplete coverage. A synthetic graph file must never compensate for an omitted
|
||||
real source, and a wrapper must not discard evidence supplied by a frontend.
|
||||
See `implementation-group-3.md` for final evidence. Eleven of 27 tasks are complete;
|
||||
change attribution, actionability admission and compact output are next.
|
||||
|
||||
## Group 4 comparison controls (2026-09-13)
|
||||
|
||||
Real JavaScript dominance rows distinguish a guarded base operation from an
|
||||
unguarded head operation while retaining a guarded sibling. The direct task 4.1
|
||||
comparator can consume these answers with declared questions and immutable Git
|
||||
span correspondence. This does not establish automatic context projection for
|
||||
that family: the existing driver's change-context evidence pass currently projects
|
||||
taint questions, and a dominance-only scan with `change_context` reports
|
||||
`context_projection_unavailable:contributor-scan`. Preserve that limitation in the
|
||||
full replay integration at 5.1; any extension belongs to the contributor-scan
|
||||
context contract, not a hook-specific graph or detector workaround.
|
||||
|
||||
Duplicate identical source lines also remain ambiguous to the snapshot adapter.
|
||||
Controls with unique lines prove the narrow supported correspondence path; they
|
||||
do not establish general identity across arbitrary refactoring. Changed source
|
||||
text alone is insufficient to distinguish a new source from a benign identifier
|
||||
rename. Those comparisons remain unknown. A completed comparable base with no flow
|
||||
to an unchanged mapped sink supplies a supported new-connection control instead.
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"feature_name": "pre-push-safety-net",
|
||||
"created_at": "2026-09-12T15:35:00Z",
|
||||
"updated_at": "2026-09-13T09:50:01.821464+00:00",
|
||||
"updated_at": "2026-09-13T10:26:47.307154+00:00",
|
||||
"language": "en",
|
||||
"phase": "implementation",
|
||||
"approvals": {
|
||||
@@ -19,7 +19,7 @@
|
||||
}
|
||||
},
|
||||
"ready_for_implementation": true,
|
||||
"implementation_approval_note": "Groups 1\u20133 verified complete. Contributor-scan frontend-retention prerequisite 2.18 approved and verified. Remaining approved tasks require independent review and fresh verification.",
|
||||
"implementation_approval_note": "Groups 1\u20133 verified complete. Maintainer explicitly authorized group 4 as a whole; each task requires independent review and fresh verification.",
|
||||
"depends_on": [
|
||||
"contributor-scan"
|
||||
],
|
||||
@@ -55,6 +55,11 @@
|
||||
"at": "2026-09-13T09:17:01.788332+00:00",
|
||||
"source": "approved",
|
||||
"action": "Approved contributor-scan frontend retention prerequisite; implement it and resume task 3.4 review and verification."
|
||||
},
|
||||
{
|
||||
"at": "2026-09-13T10:26:47.307154+00:00",
|
||||
"source": "move on to group 4 as a whole",
|
||||
"action": "Authorized all of group 4 (4.1\u20134.3), sequential implementation, independent task review and fresh verification. Groups 5\u20138 remain pending."
|
||||
}
|
||||
],
|
||||
"task_review": {
|
||||
@@ -78,14 +83,16 @@
|
||||
"3.1",
|
||||
"3.2",
|
||||
"3.3",
|
||||
"3.4"
|
||||
"3.4",
|
||||
"4.1"
|
||||
],
|
||||
"remaining_tasks": 16,
|
||||
"verification_record": "implementation-group-3.md",
|
||||
"remaining_tasks": 15,
|
||||
"verification_record": "implementation-group-4.md",
|
||||
"verification_records": [
|
||||
"implementation-group-1.md",
|
||||
"implementation-group-2.md",
|
||||
"implementation-group-3.md"
|
||||
"implementation-group-3.md",
|
||||
"implementation-group-4.md"
|
||||
],
|
||||
"next_group": 4,
|
||||
"blocked_tasks": []
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
# Implementation Plan: pre-push-safety-net
|
||||
|
||||
Status: task plan approved by the maintainer's "approve." on 2026-09-12. Groups 1–3 (tasks 1.1–3.4)
|
||||
are complete after independent review and fresh verification (11/27 subtasks). Groups 4–8 remain pending;
|
||||
are complete after independent review and fresh verification (12/27 subtasks, including task 4.1). Tasks 4.2–4.3 and groups 5–8 remain pending;
|
||||
the hook and capability admission are not yet implemented. See `implementation-group-1.md` and `implementation-group-2.md`.
|
||||
|
||||
## Execution contract
|
||||
@@ -113,10 +113,11 @@ the hook and capability admission are not yet implemented. See `implementation-g
|
||||
- _Verified 2026-09-13:_ independent review APPROVED; 1,151 passed, nine skipped; Ruff/format/mypy pass. Real PHP/JavaScript graphs retain tests/browser source, omit vendor nodes and share one ranker limit. Named census survives PHP aggregation; missing source names and unreadable discovery inputs remain explicit. Python exercises all three batch census schemas. Evidence: `benchmarks/measurements/2026-09-13-partition-scope-smoke.json`. No hook capability or latency claim.
|
||||
|
||||
- [ ] 4. Implement change attribution and actionable output
|
||||
- [ ] 4.1 Compare targeted base/head evidence for defect novelty
|
||||
- [x] 4.1 Compare targeted base/head evidence for defect novelty
|
||||
- Establish stable mechanism/operation identity with rename/span mapping and collect comparable base evidence under the same semantics and remaining deadline.
|
||||
- Distinguish new, worsened, unchanged and unknown; an absent finding from a failed/truncated base is unknown. Support removed guards and new source connections at unchanged sinks.
|
||||
- Done when benign movement and unchanged backlog do not become new defects, while a supported removed-guard case has a concrete change-to-witness relationship.
|
||||
- _Verified 2026-09-13:_ independent review APPROVED; 1,167 tests passed, nine skipped; real JavaScript guard/movement and PHP connection controls passed. Final policy rechecked against captured actual answers after malformed-schema repair. See `implementation-group-4.md`.
|
||||
- _Boundary: Delta and Actionability Policy, engine comparison integration_
|
||||
- _Depends: 2.3, 3.3, 3.4_
|
||||
- _Requirements: 2.4, 3.1, 3.4, 4.1_
|
||||
|
||||
Reference in New Issue
Block a user