mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-16 03:49:26 +00:00
feat(pre-push-safety-net): bound optional assistance to admitted witnesses
This commit is contained in:
@@ -1,6 +1,6 @@
|
||||
# Brief: pre-push-safety-net
|
||||
|
||||
Status: requirements, design and tasks approved; groups 1–6 implemented and verified (20/27 subtasks). Compatible preparation/graph/query reuse is demonstrated, including packaged JavaScript evidence equivalence. Groups 7–8 pending. Rollout remains NO-GO: changed-code feasibility and independent capability qualification are not established; normal capabilities remain disabled. Language: English.
|
||||
Status: requirements, design and tasks approved; groups 1–7 implemented and verified (23/27 subtasks). Compatible preparation/graph/query reuse is demonstrated, including packaged JavaScript evidence equivalence. Group 7 adds multi-ref Git input, non-destructive opt-in hook integration and bounded optional witness selection. Group 8 pending. Rollout remains NO-GO: changed-code feasibility and independent capability qualification are not established; normal capabilities remain disabled. Language: English.
|
||||
|
||||
## Problem
|
||||
|
||||
|
||||
@@ -413,3 +413,21 @@ admit passing capabilities in advisory mode; consider opt-in blocking only after
|
||||
LLM ranking is a later experiment, not a prerequisite. If realistic changed-unit rebuilds cannot
|
||||
meet the deadline/coverage gates, keep the hook experimental and revisit the graph strategy with
|
||||
measurements; do not solve the miss by silently shrinking context or lowering alert standards.
|
||||
|
||||
### Group 7 integration decisions
|
||||
|
||||
The Git interface produces one aggregate report, retaining all resolved updates and
|
||||
per-comparison change contexts. Deletions/no-work are not applicable; unresolved
|
||||
updates remain gaps. The hook installer resolves only the effective hook directory
|
||||
before exclusively creating the final filename, so final symlinks cannot redirect an
|
||||
installation. Explicit prior hooks retain their original input, arguments and exit
|
||||
behavior; their own runtime is separate from the Safety Net analysis deadline.
|
||||
|
||||
Optional model assistance is presentation-only witness selection after admission.
|
||||
Only an existing admitted witness index can be selected. No generated prose becomes
|
||||
an alert, and no model response changes scope, evidence strength, novelty, admission
|
||||
or enforcement. A model/prompt-specific response is not cached; deterministic analysis
|
||||
artifacts still use their existing semantic identities. Configuration, prompt
|
||||
preparation and HTTP calls share the transaction deadline. Pre-push never implicitly
|
||||
loads the working tree's `.env`; explicit model configuration uses the existing loader
|
||||
with that behavior disabled. Unknown price or missing usage is an unknown cost, not zero.
|
||||
|
||||
@@ -75,3 +75,45 @@ Verify with disposable repositories and a local bare remote, never this project'
|
||||
- CLAIM: Task 7.2 non-destructive hook integration delivered.
|
||||
- EVIDENCE: `/tmp/group7-2-full.log`, `/tmp/group7-2-final-focused.log`; static checks and shell/CLI smoke exit 0.
|
||||
- GAPS: No hook installed in this project's actual repository; rollout remains NO-GO.
|
||||
|
||||
## Task 7.3 brief
|
||||
|
||||
Requirements 3.5, 4.1, 4.2 and 7.1–7.3; design: Delta and Actionability Policy,
|
||||
Push runner and existing endpoint/redaction integration. Explicit endpoint/model only,
|
||||
one remaining deadline, no-model identity preserved. Assistance selects existing
|
||||
admitted witness indices and cannot contribute prose, evidence, a rung or a finding.
|
||||
Record prompt/model/usage/cost provenance and retain deterministic fallback. Verify
|
||||
misleading/delayed replies, outbound redaction, no unadmitted input, and an actual
|
||||
loopback HTTP request through the existing endpoint adapter. No external provider used.
|
||||
|
||||
## Status Report
|
||||
- STATUS: READY_FOR_REVIEW
|
||||
- TASK: 7.3
|
||||
- RED_PHASE_OUTPUT: `/tmp/group7-3-red.log`: six failures with the feature OFF, default no-model identity passes. ON: seven passed; switch removed. Final assistance/report suite: 27 passed, including endpoint receipts, delayed/misleading replies, redaction, dotenv isolation and witness/location binding.
|
||||
- TESTS_RUN: full suite 1295 passed, 9 skipped in 149.70s; final 27 focused tests validate the subsequent renderer binding refinement. Ruff, formatting (254 files), mypy (115 source files), compileall and diff checks pass.
|
||||
- CONCERNS: cost is explicitly unknown when receipt/pricing is missing; cancelled remote requests may still incur provider cost. All provider tests use synthetic clients or loopback HTTP. No normal capability enabled.
|
||||
|
||||
## Review Verdict
|
||||
- VERDICT: APPROVED
|
||||
- TASK: 7.3
|
||||
- MECHANICAL_RESULTS: actual code/diff reviewed against requirements 3.5, 4.1, 4.2 and 7.1–7.3. Regression/static checks pass. No new placeholders, hardcoded credentials or scope/ranking/admission changes. Synthetic key strings occur only in redaction controls.
|
||||
- FINDINGS: repaired implicit pre-deadline dotenv loading; explicit configuration reuses the existing loader with dotenv disabled. Model-selected witness location and change explanation remain bound to that witness's admitted disposition. Incomplete usage is never priced as zero. Unknown/extra model response fields cannot create claims.
|
||||
- RED phase: VERIFIED.
|
||||
- SUMMARY: Optional assistance is bounded presentation selection over existing admitted evidence, with deterministic fallback.
|
||||
|
||||
## Verification Result
|
||||
- STATUS: VERIFIED
|
||||
- CLAIM_TYPE: TASK
|
||||
- CLAIM: Task 7.3 and group 7 integration are implemented and locally verified.
|
||||
- EVIDENCE: `/tmp/group7-final-suite.log` (1295 passed, 9 skipped), `/tmp/group7-3-final-assistance-report.log` (27 passed), `/tmp/group7-3-final-focused.log` (34 passed). Detection gate 44/47, FP 0/44; map gate PASS; pair gate 3/3, labeled 28/28, zero fix leaks. Ruff, mypy, formatting, compileall and shell syntax pass. Docker build exit 0; image `sha256:af5b896b406fe98acb3d00bb2180a7549febd6dd285f0b63087538118e52e1f4`. Network-disabled packaged CLI help and real empty Git-stdin transaction both exit 0 as UID 1000; readable 1085-byte artifact records `not_applicable`, model disabled, no scans.
|
||||
- GAPS: Packaged smoke verifies liveness and input/report integration, not detection performance or alert eligibility. Main-context review fallback is not independent review. Group 8 remains pending and rollout remains NO-GO.
|
||||
|
||||
## Current specification status
|
||||
|
||||
Requirements, design and tasks remain approved. Implementation: 23/27 executable
|
||||
subtasks complete (85.2%); groups 1–7 complete, group 8 has four pending tasks.
|
||||
No new implementation blocker. Upstream contributor-scan contracts and existing ranker
|
||||
remain unchanged. Revalidate multi-ref deadline/cancellation, hook failures and model
|
||||
fallback in group 8.1, representative changed-code performance in 8.2, independent
|
||||
capability admission in 8.3, and packaged end-to-end usefulness in 8.4. The accepted
|
||||
rollout gates are unchanged; zero enabled capabilities cannot satisfy them.
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"feature_name": "pre-push-safety-net",
|
||||
"created_at": "2026-09-12T15:35:00Z",
|
||||
"updated_at": "2026-09-13T15:45:15.902603+00:00",
|
||||
"updated_at": "2026-09-13T16:03:14.583834+00:00",
|
||||
"language": "en",
|
||||
"phase": "implementation",
|
||||
"approvals": {
|
||||
@@ -19,7 +19,7 @@
|
||||
}
|
||||
},
|
||||
"ready_for_implementation": true,
|
||||
"implementation_approval_note": "Group 7 authorized; 7.1\u20137.2 verified, 7.3 next. Default capability registry empty; rollout NO-GO.",
|
||||
"implementation_approval_note": "Groups 1\u20137 implemented and verified (23/27). Group 7 authorized as a whole; manual Kiro review fallback after fresh dispatch thread limit. Multi-ref input, explicit hook integration and optional bounded witness selection verified. Default capability registry remains empty and rollout remains NO-GO; group 8 pending.",
|
||||
"depends_on": [
|
||||
"contributor-scan"
|
||||
],
|
||||
@@ -109,9 +109,10 @@
|
||||
"6.3",
|
||||
"6.4",
|
||||
"7.1",
|
||||
"7.2"
|
||||
"7.2",
|
||||
"7.3"
|
||||
],
|
||||
"remaining_tasks": 5,
|
||||
"remaining_tasks": 4,
|
||||
"verification_record": "implementation-group-7.md",
|
||||
"verification_records": [
|
||||
"implementation-group-1.md",
|
||||
@@ -122,7 +123,7 @@
|
||||
"implementation-group-6.md",
|
||||
"implementation-group-7.md"
|
||||
],
|
||||
"next_group": 7,
|
||||
"next_group": 8,
|
||||
"blocked_tasks": [],
|
||||
"feasibility": "NO-GO for rollout: compatible identical-revision reuse verified; representative changed-code p95/completion and independent alert eligibility remain unmet."
|
||||
}
|
||||
|
||||
@@ -190,7 +190,7 @@ independent capability qualification. Groups 7–8 remain pending; rollout remai
|
||||
- _Depends: 5.1, 6.2, 6.3_
|
||||
- _Requirements: 1.1, 2.2, 4.1, 4.2, 4.3, 4.4, 7.3_
|
||||
|
||||
- [ ] 7. Connect the real push interface without adding noise
|
||||
- [x] 7. Connect the real push interface without adding noise
|
||||
- [x] 7.1 Execute Git's full push input and enforcement policy
|
||||
- Extend the replay command to consume all update lines and remote arguments, aggregating per-ref comparisons under one deadline and preserving independent coverage and finding statuses.
|
||||
- Apply advisory, opt-in blocking and separate incomplete-coverage policy; keep external services disabled by default and normal alerts limited to admitted capabilities.
|
||||
@@ -204,7 +204,7 @@ independent capability qualification. Groups 7–8 remain pending; rollout remai
|
||||
- Done when a disposable Git remote accepts/rejects pushes correctly with an existing stdin-reading hook, and removal restores the preceding behavior without touching source or refs.
|
||||
- _Boundary: Result and Hook Adapter, CLI integration_
|
||||
- _Requirements: 1.2, 5.3, 5.4, 7.4_
|
||||
- [ ] 7.3 Keep optional model assistance within existing evidence rules
|
||||
- [x] 7.3 Keep optional model assistance within existing evidence rules
|
||||
- Reuse explicit endpoint configuration and redaction only when enabled; propagate the remaining deadline to model calls and record model/prompt/cost provenance.
|
||||
- Restrict explanations to existing witnesses and preserve deterministic no-model behavior; do not implement a new learned ranker or prompt optimizer in this task.
|
||||
- Done when a delayed or misleading model cannot extend the push budget, leak unredacted input in the tested path, raise a rung or bypass actionability admission.
|
||||
|
||||
@@ -46,7 +46,7 @@ engine are out of scope.
|
||||
- [ ] pre-push-safety-net — Immutable pushed tips, existing-ranker integration, compatible artifact
|
||||
reuse, total deadline, change attribution, actionability admission and compact hook behavior.
|
||||
Dependencies: contributor-scan. Requirements/design approved 2026-09-12; tasks generated
|
||||
(8 groups, 27 executable subtasks), independently reviewed and approved; groups 1–6 are implemented and verified (20/27 tasks), including the resolved frontend-retention prerequisite.
|
||||
(8 groups, 27 executable subtasks), independently reviewed and approved; groups 1–7 are implemented and verified (23/27 tasks), including the resolved frontend-retention prerequisite.
|
||||
|
||||
### Next concrete step
|
||||
|
||||
@@ -66,7 +66,8 @@ replay took 2.00s. Source/declaration changes invalidate affected units. This co
|
||||
control does not establish representative changed-code p95, independent alert precision or
|
||||
eligibility. Rollout remains NO-GO and normal capabilities remain disabled.
|
||||
|
||||
Group 7 is next for the real multi-ref push interface and opt-in hook integration. Group 8
|
||||
Group 7 now provides the real multi-ref push interface, opt-in hook integration and bounded
|
||||
optional selection among already-admitted witnesses. Group 8 is next and
|
||||
must still pass the joint gates on declared populations. Keep the existing thresholds and
|
||||
profile preparation, graph construction and queries separately when investigating misses;
|
||||
do not substitute identical-tip hits for warm changed-code performance.
|
||||
|
||||
@@ -79,9 +79,9 @@ updates as a shortcut to an unmeasured latency promise.
|
||||
The product direction above is user-requested. The amended requirements, design and initial
|
||||
evaluation profile in `pre-push-safety-net` were approved for task generation on 2026-09-12
|
||||
("ok, next kiro phase"). The maintainer then approved the task plan and starting group 1
|
||||
("approve.") and then group 2 ("ok, next kiro phase, approved"). Groups 1–6 are now
|
||||
implemented, reviewed and verified (20/27), including the frontend-retention prerequisite
|
||||
and the deadline regression exposed by full Node replay. Tasks 4.3 and groups 5–6 used the
|
||||
("approve.") and then group 2 ("ok, next kiro phase, approved"). Groups 1–7 are now
|
||||
implemented, reviewed and verified (23/27), including the frontend-retention prerequisite
|
||||
and the deadline regression exposed by full Node replay. Tasks 4.3 and groups 5–7 used the
|
||||
explicit Kiro manual review fallback after the agent thread limit.
|
||||
|
||||
The first newly frozen PMPro/Node experiment is measured **NO-GO**: all six cold cases
|
||||
@@ -96,4 +96,6 @@ dominance context projection remain unproven; the default capability registry is
|
||||
Group 6 verified equal cached/uncached evidence and current admission in a packaged authored
|
||||
JavaScript control. Source/declaration edits invalidate affected units. Identical-revision
|
||||
latency does not qualify representative changed-code performance. Requirements and rollout
|
||||
thresholds remain unchanged; groups 7–8 and independent capability admission remain pending.
|
||||
thresholds remain unchanged; group 8 and independent capability admission remain pending. Group 7 adds exact multi-ref
|
||||
transactions, explicit non-overwriting hook integration and bounded optional witness
|
||||
selection. No model output can create evidence, raise a rung or bypass admission.
|
||||
|
||||
Reference in New Issue
Block a user