mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-16 03:49:26 +00:00
feat(pre-push-safety-net): preserve exact ranker scope and outcomes
This commit is contained in:
@@ -61,3 +61,32 @@ inside the driver; this does not change the ranker. Global mypy now checks all
|
||||
nine skipped, 96.85 seconds; global Ruff, formatting (219 files), mypy and diff
|
||||
checks passed. Verification status: VERIFIED, claim type TASK. This completes
|
||||
8/27 tasks; tasks 3.2–3.4 remain in progress for this group.
|
||||
|
||||
## Task 3.2 evidence
|
||||
|
||||
Independent review APPROVED; verification status VERIFIED, claim type TASK.
|
||||
The fresh reviewer suite passed 1,105 tests with nine skipped in 101.18 seconds.
|
||||
Global Ruff, formatting (221 files), mypy (108 source files) and diff checks passed.
|
||||
The reviewer also exercised 24 combinations of order, budget and failed query
|
||||
families, retaining all 12 declared questions and reconciling coverage totals.
|
||||
|
||||
The driver accepts explicit `ranking_mode`, `unit` and `population_complete` while
|
||||
preserving ordinary defaults. Scope and execution requests are produced together
|
||||
from the existing ranker's final order. Selected/deferred identities, normalized
|
||||
evidence, unchanged tier/score, per-question outcomes and per-family completion
|
||||
travel to the CLI directly. Missing evidence stays unknown. Partial graphs cannot
|
||||
justify tier-zero pruning or completed coverage. Raw query answers survive a later
|
||||
timeout even when arbitration cannot complete. A completed outcome denotes query
|
||||
and arbitration completion within the modeled boundary, not a safety assertion.
|
||||
|
||||
`benchmarks/measurements/2026-09-13-ranker-scope-smoke.json` records a rebuilt image
|
||||
whose backend, driver, contracts and CLI hashes match the source. Real PHP (78 bytes)
|
||||
and JavaScript (69 bytes) fixtures each selected `risky` over the initially higher
|
||||
static-ranked `safe`. Both selected questions had tier 4 and score 2.5; execution
|
||||
outcome IDs matched selection, and each family reported one selected/completed,
|
||||
one deferred and zero unanswered questions. Each produced one controlled finding.
|
||||
PHP took 61.57 seconds and JavaScript 51.27 seconds under one 900-second lab budget.
|
||||
This manually declared entry-region smoke does not validate mapper discovery,
|
||||
PMPro/NodeGoat replay, capability admission or hook latency.
|
||||
|
||||
Nine of 27 tasks are now complete; tasks 3.3–3.4 remain for this group.
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"feature_name": "pre-push-safety-net",
|
||||
"created_at": "2026-09-12T15:35:00Z",
|
||||
"updated_at": "2026-09-13T08:02:59.282414+00:00",
|
||||
"updated_at": "2026-09-13T08:16:14.274231+00:00",
|
||||
"language": "en",
|
||||
"phase": "implementation",
|
||||
"approvals": {
|
||||
@@ -70,9 +70,10 @@
|
||||
"2.1",
|
||||
"2.2",
|
||||
"2.3",
|
||||
"3.1"
|
||||
"3.1",
|
||||
"3.2"
|
||||
],
|
||||
"remaining_tasks": 19,
|
||||
"remaining_tasks": 18,
|
||||
"verification_record": "implementation-group-3.md",
|
||||
"verification_records": [
|
||||
"implementation-group-1.md",
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
# Implementation Plan: pre-push-safety-net
|
||||
|
||||
Status: task plan approved by the maintainer's "approve." on 2026-09-12. Groups 1 and 2 (tasks 1.1–2.3)
|
||||
are complete after independent review and fresh verification. Group 3 is in progress (3.1 verified); groups 4–8 remain pending;
|
||||
are complete after independent review and fresh verification. Group 3 is in progress (3.1–3.2 verified); groups 4–8 remain pending;
|
||||
the hook and capability admission are not yet implemented. See `implementation-group-1.md` and `implementation-group-2.md`.
|
||||
|
||||
## Execution contract
|
||||
@@ -87,13 +87,14 @@ the hook and capability admission are not yet implemented. See `implementation-g
|
||||
- _Depends: 1.4_
|
||||
- _Requirements: 4.1, 4.2, 7.3_
|
||||
- _Verified 2026-09-13:_ independent review APPROVED; fresh suite 1,089 passed, nine skipped; global Ruff/format and mypy (108 files) passed. Real hanging probe/frontend/overlay/census/query processes obey the shared deadline and cancellation allowance; retries do not reset it. Non-root packaged PHP/JavaScript graph/query smoke passed with matching source hashes under one 900-second lab budget. See `implementation-group-3.md` and `benchmarks/measurements/2026-09-13-engine-budget-smoke.json`. Runtime readiness only; no hook latency claim. Former three mypy errors resolved by a behavior-preserving variable rename.
|
||||
- [ ] 3.2 Return the ranker's exact selected and deferred work
|
||||
- [x] 3.2 Return the ranker's exact selected and deferred work
|
||||
- Integrate the existing ranker through explicit mode selection and shared question identity; record selected/deferred IDs, evidence and reasons directly from the final order.
|
||||
- Preserve unanswered evidence as unknown rather than tier zero; report per-family coverage so taint ranking cannot masquerade as authorization/configuration coverage.
|
||||
- Add paired PHP/JavaScript normalized-evidence contract tests asserting identical tier/score and selection decisions for equivalent input and budgets, without repository/language priority exceptions.
|
||||
- Done when changing order produces matching execution and manifest IDs, with no reconstruction from an earlier list, and normalized cross-language equivalence tests pass.
|
||||
- _Boundary: Ranker Scope Contract, generic driver integration_
|
||||
- _Requirements: 2.1, 2.2, 6.4, 7.3, 8.1, 8.2_
|
||||
- _Verified 2026-09-13:_ independent review APPROVED; 1,105 passed, nine skipped; global Ruff/format/mypy passed. Twenty-four independent order/budget/missing-family combinations retained all question identities. Packaged PHP/JavaScript smoke selected the same controlled risky operation at tier 4/score 2.5, with exact execution/outcome IDs and reconciled family coverage. Missing evidence, partial graphs and interrupted arbitration remain explicit; CLI consumes authoritative scope. Evidence: `benchmarks/measurements/2026-09-13-ranker-scope-smoke.json`. No new ranking weights or hook admission claim.
|
||||
- [ ] 3.3 Connect change context to supported first-party relationships
|
||||
- Use existing entry/callee, field, hook and guard relationships to supply affected context to ranked questions, including unchanged sinks after source/guard changes.
|
||||
- Preserve current abstract semantics; absent dynamic or external relationships remain unresolved rather than guessed safe. A required new feature or fact is reported to its owning spec.
|
||||
|
||||
Reference in New Issue
Block a user