marius/Open_Ultra_SAST
1
0
Fork 0
mirror of https://github.com/norandom/OpenUltraSAST.git synced 2026-10-16 03:49:26 +00:00
Code Issues Packages Projects Releases Wiki Activity

spec(pre-push-safety-net): a harness must report the cause of its own failure

Browse Source 
Named by a week in which three separate causes looked identical as one count of unresolved. The
census reported 19 findings and 0 completed questions side by side without complaint, a stale image
measured the engine a bug had been diagnosed against, and a partition fix looked applied while an
unscoped copy of the same gap kept demoting everything.

Every harness record should carry the counts that disambiguate its failure modes, the degradations
whole rather than by reason, and the engine and ruleset identities actually used; and each harness
should assert its own consistency, because findings above zero with completion at zero is a
contradiction an instrument can notice faster than a person.

This was missed from the previous commit by an escaping error in the edit, not by intent.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
This commit is contained in:
Marius Ciepluch
2026-09-21 16:29:39 +02:00
co-authored by Claude Opus 5
parent e245040855
commit 032efc926c
1 changed files with 9 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.kiro/specs/pre-push-safety-net/tasks.md
+9
View File
@@ -594,6 +594,15 @@ Preserve all completed task evidence below.
- _Requirements: 3.1, 5.1, 5.6, 7.1_
- [ ] 16. Make a harness report the cause of its own failure (robustness)
- **Named by a week in which three separate causes looked identical.** The census reported 19 findings and 0 completed questions side by side for days without complaint; a stale image measured the engine a bug had been diagnosed against; and a partition fix looked applied while an unscoped copy of the same gap kept demoting everything. Each was found by reading code, and each would have been visible in a record that carried its own causes.
- Every harness result should carry the counts that disambiguate its failure modes, the degradations whole rather than by reason, and the engine and ruleset identities actually used. Two of these were added reactively during that week and should be the standard shape rather than a reaction.
- Add a self-consistency assertion to each harness. Findings above zero with completion at zero is a contradiction, and an instrument that says so loudly is the difference between an hour and a week.
- Done when each harness record carries its provenance and per-cause counts, and a deliberately contradictory result is reported as a contradiction rather than as a number.
- _Boundary: Evaluation harness_
- _Requirements: 8.1, 8.2_
## Implementation Notes
- Task5.2 measured PMPro timing out after both snapshots were materialized but before change context/graph analysis. Group6 must account for compatible snapshot/discovery reuse as well as graph reuse; identical-tip hits cannot substitute for the later representative changed-code latency gate. No new detector, scope heuristic or incremental CPG mutation is implied.