mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-16 03:49:26 +00:00
test(m3): Python transfer detects all three declared targets (M3 13.1)
Generalized the replay instrument so the case family is an argument rather than a hard-coded target, since a harness that only knows one language cannot say whether the design transfers. Targets, pins and expectations come from the committed inputs, so Python is asked exactly the way Node was, through the same core, ranker, comparison and admission. VAmPI-SQLI models/user_model.py:73:get_user detected, answered VAmPI-BOLA api_views/users.py:187:update_password detected, answered VAmPI-BOLA-BOOKS api_views/books.py:51:get_by_title detected, answered 35 of 35 questions complete on every case, nothing reported new, zero admitted defects. Two results stand out. The SQL injection is the evidence the milestone spec names as previously unresolved. And two of the three targets are access control, the family that could not complete a question on any repository before group 12, so this is the first time broken object-level authorization has been asked on Python at all, and it answered on both endpoints. VAmPI has the same revision on both sides, so the regression mode establishes detection and completion, never novelty. It asserts that directly: with nothing changed, a target reported new would be a comparison defect, and none was. That tests the novelty logic from the opposite side to the NodeGoat case. Identical revisions cannot show novelty, and VAmPI is inspected teaching data that cannot qualify production Python support. No capability enabled, no latency claim. 1,407 tests passed, nine skipped. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
This commit is contained in:
co-authored by
Claude Opus 5
parent
fc5dc453b6
commit
bb72fcd413
@@ -440,6 +440,23 @@ Preserve all completed task evidence below.
|
||||
- _Depends: 12.3_
|
||||
- _Requirements: 2.1, 2.2, 8.1_
|
||||
|
||||
|
||||
- [ ] 13. Show the shared design transfers across languages (release milestone M3)
|
||||
- [x] 13.1 Replay the declared Python family through the shared core
|
||||
- Generalize the replay instrument so a case family is an argument rather than a hard-coded target: a harness that only knows one language cannot say whether the design transfers. Targets, pins and expectations come from the committed inputs.
|
||||
- VAmPI is a regression snapshot with the same revision on both sides, so it can establish detection and completion but never novelty. The mode asserts that too: with nothing changed, a target reported new would be a comparison defect rather than a finding.
|
||||
- _Verified 2026-09-17:_ all three declared targets detected at their declared lines with their questions answered, 35 of 35 questions complete on every case, nothing reported new, zero admitted defects. The SQL injection at `models/user_model.py:73` is the evidence the milestone names as previously unresolved. Two of the three are access control, the family that could not complete a question anywhere before group 12, so this is the first time broken object-level authorization has been asked on Python. Evidence: `benchmarks/measurements/2026-09-17-m3-python-transfer.json`.
|
||||
- _Limits:_ identical revisions establish detection, not novelty; VAmPI is inspected teaching data and cannot qualify production Python support.
|
||||
- _Boundary: Evaluation harness, full-run integration_
|
||||
- _Requirements: 6.2, 6.3, 8.1, 8.2_
|
||||
- [ ] 13.2 Replay the declared PHP pair through the shared core
|
||||
- PMPro 2.9.7 against 2.9.8 with its upstream prepare fix, plus a comment-only benign variant. A different language, a different mechanism and a fix this project did not author.
|
||||
- The first attempt selected one case of three, because the manifest labels a case by what it is while the case id says what the comparison does; the harness now maps label to role explicitly. That run also exhausted a 900-second budget on the single case it ran.
|
||||
- Done when all three comparisons complete with their declared target evidence recorded, or the budget they need is stated as a finding rather than reported as a pass.
|
||||
- _Boundary: Evaluation harness, full-run integration_
|
||||
- _Depends: 13.1_
|
||||
- _Requirements: 6.2, 6.3, 8.1, 8.2_
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
- Task5.2 measured PMPro timing out after both snapshots were materialized but before change context/graph analysis. Group6 must account for compatible snapshot/discovery reuse as well as graph reuse; identical-tip hits cannot substitute for the later representative changed-code latency gate. No new detector, scope heuristic or incremental CPG mutation is implied.
|
||||
|
||||
Reference in New Issue
Block a user