marius/Open_Ultra_SAST
1
0
Fork 0
mirror of https://github.com/norandom/OpenUltraSAST.git synced 2026-10-18 12:39:28 +00:00
Code Issues Packages Projects Releases Wiki Activity

measure(g2): banks fixed-search MVP -- "no wrong targets, only wrong searching" demonstrated

Browse Source 
Same CPG, only the search changes. Baseline (shipped path spec) resolves 0
prompt->write_text flows on banks; injecting the two annotations an LLM steer
would supply (parameter_sources=True + write_text/read_text path sinks) makes the
engine resolve prompt -> path-join -> prompt_file.write_text(prompt.raw). First
measured instance of "#2 guided by #1" closing on an unseen target; fix needed
exactly the param-source + file-write-sink annotations.

Files the instrument bug the first MVP hit: single-query cpg.run("taint",
request_params(...)) is dead -- request_params emits 5 keys (guards/fixedOrigin/
originAnchors/quotedSanitizers/fieldParameterSources) absent from taint.sc's exec
signature; both single-query sub-routes reject them (silent None). Batch path
(run_batch->requestsFile JSON) unaffected; push/scan route uses batch.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
Marius Ciepluch
2026-10-12 14:02:41 +02:00
co-authored by Claude Opus 4.8
parent 7ab5c2ab2d
commit 8f7fc4ae83
6 changed files with 272 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.kiro/steering/roadmap.md
+8
View File
@@ -132,6 +132,14 @@ against a realistic pre-push window (≤120 s target, 300 s hard):
5 pilot regions in LLM-guided-source mode; measure source naming, traced finding, catch lift vs regex 1/5, and
false-alarm on ordinary changes; stage 5–10 → ~10–20 → full on lift. #3 change-anchoring deprioritised until
tiers yield findings.
- **BANKS FIXED-SEARCH MVP DONE (2026-10-12) — "#2 guided by #1" closes on an unseen target.** See brief
"BANKS FIXED-SEARCH MVP" + `benchmarks/measurements/2026-10-12-banks-fixed-search-mvp/`. On banks (the
library that gave the engine zero), injecting the two annotations an LLM steer would supply (param-as-source
+ `write_text`/`read_text` sink) makes the engine resolve `prompt → path-join → write_text` where the
shipped search resolved nothing. Proves the governing principle "no wrong targets, only wrong searching" and
names the exact inputs the DSPy-optimised steer must learn. Filed instrument bug:
`.kiro/specs/engine-change-scoped-analysis/bugs/taint-sc-single-query-signature-skew.md` (single-query taint
path dead on an exec-signature skew; batch path unaffected).
- LLM: L1 model latency and calls per push; L2 the recall LIFT of the LLM tier over regex+dataflow on the pool.
- Deliverable: a barrier table (budget × tier combo → fits? measured recall/FP) → the minimal releasable config.