Files
Open_Ultra_SAST/benchmarks/measurements/2026-10-12-banks-fixed-search-mvp/results.out
T
Marius CiepluchandClaude Opus 4.8 8f7fc4ae83 measure(g2): banks fixed-search MVP -- "no wrong targets, only wrong searching" demonstrated
Same CPG, only the search changes. Baseline (shipped path spec) resolves 0
prompt->write_text flows on banks; injecting the two annotations an LLM steer
would supply (parameter_sources=True + write_text/read_text path sinks) makes the
engine resolve prompt -> path-join -> prompt_file.write_text(prompt.raw). First
measured instance of "#2 guided by #1" closing on an unseen target; fix needed
exactly the param-source + file-write-sink annotations.

Files the instrument bug the first MVP hit: single-query cpg.run("taint",
request_params(...)) is dead -- request_params emits 5 keys (guards/fixedOrigin/
originAnchors/quotedSanitizers/fieldParameterSources) absent from taint.sc's exec
signature; both single-query sub-routes reject them (silent None). Batch path
(run_batch->requestsFile JSON) unaffected; push/scan route uses batch.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-10-12 14:02:41 +02:00

33 lines
1.8 KiB
Plaintext

CPG built; run_batch available.
baseline sinks: ('FileResponse', 'open', 'os.remove', 'os.unlink', 'send_file', 'send_from_directory', 'shutil.copy', 'shutil.move', 'shutil.rmtree')
annotated sinks: ('FileResponse', 'open', 'os.remove', 'os.unlink', 'read_text', 'send_file', 'send_from_directory', 'shutil.copy', 'shutil.move', 'shutil.rmtree', 'write_text')
=== BASELINE (shipped search) ===
[baseline] from_prompt_path psrc=False: 1 flow row(s)
sink=None source=None line=None evidence=None
[baseline] set psrc=False: 1 flow row(s)
sink=None source=None line=None evidence=None
[baseline] _load psrc=False: 1 flow row(s)
sink=None source=None line=None evidence=None
[baseline] _scan psrc=False: 2 flow row(s)
sink=None source=None line=None evidence=None
sink=None source=None line=None evidence=None
=== FIXED SEARCH (annotation: param-source + write_text/read_text sink) ===
[fixed] from_prompt_path psrc=True: 4 flow row(s)
sink=prompt_file.write_text(prompt.raw) source=path line=None evidence=None
sink=prompt_file.write_text(prompt.raw) source=prompt line=None evidence=None
sink=None source=None line=None evidence=None
[fixed] set psrc=True: 1 flow row(s)
sink=None source=None line=None evidence=None
[fixed] _load psrc=True: 2 flow row(s)
sink=None source=None line=None evidence=None
sink=None source=None line=None evidence=None
[fixed] _scan psrc=True: 4 flow row(s)
sink=self._index_path.write_text(self._index.model_dump_json()) source=self line=None evidence=None
sink=None source=None line=None evidence=None
sink=None source=None line=None evidence=None
=== RESULT: baseline flow rows=5 fixed-search flow rows=11 ===
FIXED SEARCH TRACES BANKS (search was wrong, not the target)