marius/Open_Ultra_SAST
1
0
Fork 0
mirror of https://github.com/norandom/OpenUltraSAST.git synced 2026-10-16 03:49:26 +00:00
Code Issues Packages Projects Releases Wiki Activity

bench(learn): harvest the advisory-fix pairs

Browse Source 
The 97 advisory-fix pairs (194 sides) harvested for the decision engine: verify a/b + agree on the 34
untrusted_destination sides and model roles on every side as plane Runs on ax ($0.38 metered, task
budgets $1.49, ceiling $5; DeepSeek balance 17.88 -> 17.73), engine features on the host (frozen
7b17eac, one container, --memory 3g: 54 ran, 58 asked no question, 82 no language model), feature
records for all 194 sides, static memory rebuilt with the merged harvest units: 724 -> 904 examples.
No unit was skipped for the 20 KB inline limit.

Instrument fix: SideReader read only the first pair-kind source, so the first rebuild left all 194
advisory sides `unread` (0 examples). It now reads every pair-kind source; test added.

Leak audit: the static profile flags only engine signals and facts.function_lines (withheld by v1);
the plane profile newly flags verify.flag_b and ms.operation (plane-only, not in any v1 prompt).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
This commit is contained in:
Marius Ciepluch
2026-10-02 09:40:00 +02:00
co-authored by Claude Opus 5.5
parent 7b17eacd44
commit d9608b9a63
4 changed files with 817 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/test_learn_examples.py
+31
View File
@@ -171,3 +171,34 @@ def test_pair_sides_join_the_record_stored_under_their_harvest_unit_pin() -> Non
assert build_examples([lab], records_by_key([stored]), lines_of, sources=SOURCES).summary()["examples"] == 0
build = build_examples([lab], records_by_key([stored]), lines_of, sources=SOURCES, pins=pins)
assert build.summary()["examples"] == 1 and build.rows[0]["pin"] == unit_pin
def test_side_reader_reads_every_pair_kind_source(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""Labels from the corpus catalog and from a second pair-kind source (advisory-fixes) are both source ``pairs``;
the reader must find either side, not only the first catalog's (the 2026-10-02 advisory rebuild read 0 of 194)."""
from types import SimpleNamespace
import openultrasast.pairs as pairs
from openultrasast.learn.examples import SideReader
files = {}
for name in ("corpus-case", "advisory-case"):
files[name] = tmp_path / f"{name}.py"
files[name].write_text("def handler(request):\n return 1\n")
catalogs = {"corpus.toml": "corpus-case", "advisory.toml": "advisory-case"}
def fake(path: Path) -> tuple[Any, ...]:
name = catalogs[Path(path).name]
return (SimpleNamespace(name=name, vuln_file=files[name], fixed_file=files[name], license="MIT"),)
monkeypatch.setattr(pairs, "load_pair_catalog", fake)
sources = Sources(
(Source("pairs", "pairs", ("corpus.toml",), (), "2026-09-30", "development"),
Source("advisory-fixes", "pairs", ("advisory.toml",), (), "2026-10-01", "development")),
frozenset(), {}, Path("sources.toml"),
) # fmt: skip
reader = SideReader(tmp_path, tmp_path, sources, [])
for name in catalogs.values():
lab = label(source="pairs", source_ref=name, pin="")
assert reader(lab, "vulnerable") == ["def handler(request):", " return 1"]
assert reader.license(lab) == "MIT"