marius/Open_Ultra_SAST
1
0
Fork 0
mirror of https://github.com/norandom/OpenUltraSAST.git synced 2026-10-16 03:49:26 +00:00
Code Issues Packages Projects Releases Wiki Activity

search-with-proof: feed the control signal back to the agent (served-root hint)

Browse Source 
When a path demo does not serve ${served_root}, the keystone control fails
(SideRecord.control_observed False) but the search only reported a generic
inconclusive, so the agent kept making the same wiring mistake. The pilot now
surfaces the affected side's control_observed in the verify result, and the
coordinator appends one advisory, deduplicated "served-root" hint (visible in the
board snapshot the agent reads) telling it to set the served-directory config to the
${served_root} placeholder. Advisory only: verdicts, _end, the oracle, the differential
and the demo validator are unchanged; the hint fires only when control_observed is False.

Tests: coordinator + pilot_run suites 78 passed; ruff clean; mypy unchanged (pre-existing
benchmarks errors only). Host-side change, no image rebuild.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
Marius Ciepluch
2026-10-07 12:03:45 +02:00
co-authored by Claude Opus 4.8
parent 84509fbdc0
commit 20c5742d84
4 changed files with 84 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/test_search_pilot_run.py
+24
View File
@@ -350,6 +350,30 @@ def test_fixed_effect_reporting_is_independent_of_differential(tmp_path, monkeyp
assert pilot.record["fixed_effect_observed"] is expected
@pytest.mark.parametrize("control_observed", [False, True, None])
@pytest.mark.parametrize("with_sides", [False, True])
def test_verify_dispatch_returns_affected_control_signal(tmp_path, monkeypatch, control_observed, with_sides):
from dataclasses import asdict
from openultrasast.search.budget import SearchBudget
from openultrasast.search.coordinator import SearchTask
from openultrasast.search.verify import SideRecord, VerificationRecord
pilot = make_pilot(tmp_path)
affected = SideRecord("observed", (), 0, "completed", control_observed=control_observed)
comparator = SideRecord("observed", (), 0, "completed", control_observed=control_observed is not True)
result = VerificationRecord("inconclusive", (affected, comparator) if with_sides else (), 0, "no differential")
monkeypatch.setattr(pilot, "verification", lambda limits: result)
task = SearchTask("task-1", "search-1", "verify", "", {}, asdict(SearchBudget()))
assert pilot.dispatch(task) == {
"status": "ok",
"outcome": "inconclusive",
"evidence_refs": ["verify:owned"],
"cost_usd": 0,
"control_observed": control_observed if with_sides else None,
}
def test_vm_preflight_refuses_before_client_or_store(tmp_path, monkeypatch):
from benchmarks.search import pilot_run