fix(hunt): lines read through a tool count as shown; an unparseable answer is an empty answer

The answer check counted only the rendered function text as shown, so 7 of
8 baseline suspicions citing lines the model had read with read_file were
marked unknown (baseline 1/12 against the pilot hunter's 6/12). Tools now
record the lines they return. A DSPy parse failure scores as zero findings
with a flag instead of a broken run. The per-run response cache now takes
effect through dspy.configure_cache.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
Marius Ciepluch
2026-10-13 13:10:40 +02:00
co-authored by Claude Opus 5.5
parent c5bf52f47d
commit 602ec23e03
8 changed files with 174 additions and 62 deletions
+11 -7
View File
@@ -21,13 +21,17 @@ def label_rows(rows: Sequence[Mapping[str, Any]], change: Mapping[str, Any]) ->
data = deepcopy(dict(row))
span = change.get("function_lines")
function = row.get("function", row.get("feature.source.function"))
site = kind == "vulnerability" and row.get("path") == change["path"] and (
function == change["function"] or (span[0] <= row.get("line", 0) <= span[1])
site = (
kind == "vulnerability"
and row.get("path") == change["path"]
and (function == change["function"] or (span[0] <= row.get("line", 0) <= span[1]))
)
data.update(
label=1 if site else 0 if kind == "ordinary" else None,
family=change["family"] if site else None,
fix_span=list(span) if site else None,
label_ref=change["case_id"] if site or kind == "ordinary" else None,
verification=None,
)
data.update(label=1 if site else 0 if kind == "ordinary" else None,
family=change["family"] if site else None,
fix_span=list(span) if site else None,
label_ref=change["case_id"] if site or kind == "ordinary" else None,
verification=None)
result.append(data)
return result