fix(hunt): lines read through a tool count as shown; an unparseable answer is an empty answer

The answer check counted only the rendered function text as shown, so 7 of
8 baseline suspicions citing lines the model had read with read_file were
marked unknown (baseline 1/12 against the pilot hunter's 6/12). Tools now
record the lines they return. A DSPy parse failure scores as zero findings
with a flag instead of a broken run. The per-run response cache now takes
effect through dspy.configure_cache.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
Marius Ciepluch
2026-10-13 13:10:40 +02:00
co-authored by Claude Opus 5.5
parent c5bf52f47d
commit 602ec23e03
8 changed files with 174 additions and 62 deletions
+1 -2
View File
@@ -87,8 +87,7 @@ class FeatureTable:
if unknown:
raise ValueError("mask names an unknown producer or feature column")
columns = {c for c in FEATURE_COLUMNS if c in names or c.split(".")[1] in names}
return FeatureTable(({k: v for k, v in row.items() if k not in columns} for row in self.rows),
masked=(*self.masked, *names))
return FeatureTable(({k: v for k, v in row.items() if k not in columns} for row in self.rows), masked=(*self.masked, *names))
def feature_columns(self) -> list[str]:
return sorted({column for row in self.rows for column in row if column.startswith("feature.")})