feat(hunt): feature table, producers, labels, metric, artifact, the DSPy hunt program, runner and CLI

hunter-learning-loop groups 1-3 (first increment). The feature table is the
contract: one row per touched function with diff, source, semantic and
taxonomy columns, provenance per producer, labels by the benchmark's rule.
Producers read the repository at head only (a planted advisory id and
commit text never reach a feature). The hunt program is a DSPy ReAct loop
over a signature whose inputs are the unmasked columns in a fixed order;
answers outside the closed family list or citing unseen lines score as
unknown. The runner writes records the benchmark scorer reads, resumable
by change, program id and mask, and refuses the held-out pilot unless the
run is a report run. Research extra [hunt] = dspy, pandas; the core
install is unchanged. 43 tests on a scripted model, no network.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
This commit is contained in:
Marius Ciepluch
2026-10-13 12:33:58 +02:00
co-authored by Claude Fable 5.1
parent e55e44d981
commit d9d6335ab3
17 changed files with 1720 additions and 0 deletions
+33
View File
@@ -0,0 +1,33 @@
"""Attach benchmark fix-site labels without exposing them to feature producers."""
from __future__ import annotations
from collections.abc import Mapping, Sequence
from copy import deepcopy
from typing import Any
def label_rows(rows: Sequence[Mapping[str, Any]], change: Mapping[str, Any]) -> list[dict[str, Any]]:
"""Label verified sites, leave other positive rows unknown and label ordinary rows zero."""
kind = change.get("kind")
if kind not in ("vulnerability", "ordinary"):
raise ValueError("change kind must be vulnerability or ordinary")
if not change.get("case_id"):
raise ValueError("a label needs a traceable case_id")
if kind == "vulnerability" and not all(change.get(k) for k in ("path", "function", "function_lines", "family")):
raise ValueError("a vulnerability label needs its path, function, span and family")
result = []
for row in rows:
data = deepcopy(dict(row))
span = change.get("function_lines")
function = row.get("function", row.get("feature.source.function"))
site = kind == "vulnerability" and row.get("path") == change["path"] and (
function == change["function"] or (span[0] <= row.get("line", 0) <= span[1])
)
data.update(label=1 if site else 0 if kind == "ordinary" else None,
family=change["family"] if site else None,
fix_span=list(span) if site else None,
label_ref=change["case_id"] if site or kind == "ordinary" else None,
verification=None)
result.append(data)
return result