3 Commits
Author SHA1 Message Date
Marius CiepluchandClaude Fable 5.1 062a7b780e refactor(p1): one source-facts module owns function ranges, callers, declaration tables and language aliases
src/openultrasast/source.py replaces the copies in semantic/functions.py,
plane/tasks/repo_facts.py, learn/features.py, learn/excerpt.py and
plane/tasks/verify.py. Two engines share one sentinel (<module>), one
DECLARATION/KEYWORDS/RESERVED set and one alias map: the parser engine
(ranges, enclosing, bounds, call_sites, callers_of, facts_for) for the
scorer, the overlay and the hunter's annotations; the line-pattern engine
(declared_functions, enclosing_at, function_span, function_bounds, the brace
matcher) that labels, features, roles, excerpts and plane facts were built
on. learn no longer imports plane.tasks. Frozen records that store the old
sentinel are mapped on read (module_function).

Gate: labels rebuilt before and after are identical, 7,921 rows, after
sentinel normalisation (benchmarks/measurements/2026-10-13-source-module-
label-diff/record.json). 490 targeted tests pass; 3,312 collect. Net: 657
lines deleted, 212 added. benchmarks/unseen keeps <global> for its own
frozen manifests.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-13 12:00:37 +02:00
Marius CiepluchandClaude Opus 5.5 56b1717246 feat(plane): scoring parity with the reference -- sites, ranges, triage cost
agree matched declared sites through benchmarks/independent/evaluate.py,
which the runner image does not carry, so site matching would have been
"not assessed". The generator now resolves the fix's old-side ranges
(evaluate.hunks, line for line) into case.json, with the declared sites in
the set and the reference's cost, and agree.matches_declared reproduces
matches_v2; a parity test runs both over the recorded outputs of the 15
validation cases (200 probes, identical verdicts) and agree on those outputs
returns the reference's 16 of 20 and $0.022.

agree reports cost_per_candidate (plane spend) and
cost_per_candidate_with_recorded_triage; Requirement 6.2 is judged on the
latter (design.md). --runner-image re-pins the templates from ops/ax/up.sh's
pin file; the committed manifests are regenerated at the current digest.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-29 19:49:28 +02:00
Marius CiepluchandClaude Fable 5.1 0cbdc5d2ac feat(plane): verify and agree tasks with metered hunts (task 6)
Requirements 5.3, 2.1, 2.2, 2.3, 7.2 of ai-service-plane.

- plane/tasks/verify.py: the batched hunt of verify_batched.py moved into
  the package (system prompt, one hunt per file, up to 6 candidates, 6
  steps, candidate-anchored sites). New against the reference: each
  candidate's known callers from facts.json in the prompt (capped at 8),
  tool turns and usage per hunt in units.jsonl, per-unit resume (rows
  with "error" are re-asked), pass label from env, MeteredClient wrapper:
  BudgetExhausted -> unfinished (exit 3), AccountError -> failed with the
  provider message (exit 2); summary.json is written last, names the bound
  Model and the summed usage, and is done only when every unit finished.
- plane/tasks/agree.py: verify_batched.agreement over two pass outputs,
  agreed.json / disputed.json, per-candidate cost and turns, declared-site
  match via benchmarks/independent/evaluate.matches_v2 loaded by file;
  metrics block in summary.json; no model calls.
- tests/test_plane_verify.py: a scripted usage-recording hunter client;
  callers line, resume, 402, budget stop and resume, model and summed
  usage, unpriced usd None, both main() env contracts, agreement metrics.
- model/audit.py + module-audit manifest: SPEC_OWNED rows for plane.tasks,
  plane.tasks.verify, plane.tasks.agree; manifest regenerated.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-29 18:05:08 +02:00