5 Commits
Author SHA1 Message Date
Marius CiepluchandClaude Fable 5.1 9feb8f1e33 fix(hunt): a row's entry points are the ones in its file or a caller's file, not the whole repository's list
The DSPy arm over the second pool failed with LMInvalidRequestError on every change with more than two rows, at no
cost: `source.facts_for` attached every entry point of the product tree to every row, 8190 records and 1.5 MB per
row on apache/seatunnel (the first pool's arm lost 11 changes the same way on trigger.dev and shopper), and the
rendered prompt was refused. The screen table of the second pool weighed 814 MB for the same reason.

A row now carries the entry points declared in its own file or in a file that calls the function, which is what the
input has described all along ("in-repository entry points that reach the file"). `source.entry_point_count`
follows that definition.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-14 14:47:08 +02:00
Marius CiepluchandClaude Fable 5.1 81ae206978 fix(hunt): build output gets no candidate rows, a minified line is not scanned once per function, the screen checkpoints every change
The second access-control screen spun for 2h20m on one ordinary mage-ai change: a Next.js build commit whose 3 MB
one-line bundles hold 12,355 functions each. `source.ranges_from_ir` ran two regex scans of the whole line once per
function, and the screen kept its 422 finished changes in memory only, so stopping it lost them.

- `source.ranges_from_ir`: a line over MINIFIED_LINE_CHARS (2000) is not checked per function; JavaScript names on it
  stay anonymous, other languages keep the parser's name.
- `source.generated_reason`: why a changed file is build output (a directory the product walk leaves out, a minified
  name, over MAX_FILE_BYTES, a line over the cap). `hunt.producers._candidates` leaves such files out and records the
  reason on the context; the fix site of a vulnerability is always kept. The DSPy arm shares the enumeration.
- The screen writes rows.jsonl and log.jsonl per change (rows first, entry last) and resumes by skipping logged ids.

Tests: the minified line never reaches the per-function scan; generated_reason on the four shapes and two source files;
build output in an ordinary change yields no rows while a vulnerability path in a one-line file is kept.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-14 13:24:45 +02:00
Marius CiepluchandClaude Fable 5.1 4cf9c85d22 perf(hunt): the caller search prepares each product file once per change
call_sites rescanned every line of every product file for declarations on each call; with one name per
candidate row that was the whole cost of the source producer. code_index prepares a file once (the names each
line declares), call_sites takes it, and callers_of keeps a cache of prepared files the producer context owns.
Same calls, tested against the unprepared scan. One Laravel change of 33 rows: 249 seconds before this pair of
commits, 20 now.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-14 09:46:46 +02:00
Marius CiepluchandClaude Fable 5.1 86d70831c8 perf(hunt): the source producer walks and reads the product tree once per change, not once per row
facts_for walked the whole product tree and re-read every product file for each candidate row; on a Laravel
application one change of 33 rows took four minutes, 96 percent of it there. The producer context already
holds the product file list and the texts it read for entry points, so facts_for and callers_of now accept
them. Same facts, tested by a call whose tree walk raises and whose file on disk is emptied after reading.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-14 09:35:16 +02:00
Marius CiepluchandClaude Fable 5.1 062a7b780e refactor(p1): one source-facts module owns function ranges, callers, declaration tables and language aliases
src/openultrasast/source.py replaces the copies in semantic/functions.py,
plane/tasks/repo_facts.py, learn/features.py, learn/excerpt.py and
plane/tasks/verify.py. Two engines share one sentinel (<module>), one
DECLARATION/KEYWORDS/RESERVED set and one alias map: the parser engine
(ranges, enclosing, bounds, call_sites, callers_of, facts_for) for the
scorer, the overlay and the hunter's annotations; the line-pattern engine
(declared_functions, enclosing_at, function_span, function_bounds, the brace
matcher) that labels, features, roles, excerpts and plane facts were built
on. learn no longer imports plane.tasks. Frozen records that store the old
sentinel are mapped on read (module_function).

Gate: labels rebuilt before and after are identical, 7,921 rows, after
sentinel normalisation (benchmarks/measurements/2026-10-13-source-module-
label-diff/record.json). 490 targeted tests pass; 3,312 collect. Net: 657
lines deleted, 212 added. benchmarks/unseen keeps <global> for its own
frozen manifests.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-13 12:00:37 +02:00