spec(contributor-scan): record 7.8 verification and one bypass

The joern-gated check passes on the real engine: the taint query streams one line per request, and
the comment-sink and literal-destination controls hold under the streamed form. Two defects the work
exposed are recorded as fixed -- the unportioned evidence pass, and a harvest that read a timed-out
pipe before killing its writer -- and one bypass is recorded as a limit rather than fixed: the reuse
layer re-query of missing ids calls the backend batch directly around the sink-weighted portioning.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
This commit is contained in:
Marius Ciepluch
2026-09-22 21:45:09 +02:00
co-authored by Claude Fable 5.1
parent a4042fdb27
commit cfd1493d69
+3 -1
View File
@@ -1281,7 +1281,9 @@ the ontology describes libraries, frameworks and language constructs, never repo
- Three coordinated changes replace the cap as the thing that governs cost. The taint query STREAMS: the census first, then one line per request as it is answered, flushed as it goes, inside the unchanged fence. The driver harvests a killed process's stdout and assembles every complete line, marking the result partial; the retry then asks only the remainder. And portion sizing ADAPTS: after each portion the observed seconds per sink visit, less the fixed start, resizes the rest toward a target duration, and a portion that did not answer halves the budget rather than being fitted. The ceiling stays as a safety net that the sizing is meant to make rare.
- Legacy single-document payloads must keep parsing unchanged; the other queries do not stream and need not. A census line is one whose only key is the census — a first cut misread a legacy document that carried the census beside its answers as a census line and dropped every answer it sat beside, which a control caught.
- Done when a killed taint batch hands back the answers it finished and the retry asks only the rest; portion sizes follow observed cost; the streamed and legacy forms both parse; and the plugin population completes more questions per unit of budget than under 7.7 with no finding lost.
- _Status:_ implemented with host controls passing for the assembler, the remainder retry and the sizer; the joern-gated streaming check and the plugin measurement wait for the transfer replay to free the machine.
- _Status 2026-09-22:_ implemented; host controls pass for the assembler in both forms and under a kill, for the remainder-only retry, and for the sizer; the joern-gated check passes on the real engine — the taint query streams one line per request, and the comment-sink and literal-destination controls hold under the streamed form. The plugin measurement is running on a frozen export of the committed source.
- _Two things the measurement exposed on the way, both fixed:_ the evidence pass was one unportioned batch protected only by the ceiling — 22,125 requests on a plugin push transaction — and now runs in portions of 4,000; and a first cut of the harvest read a timed-out process's pipe BEFORE killing it, which blocks forever because a pipe whose writer is alive never reaches end-of-file. The suite's own hanging-engine control caught that after twenty minutes of what I mistook for CPU starvation.
- _A stated limit:_ the reuse layer's re-query of `missing` ids calls the backend batch directly and so bypasses sink-weighted portioning. Those re-queries are the small remainder of a cached scan, and streaming still keeps what a killed one finished, so it is recorded here rather than fixed.
- _Boundary: cpg backend, contributor-scan scan budget, taint query_
- _Depends: 7.7_
- _Requirements: 4.1, 6.3, 8.1_