marius/Open_Ultra_SAST
1
0
Fork 0
mirror of https://github.com/norandom/OpenUltraSAST.git synced 2026-10-16 03:49:26 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(cpg): portion the work instead of capping it -- a kill keeps what finished

Browse Source 
A hard per-invocation ceiling is all-or-nothing. A batch killed at 301 s lost every answer it had
already computed, and the retry paid a fresh JVM start per part to ask them all again: measured, 116
requests killed at 300 s, then eight restarts carrying the same everywhere-sinks in every part.

Three coordinated changes make portioning, not the cap, the thing that governs cost.

  The taint query STREAMS. The census first, then one line per request as it is answered, flushed as
  it goes, inside the unchanged fence. The other queries do not stream and need not; the driver
  reads both forms, and a legacy single document parses exactly as before.

  The driver HARVESTS a kill. The process group is terminated first -- a first cut read the pipe
  before killing and a pipe whose writer is alive never reaches end-of-file; the suite own
  hanging-engine control sat on that read for twenty-eight minutes -- then the pipe is drained and
  every complete line becomes an answer, marked partial. The retry asks only the remainder.

  Portion sizing ADAPTS. After each portion the observed seconds per sink visit, less the fixed
  start, resizes the rest toward a 120 s target; a portion that did not answer halves the budget
  rather than being fitted, because its true cost is unknown and above the ceiling. A portion cut
  down to fit puts its remainder back at the front of the queue.

A census LINE carries nothing but the census. A first cut misread a legacy document that carried the
census beside its answers as a census line and dropped every answer it sat beside; a control caught
it. Task 7.8 in contributor-scan.

Verified: 1,423 host tests pass, nineteen skipped, including new controls for the assembler in both
forms and under a kill, for a retry that asks only the remainder, and for the sizer. Not yet
verified: the joern-gated streaming check and the plugin measurement, which wait for the transfer
replay to free the machine and are stated as open in the task.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
This commit is contained in:
Marius Ciepluch
2026-09-22 21:03:04 +02:00
co-authored by Claude Fable 5.1
parent 1c9d7f501f
commit a8b226a5c8
7 changed files with 316 additions and 24 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.kiro/specs/contributor-scan/tasks.md
+10
View File
@@ -1275,3 +1275,13 @@ the ontology describes libraries, frameworks and language constructs, never repo
- _Boundary: contributor-scan scan budget, cpg backend_
- _Depends: 7.3_
- _Requirements: 4.1, 6.3, 8.1_
- [ ] 7.8 Portion the work instead of capping it: a kill keeps what finished
- **Asked for by the maintainer on 2026-09-22, after 7.7.** A hard per-invocation ceiling is all-or-nothing: a batch killed at 301 s loses every answer it had already computed, and the retry then pays a fresh JVM start per part to ask them all again. Measured on the plugin: 116 requests killed at 300 s, eight restarts, and the same everywhere-sinks in every part.
- Three coordinated changes replace the cap as the thing that governs cost. The taint query STREAMS: the census first, then one line per request as it is answered, flushed as it goes, inside the unchanged fence. The driver harvests a killed process's stdout and assembles every complete line, marking the result partial; the retry then asks only the remainder. And portion sizing ADAPTS: after each portion the observed seconds per sink visit, less the fixed start, resizes the rest toward a target duration, and a portion that did not answer halves the budget rather than being fitted. The ceiling stays as a safety net that the sizing is meant to make rare.
- Legacy single-document payloads must keep parsing unchanged; the other queries do not stream and need not. A census line is one whose only key is the census — a first cut misread a legacy document that carried the census beside its answers as a census line and dropped every answer it sat beside, which a control caught.
- Done when a killed taint batch hands back the answers it finished and the retry asks only the rest; portion sizes follow observed cost; the streamed and legacy forms both parse; and the plugin population completes more questions per unit of budget than under 7.7 with no finding lost.
- _Status:_ implemented with host controls passing for the assembler, the remainder retry and the sizer; the joern-gated streaming check and the plugin measurement wait for the transfer replay to free the machine.
- _Boundary: cpg backend, contributor-scan scan budget, taint query_
- _Depends: 7.7_
- _Requirements: 4.1, 6.3, 8.1_