Reverts 6ebbc89 and e0c260c (keeping finding_dump's completed-regions
record). Measured on PMPro, comparing only questions both runs answered
(157): 37 -> 31 findings, but the six removed included the adjudicated TRUE
positive discount-codes.php:154 (REST-reachable), and only one of the six
sampled object-field false positives went away; five are still reported.
It also cut completed questions 295 -> 158. MW WP Form and WP Statistics
were unchanged either way.
The diagnosis was wrong: the object-field false positives do not come from
setter parameters. The witnesses name one field (e.g.
$update_order->BillingFrequency) and the query interpolates another
($this->membership_id), which points at field-insensitive propagation
through the object instead. To be confirmed on a flow path before any fix.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
Asked of every parameter up front, 6ebbc89's caller check ran a repository-
wide flow query per parameter of every method in a file and cut PMPro's
completed injection questions from 272 to 67 in the same budget -- so its
48 -> 26 findings could not be read as precision. The flow into the field
is asked once as before; only a parameter at the head of an unsanitized
flow is checked against its callers. finding_dump now records completed
regions so runs compare only where both answered.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
The field join (task 5.11) seeded every parameter of every method in the
file, so every setter made its field a source. Adjudicated on PMPro, six of
eight false SQL injections in a 1-in-5 sample were object fields holding
database or internal values ($this->membership_id, $morder->user_id).
A parameter now seeds the join only where some call site passes it request
data without a sanitizer on the way (memoised per method and index); a
request source read in the file still seeds as before. A real-Joern control
covers both directions -- a setter only given a constant stays quiet, a
constructor handed $_GET (the CVE-2023-6559 shape) is still found -- and
fails on the old query with exactly the PMPro false positive.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
48 findings, every 5th reviewed against source: 2 true (both privileged),
8 false; 6 of the 8 share one cause, object fields treated as attacker
input when they are loaded from the database. benchmarks/push/finding_dump.py
writes every finding of chosen families for review.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
The loop the maintainer called out: four scale defects in the PHP transfer
replay, found one per two-hour Joern run, all Python bookkeeping between
engine calls. benchmarks/push/python_overhead.py ends that: the real
replay() on the real PMPro pins, with a JoernBackend whose graphs are stubs
and whose queries answer instantly with rows recorded from the real engine.
Engine time is zero, so everything measured is ours, in about a minute,
under the sampling profiler. Its first pass named two more walls:
- Region discovery re-parsed the ruleset for every region: 89,127 TOML
parses, 275 s of a 339 s discovery, twice per push. The fact, obligation
and taxonomy loaders now keep the parsed (immutable) result keyed by each
file's name, size and mtime -- an edit in place or an added file reloads.
Discovery 339 s -> 7 s; output digest identical on PMPro.
- affected_context decoded every span's and every line anchor's path for
every context row -- a changed file has an anchor per unchanged line --
78% of a stubbed push. Decoded once per call, anchors indexed by line and
bisected, original order kept. A randomised control compares it against
the row-by-row matching over 200 contexts (>100 attachments).
Result: the whole stubbed PMPro push, 27,542 answered requests and 2,688
outcomes, runs in 58.5 s of Python. What is left is the engine.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
b4b9308 was committed with two failing tests: the module audit read
`from . import profiling` as an import of the package, called profiling
orphaned, and the committed audit manifest drifted. My check read tail's
exit status, not pytest's. Now a from-import; manifest regenerated (109
load-bearing, 0 orphaned); 1,444 pass with pytest's own exit checked.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
The cache fsynced every payload and manifest -- two per entry, ~18 ms, and a
push publishes one entry per question (~22,000 on a WordPress plugin). A
read already rejects any entry whose manifest receipt, payload size or
payload SHA-256 does not check, so a torn write is a miss, never a wrong
answer; the fsyncs bought nothing the verification does not.
The broader problem: three quadratic walls in this replay were found one
per two-hour run, because all were Python bookkeeping between engine calls
and the only record was the engine log. `OUSAST_SAMPLE_PROFILE=<path>`
starts an in-process sampler that rewrites the hottest stacks every 30 s,
so a run killed at its deadline still names where its time went. The host
cannot ptrace the container, hence in-process. The replay harness turns it
on per case.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
The third wall under the 13.3 replay, found by the streamed case log: the
evidence and context passes finished at 14:53, then nothing -- no engine
call, no log line -- for 82 minutes, until the case was killed 60 s past
its 7,200 s deadline. `affected_context` tested membership with
`relation not in relationships` on a list: 7,998,000 comparisons for 4,000
relationships (profiled, 83% of the time), and a push over a WordPress
plugin attaches tens of thousands. The deadline is checked once per
question, so a single late question could run for minutes, which is also
why the process overran its cancellation allowance.
Membership now goes through a set beside the ordered list, and each
distinct context path is resolved once instead of once per row. Measured:
4,000 relationships 7.4 s -> 0.44 s; 400,000 in 16 s. The control counts
comparisons rather than timing them, and fails on the old code with
7,998,000.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
The 13.3 replay on 7cf256b crashed on case 1: `pre-push` ran past its
7,200 s deadline plus the harness's 60 s grace, `subprocess.run` raised
TimeoutExpired, the harness died without running cases 2 and 3, and the
case's captured output died with it. Each case now streams stdout and
stderr to files with INFO logging on; an overrun kills the case's process
group, is recorded as `deadline_overrun` with its size -- the product
failing its cancellation contract -- and the next case still runs.
`--only` reruns selected cases. CLI log lines now carry a timestamp, so a
log can say where the time went.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
The real cause of the 13.3 replay wall. Every cache publication called
`_make_room`, which listed and stat'ed every entry to total the cache size,
and a push publishes one answer per question -- 22,125 on a WordPress
plugin. Measured on the live replay cache: one scan 0.68 s at 12,238
entries, ~70 minutes spent scanning by then, ~2.3 hours to finish the
evidence pass. The engine was idle while the Python process sat at 83% CPU;
the 13/31/51-minute portions were the quadratic growing.
The scan is now remembered, keyed by the cache root's modification stamp:
our own publications and evictions update the tally in place, and any
change another process makes moves the stamp and forces a rescan, so the
size limit stays exact under concurrent pushes. Lock stripes are created up
front so their first use never invalidates the tally. Eviction order and
lease handling are unchanged.
Controls: 61 publications (one a same-key replacement) take exactly one
scan and the tally equals a fresh scan; a second writer on the same root
forces a rescan and the limit holds.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
`test_hanging_process_group_killed_inside_shared_allowance` failed about one
run in thirty under load with FileNotFoundError: it checked that
/proc/<pid>/stat existed, the killed process was reaped, and the read then
failed. A reaped process is the outcome the test wants, so it now reads and
treats a missing entry as gone. The engine-stage test had the same race.
150 of 150 runs pass under engine load.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
The PHP transfer replay (pre-push 13.3) spent its whole 7,200 s case in the
head side's evidence pass. A probe on one PMPro graph named the cause: not
compute -- context adds ~20% engine time -- but OUTPUT. Each evidence request
listed every method its scope reached, up to 1,845 rows; 200 requests into
the bundled Braintree SDK returned 52,831 rows, 9.4 MB, against 211 rows
without context, over 22,125 requests in a 3 GB container.
`affected_context` only ever attaches a location in a changed file, and
every other row was checked (real file under the root, usable extent) and
discarded. The scan now sends the changed files; the engine itemises those
and folds the rest into one `context_elsewhere` row carrying exactly what
that check reads. With no safe file list the engine itemises everything,
as before.
Equivalence: unit controls over valid, missing-file, no-extent and
outside-root cases, with and without a changed-file row; a real-Joern
control; and on PMPro itself, 2,440 relationships and zero gaps decided
identically from both forms, 52,831 rows down to 3,241 with the most
referenced file as the change. Probe: benchmarks/push/context_cost_probe.py.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
Same 500-region PHP run on frozen source: completion 281/406 against 153
(hard cap) and 131 (7.8), no portion killed, per-family findings never
lower, declared target still entailed. Records the 7.9 task, its limits,
and corrects the 7.8 reuse note, which was stated backwards.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
The measured-cost run left 99 taint requests unasked to keep time for
arbitration and reported them as `query_failed` -- an engine that could not
answer -- because they were simply missing from the answer. They are now
`query_budget_reserved`, both as a degradation and as each question's outcome
reason, and are no longer double counted when nothing answered.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
The 7.8 trace showed the completion loss was a sizer death spiral, not
expensive families: after two kills the budget fell to 3 sink visits and
never recovered, because portions finishing in 25-30 s sat under an assumed
70 s fixed start and read as "no information", so the budget could only
shrink.
- taint.sc streams each answer's own cost (`ms`); the backend carries it as a
reserved `__timing__` list through shards, partitions and reuse.
- The sizer budgets predicted seconds from per-family seconds-per-visit rates
learned from those costs, and measures the fixed start as the wall time the
answers do not explain (median), instead of assuming it.
- A kill names its culprit: the first unanswered id in request order. The ids
behind it never started and are re-queued at once; the culprit is asked
alone after the rest. One that still fails alone after more than a
portion's target is reported as `query_too_expensive`, apart from
`query_failed`.
- Portions grow id by id from the queue head, and a single-portion scan now
goes through the queue too, so a kill there re-asks its remainder.
- Fix: the reuse wrapper wrapped only `run_batch`, so since 7.8 every
portioned query bypassed the graph-reuse cache and its source-root
normalization. `run_batch_once` is wrapped too.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
Final end-to-end on the 500-region PHP subject: findings identical at 125 either way, completion
slightly lower under portioning (131 of 406 vs the hard-cap-plus-reserve 153), wall time higher
(2,982 vs 2,338 s). Told straight: portioning does not beat the cap on throughput here; it matches
it on what it finds, and what it buys is kill-safety -- a killed batch keeps its streamed answers
instead of losing them. Reaching the same 125 findings is the evidence that streaming loses nothing
the cap kept.
Completion did not rise because the requeue re-asks a killed portion remainder, and on this subject
those are the expensive output-encoding questions that keep hitting the ceiling in smaller pieces,
spending budget without completing. Telling a transient kill from a too-expensive family is the next
lever, not more portioning. Neither approach is release-ready anyway: 32-38% completion against a
95% gate on development data that qualifies nothing.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
Four interacting defects surfaced in sequence when the portioner was measured end to end, each
hidden by the one before: run_batch timing polluted by its own retry, a single-attempt path missing
on the partitioned graph every plugin uses, and killed remainders dropped by a fixed chunk list. All
fixed and unit-tested; the end-to-end completion number is re-measuring on a frozen tree.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
The portioner iterated a fixed pre-split list, so a killed portion left its unanswered ids behind:
on the plugin the list ran out at 1,263 s of a 3,600 s budget with those remainders lost, and
completion sat far below what the budget could buy. It is now a work queue. A portion that comes back
short pushes its unanswered ids to the front, to be re-asked at the budget the kill just halved;
`fit` splits an over-budget head, so heads shrink toward single requests, and a single request a
whole ceiling could not answer is dropped rather than looped on forever. A control asserts both the
re-ask and that termination.
This is the last piece of portioning-over-cap: streaming keeps what a kill finished, single-attempt
timing keeps the sizer honest, coverage tells a too-big portion from a slow one, and the queue keeps
working while budget and work remain.
1,425 tests pass, nineteen skipped.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
run_batch_once was wired on the disposable and sharded paths but not the partitioned one -- and a
multi-language repository, which every WordPress plugin is, builds through the partitioned graph. So
the portioner fell back to run_batch there, whose internal split-retry folded into the time it
measured: a portion read 956 s and collapsed the sizer, exactly as before the single-attempt fix.
The fix had no effect on the one subject it was written for.
PartitionGraph now has run_batch_once, dispatching to each sub-graph the same way run_batch does --
the routing is factored into one _dispatch method that takes the per-graph method name. Sub-graphs
are built through the disposable path and already carry run_batch_once.
The lesson, again: a capability added at one construction site is not added at the others. There are
three that build a queryable graph -- disposable, sharded, partitioned -- and the one that mattered
for the measurement was the one missed.
1,424 tests pass, nineteen skipped.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
End-to-end, the adaptive portioner regressed completion -- 64 of 406 against 7.7 hard-cap 153 --
while finding the same defects. The cause: the portioner timed each portion with run_batch, which
splits on failure internally, so one portion folded its own eight-way retry into its number and read
953 s. The sizer fitted a per-visit rate from that and collapsed its budget to 14 visits, spending
the back half of the deadline on tiny portions.
Two portioning layers were fighting. The portioner is now the sole retry layer: it asks
run_batch_once, a single invocation whose kill returns the answers it streamed, so its wall time is
the cost of one attempt. And it observes COVERAGE, not emptiness -- a portion that came back short
of the ids it asked halves the budget, because it was too big for the ceiling; only a whole answer
gives a clean per-visit sample to fit from.
run_batch_once is new on CpgResult, wired on the disposable and sharded paths; a backend without it
falls back to run_batch unchanged. Host controls cover the single-attempt path and the
halve-on-short-answer rule.
1,424 tests pass, nineteen skipped. The end-to-end plugin re-measure on a frozen tree is the open
item; the mechanism (a kill keeps what finished) and the sizer logic are unit-verified.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
The helper took a destination from the caller and rm -rf it. Re-running it into the same path while
a container had that path mounted deleted taint.sc out from under a live scan, and every request
after printed no such cpg query -- the live-mount hazard the helper exists to prevent, in a new
form. It now mints base/frozen-<commit>-<pid>-<n> itself and prints it, so reuse is impossible by
construction and the caller captures the printed path.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
Measured on the plugin, first run of the adaptive sizer. A 600-visit portion answered in 184 s and
the budget was correctly cut to 262. The next portion, 199 visits, answered in 73 s -- three seconds
of work above the 70 s fixed start -- and 3 s over 199 visits read as 0.015 s each, so the budget
grew thirteen-fold to 3,404 visits. The ceiling killed that portion. The harvest kept the 13
answers it had streamed, which is the mechanism working; the sizing was not.
Two rules. A sample with less than ten seconds of work above the fixed start is not extrapolated
from, because it carries no information about cost per visit. And a budget may double per portion,
never more: the sizing may still be wrong, but by at most one factor of two, which a kill undoes.
1,423 tests pass, nineteen skipped; the sizer control asserts both rules.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
The warning once read 22126 of 22125 requests answered in parts, because the census rides the
answer under its own key and was counted as a request. An instrument that miscounts by one casts
doubt on every number beside it.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
Every long run mounts src into a container, and mounting the working tree is a live mount: edits
made during a multi-hour run are picked up mid-run, and a measurement whose code changed underneath
it is not a measurement of anything. A five-hour PHP transfer replay was lost that way today.
The export has to be checked too. git archive piped into tar once produced a tree missing the whole
ontology -- every tracked file after groovy alphabetically -- and reported success, because tar
status in a pipeline was never read. Every scan on that tree died at its first fact load and looked
like a slow run for fourteen minutes.
So: archive to a file, extract, count against git, and refuse an export without the ontology.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
The joern-gated check passes on the real engine: the taint query streams one line per request, and
the comment-sink and literal-destination controls hold under the streamed form. Two defects the work
exposed are recorded as fixed -- the unportioned evidence pass, and a harvest that read a timed-out
pipe before killing its writer -- and one bypass is recorded as a limit rather than fixed: the reuse
layer re-query of missing ids calls the backend batch directly around the sink-weighted portioning.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
The evidence pass bypasses the sink-weighted portioning by design: it is the pass that PRODUCES the
weights, and per request it is cheap, 17,700 answered in 44 s. It was still one all-or-nothing batch
protected only by the ceiling. On a plugin push transaction that batch is 22,125 requests, and under
the CPU and memory contention of concurrent test runs it exceeded the ceiling and cost the whole case.
It now runs in portions of 4,000 requests, roughly ten seconds of work behind one JVM start, with
answers merged and the census kept from the first portion that carried it. Streaming keeps what a
killed portion finished; portioning keeps that kill from being the entire pass.
Task 13.3 records the attempt that exposed this honestly: the case deadline was exhausted with no
artifact, and the failure was mine twice over -- the container mounts src live and the backend was
edited mid-run, and the evidence pass was unportioned. Neither is the algorithm. The rerun uses a
frozen export of the committed source with per-invocation logging.
1,423 tests pass, nineteen skipped; mypy clean.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
A hard per-invocation ceiling is all-or-nothing. A batch killed at 301 s lost every answer it had
already computed, and the retry paid a fresh JVM start per part to ask them all again: measured, 116
requests killed at 300 s, then eight restarts carrying the same everywhere-sinks in every part.
Three coordinated changes make portioning, not the cap, the thing that governs cost.
The taint query STREAMS. The census first, then one line per request as it is answered, flushed as
it goes, inside the unchanged fence. The other queries do not stream and need not; the driver
reads both forms, and a legacy single document parses exactly as before.
The driver HARVESTS a kill. The process group is terminated first -- a first cut read the pipe
before killing and a pipe whose writer is alive never reaches end-of-file; the suite own
hanging-engine control sat on that read for twenty-eight minutes -- then the pipe is drained and
every complete line becomes an answer, marked partial. The retry asks only the remainder.
Portion sizing ADAPTS. After each portion the observed seconds per sink visit, less the fixed
start, resizes the rest toward a 120 s target; a portion that did not answer halves the budget
rather than being fitted, because its true cost is unknown and above the ceiling. A portion cut
down to fit puts its remainder back at the front of the queue.
A census LINE carries nothing but the census. A first cut misread a legacy document that carried the
census beside its answers as a census line and dropped every answer it sat beside; a control caught
it. Task 7.8 in contributor-scan.
Verified: 1,423 host tests pass, nineteen skipped, including new controls for the assembler in both
forms and under a kill, for a retry that asks only the remainder, and for the sizer. Not yet
verified: the joern-gated streaming check and the plugin measurement, which wait for the transfer
replay to free the machine and are stated as open in the task.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
Detection is not transfer, and the status says so: the comparison run is what decides M3. But the
target that task 13.2 could not reach for months is now found, and the reason is worth keeping where
the next reader will see it -- nothing about detection changed, the population became affordable.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
classes/class.memberorder.php:936:getMemberOrderByCode, injection. A value concatenated into SQL with
no prepare, which is the declared M3 target for PHP, found for the first time.
Nothing about detection changed. The population became affordable: tier-zero pruning scoped to its
partition, batches budgeted by the sink work they imply, a per-query timeout no longer cancelling the
scan, and a reserve kept for arbitration. Before those this population completed zero questions at
every budget tried.
At 500 regions with a 3,600 s deadline: 271 of 406 questions completed, 213 findings, 2,950 s, and
eight findings in the target file including the target itself.
The earlier bounded attempt failed for a reason no budget increase could fix -- 25 regions of 4,461
is 0.6 percent of the population and the target was never asked about -- so this sizes the replay
before spending hours on it. One scan at this budget is 2,950 s, a push transaction is two scans plus
discovery and preparation, so the three declared comparisons are roughly five and a half hours.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
A census that reports what it found cannot say whether a defect it missed was examined and cleared or
never looked at, and that is the difference between a negative and a silence. It is also how a
bounded run budget gets sized without guessing: task 13.3 needs to know at what population the
declared PHP target enters scope, and asking the record is cheaper than binary-searching with
forty-minute scans.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
PMPro at 25 regions, which completed before this work and still does: 27 questions completed and 19
findings before and after, injection 1 and output encoding 18 in both. The batching changes what a
large population can afford and nothing about what a small one answers.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
On the 500-region population where every earlier run produced nothing: 153 of 406 questions completed
and 125 findings, inside the deadline. 380 requests carrying 5,807 sink visits split into 11 batches,
each answering in 80 to 197 s, none near the 300 s ceiling. Every family that can speak on this
subject now does, injection included, which is the one holding a true positive here.
Four defects, each hidden by the one before it, and three of them one rule.
Batches sized by request count, when cost is per matched sink.
Packing heaviest-first, which starved injection of all 114 of its questions.
Any per-query timeout cancelling the whole scan, abandoning batches with 33 minutes left.
No budget kept for arbitration, which removing that cancellation exposed: rows are not findings.
The reserve had to be checked per BATCH. Checked between query kinds it changed nothing at all,
because one kind is a single loop iteration that runs every one of its batches inside it.
What it does not claim: 253 of 406 questions still went unanswered and one batch still records a
timeout; the raw finding count is dominated by output encoding, the family already measured to be
imprecise on PHP; and the reserve share of a fifth is a first value, not a tuned one.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
The reserve was computed once and checked between kinds, and one kind is a single iteration that runs
every one of its batches inside it. So taint spent the whole budget and the reserve only stopped the
cheap kinds queued behind it: 0 of 406 questions completed, 2,435 s of a 2,700 s deadline in queries,
identical to having no reserve at all.
The granularity of a budget check has to match the granularity of the work it guards. It is now
checked before each batch, and the batches left unasked are logged rather than silently dropped.
1,420 tests pass, eighteen skipped.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
Querying fills rows. Arbitration turns them into findings and the outcomes are assembled from that,
and neither costs the engine anything -- so a scan that spends its whole budget on queries reports
nothing at all, however many of them answered.
Found by accident, which is the honest way to tell it. A per-query timeout used to cancel the whole
scan. Removing that cancellation was right on its own terms and made the measured result worse: the
query phase then ran the full 2,700 s deadline to exhaustion and completed 0 of 406 questions, where
the cascade had stopped it at 958 s and left time to arbitrate 78 findings. The cascade had been an
accidental budget guard. This is the deliberate one.
The query loop now stops issuing batches with a reserve in hand: a fifth of what was left when
querying started, a sixty-second floor, and never more than half. The cap is not decoration -- held
back without it the floor swallowed a whole small budget and the query loop never ran at all, which a
control caught. A scan with a fifth of a second to spend must still spend it.
What the loop skipped is recorded as query_budget_reserved with the kind and the request count, not
left to look like a query that failed.
1,420 tests pass, eighteen skipped.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
Two different timeouts reached the same handler and got the same consequence. If the SCAN deadline is
gone, cancellation is right: stop everything promptly, which is what the push two-second gate
measures. If a single query merely exceeded its own per-query ceiling, the scan may have most of its
budget left and every other batch is independent and cheaper -- and cancelling took them all.
Measured on Paid Memberships Pro: one batch of 116 requests hit a 300 s ceiling 734 s into a 2,700 s
scan, and the four batches behind it returned in 0.0 s unanswered with thirty-three minutes to
spare. The handler already distinguished the two cases in the reason it recorded, deadline_exhausted
against process_timeout, and then set the cancel deadline in both.
Fourth instance this week of one rule, now written in the design: a failure is owned by the narrowest
scope that caused it.
1,420 tests pass, eighteen skipped, including the cancellation gate.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
A taint batch costs what its matched sink NODES cost, because reachableByFlows is asked per sink.
Measured at a fixed 100 requests on one plugin graph: SQL injection 147 s with six declared sinks,
path 194 s with sixteen, output encoding 704 s with seven. The declared count does not predict cost
and the request count does not predict cost.
So batches are budgeted in sink visits rather than requests, using the count the evidence pass
already produced: a family whose sinks are rare rides in one batch, and a family whose sinks are
everywhere is split until it fits. A request over the budget on its own gets its own batch rather
than being dropped, because the scan job is to report that it is expensive, not to decide it is
unaskable.
Measured on Paid Memberships Pro at 500 regions, where every earlier run produced nothing at all:
380 arbitration requests carrying 5,807 sink visits split into 12 batches, every one answered,
heaviest 103 s and none near the 300 s ceiling. 144 findings where there were none.
The order is the ranker and must stay the ranker. A first attempt packed heaviest-first, which packs
better and analyses worse: the expensive output-encoding batches ran first, spent the deadline, and
left injection with 114 questions asked and none answered -- the one family holding a true positive
on that subject. A deadline always cuts a tail, and the tail it should cut is the one the ranker put
last.
1,420 tests pass, eighteen skipped.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
Open since September and re-measured nine times. The cost of a taint batch is the number of CALL
NODES a family sink names match in the repository. Not the request count, not the declared sink
count, not any request parameter. reachableByFlows is asked per matched sink, and PHP
soutput-encoding sinks are echo and print, which a WordPress plugin writes thousands of times.
Three controlled sweeps on one graph, built once.
Batch size exonerated: 400 requests answer in 170 s, and cost per request FALLS with size because
roughly 70 s is fixed JVM start and graph load.
Request parameters exonerated: at 200 requests the full production shape, call depth 3 with
parameter sources, costs 221 s against a 150 s baseline.
The family is the variable: at the same 100 requests, SQL injection 147 s with six declared sinks,
path 194 s with sixteen, output encoding 704 s with seven.
And the driver named its own expensive step once it was asked to: 2,435 evidence requests in 45.5 s,
then 315 arbitration requests unanswered at 1,200 s, then a split abandoned for want of time.
It survived since September because every reproduction outside the driver carried the SQL sinks --
the family the declared target belongs to, and the one family that is cheap on this subject. Nine
hypotheses were eliminated by re-measuring the cheap case.
Splitting never rescued it because a failed batch is split by REQUEST COUNT and each part carries the
same everywhere-sinks. contributor-scan 7.7 carries the fix: budget a batch by the sink nodes it
implies, using the count the evidence pass already computes, and report a family whose sinks are
everywhere as expensive rather than as failed.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
Nine hypotheses about the PHP query cost have now been eliminated by rebuilding the request outside
the driver: batch size, call depth, parameter sources, hook table, argument length, sharding,
dataflow cost, the malformed-CPG bug and the engine version. Every one of them answered in under
four minutes while the driver spent whatever ceiling it was given.
The reason the guessing continued is that a scan issues several invocations and the record could not
say which of them burned the budget. It now logs the query kind, the request count, the elapsed time
and whether it answered, for the whole batch and for each part of a split retry, including the part
where a split is abandoned for want of time.
This is task 16 applied to the backend rather than to a harness: an instrument that cannot name its
own expensive step turns a measurement into a guess, and this one has been a guess since September.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
Four census runs on this subject spent whatever query ceiling they were given: 363 s of 300, 1,255 s
of 1,200, 1,874 s of 1,800, at question counts of 2,526 and 406. Cost that tracks the budget rather
than the work is not per-question cost, and tier-zero pruning cutting the count six-fold changed
nothing. A census runs one batch, so no census could answer whether the batch itself scales.
One graph, built once, five batch sizes, every one answered:
25 requests 71.4 s 2.86 s each
50 requests 76.0 s 1.52 s each
100 requests 155.4 s 1.55 s each
200 requests 166.7 s 0.83 s each
400 requests 169.7 s 0.42 s each
Sublinear, and roughly 70 s of it is fixed JVM start and graph load. Four hundred dataflow requests
answer in 170 seconds on the graph production cannot finish 406 on.
So the difference is in the request PARAMETERS. The probe sent callDepth=0 and no parameter sources;
production sends callDepth=3 with parameter sources on every entry point, plus a hook table, a
dispatch list and context evidence. Task 13.2 recorded in September that every reproduction outside
the push path answered in under a minute and the difference was not known. Size is now removed as a
candidate and a four-variant probe is running to isolate the rest.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
The mechanism already existed. A question whose evidence vector says no sink of its family is in
reach cannot produce a flow, and tier-zero pruning defers it. That was switched off for the whole
scan whenever any of twelve degradation reasons appeared, and a multi-language repository always
carries two of them, so the measured plugin arbitrated every question it could ask. Third instance
this week of one blast radius, and the invariant written into the design on Monday named it.
A prune blocker that names a partition now withholds pruning from that partition questions and from
nobody else. cross_partition_semantics_unresolved leaves the blocker set entirely, for the reason it
left the integrity gaps: a PHP family sink is PHP, so a second language cannot be why its vector saw
none.
Measured on Paid Memberships Pro. At 25 regions arbitration falls from 127 questions to 34 with 93
recorded deferred as tier_zero, and 19 findings before and after, identical family by family. At 500
it falls from 2,526 to 406 with 2,120 deferred.
It is not sufficient, and that is the useful half. The 500-region run still fails in the same way at
the same cost, because 406 questions at roughly one and a half seconds each still exceed the
300-second per-batch ceiling. This lever moved the volume; the ceiling is a separate limit.
The census now records what a scan skipped and why, per family, so a cheaper scan and a narrower one
stop looking identical: a family whose every question was pruned reports no sink of this family in
reach rather than region offered, no question asked.
1,420 tests pass, eighteen skipped.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
NodeGoat models 18 of 175 distinct call names, covering 12 percent of 1,185 calls. That share is not
a score: most of what a program calls is not security-relevant, and a high number would mean the
tables were too loose. It is a baseline to move deliberately.
The top product candidate is res.render, 23 calls with five in a method that reads untrusted input.
Task 14.3 reasoned its way to that same gap by hand and declined to model it as a flow sink; the
instrument found it in one run, which is the validation that matters. It also surfaced Math.random in
the user data-access module, and weak randomness is a weakness class this ontology has no family for
at all -- so no family census could have found it, because the family list is itself the blind spot.
That is the kind of gap only a coverage instrument finds.
It had to be taught what is not an API. The first run ranked _tmp_3.push and __ecma.Array.factory
first, which is how the frontend lowers a destructuring and an array literal, and a Cypress suite and
a Gruntfile held four of the next five. Both are separated now, because noise at the top of a
worklist is how a worklist stops being read.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
Five of seven families established nothing in the family census, and every one failed for an ontology
reason with the flow engine ready and nothing to aim at. Each was found by investigating one family
at a time over days. vocabulary.sc asks it once: census the calls, subtract everything any fact table
names, rank the remainder by how often it is called and whether it sits in a method that reads
untrusted input. It needs no dataflow, because the structural pass already walks every call.
The harness reads its token set from the SPECS rather than from the TOML, so it measures what the
analyser actually consults: a fact the loader drops is not part of the ontology however neatly it is
written. Ontology sizes today are 152 tokens for JavaScript, 167 for Python, 84 for PHP.
Both carry the constraint the maintainer set: the output is a candidate list, never an ontology. A
name becomes a fact only when someone resolves it to a library API, states the weakness class and
records provenance, because adding a name because a subject called it is how a general-purpose tool
becomes a fixture of its own benchmark.
Also records what the bounded PHP transfer attempt found. At 25 regions the first case now runs to
completion and writes a full artifact in 1,706 s, where at 100 it had exhausted a 2,400 s deadline
with nothing. But 25 of 4,461 regions is 0.6 percent of the population and the declared target was
not among them, so the question was never asked. Task 13.3 therefore depends on 7.3, skipping
arbitration where the evidence says there is nothing to find: raising the budget is already measured
to fail, and making most of the budget unnecessary has not been tried.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
Named by a week in which three separate causes looked identical as one count of unresolved. The
census reported 19 findings and 0 completed questions side by side without complaint, a stale image
measured the engine a bug had been diagnosed against, and a partition fix looked applied while an
unscoped copy of the same gap kept demoting everything.
Every harness record should carry the counts that disambiguate its failure modes, the degradations
whole rather than by reason, and the engine and ruleset identities actually used; and each harness
should assert its own consistency, because findings above zero with completion at zero is a
contradiction an instrument can notice faster than a person.
This was missed from the previous commit by an escaping error in the edit, not by intent.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
The source, sink, sanitizer, operation, discharger and layout rows are not a pattern list. They are an
ontology of program elements and their security relations, and nearly all of this tool accuracy lives
there: the family census found five of seven families establishing nothing, every one for an ontology
reason with the flow engine ready and nothing to aim at.
The hook is general-purpose, not application-specific. The ontology describes libraries, frameworks
and language constructs; it does not describe repositories. An entry names an API and is justified by
what that API does, carries provenance and a weakness class, and may not encode a path or identifier
peculiar to one project -- a plugin bundled library tree was measured and declined on exactly that
ground. A measured gap is a candidate, never an entry: accretion by subject is how a general tool
becomes a fixture of its benchmark.
contributor-scan owns the ontology, so its design carries that statement and a new group 7 makes
growth measurable rather than anecdotal: report what a repository calls that nothing models, key
sanitizers to the weakness they prevent, skip arbitration where the evidence says there is nothing to
find, share one matcher across the queries, require a degradation to declare its scope, and commit
golden findings so a silent change cannot pass.
pre-push-safety-net carries the general-purpose test for its own reviews, and a task for harnesses
that report the cause of their own failure -- named by a week in which three separate causes looked
identical as one count.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
Task 13.4 found that a second language in a repository cost every question its completion, and so
cost every finding its admission, on the declared v0.1 target: a WordPress plugin is PHP plus its
admin JavaScript. Neither subject that passes today could reveal it, because both hold exactly one
language. A population of single-language repositories would qualify a capability that fails on the
product own target.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
_blocking_degradations already exempted declared exclusions. It now also ignores a gap another
language partition raised, because a JavaScript file missing from the JavaScript graph cannot make a
PHP answer wrong. Treating it as though it could left a WordPress plugin -- PHP plus its admin
JavaScript, which is every plugin -- unable to admit a finding it had already established.
A gap that names no partition still reaches everything, and so does every gap when the claim own
language is unknown, because unknown scope is not evidence of independence. The control asserts all
four directions.
With this and the scan fixes, task 13.4 is verified on the subject that motivated it: PMPro completes
120 of 127 questions where it completed none, its 19 findings unchanged, and the seven still
unresolved are the JavaScript partition own questions, which is the correct answer.
1,420 tests pass, eighteen skipped.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
