Measured after scoping a registration-only region to its file.
M1a, before group 12 0 of 436 head questions completed
after per-family context 298 of 436
after region attribution 299 of 299, and 299 of 299 on the base
The count falls from 436 to 299 because the mis-attributed function-scoped regions
in a route-registration module are gone, replaced by file-scoped questions the graph
can answer. Nothing is unresolved on either revision.
Detections are unchanged: the vulnerable revision is still new with
operation_absent_from_comparable_base, the fixed twin analyzed and quiet, the
unchanged revision not a regression.
What remains is recorded limits rather than failures: 210 bounded-reachability
boundaries, each owned by the question that reported it, and one vendor exclusion.
Neither blocks a comparison.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
A route-registration module registers handlers that other modules define.
app/routes/index.js requires seven handler modules and defines none, yet regions
there were scoped to handleContributionsUpdate and its siblings. The graph rightly
held no such method, so every question in those regions was unresolvable: 138
across six families on that one repository.
The entry record still names the handler. The authorization question is which
middleware guards which handler, and that is cross-file by nature. It now also
records whether the file defines what it names, and a region built from an entry
that does not is scoped to the file instead of to a method the file lacks. The
handler's own module still gets its function-scoped region, so nothing is lost.
My first attempt changed the mapper to stop naming cross-file handlers at all. An
existing control caught it by asserting that a multi-line registration names the
handler its middleware guards, which is exactly the attribution the authorization
family needs. The evidence was against me and the fix belonged one layer up, in
region construction.
1,407 tests passed, nine skipped.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
Final verification of the three security capability fixes on the pinned revisions.
head questions completed 205 -> 298 of 436
base questions completed 436 of 436
access control completed 0 -> 52
context_projection_unavailable 116 -> 0
context_scope_empty 138 -> 0
Every remaining unresolved question now reports context_function_unresolved and
names the function its file does not define, so nothing is unresolved for a reason
the tool could not explain. That is a region-attribution question for the ranker,
not a limit of what the engine can establish.
Detections are unchanged: the vulnerable revision is still new with
operation_absent_from_comparable_base and every verdict entry holds.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
After per-family context, every remaining unresolved question reported
context_scope_empty, all in three files. The cause is not a missing producer:
app/routes/index.js requires seven handler modules and defines none of them, so a
question naming handleContributionsUpdate there names a function that file
genuinely does not hold, and the empty scope is the correct answer.
Reporting that as "scope empty" hid which of two different things had happened. A
file the frontend produced no methods for is one problem; a question naming a
function defined in another module is another, and it is the common one. Taint and
configuration now separate them, reporting context_function_unresolved with the
name in the first case and context_scope_empty only when the file has no methods at
all. The driver no longer stacks its generic gap on top of a query that already
explained itself, and the new gap is question-owned so it cannot block another
question's comparison.
Completion is deliberately unchanged. The tempting fix is to widen such a question
to its whole file, which would raise the number by answering a question nobody
asked; a question whose function the graph does not hold stays unresolved, with the
name now recoverable from the question that asked it.
Group 12 is complete: admission no longer rejects genuine findings on a
mis-extracted operation symbol, both families that could never complete their change
context now can, and the remaining unresolved questions say precisely why.
1,406 tests passed, nine skipped.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
Rerunning the pinned revisions with each family producing its own change context.
head questions completed 205 -> 298 of 436
context_projection_unavailable 116 -> 0
access_control completed 0 -> 52
Access control is the headline: the family could not complete a question on any
repository before, and now completes every question it is asked. That is a
vulnerability class the tool could not reason about at all.
Detections are unchanged. The vulnerable revision is still new with
operation_absent_from_comparable_base, the fixed twin is analyzed and quiet, the
unchanged revision is not a regression.
The remaining 138 unresolved questions are all context_scope_empty and sit in three
files, none of them access control. They are function-scoped questions whose named
function the query cannot locate in the graph, so the scope it describes is honestly
empty. That is the query-to-operation identity problem M1a recorded at the witness
end, showing up at the scope end, and it is task 12.3 rather than something to patch
by widening a question to its whole file.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
Access control and configuration could not complete their change context on any
repository, which left 231 of 436 questions unresolved on the measured Node case
and made two of the three shipped families unable to answer.
The gap was larger than a missing query block. The evidence pass dispatches taint
only, because only taint has an evidence vector to rank by, so these families were
never asked for context at all. A new context pass asks each family with the flag
set.
Each query describes its own scope, which the owner research required and which
matters: copying taint's projection would describe a reachability walk these
queries never perform. Dominance exports the file's methods labelled operation,
guard or sibling, because its claim is that a file contradicts itself. Configuration
exports the file's methods labelled setting or sibling, because it asks what a file
declares. Both report an empty scope explicitly rather than implying one.
A family whose context query answers nothing keeps no context and stays unresolved,
so none can complete on another family's context. A control asserts that directly.
Confirmed on a real NodeGoat graph through the production backend: both families
produce a context summary and seven scoped methods where they previously produced
nothing. All seven are siblings for that file, which is the honest answer since it
declares no access-control operation and no matched setting.
1,405 tests passed, nine skipped.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
Admission extracted everything before the first parenthesis, so the operation for
`const preTax = eval(req.body.preTax)` was `const preTax = eval`. No real
declaration can enumerate every assignment a codebase might write, so admission
would have rejected genuine findings on operation_semantics_mismatch. The M1b
demonstration only passed because its declaration listed the assignment spellings.
The symbol is now the trailing qualified callee. Receiver spelling stays part of
it, so `$wpdb->prepare` and `prepare` remain different operations and a declaration
cannot widen from one to the other. A spelling that is not a qualified name, such
as a dynamic `handlers[name](x)`, yields the statement text, matches no declaration
and stays ineligible exactly as before.
Controls cover the assigned call, the receiver distinction, the dynamic spelling,
and that a declaration for a different operation still does not cover it. The
recorded-veto report reads the same helper instead of repeating the extraction.
1,403 tests passed, nine skipped.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
Both attacks on engine startup are measured and neither reaches the gate. The
session is equivalent for every shipped query kind and helps a long transaction,
326 s to 243 s, but its own 9 to 13 second startup is a third of a 30-second budget
so it usually declines to start. Removing JavaScript's second engine start is real,
11.0 s against 21.0 s on identical graphs, and was reverted for an asymmetric
census gate that made 12 transactions pay two starts.
What the profile names is not startup. A changed-code transaction reaches its
deadline inside the build: head build 25.2 s of 30 s, query time 0.0 s. The cost is
building a fresh whole-repository graph for every pushed revision.
Direction accepted: reuse the graph across revisions rather than optimise the
rebuild. M2 stays unmet, the gates stay unchanged, and the work is not scheduled
here. Group 12 returns to security capability, which is where the remaining release
risk sits.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
Reverts f04a839 on its own measurement. The seven-class profile rerun recorded
joern_parse_census_incomplete on 12 transactions: the faster build was rejected by
its own census gate and the frontend plus overlay pass ran as well, so those
transactions paid two engine starts instead of one. Every workload kept the same
p50 and the verdict stayed NO-GO.
The cause is an asymmetric gate. The frontend path is not census-gated at build
time, so whatever it produces is accepted, while the joern-parse path was gated and
the same shortfall rejected it. Holding the new path to a standard the old one
never has to meet guarantees the fallback fires. The probe tree and the
materialized push snapshot also differ, so equivalence on the checkout did not
transfer to the real path.
What still stands is the underlying timing fact, measured under the production
retention environment on identical graphs, same 44 files, 579 methods, file names,
overlays, Gruntfile and eighteen test files: 11.0 s against 21.0 s. JavaScript does
pay two engine starts where every other language pays one. A future attempt needs a
gate both paths are held to, or a direct comparison against what the frontend would
produce, and must be measured in the push path rather than on a checkout.
The profile also names the remaining cost plainly: a changed-code transaction
spends its budget before any query runs, head build 25.2 s of a 30 s deadline with
query time 0.0 s. Building one new graph per push is the cost, not querying it.
1,401 tests passed, nine skipped. Rollout stays NO-GO.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
JavaScript was the only language paying two engine starts: the declared frontend
and then a separate overlay pass. joern-parse applies the overlays itself. On the
pinned Node tree that is 11.0 s against 21.0 s.
The declared frontend stays as the fallback, because it is what the versioned
retention adaptation and the explicit exclusion policy run through, and the census
decides per build: a graph that does not hold every readable declared file is
rebuilt through the frontend rather than trusted. Exclusions keep the frontend path
outright.
Measured under the production retention environment, both paths produce the same 44
files, the same 579 methods, the same file names, the same five overlays, the
Gruntfile and all eighteen first-party test files, with no file present in one
graph and missing from the other.
My first measurement did not establish that. It ran without the retention
environment, both paths dropped to 25 files, and the apparent equivalence was two
runs discarding the same things. The existing control named for that decision
failed against the change built on it, which is exactly what it is for; the
evidence was wrong, not the control. That control now asserts the intent directly:
the faster path is tried first, and a short census rebuilds through the frontend.
1,401 tests passed, nine skipped. A build saving is not the latency gate; the
profile rerun decides that. Rollout stays NO-GO.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
The seven-class profile showed a 25.3 s head "build" of which the trace attributes
only 4.5 s to the frontend. The remaining twenty seconds were the session's startup
poll, which was bounded by `min(startup_timeout, deadline - now)` and therefore ran
until the transaction deadline was exhausted. The fallback then had nothing left,
so the transport's failure mode consumed exactly the budget it existed to save.
A session now refuses to start unless enough budget survives for the work it exists
to serve, and the wait is bounded by its own allowance rather than by the whole
deadline. Under the 30-second hook deadline it declines immediately instead of
burning what is left.
Controls cover both: refusing is immediate and starts no process, and a startup
wait ends at its allowance nowhere near a long deadline.
This does not make the gate reachable; it removes a self-inflicted cost that was
hiding the real one. 1,401 tests passed, nine skipped.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
The unchanged seven-class profile, three samples each, with the engine session
enabled. Every changed-code workload still times out at the 30-second deadline, and
identical-tip moved the wrong way: p50 rose from 18.5 s to 24.2 s.
The cause is in the artifacts rather than inferred. Thirteen transactions report
session_unavailable:session_startup_timeout, so under the real deadline the session
usually never starts and everything falls back to the disposable path. Startup is 9
to 13 seconds, about a third of the whole budget, and it is spent before any
analysis happens. Where a session did start, on cached-graph transactions whose
remaining work is small, it cost more than it saved.
That is the honest shape of the result: a transaction-owned session amortizes
engine startup across the invocations inside one transaction, and a warm
changed-code push does not have enough of them. It helps a long development-budget
transaction, 326 s to 243 s measured earlier, and cannot help a 30-second one.
Correctness is unchanged throughout: the same three pinned revisions through the
production rules with the session enabled reproduce the M1 verdict exactly.
Group 11 as designed cannot reach the gate, so the dominant remaining cost is named
and the decision is raised rather than worked around. Task 11.6 records the
candidates, since two of them change the product's shape: a session shared across
transactions is a resident local service, which the accepted milestone spec does
not imply, and analysing less is a scope change rather than a speed change. The
30-second and 2-second gates stay unchanged.
Rollout stays NO-GO.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
The leak was the whole cause. With the session closed by the transaction that owns
it, a session-enabled replay of the same pinned revisions under the same budget
keeps its session for the whole run: no session_query_failed appears on either
side, where the previous run reported one per side and fell back to the disposable
path for everything.
Measured on the same pins, instrument and budget, changing only the transport:
vulnerable revision 326.4 s -> 243.3 s
head query time 63.6 s -> 32.0 s
base query time 85.1 s -> 51.2 s
dominance and config ~14 s each -> ~4 s each
Detections are unchanged: witness present, fixed twin analyzed and quiet, unchanged
revision not a regression, kill criterion not triggered. Speed bought by losing a
detection would not be speed.
This is a cold transaction on one teaching repository under a 900-second
development budget. It is not the hook latency gate, which is warm changed-code p95
against the frozen seven-class profile, and that is task 11.5. Rollout stays NO-GO.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
The first session-enabled replay produced correct detections and no speedup,
reporting session_query_failed:census:session_request_failed on both sides. A
targeted probe ruled out the sequence I suspected: an overlay followed by a census
in one session both answer correctly, with the census confirming the overlays were
saved.
What the probe exposed instead is a leak. A build runs on a copy of the backend, so
the session started during the build belonged to that copy and was never closed,
leaving a JVM holding the configured 2 GB heap for the life of the process. Two
builds per transaction meant two of them, which is enough to exhaust a 3 GB
container and make every later request fail. It also matches the host running out
of memory during these runs.
The build now closes any session its copy started, and the driver ends the session
with the transaction, since that is what transaction-owned means. Controls cover
both: a build leaves nothing alive on the copy that started it, and a completed
scan closes the backend's session. Closing never fails a scan that found things.
This is a genuine defect found by measurement rather than by the suite, which
passed throughout. Whether it was the whole cause of the census failure is not yet
established; the next session-enabled run will say.
1,399 tests passed, nine skipped. Rollout stays NO-GO.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
The session-enabled NodeGoat run reported session_query_failed:census:
session_request_failed and fell back, which is correct behaviour but says nothing
about why. A bare transport failure cannot distinguish a timeout from a crash from
an out-of-memory kill, and the only account of which it was is in the server's own
log.
A transport failure now carries the exception and the tail of that log, and the
backend logs it with the process exit code alongside its own label. A control
asserts an OutOfMemoryError in the log reaches the caller.
This is diagnosis, not a repair: the run's timings were unchanged because the
session died early and everything fell back to the disposable path.
1,397 tests passed, nine skipped.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
A measured run needs to enable the transport without changing any default. The
switch reads as enabled only for the exact value "1"; unset, empty, "0", "true"
and anything else leave the disposable path in place, and an explicit constructor
argument still wins in both directions.
1,396 tests passed, nine skipped.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
The overlay was the last fixed cost bypassing the session: a whole JVM start,
15.8 s in the measured Node runs, while the census and query batches already route
through it.
`save` writes under the process working directory, which for a session is its own
private scratch rather than the build scratch, so the saved graph is now looked for
in both. Finding neither keeps the raw graph and records it rather than reporting
success, because a graph without its dataflow layer has that layer recomputed by
every later batch, 54 s each on a 637-file plugin, with nothing otherwise saying so.
The disposable invocation remains the fallback and runs unchanged whenever the
session is off or cannot answer.
Controls cover both new paths: an engine that reports success but saves no graph
keeps the raw one and notes it, and a graph saved under the session's workspace is
found and applied.
1,395 tests passed, nine skipped. The transport stays off by default; task 11.5
measures against the frozen profile. Rollout stays NO-GO.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
A session is only usable if it returns exactly what the disposable invocation
returns. On a real NodeGoat graph, taint, dominance and configuration all produce
byte-identical payloads through both transports, with parameters built by each
arbiter's own request function through the driver's family lookup rather than a
hand-written shape.
The failure shapes that would make a session dangerous rather than merely slow are
correct too. An empty answer stays present and distinguishable from an unavailable
one, which is the distinction that once turned a failed scan into a clean bill of
health. An unknown query and an absent graph are both unavailable. Two graphs in
one session each answer about the graph the request names.
Amortized over five taint requests the session took 22.5 s against 72.0 s
disposable, all five answered and identical to each other.
Two instrument defects are recorded rather than hidden. The first run created a
fresh backend per comparison, so every session served one request and paid full
startup, measuring the session's worst case and making it look slower; that is why
the amortized comparison exists. The switched-graph check crashed on a scratch
directory it never created, so it produced no verdict at all.
These timings are orientation, not the latency gate. Task 11.5 measures against the
frozen seven-class profile, and the transport stays off by default until then.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
JVM startup, not analysis, is what a push scan spends its time on: each disposable
script invocation costs 13 to 23 seconds while the work takes under a second.
The session owns one Joern server per transaction on loopback with an OS-chosen
port and no published port. Three properties are load-bearing, and the probe that
settled the design demonstrated why each is needed.
An HTTP status is not an answer. The server returns success even for a compile
error, with the diagnostic in its body, so every request redirects the evaluated
code's console output into a fresh private file behind a nonce, and the nonce is
verified before the answer is accepted. The engine's body is retained for exactly
the case where the receipt is missing, without which that failure explains nothing.
A poisoned session answers nothing afterwards, and cancellation kills the process
group rather than the launcher, which this project has already run a machine out of
memory by neglecting.
What the probe established about the engine, after three iterations that each
failed loudly: runScript is absent from the server REPL; a shipped @main script
cannot be nested in a block expression, so definitions go to the top level
unwrapped; and the REPL shadows Console with io.joern.console.Console, which has no
withOut, so capture must qualify scala.Console. With that fixed, both mechanisms
answer identically to the disposable invocation, 0.8 s against 9.8 s.
The backend routes batch and single queries through the session only when
session_transport is enabled, which it is not by default. Any failure falls back to
the disposable invocation rather than reporting a zero, and each script is defined
under its own name because every script declares exec.
Equivalence for taint, dominance and configuration is task 11.2 and gates any
measured run. 1,391 tests passed, nine skipped. Rollout stays NO-GO.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
The M1b exit run established the comparison but could not show a developer-readable
consequence or repair direction, because both belong to an evaluated capability
declaration and the installed registry is deliberately empty.
The experimental evaluation output now renders, for each change-attributed finding,
its family, exact location, witness, change attribution, provenance, engine method
and operation, together with a consequence and repair direction produced from a
supplied declaration through the same grounded-template path the admitted case
uses. No prose is invented: when no declaration matches, or a template is not
grounded in the operation and context, or the declaration does not cover the
operation symbol, the explanation is absent and the reason is stated.
Declarations reach this rendering only. The runner never passes them to admission,
so nothing in that file can enable a capability or emit an alert however it is
written, and a control asserts that even a declaration claiming PASS admits
nothing. The flag requires explicit base/head replay and the recorded-veto flag.
The evaluation instrument generates its own unreviewed declaration bound to the
run's analysis semantics, asserting that it is neither enabled nor PASS, so the
demonstration cannot drift into a qualified claim.
1,377 tests passed, nine skipped. Rollout stays NO-GO.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
All three pinned NodeGoat revisions replayed through the production rules with
exact pins and reopened source receipts.
The vulnerable revision reports all three sites new with
operation_absent_from_comparable_base, and admission now rejects on
capability_unavailable alone: dependency_unresolved is gone, which verifies 10.7
on real data since those gaps belonged to unrelated questions. The fixed twin is
analyzed and quiet with three base-only findings. The unchanged revision reports
all three sites unchanged with base locations resolved to the same lines.
Every finding carries its question, the engine-reported method and its witness.
The head census is the complete 44 files and all 436 base questions complete.
Zero admitted defects on every revision.
All five vetoes M1a recorded are repaired, plus two the repairs exposed. What
remains is recorded rather than repaired: 231 head questions still report an
incomplete change context from families whose queries export none, which did not
block this case and whose families stay experimental, and the vendor exclusion
and correspondence ambiguity remain reported as the limits they are.
M1b is not yet verified. The milestone also asks for a developer-readable finding
carrying a supported security consequence and a concrete repair direction, and
both are owned by an evaluated capability declaration that is deliberately empty.
Writing that text without a declaration is what admission exists to prevent, so
task 10.10 renders it through an explicitly experimental declaration with
eligibility still disabled.
Runtime, independent qualification and rollout all remain NO-GO.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
M1a recorded that the NodeGoat target is selected through a file-scoped question
naming no function while the engine reports method <lambda>2, and that this did
not block the comparison once the question completed. It was still only visible by
digging the method out of a serialized detail blob.
Evidence now carries the engine-reported method beside the question that produced
it, and the relationship between them is recorded: exact when the question named
that method, file_scope when the question named no function and the engine
supplied one, unresolved when the two disagree, and method_unreported when the
engine named none.
Recorded, not applied. No region is renamed, no benchmark function name is
injected into a region, and no coincident site is counted. The recorded-veto
report now reads the field instead of reparsing the detail blob.
1,371 tests passed, nine skipped. Rollout stays NO-GO.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
The runner handed every candidate the whole transaction's coverage reason list as
its unresolved dependencies, so admission always answered dependency_unresolved.
That list even included other candidates' comparison outcomes, so one unresolved
comparison anywhere marked every other claim as having an unresolved dependency.
Gaps are now attributed. A boundary carrying a question id counts only when that
question is in this candidate's required comparison scope, the same scope 10.2
established. A gap nothing attributed to any question always counts, because
unknown ownership is not evidence of independence. Integrity degradations count.
A declared exclusion, a correspondence ambiguity and a bounded reachability do
not, because each was already weighed against this specific claim: the first is a
scope choice, the second is decided per operation, and the third is decided per
claim by the comparison.
Found by the M1b production rerun rather than by the suite: the comparison reached
a correct new verdict and admission still rejected it for a dependency belonging
to unrelated questions. It is not an M1b exit blocker, since eligibility stays
disabled, but it would have blocked admission forever once capabilities qualify.
1,371 tests passed, nine skipped. Rollout stays NO-GO.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
Rerunning the three pinned revisions after tasks 10.1-10.6 shows the production
comparison classifying all of them with no lifting, and agreeing exactly with the
recorded view. That agreement is the point: the vetoes M1a recorded were repaired
rather than bypassed.
The vulnerable revision is new with operation_absent_from_comparable_base, and the
three eval witnesses at contributions.js lines 32 to 34 are attributed to the
change that introduced them. The fixed twin is analyzed and quiet with three
base-only findings. The unchanged revision is unchanged. In M1a every one of the
436 questions was demoted on both sides and production reported
witness_identity_unresolved on every finding.
All three artifacts still report finding_status none with zero admitted defects.
Admission rejects on capability_unavailable, the deliberate eligibility gate, and
on dependency_unresolved, which the rerun exposed as coarse plumbing: the runner
passes every coverage reason as every candidate's dependency gaps. That is now
task 10.7, ahead of M4, since it would block admission forever once capabilities
qualify. It is not a comparison failure and not an M1b exit blocker.
Three comparisons on one inspected teaching repository under a development budget.
No capability enabled, no alert emitted, no latency claim. Runtime, independent
qualification and rollout all remain NO-GO.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
The NodeGoat rerun after 10.4-10.6 showed both target questions completing, which
is what those tasks were for, but the comparison still reported
head_context_incomplete. Two checks inside comparison had not been updated with
the same distinction: any degradation blocked, and any scope boundary blocked.
A declared exclusion and a correspondence ambiguity both reached those checks. The
first is a scope choice rather than a failure to read the graph, and the second is
decided per operation since 10.3, so neither should block a claim. Both remain in
aggregate coverage. Integrity degradations, unknown ownership and every other
boundary still block, and bounded reachability is still filtered separately
because it blocks only claims resting on a flow not having been traced.
1,368 tests passed, nine skipped. Rollout stays NO-GO.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
vendor_semantics_unresolved was grouped with the graph integrity failures, so any
declared exclusion demoted every otherwise complete answer in the repository to
graph_incomplete. Every real repository excludes its dependencies, so this fired
on all of them, including on a complete first-party census with successful query
receipts.
Graph boundaries are now split by what they mean. An integrity failure says the
graph could not be read or is incomplete, so an answer drawn from it is demoted:
unparsed files, an empty or sharded graph, an incomplete or unavailable census,
an unresolved symlink, an unsupported frontend, unreadable source, an ambiguous
frontend path and unvalidated TypeScript property support all keep that behavior.
A declared exclusion is a deliberate scope choice. It stays physical, stays out of
targets and graphs, and stays reported in coverage, but it is not evidence that a
first-party answer is wrong.
Whether a particular answer needed the excluded semantics is carried by that
question's own unresolved call destinations, recorded per question in 10.5, rather
than assumed for every answer in the repository.
Controls exercise the real partition logic rather than an injected reason: a
vendored directory leaves the answer completed with the exclusion still reported,
an unresolved symlink still demotes, and the two together still demote.
1,365 tests passed, nine skipped. Rollout stays NO-GO.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
dynamic_external_or_depth_context_unresolved was applied as though it invalidated
everything about a question. It does not. Bounded reachability can hide further
flows; it cannot invalidate a flow that was traced, and it cannot remove a call
from a structurally scoped enumeration.
At scan level the boundary is now recorded against its owning question without
demoting that question's answer. An unavailable context projection, an empty
context scope and an unreadable method extent are unknown ownership and still
demote.
In comparison it no longer blocks the head's own claim, since the finding rests
on a path that was actually traced, and no longer blocks an absence read off the
enumerated inventory. It still blocks every claim that rests on a flow not having
been traced, including the anchored source-connection branch, which now reports
base_reachability_bounded rather than silently passing.
The query declares the basis of its enumeration. A file- or nesting-scoped
question walks calls structurally, so an unresolved destination cannot remove one
from scope. Following the call graph makes the reachable set the scope, so the
same unresolved destination can shrink what was enumerated; that inventory cannot
establish absence while reachability is bounded, and says so.
One prior control asserted a question's own reachability gap blocks its own
claim. It is updated with its reason and split, so an own projection gap is still
proven to block while an own reachability gap no longer does.
1,362 tests passed, nine skipped. Rollout stays NO-GO.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
A taint answer reports traced flows, so an absent row means either that the
operation is not present or that no traced path reached it. Change comparison
needs that distinction to say a change introduced an operation, and reachability
is deliberately bounded, so the two can never be separated from flow rows alone.
That is why the veto guarding the confusion is permanently true on any real
Express application, and why it could be neither discharged nor safely removed.
The taint query now exports one row per operation it enumerated in scope, plus a
completion marker. The marker is what makes an empty inventory meaningful:
without it a scope that was never enumerated and a scope holding no such
operation are indistinguishable.
Comparison separates three outcomes on a changed line: the operation is present
in the base without a traced flow, which is a reachability statement and not
novelty; the operation is absent from an enumerated base, which establishes that
the change introduced it; or the base scope was never enumerated, which stays
explicitly unresolved. A malformed inventory row is not an empty inventory.
The arbiter now skips any row carrying a kind, so an inventory row cannot be
normalized into a flow with an empty source that names no attacker input.
Verified against the real engine on an authored two-revision JavaScript control:
the operation is enumerated on both sides, the base carries it with zero traced
flows, the zero-operation marker appears for an empty scope, and answers and
findings are unchanged. No new sink inference, alias resolver or second source
IR. 1,354 tests passed, nine skipped. Rollout stays NO-GO.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
Rerunning the three NodeGoat revisions after tasks 10.1-10.3 shows the recorded
view now classifies the case: the vulnerable revision reaches new with
operation_absent_from_comparable_base, and the unchanged revision reaches
unchanged with its base location resolved to the same lines. Both previously
stopped at operation_correspondence_unresolved and witness_identity_unresolved.
The rerun also caught a defect in the repair itself: the new novelty reason was
missing from the admission allowlist, so admission rejected the candidate as
change_unsupported. Fixed with a control.
The production path is still blocked by the last two vetoes, and the measurement
shows neither can be scoped or relaxed as the plan described. A taint answer
reports traced flows, so an absent row means either that the operation does not
exist in the base or that no traced flow reached it.
dynamic_external_or_depth_context_unresolved exists to stop the second being read
as the first, and removing it would turn a wrong absence into a wrong new
finding. It is also computed over every in-scope method, so on any real Express
application it is always true and can never be discharged, which is why the tool
reports nothing on real projects.
The resolution is new evidence rather than a relaxed rule: export the operations
the query actually enumerated, separately from the flows it traced, so absence is
established from the inventory while a traced-flow claim still requires complete
reachability. That is now task 10.4, ahead of the two vetoes it unblocks, and it
adds no detector, alias resolver or second source IR.
1,348 tests passed, nine skipped. No capability enabled; rollout stays NO-GO.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
Two vetoes M1a recorded were global where the evidence is local.
Comparison required every selected base question to complete, so one unsupported
family anywhere prevented an otherwise complete target comparison. Completeness
is now demanded over the required scope: the counterpart question plus the
dependencies the change context actually recorded for it. Every other gap stays
in aggregate coverage and is still reported, never discharged. An incomplete
counterpart, an incomplete recorded dependency, an unknown-ownership boundary
and a truncated base scan all still yield unknown.
Correspondence ambiguity was recorded file-wide and inherited by every question,
so repeated unchanged lines in one file invalidated every question in the
repository and blocked the whole transaction comparison. It is now decided per
operation by the correspondence classifier: an operation needing an ambiguous
anchor stays uncomparable, while a uniquely mapped or newly introduced operation
in the same file is unaffected. The mapping is still withheld and still reported.
On the NodeGoat case the ambiguity is on the target's own file, so file-level
scoping would not have been enough; only per-operation resolution is.
1,347 tests passed, nine skipped. No capability enabled; rollout stays NO-GO.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
M1a measured that a newly introduced operation sits on a changed line, has no
base line anchor, and therefore stayed unclassified even with every completeness
veto lifted. Establish absence over the complete counterpart answer instead of
over a line anchor.
Correspondence now reports why a head operation does not map: `changed_line`
inside a recorded head edit, `ambiguous` where several base lines could
correspond, or `unmapped` where no anchor exists. Only the first supports
reasoning about an introduced operation, and only when the base counterpart
answer is complete and contains no operation with that spelling.
Movement is matched on the operation spelling alone, so a benign identifier
rename that changes only the source text reads as movement rather than a new
flow. The existing rename control caught that; the key was too brittle before.
One prior control is superseded and updated with its reason: a new operation on
a changed line in a comparable existing file is now new rather than unknown.
Movement, ambiguity, unmapped operations, discharged or unentailed claims and an
incomplete or failed base all remain unknown.
1,337 tests passed, nine skipped. No capability enabled; rollout stays NO-GO.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
Add group 9 to the approved task plan as the bounded owner amendment release
milestone M1a requires, implement the task-local recorded-veto evaluation flag,
and run the three pinned NodeGoat revisions under it.
The flag re-runs the production comparison and admission over a lifted copy of
the same evidence and records each veto instead of applying it. Recognized
completeness vetoes are lifted and logged; failures never are, because silence
from a failure is not a quiet fixed twin. It is off by default, accepted only on
explicit base/head replay, refused in Git stdin mode, labels every payload it
produces, and admits no defect even when a lifted candidate otherwise would.
M1a outcome, inside its one-week box and with expectations frozen on disk first:
the three entailed <lambda>2 witnesses are present at contributions.js 32-34; the
fixed twin was genuinely analyzed and quiet, answering its target question with
zero rows and no deadline reason while the same findings appear as base-only; the
unchanged revision was analyzed and recorded unchanged. The kill criterion did
not trigger.
Five vetoes fired and were recorded. M1a also corrected the diagnosis: with all
five lifted the detection still stops at operation_correspondence_unresolved,
because a newly introduced operation sits on a changed line and has no base
anchor. The unchanged revision resolves its base location on the same lines and
classifies as unchanged, which isolates correspondence as the mechanism. That
fifth veto was not in the earlier four-veto diagnosis and is now M1b's central
repair.
Diagnosis only. No defect admitted, no capability enabled, no gate changed, no
latency claim; the 900-second development budget cannot measure the hook gate.
Rollout and task 8.3 remain NO-GO.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
Add release-milestones.md as the executable outcome contract: prove one useful
Node detection, make it fast, verify PHP/Python transfer, qualify on untouched
cases, then enable a narrow advisory rollout. Link it from the roadmap, safety-net
steering, requirements, tasks, status and spec.json.
Split M1 after review. The raw NodeGoat <lambda>2 witness already exists in the
2026-09-14 post-census bundle; four admission vetoes discard it. M1a runs the
vulnerable/fixed/unchanged revisions under a task-local experimental flag that
records each veto instead of applying it, with a one-week box and a kill
criterion if the fixed twin is not quiet. M1b repairs only the vetoes that fired
and reruns through production rules. Existing approvals and thresholds unchanged.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
Two decisions the maintainer made, as rows rather than constants. A vendored tree (vendor/, node_modules/,
third_party/) is somebody else's unit: out of the targets in preprocess and out of the graph through the
frontend's --exclude, which the scan hands to the backend because the CPG seam imports only the standard
library. A test path (tests/, test-*.php, *Test.php) stays in scope -- a test that reaches a sink still
names the sink -- but its regions are not shipped and order after the product's. Measured before the row:
207 of WP Statistics's 357 PHP files were vendored, and 14 of the 67 regions ahead of MW WP Form's CVE
were test files. The position benchmark applies both, so it measures the order the scan spends.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
php2cpg drives PHP-Parser through `php` from a script inside its own install. A second install the
containerised interpreter had no mount for parsed 1 of 637 files and exited 0 -- caught by the build
census, and now refused by name before any build. On 4.0.625 with the closure exclusion gated off:
pmpro 637 of 637 in 100 s, WP Statistics 115 s with only the two data tables excluded. Phase-2 v5
verdict and the within-tier reading recorded in the brief.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
v5 (parameter-fed carried confirmed by the exact flow): pmpro 69, wpstatistics 172, mwwpform 68,
vampi 16. Reading the recorded head of each order: every region ahead of a CVE is in its tier on a tied
or higher score, and 10 of the 67 ahead of MW WP Form's are test-mail.php. The build's own word on what
is product was on the region and not in the vector; it is now, at -2.0 for an unshipped file. Orders
only, never promotes.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
v4 (provenance split): pmpro 69 and wpstatistics 172 unchanged -- their tier-4 members are source-fed
through same-file helpers that read the request; mwwpform 68 -> 248, because its CVE's region is fed by
a constructor parameter, the shape the split demotes and the exact flow had put at 30. v5 asks the flow
for exactly the parameter-fed fields a sink-bearing region reads, memoised per (family, file, field),
and reports carriedBy for diagnosis.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
methodsByFile, seedsByFile, fieldTaintMemo and hookKeysMemo were declared inside rowsFor and so memoised
nothing across a batch's requests: every pair re-walked the file's methods and re-ran the join's flow
queries. Hoisted, with the family's sources, sanitizers and parameterSources in the key.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Structural carried (v3) measured: pmpro 391 -> 69, wpstatistics 504 -> 172, mwwpform 513 -> 68 -- and
tier 4 swollen to 555 and 569 regions, because "assigned from a parameter" was folded in with "assigned
from a source" and every constructor does the former. The query now reports carriedKind; the vector
promotes on "source" and gives "parameter" a within-tier weight of 0.75.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
The flow form -- a reachableByFlows per candidate assignment -- turned pmpro's evidence pass from 255 s
into one that did not finish in 1,800 s. Evidence may be a structural proxy, so stage one of the join is
now asked as structure: a field the region reads is assigned, in its file, from a source expression, from
a parameter of the assigning method, or from a same-file method that reads a source; a hook applied in
scope has a registered callback holding a source or such a field read. Memoised per file and per
callback for the batch. Verified on WP Statistics's record(): carried = true, 36 s including the JVM.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
A pass over every region of pmpro (22,315 pairs) took 255 s without the join facts and past the scan's
300 s default with them, and a timed-out pass answers nothing -- reported once as tiers={} and the static
order. mwwpform with carried: 292 -> 30.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
budget_at_recall by a real evidence pass over every region: pmpro 391 -> 116, wpstatistics 504 -> 223,
mwwpform 513 -> 292, vampi 16 -> 16. The gate (< 50) is not met. pmpro's CVE is inside its 121 tier-4
regions; WP Statistics's and MW WP Form's are one tier low because `carried` -- stage one of the
two-stage join -- was not reported as evidence. taint.sc now reports it where a sink exists, and the
vector reads the query's answer when given.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
joern#6281 is closed as fixed in v4.0.625 (verified here on the reproducer). The version is read off the
jar beside the launcher; below 4.0.625 the miscompiled file is still excluded, at or above it nothing is,
and an install whose version cannot be read keeps the workaround.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
WP Statistics's first region is JavaScript, and a build asked for as JavaScript went looking for
jssrc2cpg, failed both launchers, lost the whole run's output and left a 9.6 MB scratch directory behind.
Also notes upstream: joern#6281 is fixed in v4.0.625.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
One flatten too many turned "base,controlflow,typerel,callgraph,dataflowOss" into "b,a,s,e,...", and
every real build then warned that its graph lacked the dataflow layer while carrying it. The compile test
now asserts the shape on a real graph.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
`Evidence.score` orders pairs within a tier by one table of weights (open sinks, cleansed sinks, local /
carried / near sources, entry, declared public), capped so volume stops counting, and zero for tier 0 so
the weights can rank but never manufacture. `ScanBudget.order_by_evidence` runs the evidence pass over
every region and cuts `max_regions` from each region's best (tier, score); `order_by_evidence()` is the
one ordering function, used by the scan and by `benchmarks/ranking/position.py --evidence`. Off by
default until the position benchmark says where the pinned CVEs land under it.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Three checks for three findings that were each silently wrong for a day (task 5.13). A clean build whose
census cannot be taken inside the query timeout is a failed build, by name: every later query would time
out and read as "nothing here", and the launcher fallback no longer rebuilds the same graph without
asking again. The census now carries the graph's saved overlays and the worker JVM's actual heap, and the
backend warns when the dataflow layer is missing (every batch would recompute it) or the heap is not the
configured one (the knob is not reaching the JVM that matters). An empty tree counts as complete, not as
a failed count.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Ten real pmpro pairs at callDepth 3 returned 3,838 flow rows. Every field the driver reads is a property
of the sink or of (source kind, source, sanitized); only `length` varies per path, and the driver ranks
by the shortest. Grouped on that key with the minimum length and a `paths` count: 1,155 rows carrying
3,838 paths, same time. The specs are corrected too -- the first write-up called the depth-3 rows
"mostly the same evidence repeated", and 3.3x is what the measurement says; the other 30x is distinct
sources.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
pmpro, 637 files, the product's own build path after 60963f5: build 97 s with overlays, evidence 76 s,
taint 4,105 s for 348 real requests (11.8 s each), arbitrate 99 s, peak child RSS 2.4 GB. The CVE is at
class.memberorder.php:936 among 250 entailed findings across 40 files -- the number the flow-aware-ranking
spec now has to concentrate. callDepth measured on the same graph: a 1.4x time term and a 100x payload
term (3,838 rows for ten requests at depth 3 against 38 at depth 1).
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Task 5.13, measured on ten real pmpro pairs at callDepth 1 with each join half switched separately and
the 54 s JVM fixed cost subtracted: both halves off 11.1 s/request, field half 10.9 s, hook half past
49 s (timeout). Stage timers put it in `hookSeedsIn("")`: every field access in all 7,061 methods put
through the field join before the first callback was answered. Seeds are now the callback's own body
and file, which is where WP Statistics's `set_current_page` and `$this->rest_hits = ...` both live.
Both halves on: 9.7 s/request. `sourceNodes` is also materialised once per request instead of once
per sink.
The WP Statistics re-verification then exposed the larger term. A `joern --script` run is two JVMs and
`-J-Xmx` sizes only the launcher, so the heap knob governed nothing; the backend now sets
JAVA_TOOL_OPTIONS too. And since PHP builds went through php2cpg directly (1ce7e9d, an hour after the
hook half was verified from a joern-parse build), every query batch has recomputed the dataflow overlay
at load -- 54 s on pmpro, and on WP Statistics an OutOfMemoryError inside OssDataFlow after 21 minutes
at 2 GB, no completion in 25 minutes at 4 GB, and no help from --max-num-def at 1000 or 200. Two
vendored browser-profile tables, one array literal each, were 60% of that graph.
Three changes, all named in the log: `queries/overlay.sc` applies the overlays once at build time and
the saved graph replaces the raw one (joern-parse --overlaysonly NPEs in 4.0.623); a source file over
1 MB is excluded as a data table and reported in `unparsed`; and the seed scoping above. After: the WP
Statistics graph reloads in 8 s, the `record` request that timed out at 520 s answers in 39 s including
the JVM, and CVE-2022-25148 is entailed again with `sourceKind = hook`. A raw-vs-saved payload diff on
pmpro shows identical evidence keys and fewer duplicate flow rows.
Also: the backend's own unit tests left fifteen ousast-cpg-* directories in /tmp per run; conftest now
keeps tempfile under the test's directory. The compile test covers overlay.sc.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
A pair with no sink of its family in reach cannot yield a flow, in any
language under any dataflow model, so its taint request is never issued. The
count is recorded as requests_pruned -- exact, so not a degradation -- and a
test asserts that every other pair's request still goes out.
637-file plugin, 500-region budget
tier_counts {0: 2161, 1: 60, 2: 20, 3: 211, 4: 48}
pruned 2,161 of 2,500 = 86% evidence pass 94.5s
taint 339 requests -> 2,400s TIMEOUT
Gate met and exceeded; the scan still does not finish, and the reason corrects
phase 0. "1,148 ms per request" was an average over a 250-request set that was
88% tier 0 and answered instantly; the ~30 real requests in it cost ~9.5s each.
Pruning removes exactly the cheap questions and keeps the expensive ones, so it
cannot make them faster. The honest cost of a taint question that CAN have an
answer is 7-10s at callDepth=3, and that is what phase 2 and 5.13 design
against -- not 1.1s, and not 6.6s either. Both specs corrected.
869 tests pass.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
flow-aware-ranking phase 0: measure before changing anything.
THE BASELINE, committed as benchmarks/ranking/baselines/2026-09-10-access-rank.json
from benchmarks/ranking/position.py, which sorts regions exactly as the scan does:
pmpro 4,463 regions 83.8% at 0.30 CVE at 390 budget_at_recall 391
mwwpform 983 regions 81.7% at 0.30 CVE at 512 budget_at_recall 513
wpstatistics 1,673 regions 75.3% at 0.30 CVE at 503 budget_at_recall 504
vampi 25 regions BOLA at 15, 10; SQLI reached transitively
That corrects the brief's own cross-assessment: "concentration, not inclusion"
was measured on pmpro alone. On the other two plugins the CVE sits OUTSIDE a
500-region budget and is never examined; the slices found those CVEs only
because a slice has too few regions to exclude anything.
THE INSTRUMENT: `evidenceOnly` mode on the taint query -- what is in scope,
which sink calls, what shape, whether a sanitizer wraps THEIR argument, whether
a source is local or reachable -- returning before reachableByFlows is ever
called. model/evidence.py turns its rows into a vector per (region, family)
and a tier; scan.py records every pair's tier on the result and acts on none of
it. Every later phase is measured against a record made by scans that did not
know they were being measured.
2,500 pairs in 98.9s incl. JVM load, ~40 ms per pair
tier 0 -- no sink of that family in reach: 2,161 of 2,500 = 86%
Two earlier attempts ran past an hour each with no payload, and both were my
fault: repository-wide walks repeated per request -- the sink scan, the
reachable-method BFS, familyInRepo. Memoised across the batch.
AND THE SAME FIX REACHES THE DATAFLOW MODE. sinkCalls and reachableMethods do
not depend on the family, and were recomputed for all 2,500 requests. The
identical 250-request set with both joins on:
2,026.3s -> 287.0s 8,105 -> 1,148 ms per request same 5,779 rows
Most of what 5.13 measured as "the base taint query" was never dataflow. A 7x
speedup with nothing about the analysis changed; both corrections recorded in
the specs. Combined with 86% tier 0, the whole-repository taint query that
timed out at 40 minutes is ~339 real requests at ~1.1s: about six minutes.
Also: the module audit manifest is regenerated (91 load-bearing, 7 standalone,
0 orphaned) because evidence.py is now imported by the scan, and the
failed-query test is order-independent since the evidence pass records its own
failure before taint's.
869 tests pass; query compile test 4 passed; gates byte-identical.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Read against today's measurements the brief had two errors and one gap. They
are corrected in place and kept on the record, because the corrections are
themselves results.
Phase 1's gate was wrong. Pruning removes 53% of requests, and 1,177 x 6.63s is
still 2.2 hours -- it halves the problem and does not finish the scan. Saying
otherwise would send someone looking for a bug in a phase that did exactly what
it can.
Ranking's job is concentration, not inclusion. CVE-2023-23488's region is at
position 390, inside the budget, examined. It goes unreported because 500
regions are unaffordable at 6.63s per family per region, not because it was
left out. The metric is therefore budget_at_recall -- the smallest K holding
every pinned CVE -- and phase 2's gate is that K below 50.
The gap: the heuristics were named, not defined, and an undefined heuristic
cannot be checked for language leakage. Now defined as an evidence vector over
(region, family) pairs whose every field is stated in terms of fact-table
categories -- source, sink, sanitizer, dispatch, access -- so a heuristic that
mentions `$wpdb` has by construction left the vector.
The architectural decision that makes it agnostic rather than merely
neutral-in-intent: the vector is a CPG query WITHOUT dataflow, a sibling of
taint.sc that never calls reachableByFlows. Tiering is the CPG query without
dataflow; arbitration is the query with it. Same graph, same facts, two costs.
The graph normalises syntax across frontends, so "this argument is a literal"
means the same thing in PHP, Python and C.
Five tiers as predicates over the vector. Only tier 0 excludes, and its
predicate has a proof: a family with no sink in reach cannot produce a flow in
any language under any dataflow model. Its scope must match the arbiter's
callDepth or it is not exact and must not be called tier 0. Everything above
orders, where being wrong costs position, which regions_truncated reports.
Where the model helps is narrower than "ranking": ambiguous cleansing (esc_sql
two statements earlier into a variable), unfamiliar sink shapes as an ordering
opinion never a fact, and within-tier ties. Where it must not be allowed:
computing tier 0, seeing the question "is this vulnerable", or seeing the
arbiter's own verdict on the same run.
Six metrics so every phase can fail, and budget_at_recall is the one that
decides the release question: below 50 and the model is a refinement, above
and the model is the plan.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
A taint request costs 6.63s, a 637-file plugin yields 4,463 regions, and 83.8%
of them share one rank ordered alphabetically. The ranker is not weak, it is
looking at the wrong thing.
The evidence that this is tractable, from one plugin:
getMemberOrderByCode (holds CVE-2023-23488) rank 0.30
$wpdb->get_var("... WHERE code = '" . $code . "' LIMIT 1")
getMemberOrderByID (sibling) rank 0.30
$wpdb->get_row("... WHERE id = '$id' LIMIT 1")
pmpro_rest_api_get_membership_level rank 0.80
no SQL sink at all
A region with no sink outranks two holding raw concatenated SQL. The signal is
in the sink call's own text -- interpolated versus bound -- and nothing in the
ranker reads it.
And the detail that decides whether a keyword rule suffices: `esc_sql` IS in
scope for the CVE's function, applied to a different statement. "A sanitizer is
present, rank it lower" demotes exactly the right answer. Knowing which call it
wraps is reading, not matching -- which is the case for a model rather than a
table.
Unit: (region, family) PAIRS, tiered by cheap evidence. Tier 0 (no sink of that
family in scope) is the only tier that excludes rather than orders, so it alone
must be exact; above it, being wrong costs position rather than the finding.
That bound is what makes a model safe here -- a ranker cannot manufacture a
finding.
Six phases, each shipping only if it beats the previous one on a repository it
was not developed against: baseline, exact pruning (the phase that makes a scan
finish), heuristic tiers, the LLM ranker, breaking the closed loop, then prompt
optimisation. Phase 2 carries an explicit possibility that the model is not
needed for tiering -- and knowing that is worth more than assuming it. Phase 5
comes last because an optimiser run against a censored metric optimises the
censorship.
Records what the prompt must carry, and that the question is never "is this
vulnerable" -- that is the arbiter's job and the one thing a ranker must not be
asked.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Feeds the day's measurements back into the spec and reorders the roadmap
around what is actually blocking.
STATUS. Three CVEs found on real WordPress plugins -- CVE-2023-23488,
CVE-2023-6559, CVE-2022-25148 -- each entailed by the graph with no model in
the loop and each absent from its fixed side. Two of them require the
two-stage join and would be invisible without it. The detector is not the
problem.
The repository is. A 637-file plugin builds in 76s to a 4.25MB graph,
dominance and config answer, taint does not finish. Three blockers in the
order they bite: cost (6.63 s/request), ranking (84% of regions share one rank,
ordered alphabetically), coverage (access_control never asked for PHP,
output_encoding untrustworthy while the sanitizer list is flat).
And what is NOT a blocker, contrary to most of what this file said this
morning: php2cpg. One genuine defect, 0.2% of files, detected and excluded and
reported. Everything else attributed to the frontend was a local php shim
proxying stdio, which cost four wrong diagnoses in a day.
NEW TASK 5.14, and it is the cheapest thing on the board. 53% of taint
requests are put to a family whose sinks are not in scope -- all 487 of PMPro's
deserialization requests go to a repository that never calls unserialize.
Skipping them cannot lose a finding, because a family with no sink cannot
produce a flow. 4.6 hours -> 2.2, before any ranking work.
ROADMAP reordered around one question: the shortest path to a scan of a real
plugin that reports its CVE. Prune the impossible questions, then rank (an LLM
job, and safe as one because ranking cannot manufacture a finding, with the
closed-loop problem treated as a design requirement rather than a caveat), then
attack the per-request cost only if those two are not enough.
857 tests pass.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Every build allocated a temp directory for the graph, its request files and
joern's own working copy of the CPG. Nothing ever reclaimed it. A day of
scanning left 1,055 directories and 355MB in /tmp on a disk already at 95%,
and the runs that leaked hardest were the ones that FAILED -- which is exactly
backwards, since a failed scan has the least to show for the space it took.
Two mechanisms, because they cover different deaths:
* `CpgResult.cleanup` disposes of the tree, and `scan_repository` calls it
when the graph has answered everything it is going to. Never raises: a scan
that found things must not fail while tidying up.
* A sweep at build time removes trees older than six hours. A process killed
mid-scan never runs its cleanup -- which is how most of the 1,055 died --
so the disposal alone is not enough. Only directories under the system temp
root carrying this module's own prefix are touched, and a scan running in
another process right now is left alone.
Note for the record what was NOT a product leak: 6.3GB of /tmp/joern-cli-*
was three copies of the Joern CLI I made by hand during today's container
experiments, and 63MB of workspace/ in the repo came from running joern from
the repo root instead of a scratch directory. The product already runs joern
with cwd set to the scratch tree, so its workspace is contained and now
disposed with it. Both are cleaned up; neither was the code's fault, and
conflating my litter with its leak would have hidden the real one.
Disk went from 6.1GB free to 19GB.
857 tests pass.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
DeepSeek announced V4.1 Flash on 2026-09-10 and routes V4 Pro requests to it
from 12:00 Beijing time on 2026-09-14, billed at Flash's price.
The consequence worth writing down is not the id change. It is that the
detector and the judge are now the SAME model, where the design assumed the
judge was a stronger and more expensive second opinion. Every claim in this
repo that a judge call is a higher-quality check is now a claim about the
PROMPT and the evidence put to it, not about model class -- and the residual
question a judge is asked ("does this sanitizer suffice for this weakness")
has to carry its own weight.
Legacy ids stay in the price table so a recorded run from before the migration
still costs out; a manifest that cannot price its own model is one nobody can
audit.
The code for this landed in 83ab533, swept in by `git add -A` under a docs
message about the ranker -- the second time today. This commit is the record
it should have had.
855 tests pass.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Three corrections and one design requirement, from measurement and from
maintainer direction.
MEASURED. 4,463 PMPro regions carry five distinct ranks, and 3,742 of them --
83.8% -- share rank 0.30, ordered alphabetically by path. That tier spans
positions 159..3,900, and 341 of the 500 examined regions come from it: two
thirds of the budget spent on an arbitrary alphabetical slice. It is not a
weak ranker, it is barely a ranker.
CORRECTED. CVE-2023-23488's region is at position 390 of 4,463 -- INSIDE the
budget and examined. I stated repeatedly that it ranked ~1036 and was never
reached; that number came from before the array-callable and method-recogniser
fixes moved it. The reason it goes unreported is the taint query timing out,
so no region got a verdict at all. Ranking is a real problem; it is not this
miss, and conflating them would have sent the fix to the wrong place.
DIRECTION. Ranking is the LLM's job, not a tie-breaker for a residue. The
judgement "is this worth looking at" is a judgement about unfamiliar code.
Facts and generic features stay as cheap evidence for the model to read and as
the fallback when none is configured.
AND THE LOOP IS CLOSED, which is worse than train-on-test and was the sharper
point. A region the ranker excludes is never arbitrated, so it never produces
a label, so the ranker never learns it was wrong -- the feedback is censored by
the policy being trained, and every run confirms the ordering it already had.
Optimising on that makes it more confident, not more correct.
Breaking the loop is therefore a design requirement, with two cheap mechanisms
recorded: an exploration slice (~10% of budget sampled from outside the top-K,
the only labels the policy did not choose) and full labelling on repositories
small enough to arbitrate every region, which is policy-independent ground
truth. Inverse-propensity weighting is noted as available if the ranking is
made stochastic, and as unnecessary otherwise.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
`joern` and `joern-parse` are shell scripts that launch a JVM as a separate
process. `subprocess.run(timeout=)` kills the script and leaves the JVM
resident, holding its heap, indefinitely. Every `query_failed: timeout` leaked
about a gigabyte.
Found by running out of memory. After a day of 300s and 2,400s taint timeouts,
four orphaned JVMs -- three hours old, ~2GB between them -- were still resident
on a 7GB machine, and background tasks started being killed. On a CI runner
doing several scans in sequence this is an OOM rather than a degradation, and
it would present as "the scan needs more memory", which is the wrong lesson
entirely: the scan's own peak is a flat ~1.5GB whether its query runs for five
minutes or forty.
The command now starts in its own session, and a timeout SIGTERMs the process
group and SIGKILLs what survives. An injected test runner is left alone,
because those spawn nothing.
Tested with a real shell script that backgrounds a child, since the bug is
specifically about grandchildren -- and the test reaps that child itself if the
assertion fails, rather than leaking the process it is complaining about.
855 tests pass.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Four configurations over one verified graph (637 files, 7,061 methods, census
confirmed BEFORE anything was timed) and one 250-request set:
both joins off 1657.5s 6,630 ms/request 5779 rows
field half only 1623.0s 6,492 ms/request 5779 rows
hook half only 1781.0s 7,124 ms/request 5779 rows
both on 2026.3s 8,105 ms/request 5779 rows
The base query is the cost. The field half is free -- inside noise, measured
slightly faster -- the hook half adds 7.5%, both add 22%. Cutting them buys a
fifth of the time and costs two of the three CVEs.
Corrects a number I published earlier: ">0.94s per request" assumed taint
completed within its 2,400s ceiling. It timed out, so that was a floor dressed
as an estimate. The real figure is seven times higher.
2,500 requests is ~4.6 hours; all 4,463 regions would be ~41. Repository-scale
PHP is not slow, it is the wrong shape of question asked thousands of times.
Caveat kept with the numbers: all four configurations return an identical 5,779
rows because these are the TOP 50 regions -- REST handlers where framework
sources already exist and the joins add nothing. The joins earn their keep
lower down (`_delete_files` at rank 0.30). This measures their cost faithfully
and says nothing about their value; the two must not be conflated when deciding
what to cut.
Levers, none of them the arbiter: fewer and better regions (5.12, ~10x on its
own), fewer families per region (five are asked of every region regardless of
whether its scope holds a sink of that family), and callDepth -- three levels
over 7,061 methods is what makes a request cost seconds.
Also notes that commit 7308570 carried the OPENULTRASAST_CPG_QUERY_TIMEOUT
change under a docs message; `git add -A` swept it in and the message does not
say so.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
