mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-16 03:49:26 +00:00
e41d6dd7eac5fa614f7a35db6e66dda0af946753
272
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Marius CiepluchandClaude Opus 5
|
e41d6dd7ea |
docs(contributor-scan): 5.11 -- a detection strategy, not a verdict of impossible
"Structurally impossible" is a verdict, and a verdict is not a plan. Replacing
both structural-miss tasks with the one mechanism that covers them, because
they are the same shape: two halves of a path joined by a key that is a
literal in the source text.
hook: $_REQUEST -> ... -> return of set_current_page
key: 'wp_statistics_current_page' literal at both ends
return of apply_filters(key) -> ... -> $wpdb->get_row
field: $_POST -> ... -> assignment to $this->attachments
key: attachments a field name at both ends
read of $this->attachments -> ... -> unlink
Neither half is missing from the graph. Only the join is. So this is not a
request for a frontend to draw an edge it cannot draw -- it is a second
question asked with the query that already exists:
stage 1 the existing taint question against a synthetic endpoint: does
untrusted data reach a callback's return, or an assignment to
$this->F?
table built by reading literals, never inferring: add_filter('H', c)
against apply_filters('H', ...); $this->F = against $this->F
stage 2 does the apply_filters return, or the field read, reach a sink?
join on the key, with the RUNG carrying the join's uncertainty -- one
registered callback and a literal key entails; several callbacks or
a computed hook name corroborates, and the witness says which
"Any hook may reach any handler" is refused absolutely: a complete graph over
the plugin would entail everything.
No new dependency, no CPG mutation, no IR -- both stages ride the batch that
already runs, which is the point of expressing it as a query-level join.
It also recovers the precision fix 5.8 had to refuse. With the field half
working, _delete_files() is reached through $this->attachments rather than
through $this being tainted wholesale, so the receiver can then leave
parameter sources and take PMPro's 11 receiver-sourced findings with it.
5.8's numbers are the before-side of that measurement, and both observables
are pairs already pinned.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
|
||
|
Marius CiepluchandClaude Opus 5
|
2ba1aaf649 |
docs(contributor-scan): scope 5.11 to the hook registry, having checked its neighbours
"String-named callable" sounds like one gap and is at least two, so the
neighbouring forms were measured rather than assumed. All three came back
sound, and none of them is what breaks WP Statistics:
* taint SURVIVES a string-named callable -- array_map('trim', $_GET['a'])
and call_user_func('trim', $_GET['a']) both reach the sink and are
entailed, exactly like a plain call;
* a sanitizer named by a string IS recognised -- array_map('esc_sql', ...)
suppresses, which is the form PMPro actually writes;
* the textual sanitizer test does not over-suppress. A query where a SIBLING
value is esc_sql-wrapped is still entailed for the unwrapped one, and a
decoy 'esc_sql' literal flowing to the same sink does not suppress either,
because flow.elements are the nodes on that value's own path.
I expected the second and third to be defects and they are not. Recording the
negative result so the next reader does not spend the same hour on it.
The hook edge is different in kind, which is why it stays its own task:
`add_filter` and `apply_filters` are not one expression carrying a callback,
they are two statements in two files joined only through a hashtable inside
WordPress core -- code that is not even in the scanned tree.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
|
||
|
Marius CiepluchandClaude Opus 5
|
4adf5ecff5 |
docs(zero-with-a-reason): brief -- the same bug seven times
Seven separate defects in this project have produced "0 findings" for reasons
that had nothing to do with the code being scanned: an unreachable batch, an
argv cap, a CPG that threw on load, a budget that halted unpaid work, a regex
that missed `protected function`, a frontend that dropped every file and
exited 0, and a Joern log line inside the fenced payload.
Every one was fixed at its own seam and every fix was correct. Not one could
have been found by the tool, because the pipeline reports the last number in
the chain and discards every number before it -- so all seven look identical
from outside.
The rung ladder cannot help by construction: it labels verdicts, and the
failure is that no verdict was produced.
Proposes three independent mechanisms and one refactor:
* the funnel -- files discovered, files parsed, methods, regions, questions,
sink sites, flows, verdicts, findings -- with the rule that a zero at one
stage is legitimate only if the previous stage is non-zero. Each of the
seven bugs is a different row of that table.
* a text floor: count the fact tables' sink names in the raw source, and
degrade when the graph contains fewer. Two independent instruments, the
dumber one bounding the clever one.
* a canary flow that must come back entailed or the run is invalid -- with
the caveat that on PHP, adding a file can itself trigger the php2cpg race,
so it belongs in `ousast doctor` rather than in-band.
* an Answer union so "no rows" and "could not ask" cannot share a value.
And what it must not become: a confidence score, a retry loop, anomaly
detection over historical counts, or more code than it saves.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
|
||
|
Marius CiepluchandClaude Opus 5
|
435c14f3ec |
docs(contributor-scan): 5.8 -- WordPress class by class, including what it cannot ask
Three plugins, each pinned at a version whose advisory was read out of the
code, each measured against its own fixed commit:
unauth injection, sink in the receiving function pmpro FOUND
arbitrary file operation mwwpform FOUND
unauth injection across a WordPress hook wpstatistics MISSED
missing capability check -- cannot be asked
unescaped output -- cannot be trusted
Both hits are entailed by the graph alone -- no judge, no spend -- and both
disappear on the fixed side, because both fixes are calls the fact tables
carry: `$wpdb->prepare` and `realpath()`. The engine is not recognising a bug;
it is failing to find a cleansing call on a path it resolved, then finding one.
Precision is stated rather than flattered. PMPro's vulnerable side reports 15
entailed, 4 from a named parameter and 11 from the receiver, and one of those
survives into 2.9.8 at a query whose every fragment is `esc_sql`-wrapped.
The miss is the useful half, and it is checkable rather than inferred. WP
Statistics' CVE has a modelled source, a modelled sink and `esc_sql` as its
fix, and reports nothing on either side, because the file holding the sink has
zero superglobals, makes no call into the class holding the source, and is
joined to it only by
add_filter('wp_statistics_current_page', array($this, 'set_current_page'));
return apply_filters('wp_statistics_current_page', $current_page);
Both ends name the callback with a string, so no frontend can draw the edge.
Most plugin data travels this way. What the engine reports there instead is two
other sites, the same class at the wrong lines, identical on both sides -- and
not noise: upstream annotates both `// phpcs:ignore WordPress.DB.PreparedSQL`
and later rewrote one with `$wpdb->prepare`.
Two classes could not be asked at all, which is not the same as a miss.
`dominance_specs(language="php")` is empty, so missing-capability is never
asked; and `output_encoding` shares the one flat sanitizer list, which holds
`esc_sql` and `intval` but not `esc_html` -- wrong in both directions, so no
XSS verdict from this engine currently means anything.
5.10's root cause is recorded too: php2cpg's parser bridge, a race rather than
a scale limit, and the mitigation is per-file exclusion. Adds 5.11 (the hook
edge, with CVE-2022-25148 as its observable) and 5.12 (taint a field where it
is assigned, with CVE-2023-6559 as its guard).
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
|
||
|
Marius CiepluchandClaude Opus 5
|
45cf43e625 |
fix(cpg): one method with no sources must not cost the whole batch its answers
`reachableByFlows` on an empty source list is not a no-op. Joern answers it by
logging
[INFO ] Attempting to determine flows from empty list of sources.
to STDOUT -- while the script is running, so the line lands between the fence
markers, ahead of the payload. `json.loads` then fails on the whole document
and the driver reports `query_failed` for every request in the batch. One
method with nothing to trace from silently cost the answers for all 205 of
the others.
Fixed at both ends, because either alone leaves the seam brittle:
* taint.sc skips the flow query when there is no source to trace from, which
is also the correct thing to do with the question.
* `extract_payload` starts a decode at each line that opens a document and
keeps the first that parses. Trimming to the outermost brackets is the
obvious approach and the wrong one -- `[INFO ]` brings its own bracket.
Found while testing whether `$this` belongs in parameter sources. It does not
belong there on the argument -- the receiver is the object, not request data --
and excluding it cuts Paid Memberships Pro's order class from 16 entailed
findings to 6, all sourced from a named parameter, and makes the 2.9.7/2.9.8
pair separate cleanly.
It also loses CVE-2023-6559: MW WP Form's `_delete_files()` takes no
parameters at all, and the attacker-controlled path arrives as
`$this->attachments`, assigned by another method. So the trade is a VERIFIED
CVE against findings merely suspected of being false, and it is refused. The
receiver stays, with the measurement written next to it. The real defect is
field-sensitivity -- a field should be tainted where it is assigned -- and
deleting the receiver removes the symptom by removing the evidence.
841 tests pass; gates byte-identical.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
|
||
|
Marius CiepluchandClaude Opus 5
|
e0442d146c |
fix(cpg): a graph the frontend emptied must not read as a clean repository
Scanning a two-file WordPress slice returned 64 regions examined, zero
findings and zero degradations. The graph it examined had no methods in it.
php2cpg 4.0.623 feeds several files to one `php-parse` process and reads its
stdout and stderr merged. php-parse writes a `====> File <next>:` banner to
stderr the moment it starts the next file, while the previous file's JSON --
megabytes of it -- is still draining from a block-buffered stdout. The banner
lands inside the JSON, ujson reports
expected json value or } got "=" at index 5011645
and the frontend drops that file. Reproducible on real plugin code, and the
size that matters is one file's AST dump, not the repository's:
class.memberorder.php alone (1834 lines) -> 142KB CPG, no failures
the same file + a two-line file -> 5.7KB CPG, BOTH dropped
One oversized file empties the whole graph. `joern-parse` then does not
propagate any of the frontend's warnings: 13 lines of output, exit status 0,
"Successfully wrote graph". This is what made a 637-file plugin unscannable
and reported it as clean -- it was never a scale limit.
Two fixes, because the failure arrives on two different paths:
* `_unparsed_files` reads the per-file warnings the frontend does emit, and
the scan reports them as `files_unparsed`. This covers the direct-frontend
path where the warnings are visible.
* The batched query now carries a `__census__` of the graph -- method and
file counts -- under a key no request id can collide with, and a scan over
a graph with no methods degrades as `cpg_empty`. This covers joern-parse,
which shows nothing. It rides the batch rather than costing its own
invocation, because avoiding a JVM start is the entire reason batching
exists.
Neither invents a verdict. There is no rung for "the graph was empty" because
the ladder labels verdicts and none was produced, which is exactly why it has
to be a degradation.
pmpro two-file slice before: 0 findings, degradations=[]
after: 0 findings, degradations=[cpg_empty]
Gates byte-identical; 840 tests pass.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
|
||
|
Marius CiepluchandClaude Opus 5
|
37a6594703 |
fix(mapping): a PHP method with a visibility modifier is a region
`^\s*function` matched a bare declaration and nothing else, so every method
written the way modern PHP writes them -- `public function`, `protected
function`, `private static function` -- was invisible to the region model.
The effect is not subtle on class-based plugins. MW WP Form's
`class.mail.php` produced ONE file-level region spanning 272 lines instead
of 16 method-scoped ones, and its `protected function _delete_files()` --
which holds CVE-2023-6559 -- was not a region at all. Paid Memberships Pro
happened to work only because that particular file omits the modifiers.
class.mail.php 1 file-level region -> 16 method-scoped regions
Also adds the MW WP Form checkout as the PATH family's plugin, pinned at
3bdd33d (5.0.3) with CVE-2023-6559 verified in the code:
protected function _delete_files() {
foreach ( $this->attachments as $file ) {
if ( file_exists( $file ) ) {
unlink( $file );
and the fix at 02ebc67 replacing the existence check with `realpath()`,
`is_file()` and a directory-confinement test. `unlink` is a modelled path
sink and `realpath` a modelled sanitizer, so this is a pair the engine
should separate rather than merely flag -- which is what makes it worth
more than a single hit.
Gates byte-identical.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
|
||
|
Marius CiepluchandClaude Opus 5
|
f13934c663 |
docs(overview): record the CVE, and what scoping is actually for
The scoreboard now carries a real finding: CVE-2023-23488 at class.memberorder.php:936, the exact line, in unmodified plugin code, with 16 of 26 unprepared $wpdb queries reported in that file and zero of the 26 using ->prepare. And the reading the maintainer's instinct produced: **the first real CVE was found by cutting, not by scanning more.** Two of 637 files load in 9 seconds and report the bug; the full checkout does not load at all. That two-file slice IS the contributor case -- a developer editing class.memberorder.php gets precisely that scan -- so scoping is not a workaround for a scale problem, it is the thesis, and the scale problem is what happens when the thesis is ignored. The distinction recorded is not whether to cut but who decides and whether the decision survives: a build declaration, a diff or a subtree is chosen by the project or the developer and is recoverable from the artifact, while a model judging "this looks like boilerplate" gives the same answer when right and tells you nothing when it drops the file holding the bug. Every failure found on this checkout was of that second shape. Also corrects "one language is demonstrated" to two, with the honest limit attached: PHP works on a real plugin slice and not at repository scale. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
96e7476c6d |
feat: the first real CVE, found in production plugin code
CVE-2023-23488 is detected at
classes/class.memberorder.php:936:getMemberOrderByCode -- the exact line
verified in the source, in unmodified Paid Memberships Pro 2.9.7.
A two-file slice carrying both ends of the CVE, 2885 lines, loads and
queries in 122 seconds and reports 16 entailed injection findings.
**The other fifteen are not noise.** The file holds 26 `$wpdb` query calls
and **zero** of them use `->prepare`; all 26 interpolate or concatenate a
variable. The engine reports 16 of 26, so it under-reports rather than
over-reports, and every sampled finding is a genuine unprepared query --
`get_row("... WHERE id = '$id'")`, `get_var("... WHERE ID = '" .
$this->user_id . "'")`. CVE-2023-23488 is simply the instance that got an
identifier, because it is reachable unauthenticated; the rest are the same
defect with different reachability.
That answers the question that prompted this: whether the system can find
one real CVE. It can, in production code, at the exact line, in two
minutes, for a cent. What it could not do was survive its own plumbing at
637 files -- four separate quiet failures on this one checkout, all between
the scan and the arbiter, none of them in the analysis.
What this does NOT settle is recorded with it: the full checkout still does
not load and that boundary is unmeasured; whether php2cpg resolves
`new X()` and `$this->method()` is still unknown, since this slice reached
the sink through a same-function flow rather than across the object; and
one CVE on one plugin is not the class-by-class table over several plugins
that 5.8 asks for.
On cutting the input with a model, proposed when size looked like the
blocker: the evidence argues for scoping rather than triage. Two of 637
files found the CVE, task 2.10 already scopes by what a project declares it
ships, and diff-scoping is this spec's thesis. A model choosing what to cut
adds a fifth quiet-failure surface -- drop the file holding the bug and the
scan reports clean with nothing recording why, which is the shape of all
four failures found today. The reverse direction is where it fits: ranking
which regions get the budget, not choosing what to remove.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
|
||
|
Marius CiepluchandClaude Opus 5
|
21fdb554aa |
fix(cpg): a query that could not be asked is not an empty answer
contributor-scan 5.7, and it was the wrong question.
**The 117k-line pmpro CPG does not load.** Importing it throws in a
ForkJoinParallelCpgPassWithAccumulator during overlay application, so every
query against it returns nothing. It is not a call-graph question at all.
Isolated by running the ARBITER against two prebuilt CPGs rather than
writing a sixth ad-hoc Scala probe, four of which had already failed on my
own mistakes:
small wpdb CPG get_order_by_code -> 1 model_entailed
pmpro CPG getMemberOrderByCode -> 0
Both have their source and sink in ONE function, both were asked with
parameter sources and depth 3, same arbiter and same spec. The CPG is the
only difference.
That clears the facts, which is worth stating: `$wpdb` sinks work, the
parameter-source rule works, and a PHP parameter reaching `$wpdb->get_var`
through a concat is entailed. Whether php2cpg resolves `new X()` and
`$this->method()` is still unmeasured -- it cannot be measured on a CPG
that will not load -- and is recorded as 5.10 along with finding the size
boundary.
**And the defect underneath it, which is the fourth quiet failure found on
this one checkout.** `query_batch` returned `{}` for a failure and `{}` for
a genuine empty answer, so a caller could not tell "could not ask" from
"nothing found". Every query on this scan failed at load and it reported
500 regions examined, no findings, and no degradation: a vulnerable plugin,
clean.
A failed batch now returns None, unparseable output counts as a failure
rather than an empty answer, and the driver records a `query_failed`
degradation naming the kind and the request count.
This is the failure mode the rung ladder structurally cannot catch. Every
other honesty mechanism here labels a VERDICT -- suspicion,
regions_unjudged, regions_unasked, stated family limits -- and when no
verdict is produced there is nothing to label.
Gates byte-identical.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
|
||
|
Marius CiepluchandClaude Opus 5
|
db273dde53 |
docs(specs): WordPress flaw classes, the corpus insight, and an IR brief
Three directions recorded, each grounded in what this session measured. **WordPress by flaw class, not by plugin** (tasks 5.8, 5.9). One plugin is a checkout; a class is a claim. WordPress's recurring shapes are narrow and each maps to a family that already exists -- unauthenticated wp_ajax_nopriv reaching $wpdb, a missing capability check, unescaped output, arbitrary file operations. Two of the four are blocked on gaps this session found: access control cannot be asked at all because PHP has no obligation facts (5.9), and unescaped output is blocked on the flat sanitizer list (5.6). The observable is a class-by-class table across several plugins saying found, missed, or out of scope -- a single found CVE is not the artifact worth teaching from. **Why a corpus of benchmarks is not enough on its own**, written into overview.md as the finding it is rather than an opinion. Twelve engine defects in one session against three real repositories, and the three gates stayed byte-identical through every fix. A gate that does not move is a corpus that could not see the bug. The reason is structural: a pair is a labelled function with a declared family and a known answer, so it cannot exercise which region to look at, what to spend, whether a capability is reachable, whether two entry points share a sink, whether the question fits in an argv slot, or whether a flow crosses a module. Three of the twelve were quiet failures where the tool reported clean and every honesty counter agreed -- which a benchmark score cannot express, because the score is computed from the verdicts that exist. The corpus is the holdout that stops per-repository tuning, and the wrong instrument for finding out whether the tool works. **ir-consolidation** (brief, unapproved) separates two questions the IR suggestion contains, which have opposite answers. The SOURCE IR is already carried rather than about to be added: semantic/ is 3218 lines, 16% of the codebase, measured at 2.2% entailment against the CPG's 41.3% and still running as a proposer -- and Vine and P-Code are binary IRs, so the work there is subtraction. The MISSING abstraction is C arithmetic, and there GhidraPAL's three-valued-logic abstract interpreter over Ghidra P-Code is exactly the right shape: "can this index exceed this allocation" is a value question, which is what a bit-level TVL domain answers and what every libpng overflow needs. Vine is dormant since 2009 and is not the candidate. The constraint is recorded as an ordering: question 1 lands before question 2 starts, or adopting an IR makes the growth problem worse. And the cost that matters is not lines but the number of things that can silently return nothing -- this session found three such in a single PHP scan, and a binary pipeline adds a build, a lifter and a domain, each able to produce an empty result that looks like a clean one. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
8579c10e7c |
docs: the WordPress checkout scans, and does not find its CVE
Reporting the result rather than the intention. Paid Memberships Pro 2.9.7 with CVE-2023-23488 is pinned, verified in the code, and scans cleanly: 637 files, 117k lines, 1577 regions, build 47s, query 74s, 140s total, no degradations. **The CVE is not detected.** Both ends are inside the budget -- the REST handler at position 290 as a recognised entry point, the `$wpdb->get_var` sink at 472 -- and the taint query runs for 23 seconds. So the gap is the middle: the flow crosses `new MemberOrder( $code )` into a constructor and then `$this->getMemberOrderByCode( $code )`, both dynamic in PHP. Whether php2cpg resolves either into a callee edge is UNVERIFIED; two attempts to query the call graph failed on script errors and were abandoned rather than guessed at. Recorded as 5.7, with the answer required before anything is concluded about the fact tables. Three engine defects had to be fixed before the scan could produce even this result, and each had previously reported the plugin clean for a different reason: 1577 regions, 0 scanned joern-parse failed where php2cpg succeeded 500 scanned, taint 0.05s a 1.49MB batch over Linux's 128KB argv cap 500 scanned, taint 21.7s the entry point was not recognised as one The middle one is the one to remember. Every honesty mechanism in this codebase labels a verdict -- suspicion, regions_unjudged, regions_unasked, stated family limits -- and there no verdict was produced to label, so a vulnerable plugin was reported clean with every counter agreeing. What is established: the WordPress engine path works end to end. What is not: detection on a real plugin. One checkout that does not find its CVE is not yet evidence either way about the facts, and 5.7 has to answer whether the call graph can carry the flow at all. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
9020318a83 |
fix(mapping): an array callable registers a handler too
`array( $this, 'method' )` is the normal way a class-based WordPress plugin registers a callback, and the recogniser only matched the bare string form. The cost was precise. Paid Memberships Pro registers every REST route that way, so `pmpro_rest_api_get_order` -- the entry point that receives CVE-2023-23488's tainted `code` parameter -- was classified as a plain function at rank 0.30, position 1036 of 1577, below the 500-region budget. It was never asked about. Meanwhile `getMemberOrderByCode`, the sink two calls away, WAS scanned at position 450 and found nothing, because its `$code` is an ordinary parameter and only an entry point's parameters are treated as untrusted. So the scan examined the sink, missed the source, and reported the plugin clean. Recognising the array callable puts the handler at position 290 with rank 0.80 and `source=entry_point`, which is what gives it the interprocedural question that task 2.4 built for exactly this shape. Gates byte-identical. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
92677136d7 |
fix(cpg): a batch goes through a file, not the command line
Scanning a 637-file WordPress plugin reported 500 regions examined, zero
findings, and a taint query time of **0.05 seconds** -- less than a JVM
takes to start. Dominance ran for 29.56s and config for 22.44s in the same
scan, so only taint had failed.
Linux caps a SINGLE argument at MAX_ARG_STRLEN (128KB) whatever ARG_MAX
says. Measured on that plugin:
taint 2500 requests 1.49 MB <- over the cap
dominance 13 requests 0.04 MB
config 13 requests 0.00 MB
As `--param requests=<json>` execve returned E2BIG, `_run` caught the
OSError, `query_batch` returned {} -- and {} reads as "no rows found". Only
taint crossed the cap, which is exactly why the failure looked like a
result rather than an error.
This is the quiet failure the rung ladder cannot catch. Every other honesty
mechanism in this codebase labels a verdict; here no verdict was produced
to label, and a scan of a vulnerable plugin reported it clean.
The payload is now staged to a file beside the CPG and the path is passed
instead; all three queries read `requestsFile` when it is set and keep
`requests` for the single-request forms and the tests. A test asserts no
argument approaches the cap, since the failure mode is invisible at the
scale the unit tests run at -- batching worked on every small tree it was
ever tried on.
Verified on vampi first: 4 entailed, unchanged, all three known
vulnerabilities still found.
Gates byte-identical.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
|
||
|
Marius CiepluchandClaude Opus 5
|
9c50e17c11 |
fix(cpg): retry a failed build through the language frontend
Scanning a 637-file WordPress plugin reported 1577 regions with 0 scanned and `cpg_build_failed`. The build was not slow or out of memory -- it failed in 42 seconds -- and `php2cpg` run directly on the same tree produced a 4.3MB CPG with no errors at all. The difference is that `joern-parse` detects the language and then shells out to the frontend in a separate process. On this tree it got as far as applying overlays and then threw, while still writing an unusable cpg.bin. Joern's own output recommends the direct route for a large codebase: "Invoking CPG generator in a separate process... If you are importing a large codebase (and/or running into memory issues), please try: invoke the frontend: php2cpg -J-Xmx3072m /tmp/src -o cpg.bin" So a failed `joern-parse` now retries through `<language>2cpg`, with the driver passing the dominant region language. A language with no mapping fails closed exactly as before -- no guessing at a frontend name. This is the documented fallback rather than a workaround, and it was diagnosed rather than assumed: the standalone build was run first, and only after it succeeded on the identical tree was the retry added. Gates byte-identical. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
922ed84053 |
feat(repos): a WordPress plugin with a verified CVE, and the WordPress DB API
contributor-scan 5.1's detection half. Paid Memberships Pro 2.9.7 pinned at
1a900be3, carrying CVE-2023-23488 -- unauthenticated SQL injection through
the `code` parameter of the /pmpro/v1/order REST route.
**Verified in the code, not taken from the advisory.** At this commit
classes/class.memberorder.php:936 reads
$id = $wpdb->get_var("SELECT id FROM $wpdb->pmpro_membership_orders
WHERE code = '" . $code . "' LIMIT 1");
and tag 2.9.8 replaces it with `$wpdb->prepare( "... code = %s ...", $code )`.
637 PHP files and 117k lines, so it is also the first PHP checkout large
enough to say anything about an envelope.
**WordPress never calls mysqli; it goes through $wpdb**, so the facts could
not have found this. Added qualified: `$wpdb->get_var`, `->get_results`,
`->get_row`, `->get_col`, `->query`, `->replace`. Qualified because a bare
`get_var` or `query` matches any method of that name anywhere -- the
generic-name trap that made `query.get` match every `.get(` in a repository
this morning. `->insert`, `->update` and `->delete` are deliberately absent:
they take arrays and WordPress prepares them itself.
**And the thing that nearly went wrong.** The obvious next step when
writing PHP facts is to add WordPress's sanitizers -- sanitize_text_field,
esc_html, esc_attr. Doing so would have made this CVE invisible, because
its handler calls `sanitize_text_field( $params['code'] )` and then
concatenates the result into a query. sanitize_text_field is a real
sanitizer that does nothing whatsoever about SQL injection.
`TaintSpec.sanitizers` is one flat list per language whose comment states
the assumption outright -- "a sanitizer that breaks one flow breaks it
whatever the sink's family" -- and PHP falsifies it. So only SQL-specific
cleansers are listed, which leaves XSS under-sanitized instead, and the
real fix is recorded as task 5.6: a discharge counts for the obligation it
discharges. That is task 2.6's finding arriving in the taint family, where
the dominance arbiter already reads kind from the facts and the taint spec
has no equivalent.
The end-to-end scan of this checkout is not yet reported: the first run
returned 1577 regions with 0 scanned, and the CPG builds standalone at
4.3MB with no errors, so the two do not yet agree. Numbers when they do.
Gates byte-identical.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
|
||
|
Marius CiepluchandClaude Opus 5
|
47b00d739b |
feat(model): one defect is one finding, however many regions reach it
contributor-scan 2.9. Task 2.4 let a region's question follow the call graph, and the moment it did, many entry points could reach one shared sink: libpng reported the same site twenty-six times. A contributor shown the same defect twenty-six times learns to scroll past it. Findings are deduplicated on (site, family), keeping the strongest rung and counting the regions that reached it -- "reached from 26 regions" is the useful half of that observation and one row is the readable half. Two families at one site stay two findings, because `echo $_GET[...]` is an injection question and an output-encoding question with different answers. The duplication that motivated this was removed by another route: the 26 rows were all the `fgets`-matched-as-`gets` false positive, and the sink matcher fix took them out. So this changes no current number and is a structural guarantee rather than a measured reduction -- the mechanism that produced them is untouched, and only the finding that made it visible is gone. +25 lines for a guarantee, in a session that has been asked to watch its growth. The audit is unchanged at 97 modules, 90 load-bearing and 7 standalone, with no orphans. Gates byte-identical. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
60d2a1a92c |
feat(model): an endpoint its contract declares open carries no obligation
contributor-scan 2.8. VAmPI's last two false positives were register_user
and login_user, both declared open by its OpenAPI spec: the arbiter was
reporting a missing guard on endpoints whose contract says there is
nothing to miss.
**The fix keys on provenance, not on the value, and that distinction is
the whole task.** The mapper says "public" in two very different
situations. An OpenAPI operation with no `security` block is DECLARED
open, because the spec is a complete contract. A Flask handler with no
`@login_required` is "public" only because nothing was found -- which is
precisely the bug this family exists to report. Suppressing on the value
alone would have silenced the detector on its primary case, so
EntryPointRecord now carries `access_declared` and only a declaration
removes the obligation.
WordPress supplied the second source of declared access on the same day --
wp_ajax_nopriv_ for a logged-out handler, wp_ajax_ for a logged-in one,
permission_callback => __return_true for an open REST route. A rule resting
on one framework's convention would be a special case; two make it a shape.
vampi entailed 6 -> 4
access-control precision 2 of 4 -> 2 of 2
known vulnerabilities 3 of 3 still entailed
access_control offered to 6 of 14 handlers, exactly the six the spec
declares authenticated
**And the guard that decides whether it is a real improvement:** the
access-control pair corpus, which nobody tuned this against, is unmoved --
pair_correct 3 and leak 1 across all nine, pair_correct 2 and leak 0.0%
across the seven from projects unrelated to VAmPI, identical to task 2.6.
The precision was not bought by silencing the family. Pair excerpts carry
no route manifest and no hook, so nothing in them is declared open and the
rule correctly does not fire.
Gates byte-identical.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
|
||
|
Marius CiepluchandClaude Opus 5
|
3e1d13e4b6 |
feat(reports): the model's own suspicion band is a proposal too
contributor-scan 2.17, completing Req 5.5. Task 2.11 made static pattern matches proportionate and left the model layer's own suspicions alone, reasoning that a proposer had reasoned about them. That does not survive inspection. A model suspicion is what the enumerator proposed at a site the graph could NOT decide, affirmed by a judge with no coverage there -- and its own rationale says exactly that: "the model resolved no flow here; its sink table may not cover this API". It is a pattern match with an opinion attached, and it reads like a reasoned claim only because a model produced it. vampi 60 findings, 61 sections -> 26, report 1085 lines -> 446 libpng 198 findings, 39 sections -> 30 **What it costs, which is the number that decides whether the rule is right.** For each known vulnerability the recipe declares, take every finding row at its location and ask whether any survives the rule. All three of vampi's -- the SQL injection and both documented BOLAs -- are still found at model_entailed and all three survive. Nothing known is hidden. Three vulnerabilities in one repository is also a statement about how small the evidence base is, and the artifact says so. The obligations checker sits at `suspicion` too and is NOT grouped: its output comes from a structural analysis, an obligated operation with no discharging guard, rather than from a guess. An earlier attempt at this rule filtered on rung alone and would have collapsed it; six tests failed and they were right to. "Pattern matches" is now "Proposed, not corroborated", since it holds two shapes. Model rows group by FAMILY rather than by rule, because their rule is always `model`. Gates byte-identical. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
af7e33ef3e |
feat(reports): say what was looked for, and what cannot be decided
contributor-scan 2.16, implementing the approved Req 5.6. `c/memory` scored zero true positives across 89k lines of C and three distinct classes of false one, for a structural reason: it models string CALLS and libpng overflows through arithmetic. The task allowed retiring the family. Kept, scoped and stated instead -- `strcpy` misuse is real in plenty of C and the family earns its place for that. Retiring it would have removed the disclosure along with the coverage, leaving a C scan silent for a reason nobody could read. **Its own description was the quiet-failure risk.** It claimed "bounds, lifetime, arithmetic", which is exactly what it does not decide. It now names the APIs it models, and a new `limits` field in families.toml says what it cannot: an overflow expressed as an index, a pointer offset, or a memcpy whose size is computed -- which is how most real C overflows are written, and how every one of libpng's is. A limit recorded in a fact table nobody reads is no better than silence, so the report renders it. **What was analysed** lists every family offered to a region whether or not it found anything, because a family that ran and reported nothing is the case a reader is most likely to misread, and it states plainly that silence means no fact table covered the code rather than that the code is safe. The counts are derived from the fact tables the arbiter actually used, so they cannot drift from what the engine looked for; only the limit sentence is authored, because "cannot decide an overflow expressed as an index" is a statement about an abstraction and no table can produce it. A C scan that finds nothing now says: | c | `memory` | 121 | 9 | **`memory` (c)** — Models CALLS, not arithmetic... Gates byte-identical. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
f488e625c2 |
fix(specs): the php2cpg "blocker" was the shim, not the frontend
Task 5.5 recorded that php2cpg emits a CPG containing only one PHP file --
the last alphabetically -- and called it a blocker for the WordPress
target. Rerun with a native php-cli, as asked, and it is not true.
php 8.3.6, no shim, same joern 4.0.623, same two files:
files: plugin.php, vuln.php
methods: mp_fetch_report, mp_save_report, mp_render_box, mp_helper,
get_user, safe_user, show, run
Both files, all eight methods, and no PHP-Parser banner warning at all --
the banner that looked like the thread to pull was a symptom of the same
thing and does not occur natively.
The cause was the workaround. This machine has no PHP interpreter, so the
shim ran each `php` invocation in its own container; php2cpg drives
PHP-Parser once per file, and that per-invocation isolation kept only one
file's JSON.
**The lesson is worth more than the fix.** Two independent reproductions
agreed, and both were measured through the same broken instrument. A
consistent result is not a correct one when every measurement shares an
apparatus -- the same trap as a corpus that cannot see the defect it is
scoring, which is this session's other recurring finding.
And with the instrument fixed, 5.4 is demonstrably done. A two-file PHP
tree yields 10 function-scoped regions and 7 entailed findings in 15
seconds: SQL injection in both WordPress handlers, XSS in the shortcode
renderer, and all three of the original probe's. safe_user is correctly
absent, its mysqli_real_escape_string dropping it to corroborated. The
wp_ajax_nopriv_ handler ranks 1.0 and the plain helper 0.3.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
|
||
|
Marius CiepluchandClaude Opus 5
|
0ffa99b5e6 |
feat(model): one finding per sink site, and WordPress's registration model
contributor-scan 5.4. Both halves, and the second found a frontend blocker.
**One verdict per region was right for a pair and wrong for a file.** A
pair is one labelled function with one bug, so one answer is correct. A
region spanning a PHP file is not: the probe's SQL injection on line 5 and
command injection on line 13 produced ONE injection verdict, `system` won
it on flow length, and the SQL injection went unreported by construction
rather than by any failure of the analysis.
`taint.verdicts()` now returns one verdict per distinct sink site,
strongest first, and its first element is exactly what `verdict()`
returns -- so the pair path and every committed measurement resting on it
are unchanged, and the gates are byte-identical. Both entry points share
`_usable_flows` so neither can drift from the other.
php probe 2 of 3 -> 3 of 3
vampi 6 entailed, unchanged, now with lines on access-control
findings from task 2.7 (users.py:187, books.py:51 -- the two
documented BOLAs at their exact lines)
libpng 1 -> 2 entailed; the second was hidden by the collapse.
Neither is validated and both are in contrib/.
**WordPress registers handlers rather than decorating them, and the hook
name is the declaration.** `wp_ajax_nopriv_*` is WordPress saying
logged-out callers reach this, so it is public; `wp_ajax_*` fires only for
logged-in ones; `permission_callback => __return_true` declares a REST
route open. That is the declared-provenance signal task 2.8 needs,
arriving from a second framework. Plus every top-level PHP function as its
own region, without which a file is one region however many functions it
holds: 1 file-level region became 10 function-scoped ones.
**And a blocker, recorded as 5.5.** php2cpg emits a CPG containing only
ONE PHP file, the last alphabetically. Reproduced twice on independent
samples; both files parse cleanly under `php -l` and under PHP-Parser
directly, and joern-parse reports success either way. php2cpg logs "Found
warning in PHP-Parser JSON output" naming the survivor, and that warning is
PHP-Parser's own `====> File X:` banner appearing in what php2cpg expects
to be pure JSON.
Not isolated yet: this machine has no PHP interpreter, so php2cpg was
driven through a container shim. The banner comes from PHP-Parser rather
than the shim, which makes a shim-only cause unlikely but not excluded, and
rerunning with the native php-cli in the shipped image is the first step.
It blocks WordPress outright -- a plugin is many files, and a scan that
silently covers one of them and reports success is the quiet failure Req
5.6 was approved for.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
|
||
|
Marius CiepluchandClaude Opus 5
|
61811512a8 |
feat(facts): PHP semantic facts, validated against a real PHP CPG
contributor-scan 5.1, the first step toward the v0.1 WordPress target. Written against a PHP CPG rather than against PHP source, because the two do not look alike. Verified with joern-parse 4.0.623 / php2cpg on a probe: $_GET["name"] -> call `<operator>.indexAccess`, CODE $_GET["name"] mysqli_query($conn, $sql) -> call NAMED mysqli_query echo $_GET["msg"] -> call NAMED echo So sources match on code and sinks match on name, and every entry the probe did not exercise is marked unverified in the file -- `include` and `require` among them, which is PHP's most characteristic path bug and the first thing to check on a real checkout. Five families derive (injection, output_encoding, path, deserialization, untrusted_destination) with no sink that is its own sanitizer -- asserted by a test, because that bug class has now appeared three times in three matchers. `php` is added to the region model's languages, without which the facts would have reached nothing. The arbiter decides 4 of 4 on the probe: the SQL concat entails, its `mysqli_real_escape_string` twin drops to corroborated, `echo` XSS entails and `system` command injection entails. **A whole scan finds 2 of those 3, and the gap is the finding.** PHP files yield FILE-LEVEL regions because no PHP entry points are mapped, and the taint arbiter returns one verdict per region per family -- so a file-level region reports at most one injection finding however many it holds. `system` won on flow length and the SQL injection went unreported. That is the mirror image of task 2.9: that one is many regions collapsing to one site, this is many sites collapsing to one verdict, and it bites hardest exactly where PHP needs it not to, since a WordPress file is large and has no decorator for the mapper to find. Recorded as task 5.4 with WordPress's registration model (add_action, add_filter, admin-ajax, REST routes), which the mapper already handles in principle for other frameworks. A 13-line probe is not a WordPress result. What is established is that the seam works end to end for PHP; everything about scale and coverage is unmeasured, and there are still zero PHP pairs in a corpus of 263 python, 351 c_cpp, 47 javascript and 49 java. Gates byte-identical. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
b5d3cd6ab6 |
docs(roadmap): a v0.1 release definition, ordered toward libpng and WordPress
The named release targets are, today, exactly the two cases that do not work -- and they fail for different reasons, which is what makes them a good pair rather than one. WordPress fails because `taint_specs(language="php")` returns zero families: Joern's php2cpg parses it and we say nothing. That is the shortest path from here to a named target, and the case this arbiter suits best, because PHP web bugs are the classic taint families. libpng fails because c/memory models string functions while libpng overflows through arithmetic, and its library never calls a sink we model. That needs either a second abstraction -- size arithmetic, a constraint problem -- or an honest stated limit. So "ready for" means two different things and the criteria say which applies where. Where a family can decide the class, find the known vulnerability. Where it cannot, run inside the envelope, report nothing false, and STATE the limit. The second is not a consolation prize: a tool that says "I do not model this class" is usable, and one that silently finds nothing is what teaches developers to distrust SAST in the first place. Seven v0.1 criteria, each tied to a requirement that already exists: unattended runs inside a stated envelope; zero known-false entailed findings on every pinned checkout; the known vulnerability found wherever a family can decide it (Req 4.6); every family's limit stated in the report (Req 5.6); a proportionate report (Req 5.5); installs without the user having Joern, a JVM or PHP; and a committed regression baseline per target. That reorders the middle of the roadmap. PHP moves up, and the c/memory decision moves up with it, because criterion 4 needs that family either earning its place or being scoped -- and scoping it is cheap. The vibe-code repository stays valuable as the guard against tuning everything on vampi, but drops below PHP because it is not a named target. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
9a65c998b4 |
docs(steering): an overview of what is true, and a roadmap ordered by one test
Fourteen specs have accumulated and the roadmap still described a pre-Joern world -- "adjudication is still Python-ast plus a tree-sitter CLI stub", "Joern-as-required deferred" -- which is stale enough to mislead someone reading it to decide what to do next. It is kept as roadmap-2026-09-04.md rather than deleted. **overview.md** is the synthesis above the specs: what the tool is, how one pass works, what has been measured, and what is not true yet. Every number cites a committed artifact. The thesis is stated plainly -- the differentiator is not the dataflow engine, since CodeQL and Semgrep have one and developers do not turn it on, but the rung: a claim you can act on without writing a query, and an explicit admission when the engine could not decide. It says what does not work as clearly as what does. The C story is structurally broken, not undertuned. One language is demonstrated. PHP, Ruby and Go have zero families, so Joern would parse them and the engine would say nothing. The evidence base is two repositories and one has never produced a true positive. Control dependence is unavailable in both CPGs. execution_confirmed has no production caller. And it records the lesson that shaped the current specs: nine defects found by pointing the tool at two real repositories, three of them the same bug in three matchers, with the three gates byte-identical through every fix -- so a corpus of function-level pairs could not have caught any of them. **roadmap.md** is ordered by the standing test, that the tool must find bugs or it is SAST noise. Which gives the ordering principle: make the output worth reading, then make it find more. A better arbiter behind an unreadable report changes nothing, and a second language before the report is fixed doubles the noise. It also says what is NOT being worked and why -- authorization-obligations and reachability-flow-model were largely delivered by other routes and should be reconciled or retired rather than carried as approved-not-started, which misrepresents both. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
290ecff9de |
docs(finding-feedback-loop): requirements approved, design generated
The design turns on an asymmetry the session measured: two of the three matcher bugs needed no judgement at all -- once the first was understood, the others were the same fix carried across. So the automation that pays is propagation and memory, and the model is confined to the one step that needs inference: naming what a group of counter-examples has in common. Two prohibitions are enforced by tests rather than by convention. No module under feedback/ may import a writer for semantic/facts, semantic/obligations or ruleset/. The boundary between evolving the QUESTION and authoring the FACTS is the whole design, and a comment saying so is not a boundary. And `evaluation_set(recipe, dismissals, *, split)` has no "all" -- no argument returns both splits, and no code path returns holdout rows during optimisation. The missing argument IS the design: the 2026-09-05 leak was possible because reading the holdout was merely discouraged, and five of eleven candidate shapes were taught by the pairs they then recovered. A dismissal is recorded AT a rung. "Not a defect at suspicion" must not silence the same site when the graph later entails it -- that is a promotion and deserves a fresh look, which is also what keeps a dismissal file from becoming a graveyard. Risk recorded rather than glossed: the evidence base is two repositories, one of which has never produced a true positive. The ledger and the dismissals are useful immediately; the evolution should not be trusted until contributor-scan 2.15 or 5.2 lands a checkout that can produce one. Phase: design, generated, not approved. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
bccc89be52 |
docs(specs): approve the group-2 amendments, generate finding-feedback-loop requirements
**Amendments approved and folded into the numbered requirements** rather than left as a proposal appended to the file: - Req 5.5 — a report proportionate to the evidence behind a finding. A rule whose only evidence is a text match is summarised past a stated number of sites; a finding a proposer reasoned about keeps its detail whatever its rung; nothing leaves findings.json, which is the record a baseline is compared against. - Req 5.6 — a family that cannot decide a class states the limit, and no scan presents an absence of findings as evidence of absence. - Req 4.6 — at least one pinned checkout carries a vulnerability a family can decide in principle, so a scan producing nothing is distinguishable from an engine that cannot see. Each carries the evidence that produced it, and the amendment record keeps what group 2 confirmed rather than changed: Req 8's overfitting warning earned its place twice, Req 3.3's honesty counters made an invisible failure visible, and Req 11.2's gates held through all nine defect fixes. Req 5.5 is half implemented. Task 2.11 made pattern matches proportionate; task 2.17 now carries the model layer's own suspicion band, which is 38 of vampi's 60 findings and is an enumerated candidate the graph could NOT decide with a judge's opinion attached -- a pattern match with an opinion, not a reasoned claim, and weaker evidence than the obligations checker's structural output. **finding-feedback-loop requirements generated**, seven of them: a dismissal file the developer owns and the tool never writes, rejecting one without a reason because a suppression list without reasons is a graveyard nobody can audit; a counter-example ledger whose entries are runnable regression cases, grouped by shared cause so a recurring bug is visible as one; a model that proposes a cause into a review queue and decides nothing, validated by reverting the three known matcher fixes and measuring what it recovers; dismissals plus a recipe's known vulnerabilities as one split evaluation set with the holdout unread and asserted; evolution scoped to the judge and residual questions with the fact tables off limits to any automated means; an adoption that reports what it traded and reverts itself when net-negative; and the loop inheriting the corpus blind spot, so its evidence base is repositories and it states how few it has. Phase: requirements, generated, not approved. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
72932ed879 |
docs(specs): group 2 feedback as amendments, and lift out what was never specified
Following the Kiro workflow rather than continuing to write tasks. **A workflow error, corrected.** Two task groups -- a counter-example ledger, and per-repository evolutionary prompting so developers can commit dismissals and evolve the harness -- were appended to contributor-scan's tasks.md on 2026-09-09 without passing requirements or design. They are not refinements of scan integration; they are a different feature, and writing them as tasks skipped the two phases that decide whether a feature is shaped right. They are lifted to .kiro/specs/finding-feedback-loop/, at brief, unapproved. That brief states the boundary the design turns on: an optimiser may evolve the QUESTION (the judge and residual prompts, which are how we ask) and never the FACTS (tables, rulesets, the declared policy file, which are what we claim). The closed-loop leak measured on 2026-09-05 is why -- five of eleven candidate shapes were taught by the holdout pairs they then recovered, reporting a -0.059 holdout Youden as +0.059. It also states the acceptance test for the model's role: revert the three matcher fixes and see whether the grouping rediscovers the known causes. **Three requirements amendments, drafted and unapproved**, at the end of contributor-scan/requirements.md: - Req 5 should state what a report may repeat. libpng emitted 198 findings and nothing in the spec forbade it. Task 2.11 fixed pattern noise; the model layer's own suspicion band is still 38 of vampi's 60, and it is the same epistemic class -- an enumerated candidate the graph could not decide and an LLM affirmed. - A family that cannot decide a class must say so. c/memory scored zero true positives on 89k lines of C for a structural reason, and a silent scan reads as a clean bill of health. Worse than noise because it is quiet, and the one failure mode the rung ladder does not already guard. - Req 4.1 is satisfied by a checkout that cannot demonstrate detection at all -- libpng's CVE is out of family and no version of it can be otherwise. It should require one that can. And what group 2 confirmed rather than changed: Req 8's overfitting warning stopped a rule being extended until one repository was clean and forced the access-control work onto pairs nobody tuned against; Req 3.3's honesty counters are what made a budget stopping free work visible at all; and Req 11.2's byte-identical gates held through all nine defect fixes, which says from the other side that the corpus could not have caught any of them. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
00e178d69e |
feat(reports): a report proportionate to what the engine can say
contributor-scan 2.11. libpng emitted 198 sections, 163 of them asserting that `memcpy` and `strcpy` are PRESENT in a library that uses them correctly throughout. That is the noise developers have learned to skip in every other tool, and shipping it costs the findings beside it their credibility. **Filtering on rung was the obvious rule and it was wrong.** The obligations checker emits at `suspicion` too, so hiding that rung would have hidden an entire detector -- six tests failed and they were right to. The discriminator is evidence_level: on libpng 163 of 198 rows are `static_corroboration`, on vampi 3 of 60 are, and the model layer and the obligations checker carry `suspicion` there instead. That is exactly the distinction the ladder exists to make -- an undecided CLAIM about a site is not an observation that an API exists. So a pattern-only rule is grouped once it has more than three sites, and keeps its full sections under that. One `python-flask-debug` is a finding a contributor can act on and must not be summarised away; fifty-four `c-unsafe-copy` are a property of the codebase. libpng 198 findings, 198 sections -> 39, 161 grouped into three lines vampi 60 findings, 61 sections -> 61, nothing grouped Counted, never dropped: findings.json stays complete because it is the record a baseline is compared against, and the grouped section says the model reached none of these and that its silence means no fact table covered the site rather than that the site is safe. What it does not fix, recorded rather than glossed: vampi still emits 61 sections for a 520-line application, 51 of them reasoned claims this rule does not touch. It fixes pattern noise, not volume. Also records what group 2 established about the three corpus asks, so none is started on a wrong assumption. Joern ships php2cpg but our fact tables have zero PHP families, so WordPress would parse and produce nothing -- fact tables first, corpus second. An older vulnerable libpng will not help for a reason no version can fix. Vibe-code repositories are the best fit and the cheapest route to a second true-positive checkout, because Python and JavaScript are the two languages whose fact tables are complete. And Group 8, for per-repository state a developer owns: a committed dismissal file the tool reads and never writes, dismissals as a split evaluation set, and GEPA/SIMBA-style evolution scoped to the QUESTION -- the judge and residual prompts -- never to the fact tables. That boundary is the whole design: a prompt is how we ask, a fact table is what we claim, and the closed-loop leak measured on 2026-09-05 is why the distinction is not negotiable. Gates byte-identical. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
d232f03597 |
feat(mcp): the findings list carries the rung, and leads with what was decided
The stated final objective is the rung plus an LLM way of diving in without being forced to rethink the code. The second half is largely plumbing, because the material already exists: the witness (`username -> db.session.execute(...) in models/user_model.py (line 73, 8 steps)`) is the reasoning, already written down and already attached to the finding. So this session's recurring shape was worth checking rather than assuming, and it was there. `openultrasast.get_finding` returns the whole record including the witness, but `openultrasast.findings` dropped the rung: an LLM listing libpng's 198 rows could not tell the one the graph decided from the 197 an enumerator proposed without fetching all 198 one at a time. That is exactly the re-derivation the list exists to spare it. The list now carries the rung, orders strongest first, and returns a per-rung tally so a caller can triage before it fetches anything. The brief records what still stands between here and the objective: the witness is one line and the query already walks the path elements before discarding all but the first; the suspicion band will be reasoned about at length by an LLM, which is worse than a human ignoring it; and a family that cannot decide its class must say so, or the model will confidently explain a silence. Gates byte-identical. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
20edd9354b |
docs(contributor-scan): what the evidence says, and what it changes
Spec feedback after group 2, recorded in the brief where the goal is stated rather than in a task where it would be lost. The standing test is that the tool must find bugs or it is SAST noise, and that most developers already have dataflow in CodeQL and Semgrep and never use it. That settles what the differentiator is: not the engine, which they already have, but the RUNG -- a claim the graph decided, reported without configuration. Everything below model_entailed is on trial. The scoreboard, kept honest: vampi finds 3 of 3 known vulnerabilities at model_entailed, including a second BOLA the engine found before it was in the recipe, with 2 false positives that are endpoints the OpenAPI spec declares public. libpng produced 198 rows and zero valid ones, for a structural reason rather than a tuning one -- the library never calls a sink the c/memory family models. Nine defects this session, each found by a measurement and none by review; three were one bug class in three matchers, two were a capability present but unreachable. The teachable finding is not any one of them, it is that a corpus of function-level pairs cannot see any of them, and every one appeared within minutes of pointing the tool at a real repository. The gates stayed byte-identical through all nine, which says the same thing from the other side. Two priorities change. 2.11, the suspicion band, becomes the critical path rather than cleanup: libpng's 163 "API is present" rows are exactly the noise the quote is about, and no arbiter accuracy survives shipping them. And a family that cannot decide its class has to say so, because a silent C scan reading as a clean bill of health is worse than noise -- it is quiet. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
2589c3d47a |
docs(contributor-scan): what c/memory covers, and what SMT covers instead
libpng's library code never calls strcpy, strcat, sprintf or gets -- all twelve files that do are under contrib/. It overflows through array and pointer arithmetic and memcpy sizing, which is a different question from the one this family asks: "can this expression exceed this allocation" rather than "does tainted data reach this call". That is a constraint problem. It is what SMT is for, and what this project deliberately did not adopt -- the architecture decision was abstract interpretation as arbiter, taken because a model layer can be built from code where execution cannot be assumed. That decision was right for the flow families and it does not reach this one. The maintainer has attacked libpng exactly this way, using SMT to lay out memory and then guiding a fuzzer to the inputs that reach it, which is evidence about the shape of the problem rather than a plan to reimplement it: the honest routes to C memory safety are constraint solving or execution, and the ladder already has a rung for the second. So 2.15 now separates the cheap route (pin a C project whose CVE runs through a string function, giving the family its first true positive) from the expensive one (model the arithmetic, a different engine), instead of implying a sink-table fix would cover both. And 2.16 asks the question this spec's predecessor would have asked: c/memory has scored zero true positives and three separate classes of false one -- a substring match, a bound it could not see, an allocation sized to fit. Every one was a real engine defect and every one is fixed, but the family has never been right. Keep it scoped and labelled, or retire it; either way with a number behind it, and never leaving a silent C scan to read as a clean bill of health. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
c1191b3a6b |
feat(model): an access-control finding names a line, and what the PDG does not give us
contributor-scan 2.7. Taint and config verdicts have always carried a `path:line:function` location; dominance never did, so every access-control finding reached a contributor as `api_views/users.py:?:update_password` -- the right file by accident, because the region was asked about that file, and no line at all. The query now reports the operation's file and the verdict carries the location, with an engine that cannot report one getting no location rather than an invented one. Two questions answered by measurement while doing it. **Does this factor in the PDG?** The data half, centrally: `reachableByFlows` is the entire taint arbiter and `joern-parse` applies the `dataflowOss` overlay. The control half, not at all -- and it is not available to us. Measured: 0 of 13,184 calls in the libpng CPG and 0 of 1,152 in the vampi CPG have a controlling control structure, so `controlledBy` returns nothing in either. Everything guard-shaped therefore runs on CFG dominance or on syntactic conditions, which is exactly why 2.12's first attempt failed: libpng's bound reaches its sink through an `error` flag, and that is a control-dependence fact and nothing else. Recorded as 2.14, including the possibility that it cannot be enabled -- which would bound what guard reasoning this engine can ever do. **Would checking out a vulnerable libpng find a known CVE?** No, and not because of the version. Every one of the twelve files using `strcpy`/`strcat`/`sprintf`/`gets` is under `contrib/`; the library itself uses none of them. libpng overflows through array and pointer arithmetic and `memcpy` sizing, and five of its buffer-overflow fixes touch no modelled sink at all. Pinning an older commit would produce the same silence for the same reason. That leaves two routes of very different size, recorded as 2.15: pin a C project whose CVE genuinely runs through a string function, which is cheap and would give the memory family its first true positive; or model how C actually overflows, which is size arithmetic rather than sink matching and should not be started by accident. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
9221ba8e40 |
feat(model): a bound on the data lowers the rung, it does not clear it
contributor-scan 2.12. c/injection and c/memory carried zero sanitizers, so no C flow could ever be discharged: every `argv -> strcpy` entailed however carefully it was bounded. Which sinks can be discharged this way is read from the sink fact's own CWE rather than declared: CWE-121 is a stack buffer overflow, so gets/sprintf/strcat/strcpy qualify. It is the only overflow CWE in the fact tables and it is C-only, so no Python or JavaScript sink is touched and the pair corpus is unaffected -- gates byte-identical. Not a sanitizer list, because `strlen` appears in the guarded and the unguarded case alike; the discharge is a guard, not a call. **The first implementation required the bound to DOMINATE the sink, and it found nothing.** libpng guards `strcpy(outname+len, ".png")` with `(len = strlen(inname)) > 250`, but that check sits on an else-if chain whose sibling branch skips it entirely, and the protection reaches the sink through an `error` flag: `++error` in the guarded branch, `if (!error)` around the copy. Joern's dominator set for that sink has 89 calls and correctly excludes the bound; `controlledBy` returns none either. Verified against the CPG before the rule was changed rather than after it failed. So the bound is searched in the method, and the consequence is corroborated rather than silence. That is the honest rung: a bound on this variable means the graph has NOT shown the copy is unguarded, and it has not shown it is guarded either -- off-by-one being how bounds fail. The judge is asked. Proving the flag-mediated case needs path-sensitive reasoning this arbiter does not do. libpng entailed 3 -> 1 The survivor is also a false positive: `strcpy(output + len, ".icc")` in iccfrompng.c, where `output = malloc(len + 5)` makes it fit exactly. That is a different discharge shape -- allocated to size, not bounds-checked -- and it is recorded as 2.13 rather than chased. Both shapes were found on one repository, and extending a rule until that repository is clean is the overfitting Req 8 exists to prevent; 2.13 requires a second C checkout with a genuine CWE-121 so the change can be measured for what it silences as well as what it clears. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
6dc25d77e4 |
feat(model): let the project say what it ships, and rank that first
contributor-scan 2.10. Every one of libpng's twenty-five entry-point regions was a `main()` -- twenty-three under contrib/, plus example.c and pngtest.c -- and those regions carried the fifty interprocedural taint requests that dominated a 300s query phase. A library has no entry points, so entry-point-driven analysis had nothing to anchor to except its own example programs. The task said to derive this rather than hardcode `contrib/`, and a project already declares it. Makefile.am names installed targets in `lib_LTLIBRARIES` and `bin_PROGRAMS`, each target's `<target>_SOURCES` names its files, and `check_PROGRAMS`/`noinst_PROGRAMS` are automake's own word for "not installed" -- exactly the distinction wanted. Taking every `_SOURCES` would have called libpng's whole test suite shipped code, since contrib/libtests declares its sources the same way the library does. interprocedural regions 25 -> 1 taint query 300.1s -> 125.0s total query 311.9s -> 138.5s regions 121 -> 121, unchanged by design The surviving interprocedural region is contrib/tools/pngfix.c:main, which libpng genuinely installs. Two rules keep it honest. **Silence is not exclusion**: a project with no build file this can read gets None back and nothing changes, so vampi's numbers are untouched. **Not shipped is not unscanned**: those regions are still arbitrated function-locally, because a contributor editing a test must still get an answer -- what they lose is the expensive treatment and first claim on the budget. Entailed findings went 0 -> 3, and all three are false positives at contrib/gregbook/wpng.c:374, `strcpy(outname+len, ".png")`. It is safe: outname is char[256] and line 325 guards with `if ((len = strlen(inname)) > 250)`. They were not introduced so much as uncovered -- a different callDepth meant a different sink set, and the 26 fgets false positives were masking them. The structural cause is recorded as 2.12: c/injection and c/memory carry ZERO sanitizers, so no C flow can ever be discharged and every `argv -> strcpy` entails however carefully it is bounded. The discharge here is not a call but a guard that dominates the sink, which is precisely what dominance.sc already does for access control and is wired to that one family. Also adds Group 7, five tasks for stopping the same false positive twice. Two of today's three false-positive classes were one bug in three matchers -- `resolve` in `resolveUrl`, `user` in `users`, `gets` in `fgets` -- each fixed locally and each next one surviving because nobody carried the fix across. A model may group counter-examples by cause and draft an edit; only a split-respecting measurement may adopt one, because the predecessor's improve lever admitted shapes taught by the holdout pairs they then recovered and turned a -0.059 Youden into +0.059. Gates byte-identical. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
67497c4d68 |
fix(cpg): a sink is matched as a word, not as a substring
`fgets` contains `gets`. The sink matcher's last clause was an unbounded
`methodFullName.contains(n)`, so every bounded read in libpng matched the
sink for the unbounded one. All 26 entailed findings on that repository
were the same false positive:
*argv -> fgets(pnmline, 256, wpng_info.infile) (line 335, 8 steps)
which is the safe API being reported as the dangerous one -- and `fgets`
is in the fact tables as a SOURCE, never a sink. Verified against the CPG:
methodFullName is exactly "fgets", and .contains("gets") is true.
Third instance of one bug class in this codebase, after the sanitizer that
matched `resolve` inside `resolveUrl` and the discharger that matched
`user` inside `users`. The sink matcher had never been given the same
treatment. All four of its clauses now run through the shared bounded
matcher, and that matcher is the punctuation-tolerant version written for
task 2.6 -- so `input(`, which an unconditional lookahead had made
unmatchable in `input(x)`, works for the first time as well.
libpng entailed 26 -> 0
Zero is the right answer for a PNG library whose only known CVE is out of
family. Gates byte-identical, so no pair outcome depended on the substring
match.
What remains on that scan is recorded, not claimed: 113 `c-unsafe-*` rows
assert an API is present rather than misused, and 50 overlay-coverage rows
are annotations rather than findings. 82 of the 113 are under `contrib/`.
Task 2.11 covers whether that band is worth putting in front of a
contributor at all; 2.10 already covers not scanning example code.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
|
||
|
Marius CiepluchandClaude Opus 5
|
68b4e551f3 |
feat(model): the envelope checkout, and no to Joern server mode
contributor-scan 2.2 for libpng and the 2.3 decision it exists to inform. 89,082 lines of C across 93 files, under a 2GB heap cap: CPG build 16.1s (25s cold) Joern queries 250.0s arbitration 122.0s (105 calls, $0.0122) whole scan 446s peak RSS 911MB regions 121, all arbitrated, 32 unasked CVE-2025-66293 out_of_scope, as the recipe declares Memory is not the constraint a fleet runner would hit: 911MB leaves room for the rest of the job on a 2-vCPU/7GB runner. The memory pressure seen while building this was concurrent JVMs on one desktop, not repository size. **Server mode: no.** The query phase splits taint 243.3s / config 10.7s over two invocations, and fixed overhead measured on the same CPG is ~13s per invocation. So ~26s of the 250s is JVM startup and ~224s is CPGQL evaluation: a warm server removes 6% of the scan and none of the part that costs. The driver now records query seconds per kind, because a single total cannot tell startup from evaluation and the two answer this oppositely. The small repository pointed the other way. vampi's 34.3s over three kinds is roughly three startups and almost no evaluation, which made a warm JVM look worth having; the ratio inverts with size. That inversion is why the task was sequenced after a real repository rather than decided on a 520-line file. On whether one whole-repository CPG is the only shape for a horizontal fleet: no, and it is also not where the pressure is. The build parallelises and costs 16s; the query does not shard and costs 250s, so splitting a repository across runners divides the cheap phase -- and after 2.4 a shard boundary hides exactly the cross-module flows that matter. Scoping to a contributor's diff cuts the expensive phase instead. 911MB for 89k lines extrapolates to roughly 5GB at a million lines, which would exceed a standard runner, so the ceiling is real and further out than this checkout can locate. Two defects recorded rather than fixed, both invisible at vampi scale: 2.9, all 26 entailed findings are the same site reported 26 times, because regions that share a sink each emit their own; and 2.10, 22 of the 30 highest-ranked regions are libpng's `contrib/` example programs rather than the library. Also ignores Joern's `workspace/`, which a maintainer querying a CPG by hand from the repository root will create. Gates byte-identical. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
d15985f695 |
fix(model): a guard discharges the obligation it is a guard of
contributor-scan 2.6. Access control entailed 7 of 9 VAmPI handlers,
including three that do authorize and three the OpenAPI spec declares
public. Five separate defects, each invisible to a corpus of one-function
excerpts and each found by asking the query what it actually matched.
**A trailing segment is not a match.** `query.get` was matched by its last
segment, so every `.get(` in the repository became a protected database
operation -- `request.headers.get('Authorization')` and
`request_data.get('username')` included. Four of the seven false positives
came from this, and so did the apparent detection of the known BOLA: it was
entailed because fetching the Authorization header had no dominating guard,
not because of the defect. A qualified token now requires its qualifier.
**Configuration is not authorization.** dominance_specs took every
obligation fact. `security_setting` operations are discharged by a VALUE,
which config_specs already claims them for; pooled in here they made
`app.run(...)` an operation requiring an authorization guard and put
thirteen tokens in BOTH lists -- an obligation that is its own discharge,
the same shape as the taint sink that used to sanitize itself.
**Input validation is not authorization.** `jsonschema.validate` was
counted as a guard, so handlers with no authorization check at all read as
guarded. No operation in the table requires validated_input.
**A guard discharges the obligation it is a guard of.** The largest one.
`orm-read` requires an identity_constraint or an ownership_check;
`token_validator` is a path_guard. Flattened into one list, authenticating
the caller discharged an object-level obligation -- a handler that
authenticates and then looks the record up by a PATH parameter read as
fully guarded. The facts state that relation and the spec was discarding
it; it is now carried and the engine asks per operation.
**A word boundary needs a word.** Found while fixing the above. Guard
matching wrapped every token in `(?<![A-Za-z0-9_])...(?![A-Za-z0-9_])`.
The identity token `['sub']` begins with `[`, and the character before it
in `resp['sub']` is `p` -- so the assertion could never hold. Every
identity token starting with punctuation was unmatchable, and the identity
constraint that separates a correct lookup from an IDOR was invisible.
Also: dominance rows were repository-wide, so the model's unguarded
`delete_user` was attributed to the API handler of the same name. Rows are
scoped to the region's file, which is what the rung's own justification
says -- the evidence is a file contradicting itself.
Measured against VAmPI's OpenAPI `security` blocks, which declare per
operation whether auth is required, and agree with the code:
vampi precision 28.6% (2/7) -> 50.0% (2/4)
known found 3 of 3, all model_entailed
And on the labelled access-control pairs, which nobody tuned this against:
all 9 pairs pair_correct 3 -> 3, leak 44.4% -> 11.1%
7 non-VAmPI pair_correct 2 -> 2, leak 42.9% -> 0.0%
Every detection lost was on a pair whose FIXED side was flagged too, which
scores nothing. No correct detection was traded for the precision.
get_by_title was flagged before it was in the recipe: its code carries the
comment "# Broken Object Level Authorization" and the README lists it, so a
real finding was being scored as a false positive. It is now the recipe's
third known vulnerability.
Both remaining false positives are endpoints the spec declares public.
Recorded as 2.8, with the reason the mapper's access_level cannot simply be
trusted: for decorator frameworks "public" means "no decorator found",
which is the bug itself.
Gates byte-identical.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
|
||
|
Marius CiepluchandClaude Opus 5
|
0fcd5feade |
feat(model): a region for the code that is in no function
contributor-scan 2.5. VAmPI's `host='0.0.0.0'` is now reported exactly once, at app.py:17:<module>. Before task 2.2 it was reported eight times in eight files that do not contain it; after 2.2 it was reported nowhere. Two causes, one shape: nothing could ask about module-level code. **The main-guard marker is not a function.** `__main__` is what the mapper emits for `if __name__ == '__main__':`. Sent to the engine as a function name it matched nothing, because a CPG calls a file's top-level code `<module>` -- verified against the VAmPI CPG rather than assumed. The region model translates the marker, which is the one place that turns what the mapper meant into what the scanner asks. **A function-scoped file had no region for its module body.** Settings, route registration and globals live in no function, and every region's query filters by function name. A file-level region beside the handlers would report each handler's findings a second time, which is why the earlier rule dropped it as "a second full-file pass". A module-scope region PARTITIONS the file instead of overlapping it. That only became cheap after batching: before it, a region was a JVM. Two things this needed to be correct rather than merely present: - A module method spans its whole file, so the taint query's line-range containment made `<module>` the parent of every function in it. Under module scope, membership is direct -- otherwise the module region reports every handler's sinks again, which is the duplication the old rule was avoiding. - A module body is not a handler. The marker-derived region inherited the handler families and entailed a missing authorization check on the main guard: a finding with no possible remediation. Caught by the first repository run that had the region, and now covered by a test. bind address 8 wrong files -> 0 -> 1, at app.py:17:<module> regions 22 -> 25 entailed 8 -> 9 query time 37.4s -> 41.3s peak RSS 827MB -> 823MB cost $0.0067 -> $0.0068 Three more regions for four seconds and no more model calls: the budget is a total and was already spent, so regions_unasked rose 17 -> 20 -- those regions got the free pass and are reported as unasked, not as clean. `<module>` is Joern's Python name for a module body; JavaScript uses `:program`, so this is verified for Python only. Gates byte-identical. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
ca7b5ba370 |
feat(model): a region that spans the flow, not just a function
contributor-scan 2.4, closing the no-go task 2.2 recorded. VAmPI's SQL injection is a request parameter reaching `get_by_username` in api_views/users.py, passed to `User.get_user` in models/user_model.py, interpolated into a query string and executed there. Source and sink are in different functions and different modules. Regions were per-function, so neither region saw both ends and the engine was silent on the most ordinary shape a real injection bug has. `reachableByFlows` was never the limitation -- it follows calls already. The limitation was the QUESTION. Sinks were filtered to the labeled function's own body, so a crossing flow had no sink to end at. Three changes, each with a reason it is sound rather than convenient: **An entry point is the trust boundary.** parameter_sources was off for repository scans because a repository's parameters are not all attacker input. An entry point's are, by definition -- that is what makes it an entry point. The driver turns it on only for regions the mapper found and named, and leaves it off for walker fallbacks and for named functions that are not entry points. **Sinks may sit below the handler.** taint.sc admits sinks in methods reachable from the labeled method through the call graph, bounded at ENTRY_POINT_CALL_DEPTH = 3: handler -> service -> data access, which is where these bugs live. Unbounded would make the question about most of the repository rather than about this entry point, and make it unaffordable. **The finding names the sink, not the asker.** A crossing flow ends in another file and another function, so Verdict carries the evidence's own `path:line:function` and the site uses it. Reporting `models/user_model.py:73:get_by_username` would name a place that does not exist: line 73 is inside get_user. Also: labeled methods are now resolved by name AND file. `get_user` exists in more than one module of a real repository, which a corpus of one-file excerpts could never surface. Measured on the same checkout and budget as 2.2: VAmPI-SQLI missed -> model_entailed at models/user_model.py:73:get_user known found 0 of 2 before 2.2's fixes, 1 of 2 in 2.2, 2 of 2 now entailed 7 -> 8 query time 40.9s -> 37.4s peak RSS 869MB -> 827MB cost $0.0067, unchanged Asking the bigger question cost nothing measurable here: file scoping and call-graph following pull in opposite directions and roughly cancel at this size. That is one 520-line repository, and VAmPI is now the repository this change was developed against -- the exact overfitting shape Req 8 exists to catch. The depth bound and the entry-point rule mean nothing general until a checkout nobody tuned against says so. Gates byte-identical. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
177c7f3719 |
feat(model): first repository measurement, and the four things it found
contributor-scan 2.2. VAmPI at a pinned commit, scanned whole rather than as excerpts, with benchmarks/repos/measure.py recording build time, query time, arbitration time, peak memory, cost, unjudged regions, and whether each known vulnerability was found and at which rung. The corpus could not have found any of this. Every defect below needs a repository to become visible, and three of them were invisible behind a passing test suite. **The batch path was unreachable.** query_batch was implemented, tested and committed while CpgResult exposed only `run`, so the driver's capability check found nothing and fell back to one JVM per region-family: the first attempt at this measurement ran eighteen minutes on 520 lines, issuing `--param function=` one at a time. Every test called query_batch directly, so none of them noticed. CpgResult now carries run_batch and a test asserts the capability is reachable through the object the driver is handed. The earlier 275s->106s improvement attributed to batching came from region deduplication alone. **The budget stopped free work.** It bounds model calls, but was enforced by breaking the region loop -- spending a paid limit to stop unpaid CPG arbitration. 6 of 22 regions were examined and VAmPI's BOLA, which the graph entails for nothing, was never reached. It now withholds the judge only. 22 of 22 regions arbitrated, 17 counted as regions_unasked (silent because unaffordable is not silent because decided), and the BOLA is entailed by the model layer. **A region with no function asked about the whole repository.** Sending function="" matched every hit in the tree and the driver attributed that answer to whoever asked: one `host='0.0.0.0'` in app.py became eight identical entailed findings in eight files that do not contain it. Queries now carry the region's file. Entailed findings 14 -> 7, query time 70.9s -> 40.9s. The single-request form stays unscoped so the committed pair measurements remain reproducible. **The JVM had no heap bound.** Joern claimed a quarter of physical RAM and runs were killed under memory pressure at different points, so the numbers described the host rather than the engine. -J-Xmx2048m by default, OPENULTRASAST_CPG_HEAP_MB to raise it. Also: arbitrated findings carried `function:line` where the report layer read `path:line`, so every entailed finding reached a contributor with a function name where its file belonged; and the witness line was parsed by splitting on a comma, which produced sites like `config.py:17): permissive literal '0.0.0.0':?`. Reading: GO on one CPG per repository -- 6.2s to build, 177s end to end, 869MB, $0.0067, nothing infeasible. NO-GO on region-scoped taint as the only shape: VAmPI's SQL injection runs from a handler parameter into a function in another module, and a per-function region sees neither end. That is the ordinary shape of a real injection bug, so it is recorded as task 2.4 along with three smaller defects this run exposed. Gates byte-identical. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
8c387306be |
feat(repos): pinned known-vulnerable checkouts, offline by default
The pair corpus scores excerpts: a vulnerable function and its fixed twin, handed to the engine with the answer already narrowed down. Nothing in it exercises a repository -- region ranking, the budget, one CPG over thousands of files, or whether a verdict that holds on fifteen lines still holds when the source, the sink and the guard live in three modules. benchmarks/repos/<name>.toml names a real checkout: url, a full 40-char commit SHA, licence, and every known vulnerability with its file, function and line. `ousast repos` lists them; `--fetch` materialises one; CI does neither, and conftest clears the switch and redirects the cache so a developer with warm checkouts cannot read them by accident. Three rules the loader enforces rather than documents: - A branch or tag is rejected. Either would make the same recipe mean different code next month while the baseline it produced kept the old numbers. - `in_scope` must be stated per vulnerability, never defaulted. It decides whether "not found" is a miss or a non-question, which is the most important thing about the result. - A recipe that declares `detection` must name something in scope to detect. `ousast repos` does not trust a recipe either: against a materialised checkout it checks HEAD against the pin, that the licence file is there, and that each declared function appears above its declared line. That earned its place immediately -- the first libpng recipe named LICENSE.md, which is the filename at HEAD today and not at the pinned commit, where it is still LICENSE. Two checkouts, verified against the real repositories: - vampi (MIT, 520 lines) is the DETECTION checkout. Excerpts of it are already in the pair corpus, including the update_password miss diagnosed earlier, so a whole-repository run says directly whether a verdict survives the move off the excerpt. Its routes register through an OpenAPI operationId, so it exercises registration-based entry-point mapping rather than the decorator path. - libpng (libpng-2.0, 91k lines of C) is the ENVELOPE checkout, pinned at the parent of the commit that added the missing clamp. Its CVE is marked out of scope: an out-of-bounds read is not a family this engine arbitrates, and recording "not found" against it as a miss is the easiest way to make a measurement lie. Nothing is vendored; the recipe records the licence so an artifact can say what it ran against. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
c35426e186 |
perf(model): one Joern invocation per query kind, not one per region
JVM startup dominated a scan: a call per region per family put a ten-line file at 275s over three regions, and would have put a thousand regions at roughly fifty hours. The queries themselves are milliseconds once the CPG is loaded, so the cost was almost entirely process starts. scan_repository() is now collect -> batch -> arbitrate. The three .sc scripts accept a `requests` JSON map and answer all of them in one run, keeping the single-request form so the committed measurements stay reproducible byte-for-byte. Arbiters are untouched: each is handed a CpgResult whose `run` returns its own prefetched rows. The batcher builds parameters by calling request_params() extracted from taint.py, dominance.py and config_value.py rather than reconstructing them. A second copy that drifts from the arbiter's is the exact shape of the five family-assumption bugs the predecessor spec spent a session finding; there is now one definition per family and the batcher borrows it. Regions are also deduplicated by (path, function), collapsing the nameless entry-point rows that were producing `model:injection:?:10`. Measured on the same ten-line file: 275s -> 106s, three duplicate findings -> one `model:injection:ping:10`, rung unchanged at model_entailed. Gates byte-identical. Server mode is recorded as task 2.3, deliberately sequenced after the repository measurement: whether the remaining ~100s floor is worth a process model, a weakened subprocess seam and a Scala-evaluating HTTP endpoint depends on a build-time-to-query-time ratio only 2.2 produces. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
e67f3ee2f0 |
feat(contributor-scan): 1.3 the model layer reaches a user
`model-grounded-detection` finished 16/16 with the model layer having zero
production callers. This closes that. A real scan of a Flask file now reports:
by_rung: {"model_entailed": 3, "model_corroborated": 0, "suspicion": 0}
model:injection:ping:10 <- os.system("ping -c 1 " + request.args["host"])
The stage is the fifth MAP proposer, mirroring _check_obligations, and is
additive: pattern, overlay and obligation findings all keep working, and without
a CPG engine it records cpg_unavailable and changes nothing else. Two scan
integrity tests pin the exact degradation set and both broke -- correctly. Their
`requested == baseline` assertion, that findings are byte-identical, still
passed, so the only change is that the scan now SAYS the arbiter was absent
instead of being silently without one. The guards are updated to assert the
model degradation specifically rather than being loosened.
ModelLayerConfig, not ModelConfig: that name is already the [models] block, and
the type checker caught the collision immediately. Two meanings of one name is
how a scan silently reads the wrong settings.
Every payload carries the same shape, skipped ones included. A payload that is
sometimes a full record and sometimes a single "skipped" key is how a report
ends up quietly omitting a section.
unjudged_sample names the lowest-ranked regions the budget did not reach.
Counting them is the honesty requirement; naming them is what lets a maintainer
write a large-repository exclusion list from evidence rather than guesswork.
The run also produced the finding behind new task 1.4: 275 seconds for a
ten-line file. cost_usd is 0.0, so those 21 invocations were not model calls --
they were Joern query launches, one per region per family, each a JVM start.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
|
||
|
Marius CiepluchandClaude Opus 5
|
8663c1a5b3 |
feat(contributor-scan): 1.1 regions from entry points, 1.2 the repository driver
Pair scoring could rely on a labelled function; a contributor cannot supply one -- which code is risky is the question they are asking. Regions now come from entry points, with a file-level fallback that says it is one, and each carries only the families its shape and language admit. Ranking uses what EntryPointRecord already decided (access_level, trust_boundary) rather than a second opinion computed from the same evidence. A family is offered only where its arbiter could answer: access_control needs a handler, so a library file with no entry point is not asked about obligations it has no siblings to be inconsistent with. The driver makes the two costs that were free at pair scale explicit. ONE CPG per repository, reused across regions -- a build is 5-30s on a forty-line excerpt and per-region builds would make this unusable. And a budget that is a TOTAL for the run rather than the enumerator's per-region cap of eight, spent highest-rank-first, with whatever it does not reach counted into regions_unjudged. A tool that silently analyses a tenth of a repository and reports a clean result is worse than one that says what it skipped. Two flaws the tests caught, both mine and both about ownership: the budget counted calls by reading a client's own usage log, so it silently never tripped for a client that keeps none -- the driver now counts its own spend through a wrapper; and the driver trusted its caller to have sorted, when the order the budget is spent in belongs to whoever holds the budget. The third was worse and only surfaced because a test failed for the right reason: the driver passed candidates=() and never enumerated, so a repository scan would have reported only what the arbiters establish unaided and never populated the suspicion band -- which supplied 15 of 26 findings in the pair measurement. It enumerates now. Sibling audit (Req 8.5) done before writing: family routing lives in exactly one production place, pipeline._arbitrate, dispatching on spec type. regions_for returns family names and the driver resolves specs, so that stays true. 750 passing, mypy clean over 96 files, gates byte-identical, zero orphans. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
64a27c4e30 |
docs(model-grounded-detection): record what 16/16 does and does not mean
Asked whether the prior spec is finished. Audited rather than counted, and the answer is: all 16 tasks are complete as written, and not all requirements are satisfied in the product. Four are built but unreachable, so the requirement is not met where it matters: model/report.py implements the per-slice and overfitting-gap discipline (Req 11.1) and has no production caller; resolve_execution_backend returns NullExecutionBackend on BOTH branches so the execution_confirmed rung (Req 10.1) can never be reached; harvest_recipe (Req 9.4) is a seam with nothing plugged into it; and the pipeline that measures 90.0% pair-correct is called only by a measurement harness, never by ousast scan. That is a spec defect rather than an execution one. No task ever said "make this reachable", and Req 11.2 -- the gates stay byte-identical -- is satisfied PRECISELY BECAUSE nothing is wired. I read that as reassurance for a whole session when it was evidence. The spec is not reopened; it remains the complete record of building and measuring the arbiter. contributor-scan takes all four: its Req 1 closes 8.x and 11.1 in the product, Req 7.1 forces the memory decision 10.1 defers, and Req 9 covers the harvest 9.4 left as a seam. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
4e4681f162 |
docs(contributor-scan): design and tasks -- wire it, then measure on a real repo
Requirements approved, design and tasks generated. 47 criteria, all traced; 13 tasks in 6 groups. The design is deliberately conservative. The model layer joins as ONE STAGE INSIDE MAP, beside the obligations checker it most resembles -- which already takes entry points, produces findings and merges them. Findings are added, never substituted; the pattern and overlay paths keep working; a machine without Joern scans exactly as it does today. Nothing here is allowed to make the tool worse for someone who does not have the engine. Two decisions carry the most weight. One CPG per scan rather than per region: a build is 5-30s on an excerpt and is the dominant cost at repository scale, so per-region builds would make this unusable. And one total budget rather than the enumerator's per-region cap of eight, which is meaningless across thousands of regions; the budget is spent highest-rank-first and whatever it does not reach is counted and reported, never silently truncated. Task 2.2 is a go/no-go and comes before shipping: the first scan of a pinned known-vulnerable checkout is a measurement nobody has taken, and if one CPG per repository proves infeasible the region and budget design changes shape. Better to learn that before groups 3-6 are built on it. Group 5 (corpus) is last on purpose. The binding constraint today is reachability, not corpus size -- 198 of 238 OWASP pairs went unvendored because volume without readability makes every number worse. The task preamble carries the predecessor's two hard-won habits: audit sibling paths in the same commit as any routing or matching change (five bugs shared that shape, each caught by a measurement rather than a check), and run the whole CI list rather than the parts one remembers (CI was red for a session because `ruff check` passing was taken as evidence for `ruff format --check`). Not approved for implementation. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
fe45081b43 |
fix(ci): run ruff format, and record run 5 at 90% pair-correct
CI was red on `ruff format --check`: 23 files. I ran `ruff check` after every
change this session and never once ran `ruff format --check`, though CI runs
both. Lint passing is not formatting passing, and I treated one as evidence for
the other for the whole session.
Formatting applied. The full CI sequence now reproduces locally: format, lint,
mypy over 94 files, 735 tests, three gates byte-identical, compileall, and the
semantic job's 16 marked tests.
Also records run 5 of the pipeline measurement, which the vampi fix lifted as
predicted:
hunter (K=5) pipeline (K=1)
pair-correct 76.7% 90.0%
leak 12.3% 6.7%
detection -- 96.7%
cost ~$0.69 $0.0068
27 of 30 pairs. Five runs, each step from a measurement contradicting an
assumption: 73.3 (no pipeline path) -> 73.3 (routing fixed, IDENTICAL, which
exposed the discarded verdict) -> 80.0 (residual question matched to its
arbiter) -> 86.7 (aiohttp settings) -> 90.0 (dominance question no longer
contradicting its own evidence).
One miss left: dsvpwa-dsvpwa-main, whose setting is an argparse --ssl default
rather than a security-setting call. Out of reach before the settings work and
still is; contorting the arbiter around one pair would be worse than the miss.
The caveat is recorded beside the number: 30 pairs of one slice, and the slice
iterated against all session. The +16% overfitting gap against vfc-js is why
contributor-scan Req 8.1/8.2 exist.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
|
||
|
Marius CiepluchandClaude Opus 5
|
14ca0a96ce |
docs(contributor-scan): requirements, including the corpus asks I first missed
10 requirements, 46 criteria. A NEW spec rather than an amendment to model-grounded-detection, which is complete and closed. Req 1-8 cover the gap that prompted it: the model layer has zero production callers, so ousast scan must use it (1), regions must come from entry points rather than a labelled function (2), ranking and budget must work at repository scale where the enumerator's per-region cap of eight is meaningless (3), the performance envelope must be measured on a real checkout (4), output must be actionable (5), packaging must be built and verified rather than merely defined (6), the two deferred decisions must close (7), and the honesty disciplines carry forward (8). Req 9 and 10 were added after the maintainer asked whether their requests had been captured. They had not been -- I folded the corpus asks implicitly into "run against a known-vulnerable repository", which is not the same thing. Now explicit: a PHP/WordPress slice (php2cpg works once an interpreter is present, which the image provides, and plugin CVEs are overwhelmingly the families the model arbitrates well); real-world C library CVEs -- libpng, image libraries, OpenSSH, libpam, kernel modules -- gated on the memory decision, because until it closes they would join 173 pairs already idle; more vibe-code pairs, since that slice is where the stated audience works and it is only 35 pairs; and regression tuning over a pinned large-repository baseline, which is the thing pair scoring structurally cannot see. Req 9.4 carries today's lesson: verify on a sample that the model can READ a slice before vendoring it. The OWASP harvest found 238 pairs and vendored 40 for exactly that reason. Not approved. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
41266ce472 |
docs(contributor-scan): brief -- the model layer is complete and unshipped
Asked what the spec state was and whether anything was forgotten. It was, and it is the largest thing outstanding: model-grounded-detection is 16/16 tasks complete and its arbiter measures 86.7% pair-correct at 6.7% leak, but the model layer has ZERO production callers. `ousast scan` still runs preprocess -> map -> overlay -> obligations -> regress -> report and touches none of it. Every number quoted today came from a measurement harness in a scratch directory. That is a scope gap in the predecessor spec rather than a slip in executing it: there was never a task saying "make this reachable", and its Req 11.2 (gates stay byte-identical) is satisfied PRECISELY BECAUSE nothing is wired. The spec is complete and the feature is unshipped, both at once. A new spec rather than an amendment, because the predecessor is coherent and closed -- it built an arbiter and measured it -- while this is a different question: making it something a developer runs on a repository they are about to contribute to. What it covers, in the order it bites: scan integration; whole-repo scanning (entry-point region selection, ranking across thousands of candidates, a budget policy -- the enumerator caps at 8 per region, meaningless at repo scale -- and a performance envelope never measured); running against a known-vulnerable checkout, since the 176 real-world CVE pairs are single-function excerpts that exercise none of it; the deferred memory decision, with 173 of those pairs idle behind a stub tier; and retiring evidence_level. It also records what was built outside any spec: the pipeline itself, the ops work (one image, compose, shell wrappers, pyinfra -- with the Docker image never actually built), and the corpus additions. And it carries the known limits forward rather than losing them: the flat permissive_values list that cannot tell debug=True from httponly=True, interprocedural resolution as the most repeated miss cause, and the five bugs that shared one shape -- a taint-specific assumption applied to every family, each caught by a measurement rather than a check. Not approved. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |