mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-16 03:49:26 +00:00
main
15
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Marius Ciepluch
|
8acada0d98 | feat(pre-push-safety-net): pin labeled evaluation inputs | ||
|
Marius CiepluchandClaude Opus 5
|
07df314e12 |
fix(cpg): prove the frontend can read the source before building, and check the graph always
Two permanent guards, and a correction that motivates both.
THE CORRECTION. There is no php2cpg defect. Every PHP build on this machine
ran through a local `php` shim proxying stdio through `docker run`, and
php2cpg depends on the INTERLEAVING of its parser's stderr banners with its
stdout JSON to attribute each document to a file. The proxy scrambles it:
streaming shim drops 72-88 767KB / 969KB / 1.2MB different every run
buffered shim drops 0 5,703 bytes empty, and silent
NATIVE php drops 0 2,872,996 bytes identical x3
The 637-file tree with native php: 0 drops, a 4,329,471-byte CPG, 20.0s,
1.9GB peak -- which is exactly what task 5.10 said before I started
"correcting" it. No intermittent failures, no malformed graph, no dataflow
assertion, no scale limit.
GUARD 1: the interpreter must prove it can read the tree, before anything is
built. Every instrument failure this project has had takes one shape -- the
input was unreadable and the tool reported a plausible ZERO -- and it has cost
four wrong diagnoses in a day:
a php that could not see ~/.cache -> "cannot handle 637 files"
a php whose stdio was proxied -> "drops half its batches"
a container missing a bind mount -> "0 dropped", 5,703-byte graph
None announce themselves: file_exists() is false, the parser writes nothing,
the frontend exits 0, and "no errors" reads exactly like success. Checking the
output cannot separate that from an empty repository. Checking that the input
arrived can, and costs one exec.
GUARD 2: the graph census is taken on EVERY build, not only one the frontend
warned about. A graph holding nothing arrives with no warnings at all --
silence is the symptom that has no symptom -- and the shortfall is now
reported as "the frontend reported no failures but the graph holds N of M
source files".
`cpg_build_failed` carries that detail through to the report instead of being
a bare reason.
Also adds tests/test_cpg_queries_compile.py: the queries are 900 lines of
Scala that nothing compiled. A rowsFor signature change shipped with 850 tests
passing and broke every query at runtime, found by a measurement failing hours
later. Each query is now compiled AND run against a real graph wherever joern
exists, and must return a parseable payload.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
|
||
|
Marius CiepluchandClaude Opus 5
|
f3f41e2fa9 |
docs: the 637-file wall is the taint query, not memory and not vendored code
Follow-up measurements, because the obvious explanations were all wrong.
Not memory: at 2GB AND 4GB the CPG never finishes loading, with no
OutOfMemoryError either time, both runs stopping in the same place --
Braintree\Util. What costs is Joern's dataflow overlay, recomputed per import.
Not scope, though scope is real and worth fixing on its own terms: 429 of the
637 files are vendored SDKs (Braintree, Stripe, each with its own LICENSE) and
31 are tests. PMPro's own code is 177 files, 62k lines. Auditing a plugin by
scanning the payment SDK it bundles is wrong regardless of cost -- those
findings are not the plugin's bugs -- and it is the same idea shipped.py
already applies to Makefile.am, which PHP has no equivalent of.
Scoping to authored code helps and does not fix it:
regions 4,463 -> 1,324 total 996.6s -> 843.7s findings still 0
600.3s of that 843.7s is the taint query hitting QUERY_TIMEOUT_SECONDS (300s)
on each of two shards. Two timeouts, not work.
So the wall is the per-request cost of the taint query at 2,500 requests. The
next measurement is what the two-stage joins cost at scale -- 1.6x on a slice
(62s -> 97s) and plausibly the dominant term here. Raising the timeout before
knowing that would only buy a slower way to find out.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
|
||
|
Marius CiepluchandClaude Opus 5
|
c03944f7dd |
docs(benchmarks): the whole-plugin envelope, next to the slice results
The class-by-class table is measured on one- and two-file slices, which invites the reading that the engine scans WordPress plugins. It does not, and the README should not let that stand unqualified. 637 files, 117k lines: the build is affordable (33s, 1.2GB, a 1.6MB CPG) and a CI runner can do it. The scan is not -- 996.6s to decide nothing, with the taint query dying in ForkJoinPool and 500 of 4,463 regions examined. What the run does correctly is refuse to look clean: files_unparsed, query_failed, cpg_sharded and the region cap all reach the reader. A scan that decides nothing and says so is a different artifact from one that decides nothing quietly. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
1ce7e9db9b |
fix(cpg): build php through the frontend and retry while it drops files
The intermittent empty-graph builds that cost four manual retries to measure
WP Statistics at all.
First, a correction: the explanation this repo carried was wrong. It said
php-parse's stderr banners interleave into block-buffered stdout and corrupt
the JSON. Capturing the exact bytes php2cpg receives -- by putting a logging
wrapper on `php` -- shows its input arriving complete and valid: two
well-formed JSON documents, 5,176,594 bytes, byte-identical across runs, with
no banner anywhere inside them. The defect is in php2cpg's own reading of that
input, and nothing outside it repairs a given attempt.
Three fixes were tried against the old theory and each changed nothing, which
is how the theory died:
* an unbuffered PHP shim via --php-parser-bin 0 of 6 builds usable
* compacting the JSON, 5,176,594 -> 943,028 bytes 0 of 6
* -J-Djava.util.concurrent.ForkJoinPool.common.parallelism=1 0 of 6
What is true is that the failure is INTERMITTENT, and that `joern-parse` is
both worse at it and silent about it. Measured on one WordPress slice:
joern-parse 5 of 8 builds usable, and says nothing when they are not
php2cpg 11 of 12 builds usable, and names every file it dropped
So php builds through the frontend directly and retries while it reports
dropped files. Retrying is sound only because the failure is intermittent AND
reported -- retrying a silent failure would be superstition; retrying a
reported one is just using the report. It gives up after four attempts and
reports what is still missing rather than pretending a partial graph is whole.
wp-vuln through the driver before: 4+ manual retries to get one build
after: 6 of 6, 3 entailed findings each time
joern-parse remains the path for every other language, and the fallback when
no frontend is on PATH.
843 tests pass; gates byte-identical; mwwpform, pmpro and VAmPI unchanged.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
|
||
|
Marius CiepluchandClaude Opus 5
|
d6908f18ba |
feat(cpg): the hook half — CVE-2022-25148 found across an add_filter/apply_filters edge
The last of the three WordPress flaw classes 5.8 measured as missed. The
value crosses a registry inside WordPress core:
add_filter('wp_statistics_current_page', array($this, 'set_current_page'));
return apply_filters('wp_statistics_current_page', $current_page);
php2cpg drops the callback argument entirely -- the CPG holds
`add_filter("wp_statistics_current_page", )` -- so the edge cannot come from
the graph at any price. It comes from the source text instead, where both ends
are literals, via `mapping.php_hook_callbacks`.
vulnerable fixed
wpstatistics 3 entailed, incl. pages.php:225:record 2, that one absent
mwwpform 1 ($this->attachments) 0
pmpro 18, incl. CVE at :936 1
VAmPI 4 unchanged
The correctly-prepared sibling query one line below the CVE is not flagged on
either side.
Four things this cost, each measured rather than guessed:
* Half one must be asked per ARRAY KEY, not per callback. "Does an
unsanitized value reach this callback's return" is TRUE on both sides: the
fix escapes `type` and `id` and leaves `search_query` alone. A
callback-level answer flags the fixed side as readily as the vulnerable
one, which is a leak, not a detection. The key is a literal at both ends,
so it joins the way the hook name and the field name do -- the same idea
for the third time.
* The sanitizer test had to stop crediting a sibling's cleansing. The
vulnerable query is ONE concatenation holding both `esc_sql($page_uri)`
and the injectable `{$current_page['id']}`, so asking whether a
sanitizer's name appears in a path element's text marked the whole flow
sanitized. An element must now BE the cleansing -- a call to it, or its
name as a string literal, which keeps `array_map('esc_sql', ...)` working.
* Hook seeds are repository-wide where field seeds are file-scoped. A hook
registry is global by construction: the callback is in one file and its
`apply_filters` in another. The query returned nothing until those two
scopes were separated.
* `sourceKind` had to be REPORTED, not inferred. By the time a row reaches
Python a hook source reads as `$current_page["id"]` and a parameter as
`$args`, and `_usable_flows` discards parameter flows whenever a modelled
one is present -- so seven sanitized `$_SERVER["REQUEST_URI"]` flows
displaced the CVE between the query and the verdict. That preference then
had to move PER SINK SITE: applied across a region it decides one site's
question with another site's evidence, and PMPro's CVE at :936 vanished
from a region still reporting fifteen other findings.
`verdict` now delegates to `verdicts` rather than running its own selection
over the union, because two copies of "which flow decides" is exactly the
duplication this module has been bitten by before -- and had been again: the
per-site preference did not reach the single-verdict path, so the two entry
points disagreed about the same rows.
841 tests pass; gates byte-identical; VAmPI unchanged.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
|
||
|
Marius CiepluchandClaude Opus 5
|
dcdbe322a1 |
feat(cpg): taint a field where it is assigned — the field half of 5.11
`_delete_files()` takes no parameters. CVE-2023-6559 arrives as a FIELD,
written by one method and read by another:
public function __construct( ..., array $attachments = array() )
$this->attachments = $attachments; // half one ends here
protected function _delete_files()
foreach ( $this->attachments as $file ) // half two starts here
unlink( $file );
Both halves were already in the graph. Only the join was missing, and the key
it joins on is the literal `$this->attachments`, written identically at both
ends -- so this is a second question asked with the query that already exists,
not an edge a frontend has to invent.
A read of `$this->F` is a source when some assignment to the same literal
`$this->F` in the same file is fed by an unsanitized flow from a seed. Scoped
three ways so a field name common to every class ever written cannot become a
wormhole: only fields the sink scope reads, only assignments in the same file,
and the right-hand side must itself be reachable from a source.
The receiver could then leave `parameterNodes` -- the precision fix refused
two commits ago, now sound, because the path it was standing in for is carried
by the field it actually travels through:
before after
mwwpform vulnerable 1, from `this` 1, from $this->attachments
mwwpform fixed 0 0
pmpro vulnerable 15 -- 4 named, 11 receiver 18 -- all named or field
pmpro fixed 1, receiver-sourced 1
VAmPI 4 entailed 4 entailed, unchanged
CVE-2023-23488 is still at class.memberorder.php:936 and CVE-2023-6559 at
class.mail.php:259. Every finding now names the value it followed instead of
naming the object.
Three things cost a measurement each, recorded in the task:
* A summary must carry the SANITIZATION status of the half it summarises,
not merely its reachability. Without that clause,
`$this->sqlQuery = "..." . esc_sql($x) . "..."` reads as tainted -- and
PMPro builds most of its queries that way, so the fixed side went from 1
finding to 13. That is the pair no longer separating, which is the only
number that matters.
* Two-phase `reachableBy` then `reachableByFlows` is SLOWER, not faster:
183s -> 256s. Running the analysis twice costs more than skipping path
reconstruction saves. Reverted.
* Memoizing per field rather than per file took it to 97s, about 1.6x the
pre-field cost for that class, because only fields some region reads are
worth the flow query.
`inScope` is factored out of `sinkCalls` so the field sources are scoped to
exactly the same place as the sinks, rather than a second copy of those three
branches drifting from the first.
841 tests pass; gates byte-identical; VAmPI unchanged.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
|
||
|
Marius CiepluchandClaude Opus 5
|
e41d6dd7ea |
docs(contributor-scan): 5.11 -- a detection strategy, not a verdict of impossible
"Structurally impossible" is a verdict, and a verdict is not a plan. Replacing
both structural-miss tasks with the one mechanism that covers them, because
they are the same shape: two halves of a path joined by a key that is a
literal in the source text.
hook: $_REQUEST -> ... -> return of set_current_page
key: 'wp_statistics_current_page' literal at both ends
return of apply_filters(key) -> ... -> $wpdb->get_row
field: $_POST -> ... -> assignment to $this->attachments
key: attachments a field name at both ends
read of $this->attachments -> ... -> unlink
Neither half is missing from the graph. Only the join is. So this is not a
request for a frontend to draw an edge it cannot draw -- it is a second
question asked with the query that already exists:
stage 1 the existing taint question against a synthetic endpoint: does
untrusted data reach a callback's return, or an assignment to
$this->F?
table built by reading literals, never inferring: add_filter('H', c)
against apply_filters('H', ...); $this->F = against $this->F
stage 2 does the apply_filters return, or the field read, reach a sink?
join on the key, with the RUNG carrying the join's uncertainty -- one
registered callback and a literal key entails; several callbacks or
a computed hook name corroborates, and the witness says which
"Any hook may reach any handler" is refused absolutely: a complete graph over
the plugin would entail everything.
No new dependency, no CPG mutation, no IR -- both stages ride the batch that
already runs, which is the point of expressing it as a query-level join.
It also recovers the precision fix 5.8 had to refuse. With the field half
working, _delete_files() is reached through $this->attachments rather than
through $this being tainted wholesale, so the receiver can then leave
parameter sources and take PMPro's 11 receiver-sourced findings with it.
5.8's numbers are the before-side of that measurement, and both observables
are pairs already pinned.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
|
||
|
Marius CiepluchandClaude Opus 5
|
435c14f3ec |
docs(contributor-scan): 5.8 -- WordPress class by class, including what it cannot ask
Three plugins, each pinned at a version whose advisory was read out of the
code, each measured against its own fixed commit:
unauth injection, sink in the receiving function pmpro FOUND
arbitrary file operation mwwpform FOUND
unauth injection across a WordPress hook wpstatistics MISSED
missing capability check -- cannot be asked
unescaped output -- cannot be trusted
Both hits are entailed by the graph alone -- no judge, no spend -- and both
disappear on the fixed side, because both fixes are calls the fact tables
carry: `$wpdb->prepare` and `realpath()`. The engine is not recognising a bug;
it is failing to find a cleansing call on a path it resolved, then finding one.
Precision is stated rather than flattered. PMPro's vulnerable side reports 15
entailed, 4 from a named parameter and 11 from the receiver, and one of those
survives into 2.9.8 at a query whose every fragment is `esc_sql`-wrapped.
The miss is the useful half, and it is checkable rather than inferred. WP
Statistics' CVE has a modelled source, a modelled sink and `esc_sql` as its
fix, and reports nothing on either side, because the file holding the sink has
zero superglobals, makes no call into the class holding the source, and is
joined to it only by
add_filter('wp_statistics_current_page', array($this, 'set_current_page'));
return apply_filters('wp_statistics_current_page', $current_page);
Both ends name the callback with a string, so no frontend can draw the edge.
Most plugin data travels this way. What the engine reports there instead is two
other sites, the same class at the wrong lines, identical on both sides -- and
not noise: upstream annotates both `// phpcs:ignore WordPress.DB.PreparedSQL`
and later rewrote one with `$wpdb->prepare`.
Two classes could not be asked at all, which is not the same as a miss.
`dominance_specs(language="php")` is empty, so missing-capability is never
asked; and `output_encoding` shares the one flat sanitizer list, which holds
`esc_sql` and `intval` but not `esc_html` -- wrong in both directions, so no
XSS verdict from this engine currently means anything.
5.10's root cause is recorded too: php2cpg's parser bridge, a race rather than
a scale limit, and the mitigation is per-file exclusion. Adds 5.11 (the hook
edge, with CVE-2022-25148 as its observable) and 5.12 (taint a field where it
is assigned, with CVE-2023-6559 as its guard).
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
|
||
|
Marius CiepluchandClaude Opus 5
|
e0442d146c |
fix(cpg): a graph the frontend emptied must not read as a clean repository
Scanning a two-file WordPress slice returned 64 regions examined, zero
findings and zero degradations. The graph it examined had no methods in it.
php2cpg 4.0.623 feeds several files to one `php-parse` process and reads its
stdout and stderr merged. php-parse writes a `====> File <next>:` banner to
stderr the moment it starts the next file, while the previous file's JSON --
megabytes of it -- is still draining from a block-buffered stdout. The banner
lands inside the JSON, ujson reports
expected json value or } got "=" at index 5011645
and the frontend drops that file. Reproducible on real plugin code, and the
size that matters is one file's AST dump, not the repository's:
class.memberorder.php alone (1834 lines) -> 142KB CPG, no failures
the same file + a two-line file -> 5.7KB CPG, BOTH dropped
One oversized file empties the whole graph. `joern-parse` then does not
propagate any of the frontend's warnings: 13 lines of output, exit status 0,
"Successfully wrote graph". This is what made a 637-file plugin unscannable
and reported it as clean -- it was never a scale limit.
Two fixes, because the failure arrives on two different paths:
* `_unparsed_files` reads the per-file warnings the frontend does emit, and
the scan reports them as `files_unparsed`. This covers the direct-frontend
path where the warnings are visible.
* The batched query now carries a `__census__` of the graph -- method and
file counts -- under a key no request id can collide with, and a scan over
a graph with no methods degrades as `cpg_empty`. This covers joern-parse,
which shows nothing. It rides the batch rather than costing its own
invocation, because avoiding a JVM start is the entire reason batching
exists.
Neither invents a verdict. There is no rung for "the graph was empty" because
the ladder labels verdicts and none was produced, which is exactly why it has
to be a degradation.
pmpro two-file slice before: 0 findings, degradations=[]
after: 0 findings, degradations=[cpg_empty]
Gates byte-identical; 840 tests pass.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
|
||
|
Marius CiepluchandClaude Opus 5
|
37a6594703 |
fix(mapping): a PHP method with a visibility modifier is a region
`^\s*function` matched a bare declaration and nothing else, so every method
written the way modern PHP writes them -- `public function`, `protected
function`, `private static function` -- was invisible to the region model.
The effect is not subtle on class-based plugins. MW WP Form's
`class.mail.php` produced ONE file-level region spanning 272 lines instead
of 16 method-scoped ones, and its `protected function _delete_files()` --
which holds CVE-2023-6559 -- was not a region at all. Paid Memberships Pro
happened to work only because that particular file omits the modifiers.
class.mail.php 1 file-level region -> 16 method-scoped regions
Also adds the MW WP Form checkout as the PATH family's plugin, pinned at
3bdd33d (5.0.3) with CVE-2023-6559 verified in the code:
protected function _delete_files() {
foreach ( $this->attachments as $file ) {
if ( file_exists( $file ) ) {
unlink( $file );
and the fix at 02ebc67 replacing the existence check with `realpath()`,
`is_file()` and a directory-confinement test. `unlink` is a modelled path
sink and `realpath` a modelled sanitizer, so this is a pair the engine
should separate rather than merely flag -- which is what makes it worth
more than a single hit.
Gates byte-identical.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
|
||
|
Marius CiepluchandClaude Opus 5
|
922ed84053 |
feat(repos): a WordPress plugin with a verified CVE, and the WordPress DB API
contributor-scan 5.1's detection half. Paid Memberships Pro 2.9.7 pinned at
1a900be3, carrying CVE-2023-23488 -- unauthenticated SQL injection through
the `code` parameter of the /pmpro/v1/order REST route.
**Verified in the code, not taken from the advisory.** At this commit
classes/class.memberorder.php:936 reads
$id = $wpdb->get_var("SELECT id FROM $wpdb->pmpro_membership_orders
WHERE code = '" . $code . "' LIMIT 1");
and tag 2.9.8 replaces it with `$wpdb->prepare( "... code = %s ...", $code )`.
637 PHP files and 117k lines, so it is also the first PHP checkout large
enough to say anything about an envelope.
**WordPress never calls mysqli; it goes through $wpdb**, so the facts could
not have found this. Added qualified: `$wpdb->get_var`, `->get_results`,
`->get_row`, `->get_col`, `->query`, `->replace`. Qualified because a bare
`get_var` or `query` matches any method of that name anywhere -- the
generic-name trap that made `query.get` match every `.get(` in a repository
this morning. `->insert`, `->update` and `->delete` are deliberately absent:
they take arrays and WordPress prepares them itself.
**And the thing that nearly went wrong.** The obvious next step when
writing PHP facts is to add WordPress's sanitizers -- sanitize_text_field,
esc_html, esc_attr. Doing so would have made this CVE invisible, because
its handler calls `sanitize_text_field( $params['code'] )` and then
concatenates the result into a query. sanitize_text_field is a real
sanitizer that does nothing whatsoever about SQL injection.
`TaintSpec.sanitizers` is one flat list per language whose comment states
the assumption outright -- "a sanitizer that breaks one flow breaks it
whatever the sink's family" -- and PHP falsifies it. So only SQL-specific
cleansers are listed, which leaves XSS under-sanitized instead, and the
real fix is recorded as task 5.6: a discharge counts for the obligation it
discharges. That is task 2.6's finding arriving in the taint family, where
the dominance arbiter already reads kind from the facts and the taint spec
has no equivalent.
The end-to-end scan of this checkout is not yet reported: the first run
returned 1577 regions with 0 scanned, and the CPG builds standalone at
4.3MB with no errors, so the two do not yet agree. Numbers when they do.
Gates byte-identical.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
|
||
|
Marius CiepluchandClaude Opus 5
|
d15985f695 |
fix(model): a guard discharges the obligation it is a guard of
contributor-scan 2.6. Access control entailed 7 of 9 VAmPI handlers,
including three that do authorize and three the OpenAPI spec declares
public. Five separate defects, each invisible to a corpus of one-function
excerpts and each found by asking the query what it actually matched.
**A trailing segment is not a match.** `query.get` was matched by its last
segment, so every `.get(` in the repository became a protected database
operation -- `request.headers.get('Authorization')` and
`request_data.get('username')` included. Four of the seven false positives
came from this, and so did the apparent detection of the known BOLA: it was
entailed because fetching the Authorization header had no dominating guard,
not because of the defect. A qualified token now requires its qualifier.
**Configuration is not authorization.** dominance_specs took every
obligation fact. `security_setting` operations are discharged by a VALUE,
which config_specs already claims them for; pooled in here they made
`app.run(...)` an operation requiring an authorization guard and put
thirteen tokens in BOTH lists -- an obligation that is its own discharge,
the same shape as the taint sink that used to sanitize itself.
**Input validation is not authorization.** `jsonschema.validate` was
counted as a guard, so handlers with no authorization check at all read as
guarded. No operation in the table requires validated_input.
**A guard discharges the obligation it is a guard of.** The largest one.
`orm-read` requires an identity_constraint or an ownership_check;
`token_validator` is a path_guard. Flattened into one list, authenticating
the caller discharged an object-level obligation -- a handler that
authenticates and then looks the record up by a PATH parameter read as
fully guarded. The facts state that relation and the spec was discarding
it; it is now carried and the engine asks per operation.
**A word boundary needs a word.** Found while fixing the above. Guard
matching wrapped every token in `(?<![A-Za-z0-9_])...(?![A-Za-z0-9_])`.
The identity token `['sub']` begins with `[`, and the character before it
in `resp['sub']` is `p` -- so the assertion could never hold. Every
identity token starting with punctuation was unmatchable, and the identity
constraint that separates a correct lookup from an IDOR was invisible.
Also: dominance rows were repository-wide, so the model's unguarded
`delete_user` was attributed to the API handler of the same name. Rows are
scoped to the region's file, which is what the rung's own justification
says -- the evidence is a file contradicting itself.
Measured against VAmPI's OpenAPI `security` blocks, which declare per
operation whether auth is required, and agree with the code:
vampi precision 28.6% (2/7) -> 50.0% (2/4)
known found 3 of 3, all model_entailed
And on the labelled access-control pairs, which nobody tuned this against:
all 9 pairs pair_correct 3 -> 3, leak 44.4% -> 11.1%
7 non-VAmPI pair_correct 2 -> 2, leak 42.9% -> 0.0%
Every detection lost was on a pair whose FIXED side was flagged too, which
scores nothing. No correct detection was traded for the precision.
get_by_title was flagged before it was in the recipe: its code carries the
comment "# Broken Object Level Authorization" and the README lists it, so a
real finding was being scored as a false positive. It is now the recipe's
third known vulnerability.
Both remaining false positives are endpoints the spec declares public.
Recorded as 2.8, with the reason the mapper's access_level cannot simply be
trusted: for decorator frameworks "public" means "no decorator found",
which is the bug itself.
Gates byte-identical.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
|
||
|
Marius CiepluchandClaude Opus 5
|
177c7f3719 |
feat(model): first repository measurement, and the four things it found
contributor-scan 2.2. VAmPI at a pinned commit, scanned whole rather than as excerpts, with benchmarks/repos/measure.py recording build time, query time, arbitration time, peak memory, cost, unjudged regions, and whether each known vulnerability was found and at which rung. The corpus could not have found any of this. Every defect below needs a repository to become visible, and three of them were invisible behind a passing test suite. **The batch path was unreachable.** query_batch was implemented, tested and committed while CpgResult exposed only `run`, so the driver's capability check found nothing and fell back to one JVM per region-family: the first attempt at this measurement ran eighteen minutes on 520 lines, issuing `--param function=` one at a time. Every test called query_batch directly, so none of them noticed. CpgResult now carries run_batch and a test asserts the capability is reachable through the object the driver is handed. The earlier 275s->106s improvement attributed to batching came from region deduplication alone. **The budget stopped free work.** It bounds model calls, but was enforced by breaking the region loop -- spending a paid limit to stop unpaid CPG arbitration. 6 of 22 regions were examined and VAmPI's BOLA, which the graph entails for nothing, was never reached. It now withholds the judge only. 22 of 22 regions arbitrated, 17 counted as regions_unasked (silent because unaffordable is not silent because decided), and the BOLA is entailed by the model layer. **A region with no function asked about the whole repository.** Sending function="" matched every hit in the tree and the driver attributed that answer to whoever asked: one `host='0.0.0.0'` in app.py became eight identical entailed findings in eight files that do not contain it. Queries now carry the region's file. Entailed findings 14 -> 7, query time 70.9s -> 40.9s. The single-request form stays unscoped so the committed pair measurements remain reproducible. **The JVM had no heap bound.** Joern claimed a quarter of physical RAM and runs were killed under memory pressure at different points, so the numbers described the host rather than the engine. -J-Xmx2048m by default, OPENULTRASAST_CPG_HEAP_MB to raise it. Also: arbitrated findings carried `function:line` where the report layer read `path:line`, so every entailed finding reached a contributor with a function name where its file belonged; and the witness line was parsed by splitting on a comma, which produced sites like `config.py:17): permissive literal '0.0.0.0':?`. Reading: GO on one CPG per repository -- 6.2s to build, 177s end to end, 869MB, $0.0067, nothing infeasible. NO-GO on region-scoped taint as the only shape: VAmPI's SQL injection runs from a handler parameter into a function in another module, and a per-function region sees neither end. That is the ordinary shape of a real injection bug, so it is recorded as task 2.4 along with three smaller defects this run exposed. Gates byte-identical. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
8c387306be |
feat(repos): pinned known-vulnerable checkouts, offline by default
The pair corpus scores excerpts: a vulnerable function and its fixed twin, handed to the engine with the answer already narrowed down. Nothing in it exercises a repository -- region ranking, the budget, one CPG over thousands of files, or whether a verdict that holds on fifteen lines still holds when the source, the sink and the guard live in three modules. benchmarks/repos/<name>.toml names a real checkout: url, a full 40-char commit SHA, licence, and every known vulnerability with its file, function and line. `ousast repos` lists them; `--fetch` materialises one; CI does neither, and conftest clears the switch and redirects the cache so a developer with warm checkouts cannot read them by accident. Three rules the loader enforces rather than documents: - A branch or tag is rejected. Either would make the same recipe mean different code next month while the baseline it produced kept the old numbers. - `in_scope` must be stated per vulnerability, never defaulted. It decides whether "not found" is a miss or a non-question, which is the most important thing about the result. - A recipe that declares `detection` must name something in scope to detect. `ousast repos` does not trust a recipe either: against a materialised checkout it checks HEAD against the pin, that the licence file is there, and that each declared function appears above its declared line. That earned its place immediately -- the first libpng recipe named LICENSE.md, which is the filename at HEAD today and not at the pinned commit, where it is still LICENSE. Two checkouts, verified against the real repositories: - vampi (MIT, 520 lines) is the DETECTION checkout. Excerpts of it are already in the pair corpus, including the update_password miss diagnosed earlier, so a whole-repository run says directly whether a verdict survives the move off the excerpt. Its routes register through an OpenAPI operationId, so it exercises registration-based entry-point mapping rather than the decorator path. - libpng (libpng-2.0, 91k lines of C) is the ENVELOPE checkout, pinned at the parent of the commit that added the missing clamp. Its CVE is marked out of scope: an out-of-bounds read is not a family this engine arbitrates, and recording "not found" against it as a miss is the easiest way to make a measurement lie. Nothing is vendored; the recipe records the licence so an artifact can say what it ran against. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |