mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-19 04:59:28 +00:00
e492ebd16e594e47e2041bae76b002f15ce0cb86
13
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Marius CiepluchandClaude Opus 5.5
|
e492ebd16e |
fix(cpg): null-guard method access in the query scripts -- a class-level call aborted the whole taint batch
On real PHP source (CPS-IT/mailqueue) a call outside any method has a null .method; inScope/nestedInLabeled dereferenced it (NullPointerException) and the script exited 1, losing every taint answer for the repository. 16 sites in taint/dominance/config/vocabulary.sc now treat a null method as out of scope; non-null behaviour is unchanged (same outputs on the repro graph for the other scripts). Static regression test added. Implemented with Codex. Co-Authored-By: Claude Opus 5.5 <[email protected]> |
||
|
Marius CiepluchandClaude Opus 5
|
e6bdac46e2 |
fix(access-control): decide the sibling scope, and see identity a destructuring spelled away
Two of NodeGoat's three corroborated claims were correct code. `const { userId } = req.session`
lowers to a temporary and then a field read, so nothing in the graph spells session.userId by the
time the value reaches dao.getByUserId(userId, ...), and the constraint the handler really applies
was invisible.
The arbiter now walks back three hops over the method's own assignments from the names the
operation is GIVEN, and req.session joins the identity sources. The token is only safe because of
that requirement: what discharges an object-level obligation is that the value passed to the
operation comes from the authenticated context, not that a session is mentioned nearby. A handler
that reads the session for logging and keys its query on a path parameter is the bug this family
exists for, and it stays unguarded.
Two exclusions the measurement forced, each caught by a page whose whole purpose is the leak it
demonstrates.
- A callback is not a constraint. Taking identifiers from the whole argument subtree credited
NodeGoat's memo listing with a discharge, because memosDAO.getAllMemos((err, docs) => ... userId
...) reads every memo in the collection and only RENDERS the session's user. Method references
are excluded from the argument names.
- An assignment is not a dominating guard. The session read dominates the query, so that clause
credited it instead and the memo listing went quiet again. An assignment is a read of the
authenticated context, not a check applied to it, so operator calls are excluded there -- the
same exclusion the sink matcher needed. The leak stays reported.
On NodeGoat displayContributions and displayProfile go silent, the allocations insecure direct
object reference, both benefits pages and the memo listing stay, and the login and signup handlers
become honestly unguarded rather than accidentally cleared by a callback's text. Six corroborated
where there were seven, and a better six. VAmPI is unchanged at two entailed findings.
The sibling scope is decided by measurement and kept at the file. On NodeGoat the region's own
file entails two handlers, both public routes and both false, and no true defect at all. Every
handler in the application entails six: four true, and the same two public routes. Both scopes
carry the same false positives, because neither can tell a public route from a protected one --
a semantic question that is already the corroborated band's residual. With the carried rung capped
the scope changes no verdict, only the witness, so widening it would buy a repository-wide walk
per question and no precision. A public-route discriminator would change that; the file scope is
not what holds this family back.
1,415 tests pass, sixteen skipped; thirteen Joern-gated query tests pass in the image.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
|
||
|
Marius CiepluchandClaude Opus 5
|
c0427e6ceb |
feat(access-control): model the collection read, by argument shape rather than by name
NodeGoat's documented insecure direct object reference reads with allocationsCol.find(searchCriteria()), and `find` was absent from an operation table holding findOne, findById, findAll and findByPk. The obligation was missing at the commonest read there is, so that defect raised nothing at any point in the pipeline. It was absent for a good reason, and the reason is the argument, not the name. Measured on wp-statistics, a WordPress plugin that ships jQuery: its assets hold 88 calls named find, of which 70 pass a jQuery selector, 13 pass a predicate to Array.prototype.find and 5 pass a selector built at run time by a template literal or a concatenation. None is a collection read. So the obligation is claimed only where the first argument has the shape of a query document. The operation fact states which argument with query_document_arg, and the arbiter that can see argument shapes decides it: no arguments at all, an object literal, or a builder call are claimed; a method reference, a literal, an operator call and a bare identifier are not. Each exclusion is one of the measured classes above rather than a guess. Zero of those 88 calls became an obligated operation, and the zero is measured rather than empty: the same run's census reports 60 files and 4,552 methods, and the files holding those calls are in it. On NodeGoat all three find calls are collection reads and all three are claimed, which raises the obligation at four more handlers -- the allocations handler that takes userId from req.params on an authenticated route, the benefits pages whose isAdmin guard is commented out in the route table, and the memo listing. Seven of fifteen handlers now reach a verdict, all corroborated, none reported without a judge, and the census is unchanged on both subjects. A bare identifier is deliberately not claimed: col.find(query) and arr.find(pred) are the same shape and neither repository measured holds one, so this is a stated false negative rather than an unmeasured false-positive risk. 1,415 tests pass, sixteen skipped; twelve Joern-gated query tests pass in the image. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo |
||
|
Marius CiepluchandClaude Opus 5
|
102a21fc4e |
feat(access-control): carry the obligation across the route-and-DAO split
The census measured this family completing 35 questions on NodeGoat with no obligation raised anywhere: every modelled data operation lives in app/data/*-dao.js, none in a route handler file, and operations were scoped to the region's own file. The family had nothing to be right or wrong about. The obligation now follows the one named call the handler writes, out of the region's file and no further. How it is followed is the point. jssrc2cpg does not resolve allocationsDAO.getByUserIdAndThreshold(...) to the DAO body: the callee is an external stub with no code. That stub's fullName records where the member was written, so the implementation is reached by the graph's own provenance rather than by a repository-wide name guess, which is the false positive the file scope was closed against. A control asserts it: a third module defining a member of the same name that nothing imports is not attributed. A row is attributed to the region's DECLARED function, which is what lets a question about displayAllocations match a graph that holds a <lambda>. A carried row is also checked for a guard at the CALL, because that is where a handler establishes who is asking. The method holding a carried operation joins the question's context scope, so the module holding the query cannot change under a claim that depends on it without the change being attributed. The witness names both sites and the finding is located at the call, which is the line a contributor changed. A carried obligation corroborates, it does not entail, and that was measured rather than assumed. Entailing it reported handleSignup, which looks a user up by name to check the name is free during registration and is public by design, because two authenticated siblings in the same file pass req.session.userId into the same module. The entailed rung's justification is that one FILE contradicts itself, same file and same trust context; carried, the compared operations live in a data-access module that handlers of different trust contexts share. The corroborated band already asks the right residual: does this operation require a check, or is the route legitimately public? Measured on NodeGoat: 15 handlers asked, obligations raised at zero before and six now, three reaching a verdict, all corroborated, none reported without a judge. VAmPI unchanged at two entailed findings, because it keeps handler and query in one file and carries nothing. What this does not reach is recorded, not implied. NodeGoat's documented insecure direct object reference still raises nothing, because its module reads with allocationsCol.find(...) and find is not a modelled operation: Array.prototype.find and jQuery's .find are everywhere, so it needs an argument-shape discriminator and its own measurement (task 14.8). Two of the three corroborated claims are correct code whose identity read a destructuring spelled away, and the cross-file contradiction that would entail the real defect needs a sibling-scope decision (task 14.7). 1,415 tests pass, fifteen skipped; eleven Joern-gated query tests pass in the image. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo |
||
|
Marius CiepluchandClaude Opus 5
|
a91551e547 |
feat(contributor-scan): give each family its own change context (M1b 12.2)
Access control and configuration could not complete their change context on any repository, which left 231 of 436 questions unresolved on the measured Node case and made two of the three shipped families unable to answer. The gap was larger than a missing query block. The evidence pass dispatches taint only, because only taint has an evidence vector to rank by, so these families were never asked for context at all. A new context pass asks each family with the flag set. Each query describes its own scope, which the owner research required and which matters: copying taint's projection would describe a reachability walk these queries never perform. Dominance exports the file's methods labelled operation, guard or sibling, because its claim is that a file contradicts itself. Configuration exports the file's methods labelled setting or sibling, because it asks what a file declares. Both report an empty scope explicitly rather than implying one. A family whose context query answers nothing keeps no context and stays unresolved, so none can complete on another family's context. A control asserts that directly. Confirmed on a real NodeGoat graph through the production backend: both families produce a context summary and seven scoped methods where they previously produced nothing. All seven are siblings for that file, which is the honest answer since it declares no access-control operation and no matched setting. 1,405 tests passed, nine skipped. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo |
||
|
Marius Ciepluch
|
6ba2b06353 | feat(pre-push-safety-net): exclude vendor inputs and partition frontend coverage | ||
|
Marius CiepluchandClaude Opus 5
|
e0442d146c |
fix(cpg): a graph the frontend emptied must not read as a clean repository
Scanning a two-file WordPress slice returned 64 regions examined, zero
findings and zero degradations. The graph it examined had no methods in it.
php2cpg 4.0.623 feeds several files to one `php-parse` process and reads its
stdout and stderr merged. php-parse writes a `====> File <next>:` banner to
stderr the moment it starts the next file, while the previous file's JSON --
megabytes of it -- is still draining from a block-buffered stdout. The banner
lands inside the JSON, ujson reports
expected json value or } got "=" at index 5011645
and the frontend drops that file. Reproducible on real plugin code, and the
size that matters is one file's AST dump, not the repository's:
class.memberorder.php alone (1834 lines) -> 142KB CPG, no failures
the same file + a two-line file -> 5.7KB CPG, BOTH dropped
One oversized file empties the whole graph. `joern-parse` then does not
propagate any of the frontend's warnings: 13 lines of output, exit status 0,
"Successfully wrote graph". This is what made a 637-file plugin unscannable
and reported it as clean -- it was never a scale limit.
Two fixes, because the failure arrives on two different paths:
* `_unparsed_files` reads the per-file warnings the frontend does emit, and
the scan reports them as `files_unparsed`. This covers the direct-frontend
path where the warnings are visible.
* The batched query now carries a `__census__` of the graph -- method and
file counts -- under a key no request id can collide with, and a scan over
a graph with no methods degrades as `cpg_empty`. This covers joern-parse,
which shows nothing. It rides the batch rather than costing its own
invocation, because avoiding a JVM start is the entire reason batching
exists.
Neither invents a verdict. There is no rung for "the graph was empty" because
the ladder labels verdicts and none was produced, which is exactly why it has
to be a degradation.
pmpro two-file slice before: 0 findings, degradations=[]
after: 0 findings, degradations=[cpg_empty]
Gates byte-identical; 840 tests pass.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
|
||
|
Marius CiepluchandClaude Opus 5
|
92677136d7 |
fix(cpg): a batch goes through a file, not the command line
Scanning a 637-file WordPress plugin reported 500 regions examined, zero
findings, and a taint query time of **0.05 seconds** -- less than a JVM
takes to start. Dominance ran for 29.56s and config for 22.44s in the same
scan, so only taint had failed.
Linux caps a SINGLE argument at MAX_ARG_STRLEN (128KB) whatever ARG_MAX
says. Measured on that plugin:
taint 2500 requests 1.49 MB <- over the cap
dominance 13 requests 0.04 MB
config 13 requests 0.00 MB
As `--param requests=<json>` execve returned E2BIG, `_run` caught the
OSError, `query_batch` returned {} -- and {} reads as "no rows found". Only
taint crossed the cap, which is exactly why the failure looked like a
result rather than an error.
This is the quiet failure the rung ladder cannot catch. Every other honesty
mechanism in this codebase labels a verdict; here no verdict was produced
to label, and a scan of a vulnerable plugin reported it clean.
The payload is now staged to a file beside the CPG and the path is passed
instead; all three queries read `requestsFile` when it is set and keep
`requests` for the single-request forms and the tests. A test asserts no
argument approaches the cap, since the failure mode is invisible at the
scale the unit tests run at -- batching worked on every small tree it was
ever tried on.
Verified on vampi first: 4 entailed, unchanged, all three known
vulnerabilities still found.
Gates byte-identical.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
|
||
|
Marius CiepluchandClaude Opus 5
|
c1191b3a6b |
feat(model): an access-control finding names a line, and what the PDG does not give us
contributor-scan 2.7. Taint and config verdicts have always carried a `path:line:function` location; dominance never did, so every access-control finding reached a contributor as `api_views/users.py:?:update_password` -- the right file by accident, because the region was asked about that file, and no line at all. The query now reports the operation's file and the verdict carries the location, with an engine that cannot report one getting no location rather than an invented one. Two questions answered by measurement while doing it. **Does this factor in the PDG?** The data half, centrally: `reachableByFlows` is the entire taint arbiter and `joern-parse` applies the `dataflowOss` overlay. The control half, not at all -- and it is not available to us. Measured: 0 of 13,184 calls in the libpng CPG and 0 of 1,152 in the vampi CPG have a controlling control structure, so `controlledBy` returns nothing in either. Everything guard-shaped therefore runs on CFG dominance or on syntactic conditions, which is exactly why 2.12's first attempt failed: libpng's bound reaches its sink through an `error` flag, and that is a control-dependence fact and nothing else. Recorded as 2.14, including the possibility that it cannot be enabled -- which would bound what guard reasoning this engine can ever do. **Would checking out a vulnerable libpng find a known CVE?** No, and not because of the version. Every one of the twelve files using `strcpy`/`strcat`/`sprintf`/`gets` is under `contrib/`; the library itself uses none of them. libpng overflows through array and pointer arithmetic and `memcpy` sizing, and five of its buffer-overflow fixes touch no modelled sink at all. Pinning an older commit would produce the same silence for the same reason. That leaves two routes of very different size, recorded as 2.15: pin a C project whose CVE genuinely runs through a string function, which is cheap and would give the memory family its first true positive; or model how C actually overflows, which is size arithmetic rather than sink matching and should not be started by accident. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
d15985f695 |
fix(model): a guard discharges the obligation it is a guard of
contributor-scan 2.6. Access control entailed 7 of 9 VAmPI handlers,
including three that do authorize and three the OpenAPI spec declares
public. Five separate defects, each invisible to a corpus of one-function
excerpts and each found by asking the query what it actually matched.
**A trailing segment is not a match.** `query.get` was matched by its last
segment, so every `.get(` in the repository became a protected database
operation -- `request.headers.get('Authorization')` and
`request_data.get('username')` included. Four of the seven false positives
came from this, and so did the apparent detection of the known BOLA: it was
entailed because fetching the Authorization header had no dominating guard,
not because of the defect. A qualified token now requires its qualifier.
**Configuration is not authorization.** dominance_specs took every
obligation fact. `security_setting` operations are discharged by a VALUE,
which config_specs already claims them for; pooled in here they made
`app.run(...)` an operation requiring an authorization guard and put
thirteen tokens in BOTH lists -- an obligation that is its own discharge,
the same shape as the taint sink that used to sanitize itself.
**Input validation is not authorization.** `jsonschema.validate` was
counted as a guard, so handlers with no authorization check at all read as
guarded. No operation in the table requires validated_input.
**A guard discharges the obligation it is a guard of.** The largest one.
`orm-read` requires an identity_constraint or an ownership_check;
`token_validator` is a path_guard. Flattened into one list, authenticating
the caller discharged an object-level obligation -- a handler that
authenticates and then looks the record up by a PATH parameter read as
fully guarded. The facts state that relation and the spec was discarding
it; it is now carried and the engine asks per operation.
**A word boundary needs a word.** Found while fixing the above. Guard
matching wrapped every token in `(?<![A-Za-z0-9_])...(?![A-Za-z0-9_])`.
The identity token `['sub']` begins with `[`, and the character before it
in `resp['sub']` is `p` -- so the assertion could never hold. Every
identity token starting with punctuation was unmatchable, and the identity
constraint that separates a correct lookup from an IDOR was invisible.
Also: dominance rows were repository-wide, so the model's unguarded
`delete_user` was attributed to the API handler of the same name. Rows are
scoped to the region's file, which is what the rung's own justification
says -- the evidence is a file contradicting itself.
Measured against VAmPI's OpenAPI `security` blocks, which declare per
operation whether auth is required, and agree with the code:
vampi precision 28.6% (2/7) -> 50.0% (2/4)
known found 3 of 3, all model_entailed
And on the labelled access-control pairs, which nobody tuned this against:
all 9 pairs pair_correct 3 -> 3, leak 44.4% -> 11.1%
7 non-VAmPI pair_correct 2 -> 2, leak 42.9% -> 0.0%
Every detection lost was on a pair whose FIXED side was flagged too, which
scores nothing. No correct detection was traded for the precision.
get_by_title was flagged before it was in the recipe: its code carries the
comment "# Broken Object Level Authorization" and the README lists it, so a
real finding was being scored as a false positive. It is now the recipe's
third known vulnerability.
Both remaining false positives are endpoints the spec declares public.
Recorded as 2.8, with the reason the mapper's access_level cannot simply be
trusted: for decorator frameworks "public" means "no decorator found",
which is the bug itself.
Gates byte-identical.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
|
||
|
Marius CiepluchandClaude Opus 5
|
c35426e186 |
perf(model): one Joern invocation per query kind, not one per region
JVM startup dominated a scan: a call per region per family put a ten-line file at 275s over three regions, and would have put a thousand regions at roughly fifty hours. The queries themselves are milliseconds once the CPG is loaded, so the cost was almost entirely process starts. scan_repository() is now collect -> batch -> arbitrate. The three .sc scripts accept a `requests` JSON map and answer all of them in one run, keeping the single-request form so the committed measurements stay reproducible byte-for-byte. Arbiters are untouched: each is handed a CpgResult whose `run` returns its own prefetched rows. The batcher builds parameters by calling request_params() extracted from taint.py, dominance.py and config_value.py rather than reconstructing them. A second copy that drifts from the arbiter's is the exact shape of the five family-assumption bugs the predecessor spec spent a session finding; there is now one definition per family and the batcher borrows it. Regions are also deduplicated by (path, function), collapsing the nameless entry-point rows that were producing `model:injection:?:10`. Measured on the same ten-line file: 275s -> 106s, three duplicate findings -> one `model:injection:ping:10`, rung unchanged at model_entailed. Gates byte-identical. Server mode is recorded as task 2.3, deliberately sequenced after the repository measurement: whether the remaining ~100s floor is worth a process model, a weakened subprocess seam and a Scala-evaluating HTTP endpoint depends on a build-time-to-query-time ratio only 2.2 produces. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
eebca3f663 |
fix(model-grounded-detection): the absence arbiter was silencing its own family
Two bugs, together making 4.1 report "no bug" on a textbook IDOR. Both found by running it against a real corpus pair rather than a fixture. 1. constraint_params were treated as dischargers. A field named `user_id` is what an IDOR is MADE OF -- `filter_by(id=user_id)` with user_id off the request is the bug, not the fix. The deleted checker only counted an identity constraint whose value's PROVENANCE was the authenticated context, so the parameter name alone evidences nothing. This is the mirror of group 2's sanitizer bug: a thing that must never count as a discharge, counted. 2. Guard tokens matched as substrings. With `user` in the token list, the string "SELECT * FROM users WHERE id = ?" contains `user`, so the query reported every operation in the handler as guarded. Matching is word-boundary now. On threatbyte-api-v1-delete the arbiter previously found three operations, all "guarded" by evidence like '"SELECT * FROM users WHERE id = ?"' and '(user_id,)', and returned no verdict at all. After the fix, on the real pairs: threatbyte-api-v1-delete vuln=model_corroborated fixed=none vampi-users-update-password vuln=model_corroborated fixed=none Both distinguished, deterministically, with no model call. Corroborated rather than entailed is right for these: they are single-handler excerpts with no guarded sibling to be inconsistent with, which is the asymmetry the rung distinction exists to express. identity_sources still discharge; request_sources and permissive_values remain excluded because they describe the vulnerable shape rather than the fix. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
fec67253f3 |
feat(model-grounded-detection): 4.1 guard dominance, 4.2 the missing sink facts
4.1 is the arbiter for the class no crash oracle reaches. An access-control bug
is the ABSENCE of a guard: no flow to follow, no crash to reproduce. What the
model can establish is a consistency violation -- an obligated operation no
discharging guard governs, in a file where its siblings are governed by one.
"Governed" turned out not to be one relation, and asking pure CFG dominance of
the operation gets two of three shapes wrong. Measured against the real engine:
identity constraint the guard is INSIDE the operation's arguments
filter_by(owner_id = current_user.id)
identity branch abort(403).controlledBy == [note.owner_id != current_user.id]
-- the check runs AFTER the fetch, so it never dominates
the operation, but it controls whether the value escapes
dominating guard a discharger-naming call that dominates the operation
That is why the query uses controlledBy rather than dominatedBy for shape 2.
Two bugs found by running it, both the same class as the sanitizer bug in group
2 -- part of a fact flattened, the part carrying the signal dropped:
* dominance_specs used only `calls` and `decorators`, omitting
identity_sources (current_user, request.user) and constraint_params
(owner_id, user_id) -- which is how an identity constraint is actually
written. The arbiter reported "no guard" on handlers that plainly had one.
request_sources and permissive_values stay excluded: they describe the
vulnerable shape, and including them would let the arbiter read a bug as
its own fix.
* matching operations by code containment yielded four or five desugared rows
per operation (the bare `tmp1.filter_by` receiver among them, carrying none
of the guards the real call has), so a guarded operation could be flagged.
Matching is by call name now.
Live, deterministic, zero model calls:
leaky -> model_entailed no discharging guard, while 2 sibling handlers are guarded
edit_note -> no finding (ownership check controls the denial)
list_notes -> no finding (identity constraint in the query)
4.2 authors the sink facts the go/no-go measured as its largest miss class (11
of 23): Django ORM raw SQL, and the four families that previously had no fact
routing to them by CWE at all -- path, output_encoding, untrusted_destination,
prototype. Each entry is a documented dangerous API of the named library with
its provenance recorded inline, and rule_ids left empty because a sink fact is
not a detection rule. Flagged for maintainer review as data, given the standing
"no LLM-authored patterns" constraint. The exact-set assertion written in task
1.1 to hold this gap open is updated rather than deleted, so it still fails if a
family ever loses its seed.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
|