mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-18 12:39:28 +00:00
c2bfcef3df9ca9c5fd1bfce89f05485eca6c8c54
112
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Marius CiepluchandClaude Opus 5
|
0fcd5feade |
feat(model): a region for the code that is in no function
contributor-scan 2.5. VAmPI's `host='0.0.0.0'` is now reported exactly once, at app.py:17:<module>. Before task 2.2 it was reported eight times in eight files that do not contain it; after 2.2 it was reported nowhere. Two causes, one shape: nothing could ask about module-level code. **The main-guard marker is not a function.** `__main__` is what the mapper emits for `if __name__ == '__main__':`. Sent to the engine as a function name it matched nothing, because a CPG calls a file's top-level code `<module>` -- verified against the VAmPI CPG rather than assumed. The region model translates the marker, which is the one place that turns what the mapper meant into what the scanner asks. **A function-scoped file had no region for its module body.** Settings, route registration and globals live in no function, and every region's query filters by function name. A file-level region beside the handlers would report each handler's findings a second time, which is why the earlier rule dropped it as "a second full-file pass". A module-scope region PARTITIONS the file instead of overlapping it. That only became cheap after batching: before it, a region was a JVM. Two things this needed to be correct rather than merely present: - A module method spans its whole file, so the taint query's line-range containment made `<module>` the parent of every function in it. Under module scope, membership is direct -- otherwise the module region reports every handler's sinks again, which is the duplication the old rule was avoiding. - A module body is not a handler. The marker-derived region inherited the handler families and entailed a missing authorization check on the main guard: a finding with no possible remediation. Caught by the first repository run that had the region, and now covered by a test. bind address 8 wrong files -> 0 -> 1, at app.py:17:<module> regions 22 -> 25 entailed 8 -> 9 query time 37.4s -> 41.3s peak RSS 827MB -> 823MB cost $0.0067 -> $0.0068 Three more regions for four seconds and no more model calls: the budget is a total and was already spent, so regions_unasked rose 17 -> 20 -- those regions got the free pass and are reported as unasked, not as clean. `<module>` is Joern's Python name for a module body; JavaScript uses `:program`, so this is verified for Python only. Gates byte-identical. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
ca7b5ba370 |
feat(model): a region that spans the flow, not just a function
contributor-scan 2.4, closing the no-go task 2.2 recorded. VAmPI's SQL injection is a request parameter reaching `get_by_username` in api_views/users.py, passed to `User.get_user` in models/user_model.py, interpolated into a query string and executed there. Source and sink are in different functions and different modules. Regions were per-function, so neither region saw both ends and the engine was silent on the most ordinary shape a real injection bug has. `reachableByFlows` was never the limitation -- it follows calls already. The limitation was the QUESTION. Sinks were filtered to the labeled function's own body, so a crossing flow had no sink to end at. Three changes, each with a reason it is sound rather than convenient: **An entry point is the trust boundary.** parameter_sources was off for repository scans because a repository's parameters are not all attacker input. An entry point's are, by definition -- that is what makes it an entry point. The driver turns it on only for regions the mapper found and named, and leaves it off for walker fallbacks and for named functions that are not entry points. **Sinks may sit below the handler.** taint.sc admits sinks in methods reachable from the labeled method through the call graph, bounded at ENTRY_POINT_CALL_DEPTH = 3: handler -> service -> data access, which is where these bugs live. Unbounded would make the question about most of the repository rather than about this entry point, and make it unaffordable. **The finding names the sink, not the asker.** A crossing flow ends in another file and another function, so Verdict carries the evidence's own `path:line:function` and the site uses it. Reporting `models/user_model.py:73:get_by_username` would name a place that does not exist: line 73 is inside get_user. Also: labeled methods are now resolved by name AND file. `get_user` exists in more than one module of a real repository, which a corpus of one-file excerpts could never surface. Measured on the same checkout and budget as 2.2: VAmPI-SQLI missed -> model_entailed at models/user_model.py:73:get_user known found 0 of 2 before 2.2's fixes, 1 of 2 in 2.2, 2 of 2 now entailed 7 -> 8 query time 40.9s -> 37.4s peak RSS 869MB -> 827MB cost $0.0067, unchanged Asking the bigger question cost nothing measurable here: file scoping and call-graph following pull in opposite directions and roughly cancel at this size. That is one 520-line repository, and VAmPI is now the repository this change was developed against -- the exact overfitting shape Req 8 exists to catch. The depth bound and the entry-point rule mean nothing general until a checkout nobody tuned against says so. Gates byte-identical. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
177c7f3719 |
feat(model): first repository measurement, and the four things it found
contributor-scan 2.2. VAmPI at a pinned commit, scanned whole rather than as excerpts, with benchmarks/repos/measure.py recording build time, query time, arbitration time, peak memory, cost, unjudged regions, and whether each known vulnerability was found and at which rung. The corpus could not have found any of this. Every defect below needs a repository to become visible, and three of them were invisible behind a passing test suite. **The batch path was unreachable.** query_batch was implemented, tested and committed while CpgResult exposed only `run`, so the driver's capability check found nothing and fell back to one JVM per region-family: the first attempt at this measurement ran eighteen minutes on 520 lines, issuing `--param function=` one at a time. Every test called query_batch directly, so none of them noticed. CpgResult now carries run_batch and a test asserts the capability is reachable through the object the driver is handed. The earlier 275s->106s improvement attributed to batching came from region deduplication alone. **The budget stopped free work.** It bounds model calls, but was enforced by breaking the region loop -- spending a paid limit to stop unpaid CPG arbitration. 6 of 22 regions were examined and VAmPI's BOLA, which the graph entails for nothing, was never reached. It now withholds the judge only. 22 of 22 regions arbitrated, 17 counted as regions_unasked (silent because unaffordable is not silent because decided), and the BOLA is entailed by the model layer. **A region with no function asked about the whole repository.** Sending function="" matched every hit in the tree and the driver attributed that answer to whoever asked: one `host='0.0.0.0'` in app.py became eight identical entailed findings in eight files that do not contain it. Queries now carry the region's file. Entailed findings 14 -> 7, query time 70.9s -> 40.9s. The single-request form stays unscoped so the committed pair measurements remain reproducible. **The JVM had no heap bound.** Joern claimed a quarter of physical RAM and runs were killed under memory pressure at different points, so the numbers described the host rather than the engine. -J-Xmx2048m by default, OPENULTRASAST_CPG_HEAP_MB to raise it. Also: arbitrated findings carried `function:line` where the report layer read `path:line`, so every entailed finding reached a contributor with a function name where its file belonged; and the witness line was parsed by splitting on a comma, which produced sites like `config.py:17): permissive literal '0.0.0.0':?`. Reading: GO on one CPG per repository -- 6.2s to build, 177s end to end, 869MB, $0.0067, nothing infeasible. NO-GO on region-scoped taint as the only shape: VAmPI's SQL injection runs from a handler parameter into a function in another module, and a per-function region sees neither end. That is the ordinary shape of a real injection bug, so it is recorded as task 2.4 along with three smaller defects this run exposed. Gates byte-identical. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
c35426e186 |
perf(model): one Joern invocation per query kind, not one per region
JVM startup dominated a scan: a call per region per family put a ten-line file at 275s over three regions, and would have put a thousand regions at roughly fifty hours. The queries themselves are milliseconds once the CPG is loaded, so the cost was almost entirely process starts. scan_repository() is now collect -> batch -> arbitrate. The three .sc scripts accept a `requests` JSON map and answer all of them in one run, keeping the single-request form so the committed measurements stay reproducible byte-for-byte. Arbiters are untouched: each is handed a CpgResult whose `run` returns its own prefetched rows. The batcher builds parameters by calling request_params() extracted from taint.py, dominance.py and config_value.py rather than reconstructing them. A second copy that drifts from the arbiter's is the exact shape of the five family-assumption bugs the predecessor spec spent a session finding; there is now one definition per family and the batcher borrows it. Regions are also deduplicated by (path, function), collapsing the nameless entry-point rows that were producing `model:injection:?:10`. Measured on the same ten-line file: 275s -> 106s, three duplicate findings -> one `model:injection:ping:10`, rung unchanged at model_entailed. Gates byte-identical. Server mode is recorded as task 2.3, deliberately sequenced after the repository measurement: whether the remaining ~100s floor is worth a process model, a weakened subprocess seam and a Scala-evaluating HTTP endpoint depends on a build-time-to-query-time ratio only 2.2 produces. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
d651455d23 |
fix(model-grounded-detection): closure scoping, and the three bugs it uncovered
The group-4 finding: JavaScript is callback-heavy, so the method containing a
sink routinely is not the labeled function. `fs.readFileSync` inside the
callback passed to `fs.stat` has enclosing method `<lambda>0`, and filtering on
`sinkMethod == function` dropped it -- the whole path family measured 0/8.
Scope is now decided by line-range containment within the same file
(astParentFullName is empty for lambdas, so nesting is not readable there), and
reported as `inLabeledScope`; the Python side trusts that flag and keeps the
name comparison only as a fallback, so an engine that cannot report scope never
has its sinks silently treated as scoped. Parameter sources stay bound to the
labeled function's OWN parameters -- a callback's `err`/`stats` are not
attacker input.
Fixing the scope exposed three further bugs, each of which would have raised
the number while making the answer untrue:
* Shortest-flow-wins picked the wrong witness. With closures in scope the
same sink has flows from `req.url...` (10 steps, the vulnerability), `res`
(3) and `this` (7). A flow whose source matches the source model now
outranks a shorter one from a bare captured name; shortest still breaks
ties, so the order stays total and the verdict deterministic.
* The JS fact file had NO sanitizers at all, so no JavaScript fix of any
family could ever be recognised. Added the documented path-normalisation
and HTML-escaping APIs, with provenance.
* Adding them regressed the witness, because `resolveUrl` CONTAINS `resolve`
and the substring test marked the vulnerable flow sanitized. This is the
same bug class already fixed in dominance.sc and not carried across -- the
exact fix-the-instance-miss-the-class failure. taint.sc now matches on word
boundaries too.
And the principled rule the fix turns on: parameter sources are a FALLBACK,
never an override. When a modelled source reaches the sink, those flows are the
evidence; otherwise a captured `res` reaching a sanitized sink reports the fix
as still vulnerable on a flow that names no attacker input.
Also recorded: req.url was NOT wrong as a diagnosis, it was untestable in
isolation -- the scoping bug masked it. Both were needed. On the pair that
motivated all of this:
vuln model_entailed req.url.split('?')[0] -> readFileSync(possibleFilename)
fixed model_corroborated same flow, sanitized by path.normalize
distinguished: true
700 passing, mypy/ruff clean, gates byte-identical.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
|
||
|
Marius CiepluchandClaude Opus 5
|
eebca3f663 |
fix(model-grounded-detection): the absence arbiter was silencing its own family
Two bugs, together making 4.1 report "no bug" on a textbook IDOR. Both found by running it against a real corpus pair rather than a fixture. 1. constraint_params were treated as dischargers. A field named `user_id` is what an IDOR is MADE OF -- `filter_by(id=user_id)` with user_id off the request is the bug, not the fix. The deleted checker only counted an identity constraint whose value's PROVENANCE was the authenticated context, so the parameter name alone evidences nothing. This is the mirror of group 2's sanitizer bug: a thing that must never count as a discharge, counted. 2. Guard tokens matched as substrings. With `user` in the token list, the string "SELECT * FROM users WHERE id = ?" contains `user`, so the query reported every operation in the handler as guarded. Matching is word-boundary now. On threatbyte-api-v1-delete the arbiter previously found three operations, all "guarded" by evidence like '"SELECT * FROM users WHERE id = ?"' and '(user_id,)', and returned no verdict at all. After the fix, on the real pairs: threatbyte-api-v1-delete vuln=model_corroborated fixed=none vampi-users-update-password vuln=model_corroborated fixed=none Both distinguished, deterministically, with no model call. Corroborated rather than entailed is right for these: they are single-handler excerpts with no guarded sibling to be inconsistent with, which is the asymmetry the rung distinction exists to express. identity_sources still discharge; request_sources and permissive_values remain excluded because they describe the vulnerable shape rather than the fix. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
d9ceeb6a09 |
feat(model-grounded-detection): 4.2 constant abstraction, the third arbiter form
A configuration bug has no flow to follow and no guard to dominate.
CORS(app, origins="*") is dangerous because of the value it is SET TO, so the
arbiter evaluates the setting's arguments against the closed permissive set in
the obligation facts -- the third form beside taint reachability and guard
dominance, and the one Req 6.3 names.
ENTAILED a security setting is passed a literal in the permissive set
CORROBORATED the argument is computed, so constant abstraction cannot decide
it -- that residual is the judge's, never silently "safe"
(none) every literal is outside the set, or the call is not a setting
Two Joern node-shape bugs, both found by running it and both of the same class
as group 2's receiver bug -- the graph's indexing is not what the source text
suggests:
* a keyword argument (`origins="*"`) is an assignment-shaped node, so
isLiteral is false on the argument itself and the literal sits in its
subtree; literals are collected from the argument's AST now.
* Joern indexes the receiver at 0, positional arguments from 1, and NAMED
arguments at -1. Settings are configured overwhelmingly by keyword, so
argumentIndexGt(0) -- correct when counting arity for the taint shape test
-- dropped exactly the arguments that matter and every keyword-configured
setting read as "computed".
Live, deterministic, zero model calls:
create_app -> model_entailed CORS(app, origins = "*"): permissive literal "*"
safe_app -> no finding (origins = "https://app.example")
All three arbiter forms now work against a real CPG: taint reachability for the
flow families, guard dominance for absence, constant abstraction for config.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
|
||
|
Marius CiepluchandClaude Opus 5
|
fec67253f3 |
feat(model-grounded-detection): 4.1 guard dominance, 4.2 the missing sink facts
4.1 is the arbiter for the class no crash oracle reaches. An access-control bug
is the ABSENCE of a guard: no flow to follow, no crash to reproduce. What the
model can establish is a consistency violation -- an obligated operation no
discharging guard governs, in a file where its siblings are governed by one.
"Governed" turned out not to be one relation, and asking pure CFG dominance of
the operation gets two of three shapes wrong. Measured against the real engine:
identity constraint the guard is INSIDE the operation's arguments
filter_by(owner_id = current_user.id)
identity branch abort(403).controlledBy == [note.owner_id != current_user.id]
-- the check runs AFTER the fetch, so it never dominates
the operation, but it controls whether the value escapes
dominating guard a discharger-naming call that dominates the operation
That is why the query uses controlledBy rather than dominatedBy for shape 2.
Two bugs found by running it, both the same class as the sanitizer bug in group
2 -- part of a fact flattened, the part carrying the signal dropped:
* dominance_specs used only `calls` and `decorators`, omitting
identity_sources (current_user, request.user) and constraint_params
(owner_id, user_id) -- which is how an identity constraint is actually
written. The arbiter reported "no guard" on handlers that plainly had one.
request_sources and permissive_values stay excluded: they describe the
vulnerable shape, and including them would let the arbiter read a bug as
its own fix.
* matching operations by code containment yielded four or five desugared rows
per operation (the bare `tmp1.filter_by` receiver among them, carrying none
of the guards the real call has), so a guarded operation could be flagged.
Matching is by call name now.
Live, deterministic, zero model calls:
leaky -> model_entailed no discharging guard, while 2 sibling handlers are guarded
edit_note -> no finding (ownership check controls the denial)
list_notes -> no finding (identity constraint in the query)
4.2 authors the sink facts the go/no-go measured as its largest miss class (11
of 23): Django ORM raw SQL, and the four families that previously had no fact
routing to them by CWE at all -- path, output_encoding, untrusted_destination,
prototype. Each entry is a documented dangerous API of the named library with
its provenance recorded inline, and rule_ids left empty because a sink fact is
not a detection rule. Flagged for maintainer review as data, given the standing
"no LLM-authored patterns" constraint. The exact-set assertion written in task
1.1 to hold this gap open is updated rather than deleted, so it still fails if a
family ever loses its seed.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
|
||
|
Marius CiepluchandClaude Opus 5
|
e127e430ed |
feat(model-grounded-detection): parameter sources, so the measurement is like-for-like
Only 11 of the 50 injection pairs contain a framework source token at all. The
other 39 receive untrusted input as a FUNCTION PARAMETER, because SVEN-style
pairs are function-level extractions:
def add_user(username, password):
os.system("useradd ... -p " + encPass + " " + username)
A framework-anchored source model finds nothing there, which caps entailment on
this slice near 22% by construction -- a property of the source model, not of
the engine. More to the point, the flat-IR baseline this feature is measured
against already counted these (source_kinds: ["parameter"]), so a run without
them is not comparable to the 2.2% it is supposed to beat.
The flag is opt-in and off by default, because the two settings are right in
different places: in a function-level pair the function boundary IS the trust
boundary, while across a whole repository most parameters carry internal
values and treating them all as untrusted is not sound. The corpus harness
opts in; a repository scan does not.
On the pair that exposed this the model now resolves both flows, including one
through an intermediate binding:
username -> os.system(...) 2 steps
password -> crypt.crypt(...) -> encPass -> ... 7 steps
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
|
||
|
Marius CiepluchandClaude Opus 5
|
1765e51da1 |
feat(model-grounded-detection): the safe-shape test, without which the model arbitrates nothing
The first live run produced the finding that shapes this design: taint
reachability alone does not distinguish a fix from a bug. On the canonical SQL
pair the flow request.args["name"] -> name -> execute exists on BOTH sides --
vuln db.execute("select ... '" + name + "'") one argument, interpolated
fixed db.execute("select ... = %s", (name,)) two arguments, bound
-- so the model entailed the fixed twin exactly as readily as the vulnerable
one, and arbitrated nothing. What separates them is the SHAPE of the sink call,
which is precisely what safe_shape_sinks preserved from the earlier sanitizer
fix. The query now reports sinkArity and sinkArg0Literal, and a flow ending at
a sink written in its declared-safe form is not a vulnerability.
Two bugs in that shape test, both found only against the real engine:
Joern's `call.argument` includes the RECEIVER at argumentIndex 0, so `db` was
counted as an argument -- making a one-argument interpolated call look like a
two-argument bound one, the exact inversion of the test -- and the "first
argument" whose literalness decides a format-string sink was the receiver
rather than the format. Both fixed with argumentIndexGt(0).
The rule is narrow on purpose: only sinks named in safe_shape_sinks are subject
to it (os.system(x) is not safe merely because it takes one argument), and a
row carrying no shape fields is judged on the flow alone -- an unknown shape
must never be read as a safe one.
End to end on the pair, deterministic and with no model call:
vuln -> model_entailed request.args["name"] -> db.execute(...) (line 4)
fixed -> None
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
|
||
|
Marius CiepluchandClaude Opus 5
|
f59e0f5b5a |
fix(model-grounded-detection): two modelling bugs the first live Joern run exposed
Both were found by running the real engine, and neither was visible offline.
1. A sink was its own sanitizer. The shipped facts mark some sanitizers with a
*shape* qualifier -- parameterized ("execute(sql, params) binds rather than
interpolates") and literal_format_arg ("printf with a constant format string
is safe"). These describe a safe FORM OF THE SINK CALL, not a call that
cleans a value on the path. Flattening them into the sanitizer list made
`execute` both the injection sink and its own sanitizer, so every SQL flow
would have reported as already-clean and nothing could ever be entailed.
Fixed as a class, not an instance: any fact carrying a shape qualifier is
excluded from `sanitizers` and kept in `safe_shape_sinks` for the shape test
task 4.2 will model properly. Fixing only the Python `parameterized` case
would have left C's entire printf family -- which the generalised test then
caught immediately.
2. labeled_family lost its deterministic derivation in group 1. The deleted
classifier resolved a pair's family by declared label, else obligation, else
mechanism, else CWE, and only asked an LLM when every link failed -- which
for a labeled corpus was never. Group 1 reduced this to the declared label
alone, which silently emptied every family whose pairs are labeled by CWE:
the injection slice went to zero pairs. The chain is restored, still with no
model call, and the family census again matches the committed entailment
ceiling exactly (injection 50, memory 176, access_control 8, ...).
Also: the Joern seam now runs in the CPG's own scratch directory, because
`joern --script` writes a `workspace/` beside the working directory and would
otherwise leave one inside the repository being analysed.
The taint query itself was corrected against the live engine: the sink is the
call's ARGUMENTS (data flows into arguments, not into the call node) and
reachableByFlows is called on the sink with the source. With that, the
three-hop chain our flat IR could not resolve is found:
request.args["cmd"] -> cmd -> "echo " + cmd -> full -> os.system(full)
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
|
||
|
Marius CiepluchandClaude Opus 5
|
2b973dc0db |
feat(model-grounded-detection): tasks 2.1-2.2, the Joern seam and the taint verdict
The whole engine integration is one subprocess boundary: joern-parse builds a cpg.bin, joern --script runs a shipped CPGQL query that prints its JSON between fence markers, because Joern's stdout carries a banner and a REPL prompt around whatever a script emits. Nothing is imported in-process, so the core install keeps dependencies = [] -- a test walks the seam's AST to assert it imports only the standard library and names no JVM binding. Fail closed everywhere. A build or query that errors, times out, or prints something unparseable yields no verdict and the caller reports suspicion. The taint verdict distinguishes this from absence deliberately: a query returning None means "the engine could not decide", which is not "there is no flow", and conflating them is how a missing engine silently becomes a clean bill of health. The verdict is where the ladder earns its keep. An unsanitized source->sink path is model_entailed -- the graph establishes it and the LLM's claim is no longer load-bearing. A path a sanitizer may break is model_corroborated: the flow is real, but whether the sanitizer suffices is the semantic residual the LLM answers, checked against this verdict rather than averaged over runs. The chosen flow is picked by a total order (shortest, then lexicographic), so two runs over one CPG cannot disagree -- Req 6.4 asserted directly. The query itself is the part our flat IR could not do: reachableByFlows walks the PDG, so `q = "..." + request.args["x"]; execute(q)` is found, which is exactly the one-hop limit that held measured entailment at 2.2%. Capability detection reuses semantic/engines.joern_available rather than growing a second notion of "installed". The audit manifest is regenerated; the new modules are named by Req 4/5/6 rather than counted as orphans. Task 2.3, the go/no-go measurement, needs Joern installed and has not run. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |