diff --git a/.kiro/specs/pre-push-safety-net/status.md b/.kiro/specs/pre-push-safety-net/status.md index 6f8279f..6c33b92 100644 --- a/.kiro/specs/pre-push-safety-net/status.md +++ b/.kiro/specs/pre-push-safety-net/status.md @@ -134,3 +134,22 @@ next implementation prerequisites, not completed fixes or changed acceptance gat An isolated Joern session probe matched the full Node census with four warm requests in 0.221–0.467s and verified cancellation/reaping in 0.043s. It identifies a promising engine lifecycle change; full-query equivalence and changed-code performance remain unverified. + +## Family detection census, 2026-09-19 + +Four of seven declared families establish findings on the two development subjects: +injection, access_control, config_secrets and untrusted_destination. Output encoding, path, +prototype and deserialization have established nothing anywhere. Every question completed on +both subjects, 299 of 299 on NodeGoat and 35 of 35 on VAmPI, so the silences are answers +rather than failures — but three of them are diagnosed gaps rather than clean code: +NodeGoat's documented SSRF is invisible because `needle.get` is not in the sink vocabulary, +its XSS is invisible because output encoding declares only DOM sinks, and its two +access-control defects raise no obligation because the modelled data operations live in DAO +files while the family's scope is the handler's own file. + +The census also found and fixed one false positive: NodeGoat's only prototype report was +the word `set` in a code comment, matched because the sink text clause searched a whole +assignment node. Nothing about this changes the release gates. No capability is qualified by +these subjects. + +Evidence: [family detection census](../../../benchmarks/measurements/2026-09-19-family-detection-census.json). diff --git a/.kiro/specs/pre-push-safety-net/tasks.md b/.kiro/specs/pre-push-safety-net/tasks.md index 4007543..736358d 100644 --- a/.kiro/specs/pre-push-safety-net/tasks.md +++ b/.kiro/specs/pre-push-safety-net/tasks.md @@ -462,6 +462,45 @@ Preserve all completed task evidence below. - _Depends: 13.1_ - _Requirements: 6.2, 6.3, 8.1, 8.2_ +- [ ] 14. Ask every declared family whether it establishes anything (precondition for M4) + - Seven families are declared and two were proven end to end. A declared family that has never been asked whether it finds anything is advertised coverage, and M4 cannot qualify a capability that establishes nothing. +- [x] 14.1 Census the seven declared families on NodeGoat and VAmPI + - Run production discovery and scan over each whole checkout and report, per family, the regions that offered it, the questions asked, the questions completed, and what was found with the witness and rung. No comparison, no novelty, no admission: detection only. + - A family that answered and found nothing must be reported differently from one that was never asked, and neither may read as clean. + - _Verified 2026-09-19:_ 1,411 tests passed, ten skipped; the six Joern-gated query tests pass in the image. Four families establish findings across the two subjects: injection (three `eval` flows on NodeGoat, one SQL flow on VAmPI), access_control (two sibling contradictions on VAmPI), config_secrets (one on each) and untrusted_destination (one open redirect on NodeGoat). All 299 NodeGoat and 35 VAmPI questions completed. Evidence: `benchmarks/measurements/2026-09-19-family-detection-census.json`. + - _The census found one false positive and it is fixed._ NodeGoat's only prototype report was the word `set` inside the comment `// set these up in case we have an error case`: the matched node is the `.assignment` holding the whole 1,000-character signup handler, and its text was searched entire. `sinkMatches` now confines its text clause to the callee and excludes operator nodes, which is the fourth instance of this bug class after `fgets`/`gets`, `resolve`/`resolveUrl` and `user`/`users`. Measured on NodeGoat's non-vendor calls: injection sinks 28 to 3, prototype 13 to 2, untrusted_destination 26 to 7, with no true positive lost. `req.query` had been an injection sink, because `query` is word-bounded inside it. + - _Boundary: contributor-scan detection, Evaluation harness_ + - _Requirements: 6.2, 6.3, 8.1_ +- [ ] 14.2 Give untrusted destination the sinks a Node server actually calls + - **Named by the 14.1 census.** NodeGoat's documented SSRF is `needle.get(req.query.url + req.query.symbol)` at `app/routes/research.js:16`, and `needle.get` is absent from a vocabulary holding axios, fetch, got, `http.get` and request. The family works and found the open redirect in the same repository; it could not ask about the request. + - This is a vocabulary gap, so fix it as data rather than as a query change, and state which client libraries the declared set now covers. + - Done when the research handler's request is asked about, with a witness naming the call, and no benign client call is reported. + - _Boundary: shipped semantic facts_ + - _Depends: 14.1_ + - _Requirements: 6.2, 8.1_ +- [ ] 14.3 Decide what output encoding claims for server-rendered JavaScript + - **Named by the 14.1 census.** All six declared JavaScript sinks are DOM sinks. NodeGoat's XSS is server-side, rendered into Swig templates and written with `res.write`, so the 44 questions this family completed could not have found it whatever the code did. + - Either the family gains server-side sinks and template-rendering sinks, or it declares itself client-side only. Both are honest; silence while declaring XSS coverage is not. + - Done when one record states the decision, and the family either asks about `res.write` and template rendering or says in its declaration what it does not cover. + - _Boundary: shipped semantic facts, family taxonomy_ + - _Depends: 14.1_ + - _Requirements: 6.2, 8.1_ +- [ ] 14.4 Raise the access-control obligation where the application splits route from query + - **Named by the 14.1 census and measured with the real dominance query.** Every modelled data operation in NodeGoat lives in `app/data/*-dao.js`; none lives in a route handler file. Operations are scoped to the region's own file and access_control regions are the handler files, so no obligation is raised where the question is asked, and inside the DAO files every operation is equally unguarded so the sibling rung has nothing to contradict. The family answers correctly on VAmPI, which keeps handler and query in one file. + - The file scope was chosen deliberately, because unscoped rows attributed a model's unguarded query to a handler of the same name. Widening it is not the fix; carrying the obligation along the call the handler actually makes might be. + - NodeGoat's two documented defects are also outside what the model can claim: an authenticated handler reading `userId` from `req.params` (`app/routes/allocations.js:17`), and a commented-out `isAdmin` in the route table (`app/routes/index.js:58`). Record which of the two this task is for, and do not silently widen the claim to the other. + - Done when a route-and-DAO split raises the obligation at the handler with a witness naming both sites, and a same-name model function is still not attributed to a handler. + - _Boundary: contributor-scan dominance, Ranker Scope Contract_ + - _Depends: 14.1_ + - _Requirements: 6.2, 6.3, 8.1_ +- [ ] 14.5 Ask the prototype and deserialization families somewhere they can answer + - **Named by the 14.1 census.** Prototype pollution has been asked on one repository and its only report was a false positive; deserialization was never asked at all, because VAmPI holds no such sink and NodeGoat is not a language where the family is declared. Neither has established anything anywhere. + - Find or record a subject where the family has a sink, and report what it does. A negative on a subject that contains the mechanism is a result; silence on a subject that does not contain it is not. + - _Boundary: Evaluation harness_ + - _Depends: 14.1_ + - _Requirements: 6.2, 8.1_ + + ## Implementation Notes - Task5.2 measured PMPro timing out after both snapshots were materialized but before change context/graph analysis. Group6 must account for compatible snapshot/discovery reuse as well as graph reuse; identical-tip hits cannot substitute for the later representative changed-code latency gate. No new detector, scope heuristic or incremental CPG mutation is implied. diff --git a/benchmarks/measurements/2026-09-19-family-detection-census.json b/benchmarks/measurements/2026-09-19-family-detection-census.json new file mode 100644 index 0000000..aaca19d --- /dev/null +++ b/benchmarks/measurements/2026-09-19-family-detection-census.json @@ -0,0 +1,363 @@ +{ + "schema_version": 1, + "experiment": "family-detection-census-v1", + "date": "2026-09-19", + "harness": "benchmarks/push/family_census.py", + "claim_scope": "Detection only, on two inspected teaching repositories. No comparison, no novelty, no admission, no capability qualified. These subjects developed the analyser and cannot qualify it.", + "question": "Which of the seven declared vulnerability families establish a finding on real code, and what does each silent family mean?", + "subjects": { + "nodegoat": { + "schema_version": 1, + "experiment": "family-detection-census-v1", + "subject": "nodegoat", + "root": "/cache/nodegoat/c5cb68a7084e", + "claim_scope": "Detection only. No comparison, no novelty, no admission, no capability enabled.", + "inputs": { + "files": 44, + "bytes": 95411, + "regions": 44 + }, + "regions_by_family": { + "access_control": 35, + "config_secrets": 44, + "injection": 44, + "output_encoding": 44, + "path": 44, + "prototype": 44, + "untrusted_destination": 44 + }, + "languages": { + "javascript": 44 + }, + "scan": { + "seconds": 137.3, + "build_seconds": 28.66, + "query_seconds": 74.48, + "hook_table_hooks": 0, + "degradations": [ + "vendor_semantics_unresolved" + ], + "questions_total": 299, + "questions_completed": 299 + }, + "families": { + "access_control": { + "regions_offering": 35, + "questions_asked": 35, + "questions_completed": 35, + "findings": 0, + "rungs": {}, + "examples": [], + "verdict": "answered, nothing found" + }, + "config_secrets": { + "regions_offering": 44, + "questions_asked": 44, + "questions_completed": 44, + "findings": 1, + "rungs": { + "model_entailed": 1 + }, + "examples": [ + { + "site": "server.js:78:?", + "rung": "model_entailed", + "witness": "session({\n // genid: (req) => {\n // return genuuid() // use UUIDs for session IDs\n //},\n secret: cookieSecret,\n // Both mandatory in Express v4\n saveUninit", + "reached_from": 1 + } + ], + "verdict": "establishes findings" + }, + "injection": { + "regions_offering": 44, + "questions_asked": 44, + "questions_completed": 44, + "findings": 3, + "rungs": { + "model_entailed": 3 + }, + "examples": [ + { + "site": "app/routes/contributions.js:32:2", + "rung": "model_entailed", + "witness": "req.body.preTax -> eval(req.body.preTax) in app/routes/contributions.js (line 32, 1 steps)", + "reached_from": 1 + }, + { + "site": "app/routes/contributions.js:33:2", + "rung": "model_entailed", + "witness": "req.body.afterTax -> eval(req.body.afterTax) in app/routes/contributions.js (line 33, 1 steps)", + "reached_from": 1 + }, + { + "site": "app/routes/contributions.js:34:2", + "rung": "model_entailed", + "witness": "req.body.roth -> eval(req.body.roth) in app/routes/contributions.js (line 34, 1 steps)", + "reached_from": 1 + } + ], + "verdict": "establishes findings" + }, + "output_encoding": { + "regions_offering": 44, + "questions_asked": 44, + "questions_completed": 44, + "findings": 0, + "rungs": {}, + "examples": [], + "verdict": "answered, nothing found" + }, + "path": { + "regions_offering": 44, + "questions_asked": 44, + "questions_completed": 44, + "findings": 0, + "rungs": {}, + "examples": [], + "verdict": "answered, nothing found" + }, + "prototype": { + "regions_offering": 44, + "questions_asked": 44, + "questions_completed": 44, + "findings": 0, + "rungs": {}, + "examples": [], + "verdict": "answered, nothing found" + }, + "untrusted_destination": { + "regions_offering": 44, + "questions_asked": 44, + "questions_completed": 44, + "findings": 1, + "rungs": { + "model_entailed": 1 + }, + "examples": [ + { + "site": "app/routes/index.js:72:1", + "rung": "model_entailed", + "witness": "req.query.url -> res.redirect(req.query.url) in app/routes/index.js (line 72, 1 steps)", + "reached_from": 1 + } + ], + "verdict": "establishes findings" + } + }, + "summary": { + "answered, nothing found": 4, + "establishes findings": 3 + } + }, + "vampi": { + "schema_version": 1, + "experiment": "family-detection-census-v1", + "subject": "vampi", + "root": "/cache/vampi/f16052dce83f", + "claim_scope": "Detection only. No comparison, no novelty, no admission, no capability enabled.", + "inputs": { + "files": 11, + "bytes": 19907, + "regions": 25 + }, + "regions_by_family": { + "config_secrets": 25, + "deserialization": 25, + "injection": 25, + "output_encoding": 25, + "path": 25, + "untrusted_destination": 25, + "access_control": 8 + }, + "languages": { + "python": 25 + }, + "scan": { + "seconds": 85.3, + "build_seconds": 8.82, + "query_seconds": 52.26, + "hook_table_hooks": 0, + "degradations": [], + "questions_total": 35, + "questions_completed": 35 + }, + "families": { + "access_control": { + "regions_offering": 8, + "questions_asked": 8, + "questions_completed": 8, + "findings": 2, + "rungs": { + "model_entailed": 2 + }, + "examples": [ + { + "site": "api_views/books.py:51:get_by_title", + "rung": "model_entailed", + "witness": "get_by_title line 51: no discharging guard, while 1 sibling handler(s) are guarded (add_new_book)", + "reached_from": 1 + }, + { + "site": "api_views/users.py:187:update_password", + "rung": "model_entailed", + "witness": "update_password line 187: no discharging guard, while 3 sibling handler(s) are guarded (delete_user, me, update_email)", + "reached_from": 1 + } + ], + "verdict": "establishes findings" + }, + "config_secrets": { + "regions_offering": 25, + "questions_asked": 25, + "questions_completed": 25, + "findings": 1, + "rungs": { + "model_entailed": 1 + }, + "examples": [ + { + "site": "app.py:17:", + "rung": "model_entailed", + "witness": "vuln_app.run(host = '0.0.0.0', port = 5000, debug = True) (line 17): permissive literal '0.0.0.0'", + "reached_from": 1 + } + ], + "verdict": "establishes findings" + }, + "deserialization": { + "regions_offering": 25, + "questions_asked": 0, + "questions_completed": 0, + "findings": 0, + "rungs": {}, + "examples": [], + "verdict": "region offered, no question asked" + }, + "injection": { + "regions_offering": 25, + "questions_asked": 2, + "questions_completed": 2, + "findings": 1, + "rungs": { + "model_entailed": 1 + }, + "examples": [ + { + "site": "models/user_model.py:73:get_user", + "rung": "model_entailed", + "witness": "username -> text(user_query) in models/user_model.py (line 73, 13 steps)", + "reached_from": 1 + } + ], + "verdict": "establishes findings" + }, + "output_encoding": { + "regions_offering": 25, + "questions_asked": 0, + "questions_completed": 0, + "findings": 0, + "rungs": {}, + "examples": [], + "verdict": "region offered, no question asked" + }, + "path": { + "regions_offering": 25, + "questions_asked": 0, + "questions_completed": 0, + "findings": 0, + "rungs": {}, + "examples": [], + "verdict": "region offered, no question asked" + }, + "untrusted_destination": { + "regions_offering": 25, + "questions_asked": 0, + "questions_completed": 0, + "findings": 0, + "rungs": {}, + "examples": [], + "verdict": "region offered, no question asked" + } + }, + "summary": { + "establishes findings": 3, + "region offered, no question asked": 4 + } + } + }, + "verdict": { + "establish_findings": [ + "injection", + "access_control", + "config_secrets", + "untrusted_destination" + ], + "answered_nothing_found": { + "nodegoat": [ + "access_control", + "output_encoding", + "path", + "prototype" + ], + "vampi": [] + }, + "region_offered_no_question_asked": { + "nodegoat": [], + "vampi": [ + "deserialization", + "output_encoding", + "path", + "untrusted_destination" + ] + }, + "never_established_anywhere": [ + "output_encoding", + "path", + "prototype", + "deserialization" + ] + }, + "false_positive_found_and_fixed": { + "family": "prototype", + "site": "app/routes/session.js:189", + "reported_witness": "req.body -> this.handleSignup = (req, res, next) => { ...", + "measured_cause": "The matched node is .assignment whose code is the entire 1,000-character arrow function. The prototype sink 'set' matched the English word in the comment '// set these up in case we have an error case'. The handler calls no set.", + "fix": "taint.sc sinkMatches confines its text clause to the callee and excludes operator nodes", + "measured_effect_on_nodegoat_non_vendor_calls": { + "injection": { + "old": 28, + "new": 3 + }, + "output_encoding": { + "old": 3, + "new": 0 + }, + "path": { + "old": 0, + "new": 0 + }, + "prototype": { + "old": 13, + "new": 2 + }, + "untrusted_destination": { + "old": 26, + "new": 7 + } + }, + "true_positives_lost": 0, + "note": "Every dropped match is an operator node or an enclosing call whose body merely spells the word. req.query was an injection sink under the old rule, because 'query' is word-bounded inside it. The three eval calls and the res.redirect survive, and their witnesses now name the sink call rather than the assignment that contained it." + }, + "silent_families_diagnosed": { + "untrusted_destination_on_nodegoat": "NodeGoat's documented SSRF is needle.get(req.query.url + req.query.symbol) at app/routes/research.js:16. needle.get is absent from the sink vocabulary, which holds axios/fetch/got/http.get/request. The family works; its vocabulary is thin. Recall gap.", + "output_encoding_on_nodegoat": "All six declared sinks are DOM sinks (innerHTML, document.write, dangerouslySetInnerHTML, insertAdjacentHTML, outerHTML, v-html). NodeGoat's XSS is server-side, rendered into Swig templates and written with res.write, so no question the family can ask could have found it. Structural gap: server-rendered output has no sink vocabulary in JavaScript.", + "access_control_on_nodegoat": "Measured with the real dominance query: every modelled data operation in NodeGoat lives in app/data/*-dao.js or artifacts/db-reset.js, and none in a route handler file. Operations are scoped to the region's own file, and access_control regions are the handler files, so no obligation is ever raised where the question is asked. In the DAO files every operation is equally unguarded, so the sibling-contradiction rung has nothing to contradict. The two documented defects are also outside the model's expressible claims: an IDOR that reads userId from req.params on an authenticated route (app/routes/allocations.js:17), and a commented-out isAdmin in the route table (app/routes/index.js:58). Structural gap: the model asks about one file, and this application splits route from query.", + "four_families_on_vampi": "deserialization, output_encoding, path and untrusted_destination asked nothing because VAmPI's eleven Python files contain no sink for any of them; the only match of any such token is os.path.join in config.py. Correct silence, not a gap.", + "prototype": "Never established anywhere. Its only report was the false positive above. VAmPI is Python and the family is JavaScript-only, so it has been asked on one repository." + }, + "limits": [ + "Two repositories, both used to develop the analyser. Nothing here qualifies a capability.", + "Findings are counted at the rung the scan reached, all model_entailed; no execution confirmation.", + "A family silent on two repositories is untested, not absent." + ] +} diff --git a/benchmarks/push/family_census.py b/benchmarks/push/family_census.py new file mode 100644 index 0000000..f66b5bb --- /dev/null +++ b/benchmarks/push/family_census.py @@ -0,0 +1,139 @@ +"""Which declared vulnerability families actually establish findings on real code? + +Seven families are declared. Two are proven end to end: injection on NodeGoat and VAmPI, and access +control on VAmPI once its change context could complete. The other five have never been checked for +whether they produce a witness on anything, and a family that is declared but silent is advertised +coverage the tool does not have. + +This runs the production discovery and scan over a whole checkout and reports, per family, how many +regions were asked, how many questions completed, and what was found with the witness and rung that +established it. It makes no novelty or admission claim: there is no comparison here, only detection. + +A family reporting nothing is recorded as nothing. That is the point of asking. +""" + +from __future__ import annotations + +import argparse +import collections +import json +import time +from pathlib import Path +from typing import Any + +from openultrasast.cpg.backend import JoernBackend +from openultrasast.mapping import analyze_entry_points, php_hook_callbacks +from openultrasast.model.contracts import ExecutionBudget +from openultrasast.model.regions import regions_for +from openultrasast.model.scan import ScanBudget, scan_repository +from openultrasast.model.shipped import declared_sources +from openultrasast.preprocess import build_file_target, enumerate_source_files + + +def main() -> int: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--out", type=Path, required=True) + parser.add_argument("--root", type=Path, required=True) + parser.add_argument("--name", required=True) + parser.add_argument("--deadline", type=float, default=1800.0) + parser.add_argument("--max-regions", type=int, default=500) + args = parser.parse_args() + args.out.mkdir(parents=True, exist_ok=True) + record: dict[str, Any] = { + "schema_version": 1, + "experiment": "family-detection-census-v1", + "subject": args.name, + "root": str(args.root), + "claim_scope": "Detection only. No comparison, no novelty, no admission, no capability enabled.", + } + target = args.out / f"{args.name}.json" + + def save() -> None: + target.write_text(json.dumps(record, indent=2, default=str) + "\n") + + files = enumerate_source_files(args.root) + targets = [build_file_target(args.root, path) for path in files] + opened = sum(path.stat().st_size for path in files if path.is_file()) + regions = regions_for(analyze_entry_points(args.root, targets), targets, shipped=declared_sources(args.root)) + record["inputs"] = {"files": len(files), "bytes": opened, "regions": len(regions)} + record["regions_by_family"] = dict(collections.Counter(f for r in regions for f in r.families)) + record["languages"] = dict(collections.Counter(r.language for r in regions)) + save() + if not files or not opened: + record["error"] = "no readable source" + save() + return 2 + + backend = JoernBackend(execution_budget=ExecutionBudget(time.monotonic() + args.deadline, 5.0)) + hooks = php_hook_callbacks(args.root, targets) if any(r.language == "php" for r in regions) else {} + started = time.monotonic() + result = scan_repository( + args.root, + regions[: args.max_regions], + backend=backend, + budget=ScanBudget(max_model_calls=0, max_regions=args.max_regions, order_by_evidence=True), + execution_budget=ExecutionBudget(time.monotonic() + args.deadline, 5.0), + ranking_mode="evidence", + unit="repository", + population_complete=True, + ) + record["scan"] = { + "seconds": round(time.monotonic() - started, 1), + "build_seconds": result.build_seconds, + "query_seconds": result.query_seconds, + "hook_table_hooks": len(hooks), + "degradations": [str(d.get("reason")) for d in result.degradations], + "questions_total": len(result.question_outcomes), + "questions_completed": sum(1 for q in result.question_outcomes if q.status == "completed"), + } + + asked = collections.Counter(q.identity.family for q in result.question_outcomes) + completed = collections.Counter(q.identity.family for q in result.question_outcomes if q.status == "completed") + found: dict[str, list[dict[str, Any]]] = collections.defaultdict(list) + for finding in result.findings: + found[finding.family].append( + { + "site": finding.site, + "rung": finding.rung.value, + "witness": (finding.witness or "")[:200], + "reached_from": finding.reached_from, + } + ) + # Families with regions but no question must appear, or the census hides its own biggest finding: on + # VAmPI four declared families were offered 25 regions each and asked nothing, and building this list + # from the questions alone made them invisible rather than negative. + offered = record["regions_by_family"] + families = sorted(set(offered) | set(asked) | set(found)) + record["families"] = { + family: { + "regions_offering": offered.get(family, 0), + "questions_asked": asked.get(family, 0), + "questions_completed": completed.get(family, 0), + "findings": len(found.get(family, [])), + "rungs": dict(collections.Counter(item["rung"] for item in found.get(family, []))), + "examples": found.get(family, [])[:3], + # The distinction that matters: a family that answered and found nothing is a different + # statement from one that could not answer, and neither is "clean". + "verdict": ( + "region offered, no question asked" + if not asked.get(family) + else "asked but no question completed" + if not completed.get(family) + else "answered, nothing found" + if not found.get(family) + else "establishes findings" + ), + } + for family in families + } + record["summary"] = dict(collections.Counter(entry["verdict"] for entry in record["families"].values())) + save() + cleanup = getattr(result, "cleanup", None) + if callable(cleanup): + cleanup() + print(json.dumps({"subject": args.name, "summary": record["summary"]}), flush=True) + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/src/openultrasast/cpg/queries/taint.sc b/src/openultrasast/cpg/queries/taint.sc index 8fea4ec..e0e2406 100644 --- a/src/openultrasast/cpg/queries/taint.sc +++ b/src/openultrasast/cpg/queries/taint.sc @@ -499,8 +499,24 @@ // which is the safe API being reported as the dangerous one. Third instance of this bug class here, after // the sanitizer that matched `resolve` inside `resolveUrl` and the discharger that matched `user` inside // `users`; the sink matcher had simply never been given the same treatment. - def sinkMatches(c: io.shiftleft.codepropertygraph.generated.nodes.Call, n: String): Boolean = - c.name == n || c.code.startsWith(n + "(") || mentionsToken(c.code, List(n)) || mentionsToken(c.methodFullName, List(n)) + // Fourth instance of the same bug class, and this one word-bounding did not catch. NodeGoat's signup handler + // is one `.assignment` node whose code is the entire 1,000-character arrow function, and inside it + // a comment reads `// set these up in case we have an error case`. `set` is a prototype-pollution sink, the + // word is bounded, and the match was reported as a prototype finding on a handler that calls no `set` at + // all. Searching a node's whole text asks "is this word anywhere near here", which is not the question. + // + // So the text clause is confined to the CALLEE -- everything before the first argument -- which is the only + // part of a call that names what is being called, and operators are excluded from it entirely: an + // assignment is not a call to a library function, whatever its right-hand side happens to spell. + def calleeText(code: String): String = { + val open = code.indexOf('(') + if (open < 0) code else code.substring(0, open) + } + + def sinkMatches(c: io.shiftleft.codepropertygraph.generated.nodes.Call, n: String): Boolean = { + val operator = c.name.startsWith(" None: overlays = str(payload.get("overlays", "")).split(",") assert "dataflowOss" in overlays, f"census reports overlays {overlays!r}, which is not a list of layer names" assert str(payload.get("maxHeapMB", "")).isdigit() + + +@pytest.mark.skipif(_joern() is None, reason="joern is not installed on this machine") +def test_a_sink_word_in_a_comment_is_not_a_sink(tmp_path: Path) -> None: + """A handler that mentions a sink word in prose must not report that sink. + + The first family census ran this exact shape: NodeGoat's signup handler is one assignment node whose code + is the whole arrow function, a comment inside it reads `// set these up in case we have an error case`, + and `set` is a prototype-pollution sink. The word is bounded and the match was real, so the census + reported a prototype finding for a handler that calls no `set`. The witness printed the function header, + because there was no sink call to print. + """ + from openultrasast.cpg.backend import extract_payload + + frontend = __import__("shutil").which("jssrc2cpg") + if frontend is None: + pytest.skip("jssrc2cpg is not installed") + source = tmp_path / "handler.js" + source.write_text( + "function handler(req, res) {\n" + " const data = req.body;\n" + " // set these up in case we have an error case\n" + " return res.end(String(data));\n" + "}\n" + ) + cpg = tmp_path / "cpg.bin" + built = subprocess.run([frontend, str(tmp_path), "-o", str(cpg)], capture_output=True, text=True, timeout=600, check=False) + if not cpg.is_file(): + pytest.skip(f"could not build a sample cpg: {(built.stderr or '')[-200:]}") + + requests = tmp_path / "requests.json" + requests.write_text(json.dumps({"0": {"sources": "req.body", "sinks": "set", "function": "handler"}})) + done = subprocess.run( + [ + _joern() or "joern", + "--script", + str((QUERIES / "taint.sc").resolve()), + "--param", + f"cpgFile={cpg}", + "--param", + f"requestsFile={requests}", + ], + capture_output=True, + text=True, + timeout=900, + check=False, + cwd=str(tmp_path), + ) + assert done.returncode == 0, f"taint.sc did not run: {(done.stderr or done.stdout or '')[-500:]}" + payload = extract_payload(done.stdout or "") + assert payload is not None, "taint.sc produced no parseable payload" + rows = payload.get("0") or [] + # The inventory row is the right answer and must survive: "zero operations in scope" is a statement, and + # the point of the census was that it differs from "the query found nothing to say". + flows = [row for row in rows if row.get("sink")] + assert flows == [], f"a commented sink word produced {len(flows)} flow row(s): {flows[:2]}" + inventory = [row for row in rows if row.get("kind") == "operation_inventory_complete"] + assert inventory, f"taint.sc reported no inventory row: {rows[:3]}" + assert all(int(row.get("operations", 0)) == 0 for row in inventory), f"a commented sink word was counted as an operation: {inventory}"