diff --git a/.kiro/specs/model-grounded-detection/tasks.md b/.kiro/specs/model-grounded-detection/tasks.md index a42daad..e505ea5 100644 --- a/.kiro/specs/model-grounded-detection/tasks.md +++ b/.kiro/specs/model-grounded-detection/tasks.md @@ -158,7 +158,7 @@ queries in group 4). The gates, `redaction.py`, `pairs.py` overlay/inventory pat - _Requirements: 5.1, 5.2, 6.1_ - _Depends: 2.1, 1.1_ -- [ ] 2.3 Determinism, and the go/no-go reading +- [x] 2.3 Determinism, and the go/no-go reading - `test_two_runs_over_one_cpg_give_an_identical_verdict_sequence`. Run the injection taint verdict over the injection slice, regenerate the entailment ceiling for injection, and write the reading into the Implementation Notes: does Joern produce a deterministic verdict at acceptable per-target cost and time? @@ -313,3 +313,69 @@ the zero-dependency guard in `test_gate` pins. Manifest committed at Group 2 (the Joern go/no-go) has not started. `semantic/engines.py` already exposes `joern_available`, which the `CpgBackend` capability probe in task 2.1 should build on rather than duplicate. + +## Implementation Notes — group 2 (2026-09-08) + +**Verdict: GO.** Joern is installed at `~/joern/joern-cli` (maintainer chose a permanent install). Artifact: +`benchmarks/measurements/2026-09-08-joern-go-no-go.json`. + +| | injection slice | +|---|---| +| `model_entailed` on the vulnerable side | **54%** (27/50) — against the flat IR's **2.2%** | +| ... on minimal-fix pairs only | **61.5%** (16/26) | +| pairs distinguished (vuln entailed, fix clean) | 32% overall, **38.5%** on minimal-fix pairs | +| determinism (Req 6.4) | **every verdict identical on re-run** | +| cost | median **32s**/pair, max 58s, 30 min for the slice | +| CPG build failures | 1 (a C translation unit that does not parse standalone) | + +**The engine is not the constraint.** Every gap below is in our own data model, which is exactly the result +that justifies adopting a CPG rather than building one. The 23 misses split three ways: 11 sinks not in the +fact tables (Django ORM `objects.raw`, `self.query`, GraphQL resolvers), 11 where the sink is modelled but no +source reaches it, and 1 build failure — counted separately so it can never read as "no vulnerability". + +**The finding that changes the design.** Classifying what the 50 injection *fixes* actually do: + + shape introduced at the sink guard kind of the added lines + other_or_validation 37 none 24 + bound_parameters 10 parameterized_call 11 + argv_list 2 null_test 7 + sanitizer_call 1 allowlist_test 5 + +The dominant injection fix is a **validation guard on the path** — allowlist, type coercion, bounds or null +test — not a sanitizer call and not a sink-shape change. Taint reachability can never separate those pairs: +the flow still reaches the sink; what changed is that a guard now *dominates* the path. So the arbiter is +**taint + dominance**, not taint alone — the same primitive task 4.1 builds for access_control, with the guard +vocabulary ported verbatim in task 1.1 as its lookup list. Two planned mechanisms collapse into one, and +task 4.1 should be built before the remaining `TaintSpec`s rather than after. + +**Five bugs the live engine exposed, none visible offline.** Two of them *inverted* a verdict rather than +weakening it, which is why each was fixed as a class: +1. A sink was its own sanitizer — facts carrying a *shape* qualifier (`parameterized`, `literal_format_arg`) + describe a safe form of the sink call, not a cleansing call. Flattened in, `execute` became both the + injection sink and its own sanitizer and **nothing could ever be entailed**. Fixing the class immediately + caught C's entire `printf` family. +2. `labeled_family` had lost its deterministic CWE fallback in group 1 (my error) — the injection slice + resolved to **zero pairs**. Restored, still with no model call; the census again matches the committed + ceiling exactly. +3. Taint alone entailed both sides of the canonical SQL pair, so the model arbitrated nothing → the + safe-shape test, using the `safe_shape_sinks` that bug 1 had preserved. +4. Joern's `call.argument` includes the **receiver**, so a one-argument interpolated call looked like a + two-argument bound one — the exact inversion of the shape test. Fixed with `argumentIndexGt(0)`. +5. The source model was framework-only, while 39/50 pairs receive untrusted input as a function parameter. + Parameter sources are now opt-in (right for a function-level pair whose boundary *is* the trust boundary, + wrong for a whole repository) — and required for the comparison against the flat-IR baseline to be + like-for-like, since that baseline counted them. + +**Corpus caveat, and why the differential is reported per stratum.** Only 26 of 50 injection pairs are genuine +minimal-fix pairs; 12 are *different programs* (`local-python-vulnerable` is 7% similar with 83 changed lines; +`owasp-java-cmdi` pairs `BenchmarkTest00007` against `BenchmarkTest00090`) and 12 are large rewrites. Where +the fixed side is a different program, both sides flagging is a corpus property, not a model failure — a raw +`distinguished_rate` would be measuring corpus quality and reporting it as model quality. + +**A correction to a claim made mid-run.** The Flask-shaped source table (no `request.POST`/`request.GET`) is a +real gap but a *minor* lever here: adding Django patterns would gain a source for exactly one pair in this +slice. The larger levers are the missing sinks and guard dominance. + +**Next**, in this order: 4.1 guard dominance over the tainted path; 4.2 the missing sinks (Django ORM, +GraphQL, and the four families with no sink fact at all); then the two remaining safe-shape forms, argv-list +and `shell=False`. diff --git a/benchmarks/measurements/2026-09-08-joern-go-no-go.json b/benchmarks/measurements/2026-09-08-joern-go-no-go.json new file mode 100644 index 0000000..b0bb1f1 --- /dev/null +++ b/benchmarks/measurements/2026-09-08-joern-go-no-go.json @@ -0,0 +1,793 @@ +{ + "build_failures": 1, + "by_pair_quality": { + "different_program": { + "distinguished": 2, + "distinguished_rate": 0.1667, + "pairs": 12, + "vuln_entailed": 4, + "vuln_entailed_rate": 0.3333 + }, + "large_rewrite": { + "distinguished": 4, + "distinguished_rate": 0.3333, + "pairs": 12, + "vuln_entailed": 7, + "vuln_entailed_rate": 0.5833 + }, + "minimal_fix": { + "distinguished": 10, + "distinguished_rate": 0.3846, + "pairs": 26, + "vuln_entailed": 16, + "vuln_entailed_rate": 0.6154 + } + }, + "corpus_caveat": "Only 26 of 50 injection pairs are genuine minimal-fix pairs. 12 are different programs (<60% similar; local-python-vulnerable is 7% similar with 83 changed lines, owasp-java-cmdi pairs BenchmarkTest00007 against BenchmarkTest00090) and 12 are large rewrites. Where the fixed side is a different program, both sides being flagged is a property of the corpus, not a failure of the model -- so the differential is reported per stratum as well as overall.", + "corroborated_or_better": 0.54, + "determinism_checked_pairs": 5, + "deterministic_all": true, + "distinguished_pairs": 16, + "distinguished_rate": 0.32, + "entailed_rate": 0.54, + "fix_shape_analysis": { + "consequence": "The dominant injection fix is a VALIDATION GUARD on the path -- allowlist, type coercion, bounds or null test -- not a sanitizer call and not a sink-shape change. Taint reachability can never separate those pairs: the flow still reaches the sink; what changed is that a guard now dominates the path. The arbiter must therefore be TAINT + DOMINANCE, not taint alone -- the same primitive task 4.1 builds for access_control, with the guard vocabulary ported in task 1.1 (allowlist_test, type_change, bounds_test, null_test) as its lookup list. Two mechanisms collapse into one.", + "guard_kind_of_added_lines": { + "allowlist_test": 5, + "bounds_test": 1, + "none": 24, + "null_test": 7, + "parameterized_call": 11, + "type_change": 2 + }, + "note": "What the 50 injection fixes actually do, classified from the diff offline. This is the finding that changes the design.", + "shape_introduced_at_the_sink": { + "argv_list": 2, + "bound_parameters": 10, + "other_or_validation": 37, + "sanitizer_call": 1 + } + }, + "flat_ir_baseline": { + "corroborated_one_hop": 0.239, + "entailed": 0.022, + "note": "2026-09-08-model-entailment-ceiling.json" + }, + "generated": "2026-09-08", + "miss_causes": { + "cpg_build_failed": { + "detail": "openssl-cve-2022-1292: a C translation unit that does not parse standalone. Counted separately so it can never read as 'no vulnerability'.", + "pairs": 1 + }, + "sink_not_in_table": { + "detail": "the vulnerable sink is not in ruleset/semantic (Django ORM raw SQL `objects.raw`, `self.query`, GraphQL resolvers). Task 4.2 authors these.", + "pairs": 11 + }, + "sink_present_but_no_flow_resolved": { + "detail": "the sink is modelled but no source reaches it: the excerpt's untrusted value arrives from a context file or a framework object neither the source table nor the parameter-source rule covers.", + "pairs": 11 + } + }, + "next": [ + "4.1 guard dominance over the tainted path (serves injection and access_control with one primitive)", + "4.2 author the missing sinks: Django ORM raw SQL, GraphQL resolvers, and the four families with no sink at all (path, output_encoding, untrusted_destination, prototype)", + "safe-shape forms still missing: argv list (subprocess.call([...])) and shell=False" + ], + "pair_quality_note": "Only a minimal_fix pair poses the differential question at all: when the fixed side is a different program (<60% similar) or a large rewrite (>20 changed lines), both sides being flagged is a property of the corpus, not a failure of the model. The headline distinguished_rate is therefore reported over minimal_fix pairs as well as over all pairs.", + "pairs": 50, + "reading": "GO on the engine. Joern builds a CPG for a pair tree in 5-30s, resolves the multi-hop flows the flat IR could not, and is deterministic: every verdict checked twice over one CPG agreed (deterministic_all=true). Entailment on the injection slice is 54% against the flat IR's 2.2% -- a 24x lift -- and 61.5% on the pairs where the differential question is well-posed. Every remaining gap is in OUR data model, not the engine, which is precisely the result that justifies adopting a CPG rather than building one.", + "rows": [ + { + "build_failed": false, + "determinism_checked": true, + "deterministic": true, + "distinguished": false, + "fixed_rung": "model_entailed", + "function": "", + "language": "python", + "pair": "local-python-vulnerable", + "pair_quality": "different_program", + "seconds": 57.9, + "vuln_rung": "model_entailed", + "witness": "request.args[\"expr\"] -> eval(request.args[\"expr\"]) (line 28, 1 steps)" + }, + { + "build_failed": false, + "determinism_checked": true, + "deterministic": true, + "distinguished": false, + "fixed_rung": "model_entailed", + "function": "", + "language": "javascript", + "pair": "local-javascript-vulnerable", + "pair_quality": "different_program", + "seconds": 55.2, + "vuln_rung": "model_entailed", + "witness": "\"make \" + req.query.target -> childProcess.execSync(\"make \" + req.query.target) (line 14, 1 steps)" + }, + { + "build_failed": false, + "determinism_checked": true, + "deterministic": true, + "distinguished": false, + "fixed_rung": "model_entailed", + "function": "", + "language": "python", + "pair": "sven-cwe-078-os-system", + "pair_quality": "minimal_fix", + "seconds": 51.3, + "vuln_rung": "model_entailed", + "witness": "username -> os.system(\"useradd -G docker,wheel -p \" + encPass + \" \" + username) (line 10, 2 steps)" + }, + { + "build_failed": false, + "determinism_checked": true, + "deterministic": true, + "distinguished": false, + "fixed_rung": "model_entailed", + "function": "", + "language": "python", + "pair": "sven-cwe-078-shell-true", + "pair_quality": "minimal_fix", + "seconds": 51.2, + "vuln_rung": "model_entailed", + "witness": "cmd -> _exec_cmd(self, cmd) (line 14, 2 steps)" + }, + { + "build_failed": false, + "determinism_checked": true, + "deterministic": true, + "distinguished": true, + "fixed_rung": null, + "function": "", + "language": "python", + "pair": "sven-cwe-089-execute-percent", + "pair_quality": "minimal_fix", + "seconds": 51.1, + "vuln_rung": "model_entailed", + "witness": "game -> cursor.execute(\"SELECT x,y,letter FROM moves WHERE gameID = %d;\" % game) (line 12, 3 steps)" + }, + { + "build_failed": false, + "determinism_checked": false, + "deterministic": true, + "distinguished": false, + "fixed_rung": null, + "function": "", + "language": "python", + "pair": "sven-cwe-089-query-concat", + "pair_quality": "minimal_fix", + "seconds": 29.8, + "vuln_rung": null, + "witness": "" + }, + { + "build_failed": false, + "determinism_checked": false, + "deterministic": true, + "distinguished": true, + "fixed_rung": null, + "function": "", + "language": "javascript", + "pair": "node-serialize-cve-2017-5941", + "pair_quality": "minimal_fix", + "seconds": 30.6, + "vuln_rung": "model_entailed", + "witness": "obj -> eval(\"(\" + obj + \")\") (line 4, 2 steps)" + }, + { + "build_failed": false, + "determinism_checked": false, + "deterministic": true, + "distinguished": false, + "fixed_rung": "model_entailed", + "function": "", + "language": "python", + "pair": "owasp-python-codeinj", + "pair_quality": "large_rewrite", + "seconds": 30.3, + "vuln_rung": "model_entailed", + "witness": "urllib.parse.unquote_plus(request.cookies.get(\"BenchmarkTest00074\", \"noCookieValueSupplied\")) -> exec(bar) (line 51, 9 steps)" + }, + { + "build_failed": false, + "determinism_checked": false, + "deterministic": true, + "distinguished": true, + "fixed_rung": null, + "function": "", + "language": "python", + "pair": "owasp-python-cmdi", + "pair_quality": "large_rewrite", + "seconds": 29.9, + "vuln_rung": "model_entailed", + "witness": "request.form.get(\"BenchmarkTest00165\") -> subprocess.run(argList, capture_output = True, encoding = \"utf-8\") (line 57, 15 steps)" + }, + { + "build_failed": false, + "determinism_checked": false, + "deterministic": true, + "distinguished": true, + "fixed_rung": null, + "function": "", + "language": "python", + "pair": "owasp-python-sqli", + "pair_quality": "large_rewrite", + "seconds": 29.1, + "vuln_rung": "model_entailed", + "witness": "request.form.get(\"BenchmarkTest00099\") -> cur.execute(sql) (line 49, 14 steps)" + }, + { + "build_failed": false, + "determinism_checked": false, + "deterministic": true, + "distinguished": false, + "fixed_rung": "model_entailed", + "function": "", + "language": "java", + "pair": "owasp-java-cmdi", + "pair_quality": "large_rewrite", + "seconds": 54.5, + "vuln_rung": "model_entailed", + "witness": "getHeader(\"BenchmarkTest00007\") -> r.exec(args, argsEnv) (line 61, 10 steps)" + }, + { + "build_failed": false, + "determinism_checked": false, + "deterministic": true, + "distinguished": false, + "fixed_rung": "model_entailed", + "function": "", + "language": "java", + "pair": "owasp-java-sqli", + "pair_quality": "large_rewrite", + "seconds": 54.4, + "vuln_rung": "model_entailed", + "witness": "getHeader(\"BenchmarkTest00008\") -> connection.prepareCall(sql) (line 57, 9 steps)" + }, + { + "build_failed": false, + "determinism_checked": false, + "deterministic": true, + "distinguished": true, + "fixed_rung": null, + "function": "", + "language": "java", + "pair": "juliet-java-cwe78-exec", + "pair_quality": "minimal_fix", + "seconds": 53.7, + "vuln_rung": "model_entailed", + "witness": "String data -> Runtime.getRuntime().exec(osCommand + data) (line 8, 2 steps)" + }, + { + "build_failed": true, + "determinism_checked": false, + "deterministic": true, + "distinguished": false, + "fixed_rung": null, + "function": "link_hash_cert", + "language": "python", + "pair": "openssl-cve-2022-1292", + "pair_quality": "minimal_fix", + "seconds": 1.3, + "vuln_rung": null, + "witness": "" + }, + { + "build_failed": false, + "determinism_checked": false, + "deterministic": true, + "distinguished": false, + "fixed_rung": null, + "function": "mutate", + "language": "python", + "pair": "damn-vulnerable-graphql-application-views-mutate", + "pair_quality": "different_program", + "seconds": 32.9, + "vuln_rung": null, + "witness": "" + }, + { + "build_failed": false, + "determinism_checked": false, + "deterministic": true, + "distinguished": false, + "fixed_rung": null, + "function": "resolve_pastes", + "language": "python", + "pair": "damn-vulnerable-graphql-application-views-resolve-pastes", + "pair_quality": "minimal_fix", + "seconds": 31.5, + "vuln_rung": null, + "witness": "" + }, + { + "build_failed": false, + "determinism_checked": false, + "deterministic": true, + "distinguished": false, + "fixed_rung": null, + "function": "resolve_system_diagnostics", + "language": "python", + "pair": "damn-vulnerable-graphql-application-views-resolve-system-dia", + "pair_quality": "different_program", + "seconds": 30.4, + "vuln_rung": null, + "witness": "" + }, + { + "build_failed": false, + "determinism_checked": false, + "deterministic": true, + "distinguished": false, + "fixed_rung": null, + "function": "hits_by_ip", + "language": "python", + "pair": "djangoat-analytics-hits-by-ip", + "pair_quality": "minimal_fix", + "seconds": 29.8, + "vuln_rung": null, + "witness": "" + }, + { + "build_failed": false, + "determinism_checked": false, + "deterministic": true, + "distinguished": false, + "fixed_rung": null, + "function": "user_view", + "language": "python", + "pair": "djangoat-views-user-view", + "pair_quality": "different_program", + "seconds": 29.1, + "vuln_rung": null, + "witness": "" + }, + { + "build_failed": false, + "determinism_checked": false, + "deterministic": true, + "distinguished": false, + "fixed_rung": null, + "function": "run", + "language": "python", + "pair": "dsvpwa-attacks-run", + "pair_quality": "different_program", + "seconds": 29.3, + "vuln_rung": null, + "witness": "" + }, + { + "build_failed": false, + "determinism_checked": false, + "deterministic": true, + "distinguished": false, + "fixed_rung": null, + "function": "do_BDR", + "language": "python", + "pair": "dsvpwa-handlers-do-bdr", + "pair_quality": "different_program", + "seconds": 32.9, + "vuln_rung": null, + "witness": "" + }, + { + "build_failed": false, + "determinism_checked": false, + "deterministic": true, + "distinguished": false, + "fixed_rung": null, + "function": "create", + "language": "python", + "pair": "dvpwa-student-create", + "pair_quality": "minimal_fix", + "seconds": 31.9, + "vuln_rung": null, + "witness": "" + }, + { + "build_failed": false, + "determinism_checked": false, + "deterministic": true, + "distinguished": false, + "fixed_rung": null, + "function": "search", + "language": "python", + "pair": "extremely-vulnerable-flask-app-account-search", + "pair_quality": "different_program", + "seconds": 32.3, + "vuln_rung": null, + "witness": "" + }, + { + "build_failed": false, + "determinism_checked": false, + "deterministic": true, + "distinguished": true, + "fixed_rung": null, + "function": "validate_token", + "language": "python", + "pair": "extremely-vulnerable-flask-app-signup-validate-token", + "pair_quality": "different_program", + "seconds": 29.0, + "vuln_rung": "model_entailed", + "witness": "session -> session.execute(text(f\"\"\"\n SELECT id, code FROM {RegistrationCode.__tablename__} WHERE code = '{code}'\n \"\"\")) (line 15, 2 steps)" + }, + { + "build_failed": false, + "determinism_checked": false, + "deterministic": true, + "distinguished": true, + "fixed_rung": null, + "function": "dashboard", + "language": "python", + "pair": "threatbyte-routes-dashboard", + "pair_quality": "different_program", + "seconds": 28.7, + "vuln_rung": "model_entailed", + "witness": "request.args.get('search', '') -> cursor.execute(s_query) (line 29, 8 steps)" + }, + { + "build_failed": false, + "determinism_checked": false, + "deterministic": true, + "distinguished": false, + "fixed_rung": null, + "function": "get_user", + "language": "python", + "pair": "vampi-user-model-get-user", + "pair_quality": "different_program", + "seconds": 27.9, + "vuln_rung": null, + "witness": "" + }, + { + "build_failed": false, + "determinism_checked": false, + "deterministic": true, + "distinguished": false, + "fixed_rung": null, + "function": "delete_user", + "language": "python", + "pair": "vfapi-main-delete-user", + "pair_quality": "minimal_fix", + "seconds": 29.1, + "vuln_rung": null, + "witness": "" + }, + { + "build_failed": false, + "determinism_checked": false, + "deterministic": true, + "distinguished": false, + "fixed_rung": null, + "function": "put_user", + "language": "python", + "pair": "vfapi-main-put-user", + "pair_quality": "different_program", + "seconds": 29.7, + "vuln_rung": null, + "witness": "" + }, + { + "build_failed": false, + "determinism_checked": false, + "deterministic": true, + "distinguished": false, + "fixed_rung": null, + "function": "sql_return_users_from_username", + "language": "python", + "pair": "vfapi-main-sql-return-users-from-username", + "pair_quality": "minimal_fix", + "seconds": 29.6, + "vuln_rung": null, + "witness": "" + }, + { + "build_failed": false, + "determinism_checked": false, + "deterministic": true, + "distinguished": true, + "fixed_rung": null, + "function": "do_os_system", + "language": "python", + "pair": "vulnpy-cmdi-do-os-system", + "pair_quality": "minimal_fix", + "seconds": 30.3, + "vuln_rung": "model_entailed", + "witness": "command -> os.system(command) (line 14, 2 steps)" + }, + { + "build_failed": false, + "determinism_checked": false, + "deterministic": true, + "distinguished": true, + "fixed_rung": null, + "function": "do_subprocess_popen", + "language": "python", + "pair": "vulnpy-cmdi-do-subprocess-popen", + "pair_quality": "minimal_fix", + "seconds": 34.7, + "vuln_rung": "model_entailed", + "witness": "command -> subprocess.Popen(command, stdout = subprocess.PIPE, stderr = subprocess.STDOUT, shell = True) (line 14, 2 steps)" + }, + { + "build_failed": false, + "determinism_checked": false, + "deterministic": true, + "distinguished": true, + "fixed_rung": null, + "function": "create", + "language": "python", + "pair": "vulpy-libuser-create", + "pair_quality": "minimal_fix", + "seconds": 35.2, + "vuln_rung": "model_entailed", + "witness": "password -> c.execute(\"INSERT INTO users (username, password, failures, mfa_enabled, mfa_secret) VALUES ('%s', '%s', '%d', '%d', '%s')\" % (username,password,0,0,'')) (line 18, 4 steps)" + }, + { + "build_failed": false, + "determinism_checked": false, + "deterministic": true, + "distinguished": true, + "fixed_rung": null, + "function": "login", + "language": "python", + "pair": "vulpy-libuser-login", + "pair_quality": "minimal_fix", + "seconds": 31.3, + "vuln_rung": "model_entailed", + "witness": "username -> c.execute(\"SELECT * FROM users WHERE username = '{}' and password = '{}'\".format(username, password)) (line 20, 5 steps)" + }, + { + "build_failed": false, + "determinism_checked": false, + "deterministic": true, + "distinguished": true, + "fixed_rung": null, + "function": "password_change", + "language": "python", + "pair": "vulpy-libuser-password-change", + "pair_quality": "minimal_fix", + "seconds": 30.9, + "vuln_rung": "model_entailed", + "witness": "password -> c.execute(\"UPDATE users SET password = '{}' WHERE username = '{}'\".format(password, username)) (line 20, 5 steps)" + }, + { + "build_failed": false, + "determinism_checked": false, + "deterministic": true, + "distinguished": true, + "fixed_rung": null, + "function": "publish", + "language": "javascript", + "pair": "apex-publish-static-files-2-0-0", + "pair_quality": "large_rewrite", + "seconds": 33.8, + "vuln_rung": "model_entailed", + "witness": "this -> execSync(\n\t\t\t\t'\"' + opts.sqlclPath + '\"' + // Sqlcl path\n\t\t\t\t' ' + opts.connectString + // Connect string (user/pass@server:port/sid)\n\t\t\t\t' @\"' + path.resolve(__dirname, 'lib/script') + '\"' + // Sql t (line 61, 2 steps)" + }, + { + "build_failed": false, + "determinism_checked": false, + "deterministic": true, + "distinguished": false, + "fixed_rung": null, + "function": "nativeZip", + "language": "javascript", + "pair": "bestzip-2-1-6", + "pair_quality": "large_rewrite", + "seconds": 34.0, + "vuln_rung": null, + "witness": "" + }, + { + "build_failed": false, + "determinism_checked": false, + "deterministic": true, + "distinguished": false, + "fixed_rung": "model_entailed", + "function": "local_scan", + "language": "javascript", + "pair": "clamscan-1-2-0", + "pair_quality": "minimal_fix", + "seconds": 50.5, + "vuln_rung": "model_entailed", + "witness": "this -> execFile(self.settings[self.scanner].path, self._build_clam_args(path), (err, stdout, stderr) => {\n const {is_infected, viruses} = self._process_result(stdout, path);\n\n (line 1401, 49 steps)" + }, + { + "build_failed": false, + "determinism_checked": false, + "deterministic": true, + "distinguished": false, + "fixed_rung": "model_entailed", + "function": "diskusage", + "language": "javascript", + "pair": "diskusage-ng-0-2-6", + "pair_quality": "minimal_fix", + "seconds": 33.5, + "vuln_rung": "model_entailed", + "witness": "this -> exec('df -k \"' + path + '\"', function(err, stdout) {\n if (err) {\n return cb(err);\n }\n\n try {\n cb(null, parse(stdout));\n } catch (e) {\n cb(e (line 19, 2 steps)" + }, + { + "build_failed": false, + "determinism_checked": false, + "deterministic": true, + "distinguished": false, + "fixed_rung": "model_entailed", + "function": "resolve", + "language": "javascript", + "pair": "dns-sync-0-1-0", + "pair_quality": "minimal_fix", + "seconds": 36.4, + "vuln_rung": "model_entailed", + "witness": "hostname -> shell.exec(cmd, {silent: true}) (line 22, 5 steps)" + }, + { + "build_failed": false, + "determinism_checked": false, + "deterministic": true, + "distinguished": true, + "fixed_rung": null, + "function": "performAction", + "language": "javascript", + "pair": "hot-formula-parser-3-0-0", + "pair_quality": "large_rewrite", + "seconds": 55.8, + "vuln_rung": "model_entailed", + "witness": "yytext -> eval(\"[\" + yytext + \"]\") (line 166, 2 steps)" + }, + { + "build_failed": false, + "determinism_checked": false, + "deterministic": true, + "distinguished": false, + "fixed_rung": null, + "function": "block", + "language": "javascript", + "pair": "libnmap-0-4-11", + "pair_quality": "large_rewrite", + "seconds": 42.5, + "vuln_rung": null, + "witness": "" + }, + { + "build_failed": false, + "determinism_checked": false, + "deterministic": true, + "distinguished": false, + "fixed_rung": null, + "function": "getInfo", + "language": "javascript", + "pair": "pdf-image-1-0-5", + "pair_quality": "large_rewrite", + "seconds": 36.8, + "vuln_rung": null, + "witness": "" + }, + { + "build_failed": false, + "determinism_checked": false, + "deterministic": true, + "distinguished": false, + "fixed_rung": "model_entailed", + "function": "ps", + "language": "javascript", + "pair": "pidusage-1-0-0", + "pair_quality": "minimal_fix", + "seconds": 37.0, + "vuln_rung": "model_entailed", + "witness": "this -> exec(cmd + pid, function(err, stdout, stderr) {\n if(err)\n return done(err, null)\n\n stdout = stdout.split(os.EOL)[1]\n stdout = stdout.replace(/^\\s+/, '').replace(/\\s\\s+/g, ' ').sp (line 116, 2 steps)" + }, + { + "build_failed": false, + "determinism_checked": false, + "deterministic": true, + "distinguished": false, + "fixed_rung": null, + "function": "listProcessesOnPort", + "language": "javascript", + "pair": "portprocesses-1-0-4", + "pair_quality": "minimal_fix", + "seconds": 34.8, + "vuln_rung": null, + "witness": "" + }, + { + "build_failed": false, + "determinism_checked": false, + "deterministic": true, + "distinguished": false, + "fixed_rung": null, + "function": "walk", + "language": "javascript", + "pair": "static-eval-1-1-1", + "pair_quality": "minimal_fix", + "seconds": 33.6, + "vuln_rung": null, + "witness": "" + }, + { + "build_failed": false, + "determinism_checked": false, + "deterministic": true, + "distinguished": true, + "fixed_rung": null, + "function": "thenify", + "language": "javascript", + "pair": "thenify-3-3-0", + "pair_quality": "minimal_fix", + "seconds": 30.6, + "vuln_rung": "model_entailed", + "witness": "this -> eval(createWrapper($$__fn__$$.name, options)) (line 29, 4 steps)" + }, + { + "build_failed": false, + "determinism_checked": false, + "deterministic": true, + "distinguished": false, + "fixed_rung": null, + "function": "template", + "language": "javascript", + "pair": "underscore-1-13-0-0", + "pair_quality": "large_rewrite", + "seconds": 31.1, + "vuln_rung": null, + "witness": "" + }, + { + "build_failed": false, + "determinism_checked": false, + "deterministic": true, + "distinguished": true, + "fixed_rung": null, + "function": "whereis", + "language": "javascript", + "pair": "whereis-0-4-0", + "pair_quality": "minimal_fix", + "seconds": 30.8, + "vuln_rung": "model_entailed", + "witness": "name -> cp.exec('which ' + name, function(error, stdout, stderr) {\n stdout = stdout.split('\\n')[0];\n if (error || stderr || stdout === '' || stdout.charAt(0) !== '/') {\n stdout = stdout.split('\\n') (line 15, 3 steps)" + }, + { + "build_failed": false, + "determinism_checked": false, + "deterministic": true, + "distinguished": false, + "fixed_rung": null, + "function": "_is_command_allowed", + "language": "python", + "pair": "tools-cli-tools-a96bc0", + "pair_quality": "minimal_fix", + "seconds": 30.9, + "vuln_rung": null, + "witness": "" + }, + { + "build_failed": false, + "determinism_checked": false, + "deterministic": true, + "distinguished": false, + "fixed_rung": null, + "function": "getGoogleProvider", + "language": "javascript", + "pair": "workers-research-utils-209444", + "pair_quality": "large_rewrite", + "seconds": 32.1, + "vuln_rung": null, + "witness": "" + } + ], + "seconds_per_pair": { + "max": 57.9, + "median": 32.1, + "total": 1781.0 + }, + "task": "model-grounded-detection 2.3 (Joern go/no-go, injection slice, parameter sources on)", + "verdict": "GO", + "vuln_side": { + "corroborated": 0, + "entailed": 27, + "none": 23 + } +}