{ "build_failures": 1, "by_pair_quality": { "different_program": { "distinguished": 2, "distinguished_rate": 0.1667, "pairs": 12, "vuln_entailed": 4, "vuln_entailed_rate": 0.3333 }, "large_rewrite": { "distinguished": 4, "distinguished_rate": 0.3333, "pairs": 12, "vuln_entailed": 7, "vuln_entailed_rate": 0.5833 }, "minimal_fix": { "distinguished": 10, "distinguished_rate": 0.3846, "pairs": 26, "vuln_entailed": 16, "vuln_entailed_rate": 0.6154 } }, "corpus_caveat": "Only 26 of 50 injection pairs are genuine minimal-fix pairs. 12 are different programs (<60% similar; local-python-vulnerable is 7% similar with 83 changed lines, owasp-java-cmdi pairs BenchmarkTest00007 against BenchmarkTest00090) and 12 are large rewrites. Where the fixed side is a different program, both sides being flagged is a property of the corpus, not a failure of the model -- so the differential is reported per stratum as well as overall.", "corroborated_or_better": 0.54, "determinism_checked_pairs": 5, "deterministic_all": true, "distinguished_pairs": 16, "distinguished_rate": 0.32, "entailed_rate": 0.54, "fix_shape_analysis": { "consequence": "The dominant injection fix is a VALIDATION GUARD on the path -- allowlist, type coercion, bounds or null test -- not a sanitizer call and not a sink-shape change. Taint reachability can never separate those pairs: the flow still reaches the sink; what changed is that a guard now dominates the path. The arbiter must therefore be TAINT + DOMINANCE, not taint alone -- the same primitive task 4.1 builds for access_control, with the guard vocabulary ported in task 1.1 (allowlist_test, type_change, bounds_test, null_test) as its lookup list. Two mechanisms collapse into one.", "guard_kind_of_added_lines": { "allowlist_test": 5, "bounds_test": 1, "none": 24, "null_test": 7, "parameterized_call": 11, "type_change": 2 }, "note": "What the 50 injection fixes actually do, classified from the diff offline. This is the finding that changes the design.", "shape_introduced_at_the_sink": { "argv_list": 2, "bound_parameters": 10, "other_or_validation": 37, "sanitizer_call": 1 } }, "flat_ir_baseline": { "corroborated_one_hop": 0.239, "entailed": 0.022, "note": "2026-09-08-model-entailment-ceiling.json" }, "generated": "2026-09-08", "miss_causes": { "cpg_build_failed": { "detail": "openssl-cve-2022-1292: a C translation unit that does not parse standalone. Counted separately so it can never read as 'no vulnerability'.", "pairs": 1 }, "sink_not_in_table": { "detail": "the vulnerable sink is not in ruleset/semantic (Django ORM raw SQL `objects.raw`, `self.query`, GraphQL resolvers). Task 4.2 authors these.", "pairs": 11 }, "sink_present_but_no_flow_resolved": { "detail": "the sink is modelled but no source reaches it: the excerpt's untrusted value arrives from a context file or a framework object neither the source table nor the parameter-source rule covers.", "pairs": 11 } }, "next": [ "4.1 guard dominance over the tainted path (serves injection and access_control with one primitive)", "4.2 author the missing sinks: Django ORM raw SQL, GraphQL resolvers, and the four families with no sink at all (path, output_encoding, untrusted_destination, prototype)", "safe-shape forms still missing: argv list (subprocess.call([...])) and shell=False" ], "pair_quality_note": "Only a minimal_fix pair poses the differential question at all: when the fixed side is a different program (<60% similar) or a large rewrite (>20 changed lines), both sides being flagged is a property of the corpus, not a failure of the model. The headline distinguished_rate is therefore reported over minimal_fix pairs as well as over all pairs.", "pairs": 50, "reading": "GO on the engine. Joern builds a CPG for a pair tree in 5-30s, resolves the multi-hop flows the flat IR could not, and is deterministic: every verdict checked twice over one CPG agreed (deterministic_all=true). Entailment on the injection slice is 54% against the flat IR's 2.2% -- a 24x lift -- and 61.5% on the pairs where the differential question is well-posed. Every remaining gap is in OUR data model, not the engine, which is precisely the result that justifies adopting a CPG rather than building one.", "rows": [ { "build_failed": false, "determinism_checked": true, "deterministic": true, "distinguished": false, "fixed_rung": "model_entailed", "function": "", "language": "python", "pair": "local-python-vulnerable", "pair_quality": "different_program", "seconds": 57.9, "vuln_rung": "model_entailed", "witness": "request.args[\"expr\"] -> eval(request.args[\"expr\"]) (line 28, 1 steps)" }, { "build_failed": false, "determinism_checked": true, "deterministic": true, "distinguished": false, "fixed_rung": "model_entailed", "function": "", "language": "javascript", "pair": "local-javascript-vulnerable", "pair_quality": "different_program", "seconds": 55.2, "vuln_rung": "model_entailed", "witness": "\"make \" + req.query.target -> childProcess.execSync(\"make \" + req.query.target) (line 14, 1 steps)" }, { "build_failed": false, "determinism_checked": true, "deterministic": true, "distinguished": false, "fixed_rung": "model_entailed", "function": "", "language": "python", "pair": "sven-cwe-078-os-system", "pair_quality": "minimal_fix", "seconds": 51.3, "vuln_rung": "model_entailed", "witness": "username -> os.system(\"useradd -G docker,wheel -p \" + encPass + \" \" + username) (line 10, 2 steps)" }, { "build_failed": false, "determinism_checked": true, "deterministic": true, "distinguished": false, "fixed_rung": "model_entailed", "function": "", "language": "python", "pair": "sven-cwe-078-shell-true", "pair_quality": "minimal_fix", "seconds": 51.2, "vuln_rung": "model_entailed", "witness": "cmd -> _exec_cmd(self, cmd) (line 14, 2 steps)" }, { "build_failed": false, "determinism_checked": true, "deterministic": true, "distinguished": true, "fixed_rung": null, "function": "", "language": "python", "pair": "sven-cwe-089-execute-percent", "pair_quality": "minimal_fix", "seconds": 51.1, "vuln_rung": "model_entailed", "witness": "game -> cursor.execute(\"SELECT x,y,letter FROM moves WHERE gameID = %d;\" % game) (line 12, 3 steps)" }, { "build_failed": false, "determinism_checked": false, "deterministic": true, "distinguished": false, "fixed_rung": null, "function": "", "language": "python", "pair": "sven-cwe-089-query-concat", "pair_quality": "minimal_fix", "seconds": 29.8, "vuln_rung": null, "witness": "" }, { "build_failed": false, "determinism_checked": false, "deterministic": true, "distinguished": true, "fixed_rung": null, "function": "", "language": "javascript", "pair": "node-serialize-cve-2017-5941", "pair_quality": "minimal_fix", "seconds": 30.6, "vuln_rung": "model_entailed", "witness": "obj -> eval(\"(\" + obj + \")\") (line 4, 2 steps)" }, { "build_failed": false, "determinism_checked": false, "deterministic": true, "distinguished": false, "fixed_rung": "model_entailed", "function": "", "language": "python", "pair": "owasp-python-codeinj", "pair_quality": "large_rewrite", "seconds": 30.3, "vuln_rung": "model_entailed", "witness": "urllib.parse.unquote_plus(request.cookies.get(\"BenchmarkTest00074\", \"noCookieValueSupplied\")) -> exec(bar) (line 51, 9 steps)" }, { "build_failed": false, "determinism_checked": false, "deterministic": true, "distinguished": true, "fixed_rung": null, "function": "", "language": "python", "pair": "owasp-python-cmdi", "pair_quality": "large_rewrite", "seconds": 29.9, "vuln_rung": "model_entailed", "witness": "request.form.get(\"BenchmarkTest00165\") -> subprocess.run(argList, capture_output = True, encoding = \"utf-8\") (line 57, 15 steps)" }, { "build_failed": false, "determinism_checked": false, "deterministic": true, "distinguished": true, "fixed_rung": null, "function": "", "language": "python", "pair": "owasp-python-sqli", "pair_quality": "large_rewrite", "seconds": 29.1, "vuln_rung": "model_entailed", "witness": "request.form.get(\"BenchmarkTest00099\") -> cur.execute(sql) (line 49, 14 steps)" }, { "build_failed": false, "determinism_checked": false, "deterministic": true, "distinguished": false, "fixed_rung": "model_entailed", "function": "", "language": "java", "pair": "owasp-java-cmdi", "pair_quality": "large_rewrite", "seconds": 54.5, "vuln_rung": "model_entailed", "witness": "getHeader(\"BenchmarkTest00007\") -> r.exec(args, argsEnv) (line 61, 10 steps)" }, { "build_failed": false, "determinism_checked": false, "deterministic": true, "distinguished": false, "fixed_rung": "model_entailed", "function": "", "language": "java", "pair": "owasp-java-sqli", "pair_quality": "large_rewrite", "seconds": 54.4, "vuln_rung": "model_entailed", "witness": "getHeader(\"BenchmarkTest00008\") -> connection.prepareCall(sql) (line 57, 9 steps)" }, { "build_failed": false, "determinism_checked": false, "deterministic": true, "distinguished": true, "fixed_rung": null, "function": "", "language": "java", "pair": "juliet-java-cwe78-exec", "pair_quality": "minimal_fix", "seconds": 53.7, "vuln_rung": "model_entailed", "witness": "String data -> Runtime.getRuntime().exec(osCommand + data) (line 8, 2 steps)" }, { "build_failed": true, "determinism_checked": false, "deterministic": true, "distinguished": false, "fixed_rung": null, "function": "link_hash_cert", "language": "python", "pair": "openssl-cve-2022-1292", "pair_quality": "minimal_fix", "seconds": 1.3, "vuln_rung": null, "witness": "" }, { "build_failed": false, "determinism_checked": false, "deterministic": true, "distinguished": false, "fixed_rung": null, "function": "mutate", "language": "python", "pair": "damn-vulnerable-graphql-application-views-mutate", "pair_quality": "different_program", "seconds": 32.9, "vuln_rung": null, "witness": "" }, { "build_failed": false, "determinism_checked": false, "deterministic": true, "distinguished": false, "fixed_rung": null, "function": "resolve_pastes", "language": "python", "pair": "damn-vulnerable-graphql-application-views-resolve-pastes", "pair_quality": "minimal_fix", "seconds": 31.5, "vuln_rung": null, "witness": "" }, { "build_failed": false, "determinism_checked": false, "deterministic": true, "distinguished": false, "fixed_rung": null, "function": "resolve_system_diagnostics", "language": "python", "pair": "damn-vulnerable-graphql-application-views-resolve-system-dia", "pair_quality": "different_program", "seconds": 30.4, "vuln_rung": null, "witness": "" }, { "build_failed": false, "determinism_checked": false, "deterministic": true, "distinguished": false, "fixed_rung": null, "function": "hits_by_ip", "language": "python", "pair": "djangoat-analytics-hits-by-ip", "pair_quality": "minimal_fix", "seconds": 29.8, "vuln_rung": null, "witness": "" }, { "build_failed": false, "determinism_checked": false, "deterministic": true, "distinguished": false, "fixed_rung": null, "function": "user_view", "language": "python", "pair": "djangoat-views-user-view", "pair_quality": "different_program", "seconds": 29.1, "vuln_rung": null, "witness": "" }, { "build_failed": false, "determinism_checked": false, "deterministic": true, "distinguished": false, "fixed_rung": null, "function": "run", "language": "python", "pair": "dsvpwa-attacks-run", "pair_quality": "different_program", "seconds": 29.3, "vuln_rung": null, "witness": "" }, { "build_failed": false, "determinism_checked": false, "deterministic": true, "distinguished": false, "fixed_rung": null, "function": "do_BDR", "language": "python", "pair": "dsvpwa-handlers-do-bdr", "pair_quality": "different_program", "seconds": 32.9, "vuln_rung": null, "witness": "" }, { "build_failed": false, "determinism_checked": false, "deterministic": true, "distinguished": false, "fixed_rung": null, "function": "create", "language": "python", "pair": "dvpwa-student-create", "pair_quality": "minimal_fix", "seconds": 31.9, "vuln_rung": null, "witness": "" }, { "build_failed": false, "determinism_checked": false, "deterministic": true, "distinguished": false, "fixed_rung": null, "function": "search", "language": "python", "pair": "extremely-vulnerable-flask-app-account-search", "pair_quality": "different_program", "seconds": 32.3, "vuln_rung": null, "witness": "" }, { "build_failed": false, "determinism_checked": false, "deterministic": true, "distinguished": true, "fixed_rung": null, "function": "validate_token", "language": "python", "pair": "extremely-vulnerable-flask-app-signup-validate-token", "pair_quality": "different_program", "seconds": 29.0, "vuln_rung": "model_entailed", "witness": "session -> session.execute(text(f\"\"\"\n SELECT id, code FROM {RegistrationCode.__tablename__} WHERE code = '{code}'\n \"\"\")) (line 15, 2 steps)" }, { "build_failed": false, "determinism_checked": false, "deterministic": true, "distinguished": true, "fixed_rung": null, "function": "dashboard", "language": "python", "pair": "threatbyte-routes-dashboard", "pair_quality": "different_program", "seconds": 28.7, "vuln_rung": "model_entailed", "witness": "request.args.get('search', '') -> cursor.execute(s_query) (line 29, 8 steps)" }, { "build_failed": false, "determinism_checked": false, "deterministic": true, "distinguished": false, "fixed_rung": null, "function": "get_user", "language": "python", "pair": "vampi-user-model-get-user", "pair_quality": "different_program", "seconds": 27.9, "vuln_rung": null, "witness": "" }, { "build_failed": false, "determinism_checked": false, "deterministic": true, "distinguished": false, "fixed_rung": null, "function": "delete_user", "language": "python", "pair": "vfapi-main-delete-user", "pair_quality": "minimal_fix", "seconds": 29.1, "vuln_rung": null, "witness": "" }, { "build_failed": false, "determinism_checked": false, "deterministic": true, "distinguished": false, "fixed_rung": null, "function": "put_user", "language": "python", "pair": "vfapi-main-put-user", "pair_quality": "different_program", "seconds": 29.7, "vuln_rung": null, "witness": "" }, { "build_failed": false, "determinism_checked": false, "deterministic": true, "distinguished": false, "fixed_rung": null, "function": "sql_return_users_from_username", "language": "python", "pair": "vfapi-main-sql-return-users-from-username", "pair_quality": "minimal_fix", "seconds": 29.6, "vuln_rung": null, "witness": "" }, { "build_failed": false, "determinism_checked": false, "deterministic": true, "distinguished": true, "fixed_rung": null, "function": "do_os_system", "language": "python", "pair": "vulnpy-cmdi-do-os-system", "pair_quality": "minimal_fix", "seconds": 30.3, "vuln_rung": "model_entailed", "witness": "command -> os.system(command) (line 14, 2 steps)" }, { "build_failed": false, "determinism_checked": false, "deterministic": true, "distinguished": true, "fixed_rung": null, "function": "do_subprocess_popen", "language": "python", "pair": "vulnpy-cmdi-do-subprocess-popen", "pair_quality": "minimal_fix", "seconds": 34.7, "vuln_rung": "model_entailed", "witness": "command -> subprocess.Popen(command, stdout = subprocess.PIPE, stderr = subprocess.STDOUT, shell = True) (line 14, 2 steps)" }, { "build_failed": false, "determinism_checked": false, "deterministic": true, "distinguished": true, "fixed_rung": null, "function": "create", "language": "python", "pair": "vulpy-libuser-create", "pair_quality": "minimal_fix", "seconds": 35.2, "vuln_rung": "model_entailed", "witness": "password -> c.execute(\"INSERT INTO users (username, password, failures, mfa_enabled, mfa_secret) VALUES ('%s', '%s', '%d', '%d', '%s')\" % (username,password,0,0,'')) (line 18, 4 steps)" }, { "build_failed": false, "determinism_checked": false, "deterministic": true, "distinguished": true, "fixed_rung": null, "function": "login", "language": "python", "pair": "vulpy-libuser-login", "pair_quality": "minimal_fix", "seconds": 31.3, "vuln_rung": "model_entailed", "witness": "username -> c.execute(\"SELECT * FROM users WHERE username = '{}' and password = '{}'\".format(username, password)) (line 20, 5 steps)" }, { "build_failed": false, "determinism_checked": false, "deterministic": true, "distinguished": true, "fixed_rung": null, "function": "password_change", "language": "python", "pair": "vulpy-libuser-password-change", "pair_quality": "minimal_fix", "seconds": 30.9, "vuln_rung": "model_entailed", "witness": "password -> c.execute(\"UPDATE users SET password = '{}' WHERE username = '{}'\".format(password, username)) (line 20, 5 steps)" }, { "build_failed": false, "determinism_checked": false, "deterministic": true, "distinguished": true, "fixed_rung": null, "function": "publish", "language": "javascript", "pair": "apex-publish-static-files-2-0-0", "pair_quality": "large_rewrite", "seconds": 33.8, "vuln_rung": "model_entailed", "witness": "this -> execSync(\n\t\t\t\t'\"' + opts.sqlclPath + '\"' + // Sqlcl path\n\t\t\t\t' ' + opts.connectString + // Connect string (user/pass@server:port/sid)\n\t\t\t\t' @\"' + path.resolve(__dirname, 'lib/script') + '\"' + // Sql t (line 61, 2 steps)" }, { "build_failed": false, "determinism_checked": false, "deterministic": true, "distinguished": false, "fixed_rung": null, "function": "nativeZip", "language": "javascript", "pair": "bestzip-2-1-6", "pair_quality": "large_rewrite", "seconds": 34.0, "vuln_rung": null, "witness": "" }, { "build_failed": false, "determinism_checked": false, "deterministic": true, "distinguished": false, "fixed_rung": "model_entailed", "function": "local_scan", "language": "javascript", "pair": "clamscan-1-2-0", "pair_quality": "minimal_fix", "seconds": 50.5, "vuln_rung": "model_entailed", "witness": "this -> execFile(self.settings[self.scanner].path, self._build_clam_args(path), (err, stdout, stderr) => {\n const {is_infected, viruses} = self._process_result(stdout, path);\n\n (line 1401, 49 steps)" }, { "build_failed": false, "determinism_checked": false, "deterministic": true, "distinguished": false, "fixed_rung": "model_entailed", "function": "diskusage", "language": "javascript", "pair": "diskusage-ng-0-2-6", "pair_quality": "minimal_fix", "seconds": 33.5, "vuln_rung": "model_entailed", "witness": "this -> exec('df -k \"' + path + '\"', function(err, stdout) {\n if (err) {\n return cb(err);\n }\n\n try {\n cb(null, parse(stdout));\n } catch (e) {\n cb(e (line 19, 2 steps)" }, { "build_failed": false, "determinism_checked": false, "deterministic": true, "distinguished": false, "fixed_rung": "model_entailed", "function": "resolve", "language": "javascript", "pair": "dns-sync-0-1-0", "pair_quality": "minimal_fix", "seconds": 36.4, "vuln_rung": "model_entailed", "witness": "hostname -> shell.exec(cmd, {silent: true}) (line 22, 5 steps)" }, { "build_failed": false, "determinism_checked": false, "deterministic": true, "distinguished": true, "fixed_rung": null, "function": "performAction", "language": "javascript", "pair": "hot-formula-parser-3-0-0", "pair_quality": "large_rewrite", "seconds": 55.8, "vuln_rung": "model_entailed", "witness": "yytext -> eval(\"[\" + yytext + \"]\") (line 166, 2 steps)" }, { "build_failed": false, "determinism_checked": false, "deterministic": true, "distinguished": false, "fixed_rung": null, "function": "block", "language": "javascript", "pair": "libnmap-0-4-11", "pair_quality": "large_rewrite", "seconds": 42.5, "vuln_rung": null, "witness": "" }, { "build_failed": false, "determinism_checked": false, "deterministic": true, "distinguished": false, "fixed_rung": null, "function": "getInfo", "language": "javascript", "pair": "pdf-image-1-0-5", "pair_quality": "large_rewrite", "seconds": 36.8, "vuln_rung": null, "witness": "" }, { "build_failed": false, "determinism_checked": false, "deterministic": true, "distinguished": false, "fixed_rung": "model_entailed", "function": "ps", "language": "javascript", "pair": "pidusage-1-0-0", "pair_quality": "minimal_fix", "seconds": 37.0, "vuln_rung": "model_entailed", "witness": "this -> exec(cmd + pid, function(err, stdout, stderr) {\n if(err)\n return done(err, null)\n\n stdout = stdout.split(os.EOL)[1]\n stdout = stdout.replace(/^\\s+/, '').replace(/\\s\\s+/g, ' ').sp (line 116, 2 steps)" }, { "build_failed": false, "determinism_checked": false, "deterministic": true, "distinguished": false, "fixed_rung": null, "function": "listProcessesOnPort", "language": "javascript", "pair": "portprocesses-1-0-4", "pair_quality": "minimal_fix", "seconds": 34.8, "vuln_rung": null, "witness": "" }, { "build_failed": false, "determinism_checked": false, "deterministic": true, "distinguished": false, "fixed_rung": null, "function": "walk", "language": "javascript", "pair": "static-eval-1-1-1", "pair_quality": "minimal_fix", "seconds": 33.6, "vuln_rung": null, "witness": "" }, { "build_failed": false, "determinism_checked": false, "deterministic": true, "distinguished": true, "fixed_rung": null, "function": "thenify", "language": "javascript", "pair": "thenify-3-3-0", "pair_quality": "minimal_fix", "seconds": 30.6, "vuln_rung": "model_entailed", "witness": "this -> eval(createWrapper($$__fn__$$.name, options)) (line 29, 4 steps)" }, { "build_failed": false, "determinism_checked": false, "deterministic": true, "distinguished": false, "fixed_rung": null, "function": "template", "language": "javascript", "pair": "underscore-1-13-0-0", "pair_quality": "large_rewrite", "seconds": 31.1, "vuln_rung": null, "witness": "" }, { "build_failed": false, "determinism_checked": false, "deterministic": true, "distinguished": true, "fixed_rung": null, "function": "whereis", "language": "javascript", "pair": "whereis-0-4-0", "pair_quality": "minimal_fix", "seconds": 30.8, "vuln_rung": "model_entailed", "witness": "name -> cp.exec('which ' + name, function(error, stdout, stderr) {\n stdout = stdout.split('\\n')[0];\n if (error || stderr || stdout === '' || stdout.charAt(0) !== '/') {\n stdout = stdout.split('\\n') (line 15, 3 steps)" }, { "build_failed": false, "determinism_checked": false, "deterministic": true, "distinguished": false, "fixed_rung": null, "function": "_is_command_allowed", "language": "python", "pair": "tools-cli-tools-a96bc0", "pair_quality": "minimal_fix", "seconds": 30.9, "vuln_rung": null, "witness": "" }, { "build_failed": false, "determinism_checked": false, "deterministic": true, "distinguished": false, "fixed_rung": null, "function": "getGoogleProvider", "language": "javascript", "pair": "workers-research-utils-209444", "pair_quality": "large_rewrite", "seconds": 32.1, "vuln_rung": null, "witness": "" } ], "seconds_per_pair": { "max": 57.9, "median": 32.1, "total": 1781.0 }, "task": "model-grounded-detection 2.3 (Joern go/no-go, injection slice, parameter sources on)", "verdict": "GO", "vuln_side": { "corroborated": 0, "entailed": 27, "none": 23 } }