marius/Open_Ultra_SAST
1
0
Fork 0
mirror of https://github.com/norandom/OpenUltraSAST.git synced 2026-10-16 03:49:26 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
1d67412e2b1ad614b4db3ee9941c2baa9bb132b6
Open_Ultra_SAST/benchmarks/measurements/2026-09-08-joern-go-no-go.json
T
Marius CiepluchandClaude Opus 5 7387d2f190 measure(model-grounded-detection): task 2.3 reads GO on Joern 
Injection slice, 50 pairs, ~30 minutes of real engine time.

  model_entailed on the vulnerable side   54%  (27/50) vs the flat IR's 2.2%
    ... on minimal-fix pairs only        61.5% (16/26)
  pairs distinguished                     32%  overall, 38.5% on minimal-fix
  determinism (Req 6.4)                   every verdict identical on re-run
  cost                                    median 32s/pair, max 58s
  CPG build failures                      1

The engine is not the constraint: all 23 misses trace to our own data model --
11 sinks absent from the fact tables (Django ORM objects.raw, self.query,
GraphQL resolvers), 11 where the sink is modelled but no source reaches it, and
one C translation unit that does not parse standalone, counted separately so it
can never read as "no vulnerability". That is exactly the result that justifies
adopting a CPG rather than building one.

The finding that changes the design: classifying what the 50 injection FIXES
actually do gives other_or_validation 37, bound_parameters 10, argv_list 2,
sanitizer_call 1. The dominant fix is a validation guard on the path, not a
sanitizer and not a sink shape -- and taint reachability can never separate
those pairs, because the flow still reaches the sink and what changed is that a
guard now dominates the path. The arbiter must be TAINT + DOMINANCE, which is
the same primitive task 4.1 builds for access_control, with the guard
vocabulary ported in task 1.1 as its lookup list. Two planned mechanisms
collapse into one, and 4.1 should now precede the remaining TaintSpecs.

Reported per stratum because only 26 of 50 pairs are genuine minimal-fix pairs:
12 are different programs (local-python-vulnerable is 7% similar with 83
changed lines) and 12 are large rewrites. A raw distinguished_rate would
measure corpus quality and report it as model quality.

Also corrected in the notes: the Flask-shaped source table is a real gap but a
minor lever -- adding Django patterns would gain a source for exactly one pair.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
2026-09-08 13:26:59 +02:00

794 lines
26 KiB
JSON
Raw Blame History

{
"build_failures": 1,
"by_pair_quality": {
"different_program": {
"distinguished": 2,
"distinguished_rate": 0.1667,
"pairs": 12,
"vuln_entailed": 4,
"vuln_entailed_rate": 0.3333
},
"large_rewrite": {
"distinguished": 4,
"distinguished_rate": 0.3333,
"pairs": 12,
"vuln_entailed": 7,
"vuln_entailed_rate": 0.5833
},
"minimal_fix": {
"distinguished": 10,
"distinguished_rate": 0.3846,
"pairs": 26,
"vuln_entailed": 16,
"vuln_entailed_rate": 0.6154
}
},
"corpus_caveat": "Only 26 of 50 injection pairs are genuine minimal-fix pairs. 12 are different programs (<60% similar; local-python-vulnerable is 7% similar with 83 changed lines, owasp-java-cmdi pairs BenchmarkTest00007 against BenchmarkTest00090) and 12 are large rewrites. Where the fixed side is a different program, both sides being flagged is a property of the corpus, not a failure of the model -- so the differential is reported per stratum as well as overall.",
"corroborated_or_better": 0.54,
"determinism_checked_pairs": 5,
"deterministic_all": true,
"distinguished_pairs": 16,
"distinguished_rate": 0.32,
"entailed_rate": 0.54,
"fix_shape_analysis": {
"consequence": "The dominant injection fix is a VALIDATION GUARD on the path -- allowlist, type coercion, bounds or null test -- not a sanitizer call and not a sink-shape change. Taint reachability can never separate those pairs: the flow still reaches the sink; what changed is that a guard now dominates the path. The arbiter must therefore be TAINT + DOMINANCE, not taint alone -- the same primitive task 4.1 builds for access_control, with the guard vocabulary ported in task 1.1 (allowlist_test, type_change, bounds_test, null_test) as its lookup list. Two mechanisms collapse into one.",
"guard_kind_of_added_lines": {
"allowlist_test": 5,
"bounds_test": 1,
"none": 24,
"null_test": 7,
"parameterized_call": 11,
"type_change": 2
},
"note": "What the 50 injection fixes actually do, classified from the diff offline. This is the finding that changes the design.",
"shape_introduced_at_the_sink": {
"argv_list": 2,
"bound_parameters": 10,
"other_or_validation": 37,
"sanitizer_call": 1
}
},
"flat_ir_baseline": {
"corroborated_one_hop": 0.239,
"entailed": 0.022,
"note": "2026-09-08-model-entailment-ceiling.json"
},
"generated": "2026-09-08",
"miss_causes": {
"cpg_build_failed": {
"detail": "openssl-cve-2022-1292: a C translation unit that does not parse standalone. Counted separately so it can never read as 'no vulnerability'.",
"pairs": 1
},
"sink_not_in_table": {
"detail": "the vulnerable sink is not in ruleset/semantic (Django ORM raw SQL `objects.raw`, `self.query`, GraphQL resolvers). Task 4.2 authors these.",
"pairs": 11
},
"sink_present_but_no_flow_resolved": {
"detail": "the sink is modelled but no source reaches it: the excerpt's untrusted value arrives from a context file or a framework object neither the source table nor the parameter-source rule covers.",
"pairs": 11
}
},
"next": [
"4.1 guard dominance over the tainted path (serves injection and access_control with one primitive)",
"4.2 author the missing sinks: Django ORM raw SQL, GraphQL resolvers, and the four families with no sink at all (path, output_encoding, untrusted_destination, prototype)",
"safe-shape forms still missing: argv list (subprocess.call([...])) and shell=False"
],
"pair_quality_note": "Only a minimal_fix pair poses the differential question at all: when the fixed side is a different program (<60% similar) or a large rewrite (>20 changed lines), both sides being flagged is a property of the corpus, not a failure of the model. The headline distinguished_rate is therefore reported over minimal_fix pairs as well as over all pairs.",
"pairs": 50,
"reading": "GO on the engine. Joern builds a CPG for a pair tree in 5-30s, resolves the multi-hop flows the flat IR could not, and is deterministic: every verdict checked twice over one CPG agreed (deterministic_all=true). Entailment on the injection slice is 54% against the flat IR's 2.2% -- a 24x lift -- and 61.5% on the pairs where the differential question is well-posed. Every remaining gap is in OUR data model, not the engine, which is precisely the result that justifies adopting a CPG rather than building one.",
"rows": [
{
"build_failed": false,
"determinism_checked": true,
"deterministic": true,
"distinguished": false,
"fixed_rung": "model_entailed",
"function": "",
"language": "python",
"pair": "local-python-vulnerable",
"pair_quality": "different_program",
"seconds": 57.9,
"vuln_rung": "model_entailed",
"witness": "request.args[\"expr\"] -> eval(request.args[\"expr\"]) (line 28, 1 steps)"
},
{
"build_failed": false,
"determinism_checked": true,
"deterministic": true,
"distinguished": false,
"fixed_rung": "model_entailed",
"function": "",
"language": "javascript",
"pair": "local-javascript-vulnerable",
"pair_quality": "different_program",
"seconds": 55.2,
"vuln_rung": "model_entailed",
"witness": "\"make \" + req.query.target -> childProcess.execSync(\"make \" + req.query.target) (line 14, 1 steps)"
},
{
"build_failed": false,
"determinism_checked": true,
"deterministic": true,
"distinguished": false,
"fixed_rung": "model_entailed",
"function": "",
"language": "python",
"pair": "sven-cwe-078-os-system",
"pair_quality": "minimal_fix",
"seconds": 51.3,
"vuln_rung": "model_entailed",
"witness": "username -> os.system(\"useradd -G docker,wheel -p \" + encPass + \" \" + username) (line 10, 2 steps)"
},
{
"build_failed": false,
"determinism_checked": true,
"deterministic": true,
"distinguished": false,
"fixed_rung": "model_entailed",
"function": "",
"language": "python",
"pair": "sven-cwe-078-shell-true",
"pair_quality": "minimal_fix",
"seconds": 51.2,
"vuln_rung": "model_entailed",
"witness": "cmd -> _exec_cmd(self, cmd) (line 14, 2 steps)"
},
{
"build_failed": false,
"determinism_checked": true,
"deterministic": true,
"distinguished": true,
"fixed_rung": null,
"function": "",
"language": "python",
"pair": "sven-cwe-089-execute-percent",
"pair_quality": "minimal_fix",
"seconds": 51.1,
"vuln_rung": "model_entailed",
"witness": "game -> cursor.execute(\"SELECT x,y,letter FROM moves WHERE gameID = %d;\" % game) (line 12, 3 steps)"
},
{
"build_failed": false,
"determinism_checked": false,
"deterministic": true,
"distinguished": false,
"fixed_rung": null,
"function": "",
"language": "python",
"pair": "sven-cwe-089-query-concat",
"pair_quality": "minimal_fix",
"seconds": 29.8,
"vuln_rung": null,
"witness": ""
},
{
"build_failed": false,
"determinism_checked": false,
"deterministic": true,
"distinguished": true,
"fixed_rung": null,
"function": "",
"language": "javascript",
"pair": "node-serialize-cve-2017-5941",
"pair_quality": "minimal_fix",
"seconds": 30.6,
"vuln_rung": "model_entailed",
"witness": "obj -> eval(\"(\" + obj + \")\") (line 4, 2 steps)"
},
{
"build_failed": false,
"determinism_checked": false,
"deterministic": true,
"distinguished": false,
"fixed_rung": "model_entailed",
"function": "",
"language": "python",
"pair": "owasp-python-codeinj",
"pair_quality": "large_rewrite",
"seconds": 30.3,
"vuln_rung": "model_entailed",
"witness": "urllib.parse.unquote_plus(request.cookies.get(\"BenchmarkTest00074\", \"noCookieValueSupplied\")) -> exec(bar) (line 51, 9 steps)"
},
{
"build_failed": false,
"determinism_checked": false,
"deterministic": true,
"distinguished": true,
"fixed_rung": null,
"function": "",
"language": "python",
"pair": "owasp-python-cmdi",
"pair_quality": "large_rewrite",
"seconds": 29.9,
"vuln_rung": "model_entailed",
"witness": "request.form.get(\"BenchmarkTest00165\") -> subprocess.run(argList, capture_output = True, encoding = \"utf-8\") (line 57, 15 steps)"
},
{
"build_failed": false,
"determinism_checked": false,
"deterministic": true,
"distinguished": true,
"fixed_rung": null,
"function": "",
"language": "python",
"pair": "owasp-python-sqli",
"pair_quality": "large_rewrite",
"seconds": 29.1,
"vuln_rung": "model_entailed",
"witness": "request.form.get(\"BenchmarkTest00099\") -> cur.execute(sql) (line 49, 14 steps)"
},
{
"build_failed": false,
"determinism_checked": false,
"deterministic": true,
"distinguished": false,
"fixed_rung": "model_entailed",
"function": "",
"language": "java",
"pair": "owasp-java-cmdi",
"pair_quality": "large_rewrite",
"seconds": 54.5,
"vuln_rung": "model_entailed",
"witness": "getHeader(\"BenchmarkTest00007\") -> r.exec(args, argsEnv) (line 61, 10 steps)"
},
{
"build_failed": false,
"determinism_checked": false,
"deterministic": true,
"distinguished": false,
"fixed_rung": "model_entailed",
"function": "",
"language": "java",
"pair": "owasp-java-sqli",
"pair_quality": "large_rewrite",
"seconds": 54.4,
"vuln_rung": "model_entailed",
"witness": "getHeader(\"BenchmarkTest00008\") -> connection.prepareCall(sql) (line 57, 9 steps)"
},
{
"build_failed": false,
"determinism_checked": false,
"deterministic": true,
"distinguished": true,
"fixed_rung": null,
"function": "",
"language": "java",
"pair": "juliet-java-cwe78-exec",
"pair_quality": "minimal_fix",
"seconds": 53.7,
"vuln_rung": "model_entailed",
"witness": "String data -> Runtime.getRuntime().exec(osCommand + data) (line 8, 2 steps)"
},
{
"build_failed": true,
"determinism_checked": false,
"deterministic": true,
"distinguished": false,
"fixed_rung": null,
"function": "link_hash_cert",
"language": "python",
"pair": "openssl-cve-2022-1292",
"pair_quality": "minimal_fix",
"seconds": 1.3,
"vuln_rung": null,
"witness": ""
},
{
"build_failed": false,
"determinism_checked": false,
"deterministic": true,
"distinguished": false,
"fixed_rung": null,
"function": "mutate",
"language": "python",
"pair": "damn-vulnerable-graphql-application-views-mutate",
"pair_quality": "different_program",
"seconds": 32.9,
"vuln_rung": null,
"witness": ""
},
{
"build_failed": false,
"determinism_checked": false,
"deterministic": true,
"distinguished": false,
"fixed_rung": null,
"function": "resolve_pastes",
"language": "python",
"pair": "damn-vulnerable-graphql-application-views-resolve-pastes",
"pair_quality": "minimal_fix",
"seconds": 31.5,
"vuln_rung": null,
"witness": ""
},
{
"build_failed": false,
"determinism_checked": false,
"deterministic": true,
"distinguished": false,
"fixed_rung": null,
"function": "resolve_system_diagnostics",
"language": "python",
"pair": "damn-vulnerable-graphql-application-views-resolve-system-dia",
"pair_quality": "different_program",
"seconds": 30.4,
"vuln_rung": null,
"witness": ""
},
{
"build_failed": false,
"determinism_checked": false,
"deterministic": true,
"distinguished": false,
"fixed_rung": null,
"function": "hits_by_ip",
"language": "python",
"pair": "djangoat-analytics-hits-by-ip",
"pair_quality": "minimal_fix",
"seconds": 29.8,
"vuln_rung": null,
"witness": ""
},
{
"build_failed": false,
"determinism_checked": false,
"deterministic": true,
"distinguished": false,
"fixed_rung": null,
"function": "user_view",
"language": "python",
"pair": "djangoat-views-user-view",
"pair_quality": "different_program",
"seconds": 29.1,
"vuln_rung": null,
"witness": ""
},
{
"build_failed": false,
"determinism_checked": false,
"deterministic": true,
"distinguished": false,
"fixed_rung": null,
"function": "run",
"language": "python",
"pair": "dsvpwa-attacks-run",
"pair_quality": "different_program",
"seconds": 29.3,
"vuln_rung": null,
"witness": ""
},
{
"build_failed": false,
"determinism_checked": false,
"deterministic": true,
"distinguished": false,
"fixed_rung": null,
"function": "do_BDR",
"language": "python",
"pair": "dsvpwa-handlers-do-bdr",
"pair_quality": "different_program",
"seconds": 32.9,
"vuln_rung": null,
"witness": ""
},
{
"build_failed": false,
"determinism_checked": false,
"deterministic": true,
"distinguished": false,
"fixed_rung": null,
"function": "create",
"language": "python",
"pair": "dvpwa-student-create",
"pair_quality": "minimal_fix",
"seconds": 31.9,
"vuln_rung": null,
"witness": ""
},
{
"build_failed": false,
"determinism_checked": false,
"deterministic": true,
"distinguished": false,
"fixed_rung": null,
"function": "search",
"language": "python",
"pair": "extremely-vulnerable-flask-app-account-search",
"pair_quality": "different_program",
"seconds": 32.3,
"vuln_rung": null,
"witness": ""
},
{
"build_failed": false,
"determinism_checked": false,
"deterministic": true,
"distinguished": true,
"fixed_rung": null,
"function": "validate_token",
"language": "python",
"pair": "extremely-vulnerable-flask-app-signup-validate-token",
"pair_quality": "different_program",
"seconds": 29.0,
"vuln_rung": "model_entailed",
"witness": "session -> session.execute(text(f\"\"\"\n SELECT id, code FROM {RegistrationCode.__tablename__} WHERE code = '{code}'\n \"\"\")) (line 15, 2 steps)"
},
{
"build_failed": false,
"determinism_checked": false,
"deterministic": true,
"distinguished": true,
"fixed_rung": null,
"function": "dashboard",
"language": "python",
"pair": "threatbyte-routes-dashboard",
"pair_quality": "different_program",
"seconds": 28.7,
"vuln_rung": "model_entailed",
"witness": "request.args.get('search', '') -> cursor.execute(s_query) (line 29, 8 steps)"
},
{
"build_failed": false,
"determinism_checked": false,
"deterministic": true,
"distinguished": false,
"fixed_rung": null,
"function": "get_user",
"language": "python",
"pair": "vampi-user-model-get-user",
"pair_quality": "different_program",
"seconds": 27.9,
"vuln_rung": null,
"witness": ""
},
{
"build_failed": false,
"determinism_checked": false,
"deterministic": true,
"distinguished": false,
"fixed_rung": null,
"function": "delete_user",
"language": "python",
"pair": "vfapi-main-delete-user",
"pair_quality": "minimal_fix",
"seconds": 29.1,
"vuln_rung": null,
"witness": ""
},
{
"build_failed": false,
"determinism_checked": false,
"deterministic": true,
"distinguished": false,
"fixed_rung": null,
"function": "put_user",
"language": "python",
"pair": "vfapi-main-put-user",
"pair_quality": "different_program",
"seconds": 29.7,
"vuln_rung": null,
"witness": ""
},
{
"build_failed": false,
"determinism_checked": false,
"deterministic": true,
"distinguished": false,
"fixed_rung": null,
"function": "sql_return_users_from_username",
"language": "python",
"pair": "vfapi-main-sql-return-users-from-username",
"pair_quality": "minimal_fix",
"seconds": 29.6,
"vuln_rung": null,
"witness": ""
},
{
"build_failed": false,
"determinism_checked": false,
"deterministic": true,
"distinguished": true,
"fixed_rung": null,
"function": "do_os_system",
"language": "python",
"pair": "vulnpy-cmdi-do-os-system",
"pair_quality": "minimal_fix",
"seconds": 30.3,
"vuln_rung": "model_entailed",
"witness": "command -> os.system(command) (line 14, 2 steps)"
},
{
"build_failed": false,
"determinism_checked": false,
"deterministic": true,
"distinguished": true,
"fixed_rung": null,
"function": "do_subprocess_popen",
"language": "python",
"pair": "vulnpy-cmdi-do-subprocess-popen",
"pair_quality": "minimal_fix",
"seconds": 34.7,
"vuln_rung": "model_entailed",
"witness": "command -> subprocess.Popen(command, stdout = subprocess.PIPE, stderr = subprocess.STDOUT, shell = True) (line 14, 2 steps)"
},
{
"build_failed": false,
"determinism_checked": false,
"deterministic": true,
"distinguished": true,
"fixed_rung": null,
"function": "create",
"language": "python",
"pair": "vulpy-libuser-create",
"pair_quality": "minimal_fix",
"seconds": 35.2,
"vuln_rung": "model_entailed",
"witness": "password -> c.execute(\"INSERT INTO users (username, password, failures, mfa_enabled, mfa_secret) VALUES ('%s', '%s', '%d', '%d', '%s')\" % (username,password,0,0,'')) (line 18, 4 steps)"
},
{
"build_failed": false,
"determinism_checked": false,
"deterministic": true,
"distinguished": true,
"fixed_rung": null,
"function": "login",
"language": "python",
"pair": "vulpy-libuser-login",
"pair_quality": "minimal_fix",
"seconds": 31.3,
"vuln_rung": "model_entailed",
"witness": "username -> c.execute(\"SELECT * FROM users WHERE username = '{}' and password = '{}'\".format(username, password)) (line 20, 5 steps)"
},
{
"build_failed": false,
"determinism_checked": false,
"deterministic": true,
"distinguished": true,
"fixed_rung": null,
"function": "password_change",
"language": "python",
"pair": "vulpy-libuser-password-change",
"pair_quality": "minimal_fix",
"seconds": 30.9,
"vuln_rung": "model_entailed",
"witness": "password -> c.execute(\"UPDATE users SET password = '{}' WHERE username = '{}'\".format(password, username)) (line 20, 5 steps)"
},
{
"build_failed": false,
"determinism_checked": false,
"deterministic": true,
"distinguished": true,
"fixed_rung": null,
"function": "publish",
"language": "javascript",
"pair": "apex-publish-static-files-2-0-0",
"pair_quality": "large_rewrite",
"seconds": 33.8,
"vuln_rung": "model_entailed",
"witness": "this -> execSync(\n\t\t\t\t'\"' + opts.sqlclPath + '\"' + // Sqlcl path\n\t\t\t\t' ' + opts.connectString + // Connect string (user/pass@server:port/sid)\n\t\t\t\t' @\"' + path.resolve(__dirname, 'lib/script') + '\"' + // Sql t (line 61, 2 steps)"
},
{
"build_failed": false,
"determinism_checked": false,
"deterministic": true,
"distinguished": false,
"fixed_rung": null,
"function": "nativeZip",
"language": "javascript",
"pair": "bestzip-2-1-6",
"pair_quality": "large_rewrite",
"seconds": 34.0,
"vuln_rung": null,
"witness": ""
},
{
"build_failed": false,
"determinism_checked": false,
"deterministic": true,
"distinguished": false,
"fixed_rung": "model_entailed",
"function": "local_scan",
"language": "javascript",
"pair": "clamscan-1-2-0",
"pair_quality": "minimal_fix",
"seconds": 50.5,
"vuln_rung": "model_entailed",
"witness": "this -> execFile(self.settings[self.scanner].path, self._build_clam_args(path), (err, stdout, stderr) => {\n const {is_infected, viruses} = self._process_result(stdout, path);\n\n (line 1401, 49 steps)"
},
{
"build_failed": false,
"determinism_checked": false,
"deterministic": true,
"distinguished": false,
"fixed_rung": "model_entailed",
"function": "diskusage",
"language": "javascript",
"pair": "diskusage-ng-0-2-6",
"pair_quality": "minimal_fix",
"seconds": 33.5,
"vuln_rung": "model_entailed",
"witness": "this -> exec('df -k \"' + path + '\"', function(err, stdout) {\n if (err) {\n return cb(err);\n }\n\n try {\n cb(null, parse(stdout));\n } catch (e) {\n cb(e (line 19, 2 steps)"
},
{
"build_failed": false,
"determinism_checked": false,
"deterministic": true,
"distinguished": false,
"fixed_rung": "model_entailed",
"function": "resolve",
"language": "javascript",
"pair": "dns-sync-0-1-0",
"pair_quality": "minimal_fix",
"seconds": 36.4,
"vuln_rung": "model_entailed",
"witness": "hostname -> shell.exec(cmd, {silent: true}) (line 22, 5 steps)"
},
{
"build_failed": false,
"determinism_checked": false,
"deterministic": true,
"distinguished": true,
"fixed_rung": null,
"function": "performAction",
"language": "javascript",
"pair": "hot-formula-parser-3-0-0",
"pair_quality": "large_rewrite",
"seconds": 55.8,
"vuln_rung": "model_entailed",
"witness": "yytext -> eval(\"[\" + yytext + \"]\") (line 166, 2 steps)"
},
{
"build_failed": false,
"determinism_checked": false,
"deterministic": true,
"distinguished": false,
"fixed_rung": null,
"function": "block",
"language": "javascript",
"pair": "libnmap-0-4-11",
"pair_quality": "large_rewrite",
"seconds": 42.5,
"vuln_rung": null,
"witness": ""
},
{
"build_failed": false,
"determinism_checked": false,
"deterministic": true,
"distinguished": false,
"fixed_rung": null,
"function": "getInfo",
"language": "javascript",
"pair": "pdf-image-1-0-5",
"pair_quality": "large_rewrite",
"seconds": 36.8,
"vuln_rung": null,
"witness": ""
},
{
"build_failed": false,
"determinism_checked": false,
"deterministic": true,
"distinguished": false,
"fixed_rung": "model_entailed",
"function": "ps",
"language": "javascript",
"pair": "pidusage-1-0-0",
"pair_quality": "minimal_fix",
"seconds": 37.0,
"vuln_rung": "model_entailed",
"witness": "this -> exec(cmd + pid, function(err, stdout, stderr) {\n if(err)\n return done(err, null)\n\n stdout = stdout.split(os.EOL)[1]\n stdout = stdout.replace(/^\\s+/, '').replace(/\\s\\s+/g, ' ').sp (line 116, 2 steps)"
},
{
"build_failed": false,
"determinism_checked": false,
"deterministic": true,
"distinguished": false,
"fixed_rung": null,
"function": "listProcessesOnPort",
"language": "javascript",
"pair": "portprocesses-1-0-4",
"pair_quality": "minimal_fix",
"seconds": 34.8,
"vuln_rung": null,
"witness": ""
},
{
"build_failed": false,
"determinism_checked": false,
"deterministic": true,
"distinguished": false,
"fixed_rung": null,
"function": "walk",
"language": "javascript",
"pair": "static-eval-1-1-1",
"pair_quality": "minimal_fix",
"seconds": 33.6,
"vuln_rung": null,
"witness": ""
},
{
"build_failed": false,
"determinism_checked": false,
"deterministic": true,
"distinguished": true,
"fixed_rung": null,
"function": "thenify",
"language": "javascript",
"pair": "thenify-3-3-0",
"pair_quality": "minimal_fix",
"seconds": 30.6,
"vuln_rung": "model_entailed",
"witness": "this -> eval(createWrapper($$__fn__$$.name, options)) (line 29, 4 steps)"
},
{
"build_failed": false,
"determinism_checked": false,
"deterministic": true,
"distinguished": false,
"fixed_rung": null,
"function": "template",
"language": "javascript",
"pair": "underscore-1-13-0-0",
"pair_quality": "large_rewrite",
"seconds": 31.1,
"vuln_rung": null,
"witness": ""
},
{
"build_failed": false,
"determinism_checked": false,
"deterministic": true,
"distinguished": true,
"fixed_rung": null,
"function": "whereis",
"language": "javascript",
"pair": "whereis-0-4-0",
"pair_quality": "minimal_fix",
"seconds": 30.8,
"vuln_rung": "model_entailed",
"witness": "name -> cp.exec('which ' + name, function(error, stdout, stderr) {\n stdout = stdout.split('\\n')[0];\n if (error || stderr || stdout === '' || stdout.charAt(0) !== '/') {\n stdout = stdout.split('\\n') (line 15, 3 steps)"
},
{
"build_failed": false,
"determinism_checked": false,
"deterministic": true,
"distinguished": false,
"fixed_rung": null,
"function": "_is_command_allowed",
"language": "python",
"pair": "tools-cli-tools-a96bc0",
"pair_quality": "minimal_fix",
"seconds": 30.9,
"vuln_rung": null,
"witness": ""
},
{
"build_failed": false,
"determinism_checked": false,
"deterministic": true,
"distinguished": false,
"fixed_rung": null,
"function": "getGoogleProvider",
"language": "javascript",
"pair": "workers-research-utils-209444",
"pair_quality": "large_rewrite",
"seconds": 32.1,
"vuln_rung": null,
"witness": ""
}
],
"seconds_per_pair": {
"max": 57.9,
"median": 32.1,
"total": 1781.0
},
"task": "model-grounded-detection 2.3 (Joern go/no-go, injection slice, parameter sources on)",
"verdict": "GO",
"vuln_side": {
"corroborated": 0,
"entailed": 27,
"none": 23
}
}
Reference in New Issue View Git Blame Copy Permalink