diff --git a/.kiro/specs/engine-change-scoped-analysis/brief.md b/.kiro/specs/engine-change-scoped-analysis/brief.md index 16bab5b..cf1ae16 100644 --- a/.kiro/specs/engine-change-scoped-analysis/brief.md +++ b/.kiro/specs/engine-change-scoped-analysis/brief.md @@ -337,13 +337,31 @@ context:** is the real "#1". - Feed confirmed source shapes (e.g. library param → path join) back as guarded static facts (#2). -## Future: GEPA-style reflective retries for the LLM arm (maintainer note 2026-10-12) +## Prompt evolution is a CORE lever for the LLM arm (maintainer directive 2026-10-12) -The LLM arm (both the corpus-label audit and the search-with-proof detector) should later use a **GEPA-style -reflective-retry loop** (DSPy GEPA: reflect on failed/low-confidence outputs, evolve the prompt, retry) rather than -single-shot calls — aligned with [[learned-decisions-not-hand-tuning]] (learned decisions + local memory, not hand -rules). Deferred to a future spec; the current sizing/audit uses single-shot DeepSeek. GLM 5.3 "cyber" model is -also future-spec (brand new); DeepSeek is the working default today though it is not a security-specialised model. +**"What is key is that we evolve the prompts as well"** — and **"GEPA or comparable; this is an evolving field."** +The bedrock hunter result makes this concrete: the agentic hunter ran (64 model calls, $0.0076) and reasoned +carefully, but with its **current prompts** it flagged adjacent concerns (network-PUBLIC default, download_file) +and never named the actual `install_packages` command injection — a *prompting/targeting* miss, not a cost or +capability limit. So the LLM arm needs a **prompt-optimisation loop** that reflects on misses/low-confidence +outputs, evolves the prompt, and retries — built on **DSPy** (its GEPA / MIPRO optimisers), kept **pluggable** +behind an interface because the field moves fast (GEPA, MIPRO, TextGrad, successors). This aligns with +[[learned-decisions-not-hand-tuning]] (learned decisions + local memory, not hand rules). It applies to BOTH LLM +uses here — the search-with-proof detector and the corpus-label auditor. + +**The optimisation objective is to learn WHAT HINTS/ANNOTATIONS the model needs** to reach the real sink — not +just reword the prompt. bedrock needed the hint "`install_packages` runs `pip install` with caller-supplied package +names → command injection"; banks needed "`prompt.name` is request-derived and joined into a filesystem path → +traversal". DSPy optimises which evidence/hints to feed the signature so the model targets the true sink instead of +adjacent noise. + +**Source those hints from the framework knowledge the project ALREADY ships** — the semantic facts taxonomy +(`ruleset/semantic/*.toml`, `ruleset/frameworks.toml`) encodes sources/sinks/sanitizers per framework WITHOUT +user annotations ([[php-detection-is-engine-only]] zero-annotation value). Where framework knowledge covers the +API, feed it as a hint; where it does not (library APIs like banks/bedrock), the LLM proposes the trust boundary +and, on repeated confirmation, it graduates into the facts (the #2 feedback loop). No new annotation burden on the +user. This is now a first-class part of the "#2 guided by #1" workstream, not a deferred nicety. (DeepSeek +`deepseek-flash` is the working default; GLM 5.3 "cyber" model is future-spec.) ## POOL-P1 LABEL AUDIT (LLM, 2026-10-12) — 47% of labels are sound @@ -360,3 +378,21 @@ pool label → sound/unsound). Full artifact: `benchmarks/measurements/2026-10-1 real detection gaps. The 23 sound targets are the fair local eval set for "#2 guided by #1". - Next: run the agentic search-with-proof hunter on the sound targets (start bedrock + graphiti), size its per-vuln call count/cost, measure catch + false-alarm. Then recompute record 07 on the sound subset. + +## MODEL HUNTER on the 2 sound targets (2026-10-12) — first LLM catch the static tiers missed + +Ran `ousast scan --mode deep` (DeepSeek `deepseek-flash`, model layer + tool_hunter, 60-call/8-hotspot bound) on +bedrock + graphiti, the two sound pilot targets the static engine missed. Artifact: +`benchmarks/measurements/2026-10-12-hunter-sound-targets/`. +- **hunter 1/2 vs static engine 0/2.** graphiti: **HIT** — `falkordb_driver.py:207`, tag `family:cypher-injection`, + rationale traces `build_fulltext_query` concatenating user input into a Cypher string at line 278 (inside the + labelled function, location-matches). bedrock: **miss** — 12 suspicions on adjacent concerns, never named the + `install_packages`→`pip` command injection. +- **Cost is a non-issue:** ~$0.007/repo, 64 model calls for a whole-repo deep scan. +- **Everything stopped at `suspicion`** — the proof arm did not elevate to corroborated/entailed/execution_confirmed. + Release-grade BLOCK needs *proven* findings; measuring the proof arm's lift is the next depth. +- bedrock's miss is the concrete case for the DSPy prompt/hints loop above. + +Next: (a) measure the proof arm (can it elevate graphiti's suspicion to proven?); (b) stand up the DSPy hint loop +and re-run bedrock; (c) widen the hunter to the 23 sound targets for a catch rate; (d) then a change-scoped + proven +run for the release-relevant shape. diff --git a/benchmarks/measurements/2026-10-12-hunter-sound-targets/NOTES.md b/benchmarks/measurements/2026-10-12-hunter-sound-targets/NOTES.md new file mode 100644 index 0000000..8eff5ea --- /dev/null +++ b/benchmarks/measurements/2026-10-12-hunter-sound-targets/NOTES.md @@ -0,0 +1,43 @@ +# Model hunter on the two sound pilot targets — 2026-10-12 + +## Why +The engine-on pilot showed the static tiers (regex + Joern dataflow) catch 0/5, and the LLM adjudication tier was +disabled in the push path. Of the 5 pilot repos, the label audit found only **bedrock + graphiti (injection)** are +sound (change-located, change-introduced) fair targets — and the static engine missed both. We ran the shipped +**model hunter** (`ousast scan --mode deep`, DeepSeek `deepseek-flash`, model layer + tool_hunter enabled, +unbounded by a push deadline, bounded to 60 model calls / 8 hotspots) on those two to test whether the LLM arm +catches what the static tiers miss. + +## Result: hunter 1/2 (static engine 0/2), cheap, but both at suspicion rung +| repo | vuln | static engine | **hunter** | model calls | cost | +|---|---|---|---|---|---| +| getzep/graphiti | Cypher injection in `build_fulltext_query` | miss | **HIT (suspicion)** | 64 | $0.0071 | +| aws/bedrock-agentcore | command injection in `install_packages` | miss | **miss (suspicions on adjacent)** | 64 | $0.0076 | + +- **graphiti — correct catch.** `falkordb_driver.py:207` finding, `tags:[family:cypher-injection]`, severity medium, + confidence low, rung **suspicion**. Rationale traces it exactly: *"build_fulltext_query concatenates the sanitized + user query into a Cypher string: full_query = group_filter + ' (' + sanitized_query + ')' (line 278)"*. Line 278 + is inside the labelled `function_lines [249,280]`; head_lines include [206],[249-251],[280] → location-matches. + The static engine produced zero findings here. +- **bedrock — miss.** 12 findings, all suspicion; the 3 on the vuln file were adjacent concerns (network-PUBLIC + default @114, caller client_token @188, download_file @583) — it never named the `install_packages` → `pip + install` command injection. A prompting/targeting miss, not cost (64 calls, $0.0076) or capability. + +## Reads +1. **The LLM arm lifts catch over the static tiers** — graphiti is the first unseen vuln caught that regex+dataflow + missed. Cost is a non-issue (~$0.007/repo, 64 calls for a whole-repo deep scan). +2. **Everything stops at `suspicion`.** Neither reached corroborated/entailed/execution_confirmed — the proof arm + (sandbox verification) did not elevate them. For release-grade BLOCK the findings must be *proven*, not just + suspected. The proof arm + verification are the next depth to measure. +3. **bedrock motivates prompt evolution.** The hunter has the capability but its current prompts did not steer it to + the real sink. Directive: a **DSPy**-based prompt-optimisation loop (GEPA/MIPRO; keep pluggable) whose objective + is to learn **what hints/annotations** the model needs (bedrock: "install_packages runs pip with caller-supplied + names → command injection") — sourced from the **framework knowledge already shipped** (`ruleset/semantic/*.toml`, + `frameworks.toml`), no new user annotation; LLM-proposed boundaries graduate into the facts (#2 feedback loop). + +## Caveats +- Whole-repo deep scan (not change-scoped), so this measures detection CAPABILITY, not pre-push latency. A + change-scoped + proven run is the release-relevant shape. +- Only 2 sound targets here; widen to the 23 sound public targets next for a rate. +- graphiti's finding is suspicion/low-confidence; a human/proof confirmation is still needed before it is a + release-grade catch. Findings JSON: `bedrock-findings.json`, `graphiti-findings.json`. diff --git a/benchmarks/measurements/2026-10-12-hunter-sound-targets/bedrock-findings.json b/benchmarks/measurements/2026-10-12-hunter-sound-targets/bedrock-findings.json new file mode 100644 index 0000000..85d6f8b --- /dev/null +++ b/benchmarks/measurements/2026-10-12-hunter-sound-targets/bedrock-findings.json @@ -0,0 +1,281 @@ +{ + "findings": [ + { + "confidence": "medium", + "evidence_level": "static_corroboration", + "finding_id": "python-unsafe-eval:tests/bedrock_agentcore/evaluation/integrations/strands_agents_evals/test_end_to_end.py:26", + "function_name": null, + "line": 26, + "path": "tests/bedrock_agentcore/evaluation/integrations/strands_agents_evals/test_end_to_end.py", + "proposed_snippet": null, + "ranking_priority": 2.9, + "rationale": "Static pattern python-unsafe-eval (CWE-95) matched eval on line 26; manual verification still required.", + "reachability_conditions": [], + "reachability_evidence": [ + { + "access_declared": false, + "access_evidence": [ + "auth boundary file tag" + ], + "access_level": "authenticated", + "conditions": [], + "defines_handler": true, + "end_line": null, + "function_name": null, + "kind": "privileged_surface", + "line": null, + "name": "auth_boundary", + "path": "tests/bedrock_agentcore/evaluation/integrations/strands_agents_evals/test_end_to_end.py", + "provenance": "entrypoint:privileged_surface:authenticated", + "rationale": "privileged_surface surface classified as authenticated at identity_boundary", + "registered_by": "", + "trust_boundary": "identity_boundary" + } + ], + "reachability_status": "inferred-file-surface", + "rung": "suspicion", + "severity": "critical", + "status": "enabled", + "tags": [ + "auth_boundary", + "syscall_entry" + ], + "title": "Dynamic Python execution needs review" + }, + { + "confidence": "medium", + "evidence_level": "static_corroboration", + "finding_id": "python-unsafe-eval:tests_integ/evaluation/integrations/strands_agents_evals/test_strands_evaluation.py:30", + "function_name": null, + "line": 30, + "path": "tests_integ/evaluation/integrations/strands_agents_evals/test_strands_evaluation.py", + "proposed_snippet": null, + "ranking_priority": 1.0, + "rationale": "Static pattern python-unsafe-eval (CWE-95) matched eval on line 30; manual verification still required.", + "reachability_conditions": [], + "reachability_evidence": [], + "reachability_status": "unknown", + "rung": "suspicion", + "severity": "critical", + "status": "enabled", + "tags": [ + "syscall_entry" + ], + "title": "Dynamic Python execution needs review" + }, + { + "confidence": "medium", + "evidence_level": "static_corroboration", + "finding_id": "overlay-coverage:tests_integ/async/interactive_async_strands.py:159:path.open", + "function_name": null, + "line": 159, + "path": "tests_integ/async/interactive_async_strands.py", + "proposed_snippet": null, + "ranking_priority": 0.0, + "rationale": "uninventoried sink path.open reached from parameter", + "reachability_conditions": [], + "reachability_evidence": [], + "reachability_status": "unknown", + "rung": "suspicion", + "severity": "medium", + "status": "enabled", + "tags": [ + "overlay-originated" + ], + "title": "Overlay coverage: path.open" + }, + { + "confidence": "medium", + "evidence_level": "static_corroboration", + "finding_id": "overlay-coverage:tests_integ/async/interactive_async_strands.py:170:path.open", + "function_name": null, + "line": 170, + "path": "tests_integ/async/interactive_async_strands.py", + "proposed_snippet": null, + "ranking_priority": 0.0, + "rationale": "uninventoried sink path.open reached from parameter", + "reachability_conditions": [], + "reachability_evidence": [], + "reachability_status": "unknown", + "rung": "suspicion", + "severity": "medium", + "status": "enabled", + "tags": [ + "overlay-originated" + ], + "title": "Overlay coverage: path.open" + }, + { + "confidence": "medium", + "evidence_level": "static_corroboration", + "finding_id": "overlay-coverage:tests_integ/async/interactive_async_strands.py:211:path.open", + "function_name": null, + "line": 211, + "path": "tests_integ/async/interactive_async_strands.py", + "proposed_snippet": null, + "ranking_priority": 0.0, + "rationale": "uninventoried sink path.open reached from parameter", + "reachability_conditions": [], + "reachability_evidence": [], + "reachability_status": "unknown", + "rung": "suspicion", + "severity": "medium", + "status": "enabled", + "tags": [ + "overlay-originated" + ], + "title": "Overlay coverage: path.open" + }, + { + "confidence": "medium", + "evidence_level": "static_corroboration", + "finding_id": "overlay-coverage:tests_integ/runtime/base_test.py:44:os.system", + "function_name": null, + "line": 44, + "path": "tests_integ/runtime/base_test.py", + "proposed_snippet": null, + "ranking_priority": 0.0, + "rationale": "uninventoried sink os.system reached from parameter", + "reachability_conditions": [], + "reachability_evidence": [], + "reachability_status": "unknown", + "rung": "suspicion", + "severity": "critical", + "status": "enabled", + "tags": [ + "overlay-originated" + ], + "title": "Overlay coverage: os.system" + }, + { + "confidence": "medium", + "evidence_level": "static_corroboration", + "finding_id": "overlay-coverage:tests_integ/runtime/http_client.py:34:outbound.request", + "function_name": null, + "line": 34, + "path": "tests_integ/runtime/http_client.py", + "proposed_snippet": null, + "ranking_priority": 0.0, + "rationale": "uninventoried sink outbound.request reached from parameter", + "reachability_conditions": [], + "reachability_evidence": [], + "reachability_status": "unknown", + "rung": "suspicion", + "severity": "medium", + "status": "enabled", + "tags": [ + "overlay-originated" + ], + "title": "Overlay coverage: outbound.request" + }, + { + "confidence": "medium", + "evidence_level": "static_corroboration", + "finding_id": "overlay-coverage:tests_integ/runtime/http_client.py:44:outbound.request", + "function_name": null, + "line": 44, + "path": "tests_integ/runtime/http_client.py", + "proposed_snippet": null, + "ranking_priority": 0.0, + "rationale": "uninventoried sink outbound.request reached from parameter", + "reachability_conditions": [], + "reachability_evidence": [], + "reachability_status": "unknown", + "rung": "suspicion", + "severity": "medium", + "status": "enabled", + "tags": [ + "overlay-originated" + ], + "title": "Overlay coverage: outbound.request" + }, + { + "confidence": "low", + "evidence_level": "suspicion", + "finding_id": "tool-hunter:src/bedrock_agentcore/tools/code_interpreter_client.py:114", + "function_name": null, + "line": 114, + "path": "src/bedrock_agentcore/tools/code_interpreter_client.py", + "proposed_snippet": null, + "ranking_priority": 0.0, + "rationale": "grep_repo and read_file show create_code_interpreter builds request_params with networkConfiguration=network_configuration or {\"networkMode\": \"PUBLIC\"} (line 173). The flagged test test_create_code_interpreter_minimal asserts exactly this PUBLIC default (test file line 120), so the tests corroborate that omitting network_configuration silently provisions a publicly networked interpreter rather than failing closed. This is the production behavior the hotspot tests exercise.", + "reachability_conditions": [], + "reachability_evidence": [], + "reachability_status": "unknown", + "rung": "suspicion", + "severity": "medium", + "status": "enabled", + "tags": [ + "tool-hunter", + "family:insecure-default network configuration" + ], + "title": "create_code_interpreter defaults networkConfiguration to PUBLIC" + }, + { + "confidence": "low", + "evidence_level": "suspicion", + "finding_id": "tool-hunter:src/bedrock_agentcore/tools/code_interpreter_client.py:188", + "function_name": null, + "line": 188, + "path": "src/bedrock_agentcore/tools/code_interpreter_client.py", + "proposed_snippet": null, + "ranking_priority": 0.0, + "rationale": "read_file shows delete_code_interpreter(interpreter_id, client_token) forwards client_token verbatim into request_params[\"clientToken\"] (line 208) with no format/length validation, and the flagged test test_delete_code_interpreter_with_client_token passes \"test-token\" through. The same pattern exists in browser_client.py lines 186-206. This is an idempotency-token passthrough, not a credential, so impact is limited, but it is the production path the flagged tests cover.", + "reachability_conditions": [], + "reachability_evidence": [], + "reachability_status": "unknown", + "rung": "suspicion", + "severity": "medium", + "status": "enabled", + "tags": [ + "tool-hunter", + "family:unvalidated idempotency token passthrough" + ], + "title": "delete_code_interpreter accepts caller-supplied client_token without validation" + }, + { + "confidence": "low", + "evidence_level": "suspicion", + "finding_id": "tool-hunter:src/bedrock_agentcore/tools/code_interpreter_client.py:583", + "function_name": null, + "line": 583, + "path": "src/bedrock_agentcore/tools/code_interpreter_client.py", + "proposed_snippet": null, + "ranking_priority": 0.0, + "rationale": "grep_repo locates download_file at line 583 and download_files at line 621 in src/bedrock_agentcore/tools/code_interpreter_client.py, matching the flagged tests test_download_file_text (test line 988), test_download_file_binary (1025), test_download_file_not_found (1059) and test_download_files_empty_result (1116). The tests mock the data plane client, so the filesystem_entry tag on the hotspots refers to sandbox-side file paths passed as strings; no local filesystem write was observed in the inspected source.", + "reachability_conditions": [], + "reachability_evidence": [], + "reachability_status": "unknown", + "rung": "suspicion", + "severity": "medium", + "status": "enabled", + "tags": [ + "tool-hunter", + "family:sandbox file download surface" + ], + "title": "download_file/download_files are the production targets of the flagged download tests" + }, + { + "confidence": "low", + "evidence_level": "suspicion", + "finding_id": "tool-hunter:tests/bedrock_agentcore/tools/test_code_interpreter_client.py:1", + "function_name": null, + "line": 1, + "path": "tests/bedrock_agentcore/tools/test_code_interpreter_client.py", + "proposed_snippet": null, + "ranking_priority": 19.983, + "rationale": "All eight flagged functions (test_create_code_interpreter_minimal, test_create_code_interpreter_with_all_options, test_delete_code_interpreter, test_delete_code_interpreter_with_client_token, test_download_file_binary, test_download_file_not_found, test_download_file_text, test_download_files_empty_result) live in tests/bedrock_agentcore/tools/test_code_interpreter_client.py. Reading the file confirms they are pytest test methods that mock boto3, get_control_plane_endpoint and get_data_plane_endpoint, so the auth_boundary/crypto/filesystem_entry/network_entry/parser tags reflect the mocked production surface, not reachable production code. The high score is driven by file-level loc=1139 and nesting=7 shared across all entries, not by per-function complexity.", + "reachability_conditions": [], + "reachability_evidence": [], + "reachability_status": "unknown", + "rung": "suspicion", + "severity": "medium", + "status": "enabled", + "tags": [ + "tool-hunter", + "family:test-only complexity hotspot" + ], + "title": "Complexity hotspots are all test functions in a single test file" + } + ] +} diff --git a/benchmarks/measurements/2026-10-12-hunter-sound-targets/graphiti-findings.json b/benchmarks/measurements/2026-10-12-hunter-sound-targets/graphiti-findings.json new file mode 100644 index 0000000..2b01a24 --- /dev/null +++ b/benchmarks/measurements/2026-10-12-hunter-sound-targets/graphiti-findings.json @@ -0,0 +1,479 @@ +{ + "findings": [ + { + "confidence": "medium", + "evidence_level": "static_corroboration", + "finding_id": "python-weak-hash:graphiti_core/llm_client/client.py:127", + "function_name": null, + "line": 127, + "path": "graphiti_core/llm_client/client.py", + "proposed_snippet": null, + "ranking_priority": 4.3, + "rationale": "Static pattern python-weak-hash (CWE-327) matched md5 on line 127; manual verification still required.", + "reachability_conditions": [], + "reachability_evidence": [ + { + "access_declared": false, + "access_evidence": [ + "auth boundary file tag" + ], + "access_level": "authenticated", + "conditions": [], + "defines_handler": true, + "end_line": null, + "function_name": null, + "kind": "privileged_surface", + "line": null, + "name": "auth_boundary", + "path": "graphiti_core/llm_client/client.py", + "provenance": "entrypoint:privileged_surface:authenticated", + "rationale": "privileged_surface surface classified as authenticated at identity_boundary", + "registered_by": "", + "trust_boundary": "identity_boundary" + }, + { + "access_declared": false, + "access_evidence": [ + "network entry file tag" + ], + "access_level": "public", + "conditions": [], + "defines_handler": true, + "end_line": null, + "function_name": null, + "kind": "route", + "line": null, + "name": "network_surface", + "path": "graphiti_core/llm_client/client.py", + "provenance": "entrypoint:route:public", + "rationale": "route surface classified as public at network_request", + "registered_by": "", + "trust_boundary": "network_request" + }, + { + "access_declared": false, + "access_evidence": [ + "parser/deserialization file tag" + ], + "access_level": "public", + "conditions": [], + "defines_handler": true, + "end_line": null, + "function_name": null, + "kind": "parser", + "line": null, + "name": "parser_input", + "path": "graphiti_core/llm_client/client.py", + "provenance": "entrypoint:parser:public", + "rationale": "parser surface classified as public at attacker_controlled_input", + "registered_by": "", + "trust_boundary": "attacker_controlled_input" + } + ], + "reachability_status": "inferred-file-surface", + "rung": "suspicion", + "severity": "medium", + "status": "enabled", + "tags": [ + "auth_boundary", + "crypto", + "network_entry", + "parser" + ], + "title": "Python weak hash needs review" + }, + { + "confidence": "medium", + "evidence_level": "static_corroboration", + "finding_id": "overlay-coverage:examples/podcast/transcript_parser.py:39:path.open", + "function_name": null, + "line": 39, + "path": "examples/podcast/transcript_parser.py", + "proposed_snippet": null, + "ranking_priority": 0.0, + "rationale": "uninventoried sink path.open reached from parameter", + "reachability_conditions": [], + "reachability_evidence": [], + "reachability_status": "unknown", + "rung": "suspicion", + "severity": "medium", + "status": "enabled", + "tags": [ + "overlay-originated" + ], + "title": "Overlay coverage: path.open" + }, + { + "confidence": "medium", + "evidence_level": "static_corroboration", + "finding_id": "overlay-coverage:examples/wizard_of_oz/parser.py:6:path.open", + "function_name": null, + "line": 6, + "path": "examples/wizard_of_oz/parser.py", + "proposed_snippet": null, + "ranking_priority": 0.0, + "rationale": "uninventoried sink path.open reached from parameter", + "reachability_conditions": [], + "reachability_evidence": [], + "reachability_status": "unknown", + "rung": "suspicion", + "severity": "medium", + "status": "enabled", + "tags": [ + "overlay-originated" + ], + "title": "Overlay coverage: path.open" + }, + { + "confidence": "medium", + "evidence_level": "static_corroboration", + "finding_id": "overlay-coverage:graphiti_core/driver/kuzu_driver.py:118:execute", + "function_name": null, + "line": 118, + "path": "graphiti_core/driver/kuzu_driver.py", + "proposed_snippet": null, + "ranking_priority": 0.0, + "rationale": "uninventoried sink execute reached from parameter", + "reachability_conditions": [], + "reachability_evidence": [], + "reachability_status": "unknown", + "rung": "suspicion", + "severity": "high", + "status": "enabled", + "tags": [ + "overlay-originated" + ], + "title": "Overlay coverage: execute" + }, + { + "confidence": "low", + "evidence_level": "suspicion", + "finding_id": "tool-hunter:examples/podcast/podcast_runner.py:130", + "function_name": null, + "line": 130, + "path": "examples/podcast/podcast_runner.py", + "proposed_snippet": null, + "ranking_priority": 15.0729, + "rationale": "The file ends with asyncio.run(main(False)) at module scope (line ~130) rather than under an if __name__ == '__main__' guard, unlike the quickstart examples which do guard (quickstart_neo4j.py:241, quickstart_falkordb.py:252). Importing this module triggers clear_data(client.driver) and graph ingestion against the configured database.", + "reachability_conditions": [], + "reachability_evidence": [], + "reachability_status": "unknown", + "rung": "suspicion", + "severity": "medium", + "status": "enabled", + "tags": [ + "tool-hunter", + "family:unsafe-side-effect" + ], + "title": "Module-level asyncio.run(main(False)) executes on import" + }, + { + "confidence": "low", + "evidence_level": "suspicion", + "finding_id": "tool-hunter:examples/podcast/podcast_runner.py:36", + "function_name": null, + "line": 36, + "path": "examples/podcast/podcast_runner.py", + "proposed_snippet": null, + "ranking_priority": 15.0729, + "rationale": "neo4j_password = os.environ.get('NEO4J_PASSWORD') or 'password' (line 36) supplies a literal default credential when the env var is unset, and the same pattern appears in examples/ecommerce/runner.py:36, examples/wizard_of_oz/runner.py:35, and examples/quickstart/quickstart_neo4j.py:51. These are example scripts, but the fallback silently connects with a well-known password.", + "reachability_conditions": [], + "reachability_evidence": [], + "reachability_status": "unknown", + "rung": "suspicion", + "severity": "medium", + "status": "enabled", + "tags": [ + "tool-hunter", + "family:hardcoded-credential" + ], + "title": "Example runner hardcodes default Neo4j password fallback" + }, + { + "confidence": "low", + "evidence_level": "suspicion", + "finding_id": "tool-hunter:graphiti_core/driver/falkordb_driver.py:207", + "function_name": null, + "line": 207, + "path": "graphiti_core/driver/falkordb_driver.py", + "proposed_snippet": null, + "ranking_priority": 0.0, + "rationale": "sanitize() only translates a separator_map and collapses whitespace (lines 244-247), then build_fulltext_query concatenates the sanitized user query into a Cypher string: full_query = group_filter + ' (' + sanitized_query + ')' (line 278). The query text is interpolated into the query string rather than passed as a bound parameter, so the sanitize step is the only barrier. This is the parser/network_entry hotspot surface for the FalkorDB driver.", + "reachability_conditions": [], + "reachability_evidence": [], + "reachability_status": "unknown", + "rung": "suspicion", + "severity": "medium", + "status": "enabled", + "tags": [ + "tool-hunter", + "family:cypher-injection" + ], + "title": "FalkorDB fulltext query built by string concatenation after weak sanitize()" + }, + { + "confidence": "low", + "evidence_level": "suspicion", + "finding_id": "tool-hunter:graphiti_core/driver/falkordb_driver.py:264", + "function_name": null, + "line": 264, + "path": "graphiti_core/driver/falkordb_driver.py", + "proposed_snippet": null, + "ranking_priority": 0.0, + "rationale": "group_values = '|'.join(group_ids) (line 264) is embedded into the group_filter portion of the fulltext query string, and the length check at line 275 uses len(group_ids or '') which is a length guard, not an escaping guard. group_ids flow from caller-supplied values (e.g. graphiti.py group_ids params) into a concatenated query.", + "reachability_conditions": [], + "reachability_evidence": [], + "reachability_status": "unknown", + "rung": "suspicion", + "severity": "medium", + "status": "enabled", + "tags": [ + "tool-hunter", + "family:cypher-injection" + ], + "title": "group_ids joined into query string without parameterization" + }, + { + "confidence": "low", + "evidence_level": "suspicion", + "finding_id": "tool-hunter:graphiti_core/edges.py:191", + "function_name": null, + "line": 191, + "path": "graphiti_core/edges.py", + "proposed_snippet": null, + "ranking_priority": 17.9696, + "rationale": "get_by_group_ids builds cursor_query as a LiteralString ('AND e.uuid < $uuid' if uuid_cursor ...) and concatenates it into the query, with group_ids passed as a bound parameter (group_ids=group_ids, line 215). The group_ids themselves are parameterized, but the cursor clause is string-concatenated; this is the high-complexity edge query path flagged in the hotspot.", + "reachability_conditions": [], + "reachability_evidence": [], + "reachability_status": "unknown", + "rung": "suspicion", + "severity": "medium", + "status": "enabled", + "tags": [ + "tool-hunter", + "family:cypher-construction" + ], + "title": "Edge.get_by_group_ids interpolates cursor clause into Cypher" + }, + { + "confidence": "low", + "evidence_level": "suspicion", + "finding_id": "tool-hunter:graphiti_core/llm_client/anthropic_client.py:130", + "function_name": null, + "line": 130, + "path": "graphiti_core/llm_client/anthropic_client.py", + "proposed_snippet": null, + "ranking_priority": 16.2076, + "rationale": "_extract_json_from_text finds the first '{' and last '}' and json.loads the slice (lines ~120-135). This is the parser-tagged hotspot: it will mis-parse when the model emits multiple JSON objects or braces inside strings, and it raises ValueError carrying the full model response text in the message.", + "reachability_conditions": [], + "reachability_evidence": [], + "reachability_status": "unknown", + "rung": "suspicion", + "severity": "medium", + "status": "enabled", + "tags": [ + "tool-hunter", + "family:unsafe-parsing" + ], + "title": "_extract_json_from_text uses brace-slicing to parse model output" + }, + { + "confidence": "low", + "evidence_level": "suspicion", + "finding_id": "tool-hunter:graphiti_core/llm_client/anthropic_client.py:95", + "function_name": null, + "line": 95, + "path": "graphiti_core/llm_client/anthropic_client.py", + "proposed_snippet": null, + "ranking_priority": 16.2076, + "rationale": "When config is None, the client constructs LLMConfig() and sets config.api_key = os.getenv('ANTHROPIC_API_KEY') (lines ~93-96). This is the auth_boundary/network_entry hotspot: the API key is read from the process environment and passed to AsyncAnthropic, so any code path that instantiates the client without an explicit config silently inherits ambient credentials.", + "reachability_conditions": [], + "reachability_evidence": [], + "reachability_status": "unknown", + "rung": "suspicion", + "severity": "medium", + "status": "enabled", + "tags": [ + "tool-hunter", + "family:credential-handling" + ], + "title": "AnthropicClient falls back to ANTHROPIC_API_KEY env var when config omitted" + }, + { + "confidence": "low", + "evidence_level": "suspicion", + "finding_id": "tool-hunter:graphiti_core/llm_client/client.py:130", + "function_name": null, + "line": 130, + "path": "graphiti_core/llm_client/client.py", + "proposed_snippet": null, + "ranking_priority": 16.3098, + "rationale": "_get_cache_key builds key_str = f'{self.model}:{message_str}' and returns hashlib.md5(key_str.encode()).hexdigest() (lines ~128-131). MD5 is used here for cache-key derivation, not for a security boundary, but it is the crypto-tagged hotspot in this file and collisions could cause cross-message cache reuse of LLM responses.", + "reachability_conditions": [], + "reachability_evidence": [], + "reachability_status": "unknown", + "rung": "suspicion", + "severity": "medium", + "status": "enabled", + "tags": [ + "tool-hunter", + "family:weak-hash" + ], + "title": "LLM response cache key uses MD5 over model+messages" + }, + { + "confidence": "low", + "evidence_level": "suspicion", + "finding_id": "tool-hunter:graphiti_core/llm_client/client.py:155", + "function_name": null, + "line": 155, + "path": "graphiti_core/llm_client/client.py", + "proposed_snippet": null, + "ranking_priority": 16.3098, + "rationale": "generate_response appends the response_model JSON schema to messages[-1].content and appends MULTILINGUAL_EXTRACTION_RESPONSES to messages[0].content (lines ~150-160) before computing the cache key. Because the same list object is mutated, repeated calls accumulate appended instructions and change the cache key, which is a correctness/state-leak concern in the shared LLM client path.", + "reachability_conditions": [], + "reachability_evidence": [], + "reachability_status": "unknown", + "rung": "suspicion", + "severity": "medium", + "status": "enabled", + "tags": [ + "tool-hunter", + "family:state-mutation" + ], + "title": "generate_response mutates caller-supplied messages list in place" + }, + { + "confidence": "low", + "evidence_level": "suspicion", + "finding_id": "tool-hunter:graphiti_core/llm_client/gemini_client.py:130", + "function_name": null, + "line": 130, + "path": "graphiti_core/llm_client/gemini_client.py", + "proposed_snippet": null, + "ranking_priority": 16.3961, + "rationale": "_check_safety_blocks collects safety_ratings category/probability into safety_info and raises Exception(f'Response blocked by Gemini safety filters: {safety_details}') (lines ~120-135). Raising a bare Exception (rather than a typed error) from the response-parsing path means callers cannot distinguish safety blocks from other failures, and the message embeds model-provided content.", + "reachability_conditions": [], + "reachability_evidence": [], + "reachability_status": "unknown", + "rung": "suspicion", + "severity": "medium", + "status": "enabled", + "tags": [ + "tool-hunter", + "family:error-handling" + ], + "title": "Safety-block handling raises bare Exception with response details" + }, + { + "confidence": "low", + "evidence_level": "suspicion", + "finding_id": "tool-hunter:graphiti_core/llm_client/gemini_client.py:95", + "function_name": null, + "line": 95, + "path": "graphiti_core/llm_client/gemini_client.py", + "proposed_snippet": null, + "ranking_priority": 16.3961, + "rationale": "In __init__, when no client is supplied, self.client = genai.Client(api_key=config.api_key) (line ~95). This is the auth_boundary/network_entry hotspot for the Gemini path; the API key is taken from LLMConfig and used to build the outbound client, with no validation that the key is present before the network client is created.", + "reachability_conditions": [], + "reachability_evidence": [], + "reachability_status": "unknown", + "rung": "suspicion", + "severity": "medium", + "status": "enabled", + "tags": [ + "tool-hunter", + "family:credential-handling" + ], + "title": "GeminiClient constructs genai.Client with config.api_key" + }, + { + "confidence": "low", + "evidence_level": "suspicion", + "finding_id": "tool-hunter:graphiti_core/nodes.py:100", + "function_name": null, + "line": 100, + "path": "graphiti_core/nodes.py", + "proposed_snippet": null, + "ranking_priority": 18.8338, + "rationale": "In the KUZU and default branches, labels are interpolated via f-string into MATCH (n:{label} {{uuid: $uuid}}) (e.g. lines ~100 and ~130). The label values are hardcoded literals ('Episodic','Community','Entity'), so this is not currently attacker-controlled, but it is the high-complexity delete path flagged in the hotspot and the pattern is unsafe if labels ever become dynamic.", + "reachability_conditions": [], + "reachability_evidence": [], + "reachability_status": "unknown", + "rung": "suspicion", + "severity": "medium", + "status": "enabled", + "tags": [ + "tool-hunter", + "family:cypher-injection" + ], + "title": "Node.delete interpolates label into f-string Cypher" + }, + { + "confidence": "low", + "evidence_level": "suspicion", + "finding_id": "tool-hunter:graphiti_core/search/search_utils.py:1", + "function_name": null, + "line": 1, + "path": "graphiti_core/search/search_utils.py", + "proposed_snippet": null, + "ranking_priority": 15.6127, + "rationale": "The hotspot reports loc=1889 and nesting=35 for graphiti_core/search/search_utils.py, the largest and most deeply nested file in the set, tagged auth_boundary and network_entry with reachability=5. The grep for query construction shows this module is the search/query layer, so its depth makes the query-building paths hard to audit for parameterization.", + "reachability_conditions": [], + "reachability_evidence": [], + "reachability_status": "unknown", + "rung": "suspicion", + "severity": "medium", + "status": "enabled", + "tags": [ + "tool-hunter", + "family:complexity" + ], + "title": "search_utils.py is a 1889-line module with nesting depth 35" + }, + { + "confidence": "low", + "evidence_level": "suspicion", + "finding_id": "tool-hunter:mcp_server/graphiti_mcp_server.py:1192", + "function_name": null, + "line": 1192, + "path": "mcp_server/graphiti_mcp_server.py", + "proposed_snippet": null, + "ranking_priority": 17.4409, + "rationale": "argparse registers parser.add_argument('--destroy-graph', action='store_true', help='Destroy all Graphiti graphs') (line 1192), and the module imports clear_data from graphiti_core.utils.maintenance.graph_data_operations (line 38). This is the network_entry/auth_boundary hotspot: a single CLI flag drives destructive deletion of all graphs with no confirmation or scoping shown in the argument definition.", + "reachability_conditions": [], + "reachability_evidence": [], + "reachability_status": "unknown", + "rung": "suspicion", + "severity": "medium", + "status": "enabled", + "tags": [ + "tool-hunter", + "family:destructive-operation" + ], + "title": "MCP server exposes --destroy-graph flag that clears all graphs" + }, + { + "confidence": "low", + "evidence_level": "suspicion", + "finding_id": "tool-hunter:mcp_server/graphiti_mcp_server.py:266", + "function_name": null, + "line": 266, + "path": "mcp_server/graphiti_mcp_server.py", + "proposed_snippet": null, + "ranking_priority": 17.4409, + "rationale": "GraphitiLLMConfig.from_cli_and_env(cls, args) (line 266) and GraphitiConfig.from_cli_and_env (line 492) build configuration by combining argparse Namespace values with environment variables, and the module imports DefaultAzureCredential/get_bearer_token_provider (line 15) and AsyncAzureOpenAI (line 17). This is the auth_boundary surface where CLI-supplied values and ambient cloud credentials converge into the LLM/embedder clients.", + "reachability_conditions": [], + "reachability_evidence": [], + "reachability_status": "unknown", + "rung": "suspicion", + "severity": "medium", + "status": "enabled", + "tags": [ + "tool-hunter", + "family:credential-handling" + ], + "title": "GraphitiLLMConfig.from_cli_and_env merges CLI args with environment credentials" + } + ] +}