Files
Open_Ultra_SAST/benchmarks/measurements/2026-09-16-operation-inventory-smoke.json
T
Marius CiepluchandClaude Opus 5 c1cc332bc1 feat(pre-push-safety-net): stop declared exclusion vetoing first-party answers (M1b 10.6)
vendor_semantics_unresolved was grouped with the graph integrity failures, so any
declared exclusion demoted every otherwise complete answer in the repository to
graph_incomplete. Every real repository excludes its dependencies, so this fired
on all of them, including on a complete first-party census with successful query
receipts.

Graph boundaries are now split by what they mean. An integrity failure says the
graph could not be read or is incomplete, so an answer drawn from it is demoted:
unparsed files, an empty or sharded graph, an incomplete or unavailable census,
an unresolved symlink, an unsupported frontend, unreadable source, an ambiguous
frontend path and unvalidated TypeScript property support all keep that behavior.
A declared exclusion is a deliberate scope choice. It stays physical, stays out of
targets and graphs, and stays reported in coverage, but it is not evidence that a
first-party answer is wrong.

Whether a particular answer needed the excluded semantics is carried by that
question's own unresolved call destinations, recorded per question in 10.5, rather
than assumed for every answer in the repository.

Controls exercise the real partition logic rather than an injected reason: a
vendored directory leaves the answer completed with the exclusion still reported,
an unresolved symlink still demotes, and the two together still demote.

1,365 tests passed, nine skipped. Rollout stays NO-GO.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-16 17:35:00 +02:00

124 lines
4.2 KiB
JSON

{
"schema_version": 1,
"experiment": "operation-inventory-engine-smoke-v1",
"milestone": "M1b task 10.4",
"purpose": "Verify on the real engine that the taint query emits the enumerated operation inventory separately from traced flows, and that adding it does not disturb answers or the arbiter.",
"provenance": {
"engine": "joern-4.0.625",
"engine_runtime": "7213365694f836758cb5fe2b05504cbbb735b8c2f0dea5d5dd19546a1f5fd43d",
"facts": "32edb6e1a78f1afb08ac4248cd436be16663e0d90dc1b3aaf948f2ce49a5a8f0",
"queries": "9e1f00272daec14cf9a4632a763fba9e5e629bf6a851f3fff81e60d665734932",
"policy": "a641f6d1241702b4290574dd70e825176fbf0f8b03517d92f4051bac1b59dc92",
"core": "5de41aaa2e7b8706fae265b7ba1a0aad724395ee05cb2bf99edcd8391b5aa48f",
"semantics": "b08156ca261c602c6626fa10546d3ae926bdfa11324a1a2d5dcbce5ca0bd70f4"
},
"source": {
"path": "app/api.js",
"base_change": "const safe = \"fixed\" -> req.body.value",
"note": "authored two-revision control, not a security claim"
},
"observations": [
{
"side": "head",
"question": {
"path": "app/api.js",
"function": "handler",
"family": "injection"
},
"status": "unresolved",
"reason": "change_context_incomplete",
"flow_rows": 1,
"inventory_operations": [
"eval(safe) at app/api.js:5"
],
"inventory_complete_marker": {
"kind": "operation_inventory_complete",
"operations": 1
}
},
{
"side": "head",
"question": {
"path": "app/api.js",
"function": "<module>",
"family": "injection"
},
"status": "unresolved",
"reason": "change_context_incomplete",
"flow_rows": 0,
"inventory_operations": [],
"inventory_complete_marker": {
"kind": "operation_inventory_complete",
"operations": 0
}
},
{
"side": "base",
"question": {
"path": "app/api.js",
"function": "handler",
"family": "injection"
},
"status": "completed",
"reason": "query_answered_and_arbitrated",
"flow_rows": 0,
"inventory_operations": [
"eval(safe) at app/api.js:5"
],
"inventory_complete_marker": {
"kind": "operation_inventory_complete",
"operations": 1
}
}
],
"limits": [
"One authored two-revision JavaScript control; no detection, precision, recall or latency claim.",
"Exercises inventory emission and answer integrity, not the changed-line novelty branch."
],
"findings_by_side": {
"head": 1,
"base": 0
},
"degradations_by_side": {
"head": [],
"base": []
},
"after_task_10_5": {
"purpose": "Same control after binding the reachability boundary to what it limits.",
"provenance": {
"engine": "joern-4.0.625",
"queries": "c170a0d445b9bf2af2dc5ffedfbf276e89887cff963fc99cd737506a840341e2",
"policy": "422ed83b1ea418a692c722554f728b843bb34e611686a8c60de393d76c8bce46",
"core": "002f0901a9f6b27d56afa6b9e6e9db83ee5d1fed5a49a7280525f53aaa9f585f",
"semantics": "21e6641c98406181446cce13b23cd52f7b3f5196d361368e0e19dbeaae0dd581"
},
"handler_injection_status": {
"head": [
"completed",
"query_answered_and_arbitrated"
],
"base": [
"completed",
"query_answered_and_arbitrated"
]
},
"candidates": [
{
"site": "app/api.js:5:handler",
"novelty": "new",
"reason": "source_connection_absent_from_comparable_base",
"witness": "req.body.value -> eval(safe) in app/api.js (line 5, 3 steps)",
"admitted": false,
"admission_reasons": [
"capability_unavailable"
]
}
],
"result": {
"finding_status": "none",
"coverage_status": "incomplete"
},
"note": "Both target questions now complete and the production comparison reaches a new verdict with an exact witness. The single remaining admission reason is capability_unavailable, which is the deliberate eligibility gate: no capability is qualified, so no alert is emitted. This is an authored one-file control, not a real vulnerability, and it qualifies nothing."
}
}