mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-16 03:49:26 +00:00
feat(pre-push-safety-net): export the enumerated operation inventory (M1b 10.4)
A taint answer reports traced flows, so an absent row means either that the operation is not present or that no traced path reached it. Change comparison needs that distinction to say a change introduced an operation, and reachability is deliberately bounded, so the two can never be separated from flow rows alone. That is why the veto guarding the confusion is permanently true on any real Express application, and why it could be neither discharged nor safely removed. The taint query now exports one row per operation it enumerated in scope, plus a completion marker. The marker is what makes an empty inventory meaningful: without it a scope that was never enumerated and a scope holding no such operation are indistinguishable. Comparison separates three outcomes on a changed line: the operation is present in the base without a traced flow, which is a reachability statement and not novelty; the operation is absent from an enumerated base, which establishes that the change introduced it; or the base scope was never enumerated, which stays explicitly unresolved. A malformed inventory row is not an empty inventory. The arbiter now skips any row carrying a kind, so an inventory row cannot be normalized into a flow with an empty source that names no attacker input. Verified against the real engine on an authored two-revision JavaScript control: the operation is enumerated on both sides, the base carries it with zero traced flows, the zero-operation marker appears for an empty scope, and answers and findings are unchanged. No new sink inference, alias resolver or second source IR. 1,354 tests passed, nine skipped. Rollout stays NO-GO. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
This commit is contained in:
co-authored by
Claude Opus 5
parent
86980d8aa0
commit
c73efd2bde
@@ -292,13 +292,14 @@ Preserve all completed task evidence below.
|
||||
- _Depends: 10.2_
|
||||
- _Requirements: 2.4, 3.4_
|
||||
- _Verified 2026-09-16:_ 1,347 tests passed, nine skipped; Ruff/format/mypy passed. Ambiguity is no longer inherited as a context gap by every question, and no longer blocks the transaction comparison; it is decided per operation by the correspondence classifier. Controls prove a uniquely mapped or newly introduced operation stays comparable in a file with repeated unchanged lines, an operation needing an ambiguous anchor stays unresolved, and any other transaction gap still blocks. The mapping is still withheld and still reported in aggregate coverage.
|
||||
- [ ] 10.4 Export the enumerated operation inventory so absence is provable
|
||||
- [x] 10.4 Export the enumerated operation inventory so absence is provable
|
||||
- **Prerequisite discovered 2026-09-16, not in the original plan.** The 10.1-10.3 rerun showed the remaining two vetoes cannot be scoped or relaxed soundly as written. A taint answer reports flows, so an absent row means either that the operation does not exist in the base or that no traced flow reached it. `dynamic_external_or_depth_context_unresolved` exists precisely to stop the second being read as the first, and on a real Express application it is always true, because ordinary framework and dependency calls have unresolved destinations. It therefore carries no discrimination and can never be discharged, while removing it would make an unreached operation look absent.
|
||||
- Export the operations the query actually enumerated in the question's scope, separately from the flows it traced, so absence of an operation is established by the inventory rather than by the absence of a flow. Sanitizer and source facts are unchanged; this adds no detector, alias resolver or second IR.
|
||||
- Done when a base answer distinguishes "this operation does not exist here" from "no traced flow reached it", with an unenumerated scope remaining explicitly unresolved.
|
||||
- _Boundary: contributor-scan query/evidence contract, Delta and Actionability Policy_
|
||||
- _Depends: 10.1_
|
||||
- _Requirements: 2.1, 2.2, 3.4, 8.1, 8.2_
|
||||
- _Verified 2026-09-16:_ 1,354 tests passed, nine skipped; Ruff/format/mypy passed. The taint query exports one row per enumerated operation plus a completion marker, which is what makes an empty inventory meaningful. Comparison now separates three outcomes: operation present in the base without a traced flow, operation absent from an enumerated base, and a base scope never enumerated, which stays explicitly unresolved along with any malformed inventory. The arbiter skips every row carrying a kind, so an inventory row cannot become a flow with an empty source. Real-engine smoke on an authored two-revision JavaScript control confirmed emission, the zero-operation marker and unchanged answer integrity: `benchmarks/measurements/2026-09-16-operation-inventory-smoke.json`. Pre-inventory fixtures were given an enumerated base, since an unenumerated one can no longer establish absence.
|
||||
- [ ] 10.5 Make the external and dynamic context boundary question-owned
|
||||
- M1a recorded `dynamic_external_or_depth_context_unresolved` on the target's own question. The flag is computed over every in-scope method, so one unresolved call anywhere vetoes every question.
|
||||
- With 10.4 supplying inventory evidence, bind the boundary to what it actually limits: reachability, not enumeration. A question establishing absence from the inventory no longer needs complete reachability; a question asserting a traced flow still does. Unknown ownership remains a blocker; absent evidence of independence is not proof of independence.
|
||||
|
||||
@@ -0,0 +1,86 @@
|
||||
{
|
||||
"schema_version": 1,
|
||||
"experiment": "operation-inventory-engine-smoke-v1",
|
||||
"milestone": "M1b task 10.4",
|
||||
"purpose": "Verify on the real engine that the taint query emits the enumerated operation inventory separately from traced flows, and that adding it does not disturb answers or the arbiter.",
|
||||
"provenance": {
|
||||
"engine": "joern-4.0.625",
|
||||
"engine_runtime": "7213365694f836758cb5fe2b05504cbbb735b8c2f0dea5d5dd19546a1f5fd43d",
|
||||
"facts": "32edb6e1a78f1afb08ac4248cd436be16663e0d90dc1b3aaf948f2ce49a5a8f0",
|
||||
"queries": "9e1f00272daec14cf9a4632a763fba9e5e629bf6a851f3fff81e60d665734932",
|
||||
"policy": "a641f6d1241702b4290574dd70e825176fbf0f8b03517d92f4051bac1b59dc92",
|
||||
"core": "5de41aaa2e7b8706fae265b7ba1a0aad724395ee05cb2bf99edcd8391b5aa48f",
|
||||
"semantics": "b08156ca261c602c6626fa10546d3ae926bdfa11324a1a2d5dcbce5ca0bd70f4"
|
||||
},
|
||||
"source": {
|
||||
"path": "app/api.js",
|
||||
"base_change": "const safe = \"fixed\" -> req.body.value",
|
||||
"note": "authored two-revision control, not a security claim"
|
||||
},
|
||||
"observations": [
|
||||
{
|
||||
"side": "head",
|
||||
"question": {
|
||||
"path": "app/api.js",
|
||||
"function": "handler",
|
||||
"family": "injection"
|
||||
},
|
||||
"status": "unresolved",
|
||||
"reason": "change_context_incomplete",
|
||||
"flow_rows": 1,
|
||||
"inventory_operations": [
|
||||
"eval(safe) at app/api.js:5"
|
||||
],
|
||||
"inventory_complete_marker": {
|
||||
"kind": "operation_inventory_complete",
|
||||
"operations": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"side": "head",
|
||||
"question": {
|
||||
"path": "app/api.js",
|
||||
"function": "<module>",
|
||||
"family": "injection"
|
||||
},
|
||||
"status": "unresolved",
|
||||
"reason": "change_context_incomplete",
|
||||
"flow_rows": 0,
|
||||
"inventory_operations": [],
|
||||
"inventory_complete_marker": {
|
||||
"kind": "operation_inventory_complete",
|
||||
"operations": 0
|
||||
}
|
||||
},
|
||||
{
|
||||
"side": "base",
|
||||
"question": {
|
||||
"path": "app/api.js",
|
||||
"function": "handler",
|
||||
"family": "injection"
|
||||
},
|
||||
"status": "completed",
|
||||
"reason": "query_answered_and_arbitrated",
|
||||
"flow_rows": 0,
|
||||
"inventory_operations": [
|
||||
"eval(safe) at app/api.js:5"
|
||||
],
|
||||
"inventory_complete_marker": {
|
||||
"kind": "operation_inventory_complete",
|
||||
"operations": 1
|
||||
}
|
||||
}
|
||||
],
|
||||
"limits": [
|
||||
"One authored two-revision JavaScript control; no detection, precision, recall or latency claim.",
|
||||
"Exercises inventory emission and answer integrity, not the changed-line novelty branch."
|
||||
],
|
||||
"findings_by_side": {
|
||||
"head": 1,
|
||||
"base": 0
|
||||
},
|
||||
"degradations_by_side": {
|
||||
"head": [],
|
||||
"base": []
|
||||
}
|
||||
}
|
||||
@@ -815,7 +815,24 @@
|
||||
}
|
||||
}
|
||||
|
||||
rows
|
||||
// What this question ENUMERATED in scope, independent of what it traced. A flow row proves a
|
||||
// path exists; the absence of one cannot distinguish an operation that is not present from an
|
||||
// operation no traced path reached. Change comparison needs that distinction to establish that
|
||||
// a change introduced an operation, and reachability is deliberately bounded, so the two can
|
||||
// never be separated from flow rows alone. The completion marker is what makes an EMPTY
|
||||
// inventory meaningful: without it, a scope that was never enumerated and a scope with no such
|
||||
// operation look identical. No new sink inference, alias resolution or fact is introduced here.
|
||||
val inventory = sinkCalls.l.map { sink =>
|
||||
ujson.Obj(
|
||||
"kind" -> "operation_inventory",
|
||||
"inventory" -> sink.code.take(200),
|
||||
"inventoryFile" -> sink.method.filename,
|
||||
"inventoryLine" -> sink.lineNumber.getOrElse(-1).toString,
|
||||
"inventoryMethod" -> sink.method.name
|
||||
)
|
||||
}
|
||||
|
||||
rows ++ inventory ++ List(ujson.Obj("kind" -> "operation_inventory_complete", "operations" -> inventory.size))
|
||||
}
|
||||
|
||||
println("---OUSAST-CPG-BEGIN---")
|
||||
|
||||
@@ -171,6 +171,11 @@ def _flows(rows: object, *, function: str) -> list[dict[str, object]]:
|
||||
for row in rows:
|
||||
if not isinstance(row, Mapping):
|
||||
continue
|
||||
# Answer rows carry no `kind`. Anything that names one is a separate export -- the
|
||||
# operation inventory, an evidence vector, a context location -- and normalizing it
|
||||
# here would manufacture a flow with an empty source that names no attacker input.
|
||||
if "kind" in row:
|
||||
continue
|
||||
if function and not _in_scope(row, function):
|
||||
continue
|
||||
kept.append(
|
||||
|
||||
@@ -136,6 +136,50 @@ def _operations(scan: ModelScanResult, budget: ExecutionBudget | None = None) ->
|
||||
return tuple(result)
|
||||
|
||||
|
||||
INVENTORY_ROW = "operation_inventory"
|
||||
INVENTORY_COMPLETE = "operation_inventory_complete"
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class OperationSite(Contract):
|
||||
"""One operation the query enumerated in a question's scope, traced or not."""
|
||||
|
||||
question: QuestionIdentity
|
||||
path: str
|
||||
line: int
|
||||
operation: str
|
||||
|
||||
|
||||
def _inventory(scan: ModelScanResult, question: QuestionIdentity) -> tuple[OperationSite, ...] | None:
|
||||
"""Operations enumerated for this question, or None when the scope was not enumerated.
|
||||
|
||||
None is the honest answer for an answer that carries no completion marker, including every
|
||||
artifact produced before the inventory existed. An unenumerated scope stays unresolved: the
|
||||
absence of a traced flow is not the absence of the operation. A malformed inventory row is
|
||||
likewise not an empty inventory.
|
||||
"""
|
||||
answer = next((q for q in scan.question_outcomes if q.identity == question), None)
|
||||
if answer is None or answer.raw_rows_json is None:
|
||||
return None
|
||||
sites: list[OperationSite] = []
|
||||
complete = False
|
||||
for row in json.loads(answer.raw_rows_json):
|
||||
if not isinstance(row, dict):
|
||||
continue
|
||||
if row.get("kind") == INVENTORY_COMPLETE:
|
||||
complete = True
|
||||
continue
|
||||
if row.get("kind") != INVENTORY_ROW:
|
||||
continue
|
||||
path, line, operation = row.get("inventoryFile"), row.get("inventoryLine"), row.get("inventory")
|
||||
if not isinstance(path, str) or not path or not isinstance(operation, str) or not operation.strip():
|
||||
return None
|
||||
if not str(line).isdigit() or int(str(line)) < 1:
|
||||
return None
|
||||
sites.append(OperationSite(question, path, int(str(line)), operation))
|
||||
return tuple(sites) if complete else None
|
||||
|
||||
|
||||
def _correspondence(op: EvidenceOperation, context: ChangeContext) -> tuple[tuple[str, int] | None, str]:
|
||||
"""Where this head operation maps in the base, and when it does not, why.
|
||||
|
||||
@@ -208,7 +252,9 @@ def _valid_answer(scan: ModelScanResult, question: QuestionIdentity, operations:
|
||||
return False
|
||||
rows = json.loads(answer.raw_rows_json)
|
||||
# Query bodies are operation rows. An unrecognized/malformed row is not an empty answer.
|
||||
return len(rows) == sum(op.question == question for op in operations)
|
||||
# The enumerated inventory is a separate export and is counted by `_inventory`, not here.
|
||||
body = [row for row in rows if not (isinstance(row, dict) and str(row.get("kind", "")).startswith(INVENTORY_ROW))]
|
||||
return len(body) == sum(op.question == question for op in operations)
|
||||
|
||||
|
||||
def _context_boundaries(scan: ModelScanResult, *questions: QuestionIdentity) -> tuple[str, ...]:
|
||||
@@ -473,8 +519,17 @@ def _compare_evidence(
|
||||
# The edit introduced these lines. A complete counterpart answer that
|
||||
# contains this operation elsewhere is movement, never novelty; only its
|
||||
# total absence establishes that the change introduced the flow.
|
||||
inventory = _inventory(base, counterpart) if base is not None else None
|
||||
if identical:
|
||||
reason = "operation_moved_within_change"
|
||||
elif inventory is None:
|
||||
# The base scope was never enumerated, so absence cannot be established
|
||||
# from the absence of a traced flow.
|
||||
reason = "base_operation_inventory_unavailable"
|
||||
elif any(site.operation == op.operation for site in inventory):
|
||||
# The operation is in the base. No traced flow reached it there, which is
|
||||
# a reachability statement, not evidence that the change introduced it.
|
||||
reason = "operation_present_in_base_without_traced_flow"
|
||||
elif finding.rung == Rung.ENTAILED and not op.discharged:
|
||||
novelty, reason = "new", "operation_absent_from_comparable_base"
|
||||
results.append(_candidate(finding, op, old, novelty, reason, change, context))
|
||||
|
||||
@@ -155,7 +155,7 @@ def test_changed_new_operation_in_comparable_existing_file():
|
||||
# map to, so a complete counterpart answer containing no such operation now establishes
|
||||
# absence. Before that repair this stayed unknown and no introduced flow could be seen.
|
||||
ctx = replace(context(), spans=(ChangedSpan("a.js", 4, 4, "head"),), line_correspondences=())
|
||||
assert delta(scan(), scan(rows=[], findings=False), ctx).novelty == "new"
|
||||
assert delta(scan(), scan(rows=inventory_rows(), findings=False), ctx).novelty == "new"
|
||||
assert delta(scan(), scan(line=6), ctx).novelty == "unknown" # movement is not new
|
||||
|
||||
|
||||
@@ -299,7 +299,7 @@ def changed_line_context(*, ambiguous=False, head_line=4):
|
||||
|
||||
def test_operation_introduced_on_a_changed_line_is_new_against_a_complete_base():
|
||||
"""M1b 10.1: absence is established over the counterpart answer, not over a line anchor."""
|
||||
candidate = delta(scan(), scan(rows=[], findings=False), changed_line_context())
|
||||
candidate = delta(scan(), scan(rows=inventory_rows(), findings=False), changed_line_context())
|
||||
assert (candidate.novelty, candidate.reason) == ("new", "operation_absent_from_comparable_base")
|
||||
|
||||
|
||||
@@ -411,16 +411,65 @@ def test_repeated_unchanged_lines_do_not_invalidate_a_distinct_operation():
|
||||
"""M1b 10.3: file-wide ambiguity is scoped to the operations whose anchor needs it."""
|
||||
ambiguous = "612e6a73:" + "ambiguous_line_correspondence"
|
||||
ctx = replace(changed_line_context(), unresolved_boundaries=(ambiguous,))
|
||||
assert delta(scan(), scan(rows=[], findings=False), ctx).novelty == "new"
|
||||
assert delta(scan(), scan(rows=inventory_rows(), findings=False), ctx).novelty == "new"
|
||||
|
||||
|
||||
def test_an_operation_needing_an_ambiguous_anchor_is_still_uncomparable():
|
||||
ambiguous = "612e6a73:" + "ambiguous_line_correspondence"
|
||||
ctx = replace(changed_line_context(ambiguous=True), unresolved_boundaries=(ambiguous,))
|
||||
assert delta(scan(), scan(rows=[], findings=False), ctx).novelty == "unknown"
|
||||
assert delta(scan(), scan(rows=inventory_rows(), findings=False), ctx).novelty == "unknown"
|
||||
|
||||
|
||||
def test_any_other_transaction_context_gap_still_blocks():
|
||||
ctx = replace(changed_line_context(), unresolved_boundaries=("snapshot:lfs_blob_unavailable",))
|
||||
candidate = delta(scan(), scan(rows=[], findings=False), ctx)
|
||||
assert (candidate.novelty, candidate.reason) == ("unknown", "change_context_unresolved")
|
||||
|
||||
|
||||
def inventory_rows(*operations, complete=True, path="a.js", line=4):
|
||||
rows = [
|
||||
{"kind": "operation_inventory", "inventory": op, "inventoryFile": path, "inventoryLine": line, "inventoryMethod": "handler"}
|
||||
for op in operations
|
||||
]
|
||||
return rows + ([{"kind": "operation_inventory_complete", "operations": len(rows)}] if complete else [])
|
||||
|
||||
|
||||
def test_an_operation_present_in_the_base_inventory_is_not_new():
|
||||
"""M1b 10.4: no traced flow is a reachability statement, not absence of the operation."""
|
||||
base = scan(rows=inventory_rows("exec(cmd)"), findings=False)
|
||||
candidate = delta(scan(), base, changed_line_context())
|
||||
assert candidate.novelty == "unknown"
|
||||
assert candidate.reason == "operation_present_in_base_without_traced_flow"
|
||||
|
||||
|
||||
def test_an_enumerated_base_without_the_operation_establishes_absence():
|
||||
base = scan(rows=inventory_rows("query(sql, params)"), findings=False)
|
||||
candidate = delta(scan(), base, changed_line_context())
|
||||
assert (candidate.novelty, candidate.reason) == ("new", "operation_absent_from_comparable_base")
|
||||
|
||||
|
||||
def test_an_empty_but_enumerated_scope_establishes_absence():
|
||||
base = scan(rows=inventory_rows(), findings=False)
|
||||
assert delta(scan(), base, changed_line_context()).novelty == "new"
|
||||
|
||||
|
||||
def test_an_unenumerated_base_scope_stays_unresolved():
|
||||
base = scan(rows=[], findings=False)
|
||||
candidate = delta(scan(), base, changed_line_context())
|
||||
assert candidate.novelty == "unknown"
|
||||
assert candidate.reason == "base_operation_inventory_unavailable"
|
||||
|
||||
|
||||
def test_a_malformed_inventory_row_is_not_an_empty_inventory():
|
||||
broken = [{"kind": "operation_inventory", "inventory": "exec(cmd)", "inventoryFile": "a.js", "inventoryLine": "0"}]
|
||||
broken.append({"kind": "operation_inventory_complete", "operations": 1})
|
||||
candidate = delta(scan(), scan(rows=broken, findings=False), changed_line_context())
|
||||
assert candidate.reason == "base_operation_inventory_unavailable"
|
||||
|
||||
|
||||
def test_inventory_rows_do_not_count_as_unparsed_answer_body():
|
||||
"""A completed answer carrying an inventory is still a valid answer, not a malformed one."""
|
||||
head = scan()
|
||||
rows = json.loads(head.question_outcomes[0].raw_rows_json) + inventory_rows("exec(cmd)")
|
||||
head = replace(head, question_outcomes=(replace(head.question_outcomes[0], raw_rows_json=json.dumps(rows)),))
|
||||
assert delta(head, scan(rows=inventory_rows("query(sql, params)"), findings=False), changed_line_context()).novelty == "new"
|
||||
|
||||
Reference in New Issue
Block a user