feat(pre-push-safety-net): exclude vendor inputs and partition frontend coverage

This commit is contained in:
Marius Ciepluch
2026-09-13 11:53:29 +02:00
parent 51fabef9ea
commit 6ba2b06353
29 changed files with 1996 additions and 72 deletions
+1 -1
View File
@@ -1,6 +1,6 @@
# Brief: pre-push-safety-net # Brief: pre-push-safety-net
Status: requirements, design and tasks approved on 2026-09-12; groups 1 and 2 plus tasks 3.1–3.3 verified complete (10/27 subtasks); task 3.4 blocked on frontend test retention; groups 4–8 pending. Language: English. Status: requirements, design and tasks approved on 2026-09-12; groups 1–3 verified complete (11/27 subtasks), including the approved contributor-scan frontend-retention prerequisite; groups 4–8 pending. Language: English.
## Problem ## Problem
@@ -15,10 +15,10 @@ coverage limits.
## Verification state ## Verification state
Tasks 3.1–3.3 are independently approved and verified (10/27 tasks overall). Group 3 is independently approved and VERIFIED (TASK scope): tasks 3.1–3.4 are
Task 3.4 is blocked on a confirmed frontend retention prerequisite; group 3 is complete, bringing the pre-push spec to 11/27 tasks. The approved contributor-scan
not complete. The proposed foundation repair is in frontend-retention prerequisite 2.18 is also verified. Groups 4–8 remain pending;
`../contributor-scan/frontend-retention-prerequisite.md`. this is not feature GO, capability admission or a hook latency claim.
## Baseline ## Baseline
@@ -130,7 +130,7 @@ line enforced security, that the defect is new, or that a hook meets its latency
Ten of 27 tasks are now complete; task 3.4 remains for this group. Ten of 27 tasks are now complete; task 3.4 remains for this group.
## Task 3.4 blocked verification ## Historical task 3.4 blocker
Claim type TASK; status NOT_VERIFIED. The implementer returned BLOCKED, and an Claim type TASK; status NOT_VERIFIED. The implementer returned BLOCKED, and an
independent fresh-context Kiro debug investigation returned STOP_FOR_HUMAN with independent fresh-context Kiro debug investigation returned STOP_FOR_HUMAN with
@@ -169,3 +169,44 @@ uncommitted pending the foundation repair, review and fresh runtime verification
The packaged failure preceded a comment-only clarification in `cpg/backend.py` The packaged failure preceded a comment-only clarification in `cpg/backend.py`
removing the disproven attribution to legacy frontend autodetection. No executable removing the disproven attribution to legacy frontend autodetection. No executable
backend behavior changed after that measurement. backend behavior changed after that measurement.
## Task 3.4 final verification
Status VERIFIED, claim type TASK; independent review APPROVED after two concrete
error-path repairs. The final suite passed 1,151 tests with nine skipped in
100.55 seconds. Ruff, formatting (230 files), mypy (109 source files), compileall,
diff and all three regression gates passed.
The approved foundation repair is committed as `51fabef`. Physically filtered
frontend inputs retain original source paths and bytes; the existing layout facts
mark JavaScript/TypeScript test conventions unshipped. One existing ranker decision
and region/deadline budget cover supported partitions. Browser-path source remains
available, while runtime origin stays unspecified without declarations. Vendor,
cross-language, TypeScript-property and C-bounds limits remain explicit.
Exact source names now travel in taint, dominance and configuration batch censuses.
PHP aggregation preserves them only when every shard provides named evidence.
Synthetic `<unknown>` nodes cannot mask missing real files. Unreadable unknown
extensions produce `source_unreadable` coverage instead of raising PermissionError.
Legacy callbacks that omit census entirely retain their existing compatibility
behavior; production queries emit the named census, and unsupported/partial
sharded evidence cannot establish a complete negative.
`benchmarks/measurements/2026-09-13-partition-scope-smoke.json` records the final
image and ten matching source/query/fact hashes. The real mixed graph retains
seven JavaScript files (296 bytes), including tests and browser source, plus two
PHP files (97 bytes), with vendor nodes absent. One question is selected and three
deferred; unsupported Go is reported. No false empty/unavailable census remains.
PHP and JavaScript change-context controls both complete with unchanged-helper
witnesses, preserving dirty checkout/index state and cleaning snapshot scratch.
A real Python graph supplies named census from all three production query kinds.
Those unchanged Scala sources were verified before the separate PHP transport and
discovery repairs; the artifact records that narrower provenance explicitly. A
real filesystem permission probe verifies the repaired error path, using a clearly
labeled synthetic graph callback.
The mixed smoke took 197.76 seconds under a 900-second lab allowance; PHP/JavaScript
context checks took 117.3/95.9 seconds. These manually declared controls do not
establish automatic scope discovery, production hook latency or alert quality.
Group 4 now owns novelty comparison, actionability admission and compact output.
+12 -1
View File
@@ -6,7 +6,7 @@
- User direction: a scalable pre-push safety net as AI accelerates development; WordPress is one case. - User direction: a scalable pre-push safety net as AI accelerates development; WordPress is one case.
- Clarifications: advisory must be actionable, not SAST noise; the ranker is the scope mechanism; - Clarifications: advisory must be actionable, not SAST noise; the ranker is the scope mechanism;
third-party vendor code must remain outside targets and graphs. third-party vendor code must remain outside targets and graphs.
- Requirements, design and tasks are now approved, and groups 1 and 2 plus tasks 3.1–3.3 are verified complete; task 3.4 is blocked on JavaScript frontend test retention; see - Requirements, design and tasks are now approved, and groups 1–3 are verified complete after the approved frontend-retention repair; see
`tasks.md` for verified completion. The discovery log below preserves the earlier drafting state `tasks.md` for verified completion. The discovery log below preserves the earlier drafting state
and measurements. No hook latency claim is made. and measurements. No hook latency claim is made.
@@ -167,3 +167,14 @@ Direct frontend invocation reproduced the omission. Two unconditional upstream
test filters require an owning foundation repair; see test filters require an owning foundation repair; see
`../contributor-scan/frontend-retention-prerequisite.md`. The runtime failure is `../contributor-scan/frontend-retention-prerequisite.md`. The runtime failure is
not overridden by passing mocks, static checks or a populated server graph. not overridden by passing mocks, static checks or a populated server graph.
### Group 3 completed after frontend and coverage repairs
The approved contributor-scan 2.18 adaptation preserves test directories and
filename suffixes through both pinned JavaScript frontend filters. Task 3.4 then
passed independent review with exact graph-path census, including PHP aggregation.
Review also repaired unreadable discovery inputs throwing rather than reporting
incomplete coverage. A synthetic graph file must never compensate for an omitted
real source, and a wrapper must not discard evidence supplied by a frontend.
See `implementation-group-3.md` for final evidence. Eleven of 27 tasks are complete;
change attribution, actionability admission and compact output are next.
+7 -13
View File
@@ -1,7 +1,7 @@
{ {
"feature_name": "pre-push-safety-net", "feature_name": "pre-push-safety-net",
"created_at": "2026-09-12T15:35:00Z", "created_at": "2026-09-12T15:35:00Z",
"updated_at": "2026-09-13T09:17:01.788332+00:00", "updated_at": "2026-09-13T09:50:01.821464+00:00",
"language": "en", "language": "en",
"phase": "implementation", "phase": "implementation",
"approvals": { "approvals": {
@@ -19,7 +19,7 @@
} }
}, },
"ready_for_implementation": true, "ready_for_implementation": true,
"implementation_approval_note": "Groups 1 and 2 verified complete. Maintainer authorized continuing group 3; remaining approved tasks execute in dependency order with independent review and fresh verification.", "implementation_approval_note": "Groups 1\u20133 verified complete. Contributor-scan frontend-retention prerequisite 2.18 approved and verified. Remaining approved tasks require independent review and fresh verification.",
"depends_on": [ "depends_on": [
"contributor-scan" "contributor-scan"
], ],
@@ -77,23 +77,17 @@
"2.3", "2.3",
"3.1", "3.1",
"3.2", "3.2",
"3.3" "3.3",
"3.4"
], ],
"remaining_tasks": 17, "remaining_tasks": 16,
"verification_record": "implementation-group-3.md", "verification_record": "implementation-group-3.md",
"verification_records": [ "verification_records": [
"implementation-group-1.md", "implementation-group-1.md",
"implementation-group-2.md", "implementation-group-2.md",
"implementation-group-3.md" "implementation-group-3.md"
], ],
"next_group": 3, "next_group": 4,
"blocked_tasks": [ "blocked_tasks": []
{
"task": "3.4",
"reason": "Pinned JavaScript frontend silently excludes first-party tests; missing contributor-scan prerequisite.",
"record": "../contributor-scan/frontend-retention-prerequisite.md",
"next_action": "implement_approved_contributor_scan_2.18"
}
]
} }
} }
+6 -5
View File
@@ -1,7 +1,7 @@
# Implementation Plan: pre-push-safety-net # Implementation Plan: pre-push-safety-net
Status: task plan approved by the maintainer's "approve." on 2026-09-12. Groups 1 and 2 (tasks 1.1–2.3) Status: task plan approved by the maintainer's "approve." on 2026-09-12. Groups 1–3 (tasks 1.1–3.4)
are complete after independent review and fresh verification. Group 3 has 3.1–3.3 verified; 3.4 is blocked on frontend test retention. Groups 4–8 remain pending; are complete after independent review and fresh verification (11/27 subtasks). Groups 4–8 remain pending;
the hook and capability admission are not yet implemented. See `implementation-group-1.md` and `implementation-group-2.md`. the hook and capability admission are not yet implemented. See `implementation-group-1.md` and `implementation-group-2.md`.
## Execution contract ## Execution contract
@@ -78,7 +78,7 @@ the hook and capability admission are not yet implemented. See `implementation-g
- _Requirements: 1.4, 2.1, 2.4, 3.4_ - _Requirements: 1.4, 2.1, 2.4, 3.4_
- _Verified 2026-09-12:_ independent review APPROVED; fresh final suite 1,069 passed, 9 skipped; focused 148 passed. Global Ruff/format, touched mypy and regression gates passed. PHP/JavaScript fixtures retain removed spans and unchanged-operation anchors across declaration renames; uncertain moves, repeated lines, binary and unavailable inputs remain unresolved. Forty independent real-Git comparisons checked exact spans/anchors. Network-disabled image smoke preserved committed PHP/JavaScript bytes and dirty non-HEAD state, with cleanup verified; see `implementation-group-2.md`. Lexical anchors do not establish semantic impact or novelty. - _Verified 2026-09-12:_ independent review APPROVED; fresh final suite 1,069 passed, 9 skipped; focused 148 passed. Global Ruff/format, touched mypy and regression gates passed. PHP/JavaScript fixtures retain removed spans and unchanged-operation anchors across declaration renames; uncertain moves, repeated lines, binary and unavailable inputs remain unresolved. Forty independent real-Git comparisons checked exact spans/anchors. Network-disabled image smoke preserved committed PHP/JavaScript bytes and dirty non-HEAD state, with cleanup verified; see `implementation-group-2.md`. Lexical anchors do not establish semantic impact or novelty.
- [ ] 3. Integrate bounded execution and ranker scope with the existing engine - [x] 3. Integrate bounded execution and ranker scope with the existing engine
- [x] 3.1 (P) Enforce one remaining-time budget in the backend - [x] 3.1 (P) Enforce one remaining-time budget in the backend
- Thread the absolute deadline through readability probes, frontend builds, overlays, census, query batches, fallbacks and retries; no phase gets a fresh full budget. - Thread the absolute deadline through readability probes, frontend builds, overlays, census, query batches, fallbacks and retries; no phase gets a fresh full budget.
- Bound process-group termination and lease/scratch cleanup by the cancellation allowance, replacing any longer fixed cleanup waits on the push path. - Bound process-group termination and lease/scratch cleanup by the cancellation allowance, replacing any longer fixed cleanup waits on the push path.
@@ -103,13 +103,14 @@ the hook and capability admission are not yet implemented. See `implementation-g
- _Depends: 2.3, 3.2_ - _Depends: 2.3, 3.2_
- _Requirements: 2.1, 2.2, 2.4, 8.1, 8.2_ - _Requirements: 2.1, 2.2, 2.4, 8.1, 8.2_
- _Verified 2026-09-13:_ independent review APPROVED after an unknown-call boundary repair; fresh suite 1,128 passed, nine skipped; global Ruff/format/mypy passed. Real PHP/JavaScript Git guard-removal snapshots reached unchanged helper-file witnesses with local dirty state preserved. Production negative regression reproduced then fixed unknown(req.body) being pruned as tier zero: modeled arguments no longer summarize unknown consumers. Existing ranking and detector semantics unchanged; unsupported projections remain explicit. Evidence: `benchmarks/measurements/2026-09-13-change-context-smoke.json`. Lexical attribution is not semantic novelty or alert admission. - _Verified 2026-09-13:_ independent review APPROVED after an unknown-call boundary repair; fresh suite 1,128 passed, nine skipped; global Ruff/format/mypy passed. Real PHP/JavaScript Git guard-removal snapshots reached unchanged helper-file witnesses with local dirty state preserved. Production negative regression reproduced then fixed unknown(req.body) being pruned as tier zero: modeled arguments no longer summarize unknown consumers. Existing ranking and detector semantics unchanged; unsupported projections remain explicit. Evidence: `benchmarks/measurements/2026-09-13-change-context-smoke.json`. Lexical attribution is not semantic novelty or alert admission.
- [ ] 3.4 Apply vendor exclusion and explicit language partitions - [x] 3.4 Apply vendor exclusion and explicit language partitions
- _Blocked:_ Joern 4.0.625/ASTGen 3.50.1 omit first-party JavaScript tests despite readable filtered inputs. Independent debug returned STOP_FOR_HUMAN; review the owning `contributor-scan/frontend-retention-prerequisite.md` before resuming. Implementation remains uncommitted and unapproved; runtime evidence: `benchmarks/measurements/2026-09-13-partition-retention-blocker.json`. - _Prerequisite resolved 2026-09-13:_ contributor-scan 2.18 independently approved and verified at `51fabef`; both JavaScript test filters repaired by explicit versioned input policy. Task 3.4 subsequently passed its own independent review.
- Reuse declared layout exclusions before target enumeration and graph build; retain unshipped first-party tests and separate browser/server or language capability coverage. - Reuse declared layout exclusions before target enumeration and graph build; retain unshipped first-party tests and separate browser/server or language capability coverage.
- Build supported frontend partitions explicitly; preserve unknown cross-language and vendor semantics without restoring excluded sources. - Build supported frontend partitions explicitly; preserve unknown cross-language and vendor semantics without restoring excluded sources.
- Done when PHP and Node graphs/targets contain no declared vendor code, first-party callers remain visible, and unsupported partitions appear as coverage limits rather than clean results. - Done when PHP and Node graphs/targets contain no declared vendor code, first-party callers remain visible, and unsupported partitions appear as coverage limits rather than clean results.
- _Boundary: Ranker Scope Contract, Cache and Engine Boundary integration_ - _Boundary: Ranker Scope Contract, Cache and Engine Boundary integration_
- _Requirements: 2.2, 2.3, 7.3, 8.2, 8.4_ - _Requirements: 2.2, 2.3, 7.3, 8.2, 8.4_
- _Verified 2026-09-13:_ independent review APPROVED; 1,151 passed, nine skipped; Ruff/format/mypy pass. Real PHP/JavaScript graphs retain tests/browser source, omit vendor nodes and share one ranker limit. Named census survives PHP aggregation; missing source names and unreadable discovery inputs remain explicit. Python exercises all three batch census schemas. Evidence: `benchmarks/measurements/2026-09-13-partition-scope-smoke.json`. No hook capability or latency claim.
- [ ] 4. Implement change attribution and actionable output - [ ] 4. Implement change attribution and actionable output
- [ ] 4.1 Compare targeted base/head evidence for defect novelty - [ ] 4.1 Compare targeted base/head evidence for defect novelty
+8 -7
View File
@@ -27,7 +27,7 @@ engine are out of scope.
- Equal-score tie-breaking alone cannot bring PMPro or WP Statistics under 50. The existing - Equal-score tie-breaking alone cannot bring PMPro or WP Statistics under 50. The existing
ranker is useful, but its measured ordering is not enabled by the ordinary CLI caller. ranker is useful, but its measured ordering is not enabled by the ordinary CLI caller.
- Docker/Compose/ops now default to 4.0.625; task 1.1 passed local PHP/JavaScript image smoke and independent review. No host deployment or hook latency claim follows from that runtime evidence. - Docker/Compose/ops now default to 4.0.625; task 1.1 passed local PHP/JavaScript image smoke and independent review. No host deployment or hook latency claim follows from that runtime evidence.
- `contributor-scan` has 24/40 checked tasks; historical checkboxes are not rewritten by this review. - `contributor-scan` has 25/41 checked tasks, including the approved and verified frontend-retention prerequisite 2.18.
`flow-aware-ranking` is still an unapproved brief despite implementation history. `flow-aware-ranking` is still an unapproved brief despite implementation history.
## Existing Spec Updates ## Existing Spec Updates
@@ -46,15 +46,16 @@ engine are out of scope.
- [ ] pre-push-safety-net — Immutable pushed tips, existing-ranker integration, compatible artifact - [ ] pre-push-safety-net — Immutable pushed tips, existing-ranker integration, compatible artifact
reuse, total deadline, change attribution, actionability admission and compact hook behavior. reuse, total deadline, change attribution, actionability admission and compact hook behavior.
Dependencies: contributor-scan. Requirements/design approved 2026-09-12; tasks generated Dependencies: contributor-scan. Requirements/design approved 2026-09-12; tasks generated
(8 groups, 27 executable subtasks), independently reviewed and approved; groups 1 and 2 and tasks 3.1–3.3 are complete (10/27 tasks); 3.4 is blocked. (8 groups, 27 executable subtasks), independently reviewed and approved; groups 1–3 are complete (11/27 tasks), including the resolved frontend-retention prerequisite.
### Next concrete step ### Next concrete step
Groups 1 and 2 and tasks 3.1–3.3 have passed independent review: runtime, inputs, Groups 1–3 have passed independent review: runtime, inputs, scoring, snapshots,
scoring, snapshots, shared deadline, exact ranker scope and change-context evidence. shared deadline, exact ranker scope, change-context evidence and vendor-free language
Task 3.4 is blocked: real Joern graphs omit first-party JavaScript tests even though partitions. The approved contributor-scan prerequisite 2.18 (`51fabef`) repairs both
filtered inputs contain them. Review `contributor-scan/frontend-retention-prerequisite.md`, JavaScript test filters; real graphs retain first-party tests and exclude vendor code.
repair that foundation contract, then rerun the partition census and independent review. Task 3.4 also repairs unreadable-input handling and preserves named PHP census evidence.
Group 4 is next: novelty comparison, actionability admission and compact advisory output.
See `pre-push-safety-net/implementation-group-3.md` for evidence and limits. See `pre-push-safety-net/implementation-group-3.md` for evidence and limits.
The first complete experiment is 5.2, before persistent caching and hook The first complete experiment is 5.2, before persistent caching and hook
installation work. Keep PMPro as the primary core-optimization example, then freeze the core and installation work. Keep PMPro as the primary core-optimization example, then freeze the core and
+7 -5
View File
@@ -79,8 +79,10 @@ updates as a shortcut to an unmeasured latency promise.
The product direction above is user-requested. The amended requirements, design and initial The product direction above is user-requested. The amended requirements, design and initial
evaluation profile in `pre-push-safety-net` were approved for task generation on 2026-09-12 evaluation profile in `pre-push-safety-net` were approved for task generation on 2026-09-12
("ok, next kiro phase"). The maintainer then approved the task plan and starting group 1 ("ok, next kiro phase"). The maintainer then approved the task plan and starting group 1
("approve.") and then group 2 ("ok, next kiro phase, approved"). Groups 1 and 2 are now verified ("approve.") and then group 2 ("ok, next kiro phase, approved"). Groups 1–3 are now verified
complete, as are group 3 tasks 3.1–3.3 (10/27 total). Task 3.4 is blocked on a complete (11/27 total). The maintainer approved the frontend-retention prerequisite;
confirmed frontend test-retention gap; review the proposed contributor-scan contributor-scan 2.18 and pre-push task 3.4 have both passed independent review and
frontend-retention prerequisite before resuming it. Implementation is authorized under that plan; task completion still requires fresh verification. Group 4 is next: novelty comparison, actionability admission and
independent review and fresh evidence. Hook performance and capability admission remain unproven. compact advisory output. Implementation is authorized under the approved plan;
task completion still requires independent review and fresh evidence. Hook performance
and capability admission remain unproven.
@@ -1,10 +1,10 @@
{ {
"counts": { "counts": {
"load_bearing": 98, "load_bearing": 99,
"standalone_capability": 9 "standalone_capability": 9
}, },
"modules": 107, "modules": 108,
"note": "Refreshed 2026-09-12 after pre-push-safety-net group 2: immutable comparison, materialization and lexical change context are spec-owned; ranker/runner integration remains pending.", "note": "Refreshed after pre-push-safety-net task 3.4: first-party frontend partitions share the authoritative ranker and execution budget.",
"orphaned": [], "orphaned": [],
"rows": [ "rows": [
{ {
@@ -129,6 +129,7 @@
"model.config_value", "model.config_value",
"model.dominance", "model.dominance",
"model.judge", "model.judge",
"model.partitions",
"model.pipeline", "model.pipeline",
"model.scan", "model.scan",
"model.taint" "model.taint"
@@ -149,6 +150,7 @@
"model.config_value", "model.config_value",
"model.dominance", "model.dominance",
"model.judge", "model.judge",
"model.partitions",
"model.pipeline", "model.pipeline",
"model.scan", "model.scan",
"model.taint" "model.taint"
@@ -338,12 +340,14 @@
"classification": "load_bearing", "classification": "load_bearing",
"importers": [ "importers": [
"cli", "cli",
"cpg.backend",
"model.candidates", "model.candidates",
"model.config_value", "model.config_value",
"model.dominance", "model.dominance",
"model.execution", "model.execution",
"model.judge", "model.judge",
"model.layout", "model.layout",
"model.partitions",
"model.pipeline", "model.pipeline",
"model.regions", "model.regions",
"model.report", "model.report",
@@ -390,6 +394,10 @@
{ {
"classification": "load_bearing", "classification": "load_bearing",
"importers": [ "importers": [
"cpg.backend",
"model.partitions",
"model.regions",
"model.scan",
"push.contracts", "push.contracts",
"push.snapshot" "push.snapshot"
], ],
@@ -450,12 +458,21 @@
{ {
"classification": "load_bearing", "classification": "load_bearing",
"importers": [ "importers": [
"model.partitions",
"model.scan", "model.scan",
"preprocess" "preprocess"
], ],
"module": "model.layout", "module": "model.layout",
"reason": "reachable from gate, map_gate, pair_gate or the CLI" "reason": "reachable from gate, map_gate, pair_gate or the CLI"
}, },
{
"classification": "load_bearing",
"importers": [
"model.scan"
],
"module": "model.partitions",
"reason": "reachable from gate, map_gate, pair_gate or the CLI"
},
{ {
"classification": "load_bearing", "classification": "load_bearing",
"importers": [ "importers": [
@@ -596,6 +613,7 @@
"map_gate", "map_gate",
"mapping", "mapping",
"model.candidates", "model.candidates",
"model.partitions",
"model.scan", "model.scan",
"pairs", "pairs",
"rank", "rank",
File diff suppressed because it is too large Load Diff
+16 -5
View File
@@ -181,8 +181,19 @@ unsupported Go partition is reported explicitly. One ranker decision and one reg
limit cover both supported frontends. The shared 900-second budget is a lab allowance; limit cover both supported frontends. The shared 900-second budget is a lab allowance;
this check establishes neither production hook latency nor framework admission. this check establishes neither production hook latency nor framework admission.
Current status (2026-09-13): this is a failing regression on Joern 4.0.625. The Stock Joern 4.0.625 fails the retention check: the projected test file is readable
projected JavaScript test file is readable but absent from the resulting graph. but absent from the graph. The approved source adaptation fixes both test filters;
See `benchmarks/measurements/2026-09-13-partition-retention-blocker.json` and the see `frontend-retention/README.md` for build pins and inclusion policy, and
proposed contributor-scan frontend-retention prerequisite. An exit 1 is expected `benchmarks/measurements/2026-09-13-frontend-retention-smoke.json` for passing
until that engine contract is repaired; do not treat it as a passing partition smoke. foundation verification. The earlier failing evidence remains in
`benchmarks/measurements/2026-09-13-partition-retention-blocker.json`.
Setting `OUSAST_SMOKE_INCLUDE_TESTS=0` in the container explicitly disables the
backend inclusion policy; that negative control must fail the retention assertion.
The expanded task 3.4 smoke also verifies first-party browser-path retention and
unshipped test-path records. Runtime origin remains unspecified without declarations.
`ops/smoke_partition_census.py`, using the same container invocation, builds a small
readable Python graph and exercises the production taint, dominance and configuration
batch queries. Each must return the actual source filename in its census. This
checks query compilation and census transport, not security-property coverage.
+4
View File
@@ -95,6 +95,10 @@ def main() -> None:
enriched = result.change_context enriched = result.change_context
assert enriched is not None and enriched.relationships, "changed context did not reach any question" assert enriched is not None and enriched.relationships, "changed context did not reach any question"
assert result.scope is not None and result.scope.selected assert result.scope is not None and result.scope.selected
invalid_census = {"cpg_empty", "files_unparsed", "partition_file_census_unavailable", "partition_file_census_incomplete"}
assert not invalid_census.intersection(result.scope.unresolved_boundaries), result.scope
assert not any(d.get("reason") in invalid_census for d in result.degradations), result.degradations
assert result.question_outcomes and all(outcome.status == "completed" for outcome in result.question_outcomes), result
selected = {question.identity for question in result.scope.selected} selected = {question.identity for question in result.scope.selected}
assert any(relation.target in selected for relation in enriched.relationships) assert any(relation.target in selected for relation in enriched.relationships)
assert result.findings, "controlled unchanged sink has no witness" assert result.findings, "controlled unchanged sink has no witness"
+54
View File
@@ -0,0 +1,54 @@
"""Compile all production batch censuses on a readable real Python graph."""
from __future__ import annotations
import hashlib
import json
import tempfile
import time
from pathlib import Path
from openultrasast.cpg.backend import JoernBackend
from openultrasast.model.contracts import ExecutionBudget
def main() -> None:
budget = ExecutionBudget(time.monotonic() + 600, 2.0)
results = {}
with tempfile.TemporaryDirectory(prefix="ousast-census-smoke-") as temporary:
root = Path(temporary)
source = root / "probe.py"
source.write_text("def handler(value):\n return value\n")
data = source.read_bytes()
assert data, "source input unreadable"
backend = JoernBackend(heap_mb=1024, execution_budget=budget)
graph = backend.build(root, language="python")
assert graph is not None and graph.run_batch is not None, "real Python graph unavailable"
try:
for kind in ("taint", "dominance", "config"):
answer = graph.run_batch(kind, {"probe": {"file": "probe.py", "function": "handler"}})
assert isinstance(answer, dict) and "probe" in answer, f"{kind} batch unanswered"
census = answer.get("__census__")
assert isinstance(census, list) and census, f"{kind} census missing"
names = census[0].get("file_names")
assert isinstance(names, list) and any(str(p).endswith("probe.py") for p in names), census
assert int(census[0]["methods"]) > 0, census
results[kind] = census
finally:
graph.cleanup()
print(
json.dumps(
{
"purpose": "Real Python batch schema and query compilation; no security property admission claim",
"source_bytes": len(data),
"source_sha256": hashlib.sha256(data).hexdigest(),
"shared_budget_seconds": 600,
"censuses": results,
},
indent=2,
)
)
if __name__ == "__main__":
main()
+11
View File
@@ -39,6 +39,7 @@ def main() -> None:
sources = { sources = {
"src/api.php": "<?php\nfunction php_handler($req) { eval($req); }\n", "src/api.php": "<?php\nfunction php_handler($req) { eval($req); }\n",
"server/api.js": "function js_handler(req) { eval(req.query.code); }\n", "server/api.js": "function js_handler(req) { eval(req.query.code); }\n",
"browser/client.js": "function browser_marker() { return 42; }\n",
"tests/probe.php": "<?php\nfunction php_test_marker() { return 42; }\n", "tests/probe.php": "<?php\nfunction php_test_marker() { return 42; }\n",
"tests/probe.js": "function js_test_marker() { return 42; }\n", "tests/probe.js": "function js_test_marker() { return 42; }\n",
"vendor/library.php": "<?php\nfunction vendor_php_marker($req) { eval($req); }\n", "vendor/library.php": "<?php\nfunction vendor_php_marker($req) { eval($req); }\n",
@@ -91,6 +92,8 @@ def main() -> None:
assert any(str(filename).endswith(path) for filename in files), witness assert any(str(filename).endswith(path) for filename in files), witness
assert method in methods, witness assert method in methods, witness
assert "php_handler" not in methods, witness assert "php_handler" not in methods, witness
assert any(str(path).endswith("browser/client.js") for path in files), witness
assert "browser_marker" in methods, witness
else: else:
raise AssertionError(f"unexpected frontend: {language}") raise AssertionError(f"unexpected frontend: {language}")
records.append({"frontend": language, "input_root": str(root), "graph_witness": witness}) records.append({"frontend": language, "input_root": str(root), "graph_witness": witness})
@@ -116,12 +119,20 @@ def main() -> None:
) )
assert {record["frontend"] for record in records} == {"php", "javascript"}, records assert {record["frontend"] for record in records} == {"php", "javascript"}, records
assert result.scope is not None assert result.scope is not None
invalid_census = {"cpg_empty", "files_unparsed", "partition_file_census_unavailable", "partition_file_census_incomplete"}
assert not invalid_census.intersection(result.scope.unresolved_boundaries), result.scope
assert not any(d.get("reason") in invalid_census for d in result.degradations), result.degradations
assert [q.identity.function for q in result.scope.selected] == ["php_handler"], result.scope assert [q.identity.function for q in result.scope.selected] == ["php_handler"], result.scope
assert {q.identity.function for q in result.scope.deferred} == {"js_handler", "php_test_marker", "js_test_marker"}, result.scope assert {q.identity.function for q in result.scope.deferred} == {"js_handler", "php_test_marker", "js_test_marker"}, result.scope
assert len(result.question_outcomes) == 1, result.question_outcomes assert len(result.question_outcomes) == 1, result.question_outcomes
assert result.question_outcomes[0].identity == result.scope.selected[0].identity assert result.question_outcomes[0].identity == result.scope.selected[0].identity
assert all("vendor" not in q.identity.path and "node_modules" not in q.identity.path for q in result.scope.selected), result.scope assert all("vendor" not in q.identity.path and "node_modules" not in q.identity.path for q in result.scope.selected), result.scope
assert any(p.language == "go" and p.status == "unsupported" for p in result.partitions), result.partitions assert any(p.language == "go" and p.status == "unsupported" for p in result.partitions), result.partitions
javascript = next(p for p in result.partitions if p.language == "javascript")
assert set(javascript.unshipped_paths) == {"tests/probe.js", *suffix_tests}, javascript
assert javascript.runtime == "unspecified", "runtime role needs declared evidence"
php = next(p for p in result.partitions if p.language == "php")
assert php.unshipped_paths == ("tests/probe.php",), php
assert all(not Path(record["input_root"]).exists() for record in records), "partition scratch leaked" assert all(not Path(record["input_root"]).exists() for record in records), "partition scratch leaked"
report = { report = {
"purpose": "Real mixed PHP/JavaScript graph census and one global ranker budget; no admission or latency claim", "purpose": "Real mixed PHP/JavaScript graph census and one global ranker budget; no admission or latency claim",
+2 -1
View File
@@ -4,7 +4,7 @@ import argparse
import json import json
import tempfile import tempfile
from collections.abc import Mapping, Sequence from collections.abc import Mapping, Sequence
from dataclasses import dataclass, replace from dataclasses import asdict, dataclass, replace
from pathlib import Path from pathlib import Path
from time import perf_counter from time import perf_counter
from typing import TYPE_CHECKING, Any from typing import TYPE_CHECKING, Any
@@ -903,6 +903,7 @@ def _model_payload(
"scope": scope.to_payload() if scope is not None else None, "scope": scope.to_payload() if scope is not None else None,
"question_outcomes": [q.to_payload() for q in outcomes], "question_outcomes": [q.to_payload() for q in outcomes],
"family_coverage": [c.to_payload() for c in getattr(result, "family_coverage", ())], "family_coverage": [c.to_payload() for c in getattr(result, "family_coverage", ())],
"partitions": [asdict(p) for p in getattr(result, "partitions", ())],
"by_rung": dict(getattr(result, "by_rung", {}) or {}), "by_rung": dict(getattr(result, "by_rung", {}) or {}),
"regions_scanned": getattr(result, "regions_scanned", 0), "regions_scanned": getattr(result, "regions_scanned", 0),
"regions_unjudged": getattr(result, "regions_unjudged", 0), "regions_unjudged": getattr(result, "regions_unjudged", 0),
+14 -1
View File
@@ -697,9 +697,12 @@ class JoernBackend:
methods = 0 methods = 0
files = 0 files = 0
answered = 0 answered = 0
file_names: set[str] = set()
named_census = True
for shard in shards: for shard in shards:
one = self.query_batch(shard, query, requests) one = self.query_batch(shard, query, requests)
if one is None: if one is None:
named_census = False
if self.execution_budget is not None: if self.execution_budget is not None:
self._note("shard_query_incomplete") self._note("shard_query_incomplete")
return None return None
@@ -709,11 +712,21 @@ class JoernBackend:
if census and isinstance(census[0], Mapping): if census and isinstance(census[0], Mapping):
methods += int(str(census[0].get("methods", "0")) or 0) methods += int(str(census[0].get("methods", "0")) or 0)
files += int(str(census[0].get("files", "0")) or 0) files += int(str(census[0].get("files", "0")) or 0)
names = census[0].get("file_names")
if isinstance(names, list) and all(isinstance(name, str) for name in names):
file_names.update(names)
else:
named_census = False
else:
named_census = False
for rid, rows in one.items(): for rid, rows in one.items():
merged.setdefault(rid, []).extend(rows) merged.setdefault(rid, []).extend(rows)
if not answered: if not answered:
return None return None
merged["__census__"] = [{"methods": str(methods), "files": str(files), "shards": str(len(shards))}] total: dict[str, object] = {"methods": str(methods), "files": str(files), "shards": str(len(shards))}
if named_census:
total["file_names"] = sorted(file_names)
merged["__census__"] = [total]
return merged return merged
def _apply_overlays(self, cpg_path: Path, scratch: Path) -> bool: def _apply_overlays(self, cpg_path: Path, scratch: Path) -> bool:
+1 -1
View File
@@ -78,7 +78,7 @@
// per-file warnings -- so "no rows" and "no graph" are indistinguishable to the driver without this. // per-file warnings -- so "no rows" and "no graph" are indistinguishable to the driver without this.
// It rides the batch rather than costing its own invocation, because JVM startup is what this whole // It rides the batch rather than costing its own invocation, because JVM startup is what this whole
// batching design exists to avoid. // batching design exists to avoid.
val census = ujson.Arr(ujson.Obj("methods" -> cpg.method.size.toString, "files" -> cpg.file.size.toString)) val census = ujson.Arr(ujson.Obj("methods" -> cpg.method.size.toString, "files" -> cpg.file.size.toString, "file_names" -> ujson.Arr.from(cpg.file.name.l)))
println(ujson.write(ujson.Obj.from(answers.toSeq :+ ("__census__" -> census)))) println(ujson.write(ujson.Obj.from(answers.toSeq :+ ("__census__" -> census))))
} else { } else {
println(ujson.write(ujson.Arr(rowsFor(settings, function, file): _*))) println(ujson.write(ujson.Arr(rowsFor(settings, function, file): _*)))
+1 -1
View File
@@ -190,7 +190,7 @@
// per-file warnings -- so "no rows" and "no graph" are indistinguishable to the driver without this. // per-file warnings -- so "no rows" and "no graph" are indistinguishable to the driver without this.
// It rides the batch rather than costing its own invocation, because JVM startup is what this whole // It rides the batch rather than costing its own invocation, because JVM startup is what this whole
// batching design exists to avoid. // batching design exists to avoid.
val census = ujson.Arr(ujson.Obj("methods" -> cpg.method.size.toString, "files" -> cpg.file.size.toString)) val census = ujson.Arr(ujson.Obj("methods" -> cpg.method.size.toString, "files" -> cpg.file.size.toString, "file_names" -> ujson.Arr.from(cpg.file.name.l)))
println(ujson.write(ujson.Obj.from(answers.toSeq :+ ("__census__" -> census)))) println(ujson.write(ujson.Obj.from(answers.toSeq :+ ("__census__" -> census))))
} else { } else {
println(ujson.write(ujson.Arr(rowsFor(operations, dischargers, function, file, operationRequires, dischargersByKind): _*))) println(ujson.write(ujson.Arr(rowsFor(operations, dischargers, function, file, operationRequires, dischargersByKind): _*)))
+1 -1
View File
@@ -852,7 +852,7 @@
// per-file warnings -- so "no rows" and "no graph" are indistinguishable to the driver without this. // per-file warnings -- so "no rows" and "no graph" are indistinguishable to the driver without this.
// It rides the batch rather than costing its own invocation, because JVM startup is what this whole // It rides the batch rather than costing its own invocation, because JVM startup is what this whole
// batching design exists to avoid. // batching design exists to avoid.
val census = ujson.Arr(ujson.Obj("methods" -> cpg.method.size.toString, "files" -> cpg.file.size.toString)) val census = ujson.Arr(ujson.Obj("methods" -> cpg.method.size.toString, "files" -> cpg.file.size.toString, "file_names" -> ujson.Arr.from(cpg.file.name.l)))
println(ujson.write(ujson.Obj.from(answers.toSeq :+ ("__census__" -> census)))) println(ujson.write(ujson.Obj.from(answers.toSeq :+ ("__census__" -> census))))
} else { } else {
println( println(
+6 -1
View File
@@ -26,7 +26,12 @@ def _matches(relative: str, patterns: Iterable[str]) -> bool:
parts = relative.split("/") parts = relative.split("/")
for pattern in patterns: for pattern in patterns:
if pattern.endswith("/"): if pattern.endswith("/"):
if pattern.rstrip("/") in parts[:-1] or pattern.rstrip("/") in parts: directory = pattern.rstrip("/")
if any(
fnmatch.fnmatch("/".join(parts[start:end]), directory)
for start in range(len(parts))
for end in range(start + 1, len(parts) + 1)
):
return True return True
elif fnmatch.fnmatch(relative, pattern) or fnmatch.fnmatch(parts[-1], pattern): elif fnmatch.fnmatch(relative, pattern) or fnmatch.fnmatch(parts[-1], pattern):
return True return True
+279
View File
@@ -0,0 +1,279 @@
"""Physical first-party frontend inputs and routing; selection remains in the scan ranker."""
from __future__ import annotations
import os
import stat
import tempfile
import time
from collections.abc import Callable, Mapping
from dataclasses import dataclass
from pathlib import Path
from typing import Any
from ..cpg.backend import _DeadlineExpired, _remove_owned_tree
from ..preprocess import LANGUAGE_BY_EXTENSION
from .contracts import ExecutionBudget
from .layout import is_test_path, is_vendored, layout_facts
FRONTENDS = {
"php": "php",
"javascript": "javascript",
"typescript": "typescript",
"python": "python",
"java": "java",
"c": "c",
"cpp": "c_cpp",
}
@dataclass(frozen=True)
class PartitionCoverage:
language: str
frontend: str | None
paths: tuple[str, ...]
source_bytes: int
status: str
boundaries: tuple[str, ...] = ()
runtime: str = "unspecified"
excluded_paths: tuple[str, ...] = ()
unshipped_paths: tuple[str, ...] = ()
class PartitionGraph:
"""Routes existing requests without reordering or allocating another scan budget."""
def __init__(self, budget: ExecutionBudget | None = None) -> None:
self.budget = budget
self.diagnostics: list[str] = []
self.graphs: dict[str, Any] = {}
self.roots: dict[str, Path] = {}
self.path_languages: dict[str, str] = {}
self.partitions: tuple[PartitionCoverage, ...] = ()
self.boundaries: tuple[str, ...] = ()
self.unparsed: tuple[str, ...] = ()
self.cpg_path = Path("partitioned-cpg")
def cleanup(self) -> None:
try:
for graph in self.graphs.values():
cleanup = getattr(graph, "cleanup", None)
if callable(cleanup):
cleanup()
finally:
for root in self.roots.values():
deadline = self.budget.deadline_monotonic + self.budget.cancellation_allowance_seconds if self.budget else float("inf")
try:
_remove_owned_tree(root, deadline)
except (OSError, _DeadlineExpired):
self.diagnostics.append("scratch_cleanup_incomplete")
def execution_diagnostics(self) -> tuple[str, ...]:
reasons = list(self.diagnostics)
for graph in self.graphs.values():
read = getattr(graph, "execution_diagnostics", None)
if callable(read):
reasons.extend(read())
return tuple(reasons)
def _normalize(self, value: Any) -> Any:
if isinstance(value, str):
for root in self.roots.values():
value = value.replace(str(root) + "/", "")
return value
if isinstance(value, list):
return [self._normalize(v) for v in value]
if isinstance(value, dict):
return {k: self._normalize(v) for k, v in value.items()}
return value
def run(self, kind: str, params: Mapping[str, object]) -> list[object] | None:
return self.run_batch(kind, {"one": params}).get("one")
def run_batch(self, kind: str, requests: Mapping[str, Mapping[str, object]]) -> dict[str, list[object]]:
answers: dict[str, list[object]] = {}
languages = dict.fromkeys(self.path_languages.get(str(params.get("file", "")), "") for params in requests.values())
for language in languages:
graph = self.graphs.get(language)
if graph is None:
continue
if self.budget is not None and time.monotonic() >= self.budget.deadline_monotonic:
break
selected = {rid: params for rid, params in requests.items() if self.path_languages.get(str(params.get("file", ""))) == language}
if not selected:
continue
batch = getattr(graph, "run_batch", None)
raw = batch(kind, selected) if callable(batch) else {rid: graph.run(kind, params) for rid, params in selected.items()}
if not isinstance(raw, Mapping):
continue
for rid, rows in raw.items():
if rid in selected and isinstance(rows, list):
answers[rid] = self._normalize(rows)
elif rid == "__census__" and isinstance(rows, list):
expected = set(next(p.paths for p in self.partitions if p.language == language))
normalized = self._normalize(rows)
names = [r.get("file_names") for r in normalized if isinstance(r, Mapping)]
named = bool(names) and all(isinstance(n, list) and all(isinstance(p, str) for p in n) for n in names)
observed = {p.removeprefix("./") for n in names if isinstance(n, list) for p in n if isinstance(p, str)}
if not named or not expected <= observed:
self.diagnostics.append("partition_file_census_incomplete" if named else "partition_file_census_unavailable")
rows = [
{
"methods": 0,
"files": 0,
"shards": max((int(str(r.get("shards", 1))) for r in rows if isinstance(r, Mapping)), default=1),
}
]
# Preserve any individual empty or sharded graph's degradation.
previous = answers.get(rid, [])
if not previous or any(
isinstance(r, Mapping) and (int(str(r.get("methods", 0))) == 0 or int(str(r.get("shards", 1))) > 1) for r in rows
):
answers[rid] = rows
return answers
def build_partitions(
root: Path, build: Callable[[Path, str], Any], budget: ExecutionBudget | None, declared: Mapping[str, tuple[str, ...]] | None = None
) -> PartitionGraph:
graph = PartitionGraph(budget)
paths: dict[str, list[Path]] = {}
boundaries: set[str] = set()
excluded: set[str] = set()
facts = layout_facts()
root_fd = os.open(root, os.O_RDONLY | os.O_DIRECTORY | os.O_NOFOLLOW)
try:
for directory, dirs, files, directory_fd in os.fwalk(
".", follow_symlinks=False, dir_fd=root_fd, onerror=lambda error: boundaries.add("source_unreadable")
):
if budget is not None and time.monotonic() >= budget.deadline_monotonic:
boundaries.add("deadline_exhausted")
break
base = root / directory
for name in list(dirs):
if budget is not None and time.monotonic() >= budget.deadline_monotonic:
boundaries.add("deadline_exhausted")
dirs.clear()
break
path = base / name
relative = path.relative_to(root).as_posix()
if name in {".git", ".hg", ".svn", ".openultrasast", ".venv", "venv", "__pycache__"}:
excluded.add(relative + "/")
dirs.remove(name)
continue
if path.is_symlink() or is_vendored(relative + "/", facts):
excluded.add(relative + "/")
boundaries.add("symlink_context_unresolved" if path.is_symlink() else "vendor_semantics_unresolved")
dirs.remove(name)
for name in files:
if budget is not None and time.monotonic() >= budget.deadline_monotonic:
boundaries.add("deadline_exhausted")
break
path = base / name
relative = path.relative_to(root).as_posix()
if is_vendored(relative, facts):
excluded.add(relative)
boundaries.add("vendor_semantics_unresolved")
continue
if path.is_symlink() or not path.is_file():
excluded.add(relative)
boundaries.add("symlink_context_unresolved")
continue
language = LANGUAGE_BY_EXTENSION.get(path.suffix.lower(), "unknown")
if language == "unknown":
try:
fd = os.open(name, os.O_RDONLY | os.O_NOFOLLOW | os.O_NONBLOCK, dir_fd=directory_fd)
with os.fdopen(fd, "rb") as handle:
first = handle.readline(4096).lower()
except OSError:
boundaries.add("source_unreadable")
continue
if first.startswith(b"#!"):
language = "python" if b"python" in first else "javascript" if b"node" in first else "unknown"
languages = (language,) if language != "unknown" else (declared or {}).get(relative, ())
for item in languages:
paths.setdefault(item, []).append(path)
if len(languages) == 1:
graph.path_languages[relative] = languages[0]
elif languages:
boundaries.add("ambiguous_frontend_path")
if len(paths) > 1:
boundaries.add("cross_partition_semantics_unresolved")
records: list[PartitionCoverage] = []
for language, sources in sorted(paths.items()):
relative_paths = tuple(sorted(p.relative_to(root).as_posix() for p in sources))
frontend = FRONTENDS.get(language)
if frontend is None:
records.append(PartitionCoverage(language, None, relative_paths, 0, "unsupported", ("frontend_unsupported",)))
boundaries.add("frontend_unsupported")
continue
scratch = Path(tempfile.mkdtemp(prefix="ousast-input-"))
graph.roots[language] = scratch
size = 0
complete = True
for source in sources:
if budget is not None and time.monotonic() >= budget.deadline_monotonic:
complete = False
break
target = scratch / source.relative_to(root)
target.parent.mkdir(parents=True, exist_ok=True)
# A no-follow open prevents file symlink substitution between census and read.
try:
fd = _open_source(root_fd, source.relative_to(root).parts)
except OSError:
boundaries.add("source_unreadable")
complete = False
break
with os.fdopen(fd, "rb") as handle, target.open("wb") as output:
while chunk := handle.read(1024 * 1024):
if budget is not None and time.monotonic() >= budget.deadline_monotonic:
complete = False
break
output.write(chunk)
size += len(chunk)
if not complete:
break
built = build(scratch, frontend) if complete else None
if built is not None:
graph.graphs[language] = built
graph.unparsed += tuple(graph._normalize(list(getattr(built, "unparsed", ()))))
record_limits: tuple[str, ...] = ("runtime_unspecified",) if language in {"javascript", "typescript"} else ()
if language == "typescript":
record_limits += ("typescript_property_support_unvalidated",)
boundaries.add("typescript_property_support_unvalidated")
if language in {"c", "cpp"}:
record_limits += ("c_bounds_arithmetic_unmodeled",)
records.append(
PartitionCoverage(language, frontend, relative_paths, size, "built" if built is not None else "build_failed", record_limits)
)
from dataclasses import replace
graph.partitions = tuple(
replace(
record, excluded_paths=tuple(sorted(excluded)), unshipped_paths=tuple(p for p in record.paths if is_test_path(p, facts))
)
for record in records
)
graph.boundaries = tuple(sorted(boundaries))
return graph
except BaseException:
graph.cleanup()
raise
finally:
os.close(root_fd)
def _open_source(root_fd: int, parts: tuple[str, ...]) -> int:
descriptor = os.dup(root_fd)
try:
for component in parts[:-1]:
child = os.open(component, os.O_RDONLY | os.O_DIRECTORY | os.O_NOFOLLOW, dir_fd=descriptor)
os.close(descriptor)
descriptor = child
result = os.open(parts[-1], os.O_RDONLY | os.O_NOFOLLOW | os.O_NONBLOCK, dir_fd=descriptor)
if not stat.S_ISREG(os.fstat(result).st_mode):
os.close(result)
raise OSError("source is not a regular file")
return result
finally:
os.close(descriptor)
+59 -7
View File
@@ -43,7 +43,8 @@ from .contracts import (
from .dominance import request_params as dominance_params from .dominance import request_params as dominance_params
from .evidence import TIER_EXCLUDE, Evidence, evidence_from_rows from .evidence import TIER_EXCLUDE, Evidence, evidence_from_rows
from .ladder import Rung from .ladder import Rung
from .layout import with_layout from .layout import is_vendored, layout_facts, with_layout
from .partitions import PartitionCoverage, build_partitions
from .pipeline import ModelFinding, scan_region from .pipeline import ModelFinding, scan_region
from .regions import MODULE_SCOPE, ScanRegion, affected_context from .regions import MODULE_SCOPE, ScanRegion, affected_context
from .specs import ConfigSpec, DominanceSpec, TaintSpec, config_specs, dominance_specs, taint_specs from .specs import ConfigSpec, DominanceSpec, TaintSpec, config_specs, dominance_specs, taint_specs
@@ -126,6 +127,7 @@ class ModelScanResult:
# could have been found was skipped. Reported so the request count is legible. # could have been found was skipped. Reported so the request count is legible.
requests_pruned: int = 0 requests_pruned: int = 0
partitions: tuple[PartitionCoverage, ...] = ()
change_context: ChangeContext | None = None change_context: ChangeContext | None = None
scope: ScopeDecision | None = None scope: ScopeDecision | None = None
question_outcomes: tuple[QuestionOutcome, ...] = () question_outcomes: tuple[QuestionOutcome, ...] = ()
@@ -158,6 +160,8 @@ def scan_repository(
if ranking_mode is not None: if ranking_mode is not None:
limits = replace(limits, order_by_evidence=ranking_mode == "evidence") limits = replace(limits, order_by_evidence=ranking_mode == "evidence")
mode = "evidence" if limits.order_by_evidence else "static" mode = "evidence" if limits.order_by_evidence else "static"
layout = layout_facts()
regions = tuple(r for r in regions if not is_vendored(r.path, layout))
# Reject duplicate canonical questions before any engine work or resource ownership. # Reject duplicate canonical questions before any engine work or resource ownership.
identities = [_identity(unit, r, f) for r in regions for f in r.families] identities = [_identity(unit, r, f) for r in regions for f in r.families]
if len(set(identities)) != len(identities): if len(set(identities)) != len(identities):
@@ -223,13 +227,29 @@ def _scan_repository_impl(
for region in regions: for region in regions:
counts[region.language] = counts.get(region.language, 0) + 1 counts[region.language] = counts.get(region.language, 0) + 1
dominant = max(counts, key=lambda name: (counts[name], name)) if counts else "" dominant = max(counts, key=lambda name: (counts[name], name)) if counts else ""
declarations: dict[str, set[str]] = {}
for region in regions:
declarations.setdefault(region.path, set()).add(region.language)
if root.is_dir():
cpg = build_partitions(
root,
lambda source, language: _build(backend, source, language, execution_budget=execution_budget),
execution_budget,
{path: tuple(sorted(languages)) for path, languages in declarations.items()},
)
degradations.extend({"stage": "model", "reason": reason} for reason in cpg.boundaries)
else:
cpg = _build(backend, root, dominant, execution_budget=execution_budget) cpg = _build(backend, root, dominant, execution_budget=execution_budget)
build_seconds = round(time.monotonic() - build_started, 2) build_seconds = round(time.monotonic() - build_started, 2)
if cpg is None: if cpg is None or (hasattr(cpg, "graphs") and not cpg.graphs):
if cpg is not None:
owned.append(cpg)
scope, _ = _scope_work( scope, _ = _scope_work(
regions, {}, limits, unit, ranking_mode, population_complete, failure="cpg_build_failed", change_context=change_context regions, {}, limits, unit, ranking_mode, population_complete, failure="cpg_build_failed", change_context=change_context
) )
scope = replace(scope, unresolved_boundaries=tuple(dict.fromkeys((*scope.unresolved_boundaries, *getattr(cpg, "boundaries", ())))))
return ModelScanResult( return ModelScanResult(
partitions=getattr(cpg, "partitions", ()),
change_context=change_context, change_context=change_context,
scope=scope, scope=scope,
family_coverage=_family_coverage(scope, ()), family_coverage=_family_coverage(scope, ()),
@@ -322,7 +342,22 @@ def _scan_repository_impl(
# Stable question IDs are the IDs sent to every execution request below. # Stable question IDs are the IDs sent to every execution request below.
scope_limits = ( scope_limits = (
replace(limits, prune_tier0=False) replace(limits, prune_tier0=False)
if any(d.get("reason") in {"files_unparsed", "cpg_empty", "cpg_sharded"} for d in degradations) if any(
d.get("reason")
in {
"files_unparsed",
"cpg_empty",
"cpg_sharded",
"cross_partition_semantics_unresolved",
"vendor_semantics_unresolved",
"symlink_context_unresolved",
"frontend_unsupported",
"source_unreadable",
"ambiguous_frontend_path",
"typescript_property_support_unvalidated",
}
for d in degradations
)
else limits else limits
) )
scope, work = _scope_work( scope, work = _scope_work(
@@ -457,16 +492,32 @@ def _scan_repository_impl(
if unasked: if unasked:
degradations.append({"stage": "model", "reason": "budget_exhausted", "regions_unasked": len(unasked)}) degradations.append({"stage": "model", "reason": "budget_exhausted", "regions_unasked": len(unasked)})
graph_gaps = tuple(str(d["reason"]) for d in degradations if d.get("reason") in {"files_unparsed", "cpg_empty", "cpg_sharded"}) graph_gaps = tuple(
str(d["reason"])
for d in degradations
if d.get("reason")
in {
"files_unparsed",
"cpg_empty",
"cpg_sharded",
"cross_partition_semantics_unresolved",
"vendor_semantics_unresolved",
"symlink_context_unresolved",
"frontend_unsupported",
"source_unreadable",
"ambiguous_frontend_path",
"typescript_property_support_unvalidated",
}
)
for question in scope.selected: for question in scope.selected:
rid = question.identity.question_id rid = question.identity.question_id
if rid not in outcomes: if rid not in outcomes:
answered = rid in rows_by_id has_answer = rid in rows_by_id
outcomes[rid] = QuestionOutcome( outcomes[rid] = QuestionOutcome(
question.identity, question.identity,
"not_arbitrated" if answered else "unanswered", "not_arbitrated" if has_answer else "unanswered",
"deadline_exhausted" if not _within_deadline(execution_budget) else "query_unanswered", "deadline_exhausted" if not _within_deadline(execution_budget) else "query_unanswered",
_rows_json(rows_by_id[rid]) if answered else None, _rows_json(rows_by_id[rid]) if has_answer else None,
) )
if question.identity in context_gaps and outcomes[rid].status == "completed": if question.identity in context_gaps and outcomes[rid].status == "completed":
outcomes[rid] = replace(outcomes[rid], status="unresolved", reason="change_context_incomplete") outcomes[rid] = replace(outcomes[rid], status="unresolved", reason="change_context_incomplete")
@@ -480,6 +531,7 @@ def _scan_repository_impl(
# The graph has answered everything it is going to. Its scratch tree holds the CPG, the request files and # The graph has answered everything it is going to. Its scratch tree holds the CPG, the request files and
# joern's own working copy -- tens of megabytes per scan -- and nothing else reclaims it. # joern's own working copy -- tens of megabytes per scan -- and nothing else reclaims it.
return ModelScanResult( return ModelScanResult(
partitions=getattr(cpg, "partitions", ()),
findings=findings, findings=findings,
by_rung=_tally(findings), by_rung=_tally(findings),
regions_scanned=scanned, regions_scanned=scanned,
+7 -1
View File
@@ -97,7 +97,13 @@ def enumerate_source_files(root: Path) -> list[Path]:
patterns = _load_ignore_patterns(root) patterns = _load_ignore_patterns(root)
paths: list[Path] = [] paths: list[Path] = []
for path in root.rglob("*"): for path in root.rglob("*"):
if not path.is_file() or _has_ignored_dir(root, path) or _is_ignored(root, path, patterns) or _is_vendored(root, path): if (
path.is_symlink()
or not path.is_file()
or _has_ignored_dir(root, path)
or _is_ignored(root, path, patterns)
or _is_vendored(root, path)
):
continue continue
if detect_language(path) != "unknown": if detect_language(path) != "unknown":
paths.append(path) paths.append(path)
@@ -92,3 +92,18 @@ calls = ["path.normalize", "path.resolve"]
# CWE-79. The standard encoders: the DOM/global URI encoders and the common escaping helpers. # CWE-79. The standard encoders: the DOM/global URI encoders and the common escaping helpers.
id = "html.escape" id = "html.escape"
calls = ["encodeURIComponent", "encodeURI", "escapeHtml", "escape", "sanitizeHtml", "DOMPurify.sanitize"] calls = ["encodeURIComponent", "encodeURI", "escapeHtml", "escape", "sanitizeHtml", "DOMPurify.sanitize"]
[[layout]]
# First-party test conventions are retained as graph context, ordered as unshipped.
# These declarations cover JS module/JSX and TS/TSX filenames; they do not assert
# TypeScript property support or a browser/server runtime classification.
id = "javascript_first_party_tests"
vendored = ["node_modules/"]
tests = [
"tests/", "test/", "spec/", "__tests__/", "__mocks__/", "e2e/",
"*.test.js", "*.test.mjs", "*.test.cjs", "*.test.jsx", "*.test.ts", "*.test.mts", "*.test.cts", "*.test.tsx",
"*.spec.js", "*.spec.mjs", "*.spec.cjs", "*.spec.jsx", "*.spec.ts", "*.spec.mts", "*.spec.cts", "*.spec.tsx",
"*.mock.js", "*.mock.mjs", "*.mock.cjs", "*.mock.jsx", "*.mock.ts", "*.mock.mts", "*.mock.cts", "*.mock.tsx",
"*.e2e.js", "*.e2e.mjs", "*.e2e.cjs", "*.e2e.jsx", "*.e2e.ts", "*.e2e.mts", "*.e2e.cts", "*.e2e.tsx",
]
+30
View File
@@ -956,6 +956,36 @@ def test_a_source_file_the_size_of_a_data_table_is_excluded_and_named(tmp_path:
assert [Path(name).name for name in result.unparsed] == ["profiles.php"], "and it is reported, not silently dropped" assert [Path(name).name for name in result.unparsed] == ["profiles.php"], "and it is reported, not silently dropped"
@pytest.mark.parametrize("shard_names", [("app.php",), ("app.php", "helper.php")])
def test_shard_aggregation_preserves_complete_named_census(monkeypatch, shard_names):
from openultrasast.cpg.backend import JoernBackend
def batch(self, path, query, requests):
return {"__census__": [{"methods": "2", "files": "2", "file_names": ["<unknown>", path.name]}]}
monkeypatch.setattr(JoernBackend, "query_batch", batch)
result = JoernBackend().query_batch_across([Path(name) for name in shard_names], "taint", {})
assert result is not None
assert set(result["__census__"][0]["file_names"]) == {"<unknown>", *shard_names}
assert result["__census__"][0]["shards"] == str(len(shard_names))
@pytest.mark.parametrize("missing", ["names", "census", "answer"])
def test_shard_aggregation_does_not_invent_complete_named_census(monkeypatch, missing):
from openultrasast.cpg.backend import JoernBackend
def batch(self, path, query, requests):
if path.name == "missing":
return None if missing == "answer" else {} if missing == "census" else {"__census__": [{"methods": "2", "files": "2"}]}
return {"__census__": [{"methods": "2", "files": "2", "file_names": ["app.php", "<unknown>"]}]}
monkeypatch.setattr(JoernBackend, "query_batch", batch)
result = JoernBackend().query_batch_across([Path("app.php"), Path("missing")], "taint", {})
assert result is not None
assert "file_names" not in result["__census__"][0]
assert result["__census__"][0]["shards"] == "2"
def test_a_clean_build_whose_census_cannot_be_taken_is_a_failed_build(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: def test_a_clean_build_whose_census_cannot_be_taken_is_a_failed_build(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""`cpg query census failed: timeout` was logged as a warning during a build whose every later query """`cpg query census failed: timeout` was logged as a warning during a build whose every later query
then timed out too. A graph that cannot answer the census inside the query timeout cannot answer a taint then timed out too. A graph that cannot answer the census inside the query timeout cannot answer a taint
+3
View File
@@ -261,6 +261,7 @@ def test_cleanup_does_not_follow_substituted_directory_links(tmp_path: Path) ->
def test_driver_propagates_budget_and_rejects_legacy_without_launch(tmp_path: Path) -> None: def test_driver_propagates_budget_and_rejects_legacy_without_launch(tmp_path: Path) -> None:
(tmp_path / "input.py").write_bytes(b"print(1)\n")
from openultrasast.cpg.backend import CpgResult from openultrasast.cpg.backend import CpgResult
from openultrasast.model.scan import scan_repository from openultrasast.model.scan import scan_repository
@@ -296,6 +297,7 @@ def test_driver_propagates_budget_and_rejects_legacy_without_launch(tmp_path: Pa
def test_budgeted_driver_does_not_retry_internal_typeerror(tmp_path: Path) -> None: def test_budgeted_driver_does_not_retry_internal_typeerror(tmp_path: Path) -> None:
(tmp_path / "input.py").write_bytes(b"print(1)\n")
from openultrasast.model.scan import scan_repository from openultrasast.model.scan import scan_repository
class Backend: class Backend:
@@ -337,6 +339,7 @@ def test_cancelled_graph_session_does_not_poison_next_build(tmp_path: Path, monk
def test_driver_cleans_graph_on_unexpected_exception(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: def test_driver_cleans_graph_on_unexpected_exception(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
(tmp_path / "input.py").write_bytes(b"print(1)\n")
from openultrasast.cpg.backend import CpgResult from openultrasast.cpg.backend import CpgResult
from openultrasast.model import scan from openultrasast.model import scan
+6 -2
View File
@@ -65,11 +65,13 @@ def test_test_regions_are_not_product_and_order_last() -> None:
) )
def test_the_scan_hands_the_vendored_trees_to_the_build(tmp_path: Path) -> None: def test_the_scan_excludes_vendored_trees_before_the_build(tmp_path: Path) -> None:
from openultrasast.model.regions import ScanRegion from openultrasast.model.regions import ScanRegion
from openultrasast.model.scan import ScanBudget, scan_repository from openultrasast.model.scan import ScanBudget, scan_repository
(tmp_path / "vendor").mkdir() (tmp_path / "vendor").mkdir()
(tmp_path / "a.php").write_bytes(b"<?php echo 1;")
(tmp_path / "vendor/b.php").write_bytes(b"<?php vendor_secret();")
seen: dict[str, object] = {} seen: dict[str, object] = {}
class _Backend: class _Backend:
@@ -80,8 +82,10 @@ def test_the_scan_hands_the_vendored_trees_to_the_build(tmp_path: Path) -> None:
from openultrasast.cpg.backend import CpgResult from openultrasast.cpg.backend import CpgResult
seen["exclude"] = tuple(exclude) seen["exclude"] = tuple(exclude)
seen["files"] = tuple(p.relative_to(root).as_posix() for p in root.rglob("*") if p.is_file())
return CpgResult(cpg_path=Path("cpg.bin"), run=lambda q, p: [], run_batch=lambda q, r: {rid: [] for rid in r}) return CpgResult(cpg_path=Path("cpg.bin"), run=lambda q, p: [], run_batch=lambda q, r: {rid: [] for rid in r})
region = ScanRegion(path="a.php", function="f", language="php", families=("injection",), rank=0.5, source="x") region = ScanRegion(path="a.php", function="f", language="php", families=("injection",), rank=0.5, source="x")
scan_repository(tmp_path, [region], backend=_Backend(), budget=ScanBudget(max_model_calls=0)) scan_repository(tmp_path, [region], backend=_Backend(), budget=ScanBudget(max_model_calls=0))
assert seen["exclude"] == ("vendor",) assert seen["exclude"] == ()
assert seen["files"] == ("a.php",)
+337
View File
@@ -0,0 +1,337 @@
from pathlib import Path
from openultrasast.cpg.backend import CpgResult
from openultrasast.model.regions import ScanRegion
from openultrasast.model.scan import ScanBudget, scan_repository
def test_polyglot_graph_inputs_and_single_scope(tmp_path: Path) -> None:
sources = {
"api.php": "<?php function f($x) { echo $x; }",
"api.js": "function f(x) { return x; }",
"vendor/bad.php": "<?php echo 1;",
"node_modules/bad.js": "bad()",
"tests/test.js": "test()",
"other.rs": "fn main() {}",
}
for name, content in sources.items():
path = tmp_path / name
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(content)
seen = []
class Backend:
def build(self, root, *, language="", **kwargs):
files = tuple(sorted(p.relative_to(root).as_posix() for p in root.rglob("*") if p.is_file()))
assert all(p.read_bytes() for p in root.rglob("*") if p.is_file())
seen.append((language, files))
return CpgResult(Path("cpg.bin"), lambda q, p: [], run_batch=lambda q, requests: {rid: [] for rid in requests})
regions = [
ScanRegion(path=p, function="f", language=language, families=("injection",), rank=r, source="entry_point")
for p, language, r in [
("api.php", "php", 1.0),
("api.js", "javascript", 0.9),
("vendor/bad.php", "php", 2.0),
("tests/test.js", "javascript", 3.0),
]
]
result = scan_repository(
tmp_path, regions, backend=Backend(), budget=ScanBudget(max_regions=1, tiering=False), population_complete=True
)
assert seen == [("javascript", ("api.js", "tests/test.js")), ("php", ("api.php",))]
assert result.scope.selected[0].identity.path == "api.php"
assert all(q.identity.path != "vendor/bad.php" for q in (*result.scope.selected, *result.scope.deferred))
assert any(p.language == "rust" and p.status == "unsupported" for p in result.partitions)
assert "cross_partition_semantics_unresolved" in result.scope.unresolved_boundaries
def test_file_patterns_symlinks_and_witness_paths(tmp_path, monkeypatch):
import dataclasses
from openultrasast.model import layout, partitions
from openultrasast.preprocess import preprocess_repository
from openultrasast.semantic.facts import LayoutFact
facts = (LayoutFact(id="declared", language="javascript", vendored=("*.bundle.js", "bundled/lib/"), tests=("tests/",)),)
monkeypatch.setattr(layout, "layout_facts", lambda *args: facts)
monkeypatch.setattr(partitions, "layout_facts", lambda *args: facts)
outside = tmp_path.parent / "external.js"
outside.write_text("external_secret()")
(tmp_path / "escape.js").symlink_to(outside)
(tmp_path / "bad.bundle.js").write_text("vendor_secret()")
(tmp_path / "app.js").write_text("function f(x) { return x; }")
(tmp_path / "bundled/lib").mkdir(parents=True)
(tmp_path / "bundled/lib/hidden.js").write_text("vendor_secret()")
_, targets = preprocess_repository(tmp_path)
assert [t.path for t in targets] == ["app.js"]
roots = []
def build(root, language):
roots.append(root)
assert [p.relative_to(root).as_posix() for p in root.rglob("*") if p.is_file()] == ["app.js"]
return CpgResult(
Path("graph"),
lambda q, p: [],
run_batch=lambda q, reqs: {rid: [{"file": str(root / "app.js"), "site": str(root / "app.js") + ":1"}] for rid in reqs},
)
graph = partitions.build_partitions(tmp_path, build, None)
try:
assert graph.partitions[0].source_bytes == len((tmp_path / "app.js").read_bytes())
assert graph.run("taint", {"file": "app.js"}) == [{"file": "app.js", "site": "app.js:1"}]
assert {"vendor_semantics_unresolved", "symlink_context_unresolved"} <= set(graph.boundaries)
assert dataclasses.asdict(graph.partitions[0])["runtime"] == "unspecified"
finally:
graph.cleanup()
assert all(not root.exists() for root in roots)
def test_shared_deadline_stops_later_frontend_and_no_false_clean(tmp_path):
import time
from openultrasast.model.contracts import ExecutionBudget
from openultrasast.model.partitions import build_partitions
for name in ("a.js", "b.php"):
(tmp_path / name).write_text("readable")
seen = []
budget = ExecutionBudget(time.monotonic() + 0.1, 0.1)
def build(root, language):
seen.append(language)
time.sleep(0.12)
return CpgResult(Path("graph"), lambda q, p: [])
graph = build_partitions(tmp_path, build, budget)
try:
assert seen == ["javascript"]
assert [p.status for p in graph.partitions] == ["built", "build_failed"]
assert graph.run("taint", {"file": "b.php"}) is None
finally:
graph.cleanup()
def test_evidence_ranker_selects_once_across_languages(tmp_path):
from test_model_scope import vector
for name in ("a.php", "b.js"):
(tmp_path / name).write_text("readable source")
executed = []
class Backend:
def build(self, root, *, language="", **kwargs):
def batch(kind, requests):
if any(p.get("evidenceOnly") for p in requests.values()):
return {rid: vector(language == "javascript") for rid in requests}
executed.extend(p["file"] for p in requests.values())
return {rid: [] for rid in requests}
return CpgResult(Path("graph"), lambda q, p: [], run_batch=batch)
regions = [
ScanRegion(path=path, function="f", language=language, families=("injection",), rank=rank, source="entry_point")
for path, language, rank in [("a.php", "php", 1.0), ("b.js", "javascript", 0.5)]
]
result = scan_repository(tmp_path, regions, backend=Backend(), ranking_mode="evidence", budget=ScanBudget(max_regions=1))
assert len(result.scope.selected) == 1
assert result.scope.selected[0].identity.path == "b.js"
assert executed == ["b.js"]
assert result.question_outcomes[0].status == "unresolved"
def test_unsupported_only_population_is_named(tmp_path):
(tmp_path / "main.go").write_text("package main")
class Backend:
def build(self, *args, **kwargs):
raise AssertionError("unsupported frontend must not build")
result = scan_repository(tmp_path, [], backend=Backend(), population_complete=True)
assert result.partitions[0].language == "go"
assert result.partitions[0].status == "unsupported"
assert "frontend_unsupported" in result.scope.unresolved_boundaries
def test_partial_partition_census_never_completes_questions(tmp_path):
for name in ("a.py", "b.py"):
(tmp_path / name).write_text("print('readable')")
class Backend:
def build(self, root, **kwargs):
return CpgResult(
Path("graph"),
lambda q, p: [],
run_batch=lambda q, reqs: {**{rid: [] for rid in reqs}, "__census__": [{"methods": 2, "files": 1}]},
)
region = ScanRegion(path="a.py", function="f", language="python", families=("injection",), rank=1.0, source="entry_point")
result = scan_repository(tmp_path, [region], backend=Backend(), budget=ScanBudget(tiering=False))
assert result.question_outcomes[0].status == "unresolved"
assert "cpg_empty" in result.scope.unresolved_boundaries
def test_projection_refuses_substituted_ancestor(tmp_path):
import os
import pytest
from openultrasast.model.partitions import _open_source
source = tmp_path / "source"
source.mkdir()
outside = tmp_path / "outside"
outside.mkdir()
(outside / "secret.js").write_bytes(b"must not enter the graph")
(source / "nested").symlink_to(outside, target_is_directory=True)
descriptor = os.open(source, os.O_RDONLY | os.O_DIRECTORY)
try:
with pytest.raises(OSError):
_open_source(descriptor, ("nested", "secret.js"))
finally:
os.close(descriptor)
def test_partition_records_reach_cli(tmp_path):
from openultrasast.cli import _model_payload
(tmp_path / "main.go").write_bytes(b"package main")
result = scan_repository(tmp_path, [], backend=object())
payload = _model_payload(result)
assert payload["partitions"][0]["language"] == "go"
assert payload["partitions"][0]["status"] == "unsupported"
def test_stale_region_cannot_bypass_vendor_graph_exclusion(tmp_path):
(tmp_path / "api.php").write_bytes(b"<?php echo 1;")
(tmp_path / "vendor").mkdir()
(tmp_path / "vendor/bad.php").write_bytes(b"<?php vendor_secret();")
seen = []
class Backend:
def build(self, root, **kwargs):
seen.extend(p.relative_to(root).as_posix() for p in root.rglob("*") if p.is_file())
return CpgResult(Path("graph"), lambda q, p: [])
region = ScanRegion(path="missing.php", function="f", language="php", families=("injection",), rank=1.0, source="entry_point")
result = scan_repository(tmp_path, [region], backend=Backend())
assert seen == ["api.php"]
assert result.question_outcomes[0].status == "unanswered"
def test_unreadable_unknown_extension_reports_incomplete_without_crashing(tmp_path, monkeypatch):
from openultrasast.model import partitions
(tmp_path / "app.js").write_bytes(b"function handler(value) { return value; }\n")
(tmp_path / "README.md").write_bytes(b"unreadable declaration candidate\n")
original_open = partitions.os.open
def unreadable_open(path, flags, *args, **kwargs):
if str(path) == "README.md":
raise PermissionError("controlled unreadable discovery input")
return original_open(path, flags, *args, **kwargs)
monkeypatch.setattr(partitions.os, "open", unreadable_open)
class Backend:
def build(self, root, **kwargs):
assert (root / "app.js").read_bytes()
return CpgResult(Path("graph"), lambda query, params: [])
region = ScanRegion("app.js", "handler", "javascript", ("injection",), 1.0, "entry_point")
result = scan_repository(tmp_path, [region], backend=Backend(), budget=ScanBudget(tiering=False))
assert result.partitions[0].status == "built"
assert "source_unreadable" in result.scope.unresolved_boundaries
assert result.question_outcomes[0].status == "unresolved"
def test_declared_javascript_tests_remain_inputs_but_unshipped(tmp_path):
from openultrasast.model.layout import with_layout
from openultrasast.model.partitions import build_partitions
names = ("unit.test.js", "unit.spec.mjs", "unit.mock.cjs", "unit.e2e.jsx", "unit.test.ts", "unit.spec.tsx", "__tests__/unit.js")
for name in (*names, "app.js"):
source = tmp_path / name
source.parent.mkdir(parents=True, exist_ok=True)
source.write_text("function readable() {}")
graph = build_partitions(tmp_path, lambda root, language: CpgResult(Path("graph"), lambda q, p: []), None)
try:
assert set(names) <= {p for record in graph.partitions for p in record.paths}
assert set(names) == {p for record in graph.partitions for p in record.unshipped_paths}
regions = [
ScanRegion(path=p, function="readable", language="javascript", families=("injection",), rank=1.0, source="entry_point")
for p in (*names, "app.js")
]
assert {r.path for r in with_layout(regions) if not r.shipped} == set(names)
finally:
graph.cleanup()
def test_synthetic_census_node_cannot_mask_missing_source(tmp_path):
from openultrasast.model.partitions import build_partitions
for name in ("a.php", "b.php"):
(tmp_path / name).write_text("<?php echo 1;")
def build(root, language):
return CpgResult(
Path("graph"),
lambda q, p: [],
run_batch=lambda q, reqs: {
**{rid: [] for rid in reqs},
"__census__": [{"methods": 3, "files": 2, "file_names": ["<unknown>", str(root / "a.php")]}],
},
)
graph = build_partitions(tmp_path, build, None)
try:
result = graph.run_batch("taint", {"q": {"file": "a.php"}})
assert result["__census__"][0]["files"] == 0
assert "partition_file_census_incomplete" in graph.execution_diagnostics()
finally:
graph.cleanup()
def test_named_census_accepts_exact_paths_for_every_query_kind(tmp_path):
from openultrasast.model.partitions import build_partitions
(tmp_path / "app.php").write_text("<?php echo 1;")
def build(root, language):
return CpgResult(
Path("graph"),
lambda q, p: [],
run_batch=lambda q, reqs: {
**{rid: [] for rid in reqs},
"__census__": [{"methods": 2, "files": 2, "file_names": ["<unknown>", str(root / "app.php")]}],
},
)
graph = build_partitions(tmp_path, build, None)
try:
for kind in ("taint", "dominance", "config"):
assert graph.run_batch(kind, {"q": {"file": "app.php"}})["__census__"][0]["files"] == 2
assert not graph.execution_diagnostics()
finally:
graph.cleanup()
def test_unnamed_sharded_census_remains_explicitly_incomplete(tmp_path):
from openultrasast.model.partitions import build_partitions
(tmp_path / "app.php").write_text("<?php echo 1;")
graph = build_partitions(
tmp_path,
lambda root, language: CpgResult(
Path("graph"),
lambda q, p: [],
run_batch=lambda q, reqs: {**{rid: [] for rid in reqs}, "__census__": [{"methods": 20, "files": 20, "shards": 2}]},
),
None,
)
try:
result = graph.run_batch("taint", {"q": {"file": "app.php"}})
assert result["__census__"] == [{"methods": 0, "files": 0, "shards": 2}]
assert "partition_file_census_unavailable" in graph.execution_diagnostics()
finally:
graph.cleanup()
+12 -7
View File
@@ -394,7 +394,7 @@ def test_a_query_the_engine_could_not_answer_is_recorded_not_ignored() -> None:
assert result.findings == () assert result.findings == ()
def test_a_region_cap_that_hides_most_of_a_repository_is_reported() -> None: def test_a_region_cap_that_hides_most_of_a_repository_is_reported(tmp_path: Path) -> None:
"""Paid Memberships Pro yields 4,463 regions from 637 files. A default budget of 500 examines 11% of it, """Paid Memberships Pro yields 4,463 regions from 637 files. A default budget of 500 examines 11% of it,
and a report that does not say so invites its silence to be read as a clean bill of health for the rest. and a report that does not say so invites its silence to be read as a clean bill of health for the rest.
""" """
@@ -414,14 +414,16 @@ def test_a_region_cap_that_hides_most_of_a_repository_is_reported() -> None:
ScanRegion(path=f"f{n}.py", function="handler", language="python", families=("injection",), rank=0.5, source="entry_point") ScanRegion(path=f"f{n}.py", function="handler", language="python", families=("injection",), rank=0.5, source="entry_point")
for n in range(30) for n in range(30)
] ]
result = scan_repository(Path("."), regions, backend=_Backend(), budget=ScanBudget(max_model_calls=0, max_regions=10)) for region in regions:
(tmp_path / region.path).write_bytes(b"print(1)\n")
result = scan_repository(tmp_path, regions, backend=_Backend(), budget=ScanBudget(max_model_calls=0, max_regions=10))
truncated = [d for d in result.degradations if d.get("reason") == "regions_truncated"] truncated = [d for d in result.degradations if d.get("reason") == "regions_truncated"]
assert truncated, "the cap hid two thirds of the regions and said nothing" assert truncated, "the cap hid two thirds of the regions and said nothing"
assert truncated[0]["examined"] == 10 and truncated[0]["total"] == 30 assert truncated[0]["examined"] == 10 and truncated[0]["total"] == 30
def test_tier_zero_pairs_are_not_asked_and_nothing_else_is_skipped() -> None: def test_tier_zero_pairs_are_not_asked_and_nothing_else_is_skipped(tmp_path: Path) -> None:
"""Phase 1 of flow-aware-ranking. A pair with no sink of its family in reach cannot yield a flow, so its """Phase 1 of flow-aware-ranking. A pair with no sink of its family in reach cannot yield a flow, so its
taint request is never issued -- and every other pair's request still is.""" taint request is never issued -- and every other pair's request still is."""
from openultrasast.model.regions import ScanRegion from openultrasast.model.regions import ScanRegion
@@ -466,7 +468,8 @@ def test_tier_zero_pairs_are_not_asked_and_nothing_else_is_skipped() -> None:
region = ScanRegion( region = ScanRegion(
path="a.py", function="run", language="python", families=("injection", "path", "deserialization"), rank=0.9, source="entry_point" path="a.py", function="run", language="python", families=("injection", "path", "deserialization"), rank=0.9, source="entry_point"
) )
result = scan_repository(Path("."), [region], backend=_Backend(), budget=ScanBudget(max_model_calls=0)) (tmp_path / "a.py").write_bytes(b"print(1)\n")
result = scan_repository(tmp_path, [region], backend=_Backend(), budget=ScanBudget(max_model_calls=0))
taint = next(i for i in issued if i["kind"] == "taint") taint = next(i for i in issued if i["kind"] == "taint")
assert taint["count"] == 1, "only the pair with a sink in reach is asked" assert taint["count"] == 1, "only the pair with a sink in reach is asked"
@@ -474,7 +477,7 @@ def test_tier_zero_pairs_are_not_asked_and_nothing_else_is_skipped() -> None:
assert not any(d.get("reason") == "query_failed" for d in result.degradations) assert not any(d.get("reason") == "query_failed" for d in result.degradations)
def test_the_budget_is_spent_by_evidence_when_asked() -> None: def test_the_budget_is_spent_by_evidence_when_asked(tmp_path: Path) -> None:
"""Phase 2 of flow-aware-ranking. With `order_by_evidence`, the evidence pass runs over every region and """Phase 2 of flow-aware-ranking. With `order_by_evidence`, the evidence pass runs over every region and
the region cap is cut from the (tier, score) order, so a low-ranked region with an open sink and a the region cap is cut from the (tier, score) order, so a low-ranked region with an open sink and a
local source is examined before a top-ranked region with no sink in reach. Off, the static order holds.""" local source is examined before a top-ranked region with no sink in reach. Off, the static order holds."""
@@ -523,15 +526,17 @@ def test_the_budget_is_spent_by_evidence_when_asked() -> None:
top = ScanRegion(path="top.py", function="handle", language="python", families=("injection",), rank=1.0, source="entry_point") top = ScanRegion(path="top.py", function="handle", language="python", families=("injection",), rank=1.0, source="entry_point")
low = ScanRegion(path="low.py", function="run", language="python", families=("injection",), rank=0.3, source="entry_point") low = ScanRegion(path="low.py", function="run", language="python", families=("injection",), rank=0.3, source="entry_point")
(tmp_path / "top.py").write_bytes(b"print(1)\n")
(tmp_path / "low.py").write_bytes(b"print(2)\n")
static = _Backend() static = _Backend()
scan_repository(Path("."), [top, low], backend=static, budget=ScanBudget(max_model_calls=0, max_regions=1)) scan_repository(tmp_path, [top, low], backend=static, budget=ScanBudget(max_model_calls=0, max_regions=1))
assert static.evidence_sizes == [1] and static.taint_files == [], ( assert static.evidence_sizes == [1] and static.taint_files == [], (
"static order: the budget holds `top`, whose pair is tier 0 and pruned" "static order: the budget holds `top`, whose pair is tier 0 and pruned"
) )
ordered = _Backend() ordered = _Backend()
result = scan_repository( result = scan_repository(
Path("."), [top, low], backend=ordered, budget=ScanBudget(max_model_calls=0, max_regions=1, order_by_evidence=True) tmp_path, [top, low], backend=ordered, budget=ScanBudget(max_model_calls=0, max_regions=1, order_by_evidence=True)
) )
assert ordered.evidence_sizes == [2], "the evidence pass covers every region, not only the budget" assert ordered.evidence_sizes == [2], "the evidence pass covers every region, not only the budget"
assert ordered.taint_files == [["low.py"]], "and the budget is spent on the region the evidence points at" assert ordered.taint_files == [["low.py"]], "and the budget is spent on the region the evidence points at"
+3 -3
View File
@@ -23,7 +23,7 @@ class Backend:
def build(self, root, **kwargs): def build(self, root, **kwargs):
# Prove these tests provide real readable source to the driver boundary. # Prove these tests provide real readable source to the driver boundary.
assert any(p.read_bytes() for p in root.iterdir() if p.is_file()) assert any(p.read_bytes() for p in root.rglob("*") if p.is_file())
if self.fail: if self.fail:
return None return None
return SimpleNamespace(cpg_path=Path("test.cpg"), run_batch=self.batch, cleanup=lambda: None) return SimpleNamespace(cpg_path=Path("test.cpg"), run_batch=self.batch, cleanup=lambda: None)
@@ -162,7 +162,7 @@ def test_later_timeout_retains_raw_answer_without_claiming_arbitration(tmp_path)
def batch(self, kind, requests): def batch(self, kind, requests):
rows = super().batch(kind, requests) rows = super().batch(kind, requests)
if kind == "taint" and not any(p.get("evidenceOnly") for p in requests.values()): if kind == "taint" and not any(p.get("evidenceOnly") for p in requests.values()):
time.sleep(0.08) time.sleep(0.3)
return {rid: [{"kind": "retained", "value": 42}] for rid in requests} return {rid: [{"kind": "retained", "value": 42}] for rid in requests}
return rows return rows
@@ -171,7 +171,7 @@ def test_later_timeout_retains_raw_answer_without_claiming_arbitration(tmp_path)
[region("a.py", families=("injection", "access_control"))], [region("a.py", families=("injection", "access_control"))],
backend=Slow(), backend=Slow(),
ranking_mode="static", ranking_mode="static",
execution_budget=ExecutionBudget(time.monotonic() + 0.05, 0.2), execution_budget=ExecutionBudget(time.monotonic() + 0.2, 0.2),
) )
outcomes = {q.identity.family: q for q in result.question_outcomes} outcomes = {q.identity.family: q for q in result.question_outcomes}
assert outcomes["injection"].status == "not_arbitrated" assert outcomes["injection"].status == "not_arbitrated"