diff --git a/.kiro/specs/ai-service-plane/design.md b/.kiro/specs/ai-service-plane/design.md index 83ee4eb..d22c240 100644 --- a/.kiro/specs/ai-service-plane/design.md +++ b/.kiro/specs/ai-service-plane/design.md @@ -141,7 +141,7 @@ before `ax apply`, and a task without a Model gets none. The value is never rend Every actor connection except DNS goes through Agent Substrate's egress gateway (`atenet-egress`), which denies an actor without an `EgressPolicy`. Three pieces make the plane work under that rule: -- **Receiver by name.** `ops/ax/receiver-service.yaml.tmpl` (applied by `up.sh`) is a selector-less Service +- Receiver by name. `ops/ax/receiver-service.yaml.tmpl` (applied by `up.sh`) is a selector-less Service `ousast-receiver` in `ax-system`, port 80, with an EndpointSlice on the kind gateway at `OUSAST_ARTIFACT_PORT` (fixed, 18090), where the reconciler binds its receiver. Tasks get `OUSAST_ARTIFACT_URL=http://ousast-receiver.ax-system.svc.cluster.local/` plus `OUSAST_ARTIFACT_DIAL=` inside the labeled function counts. -- `src/openultrasast/semantic/mechanisms.py` — no schema change; `corpus_mechanisms()` unchanged; a helper `obligation_mechanisms(records)` selects `shape["family"] == "obligation"` (added in `obligations/shapes.py`, not in the store module). +- `src/openultrasast/cli.py`: `_check_obligations` after `_search_variants` in MAP (standard/deep); manifest `obligations` block; `mechanisms export` writes obligation rows too. +- `src/openultrasast/pairs.py`: `ExpectedFinding.obligation: str | None`; `_detections`-side additive rule: for a row with `obligation`, a finding with tag `obligation:` inside the labeled function counts. +- `src/openultrasast/semantic/mechanisms.py`: no schema change; `corpus_mechanisms()` unchanged; a helper `obligation_mechanisms(records)` selects `shape["family"] == "obligation"` (added in `obligations/shapes.py`, not in the store module). - `src/openultrasast/reports.py`, `rank.py`, `config.py`, `improve/*`, `semantic/seed.py`, `semantic/loo.py`, `benchmarks/pairs/harvest.py`, `benchmarks/pairs/mechanisms.toml` as listed above. ## System Flows @@ -177,7 +177,7 @@ flowchart LR I --> J[StaticFinding suspicion; tags obligation:*, discharger:*, obligation_evidence:*] ``` -Gating: the checker emits a finding only when (1) the operation is reached — from an entry point along a `PathRecord`, or, without paths, when the operation sits in a handler that is itself an entry point — and (2) no discharger of the required kind dominates, and (3) the obligation is justified by a sibling set (`consistency_violation`), a policy clause (`declared_policy_violation`) or, in degraded mode without either, by the operation fact alone (`function_local`). A declared public route suppresses path-guard obligations for that route only. +Gating: the checker emits a finding only when (1) the operation is reached (from an entry point along a `PathRecord`, or, without paths, when the operation sits in a handler that is itself an entry point) and (2) no discharger of the required kind dominates, and (3) the obligation is justified by a sibling set (`consistency_violation`), a policy clause (`declared_policy_violation`) or, in degraded mode without either, by the operation fact alone (`function_local`). A declared public route suppresses path-guard obligations for that route only. ### Teaching and measuring diff --git a/.kiro/specs/authorization-obligations/requirements.md b/.kiro/specs/authorization-obligations/requirements.md index 8f76430..beba1a4 100644 --- a/.kiro/specs/authorization-obligations/requirements.md +++ b/.kiro/specs/authorization-obligations/requirements.md @@ -8,9 +8,9 @@ AppSec engineers and maintainers of OpenUltraSAST scan web code, increasingly vi ## Boundary Context -- **In scope**: obligated operations and discharger kinds as closed facts data; obligation shapes learned from trusted pairs; consistency detection across sibling handlers; an optional declared policy file with a closed schema; path-aware violation evaluation with a function-local degraded mode; obligation evidence rungs, report wording and worth-fixing weighting; a corpus recipe shape that keeps the handler's registration context; leave-one-out per obligation kind; an `obligations` improve lever; offline tests. -- **Out of scope**: flow taint, walker or sink facts; the call graph, path records and dominance computation (owned by `reachability-flow-model` and `guard-dominance-regime`); model-authored policy or shapes; sandbox proof of absence bugs; any merge gate; vendoring unlicensed code. -- **Adjacent expectations**: the entry-point mapper keeps providing routes with an access level and their decorators; the reachability spec provides entry-point-to-operation paths and the guard-dominance spec provides "a guard dominates the path"; the mechanism store, its deterministic ids and the lever contract from `corpus-seeded-mechanisms` are reused unchanged; the pair scorer accepts one additive label field; `local` remains the only slice that can fail CI. +- In scope: obligated operations and discharger kinds as closed facts data; obligation shapes learned from trusted pairs; consistency detection across sibling handlers; an optional declared policy file with a closed schema; path-aware violation evaluation with a function-local degraded mode; obligation evidence rungs, report wording and worth-fixing weighting; a corpus recipe shape that keeps the handler's registration context; leave-one-out per obligation kind; an `obligations` improve lever; offline tests. +- Out of scope: flow taint, walker or sink facts; the call graph, path records and dominance computation (owned by `reachability-flow-model` and `guard-dominance-regime`); model-authored policy or shapes; sandbox proof of absence bugs; any merge gate; vendoring unlicensed code. +- Adjacent expectations: the entry-point mapper keeps providing routes with an access level and their decorators; the reachability spec provides entry-point-to-operation paths and the guard-dominance spec provides "a guard dominates the path"; the mechanism store, its deterministic ids and the lever contract from `corpus-seeded-mechanisms` are reused unchanged; the pair scorer accepts one additive label field; `local` remains the only slice that can fail CI. ## Requirements diff --git a/.kiro/specs/authorization-obligations/research.md b/.kiro/specs/authorization-obligations/research.md index d550afb..52d0c12 100644 --- a/.kiro/specs/authorization-obligations/research.md +++ b/.kiro/specs/authorization-obligations/research.md @@ -2,9 +2,9 @@ ## Summary -- **Feature**: `authorization-obligations` -- **Discovery Scope**: Extension of the semantic layer (facts, shapes, store, MAP proposers), the entry-point mapper, reports, the pair corpus, leave-one-out and the improve lever; complex integration because it consumes two unimplemented upstream specs. -- **Key Findings**: +- Feature: `authorization-obligations` +- Discovery Scope: Extension of the semantic layer (facts, shapes, store, MAP proposers), the entry-point mapper, reports, the pair corpus, leave-one-out and the improve lever; complex integration because it consumes two unimplemented upstream specs. +- Key Findings: - The absence pairs are real and structured. `vampi-books-get-by-title` has identical code on both sides except a constraint `user=user` bound from `token_validator(request.headers.get('Authorization'))['sub']`; the vulnerable branch queries `Book` by title alone. The difference is the provenance of a constraint, not a flow. 15 of 29 `agent-vfc` rows and 6 of the 19 non-teaching `vibe-py` pairs carry absence labels (`missing_auth_guard`, `permissive_default`, `identity_from_request_body`, `secret_in_client`, `validation_strength`). - The entry-point mapper already classifies every route as `public`, `authenticated` or `role-restricted` from its decorators (`mapping._python_route_access`), records the decorators as `access_evidence`, and lists `conditions` (`if` tests that look like checks). That is the sibling detector's upstream for path guards and needs no change. - `FileIR` binds carry `value_text`, `names` and `call_name`; `variants.source_kind_of_name` already traces a name through binds to a fact source, a parameter or a container read. Provenance of a constraint's value (authenticated context versus request input versus constant) is the same trace with one more source class. @@ -17,29 +17,29 @@ ### What the absence pairs actually differ by -- **Context**: Decide whether absence bugs can be a shape family at all. -- **Sources Consulted**: `benchmarks/pairs/vibe-py` excerpts for `missing_auth_guard` and `permissive_default` rows; `benchmarks/pairs/agent-vfc/recipes.toml` mechanisms; `mechanisms.toml` definitions. -- **Findings**: three recurring differences: (a) a constraint added to a data access whose value comes from the authenticated context; (b) a guard call or decorator added before the operation (early return on a failed token check, `login_required`); (c) a literal changed from permissive to restrictive (`CORS(app)` → origins list, `DEBUG=True` → `False`, RLS enabled). A fourth, rarer: identity read from the body replaced by identity from the token (same as (a) seen from the other side). -- **Implications**: discharger kinds `identity_constraint`, `path_guard`, `ownership_check`, `non_permissive_value`, `validated_input`; the value provenance is the discriminating feature for (a) and (d). +- Context: Decide whether absence bugs can be a shape family at all. +- Sources Consulted: `benchmarks/pairs/vibe-py` excerpts for `missing_auth_guard` and `permissive_default` rows; `benchmarks/pairs/agent-vfc/recipes.toml` mechanisms; `mechanisms.toml` definitions. +- Findings: three recurring differences: (a) a constraint added to a data access whose value comes from the authenticated context; (b) a guard call or decorator added before the operation (early return on a failed token check, `login_required`); (c) a literal changed from permissive to restrictive (`CORS(app)` → origins list, `DEBUG=True` → `False`, RLS enabled). A fourth, rarer: identity read from the body replaced by identity from the token (same as (a) seen from the other side). +- Implications: discharger kinds `identity_constraint`, `path_guard`, `ownership_check`, `non_permissive_value`, `validated_input`; the value provenance is the discriminating feature for (a) and (d). ### Where the policy comes from -- **Context**: Absence is relative to intent; the tool must not invent intent. -- **Sources Consulted**: `mapping.analyze_entry_points` (routes, access level, evidence), Real-Vuln repositories (many handlers per resource), Clearwing's sibling-comparison practice. -- **Findings**: in every Real-Vuln web app the handlers on one resource share a router and a model; the vulnerable handler is the odd one out. Route access classification already exists. A declared policy is the only way to say "this route is meant to be public"; the hunter can only guess it. -- **Implications**: consistency first (no declaration needed, measurable now), declared policy second (stronger rung), hunter last (records intent, never raises). +- Context: Absence is relative to intent; the tool must not invent intent. +- Sources Consulted: `mapping.analyze_entry_points` (routes, access level, evidence), Real-Vuln repositories (many handlers per resource), Clearwing's sibling-comparison practice. +- Findings: in every Real-Vuln web app the handlers on one resource share a router and a model; the vulnerable handler is the odd one out. Route access classification already exists. A declared policy is the only way to say "this route is meant to be public"; the hunter can only guess it. +- Implications: consistency first (no declaration needed, measurable now), declared policy second (stronger rung), hunter last (records intent, never raises). ### Dominance before guard-dominance-regime exists -- **Context**: Req 5 needs "a discharger dominates the path"; the spec that owns dominance is not written. -- **Findings**: within one handler, the existing `FunctionIR` order of binds and calls gives statement order; a guard call followed by an early return before the operation line is dominance in the common case; decorators dominate the whole handler; middleware attached to the router dominates every route under it. Across calls, `PathRecord.hops` gives the sequence of nodes; a discharger on any hop before the operation node dominates the path when the hop's function returns early on failure. -- **Implications**: define `Dominance` as a narrow protocol (`dominates(witness, operation) -> bool`) with a built-in `OrderDominance` implementation (statement order in the handler, decorators, router middleware, hop order) that `guard-dominance-regime` may replace with nesting-aware dominance. The degraded mode (`function_local`) is `OrderDominance` without path records. +- Context: Req 5 needs "a discharger dominates the path"; the spec that owns dominance is not written. +- Findings: within one handler, the existing `FunctionIR` order of binds and calls gives statement order; a guard call followed by an early return before the operation line is dominance in the common case; decorators dominate the whole handler; middleware attached to the router dominates every route under it. Across calls, `PathRecord.hops` gives the sequence of nodes; a discharger on any hop before the operation node dominates the path when the hop's function returns early on failure. +- Implications: define `Dominance` as a narrow protocol (`dominates(witness, operation) -> bool`) with a built-in `OrderDominance` implementation (statement order in the handler, decorators, router middleware, hop order) that `guard-dominance-regime` may replace with nesting-aware dominance. The degraded mode (`function_local`) is `OrderDominance` without path records. ### Store and lever reuse -- **Context**: Avoid a second store and a second lever machinery. -- **Findings**: `Mechanism.shape` is a dict validated where used; `corpus_mechanism_id` hashes `shape.key()`; `validate_mechanism` checks guard and source kinds against `variants` constants. A second family needs its own closed kinds check and a `family` field in the shape dict; the tombstone, snapshot and profile clause need nothing. -- **Implications**: `ObligationShape.to_dict()` carries `family = "obligation"`; the lever validator dispatches on `family`; `search_tree` ignores obligation rows (they are not sink shapes) and the obligation checker ignores sink rows. +- Context: Avoid a second store and a second lever machinery. +- Findings: `Mechanism.shape` is a dict validated where used; `corpus_mechanism_id` hashes `shape.key()`; `validate_mechanism` checks guard and source kinds against `variants` constants. A second family needs its own closed kinds check and a `family` field in the shape dict; the tombstone, snapshot and profile clause need nothing. +- Implications: `ObligationShape.to_dict()` carries `family = "obligation"`; the lever validator dispatches on `family`; `search_tree` ignores obligation rows (they are not sink shapes) and the obligation checker ignores sink rows. ## Architecture Pattern Evaluation @@ -54,23 +54,23 @@ ### Decision: Obligation findings are `suspicion` with a data label, not new ladder rungs -- **Context**: Req 6.1 names three labels; the ladder is closed and owned elsewhere. -- **Selected Approach**: `StaticFinding.evidence_level = "suspicion"`, tags `obligation:`, `discharger:`, `obligation_evidence: