mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-16 03:49:26 +00:00
bd53b6dc9de02c7d34922bf4ae6cec1ccc2ab65a
210
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Marius CiepluchandClaude Opus 5
|
d15985f695 |
fix(model): a guard discharges the obligation it is a guard of
contributor-scan 2.6. Access control entailed 7 of 9 VAmPI handlers,
including three that do authorize and three the OpenAPI spec declares
public. Five separate defects, each invisible to a corpus of one-function
excerpts and each found by asking the query what it actually matched.
**A trailing segment is not a match.** `query.get` was matched by its last
segment, so every `.get(` in the repository became a protected database
operation -- `request.headers.get('Authorization')` and
`request_data.get('username')` included. Four of the seven false positives
came from this, and so did the apparent detection of the known BOLA: it was
entailed because fetching the Authorization header had no dominating guard,
not because of the defect. A qualified token now requires its qualifier.
**Configuration is not authorization.** dominance_specs took every
obligation fact. `security_setting` operations are discharged by a VALUE,
which config_specs already claims them for; pooled in here they made
`app.run(...)` an operation requiring an authorization guard and put
thirteen tokens in BOTH lists -- an obligation that is its own discharge,
the same shape as the taint sink that used to sanitize itself.
**Input validation is not authorization.** `jsonschema.validate` was
counted as a guard, so handlers with no authorization check at all read as
guarded. No operation in the table requires validated_input.
**A guard discharges the obligation it is a guard of.** The largest one.
`orm-read` requires an identity_constraint or an ownership_check;
`token_validator` is a path_guard. Flattened into one list, authenticating
the caller discharged an object-level obligation -- a handler that
authenticates and then looks the record up by a PATH parameter read as
fully guarded. The facts state that relation and the spec was discarding
it; it is now carried and the engine asks per operation.
**A word boundary needs a word.** Found while fixing the above. Guard
matching wrapped every token in `(?<![A-Za-z0-9_])...(?![A-Za-z0-9_])`.
The identity token `['sub']` begins with `[`, and the character before it
in `resp['sub']` is `p` -- so the assertion could never hold. Every
identity token starting with punctuation was unmatchable, and the identity
constraint that separates a correct lookup from an IDOR was invisible.
Also: dominance rows were repository-wide, so the model's unguarded
`delete_user` was attributed to the API handler of the same name. Rows are
scoped to the region's file, which is what the rung's own justification
says -- the evidence is a file contradicting itself.
Measured against VAmPI's OpenAPI `security` blocks, which declare per
operation whether auth is required, and agree with the code:
vampi precision 28.6% (2/7) -> 50.0% (2/4)
known found 3 of 3, all model_entailed
And on the labelled access-control pairs, which nobody tuned this against:
all 9 pairs pair_correct 3 -> 3, leak 44.4% -> 11.1%
7 non-VAmPI pair_correct 2 -> 2, leak 42.9% -> 0.0%
Every detection lost was on a pair whose FIXED side was flagged too, which
scores nothing. No correct detection was traded for the precision.
get_by_title was flagged before it was in the recipe: its code carries the
comment "# Broken Object Level Authorization" and the README lists it, so a
real finding was being scored as a false positive. It is now the recipe's
third known vulnerability.
Both remaining false positives are endpoints the spec declares public.
Recorded as 2.8, with the reason the mapper's access_level cannot simply be
trusted: for decorator frameworks "public" means "no decorator found",
which is the bug itself.
Gates byte-identical.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
|
||
|
Marius CiepluchandClaude Opus 5
|
0fcd5feade |
feat(model): a region for the code that is in no function
contributor-scan 2.5. VAmPI's `host='0.0.0.0'` is now reported exactly once, at app.py:17:<module>. Before task 2.2 it was reported eight times in eight files that do not contain it; after 2.2 it was reported nowhere. Two causes, one shape: nothing could ask about module-level code. **The main-guard marker is not a function.** `__main__` is what the mapper emits for `if __name__ == '__main__':`. Sent to the engine as a function name it matched nothing, because a CPG calls a file's top-level code `<module>` -- verified against the VAmPI CPG rather than assumed. The region model translates the marker, which is the one place that turns what the mapper meant into what the scanner asks. **A function-scoped file had no region for its module body.** Settings, route registration and globals live in no function, and every region's query filters by function name. A file-level region beside the handlers would report each handler's findings a second time, which is why the earlier rule dropped it as "a second full-file pass". A module-scope region PARTITIONS the file instead of overlapping it. That only became cheap after batching: before it, a region was a JVM. Two things this needed to be correct rather than merely present: - A module method spans its whole file, so the taint query's line-range containment made `<module>` the parent of every function in it. Under module scope, membership is direct -- otherwise the module region reports every handler's sinks again, which is the duplication the old rule was avoiding. - A module body is not a handler. The marker-derived region inherited the handler families and entailed a missing authorization check on the main guard: a finding with no possible remediation. Caught by the first repository run that had the region, and now covered by a test. bind address 8 wrong files -> 0 -> 1, at app.py:17:<module> regions 22 -> 25 entailed 8 -> 9 query time 37.4s -> 41.3s peak RSS 827MB -> 823MB cost $0.0067 -> $0.0068 Three more regions for four seconds and no more model calls: the budget is a total and was already spent, so regions_unasked rose 17 -> 20 -- those regions got the free pass and are reported as unasked, not as clean. `<module>` is Joern's Python name for a module body; JavaScript uses `:program`, so this is verified for Python only. Gates byte-identical. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
ca7b5ba370 |
feat(model): a region that spans the flow, not just a function
contributor-scan 2.4, closing the no-go task 2.2 recorded. VAmPI's SQL injection is a request parameter reaching `get_by_username` in api_views/users.py, passed to `User.get_user` in models/user_model.py, interpolated into a query string and executed there. Source and sink are in different functions and different modules. Regions were per-function, so neither region saw both ends and the engine was silent on the most ordinary shape a real injection bug has. `reachableByFlows` was never the limitation -- it follows calls already. The limitation was the QUESTION. Sinks were filtered to the labeled function's own body, so a crossing flow had no sink to end at. Three changes, each with a reason it is sound rather than convenient: **An entry point is the trust boundary.** parameter_sources was off for repository scans because a repository's parameters are not all attacker input. An entry point's are, by definition -- that is what makes it an entry point. The driver turns it on only for regions the mapper found and named, and leaves it off for walker fallbacks and for named functions that are not entry points. **Sinks may sit below the handler.** taint.sc admits sinks in methods reachable from the labeled method through the call graph, bounded at ENTRY_POINT_CALL_DEPTH = 3: handler -> service -> data access, which is where these bugs live. Unbounded would make the question about most of the repository rather than about this entry point, and make it unaffordable. **The finding names the sink, not the asker.** A crossing flow ends in another file and another function, so Verdict carries the evidence's own `path:line:function` and the site uses it. Reporting `models/user_model.py:73:get_by_username` would name a place that does not exist: line 73 is inside get_user. Also: labeled methods are now resolved by name AND file. `get_user` exists in more than one module of a real repository, which a corpus of one-file excerpts could never surface. Measured on the same checkout and budget as 2.2: VAmPI-SQLI missed -> model_entailed at models/user_model.py:73:get_user known found 0 of 2 before 2.2's fixes, 1 of 2 in 2.2, 2 of 2 now entailed 7 -> 8 query time 40.9s -> 37.4s peak RSS 869MB -> 827MB cost $0.0067, unchanged Asking the bigger question cost nothing measurable here: file scoping and call-graph following pull in opposite directions and roughly cancel at this size. That is one 520-line repository, and VAmPI is now the repository this change was developed against -- the exact overfitting shape Req 8 exists to catch. The depth bound and the entry-point rule mean nothing general until a checkout nobody tuned against says so. Gates byte-identical. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
177c7f3719 |
feat(model): first repository measurement, and the four things it found
contributor-scan 2.2. VAmPI at a pinned commit, scanned whole rather than as excerpts, with benchmarks/repos/measure.py recording build time, query time, arbitration time, peak memory, cost, unjudged regions, and whether each known vulnerability was found and at which rung. The corpus could not have found any of this. Every defect below needs a repository to become visible, and three of them were invisible behind a passing test suite. **The batch path was unreachable.** query_batch was implemented, tested and committed while CpgResult exposed only `run`, so the driver's capability check found nothing and fell back to one JVM per region-family: the first attempt at this measurement ran eighteen minutes on 520 lines, issuing `--param function=` one at a time. Every test called query_batch directly, so none of them noticed. CpgResult now carries run_batch and a test asserts the capability is reachable through the object the driver is handed. The earlier 275s->106s improvement attributed to batching came from region deduplication alone. **The budget stopped free work.** It bounds model calls, but was enforced by breaking the region loop -- spending a paid limit to stop unpaid CPG arbitration. 6 of 22 regions were examined and VAmPI's BOLA, which the graph entails for nothing, was never reached. It now withholds the judge only. 22 of 22 regions arbitrated, 17 counted as regions_unasked (silent because unaffordable is not silent because decided), and the BOLA is entailed by the model layer. **A region with no function asked about the whole repository.** Sending function="" matched every hit in the tree and the driver attributed that answer to whoever asked: one `host='0.0.0.0'` in app.py became eight identical entailed findings in eight files that do not contain it. Queries now carry the region's file. Entailed findings 14 -> 7, query time 70.9s -> 40.9s. The single-request form stays unscoped so the committed pair measurements remain reproducible. **The JVM had no heap bound.** Joern claimed a quarter of physical RAM and runs were killed under memory pressure at different points, so the numbers described the host rather than the engine. -J-Xmx2048m by default, OPENULTRASAST_CPG_HEAP_MB to raise it. Also: arbitrated findings carried `function:line` where the report layer read `path:line`, so every entailed finding reached a contributor with a function name where its file belonged; and the witness line was parsed by splitting on a comma, which produced sites like `config.py:17): permissive literal '0.0.0.0':?`. Reading: GO on one CPG per repository -- 6.2s to build, 177s end to end, 869MB, $0.0067, nothing infeasible. NO-GO on region-scoped taint as the only shape: VAmPI's SQL injection runs from a handler parameter into a function in another module, and a per-function region sees neither end. That is the ordinary shape of a real injection bug, so it is recorded as task 2.4 along with three smaller defects this run exposed. Gates byte-identical. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
8c387306be |
feat(repos): pinned known-vulnerable checkouts, offline by default
The pair corpus scores excerpts: a vulnerable function and its fixed twin, handed to the engine with the answer already narrowed down. Nothing in it exercises a repository -- region ranking, the budget, one CPG over thousands of files, or whether a verdict that holds on fifteen lines still holds when the source, the sink and the guard live in three modules. benchmarks/repos/<name>.toml names a real checkout: url, a full 40-char commit SHA, licence, and every known vulnerability with its file, function and line. `ousast repos` lists them; `--fetch` materialises one; CI does neither, and conftest clears the switch and redirects the cache so a developer with warm checkouts cannot read them by accident. Three rules the loader enforces rather than documents: - A branch or tag is rejected. Either would make the same recipe mean different code next month while the baseline it produced kept the old numbers. - `in_scope` must be stated per vulnerability, never defaulted. It decides whether "not found" is a miss or a non-question, which is the most important thing about the result. - A recipe that declares `detection` must name something in scope to detect. `ousast repos` does not trust a recipe either: against a materialised checkout it checks HEAD against the pin, that the licence file is there, and that each declared function appears above its declared line. That earned its place immediately -- the first libpng recipe named LICENSE.md, which is the filename at HEAD today and not at the pinned commit, where it is still LICENSE. Two checkouts, verified against the real repositories: - vampi (MIT, 520 lines) is the DETECTION checkout. Excerpts of it are already in the pair corpus, including the update_password miss diagnosed earlier, so a whole-repository run says directly whether a verdict survives the move off the excerpt. Its routes register through an OpenAPI operationId, so it exercises registration-based entry-point mapping rather than the decorator path. - libpng (libpng-2.0, 91k lines of C) is the ENVELOPE checkout, pinned at the parent of the commit that added the missing clamp. Its CVE is marked out of scope: an out-of-bounds read is not a family this engine arbitrates, and recording "not found" against it as a miss is the easiest way to make a measurement lie. Nothing is vendored; the recipe records the licence so an artifact can say what it ran against. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
c35426e186 |
perf(model): one Joern invocation per query kind, not one per region
JVM startup dominated a scan: a call per region per family put a ten-line file at 275s over three regions, and would have put a thousand regions at roughly fifty hours. The queries themselves are milliseconds once the CPG is loaded, so the cost was almost entirely process starts. scan_repository() is now collect -> batch -> arbitrate. The three .sc scripts accept a `requests` JSON map and answer all of them in one run, keeping the single-request form so the committed measurements stay reproducible byte-for-byte. Arbiters are untouched: each is handed a CpgResult whose `run` returns its own prefetched rows. The batcher builds parameters by calling request_params() extracted from taint.py, dominance.py and config_value.py rather than reconstructing them. A second copy that drifts from the arbiter's is the exact shape of the five family-assumption bugs the predecessor spec spent a session finding; there is now one definition per family and the batcher borrows it. Regions are also deduplicated by (path, function), collapsing the nameless entry-point rows that were producing `model:injection:?:10`. Measured on the same ten-line file: 275s -> 106s, three duplicate findings -> one `model:injection:ping:10`, rung unchanged at model_entailed. Gates byte-identical. Server mode is recorded as task 2.3, deliberately sequenced after the repository measurement: whether the remaining ~100s floor is worth a process model, a weakened subprocess seam and a Scala-evaluating HTTP endpoint depends on a build-time-to-query-time ratio only 2.2 produces. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
e67f3ee2f0 |
feat(contributor-scan): 1.3 the model layer reaches a user
`model-grounded-detection` finished 16/16 with the model layer having zero
production callers. This closes that. A real scan of a Flask file now reports:
by_rung: {"model_entailed": 3, "model_corroborated": 0, "suspicion": 0}
model:injection:ping:10 <- os.system("ping -c 1 " + request.args["host"])
The stage is the fifth MAP proposer, mirroring _check_obligations, and is
additive: pattern, overlay and obligation findings all keep working, and without
a CPG engine it records cpg_unavailable and changes nothing else. Two scan
integrity tests pin the exact degradation set and both broke -- correctly. Their
`requested == baseline` assertion, that findings are byte-identical, still
passed, so the only change is that the scan now SAYS the arbiter was absent
instead of being silently without one. The guards are updated to assert the
model degradation specifically rather than being loosened.
ModelLayerConfig, not ModelConfig: that name is already the [models] block, and
the type checker caught the collision immediately. Two meanings of one name is
how a scan silently reads the wrong settings.
Every payload carries the same shape, skipped ones included. A payload that is
sometimes a full record and sometimes a single "skipped" key is how a report
ends up quietly omitting a section.
unjudged_sample names the lowest-ranked regions the budget did not reach.
Counting them is the honesty requirement; naming them is what lets a maintainer
write a large-repository exclusion list from evidence rather than guesswork.
The run also produced the finding behind new task 1.4: 275 seconds for a
ten-line file. cost_usd is 0.0, so those 21 invocations were not model calls --
they were Joern query launches, one per region per family, each a JVM start.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
|
||
|
Marius CiepluchandClaude Opus 5
|
8663c1a5b3 |
feat(contributor-scan): 1.1 regions from entry points, 1.2 the repository driver
Pair scoring could rely on a labelled function; a contributor cannot supply one -- which code is risky is the question they are asking. Regions now come from entry points, with a file-level fallback that says it is one, and each carries only the families its shape and language admit. Ranking uses what EntryPointRecord already decided (access_level, trust_boundary) rather than a second opinion computed from the same evidence. A family is offered only where its arbiter could answer: access_control needs a handler, so a library file with no entry point is not asked about obligations it has no siblings to be inconsistent with. The driver makes the two costs that were free at pair scale explicit. ONE CPG per repository, reused across regions -- a build is 5-30s on a forty-line excerpt and per-region builds would make this unusable. And a budget that is a TOTAL for the run rather than the enumerator's per-region cap of eight, spent highest-rank-first, with whatever it does not reach counted into regions_unjudged. A tool that silently analyses a tenth of a repository and reports a clean result is worse than one that says what it skipped. Two flaws the tests caught, both mine and both about ownership: the budget counted calls by reading a client's own usage log, so it silently never tripped for a client that keeps none -- the driver now counts its own spend through a wrapper; and the driver trusted its caller to have sorted, when the order the budget is spent in belongs to whoever holds the budget. The third was worse and only surfaced because a test failed for the right reason: the driver passed candidates=() and never enumerated, so a repository scan would have reported only what the arbiters establish unaided and never populated the suspicion band -- which supplied 15 of 26 findings in the pair measurement. It enumerates now. Sibling audit (Req 8.5) done before writing: family routing lives in exactly one production place, pipeline._arbitrate, dispatching on spec type. regions_for returns family names and the driver resolves specs, so that stays true. 750 passing, mypy clean over 96 files, gates byte-identical, zero orphans. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
fe45081b43 |
fix(ci): run ruff format, and record run 5 at 90% pair-correct
CI was red on `ruff format --check`: 23 files. I ran `ruff check` after every
change this session and never once ran `ruff format --check`, though CI runs
both. Lint passing is not formatting passing, and I treated one as evidence for
the other for the whole session.
Formatting applied. The full CI sequence now reproduces locally: format, lint,
mypy over 94 files, 735 tests, three gates byte-identical, compileall, and the
semantic job's 16 marked tests.
Also records run 5 of the pipeline measurement, which the vampi fix lifted as
predicted:
hunter (K=5) pipeline (K=1)
pair-correct 76.7% 90.0%
leak 12.3% 6.7%
detection -- 96.7%
cost ~$0.69 $0.0068
27 of 30 pairs. Five runs, each step from a measurement contradicting an
assumption: 73.3 (no pipeline path) -> 73.3 (routing fixed, IDENTICAL, which
exposed the discarded verdict) -> 80.0 (residual question matched to its
arbiter) -> 86.7 (aiohttp settings) -> 90.0 (dominance question no longer
contradicting its own evidence).
One miss left: dsvpwa-dsvpwa-main, whose setting is an argparse --ssl default
rather than a security-setting call. Out of reach before the settings work and
still is; contorting the arbiter around one pair would be worse than the miss.
The caveat is recorded beside the number: 30 pairs of one slice, and the slice
iterated against all session. The +16% overfitting gap against vfc-js is why
contributor-scan Req 8.1/8.2 exist.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
|
||
|
Marius CiepluchandClaude Opus 5
|
3fe7b9ded7 |
fix(model-grounded-detection): the dominance question contradicted its own evidence
vampi-users-update-password scored both_silent while the arbiter returned model_corroborated on the vulnerable side and nothing on the fixed one -- so the finding existed and the pipeline threw it away. The residual question was written for the ENTAILED case, but entailed returns before the judge is ever consulted, so the question is only ever asked for CORROBORATED. It asserted "in a file where sibling handlers are guarded" and asked about "the guard its siblings have", printed directly above a witness reading "and no guarded sibling" -- because corroborated means precisely that no sibling is guarded. The judge was handed a prompt contradicting its own evidence line and reasonably answered no. The question now describes what corroborated actually is: an obligated operation no guard governs, with no sibling evidence either way, so the model cannot tell from consistency alone whether the guard is missing or the module is deliberately public. That is the real open question and the one worth asking. Verified end to end against a live endpoint: vuln model_corroborated, fixed silent, the pair splits. The taint and config questions were checked for the same defect and are correct -- each describes the corroborated case it is actually asked about. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
db3db6056d |
feat(model-grounded-detection): widen the settings table, wire the weak-algorithm arm
Two things, and a finding that says neither fixes the pairs that prompted them.
The settings table was Flask/FastAPI-shaped (CORS, set_cookie, app.run) while
the corpus contains aiohttp and Django applications, so three config_secrets
pairs contained no modelled security setting at all. Added aiohttp's
session_setup and cookie storages and Django's SECURE_* / DEBUG settings: 8
calls to 24.
And config_secrets spans three abstractions while only one was wired -- a
permissive setting VALUE. Weak algorithm CHOICE (CWE-326/327) is a different
thing: the danger is the algorithm NAMED, not a flag left open. Its data has
been sitting on the sink facts as `weak_literals` since before this feature and
nothing read it. Both abstractions read the same literal, so the arm is small:
hashlib.new("md5") is now entailed and hashlib.new("sha256") is not.
The finding: the three silent config pairs are CORPUS DEFECTS, not model gaps.
Measured similarity 23%, 45% and 53% -- dsvpwa-dsvpwa-main pairs `def main()`
against `def run(self, handler)`, a different function entirely. No settings
table can make those split, and it is worth saying so rather than letting the
work imply otherwise. The table gap was real and worth closing on its own
merits; it just is not why those pairs are silent.
734 passing, mypy/ruff clean, gates byte-identical.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
|
||
|
Marius CiepluchandClaude Opus 5
|
52f2ab5b4d |
fix(model-grounded-detection): ask the residual question the arbiter actually raised
The corrected re-measure returned numbers IDENTICAL to the buggy one, which was the clue. Routing was fixed, the dominance arbiter was reached, and the finding was discarded anyway -- because every corroboration was put to the judge as a TAINT question: "a sanitizer appears on the path: yes, does it suffice?". For a missing authorization guard that question is meaningless. The judge answers no to something that does not apply and the claim is dropped. All three vibe-py access_control pairs scored both_silent this way even though the dominance arbiter had returned model_corroborated for each of them, which I had verified by hand earlier in the session. Fourth instance in this feature of one shape: taint-specific behaviour applied to every family. Sanitizer matching, config routing, family dispatch, now the residual question. Each was exposed by a measurement rather than by a check. The question is now phrased for the arbiter that raised it -- a guard asymmetry for dominance, a permissive setting for constant abstraction, sanitizer sufficiency only for taint. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
7a76fcb028 |
fix(model-grounded-detection): the pipeline arbitrated only flow families
scan_region took a TaintSpec and called taint_verdict, so access_control and config_secrets -- whose dominance and constant-abstraction arbiters have worked since group 4 -- had no pipeline path at all. On the first end-to-end vibe-py run every one of their pairs scored `both_silent`, which reads as the model failing when in fact no code reached the arbiter. This is the same class as the ceiling harness routing config_secrets through taint: family routing fixed in one place and not carried to the other. Third time this shape has appeared in this feature, which is worth saying plainly. Dispatch is on the spec TYPE rather than a family name, so a family without an arbiter cannot be silently routed to the wrong one -- it simply has no spec and is skipped, visibly. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
cdedb372dd |
feat(model-grounded-detection): wire the detector -- the enumerator proposes, the model disposes
Everything before this built an ARBITER: something that judges a flow taint
reachability had already found unaided. Nothing proposed to it. That is why its
recall was never comparable to a detector's, and why comparing 41.3% entailment
against the LLM hunter's 76.7% pair-correct was comparing two different kinds
of thing. model/judge.py and enumerate_candidates were called by no production
code at all; every number so far came from a measurement harness.
The wiring turns on a distinction judge() alone conflated:
the model CONTRADICTS it has coverage at this site and says no -- a
sanitizer on the path, or a declared-safe sink shape.
The claim is dropped (Req 8.2).
the model CANNOT DECIDE no modelled sink or source reaches the site at all.
The model is SILENT, not negative, so the claim is
reported at `suspicion` (Req 8.4).
Treating silence as refutation is the tempting mistake and would be wrong here:
23 of the 39 misses in the entailment ceiling were sinks simply absent from the
fact tables, so "no flow" usually means the model cannot see it rather than
that nothing is there. The rung carries that uncertainty on the finding's face
instead of a confidence score nobody can act on.
An entailed site still costs zero model calls. The candidate budget is bounded
so a dense region cannot become an unbounded spend.
Smoke-tested end to end against a live endpoint: 2 pairs, detection 100%,
$0.0024. The full vibe-py measurement -- the first like-for-like comparison
with the hunter's 76.7% pair-correct and 12.3% leak -- is running.
727 passing, mypy/ruff clean, zero orphaned modules.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
|
||
|
Marius CiepluchandClaude Opus 5
|
1af3db37f6 |
feat(corpus): vendor 40 OWASP xss pairs as the owasp slice
output_encoding had exactly ONE pair in the whole corpus. It now has 41. 238 pairs were harvested from OWASP BenchmarkJava (2740 generated servlets, GPL-2.0, with a ground-truth CSV) by matching each really-vulnerable test to the safe test of the same category it most resembles -- adjacent tests come from one template, so they differ in the sink or the sanitiser rather than being unrelated programs. Median similarity 0.91, against the 64% of the older `sast` OWASP entries that the group-4 stratification flagged as different programs. Only 40 are vendored. One representative of each of the seven categories was run through the model BEFORE vendoring, and only xss distinguished: its fix adds ESAPI encoding, which the Java sanitizer facts recognise, so the vulnerable side entails and the fixed side drops to corroborated. cmdi, sqli, ldapi and xpathi put the sink in a helper class outside the servlet and resolved no flow at all; pathtraver found the flow but its "safe" twin carries no distinguishing call, so text similarity had matched a twin that is not semantically the fix -- a subtler form of the very defect the harvest was designed to avoid. Those 198 pairs are deliberately left out: adding pairs the model cannot read would inflate the corpus while making every number less interpretable. Evidence for the vendored slice is 3 of 3 distinguished, which is a small sample and is stated as such in the catalog header rather than implied to be more. The slice is marked SYNTHETIC in its header: it measures breadth across a weakness class and must never stand in for real-world performance. The corpus already shows a +16% overfitting gap, and a large generated slice blended into a headline would widen it invisibly. Two things recorded rather than hidden: BenchmarkJava's LICENSE is GPL-2.0 while the older sast entries declare GPL-3.0 for OWASP, which looks like an error and is not copied; and OWASP-Benchmark/BenchmarkJava already straddled the train/holdout split before these pairs (owasp-java-cmdi train, owasp-java-sqli holdout), so the new all-holdout pairs enlarge an existing straddle rather than create one. The two vendored-count guards did their job -- a corpus change could not land silently -- and are updated as the acknowledgement. Hygiene artifact regenerated: 277 -> 317 vendored pairs. 720 passing, mypy/ruff clean, gates byte-identical. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
0c71c956fd |
feat(model-grounded-detection): group 6, the execution tier as a seam not a build
The rung this project deliberately does not build. Clearwing already runs sandbox lifecycles, container pools, sanitizer images, PoC replay and stability classification, and the corrective brief concluded -- from this project's own defect history -- that reimplementing that machinery is where we go wrong. So model/execution.py decides eligibility, calls an adopted tool, and records what happened. A test asserts structurally that it imports no container or subprocess machinery and defines no build/run/compile/patch function, because the failure mode to prevent is this file quietly growing a sandbox. Eligibility is narrow in both directions: only a family a static model cannot arbitrate (memory safety in C, where the bug turns on runtime layout -- Req 10.3 forbids any web/logic family depending on this tier, and each now has taint, dominance or constant abstraction instead), and only a finding the model left undecided. The tier escalates uncertainty; it never re-checks a verdict. One deliberate asymmetry: a failure to reproduce is NOT a refutation. A sandbox that does not trigger a bug has not shown the bug is absent, only that this attempt did not trigger it, so the candidate stays where the model left it. Reading a failed reproduction as a clean bill of health is the same error class as reading an absent engine as "no vulnerability". With no adopted tool present -- the normal case -- the rung is never reached and clearwing_unavailable is recorded. All 16 tasks of model-grounded-detection are now done. 720 passing, mypy/ruff clean, gates byte-identical, zero orphaned modules. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
f1c341df6c |
feat(model-grounded-detection): group 5, corpus calibration and honest reporting
5.1 inverts what the corpus is for. Previously a pair the detector missed was a
DETECTOR failure, to be averaged away over K runs and chased with a prompt
change. The arbiter is deterministic: it will miss that pair the same way every
time, and the only thing that moves it is better modelling. So a pair the model
cannot split is a NAMED GAP IN THE MODEL, recorded with its reason, and every
gap carries one even when the caller supplies none -- a bare gap teaches
nothing.
`covered` means the model reached a strictly higher rung on the vulnerable side
than on its fix. Flagging both sides equally is not coverage: it distinguishes
nothing, which is exactly what group 2 measured taint reachability doing on
every parameterised SQL pair. The harvest seam takes the fix commit and derives
the vulnerable side as its parent, with the fix diff as the oracle -- nothing
hand-labelled.
5.2 keeps three disciplines from the deleted publish layer, each because its
absence caused a real error here:
* per-slice rows always -- `render` RAISES without them. A mixed headline
once hid agent-written code scoring 44% against the development slice's 83%.
* the overfitting gap beside the headline, and ABSENT rather than zero when a
slice is missing: reading an unmeasured slice as "no overfitting" is the
flattering failure, not a neutral one.
* the rung on every finding, so a suspicion cannot be read as a result.
Run end to end against the committed ceiling artifact, the report says things
worth not hiding: agent-vfc 0/28 covered, vibe-py 40% against vfc-js 24%, an
overfitting gap of +16%, and gaps grouped by family with their reasons --
injection 32 (21 no verdict, 11 distinguished nothing), path 8 (all no
verdict), untrusted_destination 5 (4 entailed on BOTH sides).
713 passing, mypy/ruff clean, gates byte-identical, zero orphaned modules.
A full re-measure with the closure-scoping fix is in flight; these numbers are
the pre-fix baseline and will move.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
|
||
|
Marius CiepluchandClaude Opus 5
|
d651455d23 |
fix(model-grounded-detection): closure scoping, and the three bugs it uncovered
The group-4 finding: JavaScript is callback-heavy, so the method containing a
sink routinely is not the labeled function. `fs.readFileSync` inside the
callback passed to `fs.stat` has enclosing method `<lambda>0`, and filtering on
`sinkMethod == function` dropped it -- the whole path family measured 0/8.
Scope is now decided by line-range containment within the same file
(astParentFullName is empty for lambdas, so nesting is not readable there), and
reported as `inLabeledScope`; the Python side trusts that flag and keeps the
name comparison only as a fallback, so an engine that cannot report scope never
has its sinks silently treated as scoped. Parameter sources stay bound to the
labeled function's OWN parameters -- a callback's `err`/`stats` are not
attacker input.
Fixing the scope exposed three further bugs, each of which would have raised
the number while making the answer untrue:
* Shortest-flow-wins picked the wrong witness. With closures in scope the
same sink has flows from `req.url...` (10 steps, the vulnerability), `res`
(3) and `this` (7). A flow whose source matches the source model now
outranks a shorter one from a bare captured name; shortest still breaks
ties, so the order stays total and the verdict deterministic.
* The JS fact file had NO sanitizers at all, so no JavaScript fix of any
family could ever be recognised. Added the documented path-normalisation
and HTML-escaping APIs, with provenance.
* Adding them regressed the witness, because `resolveUrl` CONTAINS `resolve`
and the substring test marked the vulnerable flow sanitized. This is the
same bug class already fixed in dominance.sc and not carried across -- the
exact fix-the-instance-miss-the-class failure. taint.sc now matches on word
boundaries too.
And the principled rule the fix turns on: parameter sources are a FALLBACK,
never an override. When a modelled source reaches the sink, those flows are the
evidence; otherwise a captured `res` reaching a sanitized sink reports the fix
as still vulnerable on a flow that names no attacker input.
Also recorded: req.url was NOT wrong as a diagnosis, it was untestable in
isolation -- the scoping bug masked it. Both were needed. On the pair that
motivated all of this:
vuln model_entailed req.url.split('?')[0] -> readFileSync(possibleFilename)
fixed model_corroborated same flow, sanitized by path.normalize
distinguished: true
700 passing, mypy/ruff clean, gates byte-identical.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
|
||
|
Marius CiepluchandClaude Opus 5
|
eebca3f663 |
fix(model-grounded-detection): the absence arbiter was silencing its own family
Two bugs, together making 4.1 report "no bug" on a textbook IDOR. Both found by running it against a real corpus pair rather than a fixture. 1. constraint_params were treated as dischargers. A field named `user_id` is what an IDOR is MADE OF -- `filter_by(id=user_id)` with user_id off the request is the bug, not the fix. The deleted checker only counted an identity constraint whose value's PROVENANCE was the authenticated context, so the parameter name alone evidences nothing. This is the mirror of group 2's sanitizer bug: a thing that must never count as a discharge, counted. 2. Guard tokens matched as substrings. With `user` in the token list, the string "SELECT * FROM users WHERE id = ?" contains `user`, so the query reported every operation in the handler as guarded. Matching is word-boundary now. On threatbyte-api-v1-delete the arbiter previously found three operations, all "guarded" by evidence like '"SELECT * FROM users WHERE id = ?"' and '(user_id,)', and returned no verdict at all. After the fix, on the real pairs: threatbyte-api-v1-delete vuln=model_corroborated fixed=none vampi-users-update-password vuln=model_corroborated fixed=none Both distinguished, deterministically, with no model call. Corroborated rather than entailed is right for these: they are single-handler excerpts with no guarded sibling to be inconsistent with, which is the asymmetry the rung distinction exists to express. identity_sources still discharge; request_sources and permissive_values remain excluded because they describe the vulnerable shape rather than the fix. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
d9ceeb6a09 |
feat(model-grounded-detection): 4.2 constant abstraction, the third arbiter form
A configuration bug has no flow to follow and no guard to dominate.
CORS(app, origins="*") is dangerous because of the value it is SET TO, so the
arbiter evaluates the setting's arguments against the closed permissive set in
the obligation facts -- the third form beside taint reachability and guard
dominance, and the one Req 6.3 names.
ENTAILED a security setting is passed a literal in the permissive set
CORROBORATED the argument is computed, so constant abstraction cannot decide
it -- that residual is the judge's, never silently "safe"
(none) every literal is outside the set, or the call is not a setting
Two Joern node-shape bugs, both found by running it and both of the same class
as group 2's receiver bug -- the graph's indexing is not what the source text
suggests:
* a keyword argument (`origins="*"`) is an assignment-shaped node, so
isLiteral is false on the argument itself and the literal sits in its
subtree; literals are collected from the argument's AST now.
* Joern indexes the receiver at 0, positional arguments from 1, and NAMED
arguments at -1. Settings are configured overwhelmingly by keyword, so
argumentIndexGt(0) -- correct when counting arity for the taint shape test
-- dropped exactly the arguments that matter and every keyword-configured
setting read as "computed".
Live, deterministic, zero model calls:
create_app -> model_entailed CORS(app, origins = "*"): permissive literal "*"
safe_app -> no finding (origins = "https://app.example")
All three arbiter forms now work against a real CPG: taint reachability for the
flow families, guard dominance for absence, constant abstraction for config.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
|
||
|
Marius CiepluchandClaude Opus 5
|
fec67253f3 |
feat(model-grounded-detection): 4.1 guard dominance, 4.2 the missing sink facts
4.1 is the arbiter for the class no crash oracle reaches. An access-control bug
is the ABSENCE of a guard: no flow to follow, no crash to reproduce. What the
model can establish is a consistency violation -- an obligated operation no
discharging guard governs, in a file where its siblings are governed by one.
"Governed" turned out not to be one relation, and asking pure CFG dominance of
the operation gets two of three shapes wrong. Measured against the real engine:
identity constraint the guard is INSIDE the operation's arguments
filter_by(owner_id = current_user.id)
identity branch abort(403).controlledBy == [note.owner_id != current_user.id]
-- the check runs AFTER the fetch, so it never dominates
the operation, but it controls whether the value escapes
dominating guard a discharger-naming call that dominates the operation
That is why the query uses controlledBy rather than dominatedBy for shape 2.
Two bugs found by running it, both the same class as the sanitizer bug in group
2 -- part of a fact flattened, the part carrying the signal dropped:
* dominance_specs used only `calls` and `decorators`, omitting
identity_sources (current_user, request.user) and constraint_params
(owner_id, user_id) -- which is how an identity constraint is actually
written. The arbiter reported "no guard" on handlers that plainly had one.
request_sources and permissive_values stay excluded: they describe the
vulnerable shape, and including them would let the arbiter read a bug as
its own fix.
* matching operations by code containment yielded four or five desugared rows
per operation (the bare `tmp1.filter_by` receiver among them, carrying none
of the guards the real call has), so a guarded operation could be flagged.
Matching is by call name now.
Live, deterministic, zero model calls:
leaky -> model_entailed no discharging guard, while 2 sibling handlers are guarded
edit_note -> no finding (ownership check controls the denial)
list_notes -> no finding (identity constraint in the query)
4.2 authors the sink facts the go/no-go measured as its largest miss class (11
of 23): Django ORM raw SQL, and the four families that previously had no fact
routing to them by CWE at all -- path, output_encoding, untrusted_destination,
prototype. Each entry is a documented dangerous API of the named library with
its provenance recorded inline, and rule_ids left empty because a sink fact is
not a detection rule. Flagged for maintainer review as data, given the standing
"no LLM-authored patterns" constraint. The exact-set assertion written in task
1.1 to hold this gap open is updated rather than deleted, so it still fails if a
family ever loses its seed.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
|
||
|
Marius CiepluchandClaude Opus 5
|
b8f8f727c6 |
feat(model-grounded-detection): group 3, the evidence ladder and the judge
3.1 adds StaticFinding.rung beside evidence_level rather than renaming it. The existing evidence_level carries `static_corroboration`, emitted by the flat IR overlay -- a path Req 4.4 demotes to the suspicion band. Restating that as model_corroborated would claim a CPG arbitrated something it never saw, and rewriting every legacy finding to suspicion would be a product-visible regression nobody asked for. So `rung` defaults to suspicion and is raised only by ladder.at_rung(finding, verdict), which is the structural form of Req 5.4: nothing rises above suspicion without a Verdict behind it. The witness travels with the rung, because a rung without its evidence is just a louder assertion. It reaches markdown, JSON and SARIF. 3.2 is the judge, and it is defined as much by what it lacks. Three exhaustive outcomes: the model entails (report it, never call the model), the model contradicts (drop the claim, record why), the model cannot decide (report suspicion). The LLM gets one bounded typed question with tools=[] so it cannot search for the site, asked only about the residual the graph cannot settle -- whether a sanitizer on a REAL flow suffices. Its answer can only confirm a flow the CPG already found; it can never conjure one. Req 8.5 is asserted structurally: a test parses judge.py and fails on any AST name like k_runs/majority/vote/average/acceptance/floor/budget, any for/while loop, or an import from the deleted round machinery. That test first matched the word "votes" in the module's own docstring, which is why it now inspects code rather than text. Verified live against a real CPG, not only scripted rows: vuln rung=model_entailed request.args["name"] -> db.execute(...) fixed rung=suspicion no source reaches this sink model calls made: 0 Both sides arbitrated by the graph with the LLM never consulted -- the thing the prior architecture could not do at any price. 678 passing, mypy/ruff clean, gates byte-identical, zero orphaned modules. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
e127e430ed |
feat(model-grounded-detection): parameter sources, so the measurement is like-for-like
Only 11 of the 50 injection pairs contain a framework source token at all. The
other 39 receive untrusted input as a FUNCTION PARAMETER, because SVEN-style
pairs are function-level extractions:
def add_user(username, password):
os.system("useradd ... -p " + encPass + " " + username)
A framework-anchored source model finds nothing there, which caps entailment on
this slice near 22% by construction -- a property of the source model, not of
the engine. More to the point, the flat-IR baseline this feature is measured
against already counted these (source_kinds: ["parameter"]), so a run without
them is not comparable to the 2.2% it is supposed to beat.
The flag is opt-in and off by default, because the two settings are right in
different places: in a function-level pair the function boundary IS the trust
boundary, while across a whole repository most parameters carry internal
values and treating them all as untrusted is not sound. The corpus harness
opts in; a repository scan does not.
On the pair that exposed this the model now resolves both flows, including one
through an intermediate binding:
username -> os.system(...) 2 steps
password -> crypt.crypt(...) -> encPass -> ... 7 steps
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
|
||
|
Marius CiepluchandClaude Opus 5
|
1765e51da1 |
feat(model-grounded-detection): the safe-shape test, without which the model arbitrates nothing
The first live run produced the finding that shapes this design: taint
reachability alone does not distinguish a fix from a bug. On the canonical SQL
pair the flow request.args["name"] -> name -> execute exists on BOTH sides --
vuln db.execute("select ... '" + name + "'") one argument, interpolated
fixed db.execute("select ... = %s", (name,)) two arguments, bound
-- so the model entailed the fixed twin exactly as readily as the vulnerable
one, and arbitrated nothing. What separates them is the SHAPE of the sink call,
which is precisely what safe_shape_sinks preserved from the earlier sanitizer
fix. The query now reports sinkArity and sinkArg0Literal, and a flow ending at
a sink written in its declared-safe form is not a vulnerability.
Two bugs in that shape test, both found only against the real engine:
Joern's `call.argument` includes the RECEIVER at argumentIndex 0, so `db` was
counted as an argument -- making a one-argument interpolated call look like a
two-argument bound one, the exact inversion of the test -- and the "first
argument" whose literalness decides a format-string sink was the receiver
rather than the format. Both fixed with argumentIndexGt(0).
The rule is narrow on purpose: only sinks named in safe_shape_sinks are subject
to it (os.system(x) is not safe merely because it takes one argument), and a
row carrying no shape fields is judged on the flow alone -- an unknown shape
must never be read as a safe one.
End to end on the pair, deterministic and with no model call:
vuln -> model_entailed request.args["name"] -> db.execute(...) (line 4)
fixed -> None
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
|
||
|
Marius CiepluchandClaude Opus 5
|
f59e0f5b5a |
fix(model-grounded-detection): two modelling bugs the first live Joern run exposed
Both were found by running the real engine, and neither was visible offline.
1. A sink was its own sanitizer. The shipped facts mark some sanitizers with a
*shape* qualifier -- parameterized ("execute(sql, params) binds rather than
interpolates") and literal_format_arg ("printf with a constant format string
is safe"). These describe a safe FORM OF THE SINK CALL, not a call that
cleans a value on the path. Flattening them into the sanitizer list made
`execute` both the injection sink and its own sanitizer, so every SQL flow
would have reported as already-clean and nothing could ever be entailed.
Fixed as a class, not an instance: any fact carrying a shape qualifier is
excluded from `sanitizers` and kept in `safe_shape_sinks` for the shape test
task 4.2 will model properly. Fixing only the Python `parameterized` case
would have left C's entire printf family -- which the generalised test then
caught immediately.
2. labeled_family lost its deterministic derivation in group 1. The deleted
classifier resolved a pair's family by declared label, else obligation, else
mechanism, else CWE, and only asked an LLM when every link failed -- which
for a labeled corpus was never. Group 1 reduced this to the declared label
alone, which silently emptied every family whose pairs are labeled by CWE:
the injection slice went to zero pairs. The chain is restored, still with no
model call, and the family census again matches the committed entailment
ceiling exactly (injection 50, memory 176, access_control 8, ...).
Also: the Joern seam now runs in the CPG's own scratch directory, because
`joern --script` writes a `workspace/` beside the working directory and would
otherwise leave one inside the repository being analysed.
The taint query itself was corrected against the live engine: the sink is the
call's ARGUMENTS (data flows into arguments, not into the call node) and
reachableByFlows is called on the sink with the source. With that, the
three-hop chain our flat IR could not resolve is found:
request.args["cmd"] -> cmd -> "echo " + cmd -> full -> os.system(full)
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
|
||
|
Marius CiepluchandClaude Opus 5
|
2b973dc0db |
feat(model-grounded-detection): tasks 2.1-2.2, the Joern seam and the taint verdict
The whole engine integration is one subprocess boundary: joern-parse builds a cpg.bin, joern --script runs a shipped CPGQL query that prints its JSON between fence markers, because Joern's stdout carries a banner and a REPL prompt around whatever a script emits. Nothing is imported in-process, so the core install keeps dependencies = [] -- a test walks the seam's AST to assert it imports only the standard library and names no JVM binding. Fail closed everywhere. A build or query that errors, times out, or prints something unparseable yields no verdict and the caller reports suspicion. The taint verdict distinguishes this from absence deliberately: a query returning None means "the engine could not decide", which is not "there is no flow", and conflating them is how a missing engine silently becomes a clean bill of health. The verdict is where the ladder earns its keep. An unsanitized source->sink path is model_entailed -- the graph establishes it and the LLM's claim is no longer load-bearing. A path a sanitizer may break is model_corroborated: the flow is real, but whether the sanitizer suffices is the semantic residual the LLM answers, checked against this verdict rather than averaged over runs. The chosen flow is picked by a total order (shortest, then lexicographic), so two runs over one CPG cannot disagree -- Req 6.4 asserted directly. The query itself is the part our flat IR could not do: reachableByFlows walks the PDG, so `q = "..." + request.args["x"]; execute(q)` is found, which is exactly the one-hop limit that held measured entailment at 2.2%. Capability detection reuses semantic/engines.joern_available rather than growing a second notion of "installed". The audit manifest is regenerated; the new modules are named by Req 4/5/6 rather than counted as orphans. Task 2.3, the go/no-go measurement, needs Joern installed and has not run. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
9ed0e45c5f |
feat(model-grounded-detection): group 1, subtract the noise architecture
99 source modules -> 81. 832 tests -> 646. The whole learning/ package (16 modules, ~4,100 lines) and six semantic/ modules removed, with the three gates byte-identical to 2026-09-06-gate-baseline.txt at every step. The sequence is what made it safe: vocabularies ported first (1.1), importers detached second (1.2-1.3), deletion third (1.4). Every import break during the deletion was therefore a detachment miss in the importer and was fixed there -- no module was restored to satisfy one. Both misses were the same shape and were fixed as a class: operations.py and shapes.py each imported trailing_name from the deleted variants.py, so it moved to semantic/ir.py, its natural home, and both importers changed in one edit. Ported before deleting: the closed guard vocabulary (verbatim, nothing else holds it) into model/specs.py, which derives TaintSpec and DominanceSpec from the retained ruleset facts rather than copying them. Re-homed: families -> model/taxonomy, candidates -> model/candidates, endpoint -> model/endpoint, plus Region, labeled_family and labeled_spans. labeled_family no longer falls back to the LLM classifier -- an unlabeled pair is honestly unknown. Four deviations, each recorded in the spec notes with its reason. improve/ evolve.py and validator.py were stripped, not deleted: each mixes the mechanism lever (gone) with the deterministic rule-status loop `ousast improve` still runs, which has no LLM in it; a test now asserts the lever's absence by symbol so it cannot creep back. The learning CLI group became `model candidates`, because deleting it outright would have orphaned the retained enumerator by removing its only entry point. thinking moved from [learning] to [models]. A gap named rather than papered over: no retained sink fact routes by CWE to path, output_encoding, untrusted_destination or prototype -- the same hole the entailment ceiling diagnosed. The test asserts the seeded set and the gap set exactly, so task 4.2 must update it when it authors those sinks. The audit found exactly one genuine orphan: vectorstore.py, a bakeoff harness with no production caller and no CLI, alive only through its own test. The five named subsystems each carry an explicit decision. Manifest committed, with a test asserting the tree still matches it. Group 2 (the Joern go/no-go) has not started. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
3ebcc6729a |
fix: a body that stops mid-read is transient, and a round that dies still says so
Round 5 died on `http.client.IncompleteRead(0 bytes read)` and left no record at all. Two defects, both ours. The retry classified only connection-level failures. IncompleteRead and RemoteDisconnected arrive while the *body* is being read, after the connection succeeded, so they are not URLError -- they escaped both the retry and the error wrapper and walked out of the round. The network is allowed to be unreliable; losing a paid round to it is not the provider's defect. And the exception path was wrapped around the stages that come *after* the proposal is applied, because those are the ones with something to revert. The stages before it had already spent money, and a round that leaves no record is a round nobody can account for. Every outcome is journalled now, with the exception type in the reason. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
cde57e4293 |
fix(learning-harness): say why the proposer had nothing, and stop hard-coding its vendor
The meta-agent round journalled `no_proposal`. The real cause was an authentication failure: `_model_config` hard-coded the anthropic provider while the detector was talking to DeepSeek and no ANTHROPIC_API_KEY exists here. The composition itself was fine -- MetaAgent constructed, evolve started, budget $5 -- and the trace was thrown away. `no_proposal` covered two different worlds: a proposer that considered the facts and declined, and one that could not start at all. The journal now carries the proposer's own reason, and the vendor comes from `[harnessx] provider` instead of being assumed. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
84b1ba7a72 |
fix(learning-harness): the published table lost a whole slice
`publish` keyed baselines by model alone, so the vibe-py run silently replaced the agent-vfc run and the table showed hand-written teaching code as if it were the corpus. That is precisely the failure the constrained-detector requirements exist to prevent, live in a published artifact. Third time today that a key was fixed in one place and not another -- `compare_baselines` got the model+slice label and `_baselines` did not, the same way the exception path got checkpointing and the timeout path did not. The table now carries a slice column and one row per slice, and the artifact is keyed by what identifies a run. Republished: agent-vfc's six families are back, including access_control at 0.000 recall on four pairs, which is the number that was missing. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
506bbe5f5c |
fix(learning-harness): round zero writes down every pair it has paid for
The 176-pair vfc baseline hit its four-hour timeout and wrote nothing. Scores were accumulated in memory and persisted once at the end, so roughly four hours of paid API calls were lost. I fixed the exception path this morning -- a provider hiccup costs one pair -- and missed that a timeout, a kill or a machine going away cost all of them. Each pair is now appended to `scores.jsonl` and flushed as it is measured, and `--resume` keeps what a previous run of that exact baseline already bought. This also gives the per-pair outcomes the round-3 review asked for, which make the noise floor auditable after the fact instead of only in aggregate. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
78b4f33a8e |
feat(constrained-detector): the ceiling, measured offline across every slice
Group 1 complete. Tasks 1.2 and 1.3: `candidates.ceiling`, `ousast learning
candidates`, and the stop-or-go reading. No model is called anywhere in this
group, so the condition that could have ended the feature was reachable for
free.
Verdict: go.
agent-vfc (deciding) 89.7% ceiling against 44.4% round-zero detection
vfc (deciding) 84.7%, baseline still running, non-gating
vfc-js (deciding) 52.9% -- see below
github (deciding) 100.0%
vibe-py (development)100.0% against 83.3%
The margin is largest on agent-vfc, which is the slice the tool is actually
for.
The measurement forced a shape correction before the detector existed.
config_secrets first read 63.6% with a bind-and-literal shape, because a
permissive default is as often an argument to a call -- cors({origin: '*'}),
app.use(session({secure: false})) -- as it is a binding. Adding call sites to
that family's shape took it to 100% and recovered four agent-vfc rows.
Carry-forward, and not this feature's to fix: vfc-js sits at 52.9% because
eight of seventeen rows label a function the JavaScript CST names `<anon>` --
`module.exports.publish = function () {}` and its relatives. That is a
function-naming gap, it caps the existing hunter on that slice exactly as
much, and it is the same class as the `_declarator_start` defects found in
learning-harness task 4.1.
The ruleset arm of `ceiling` is kept so the 12.1% that rejected it stays a
measurement rather than a remembered number.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
|
||
|
Marius CiepluchandClaude Opus 5
|
8d771901bf |
feat(constrained-detector): candidates from the semantic IR, shaped by family
Task 1.1. The detector stops searching: something deterministic decides what
it looks at, ordered by (path, line, name) and bounded by a constant, so the
batches a later task forms are determined by the input alone.
A call candidate carries what the model used to grep for -- the arguments,
and the binding chain the IR resolved for them, nearest first. An access
control candidate is the operation with the guards in scope, because an
absence bug has no sink.
Two things the fixture taught while writing it. The IR emits a chained call
twice at one line, once bare and once carrying the keyword arguments, so
deduplication keeps the richest: a bare `filter_by` and
`filter_by(user_id=owner)` are the same site and only one of them says what
the constraint was. And `leaky` and `constrained` in the shared obligations
fixture both discharge through `filter_by` -- what separates them is that one
chain ends at `request.args.get('owner')` and the other at `request.user.id`.
That contrast is now the test, and it is exactly the judgement the model is
being narrowed down to make.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
|
||
|
Marius CiepluchandClaude Opus 5
|
309faad266 |
measure: the candidate generator must be the IR, not the ruleset
A candidate-driven detector can only judge sites something else enumerates,
so the enumerator caps recall. Measured over the 58 scorable web pairs before
building anything:
pattern ruleset: 12.1% ceiling -- and the misses are not near-misses, the
ruleset finds nothing at all in most of these files
semantic IR: 96.6% ceiling, a median of ten call sites and bindings
inside the labeled function
The ruleset would have capped us at 12.1% against the 64.9% the unconstrained
loop already reaches. That design was one commit from being built.
Two corpus defects found on the way, both mine from task 4.1. The overleaf
excerpt started mid-import, because `_declarator_start` walks back to the
nearest `}` and Overleaf writes JavaScript without semicolons, so the nearest
one was inside `import { expressify }`. It then ended mid-object-literal,
because registration mentions were appended as bare lines. Neither parsed,
and a pair that cannot be parsed cannot be scored. Registration mentions now
expand to the statement that contains them; the ceiling moved 94.8% -> 96.6%.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
|
||
|
Marius CiepluchandClaude Opus 5
|
b57b1a35d8 |
feat(learning-harness): thinking is a measured condition, and every round
records whether its decision beat noise Two things the maintainer was right about. Thinking was disabled so `temperature: 0` would not be silently ignored -- a trade that only pays if it buys determinism. Round zero says it did not: 11 of 20 injection pairs disagreed with themselves across five runs, so the variance is in the agentic path, not the decoder. We paid the reasoning cost on a reasoning model and got nothing. It is now `[learning] thinking`, recorded in the artifact and part of its key so the two modes cannot be read as one number. The default stays off only because that is what round zero was measured under. And the deeper point: the spec built `sign_test` and `reliable_change` and the acceptance rule calls neither -- `decide` compares raw deltas. On a detector with that floor a raw delta is mostly noise, and a rule that brittle is exactly what makes a deterministic decoder look necessary. It is not; the alternative is an acceptance rule that models the score as noisy, the way reflective-evolution and Bayesian prompt optimizers do. Changing Req 9.5 is a maintainer's call, so this does not change the decision: every round now journals the two-sided sign test over the pairs that moved, and after four rounds the journal will say whether any decision could be told from a coin flip. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
1e4f212868 |
fix(learning-harness): logprobs are on the choice, not the message
OpenAI-shaped providers put `logprobs` beside the message rather than inside it, so the confidence signal the classifier is meant to abstain on was always None -- a knob that reads 'never unsure' whatever the model said. Measured against DeepSeek while checking: it returns the structure but not the values. Every chosen token comes back at logprob 0.0 and every alternative at -9999.0, on a deliberately uncertain prompt as much as a certain one. The field is there; the information is not. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
ce89b0dbe0 |
fix(learning-harness): a slice is part of what identifies a baseline
Keyed by model alone, the memory-safety run over the vfc slice silently replaced the run over the web families, and the published table would have shown one of them as if it were everything. The artifact key and the comparison label now carry the slice selection when it is not the default. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
c1ef3ca539 |
fix(learning-harness): a provider hiccup costs one pair, not the whole run
Round zero is the most expensive thing this harness does. An error two thirds of the way through raised straight out of `run_baseline` and took everything already measured and paid for with it, so the only honest response to a flaky endpoint was to run the whole thing again and hope. The pair is now recorded as unscorable with the reason `detector_unreachable` and the run finishes. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
d2ebb58007 |
fix(learning-harness): a hunt that ends in prose is asked to format it
Measured live against DeepSeek on one pair: the model spends its step budget on tools and then answers in a paragraph that names the sink, the argument and the line -- which parses as nothing. A detector that found the bug and reported zero is indistinguishable from one that found nothing, and every family in round zero would have baselined at zero for this reason alone. The loop now takes one more turn when the hunt ends in prose or runs out of steps, asking the provider for JSON and handing back the reasoning the model already produced. It formats an answer that was reached; it never asks for a new one, and a hunt that called no tool never gets there. Same pair, same model, before and after: recall 0.00 -> 1.00, Youden 0.00 -> 1.00 over three runs per side. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
a1e7b1a20f |
fix(learning-harness): the run reads the meter it publishes
The chat client meters itself and nothing passed the meter to the baseline or the round, so every published cost would have been 0.00 -- and the round's cost cap, which reads the same meter, could never have fired. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
32f16f78ff |
fix(learning-harness): a baseline over a named slice measures that slice
`--slice vfc` selected the vfc rows and `run_baseline` filtered them all out again against its own default of the two web slices, so the memory-safety baseline Req 8.4 asks for produced an empty report and said nothing about it. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
2dad25d312 |
fix(learning-harness): the round-4 review findings
Three defects, two of them the same shape as ones already rejected once: - The meta-agent proposer still could not propose. `MetaAgent(inner_model=)` takes a ModelConfig; it was handed a model name, a `# type: ignore` hid that from mypy, and a bare `except Exception` turned the resulting AttributeError into `meta_agent_failed` -- which reads exactly like the model declining. The composition is now a seam a test can pin, and a construction error is re-raised rather than reported as a quiet round. - `ousast pairs --hunter` crashed in its documented form: the K floor landed with `--k-runs` still defaulting to 1. The covering test always passed `--k-runs 4`, which is why nothing noticed. - 28 rows (12% of the corpus) resolved a span on the vulnerable side and none on the fixed side, so every fixed-side finding read as outside the labeled function and the pair scored as silence whatever the detector said. `fix_function` now reaches the catalog and the scorer, and the generator derives it from the fixed excerpt for the 25 trap rows that never declared one. Measured after: 0 rows. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
75ae9ee47b |
test(learning-harness): the gate baseline and the degradation matrix
The three gate outputs are captured at
|
||
|
Marius CiepluchandClaude Opus 5
|
3e71e9e88b |
feat(learning-harness): fix dates and the post-cutoff slice
The catalog generator fills `fix_date` from the recipe at the precision the recipe has -- a full ISO day when one was recorded, otherwise the year. Every harvested recipe carries a year and almost none carries a day, and writing `-01-01` would turn a year into a claim about a month. A row counts as post-cutoff only when its earliest possible date is after the cutoff, so a coarse date can under-claim the clean slice but never over-claim it. Each family now reports the same numbers over the rows fixed after the cutoff, beside the whole number and never instead of it, with the undated count on its own line. Without a configured cutoff every scoreboard says so rather than inventing one: guessing a model's training cutoff and then publishing a contamination number from the guess would be worse than reporting none. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
d1faa04771 |
feat(learning-harness): declared twins and a fixture-difficulty check
The five byte-identical vibe-py twins now say so in the catalog instead of only hashing to it, and a declared limit that names one of the unscorable vocabulary's own reasons keeps that spelling, so `identical_twin` stays one fact. Leave-one-out over vibe-py consequently runs on 30 rows, not 35. The difficulty check measures three axes the corpus varies on: binding hops from the identity source to the constrained use, registration distance, and guard distance. Measured first, and it failed: the shared obligations fixture reached (1, 0, 2) where the vibe-py absence rows reach (5, 2, 2). Every fixture handler carried its own route decorator, so nothing in the suite exercised a handler registered from another document and no binding chain was longer than one hop -- the checker was passing on a simpler world than the one it meets. `from_spec` closes that with the connexion shape those rows really have. The verdict is committed beside the numbers. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
1843ef9a02 |
fix(learning-harness): the round-3 review findings, remediated RED-first
Eight of nine group-3 tasks were rejected. Each defect was hiding a number, and the numbers are why the loop was not learning: - The fixed side was judged against the *vulnerable* side's function ranges. 191 of 227 pairs with a resolvable labeled span (84%) have different spans on the two sides, so a leak was recorded as silence, always in the direction that flatters Youden. - Round zero measured every family with the generalist. `family:unknown` earns no credit against any real family, so every family but `unknown` baselined at a structural zero and every noise floor with it. - The cross-family sweep counted absolute failures instead of negative flips. On a corpus where most pairs fail at round zero that rejects every round for damage it did not do. Unscorable rows voted too. - K runs was declared and never threaded; `pairs --k-runs` was parsed and discarded, and its only test grepped `--help`. - The meta-agent proposer was a stub, so `learning round` could never propose anything. It now composes MetaAgent.evolve against a scratch copy of the family directory, and `--proposals FILE` runs a recorded sequence offline. - `refused_holdout` was a vocabulary word no code path could produce, and the covering test accepted either outcome. - Cost was metered once, after everything had been paid for; a stage that raised left no record at all. - The corrected lever number was published beside the earlier one rather than in place of it, and the test named for that property never checked. - Refusals were `continue` statements: `Refusal.degradation()` existed and nothing called it. A train-split identical twin was reported as `split:train`, which is untrue. Test repairs that each changed a number: two split tests measured an `x = 1` fixture that teaches nothing whatever the rule says; two CLI scoreboard tests asserted against all-zero output produced by a stub that answers twice and puts its finding outside the labeled function. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
c2d9c4ad01 |
feat(learning-harness): absence rows travel with their registration
Eight of the ten vendored access-control rows had no named handler at all, so every one of them would have scored `missing_context` forever rather than as a detector miss. After the re-harvest nine are named and the tenth states its reason. The handler-context mode now carries the class its framework registers, multi-line decorators, and commented-out guards -- `#@token_required` above a ThreatByte handler is the entire bug. Where the registration lives in another document (connexion's `operationId`, an express router), a recipe names `context = [...]`, the harvester writes that stanza per side, and the pair materializes both files at their repository-relative paths so the entry-point mapper sees the layout the project has. `harvest_recipe` fetches every blob before the first write and the driver takes `--report`, so a failed fetch leaves a reviewed excerpt exactly as it was. Two defects the re-harvest exposed: `.mjs` was not a known suffix, so the one pair vendored as an ES module had no file target at all; and `_python_route_access` matched auth words inside decorator prose, reading `@ns.doc(description='... without proper authorization checks')` as a guard. Recorded in the hygiene artifact: nine of the ten duplicate groups sit on both sides of the split, which the repository-level split cannot see. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
7a33756a7e |
feat(learning-harness): the learning subcommands
Task 3.9. Five entry points make the classified detector set reachable: classify a corpus and report the classifier against itself, score a catalog by family, run round zero, run one evolve round, and publish every number from its artifacts. The pairs command gains a runs-per-pair flag, since the family scorer needs more than one. Every step degrades with a reason rather than failing: with no chat endpoint the run says so and still records what it could measure without a detector, and a missing configuration directory is written from the current hunter prompt rather than being an error. Artifacts are keyed by model throughout, so comparing a second detector is a flag rather than a rewrite. Both matrices are green and all three gate outputs stay byte-identical. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
b8ad42c57e |
feat(learning-harness): publish every number from the artifacts that produced it
Task 3.8. A number in a roadmap that nobody can regenerate is a claim, not a measurement. One entry point now writes them all from the committed artifacts and puts the denominator beside each: how many pairs could be scored, why the rest could not, which taxonomy version scored them, the noise the family shows against itself, and what the run cost per pair it got right. Every table says whether hierarchical partial credit could apply, which today it cannot. The roadmap section is replaced in place between markers rather than appended, so the file cannot accumulate a stale number beside a fresh one, and running the command twice without new artifacts changes nothing. With no baseline recorded it says so instead of printing an empty table. The corrected mechanism-lever number travels with the rest: detected five of seventeen at a negative Youden once only train-split pairs teach, against seven when every pair taught, with the difference named as what holdout pairs stopped teaching. Round zero now records what it spent so the cost column has a source. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |
||
|
Marius CiepluchandClaude Opus 5
|
b9552518b9 |
feat(learning-harness): one evolve round, end to end
Task 3.7. A round proposes one change for one family and then lets the evidence decide whether it stays. The stages run cheapest first: a train minibatch, then the family's held-out pairs, then every other family. Each stage scores several runs per pair, and asking for fewer than three is refused outright, because a single run misses most real changes. The family directory is snapshotted before the change, so every outcome other than acceptance leaves the tree exactly as it was found: a tie, a regression on either side, another family below its floor, a proposal that reaches beyond its directory, or a round that crosses its cost cap. Acceptance bumps only that family's version and records which configuration won which pair. Every outcome is journalled, including having no proposal at all, and the round directory holds the proposal, the scores, the pair identities behind the attribution counts and the detector trajectories, all redacted, so a round can be replayed and its attribution audited rather than taken on trust. A round number is never reused. Recorded while writing the sweep: with per-family directories a change cannot reach another family, so the sweep is a floor check over every other family at the moment this round would land, not a causal claim about this change. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG |