7 Commits

Include renames

Author	SHA1	Message	Date
Marius CiepluchandClaude Opus 5	8c387306be	feat(repos): pinned known-vulnerable checkouts, offline by default ... The pair corpus scores excerpts: a vulnerable function and its fixed twin, handed to the engine with the answer already narrowed down. Nothing in it exercises a repository -- region ranking, the budget, one CPG over thousands of files, or whether a verdict that holds on fifteen lines still holds when the source, the sink and the guard live in three modules. benchmarks/repos/<name>.toml names a real checkout: url, a full 40-char commit SHA, licence, and every known vulnerability with its file, function and line. `ousast repos` lists them; `--fetch` materialises one; CI does neither, and conftest clears the switch and redirects the cache so a developer with warm checkouts cannot read them by accident. Three rules the loader enforces rather than documents: - A branch or tag is rejected. Either would make the same recipe mean different code next month while the baseline it produced kept the old numbers. - `in_scope` must be stated per vulnerability, never defaulted. It decides whether "not found" is a miss or a non-question, which is the most important thing about the result. - A recipe that declares `detection` must name something in scope to detect. `ousast repos` does not trust a recipe either: against a materialised checkout it checks HEAD against the pin, that the licence file is there, and that each declared function appears above its declared line. That earned its place immediately -- the first libpng recipe named LICENSE.md, which is the filename at HEAD today and not at the pinned commit, where it is still LICENSE. Two checkouts, verified against the real repositories: - vampi (MIT, 520 lines) is the DETECTION checkout. Excerpts of it are already in the pair corpus, including the update_password miss diagnosed earlier, so a whole-repository run says directly whether a verdict survives the move off the excerpt. Its routes register through an OpenAPI operationId, so it exercises registration-based entry-point mapping rather than the decorator path. - libpng (libpng-2.0, 91k lines of C) is the ENVELOPE checkout, pinned at the parent of the commit that added the missing clamp. Its CVE is marked out of scope: an out-of-bounds read is not a family this engine arbitrates, and recording "not found" against it as a miss is the easiest way to make a measurement lie. Nothing is vendored; the recipe records the licence so an artifact can say what it ran against. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG	2026-09-09 08:10:59 +02:00
Marius CiepluchandClaude Opus 5	c35426e186	perf(model): one Joern invocation per query kind, not one per region ... JVM startup dominated a scan: a call per region per family put a ten-line file at 275s over three regions, and would have put a thousand regions at roughly fifty hours. The queries themselves are milliseconds once the CPG is loaded, so the cost was almost entirely process starts. scan_repository() is now collect -> batch -> arbitrate. The three .sc scripts accept a `requests` JSON map and answer all of them in one run, keeping the single-request form so the committed measurements stay reproducible byte-for-byte. Arbiters are untouched: each is handed a CpgResult whose `run` returns its own prefetched rows. The batcher builds parameters by calling request_params() extracted from taint.py, dominance.py and config_value.py rather than reconstructing them. A second copy that drifts from the arbiter's is the exact shape of the five family-assumption bugs the predecessor spec spent a session finding; there is now one definition per family and the batcher borrows it. Regions are also deduplicated by (path, function), collapsing the nameless entry-point rows that were producing `model:injection:?:10`. Measured on the same ten-line file: 275s -> 106s, three duplicate findings -> one `model:injection:ping:10`, rung unchanged at model_entailed. Gates byte-identical. Server mode is recorded as task 2.3, deliberately sequenced after the repository measurement: whether the remaining ~100s floor is worth a process model, a weakened subprocess seam and a Scala-evaluating HTTP endpoint depends on a build-time-to-query-time ratio only 2.2 produces. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG	2026-09-09 07:57:35 +02:00
Marius CiepluchandClaude Opus 5	e67f3ee2f0	feat(contributor-scan): 1.3 the model layer reaches a user ... `model-grounded-detection` finished 16/16 with the model layer having zero production callers. This closes that. A real scan of a Flask file now reports: by_rung: {"model_entailed": 3, "model_corroborated": 0, "suspicion": 0} model:injection:ping:10 <- os.system("ping -c 1 " + request.args["host"]) The stage is the fifth MAP proposer, mirroring _check_obligations, and is additive: pattern, overlay and obligation findings all keep working, and without a CPG engine it records cpg_unavailable and changes nothing else. Two scan integrity tests pin the exact degradation set and both broke -- correctly. Their `requested == baseline` assertion, that findings are byte-identical, still passed, so the only change is that the scan now SAYS the arbiter was absent instead of being silently without one. The guards are updated to assert the model degradation specifically rather than being loosened. ModelLayerConfig, not ModelConfig: that name is already the [models] block, and the type checker caught the collision immediately. Two meanings of one name is how a scan silently reads the wrong settings. Every payload carries the same shape, skipped ones included. A payload that is sometimes a full record and sometimes a single "skipped" key is how a report ends up quietly omitting a section. unjudged_sample names the lowest-ranked regions the budget did not reach. Counting them is the honesty requirement; naming them is what lets a maintainer write a large-repository exclusion list from evidence rather than guesswork. The run also produced the finding behind new task 1.4: 275 seconds for a ten-line file. cost_usd is 0.0, so those 21 invocations were not model calls -- they were Joern query launches, one per region per family, each a JVM start. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG	2026-09-09 07:42:44 +02:00
Marius CiepluchandClaude Opus 5	8663c1a5b3	feat(contributor-scan): 1.1 regions from entry points, 1.2 the repository driver ... Pair scoring could rely on a labelled function; a contributor cannot supply one -- which code is risky is the question they are asking. Regions now come from entry points, with a file-level fallback that says it is one, and each carries only the families its shape and language admit. Ranking uses what EntryPointRecord already decided (access_level, trust_boundary) rather than a second opinion computed from the same evidence. A family is offered only where its arbiter could answer: access_control needs a handler, so a library file with no entry point is not asked about obligations it has no siblings to be inconsistent with. The driver makes the two costs that were free at pair scale explicit. ONE CPG per repository, reused across regions -- a build is 5-30s on a forty-line excerpt and per-region builds would make this unusable. And a budget that is a TOTAL for the run rather than the enumerator's per-region cap of eight, spent highest-rank-first, with whatever it does not reach counted into regions_unjudged. A tool that silently analyses a tenth of a repository and reports a clean result is worse than one that says what it skipped. Two flaws the tests caught, both mine and both about ownership: the budget counted calls by reading a client's own usage log, so it silently never tripped for a client that keeps none -- the driver now counts its own spend through a wrapper; and the driver trusted its caller to have sorted, when the order the budget is spent in belongs to whoever holds the budget. The third was worse and only surfaced because a test failed for the right reason: the driver passed candidates=() and never enumerated, so a repository scan would have reported only what the arbiters establish unaided and never populated the suspicion band -- which supplied 15 of 26 findings in the pair measurement. It enumerates now. Sibling audit (Req 8.5) done before writing: family routing lives in exactly one production place, pipeline._arbitrate, dispatching on spec type. regions_for returns family names and the driver resolves specs, so that stays true. 750 passing, mypy clean over 96 files, gates byte-identical, zero orphans. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG	2026-09-09 07:28:30 +02:00
Marius CiepluchandClaude Opus 5	4e4681f162	docs(contributor-scan): design and tasks -- wire it, then measure on a real repo ... Requirements approved, design and tasks generated. 47 criteria, all traced; 13 tasks in 6 groups. The design is deliberately conservative. The model layer joins as ONE STAGE INSIDE MAP, beside the obligations checker it most resembles -- which already takes entry points, produces findings and merges them. Findings are added, never substituted; the pattern and overlay paths keep working; a machine without Joern scans exactly as it does today. Nothing here is allowed to make the tool worse for someone who does not have the engine. Two decisions carry the most weight. One CPG per scan rather than per region: a build is 5-30s on an excerpt and is the dominant cost at repository scale, so per-region builds would make this unusable. And one total budget rather than the enumerator's per-region cap of eight, which is meaningless across thousands of regions; the budget is spent highest-rank-first and whatever it does not reach is counted and reported, never silently truncated. Task 2.2 is a go/no-go and comes before shipping: the first scan of a pinned known-vulnerable checkout is a measurement nobody has taken, and if one CPG per repository proves infeasible the region and budget design changes shape. Better to learn that before groups 3-6 are built on it. Group 5 (corpus) is last on purpose. The binding constraint today is reachability, not corpus size -- 198 of 238 OWASP pairs went unvendored because volume without readability makes every number worse. The task preamble carries the predecessor's two hard-won habits: audit sibling paths in the same commit as any routing or matching change (five bugs shared that shape, each caught by a measurement rather than a check), and run the whole CI list rather than the parts one remembers (CI was red for a session because `ruff check` passing was taken as evidence for `ruff format --check`). Not approved for implementation. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG	2026-09-09 07:17:20 +02:00
Marius CiepluchandClaude Opus 5	14ca0a96ce	docs(contributor-scan): requirements, including the corpus asks I first missed ... 10 requirements, 46 criteria. A NEW spec rather than an amendment to model-grounded-detection, which is complete and closed. Req 1-8 cover the gap that prompted it: the model layer has zero production callers, so ousast scan must use it (1), regions must come from entry points rather than a labelled function (2), ranking and budget must work at repository scale where the enumerator's per-region cap of eight is meaningless (3), the performance envelope must be measured on a real checkout (4), output must be actionable (5), packaging must be built and verified rather than merely defined (6), the two deferred decisions must close (7), and the honesty disciplines carry forward (8). Req 9 and 10 were added after the maintainer asked whether their requests had been captured. They had not been -- I folded the corpus asks implicitly into "run against a known-vulnerable repository", which is not the same thing. Now explicit: a PHP/WordPress slice (php2cpg works once an interpreter is present, which the image provides, and plugin CVEs are overwhelmingly the families the model arbitrates well); real-world C library CVEs -- libpng, image libraries, OpenSSH, libpam, kernel modules -- gated on the memory decision, because until it closes they would join 173 pairs already idle; more vibe-code pairs, since that slice is where the stated audience works and it is only 35 pairs; and regression tuning over a pinned large-repository baseline, which is the thing pair scoring structurally cannot see. Req 9.4 carries today's lesson: verify on a sample that the model can READ a slice before vendoring it. The OWASP harvest found 238 pairs and vendored 40 for exactly that reason. Not approved. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG	2026-09-08 19:42:53 +02:00
Marius CiepluchandClaude Opus 5	41266ce472	docs(contributor-scan): brief -- the model layer is complete and unshipped ... Asked what the spec state was and whether anything was forgotten. It was, and it is the largest thing outstanding: model-grounded-detection is 16/16 tasks complete and its arbiter measures 86.7% pair-correct at 6.7% leak, but the model layer has ZERO production callers. `ousast scan` still runs preprocess -> map -> overlay -> obligations -> regress -> report and touches none of it. Every number quoted today came from a measurement harness in a scratch directory. That is a scope gap in the predecessor spec rather than a slip in executing it: there was never a task saying "make this reachable", and its Req 11.2 (gates stay byte-identical) is satisfied PRECISELY BECAUSE nothing is wired. The spec is complete and the feature is unshipped, both at once. A new spec rather than an amendment, because the predecessor is coherent and closed -- it built an arbiter and measured it -- while this is a different question: making it something a developer runs on a repository they are about to contribute to. What it covers, in the order it bites: scan integration; whole-repo scanning (entry-point region selection, ranking across thousands of candidates, a budget policy -- the enumerator caps at 8 per region, meaningless at repo scale -- and a performance envelope never measured); running against a known-vulnerable checkout, since the 176 real-world CVE pairs are single-function excerpts that exercise none of it; the deferred memory decision, with 173 of those pairs idle behind a stub tier; and retiring evidence_level. It also records what was built outside any spec: the pipeline itself, the ops work (one image, compose, shell wrappers, pyinfra -- with the Docker image never actually built), and the corpus additions. And it carries the known limits forward rather than losing them: the flat permissive_values list that cannot tell debug=True from httponly=True, interprocedural resolution as the most repeated miss cause, and the five bugs that shared one shape -- a taint-specific assumption applied to every family, each caught by a measurement rather than a check. Not approved. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG	2026-09-08 19:40:38 +02:00