From b5d3cd6ab6a612080552cecac785f8334df4ebd2 Mon Sep 17 00:00:00 2001 From: Marius Ciepluch <11855163+norandom@users.noreply.github.com> Date: Wed, 9 Sep 2026 14:06:11 +0200 Subject: [PATCH] docs(roadmap): a v0.1 release definition, ordered toward libpng and WordPress The named release targets are, today, exactly the two cases that do not work -- and they fail for different reasons, which is what makes them a good pair rather than one. WordPress fails because `taint_specs(language="php")` returns zero families: Joern's php2cpg parses it and we say nothing. That is the shortest path from here to a named target, and the case this arbiter suits best, because PHP web bugs are the classic taint families. libpng fails because c/memory models string functions while libpng overflows through arithmetic, and its library never calls a sink we model. That needs either a second abstraction -- size arithmetic, a constraint problem -- or an honest stated limit. So "ready for" means two different things and the criteria say which applies where. Where a family can decide the class, find the known vulnerability. Where it cannot, run inside the envelope, report nothing false, and STATE the limit. The second is not a consolation prize: a tool that says "I do not model this class" is usable, and one that silently finds nothing is what teaches developers to distrust SAST in the first place. Seven v0.1 criteria, each tied to a requirement that already exists: unattended runs inside a stated envelope; zero known-false entailed findings on every pinned checkout; the known vulnerability found wherever a family can decide it (Req 4.6); every family's limit stated in the report (Req 5.6); a proportionate report (Req 5.5); installs without the user having Joern, a JVM or PHP; and a committed regression baseline per target. That reorders the middle of the roadmap. PHP moves up, and the c/memory decision moves up with it, because criterion 4 needs that family either earning its place or being scoped -- and scoping it is cheap. The vibe-code repository stays valuable as the guard against tuning everything on vampi, but drops below PHP because it is not a named target. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG --- .kiro/steering/roadmap.md | 60 +++++++++++++++++++++++++++++++++------ 1 file changed, 52 insertions(+), 8 deletions(-) diff --git a/.kiro/steering/roadmap.md b/.kiro/steering/roadmap.md index eaf1058..bce2cfd 100644 --- a/.kiro/steering/roadmap.md +++ b/.kiro/steering/roadmap.md @@ -5,6 +5,43 @@ adopted, the model layer was built, wired and measured, and two repositories have been scanned. See `overview.md` for what is true today; this is what happens next and in what order. +## The target: a first release for small projects + +> "a first release that is ready for smaller projects like libpng or wordpress" + +Those two are, today, exactly the cases that do not work — and they fail for **different** reasons, which is +what makes them a good pair of targets rather than one. + +| target | why it fails now | what a release needs | +| --- | --- | --- | +| **WordPress** (php) | `taint_specs(language="php")` returns **zero families**. Joern's `php2cpg` parses it; we say nothing. | Fact tables and entry-point mapping. The engine is right for this: PHP web bugs are the classic taint families, which is what the taint arbiter already decides well. | +| **libpng** (c) | `c/memory` models string functions; libpng overflows through *arithmetic*, and its library never calls a sink we model. | Either a second abstraction (size arithmetic — a constraint problem) or an honest, stated limit. | + +So "ready for" has to mean two different things, and the release criteria say which applies where: + +* For a project whose bug class a family **can** decide (WordPress, vibe-code): find the known vulnerability. +* For one it **cannot** (libpng today): run inside its envelope, report nothing false, and *state the limit* + rather than returning a quiet clean bill of health. + +The second is not a consolation. A tool that says "I do not model this class" is usable; one that silently +finds nothing is the thing that teaches developers to distrust SAST. + +### Release criteria (v0.1) + +1. Runs unattended on a pinned checkout of each target within a stated time and memory envelope. +2. **Zero known-false entailed findings** on every pinned checkout. +3. Finds the known vulnerability wherever a family can decide it in principle (Req 4.6). +4. States what each family cannot decide, in the report (Req 5.6). +5. The report is proportionate — no rule repeating itself past a stated count (Req 5.5). +6. Installs and runs without the user having Joern, a JVM, or a PHP interpreter (docker compose + wrapper). +7. A committed regression baseline per target, so the next change is measurable rather than argued about. + +### What that reorders + +PHP moves **up**: it is the shortest path from here to a target the maintainer named, and it is the case the +existing arbiter is best suited to. The C decision moves up with it, because criterion 4 needs `c/memory` +either earning its place or being scoped — and scoping it is cheap. + ## The one test everything is ordered by > "it must find the bugs, otherwise it's SAST noise. most devs know codeql / semgrep have dataflow @@ -28,15 +65,18 @@ Everything here is in `contributor-scan`, group 2, and all of it is measurable o Needs the access level's *provenance*, because for decorator frameworks "public" means "no decorator found", which is the bug itself. -## Next — make it find more, in the cheapest order +## Next — reach the named targets -5. **5.2 — a pinned vibe-code repository.** The cheapest route to a *second* checkout that can produce a true - positive, because Python and JavaScript are the two languages whose fact tables are complete. Everything - downstream is tuned on two repositories until this lands. -6. **2.15 — a C checkout whose CVE goes through a sink we model.** Decides whether `c/memory` earns its place - or gets scoped (2.16). libpng cannot answer this; no version of it can. -7. **5.1 — PHP fact tables, then WordPress.** `php2cpg` exists; our PHP families are empty. Tables first, - corpus second — the other order produces a confident silence. +5. **5.1 — PHP fact tables, then a WordPress checkout.** The shortest path to a target the maintainer named, + and the case this arbiter suits best. Sources, sinks and sanitizers for PHP; entry points for WordPress's + registration model (`add_action`, `add_filter`, admin-ajax, REST routes) which the mapper already handles + in principle. **Validate on pairs before pointing it at WordPress** — tables first, corpus second, because + the other order produces a confident silence. +6. **2.15 / 2.16 — settle `c/memory`.** Either a C checkout whose CVE runs through a sink we model, giving the + family its first true positive, or the family's scope narrowed and stated. Release criterion 4 needs one + or the other, and the second is cheap. libpng cannot answer the first; no version of it can. +7. **5.2 — a pinned vibe-code repository.** Still the cheapest *second* true-positive checkout, and the guard + against tuning everything on vampi. Below PHP now only because it is not a named release target. ## Then — close the deferred engine questions @@ -72,3 +112,7 @@ Everything here is in `contributor-scan`, group 2, and all of it is measurable o The scoreboard in `overview.md`, and one question: **can a developer run this on a repository they did not write and get something they act on?** Today that is true for one Python application and false for one C library, and the specs above are ordered by what changes that. + +A release is ready when that question is answered **yes for WordPress and honestly for libpng** — found for +the one whose class we decide, and a stated limit for the one we do not. Both are release criteria; only one +of them is a detection claim.