The stated final objective is the rung plus an LLM way of diving in
without being forced to rethink the code. The second half is largely
plumbing, because the material already exists: the witness
(`username -> db.session.execute(...) in models/user_model.py (line 73, 8
steps)`) is the reasoning, already written down and already attached to
the finding.
So this session's recurring shape was worth checking rather than assuming,
and it was there. `openultrasast.get_finding` returns the whole record
including the witness, but `openultrasast.findings` dropped the rung: an
LLM listing libpng's 198 rows could not tell the one the graph decided
from the 197 an enumerator proposed without fetching all 198 one at a
time. That is exactly the re-derivation the list exists to spare it.
The list now carries the rung, orders strongest first, and returns a
per-rung tally so a caller can triage before it fetches anything.
The brief records what still stands between here and the objective: the
witness is one line and the query already walks the path elements before
discarding all but the first; the suspicion band will be reasoned about at
length by an LLM, which is worse than a human ignoring it; and a family
that cannot decide its class must say so, or the model will confidently
explain a silence.
Gates byte-identical.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Spec feedback after group 2, recorded in the brief where the goal is
stated rather than in a task where it would be lost.
The standing test is that the tool must find bugs or it is SAST noise, and
that most developers already have dataflow in CodeQL and Semgrep and never
use it. That settles what the differentiator is: not the engine, which
they already have, but the RUNG -- a claim the graph decided, reported
without configuration. Everything below model_entailed is on trial.
The scoreboard, kept honest: vampi finds 3 of 3 known vulnerabilities at
model_entailed, including a second BOLA the engine found before it was in
the recipe, with 2 false positives that are endpoints the OpenAPI spec
declares public. libpng produced 198 rows and zero valid ones, for a
structural reason rather than a tuning one -- the library never calls a
sink the c/memory family models.
Nine defects this session, each found by a measurement and none by review;
three were one bug class in three matchers, two were a capability present
but unreachable. The teachable finding is not any one of them, it is that
a corpus of function-level pairs cannot see any of them, and every one
appeared within minutes of pointing the tool at a real repository. The
gates stayed byte-identical through all nine, which says the same thing
from the other side.
Two priorities change. 2.11, the suspicion band, becomes the critical path
rather than cleanup: libpng's 163 "API is present" rows are exactly the
noise the quote is about, and no arbiter accuracy survives shipping them.
And a family that cannot decide its class has to say so, because a silent
C scan reading as a clean bill of health is worse than noise -- it is
quiet.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Asked what the spec state was and whether anything was forgotten. It was, and
it is the largest thing outstanding: model-grounded-detection is 16/16 tasks
complete and its arbiter measures 86.7% pair-correct at 6.7% leak, but the
model layer has ZERO production callers. `ousast scan` still runs
preprocess -> map -> overlay -> obligations -> regress -> report and touches
none of it. Every number quoted today came from a measurement harness in a
scratch directory.
That is a scope gap in the predecessor spec rather than a slip in executing it:
there was never a task saying "make this reachable", and its Req 11.2 (gates
stay byte-identical) is satisfied PRECISELY BECAUSE nothing is wired. The spec
is complete and the feature is unshipped, both at once.
A new spec rather than an amendment, because the predecessor is coherent and
closed -- it built an arbiter and measured it -- while this is a different
question: making it something a developer runs on a repository they are about
to contribute to.
What it covers, in the order it bites: scan integration; whole-repo scanning
(entry-point region selection, ranking across thousands of candidates, a budget
policy -- the enumerator caps at 8 per region, meaningless at repo scale -- and
a performance envelope never measured); running against a known-vulnerable
checkout, since the 176 real-world CVE pairs are single-function excerpts that
exercise none of it; the deferred memory decision, with 173 of those pairs idle
behind a stub tier; and retiring evidence_level.
It also records what was built outside any spec: the pipeline itself, the ops
work (one image, compose, shell wrappers, pyinfra -- with the Docker image
never actually built), and the corpus additions.
And it carries the known limits forward rather than losing them: the flat
permissive_values list that cannot tell debug=True from httponly=True,
interprocedural resolution as the most repeated miss cause, and the five bugs
that shared one shape -- a taint-specific assumption applied to every family,
each caught by a measurement rather than a check.
Not approved.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
