Commit Graph
12 Commits
Author SHA1 Message Date
Marius CiepluchandClaude Opus 5 bb72fcd413 test(m3): Python transfer detects all three declared targets (M3 13.1)
Generalized the replay instrument so the case family is an argument rather than a
hard-coded target, since a harness that only knows one language cannot say whether
the design transfers. Targets, pins and expectations come from the committed inputs,
so Python is asked exactly the way Node was, through the same core, ranker,
comparison and admission.

  VAmPI-SQLI        models/user_model.py:73:get_user        detected, answered
  VAmPI-BOLA        api_views/users.py:187:update_password  detected, answered
  VAmPI-BOLA-BOOKS  api_views/books.py:51:get_by_title      detected, answered

35 of 35 questions complete on every case, nothing reported new, zero admitted
defects.

Two results stand out. The SQL injection is the evidence the milestone spec names
as previously unresolved. And two of the three targets are access control, the
family that could not complete a question on any repository before group 12, so
this is the first time broken object-level authorization has been asked on Python
at all, and it answered on both endpoints.

VAmPI has the same revision on both sides, so the regression mode establishes
detection and completion, never novelty. It asserts that directly: with nothing
changed, a target reported new would be a comparison defect, and none was. That
tests the novelty logic from the opposite side to the NodeGoat case.

Identical revisions cannot show novelty, and VAmPI is inspected teaching data that
cannot qualify production Python support. No capability enabled, no latency claim.

1,407 tests passed, nine skipped.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-17 18:17:31 +02:00
Marius CiepluchandClaude Opus 5 d93dee9f1a test(cpg): prove transport equivalence for every shipped query kind (M2 11.2)
A session is only usable if it returns exactly what the disposable invocation
returns. On a real NodeGoat graph, taint, dominance and configuration all produce
byte-identical payloads through both transports, with parameters built by each
arbiter's own request function through the driver's family lookup rather than a
hand-written shape.

The failure shapes that would make a session dangerous rather than merely slow are
correct too. An empty answer stays present and distinguishable from an unavailable
one, which is the distinction that once turned a failed scan into a clean bill of
health. An unknown query and an absent graph are both unavailable. Two graphs in
one session each answer about the graph the request names.

Amortized over five taint requests the session took 22.5 s against 72.0 s
disposable, all five answered and identical to each other.

Two instrument defects are recorded rather than hidden. The first run created a
fresh backend per comparison, so every session served one request and paid full
startup, measuring the session's worst case and making it look slower; that is why
the amortized comparison exists. The switched-graph check crashed on a scratch
directory it never created, so it produced no verdict at all.

These timings are orientation, not the latency gate. Task 11.5 measures against the
frozen seven-class profile, and the transport stays off by default until then.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-16 20:37:47 +02:00
Marius CiepluchandClaude Opus 5 7e3f12bc00 feat(cpg): add a transaction-owned engine session and route it opt-in (M2 11.1, 11.3)
JVM startup, not analysis, is what a push scan spends its time on: each disposable
script invocation costs 13 to 23 seconds while the work takes under a second.

The session owns one Joern server per transaction on loopback with an OS-chosen
port and no published port. Three properties are load-bearing, and the probe that
settled the design demonstrated why each is needed.

An HTTP status is not an answer. The server returns success even for a compile
error, with the diagnostic in its body, so every request redirects the evaluated
code's console output into a fresh private file behind a nonce, and the nonce is
verified before the answer is accepted. The engine's body is retained for exactly
the case where the receipt is missing, without which that failure explains nothing.

A poisoned session answers nothing afterwards, and cancellation kills the process
group rather than the launcher, which this project has already run a machine out of
memory by neglecting.

What the probe established about the engine, after three iterations that each
failed loudly: runScript is absent from the server REPL; a shipped @main script
cannot be nested in a block expression, so definitions go to the top level
unwrapped; and the REPL shadows Console with io.joern.console.Console, which has no
withOut, so capture must qualify scala.Console. With that fixed, both mechanisms
answer identically to the disposable invocation, 0.8 s against 9.8 s.

The backend routes batch and single queries through the session only when
session_transport is enabled, which it is not by default. Any failure falls back to
the disposable invocation rather than reporting a zero, and each script is defined
under its own name because every script declares exec.

Equivalence for taint, dominance and configuration is task 11.2 and gates any
measured run. 1,391 tests passed, nine skipped. Rollout stays NO-GO.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-16 20:24:34 +02:00
Marius CiepluchandClaude Opus 5 cec1c4a428 feat(pre-push-safety-net): render the experimental evaluation explanation (M1b 10.10)
The M1b exit run established the comparison but could not show a developer-readable
consequence or repair direction, because both belong to an evaluated capability
declaration and the installed registry is deliberately empty.

The experimental evaluation output now renders, for each change-attributed finding,
its family, exact location, witness, change attribution, provenance, engine method
and operation, together with a consequence and repair direction produced from a
supplied declaration through the same grounded-template path the admitted case
uses. No prose is invented: when no declaration matches, or a template is not
grounded in the operation and context, or the declaration does not cover the
operation symbol, the explanation is absent and the reason is stated.

Declarations reach this rendering only. The runner never passes them to admission,
so nothing in that file can enable a capability or emit an alert however it is
written, and a control asserts that even a declaration claiming PASS admits
nothing. The flag requires explicit base/head replay and the recorded-veto flag.

The evaluation instrument generates its own unreviewed declaration bound to the
run's analysis semantics, asserting that it is neither enabled nor PASS, so the
demonstration cannot drift into a qualified claim.

1,377 tests passed, nine skipped. Rollout stays NO-GO.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-16 19:59:12 +02:00
Marius CiepluchandClaude Opus 5 3224884d8f feat(pre-push-safety-net): record admission vetoes and meet M1a on NodeGoat
Add group 9 to the approved task plan as the bounded owner amendment release
milestone M1a requires, implement the task-local recorded-veto evaluation flag,
and run the three pinned NodeGoat revisions under it.

The flag re-runs the production comparison and admission over a lifted copy of
the same evidence and records each veto instead of applying it. Recognized
completeness vetoes are lifted and logged; failures never are, because silence
from a failure is not a quiet fixed twin. It is off by default, accepted only on
explicit base/head replay, refused in Git stdin mode, labels every payload it
produces, and admits no defect even when a lifted candidate otherwise would.

M1a outcome, inside its one-week box and with expectations frozen on disk first:
the three entailed <lambda>2 witnesses are present at contributions.js 32-34; the
fixed twin was genuinely analyzed and quiet, answering its target question with
zero rows and no deadline reason while the same findings appear as base-only; the
unchanged revision was analyzed and recorded unchanged. The kill criterion did
not trigger.

Five vetoes fired and were recorded. M1a also corrected the diagnosis: with all
five lifted the detection still stops at operation_correspondence_unresolved,
because a newly introduced operation sits on a changed line and has no base
anchor. The unchanged revision resolves its base location on the same lines and
classifies as unchanged, which isolates correspondence as the mechanism. That
fifth veto was not in the earlier four-veto diagnosis and is now M1b's central
repair.

Diagnosis only. No defect admitted, no capability enabled, no gate changed, no
latency claim; the 900-second development budget cannot measure the hook gate.
Rollout and task 8.3 remain NO-GO.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-16 15:31:34 +02:00
Marius Ciepluch 88216c5072 test(pre-push-safety-net): validate packaged outcome and retain rollout no-go 2026-09-13 19:39:19 +02:00
Marius Ciepluch b506516d87 feat(pre-push-safety-net): enforce evidence eligibility and record blocked qualification 2026-09-13 19:32:43 +02:00
Marius Ciepluch 2491ce497c feat(pre-push-safety-net): measure frozen runtime feasibility and retain no-go 2026-09-13 19:06:02 +02:00
Marius Ciepluch 530b9c9879 feat(pre-push-safety-net): reuse compatible artifacts throughout replay 2026-09-13 17:08:49 +02:00
Marius Ciepluch 78a7dfc8c4 feat(pre-push-safety-net): measure frozen PMPro and Node replay feasibility 2026-09-13 15:24:06 +02:00
Marius Ciepluch 051f3b0058 feat(pre-push-safety-net): preserve uncertainty in benchmark scoring 2026-09-12 19:04:20 +02:00
Marius Ciepluch 8acada0d98 feat(pre-push-safety-net): pin labeled evaluation inputs 2026-09-12 18:45:54 +02:00