Open_Ultra_SAST/.kiro/specs/pre-push-safety-net/m1a-recorded-vetoes.md

M1a evidence: the NodeGoat detection with every veto recorded

Run date 2026-09-16. Milestone M1a of release-milestones.md. Instrument benchmarks/push/m1a_record_vetoes.py; flag --experimental-record-vetoes. Evidence: summary and its 11-file bundle.

This record diagnoses. It admits no defect, enables no capability, qualifies no language and makes no latency claim. All three artifacts report finding_status: none with zero admitted defects; every recorded section carries its experimental label.

What was declared before running

Expected base/head pins, expected operation and expected outcome for all three comparisons were written to m1a-record.json before the first scan. The declared target is app/routes/contributions.js, handleContributionsUpdate, family injection, lines 32–34.

Case	Base	Head	Declared expectation
nodegoat-introduce	fixed 0b453d5a	vulnerable c5cb68a7	raw finding at the three eval lines, recorded novelty new
nodegoat-repair	vulnerable c5cb68a7	fixed 0b453d5a	fixed twin analyzed and quiet; three base-only findings
nodegoat-benign	vulnerable c5cb68a7	benign f772c59e	backlog unchanged; never new or worsened

Each revision's target file was reopened before the run: 2,485, 2,491 and 2,498 bytes with recorded SHA-256 digests. Each head scan reported a complete 44-file JavaScript census. Installed engine, facts, query, policy, core and semantics provenance matched the frozen values in all three artifacts.

Observed outcome

Case	Elapsed	Target findings	Target question answered	Deadline reached
nodegoat-introduce	349.8 s	3	yes, 9 rows	no
nodegoat-repair	153.8 s	0	yes, 0 rows	no
nodegoat-benign	261.4 s	3	yes	no

All three ran under an explicitly named 900-second development budget. These durations are development measurements and cannot satisfy or measure the 30-second hook latency gate.

Verdict recorded by the instrument: raw witness present, fixed twin quiet, unchanged revision not reported as a new regression, recorded evaluation produced for all three. The milestone's kill criterion did not trigger: the fixed twin was genuinely analyzed, answering its target question with zero rows and no deadline reason, while the same three findings appear on its base side as base-only findings. Silence from failure was not accepted as a pass.

The vetoes that actually fired

Identical for each of the three eval witnesses on the vulnerable revision. Five recognized vetoes fired and were recorded rather than applied:

Stage	Veto	Evidence
head question	change_context_incomplete	the answered target question carried 9 rows and was then demoted
head scan	dynamic_external_or_depth_context_unresolved	on the target's own question identity, not an unrelated one
head scan	vendor_semantics_unresolved	recorded both as a degradation and as a scope boundary
base scan	graph_incomplete	the base question answered with 0 rows and was then demoted
change context	ambiguous_line_correspondence	file-level, on the target file

The earlier owner diagnosis named four independent vetoes and expected query-to-operation provenance and question-owned vendor relevance to be the two that mattered. Both fired. The run also confirms the third and fourth, so all four named vetoes are real on this case.

What M1a corrected in the diagnosis

A fifth veto is the actual remaining blocker. With all five recognized vetoes lifted, the detection does not become new. Its reason advances from witness_identity_unresolved to operation_correspondence_unresolved, because the head operation sits on a changed line and therefore has no base line anchor. The comparison can classify an operation only when it maps to a base location.

The unchanged revision proves this is the mechanism rather than a coincidence. Its comment-only edit leaves lines 32–34 unchanged, the base location resolves to the same lines, and the recorded novelty becomes unchanged / same_supported_mechanism. Same evidence, same policy, opposite correspondence outcome:

Revision	Operation line	Base location	Recorded novelty
vulnerable	changed	unresolved	unknown, operation_correspondence_unresolved
unchanged	unchanged	resolved to the same line	unchanged, same_supported_mechanism

Query identity differs from witness identity, as predicted. The target is selected through a file-fallback question with function=None, while the rows report method <lambda>2 at lines 32–34. The declared benchmark function is handleContributionsUpdate. This is recorded, not applied: it is why production reported witness_identity_unresolved before any lifting, since an unresolved question yields no comparable operation at all.

Admission reasons that no flag can remove. capability_unavailable remains on every candidate because the registry is empty and this flag cannot enable a capability. dependency_unresolved remains because unlifted coverage reasons still reach admission.

What M1b must repair, ordered by this evidence

1. Authentic per-family change context. change_context_incomplete demoted every answered question in all three cases. Dominance and configuration questions need their own context producers describing their actual query scope; taint context must not be copied into them.
2. Question-owned vendor relevance. vendor_semantics_unresolved turned an otherwise complete 44-file first-party census into graph_incomplete on every base question. Keep the exclusion physical; establish which question actually requires excluded semantics.
3. Operation correspondence for changed lines. The blocker M1a surfaced. Bind the required base/head correspondence to the specific operation rather than to a line anchor, with absent, renamed, removed-guard and ambiguous-operation controls. Do not delete the ambiguity check and do not treat lexical alignment as semantic proof.
4. Correspondence ambiguity scoped to its file region. ambiguous_line_correspondence is recorded file-wide and inherited by every question, including uniquely mapped operations.
5. Query-to-operation provenance. Carry verified provenance from the selected question to the reported operation. Do not inject benchmark function names into regions.

Vetoes that did not fire on this case stay as they are and their families stay experimental.

Limits

Three comparisons on one inspected teaching repository. NodeGoat is development and transfer data; it cannot qualify production Node support, and Node evidence cannot qualify PHP or Python. The lifted view is a diagnostic reconstruction, not a comparison result: it shows what the existing evidence would support if those vetoes were repaired, and no repair has been made. Runtime, independent qualification and rollout all remain NO-GO.

M1b step 1: what the first three repairs changed

Rerun 2026-09-16 after tasks 10.1–10.3, same instrument, same pins, same budget. Evidence: summary and its bundle.

The recorded view now classifies the case correctly. The vulnerable revision reaches new / operation_absent_from_comparable_base, and the unchanged revision reaches unchanged / same_supported_mechanism with its base location resolved to the same lines. Before the repairs both stopped at operation_correspondence_unresolved and witness_identity_unresolved.

Revision	Before 10.1–10.3	After 10.1–10.3
vulnerable	unknown, operation_correspondence_unresolved	new, operation_absent_from_comparable_base
unchanged	unknown, witness_identity_unresolved	unchanged, same_supported_mechanism

The rerun also caught a defect in the repair itself: the new novelty reason was missing from the admission allowlist, so admission rejected the candidate as change_unsupported. Fixed with a control.

The production path is unchanged so far, because two vetoes still demote the target questions: change_context_incomplete on the head from the question's own dynamic_external_or_depth_context_unresolved, and graph_incomplete on the base from the vendor exclusion.

Why the last two vetoes need new query evidence

Neither remaining veto can be scoped or relaxed soundly as the plan originally described, and the rerun is what showed it.

A taint answer reports traced flows. An absent row therefore means either that the operation does not exist in the base, or that it exists and no traced flow reached it. dynamic_external_or_depth_context_unresolved exists precisely to stop the second being read as the first. Removing or narrowing it would let an unreached operation look absent, which turns a wrong absence into a wrong new finding: the false-positive direction this project cannot afford.

At the same time the flag carries no discrimination. It is computed over every in-scope method, and on any real Express application ordinary framework and dependency calls have unresolved destinations, so it is always true and can never be discharged. A veto that is always on and can never be lifted is why the tool reports nothing on real projects.

The resolution is neither to keep it nor to drop it, but to supply the evidence that makes absence provable independently of reachability: export the operations the query actually enumerated in the question's scope, separately from the flows it traced. Absence is then established from the inventory, while a traced-flow claim still requires complete reachability. Vendor exclusion can then stop acting as a global completion veto for the same reason, since dependency on excluded code becomes visible in the evidence rather than assumed.

That prerequisite is now task 10.4, ahead of the two vetoes it unblocks. It adds no detector, alias resolver or second source IR.

M1b outcome: the same case through the production rules

Rerun 2026-09-16 after tasks 10.1 to 10.6, same instrument, same pins, same budget. Evidence: summary and its bundle.

The production comparison now classifies all three revisions without any lifting, and its verdicts match the recorded view exactly. That match is the point: the vetoes M1a recorded were repaired rather than bypassed.

Revision	Production verdict	Recorded verdict
vulnerable	new, operation_absent_from_comparable_base	identical
fixed twin	quiet, analyzed, three base-only findings	identical
unchanged	unchanged, same_supported_mechanism	identical

Compared with M1a, where production reported witness_identity_unresolved on every finding and every one of the 436 questions was demoted on both sides.

The head target questions now complete, the base scan completes all 436 questions, and the three eval witnesses at contributions.js lines 32 to 34 are attributed to the change that introduced them. Timings were 325.7 s, 158.9 s and 260.3 s under the named 900-second development budget, which measures nothing about the hook gate.

All three artifacts still report finding_status: none with zero admitted defects. Admission rejects on capability_unavailable, which is the deliberate eligibility gate, and on dependency_unresolved, which is a coarse plumbing defect recorded as its own task: the runner passes every coverage reason as every candidate's dependency gaps rather than the gaps that pertain to that candidate. Neither is a comparison failure.

What each repair contributed, measured

1. 10.1 operation correspondence. Moved the vulnerable revision from operation_correspondence_unresolved to a classifiable operation.
2. 10.2 required comparison scope. Stopped unrelated unsupported families from blocking a complete target comparison.
3. 10.3 ambiguity scoping. Stopped one file's repeated unchanged lines invalidating every question in the repository.
4. 10.4 operation inventory. Made absence provable from enumeration rather than from the absence of a traced flow, which is what let 10.5 and 10.6 be done soundly.
5. 10.5 reachability binding. Let the head questions complete, since bounded reachability cannot invalidate a flow that was traced.
6. 10.6 declared exclusion. Let the base scan's 436 answers stand, since excluding dependencies is a scope choice rather than a failed read.

A seventh change was needed after the first rerun: the same exclusion and ambiguity distinction was still missing inside the comparison's own degradation and boundary checks, so the verdict stayed at head_context_incomplete even once both scans completed. The rerun is what surfaced it, not the unit suite.

Limits

Three comparisons on one inspected teaching repository, under a development budget. NodeGoat is development and transfer data: it cannot qualify production Node support, and Node evidence cannot qualify PHP or Python. No capability is enabled, no defect is admitted, no alert is emitted, and no latency claim is made. Runtime, independent qualification and rollout all remain NO-GO.

M1b exit run (tasks 10.1 to 10.8)

Rerun 2026-09-16 after every repair, same instrument, same pins, same 900-second development budget. Evidence: summary and its bundle.

Revision	Verdict	Admission	Elapsed
vulnerable	all three sites new, operation_absent_from_comparable_base	capability_unavailable only	326.4 s
fixed twin	analyzed and quiet, three base-only findings	none to admit	157.8 s
unchanged	all three sites unchanged, same_supported_mechanism	capability_unavailable, unchanged	256.8 s

The vulnerable revision's only remaining admission reason is the deliberate eligibility gate. dependency_unresolved is gone, which verifies task 10.7 on real data: those gaps belonged to unrelated questions.

Each finding now carries its provenance explicitly: question function None, engine method <lambda>2, witness eval(req.body.preTax) -> const preTax = eval(req.body.preTax) at contributions.js line 32, and likewise for lines 33 and 34. The unchanged revision resolves each base location to the same line, which is why it reads unchanged rather than new.

Source receipts were reopened before the run, and the head census is the complete 44 files.

Which vetoes were repaired and which remain

All five vetoes M1a recorded are repaired, plus two the repairs exposed:

Veto	State
change_context_incomplete on the target question	repaired, question completes
graph_incomplete on the base from vendor exclusion	repaired, all 436 base questions complete
ambiguous_line_correspondence inherited repository-wide	repaired, decided per operation
dynamic_external_or_depth_context_unresolved as a blanket veto	repaired, bound to reachability claims
operation_correspondence_unresolved on a changed line	repaired via the enumerated inventory
comparison's own degradation and boundary checks	repaired after the first rerun exposed them
dependency_unresolved on every candidate	repaired, gaps attributed per candidate

What remains, recorded rather than repaired, because each is outside this case or a later milestone:

• 231 of 436 head questions still report change_context_incomplete, from context_projection_unavailable and context_scope_empty on families whose queries export no context. Those are dominance and configuration questions. They did not block this case and their families stay experimental, so per the milestone they stay as they are.
• The vendor exclusion and the file's correspondence ambiguity remain reported in coverage. That is intended: they are recorded limits, not repaired absences.

What M1b has not yet demonstrated

The milestone also asks for a developer-readable evaluation finding carrying a supported security consequence and a concrete repair direction. Those two come from an evaluated capability declaration, and the registry is deliberately empty, so nothing renders them today. Writing that text without a declaration is precisely what admission exists to prevent, so it is not something to work around here. The comparison half of M1b is verified; the explanation half waits on a declaration, which is M4's subject.

Limits

Three comparisons on one inspected teaching repository under a development budget. NodeGoat is development and transfer data: it cannot qualify production Node support, and Node evidence cannot qualify PHP or Python. No capability is enabled, no defect is admitted, no alert is emitted and no latency claim is made. Runtime, independent qualification and rollout all remain NO-GO.

M1b explanation: the finding as a developer would read it

Run 2026-09-16 after task 10.10, same pins and budget, with an unreviewed experimental declaration generated by the instrument and bound to the run's own analysis semantics. Evidence: summary and its bundle.

Three evaluation findings were rendered on the vulnerable revision, one per eval site. For line 32:

• Location app/routes/contributions.js:32, family injection, novelty new with operation_absent_from_comparable_base.
• Witness eval(req.body.preTax) -> const preTax = eval(req.body.preTax).
• Change attribution base:app/routes/contributions.js:32-34.
• Provenance file_scope: the question named no function, the engine reported <lambda>2.
• Consequence "A request field reaches const preTax = eval(req.body.preTax), where it is evaluated as JavaScript, so an attacker controlling that field can execute arbitrary code in the server process."
• Repair "Replace the evaluation at const preTax = eval(req.body.preTax) with a numeric conversion such as Number(...) or parseInt(...), then validate the result before use."

Both sentences come from the declaration's templates through the grounded-template path, not from generated prose. The declaration reports verdict experimental and enabled: false, the production result still reports finding_status: none, and all three artifacts carry zero admitted defects. The rendering path never reaches admission, so no supplied declaration can change that.

A defect this surfaced, recorded not repaired

The operation symbol admission extracts is the text before the first parenthesis, which for const preTax = eval(req.body.preTax) is const preTax = eval, not eval. The demonstration declaration had to enumerate each assignment spelling for its coverage check to pass. A real qualified declaration cannot enumerate every assignment a codebase might write, so admission would reject genuine findings on operation_semantics_mismatch. That is recorded as its own task ahead of M4; it is not worked around here.

M1b is met: the comparison classifies all three revisions through the production rules, and the finding reads as a developer would need it to, with every claim traceable to its evidence.