- The overlay scorer ignores `coverage`. The Java sqli vuln file already produces a coverage record on `prepareCall` from a servlet source; the scorer reports MISS.
- The overlay column equals the inventory column on all 176 vfc pairs because C facts have no memory sink, so every proposal is `flow_incomplete` and the scorer falls back to inventory. The dashboard cannot show engine progress until the scorer reports parser and adjudication loss explicitly.
- Harvest anchors on the first textual occurrence of the function name. For curl that is the doc comment mention (`Curl_follow()`), so 25 pairs start mid-comment and fail to parse.
- 129 of 176 vfc expected rows have `sink = "unknown"`; 4 have a rule id. Matching degrades to CWE equality.
- Real-Vuln-Benchmark (Apache-2.0) ships 66 pinned Python web repos, 40 of them LLM-generated, with file, line range, function, primary and acceptable CWEs, severity, authorship, and 120 false-positive traps. It is the only released corpus with an authorship column.
- SecBench.js ships 600 npm vulnerabilities with `fixCommit`, `fixedVersion`, and `sinkLocation`, but no license file. Pointer plus harvest from upstream repos only.
- GitHub commit search for a Claude co-author trailer plus "fix IDOR" returns over twenty thousand commits; a sampled June 2026 FastAPI repository with `AGENTS.md` had a 12-line fix moving identity from the request body to the auth context. Agent-authored vuln-to-fix pairs are harvestable.
- VibeApps/VibeVulns (arXiv 2606.23130) is not released, but its identification method is: `.claude/` directory, Lovable meta tag, Claude trailers, and an 85% AI-authored threshold.
- Context: Why the sast overlay number did not move after the tree-sitter extra.
- Sources Consulted: `src/openultrasast/pairs.py` (`_evaluate_overlay_pair`, `_overlay_matches_expected`, `_overlay_leaks`), `semantic/overlay.py`, direct runs of `adjudicate` on the sast files.
- Findings: only `promote` is counted; `coverage` is excluded on both sides. `_overlay_matches_expected` accepts a CWE-only match. `ExpectedFinding` already has `function` and `line`, and `benchmark._matches_expected` honors them, but the overlay path does not. `OverlayRecord` has no function field; `FunctionIR` has line ranges.
- Implications: count coverage symmetrically; add optional `function` to `OverlayRecord`; require function or sink or rule id for a match when present; flag CWE-only rows.
- Sources Consulted: `benchmarks/pairs/vfc/harvest.py``extract_function`; tree-sitter ERROR nodes on `curl-cve-2022-27774/vuln.c` and `curl-cve-2023-38545-do_socks5/vuln.c`.
- Findings: the name search accepts a match followed by `(` even inside a comment, and the brace search then finds the real body. A second, smaller class is `#if`-split `else if` chains, which tree-sitter cannot attach; that is an engine tolerance question, not harvest.
- Implications: skip comment and string regions before searching; anchor on a line that starts a declarator; add line-range and hunk modes so Python and JS/TS are covered; report parse loss on the dashboard so the engine spec can measure its tolerance work.
- Findings: the accept gate already ANDs four clauses and reverts byte-for-byte. Pair signals carry pair name but no profile. Ledger is one file per target. Preprocess tags are deterministic and per file.
- Implications: classify on two axes (provenance for gating, mechanism for reporting). Branch the acceptance decision, not the ledger format: reject a change if any profile's holdout regresses. Profile-specific facts and thresholds are deferred to the engine specs where those data live.
| Reopen `real-world-vfc-slice` | Add tasks to the closed spec | One place | Mixes closed seed work with scorer redesign; roadmap says do not reopen | Rejected |
| New superseding spec, corpus plane only | Scorer, labels, harvest, slices, provenance, profile gate | Measurable before engine work; engine specs stay pure | Two engine specs still needed | **Selected** |
| One big spec including engine | Corpus plus walker plus guards | Fewer specs | Unreviewable size; engine changes would be measured by a lossy scorer | Rejected |
| Profile as ledger fork | One `rule_policy.<profile>.json` per profile | Full branching | Changes ledger format owned by another spec; overfits per slice | Deferred |
## Design Decisions
### Decision: Count coverage-with-flow as detection and as leak
- Selected Approach: expected rows carry `function`; match requires the record inside that function's lines; rows with no function, sink, or rule id are `weak_label` and never match by CWE alone.
- Trade-offs: vfc recall may drop on first run; the drop is real.
- Selected Approach: signals from `.claude/`, `CLAUDE.md`, `AGENTS.md`, `.cursor/`, `.github/copilot-instructions.md`, Lovable meta tag, "Generated with Claude Code", and co-author trailers for Claude, Codex, Copilot, Cursor, Devin, Jules when a git log is readable. Threshold for `agent` on a git tree: at least 85% of commits or lines in the last N commits carry a signal, following the VibeApps method; otherwise `mixed` when any signal exists; `human` when none; `synthetic` only from catalog declaration.
- Trade-offs: repos that scrub trailers read as human. Acceptable; the label is a profile, not a verdict.
