feat(reports): say what was looked for, and what cannot be decided

contributor-scan 2.16, implementing the approved Req 5.6. `c/memory`
scored zero true positives across 89k lines of C and three distinct
classes of false one, for a structural reason: it models string CALLS and
libpng overflows through arithmetic.

The task allowed retiring the family. Kept, scoped and stated instead --
`strcpy` misuse is real in plenty of C and the family earns its place for
that. Retiring it would have removed the disclosure along with the
coverage, leaving a C scan silent for a reason nobody could read.

**Its own description was the quiet-failure risk.** It claimed "bounds,
lifetime, arithmetic", which is exactly what it does not decide. It now
names the APIs it models, and a new `limits` field in families.toml says
what it cannot: an overflow expressed as an index, a pointer offset, or a
memcpy whose size is computed -- which is how most real C overflows are
written, and how every one of libpng's is.

A limit recorded in a fact table nobody reads is no better than silence, so
the report renders it. **What was analysed** lists every family offered to
a region whether or not it found anything, because a family that ran and
reported nothing is the case a reader is most likely to misread, and it
states plainly that silence means no fact table covered the code rather
than that the code is safe.

The counts are derived from the fact tables the arbiter actually used, so
they cannot drift from what the engine looked for; only the limit sentence
is authored, because "cannot decide an overflow expressed as an index" is a
statement about an abstraction and no table can produce it.

A C scan that finds nothing now says:

  | c | `memory` | 121 | 9 |
  **`memory` (c)** — Models CALLS, not arithmetic...

Gates byte-identical.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
This commit is contained in:
Marius Ciepluch
2026-09-09 15:08:01 +02:00
co-authored by Claude Opus 5
parent f488e625c2
commit af7e33ef3e
8 changed files with 173 additions and 6 deletions
+17 -3
View File
@@ -3,7 +3,7 @@ from __future__ import annotations
import argparse
import json
import tempfile
from collections.abc import Sequence
from collections.abc import Mapping, Sequence
from dataclasses import dataclass, replace
from pathlib import Path
from time import perf_counter
@@ -538,6 +538,10 @@ def _run_scan(path: Path, config_path: Path, mode: str, fail_on: str) -> ScanOut
wrote_verdicts = True
plan = record_completed(plan, Stage.REGRESS)
worth_fixing_payload = _worth_fixing_payload(verdict_records) if wrote_verdicts else None
# Req 5.6: what each family could see, for the report. Absent when the model layer did not run --
# and a scan with no model layer states no limits because it made no claims.
coverage_rows = model_payload.get("coverage") if isinstance(model_payload, dict) else None
coverage_for_report = coverage_rows if isinstance(coverage_rows, list) and coverage_rows else None
runtime.run_stage(
"report",
lambda: write_markdown_report(
@@ -550,6 +554,7 @@ def _run_scan(path: Path, config_path: Path, mode: str, fail_on: str) -> ScanOut
overlay=overlay_records if wrote_overlay else None,
obligations=obligations_cited or None,
obligations_summary=obligations_payload or None,
coverage=coverage_for_report,
),
)
runtime.run_stage(
@@ -872,7 +877,9 @@ def _model_skipped(reason: str) -> dict[str, object]:
return {**_model_payload(None), "skipped": reason}
def _model_payload(result: object, *, unjudged_paths: tuple[str, ...] = ()) -> dict[str, object]:
def _model_payload(
result: object, *, unjudged_paths: tuple[str, ...] = (), coverage: Sequence[Mapping[str, object]] = ()
) -> dict[str, object]:
"""What the scan records about the model layer.
`unjudged_sample` names the regions the budget did not reach, lowest-ranked first. Counting them is the
@@ -892,6 +899,7 @@ def _model_payload(result: object, *, unjudged_paths: tuple[str, ...] = ()) -> d
"query_seconds": getattr(result, "query_seconds", 0.0),
"query_seconds_by_kind": dict(getattr(result, "query_seconds_by_kind", {}) or {}),
"arbitrate_seconds": getattr(result, "arbitrate_seconds", 0.0),
"coverage": list(coverage),
}
@@ -929,6 +937,12 @@ def _run_model_layer(
if not regions:
return [], _model_skipped("no_regions")
# Req 5.6: what each family could see, recorded whether or not it found anything. A family that ran
# and reported nothing is the case a reader is most likely to misread.
from .model.report import coverage_rows
coverage = coverage_rows(regions)
resolved = resolve_chat_endpoint(config)
client = resolved[0] if resolved else None
if client is None:
@@ -952,7 +966,7 @@ def _run_model_layer(
# The regions the budget did not reach, weakest first -- the evidence an exclusion list is built from.
unjudged = tuple(region.path for region in regions[result.regions_scanned :][:10])
findings = [_finding_from_model(item) for item in result.findings]
return findings, _model_payload(result, unjudged_paths=unjudged)
return findings, _model_payload(result, unjudged_paths=unjudged, coverage=coverage)
def _finding_from_model(item: object) -> StaticFinding: