Open_Ultra_SAST/benchmarks/measurements/2026-09-21-partition-gap-blast-radius.json

{
  "schema_version": 1,
  "experiment": "partition-gap-blast-radius-v1",
  "date": "2026-09-21",
  "task": "pre-push-safety-net 13.4",
  "claim_scope": "Why a multi-language repository completes no question, in two parts. Measured on Paid Memberships Pro.",
  "part_one_done": {
    "change": "cross_partition_semantics_unresolved moved from GRAPH_INTEGRITY_GAPS to DECLARED_EXCLUSION_GAPS",
    "argument": "Membership of the integrity set should turn on whether a gap can make a reported answer WRONG, not on whether it cost coverage. An unparsed file may hold the sanitizer that would have cleared a reported flow, so it belongs. A flow this tool cannot follow from one language into another can hide a finding and never invent one, because a PHP sink is sanitised in PHP.",
    "what_it_touches": "Two places, both already honouring the distinction: the outcome demotion in scan.py and _blocking_degradations in the push policy, which exempts declared exclusions. The reason stays published in the scope's unresolved boundaries either way, so nothing is hidden.",
    "correction": "The task note written earlier today said NOT to reclassify, on the grounds that it would claim cross-language flows are out of scope rather than unresolved. That was a misreading of the class: both classes are added to unresolved_boundaries, and membership decides only whether every answer in the repository is demoted."
  },
  "part_two_open": {
    "symptom": "PMPro still completes 0 of 127 questions after the reclassification, while establishing 19 findings",
    "cause": "partition_file_census_incomplete, which is still an integrity gap and still fires",
    "the_missing_files": [
      "js/blocks/blocks.build.js",
      "js/jsapi.js",
      "js/select2.min.js",
      "webpack.config.js"
    ],
    "reading": "The JAVASCRIPT partition expected 36 files and its graph holds 32. The four absent ones are a webpack build output, a minified third-party library, a build config and one source file. The PHP partition is intact. So a plugin's bundled select2.min.js costs the entire PHP analysis its admissibility: 4,425 PHP regions demoted by four JavaScript files.",
    "why_the_gap_is_right_and_its_blast_radius_is_not": "A file expected but missing from a graph could hold a sanitizer, so answers drawn from THAT partition may be wrong and demoting them is conservative. A missing JavaScript file cannot make a PHP answer wrong. This is the same over-broad blast radius as part one and the third instance of one principle in this thread: a gap should reach the questions it can actually affect.",
    "fix_direction": "Scope a partition's census gap to that partition's questions. The language is known where the gap is raised, and path-to-language is already computed and used to dispatch every request, so the data exists; it is not threaded to the demotion.",
    "second_order": "Three of the four files are build output or minified vendored code. The JavaScript vendored list holds only node_modules/, so a minified library under js/ is treated as first-party. Excluding build artefacts would remove this instance without addressing the blast radius.",
    "superseded_by": "part_two_closed"
  },
  "measured_after_part_one": {
    "questions_completed": 0,
    "questions_total": 127,
    "findings_total": 19,
    "outcome_status": {
      "unresolved": 127
    },
    "degradations": [
      "cross_partition_semantics_unresolved",
      "partition_file_census_incomplete"
    ]
  },
  "limits": [
    "One multi-language subject. A single-language PHP repository would isolate part one's effect and has not been measured.",
    "Part one is verified by argument and by the test suite, not yet by a subject that completes because of it."
  ],
  "part_two_closed": {
    "change": "A census gap now carries the partition it was raised for, and an integrity gap reaches only that partition's questions. The same gap is raised in TWO passes, arbitration and evidence, and carrying the language in one but not the other left an unscoped copy that demoted everything: the fix looked applied and was not, until the record printed each outcome's reason rather than only its status.",
    "measured_on_pmpro": {
      "questions_completed_before": 0,
      "questions_completed_after": 120,
      "questions_total": 127,
      "findings_unchanged": 19,
      "outcome_reason": {
        "completed:query_answered_and_arbitrated": 120,
        "unresolved:graph_incomplete": 7
      },
      "the_seven_still_unresolved": "the JavaScript partition's own questions, which is correct: the four missing files are its files, so its answers are the ones that could be wrong"
    },
    "still_open": "The push policy's _blocking_degradations exempts declared exclusions but does not yet respect a partition, and a post-hoc diagnostic copy carries no language. A JavaScript partition's gap would still block admission of a PHP finding."
  }
}