measure(m3): two languages in a repository defeat completion, and so defeat admission

No PHP question is ever marked completed, and PHP was never the variable. partitions.py raises
cross_partition_semantics_unresolved whenever more than one language partition exists, that reason
is a member of GRAPH_INTEGRITY_GAPS, and a graph-integrity gap demotes every completed outcome in
the scan. Admission requires a completed outcome.

The evidence is one column wide. NodeGoat is JavaScript alone and completes 299 of 299. VAmPI is
Python alone and completes 35 of 35. PMPro is PHP with 36 regions of bundled JavaScript and
completes 0 of 127 and 0 of 520 while establishing 19 and 43 findings. NodeGoat own degradation,
vendor_semantics_unresolved, is a DECLARED exclusion rather than an integrity gap, which is exactly
why it does not demote.

This lands on the declared v0.1 target. A WordPress plugin is PHP plus its admin JavaScript, so
every one of them is multi-language, so on the target this tool was built for it can detect defects
and never admit one. Neither subject that currently passes could reveal it, which is why a census
across a third subject was worth running.

Recording the boundary is right: a flow from PHP into JavaScript is semantics the partitioned graph
cannot follow. Demoting every question is not, including one wholly inside a single PHP file. Task
12.3 already fixed this bug class one level down by making a gap question-owned so it could not
block another question comparison; the same principle applies to a partition boundary. Reclassifying
the reason as a declared exclusion is explicitly the wrong fix, because that would claim
cross-language flows are out of scope when they are unresolved.

Task 13.4 rewritten with the diagnosis, and the milestone amendment now ranks it with M2 rather
than behind it: a narrow JavaScript-only release can proceed without it, the WordPress target
cannot proceed at all.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
This commit is contained in:
Marius Ciepluch
2026-09-21 10:57:19 +02:00
co-authored by Claude Opus 5
parent 6bd6e3531e
commit 73e9c1fed6
3 changed files with 86 additions and 4 deletions
@@ -0,0 +1,69 @@
{
"schema_version": 1,
"experiment": "two-languages-defeat-completion-v1",
"date": "2026-09-21",
"task": "pre-push-safety-net 13.4",
"claim_scope": "Why no PHP question is ever marked completed. Measured across four census runs on three subjects.",
"finding": "A repository that holds TWO languages can never complete a question, so it can never admit a finding. The cause is structural and deliberate, not a bug in PHP support: `partitions.py` raises `cross_partition_semantics_unresolved` whenever more than one language partition exists, that reason is a member of `GRAPH_INTEGRITY_GAPS`, and a graph-integrity gap demotes EVERY completed outcome in the scan. PHP was never the variable.",
"evidence": {
"nodegoat": {
"languages": {
"javascript": 44
},
"language_count": 1,
"questions_total": 299,
"questions_completed": 299,
"findings": 8,
"degradations": [
"vendor_semantics_unresolved"
]
},
"vampi": {
"languages": {
"python": 25
},
"language_count": 1,
"questions_total": 35,
"questions_completed": 35,
"findings": 4,
"degradations": []
},
"pmpro-n25": {
"languages": {
"php": 4425,
"javascript": 36
},
"language_count": 2,
"questions_total": 127,
"questions_completed": 0,
"findings": 19,
"degradations": [
"cross_partition_semantics_unresolved",
"partition_file_census_incomplete"
]
},
"pmpro-n100": {
"languages": {
"php": 4425,
"javascript": 36
},
"language_count": 2,
"questions_total": 520,
"questions_completed": 0,
"findings": 43,
"degradations": [
"cross_partition_semantics_unresolved",
"partition_file_census_incomplete"
]
}
},
"the_reading": "NodeGoat is JavaScript alone and completes 299 of 299. VAmPI is Python alone and completes 35 of 35. PMPro is PHP with 36 regions of bundled JavaScript and completes 0 of 127 and 0 of 520, while establishing 19 and 43 findings. The only difference between the subjects that complete and the one that does not is the number of languages. NodeGoat's own degradation, `vendor_semantics_unresolved`, is a DECLARED exclusion rather than an integrity gap, which is exactly why it does not demote.",
"why_this_matters_for_the_release": "The declared v0.1 target is WordPress plugins, and a WordPress plugin is PHP plus its admin JavaScript. Every such repository is multi-language, so on the declared target the analyser can detect defects and can never admit one. That is a larger obstacle than the PHP runtime limit, and it is invisible from either subject that currently passes.",
"why_the_gap_exists": "A flow that crosses a language boundary is not modelled: PHP calling into JavaScript, or a script posting to an endpoint, is semantics the partitioned graph cannot follow. Recording that is right.",
"why_the_response_is_too_coarse": "The gap demotes every question, including one wholly inside a single PHP file with no cross-language element in its scope. This project has already fixed this bug class once, at the question level: task 12.3 made an empty-scope gap question-owned 'so it cannot block another question's comparison'. The same principle applies one level up. A partition boundary should demote the questions whose scope could cross it, not every question in the repository.",
"what_to_do": "Scope the partition boundary to the questions it can actually affect, and leave a question whose scope lies inside one partition completable. Do not simply reclassify the reason as a declared exclusion: that would silently claim cross-language flows are out of scope rather than unresolved.",
"limits": [
"Three subjects, one of them multi-language. A single-language PHP repository has not been measured and would isolate the cause further.",
"`partition_file_census_incomplete` is present on PMPro too and is also an integrity gap, so fixing only the cross-partition reason may not be enough on this subject."
]
}