Re-verify 1ph: the iio bug could never have touched it
d86 asked for the rigorous version of 35's spot-check -- check out the codebase at the 1ph commit, backport the iio stale-leaf-share fix, re-run the historical seed sets. One constraint had to be worked around: this repository's history begins 2026-07-30, six days AFTER the 1ph commit (2026-07-24), so that checkout does not exist here. Closest reachable stand-in is391f510(2026-07-30), a genuine ancestor of the iio fix and therefore pre-iio, carrying that era's examples/ and objective. Measured there, not at the true 1ph commit, and that is a real limitation of the reproduction. Protocol as published: programme-house, budget 3000, 4 workers, seeds 1-20, ON vs OFF, both arms finished with --collapse. Run twice over one worktree, as-is then with the 22-line iio hunk applied. OFF ON W/L/T mean diff t(df=19) published 1ph 7.95 7.10 11/6/3 +0.85 2.38 pre-iio391f5108.05 7.10 11/6/3 +0.95 2.59 same + iio fix 8.05 7.10 11/6/3 +0.95 2.59 The published verdict reproduces, and the iio fix changes nothing: 0 of 40 (seed, arm) cells differ, per-seed counts equal cell by cell. It could not have been otherwise. The bug needs a leaf carrying a STALE share/share_type, and leaf-sharing only stamps a share where a code has count > 1 -- programme-house declares count: 1 for all six codes. Over 8 constructed seeds at that commit, programme-house had share_type set on 0 of 56 leaves; harbor-house on 24 of 128. _collapse_value reads leaf.share_type, so the bug is structurally unreachable on the 1ph protocol. That is why 35 saw 2/3 harbor seeds diverge by 5-8 fails while programme-house at N=20 moves not one cell: harbor has codes at counts 10, 6 and 5. 20's retroactive caveat is discharged for the 1ph section and stays live for harbor-house/qpk, where shares exist and divergence was measured. Kept experiments/rerun_1ph_protocol.sh and both result TSVs. Closes homemaker-py-d86. Lint at parity (46); tests 384 passed, 0 failed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
This commit is contained in:
parent
f6eeee7191
commit
50fb293fa5
5 changed files with 167 additions and 1 deletions
File diff suppressed because one or more lines are too long
60
DESIGN.md
60
DESIGN.md
|
|
@ -2784,6 +2784,13 @@ is real per-seed noise (not a directional bias) that does not appear to overturn
|
|||
verdict above, but the exact historical per-seed numbers quoted in this section were not re-measured
|
||||
under the fix. See §35 for the mechanism and what was (and wasn't) re-confirmed.
|
||||
|
||||
**Caveat DISCHARGED for this section (`homemaker-py-d86`, §38.18).** The 1ph protocol has now been
|
||||
re-run at N=20 on a pre-iio commit with and without the fix backported: all 40 (seed, arm) cells are
|
||||
identical, and the reason is structural rather than lucky — programme-house declares `count: 1` for
|
||||
every code, so no leaf ever acquires a share and the stale-share bug cannot fire on this programme
|
||||
at all. The numbers above stand. The caveat remains live for harbor-house/qpk, where shares do exist
|
||||
and §35 measured real divergence.
|
||||
|
||||
## 21. Insert/relocate-circulation repair operator (`homemaker-py-8sh`) — DONE (mixed, kept off)
|
||||
|
||||
**Motivation.** qi6's remaining candidate (§18): mechanism (a), an explicit search-time
|
||||
|
|
@ -5778,6 +5785,59 @@ over 2/3/4 workers, asserting each is internally stable and deliberately not
|
|||
asserting they agree with each other) and
|
||||
`test_scoring_a_frozen_design_is_deterministic`.
|
||||
|
||||
### 38.18 The 1ph verdict re-verified: the iio bug could never have touched it (`homemaker-py-d86`)
|
||||
|
||||
`d86` asked for the rigorous version of §35's spot-check: take the codebase at the
|
||||
`1ph` commit, backport the `iio` stale-leaf-share fix, and re-run the actual
|
||||
historical seed sets to see whether the published verdict would have changed.
|
||||
|
||||
**One constraint had to be worked around.** This repository's history begins
|
||||
**2026-07-30**, six days *after* the 1ph commit (2026-07-24) — the historical
|
||||
checkout the issue asks for does not exist here. The closest reachable stand-in
|
||||
is `391f510` (2026-07-30), which is a genuine ancestor of the iio fix
|
||||
(`929be5b`, 2026-08-01) and therefore pre-iio, and which carries that era's
|
||||
`examples/` and objective. Everything below is measured there, not at the true
|
||||
1ph commit, and that is a real limitation of the reproduction.
|
||||
|
||||
**Protocol as published:** programme-house `init.dom`, budget 3000, 4 workers,
|
||||
seeds 1–20, ON vs OFF, both arms finished with `--collapse`. Run twice over the
|
||||
same worktree — once as-is, once with the 22-line `iio` `fitness.py` hunk applied.
|
||||
|
||||
| codebase | OFF | ON | W/L/T | mean diff | t (df=19) |
|
||||
|---|---|---|---|---|---|
|
||||
| published 1ph (2026-07-24) | 7.95 | 7.10 | 11/6/3 | +0.85 | 2.38 |
|
||||
| pre-iio `391f510` | **8.05** | **7.10** | **11/6/3** | **+0.95** | **2.59** |
|
||||
| the same, + iio fix backported | 8.05 | 7.10 | 11/6/3 | +0.95 | 2.59 |
|
||||
|
||||
Two results:
|
||||
|
||||
1. **The published verdict reproduces.** ON beats OFF, significant at N=20, with
|
||||
a win/loss/tie split identical to the published 11/6/3.
|
||||
2. **The iio fix changes nothing — 0 of 40 (seed, arm) cells differ.** Not a
|
||||
coincidence of means: the per-seed fail counts are equal cell by cell.
|
||||
|
||||
**And it could not have been otherwise.** The iio bug needs a leaf carrying a
|
||||
*stale* `share`/`share_type` — leftover multiplicity from a code the leaf has
|
||||
since been retyped away from. Leaf-sharing only ever stamps a share when a code
|
||||
has `count > 1`, and **programme-house declares `count: 1` for all six of its
|
||||
codes**. Measured directly over 8 constructed seeds at that commit:
|
||||
|
||||
| programme | leaves | `share > 1` | `share_type` set |
|
||||
|---|---|---|---|
|
||||
| programme-house | 56 | **0** | **0** |
|
||||
| harbor-house | 128 | 24 | 24 |
|
||||
|
||||
`_collapse_value` reads `leaf.share_type`; on programme-house it is never set,
|
||||
so the bug is **structurally unreachable on the 1ph protocol**. That is why §35
|
||||
saw 2 of 3 harbor seeds diverge by 5–8 fails while programme-house at N=20 moves
|
||||
not one cell — harbor has codes at counts 10, 6 and 5.
|
||||
|
||||
§20's retroactive caveat is therefore **discharged for the 1ph section** and
|
||||
stays live for harbor-house/qpk, where shares exist and the divergence was
|
||||
measured. Worth noting for future archaeology: "was this measurement affected by
|
||||
bug X" is often answerable from the programme's structure without re-running
|
||||
anything.
|
||||
|
||||
## 39. Config audit: requirements that actively fight the engine (`homemaker-py-ju3`) — measured 2026-08-25
|
||||
|
||||
The corpus `patterns.config` targets and `costs.config` values were estimated
|
||||
|
|
|
|||
26
experiments/rerun_1ph_protocol.sh
Executable file
26
experiments/rerun_1ph_protocol.sh
Executable file
|
|
@ -0,0 +1,26 @@
|
|||
#!/usr/bin/env bash
|
||||
# Re-run the 1ph protocol (DESIGN.md §20): programme-house init.dom, budget 3000,
|
||||
# 4 workers, seeds 1-20, ON vs OFF, BOTH arms finished with --collapse.
|
||||
set -u
|
||||
W="$1"; TAG="$2"; OUT="$3"
|
||||
cd "$W/examples/programme-house"
|
||||
: > "$OUT"
|
||||
for seed in $(seq 1 20); do
|
||||
for arm in on off; do
|
||||
flag=""; [ "$arm" = "off" ] && flag="--no-collapse-insearch"
|
||||
t0=$(date +%s)
|
||||
PYTHONPATH="$W/src" timeout 600 python -m homemaker_layout.evolve init.dom \
|
||||
--budget 3000 --seed "$seed" --workers 4 --collapse $flag \
|
||||
--output "$OUT.$arm.dom" > "$OUT.$arm.log" 2>&1
|
||||
t1=$(date +%s)
|
||||
n=$(PYTHONPATH="$W/src" python - "$OUT.$arm.dom" <<'PY'
|
||||
import copy, sys
|
||||
from homemaker_layout import dom, fitness
|
||||
conf, cost = fitness.load_config(".")
|
||||
_, f = fitness.Fitness(conf, cost).score_with_fails(copy.deepcopy(dom.load(sys.argv[1])))
|
||||
print(len(f))
|
||||
PY
|
||||
)
|
||||
echo -e "$TAG\t$seed\t$arm\t$n\t$((t1-t0))" >> "$OUT"
|
||||
done
|
||||
done
|
||||
40
experiments/results/d86_1ph_iiofix.tsv
Normal file
40
experiments/results/d86_1ph_iiofix.tsv
Normal file
|
|
@ -0,0 +1,40 @@
|
|||
iiofix 1 on 5 8
|
||||
iiofix 1 off 8 6
|
||||
iiofix 2 on 7 8
|
||||
iiofix 2 off 8 6
|
||||
iiofix 3 on 9 8
|
||||
iiofix 3 off 8 6
|
||||
iiofix 4 on 9 7
|
||||
iiofix 4 off 11 5
|
||||
iiofix 5 on 9 8
|
||||
iiofix 5 off 8 7
|
||||
iiofix 6 on 8 7
|
||||
iiofix 6 off 11 5
|
||||
iiofix 7 on 9 8
|
||||
iiofix 7 off 11 6
|
||||
iiofix 8 on 8 8
|
||||
iiofix 8 off 7 7
|
||||
iiofix 9 on 6 7
|
||||
iiofix 9 off 9 6
|
||||
iiofix 10 on 4 8
|
||||
iiofix 10 off 7 7
|
||||
iiofix 11 on 5 7
|
||||
iiofix 11 off 5 6
|
||||
iiofix 12 on 7 8
|
||||
iiofix 12 off 6 7
|
||||
iiofix 13 on 6 7
|
||||
iiofix 13 off 9 6
|
||||
iiofix 14 on 8 8
|
||||
iiofix 14 off 7 6
|
||||
iiofix 15 on 6 8
|
||||
iiofix 15 off 8 6
|
||||
iiofix 16 on 8 7
|
||||
iiofix 16 off 8 6
|
||||
iiofix 17 on 6 7
|
||||
iiofix 17 off 8 6
|
||||
iiofix 18 on 7 7
|
||||
iiofix 18 off 8 6
|
||||
iiofix 19 on 8 7
|
||||
iiofix 19 off 7 6
|
||||
iiofix 20 on 7 7
|
||||
iiofix 20 off 7 6
|
||||
|
40
experiments/results/d86_1ph_preiio.tsv
Normal file
40
experiments/results/d86_1ph_preiio.tsv
Normal file
|
|
@ -0,0 +1,40 @@
|
|||
preiio 1 on 5 8
|
||||
preiio 1 off 8 6
|
||||
preiio 2 on 7 7
|
||||
preiio 2 off 8 6
|
||||
preiio 3 on 9 7
|
||||
preiio 3 off 8 7
|
||||
preiio 4 on 9 8
|
||||
preiio 4 off 11 6
|
||||
preiio 5 on 9 8
|
||||
preiio 5 off 8 7
|
||||
preiio 6 on 8 8
|
||||
preiio 6 off 11 6
|
||||
preiio 7 on 9 7
|
||||
preiio 7 off 11 6
|
||||
preiio 8 on 8 7
|
||||
preiio 8 off 7 6
|
||||
preiio 9 on 6 8
|
||||
preiio 9 off 9 5
|
||||
preiio 10 on 4 7
|
||||
preiio 10 off 7 7
|
||||
preiio 11 on 5 7
|
||||
preiio 11 off 5 6
|
||||
preiio 12 on 7 8
|
||||
preiio 12 off 6 7
|
||||
preiio 13 on 6 8
|
||||
preiio 13 off 9 6
|
||||
preiio 14 on 8 8
|
||||
preiio 14 off 7 6
|
||||
preiio 15 on 6 8
|
||||
preiio 15 off 8 6
|
||||
preiio 16 on 8 7
|
||||
preiio 16 off 8 5
|
||||
preiio 17 on 6 7
|
||||
preiio 17 off 8 6
|
||||
preiio 18 on 7 7
|
||||
preiio 18 off 8 5
|
||||
preiio 19 on 8 7
|
||||
preiio 19 off 7 5
|
||||
preiio 20 on 7 7
|
||||
preiio 20 off 7 6
|
||||
|
Loading…
Add table
Reference in a new issue