homemaker-layout/experiments/rerun_1ph_protocol.sh
Claude 50fb293fa5
Re-verify 1ph: the iio bug could never have touched it
d86 asked for the rigorous version of 35's spot-check -- check out the
codebase at the 1ph commit, backport the iio stale-leaf-share fix, re-run
the historical seed sets.

One constraint had to be worked around: this repository's history begins
2026-07-30, six days AFTER the 1ph commit (2026-07-24), so that checkout
does not exist here. Closest reachable stand-in is 391f510 (2026-07-30), a
genuine ancestor of the iio fix and therefore pre-iio, carrying that era's
examples/ and objective. Measured there, not at the true 1ph commit, and
that is a real limitation of the reproduction.

Protocol as published: programme-house, budget 3000, 4 workers, seeds 1-20,
ON vs OFF, both arms finished with --collapse. Run twice over one worktree,
as-is then with the 22-line iio hunk applied.

                            OFF    ON     W/L/T    mean diff  t(df=19)
  published 1ph             7.95   7.10   11/6/3   +0.85      2.38
  pre-iio 391f510           8.05   7.10   11/6/3   +0.95      2.59
  same + iio fix            8.05   7.10   11/6/3   +0.95      2.59

The published verdict reproduces, and the iio fix changes nothing: 0 of 40
(seed, arm) cells differ, per-seed counts equal cell by cell.

It could not have been otherwise. The bug needs a leaf carrying a STALE
share/share_type, and leaf-sharing only stamps a share where a code has
count > 1 -- programme-house declares count: 1 for all six codes. Over 8
constructed seeds at that commit, programme-house had share_type set on 0
of 56 leaves; harbor-house on 24 of 128. _collapse_value reads
leaf.share_type, so the bug is structurally unreachable on the 1ph
protocol. That is why 35 saw 2/3 harbor seeds diverge by 5-8 fails while
programme-house at N=20 moves not one cell: harbor has codes at counts 10,
6 and 5.

20's retroactive caveat is discharged for the 1ph section and stays live
for harbor-house/qpk, where shares exist and divergence was measured.

Kept experiments/rerun_1ph_protocol.sh and both result TSVs.

Closes homemaker-py-d86.

Lint at parity (46); tests 384 passed, 0 failed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
2026-08-29 13:21:29 +00:00

26 lines
945 B
Bash
Executable file

#!/usr/bin/env bash
# Re-run the 1ph protocol (DESIGN.md §20): programme-house init.dom, budget 3000,
# 4 workers, seeds 1-20, ON vs OFF, BOTH arms finished with --collapse.
set -u
W="$1"; TAG="$2"; OUT="$3"
cd "$W/examples/programme-house"
: > "$OUT"
for seed in $(seq 1 20); do
for arm in on off; do
flag=""; [ "$arm" = "off" ] && flag="--no-collapse-insearch"
t0=$(date +%s)
PYTHONPATH="$W/src" timeout 600 python -m homemaker_layout.evolve init.dom \
--budget 3000 --seed "$seed" --workers 4 --collapse $flag \
--output "$OUT.$arm.dom" > "$OUT.$arm.log" 2>&1
t1=$(date +%s)
n=$(PYTHONPATH="$W/src" python - "$OUT.$arm.dom" <<'PY'
import copy, sys
from homemaker_layout import dom, fitness
conf, cost = fitness.load_config(".")
_, f = fitness.Fitness(conf, cost).score_with_fails(copy.deepcopy(dom.load(sys.argv[1])))
print(len(f))
PY
)
echo -e "$TAG\t$seed\t$arm\t$n\t$((t1-t0))" >> "$OUT"
done
done