homemaker-layout/experiments/results
Claude 50fb293fa5
Re-verify 1ph: the iio bug could never have touched it
d86 asked for the rigorous version of 35's spot-check -- check out the
codebase at the 1ph commit, backport the iio stale-leaf-share fix, re-run
the historical seed sets.

One constraint had to be worked around: this repository's history begins
2026-07-30, six days AFTER the 1ph commit (2026-07-24), so that checkout
does not exist here. Closest reachable stand-in is 391f510 (2026-07-30), a
genuine ancestor of the iio fix and therefore pre-iio, carrying that era's
examples/ and objective. Measured there, not at the true 1ph commit, and
that is a real limitation of the reproduction.

Protocol as published: programme-house, budget 3000, 4 workers, seeds 1-20,
ON vs OFF, both arms finished with --collapse. Run twice over one worktree,
as-is then with the 22-line iio hunk applied.

                            OFF    ON     W/L/T    mean diff  t(df=19)
  published 1ph             7.95   7.10   11/6/3   +0.85      2.38
  pre-iio 391f510           8.05   7.10   11/6/3   +0.95      2.59
  same + iio fix            8.05   7.10   11/6/3   +0.95      2.59

The published verdict reproduces, and the iio fix changes nothing: 0 of 40
(seed, arm) cells differ, per-seed counts equal cell by cell.

It could not have been otherwise. The bug needs a leaf carrying a STALE
share/share_type, and leaf-sharing only stamps a share where a code has
count > 1 -- programme-house declares count: 1 for all six codes. Over 8
constructed seeds at that commit, programme-house had share_type set on 0
of 56 leaves; harbor-house on 24 of 128. _collapse_value reads
leaf.share_type, so the bug is structurally unreachable on the 1ph
protocol. That is why 35 saw 2/3 harbor seeds diverge by 5-8 fails while
programme-house at N=20 moves not one cell: harbor has codes at counts 10,
6 and 5.

20's retroactive caveat is discharged for the 1ph section and stays live
for harbor-house/qpk, where shares exist and divergence was measured.

Kept experiments/rerun_1ph_protocol.sh and both result TSVs.

Closes homemaker-py-d86.

Lint at parity (46); tests 384 passed, 0 failed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
2026-08-29 13:21:29 +00:00
..
ab_cpsat_assign_20k_harbor_maple.log homemaker-py-2g7.5: full acceptance-criteria A/B (harbor+maple, 3 seeds, 20k budget) 2026-08-05 00:01:00 +01:00
ab_ssz_power.csv ssz: 61% of reported crinkliness fails are not defects; two corrections 2026-08-26 18:07:01 +00:00
ab_ssz_search.csv ssz: record the A/B result honestly -- not a pass at n=3 2026-08-26 17:57:11 +00:00
coldstart_baseline.tsv Checkpoint long searches; the cold-start runs were lost to a reclaimed box 2026-08-29 05:43:58 +00:00
d86_1ph_iiofix.tsv Re-verify 1ph: the iio bug could never have touched it 2026-08-29 13:21:29 +00:00
d86_1ph_preiio.tsv Re-verify 1ph: the iio bug could never have touched it 2026-08-29 13:21:29 +00:00