caf480aa32
0wr asked which harbor A/Bs were decided by a narrow margin before 39.4. Measuring harbor's variance makes the margin-by-margin triage moot. Harbor's paired seed-to-seed sd is 6.19 fails (24 paired ON/OFF runs, budget 2500), so n=3 resolves nothing finer than 15.4 fails. Every recorded harbor margin is below that: 13.9 share_edge_cap 3.7, 20 qpk 8.3, 23 f1d mixed, 37.1 tiering 6.3. So it is not that SOME harbor results were narrow -- no harbor A/B run at three seeds could resolve the margin it reported, independently of what 39.4 did to the programme. Of the 220 possible 3-seed subsets of the 24 runs, 56 (25%) show a clean 3/3 sweep for ON. Re-measured 20's harbor arm, the one backing a live default: N=3 2W/1L/0T +2.67 p=0.560 N=12 8W/3L/1T +3.50 p=0.076 N=24 13W/10L/1T +1.21 p=0.502 CI [-2.46,+4.88] Null. The published "harbor: ON wins 3/3, 80.3 -> 72.0" was a lucky draw -- even seeds 1-3 measured here give 2W/1L, not a sweep. So 20's claim that the qpk verdict "holds at both example scales tested" is withdrawn and annotated in place. collapse_insearch's default rests on programme-house alone (38.19, N=60, +0.57, p=0.017). It is not refuted on harbor -- direction positive but indistinguishable from zero -- but harbor must not be cited as corroboration. Harness generalised (PROG/BUDGET/WORKERS) and results kept. Filed homemaker-py-... : A/B harnesses should report the minimum detectable difference for the N they run, so an underpowered verdict is visible when it is made rather than years later. Closes homemaker-py-0wr. Lint at parity (46). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
924 B
924 B
| 1 | harbor | 1 | on | 96 | 52 |
|---|---|---|---|---|---|
| 2 | harbor | 1 | off | 103 | 24 |
| 3 | harbor | 2 | on | 81 | 53 |
| 4 | harbor | 2 | off | 76 | 25 |
| 5 | harbor | 3 | on | 91 | 55 |
| 6 | harbor | 3 | off | 97 | 26 |
| 7 | harbor | 4 | on | 99 | 54 |
| 8 | harbor | 4 | off | 91 | 25 |
| 9 | harbor | 5 | on | 82 | 54 |
| 10 | harbor | 5 | off | 82 | 25 |
| 11 | harbor | 6 | on | 90 | 53 |
| 12 | harbor | 6 | off | 95 | 25 |
| 13 | harbor | 7 | on | 88 | 55 |
| 14 | harbor | 7 | off | 87 | 25 |
| 15 | harbor | 8 | on | 90 | 53 |
| 16 | harbor | 8 | off | 100 | 27 |
| 17 | harbor | 9 | on | 87 | 53 |
| 18 | harbor | 9 | off | 98 | 26 |
| 19 | harbor | 10 | on | 86 | 55 |
| 20 | harbor | 10 | off | 97 | 26 |
| 21 | harbor | 11 | on | 79 | 56 |
| 22 | harbor | 11 | off | 84 | 25 |
| 23 | harbor | 12 | on | 102 | 52 |
| 24 | harbor | 12 | off | 103 | 26 |
| 25 | harbor | 13 | on | 93 | 53 |
| 26 | harbor | 13 | off | 98 | 24 |
| 27 | harbor | 14 | on | 77 | 53 |
| 28 | harbor | 14 | off | 83 | 25 |
| 29 | harbor | 15 | on | 76 | 53 |
| 30 | harbor | 15 | off | 75 | 26 |
| 31 | harbor | 16 | on | 102 | 56 |
| 32 | harbor | 16 | off | 87 | 26 |
| 33 | harbor | 17 | on | 78 | 55 |
| 34 | harbor | 17 | off | 89 | 27 |
| 35 | harbor | 18 | on | 89 | 54 |
| 36 | harbor | 18 | off | 83 | 26 |
| 37 | harbor | 19 | on | 102 | 53 |
| 38 | harbor | 19 | off | 82 | 24 |
| 39 | harbor | 20 | on | 72 | 51 |
| 40 | harbor | 20 | off | 74 | 26 |
| 41 | harbor | 21 | on | 89 | 54 |
| 42 | harbor | 21 | off | 82 | 24 |
| 43 | harbor | 22 | on | 84 | 54 |
| 44 | harbor | 22 | off | 80 | 25 |
| 45 | harbor | 23 | on | 91 | 53 |
| 46 | harbor | 23 | off | 90 | 26 |
| 47 | harbor | 24 | on | 81 | 53 |
| 48 | harbor | 24 | off | 98 | 27 |