caf480aa32
0wr asked which harbor A/Bs were decided by a narrow margin before 39.4. Measuring harbor's variance makes the margin-by-margin triage moot. Harbor's paired seed-to-seed sd is 6.19 fails (24 paired ON/OFF runs, budget 2500), so n=3 resolves nothing finer than 15.4 fails. Every recorded harbor margin is below that: 13.9 share_edge_cap 3.7, 20 qpk 8.3, 23 f1d mixed, 37.1 tiering 6.3. So it is not that SOME harbor results were narrow -- no harbor A/B run at three seeds could resolve the margin it reported, independently of what 39.4 did to the programme. Of the 220 possible 3-seed subsets of the 24 runs, 56 (25%) show a clean 3/3 sweep for ON. Re-measured 20's harbor arm, the one backing a live default: N=3 2W/1L/0T +2.67 p=0.560 N=12 8W/3L/1T +3.50 p=0.076 N=24 13W/10L/1T +1.21 p=0.502 CI [-2.46,+4.88] Null. The published "harbor: ON wins 3/3, 80.3 -> 72.0" was a lucky draw -- even seeds 1-3 measured here give 2W/1L, not a sweep. So 20's claim that the qpk verdict "holds at both example scales tested" is withdrawn and annotated in place. collapse_insearch's default rests on programme-house alone (38.19, N=60, +0.57, p=0.017). It is not refuted on harbor -- direction positive but indistinguishable from zero -- but harbor must not be cited as corroboration. Harness generalised (PROG/BUDGET/WORKERS) and results kept. Filed homemaker-py-... : A/B harnesses should report the minimum detectable difference for the N they run, so an underpowered verdict is visible when it is made rather than years later. Closes homemaker-py-0wr. Lint at parity (46). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
48 lines
924 B
Text
48 lines
924 B
Text
harbor 1 on 96 52
|
|
harbor 1 off 103 24
|
|
harbor 2 on 81 53
|
|
harbor 2 off 76 25
|
|
harbor 3 on 91 55
|
|
harbor 3 off 97 26
|
|
harbor 4 on 99 54
|
|
harbor 4 off 91 25
|
|
harbor 5 on 82 54
|
|
harbor 5 off 82 25
|
|
harbor 6 on 90 53
|
|
harbor 6 off 95 25
|
|
harbor 7 on 88 55
|
|
harbor 7 off 87 25
|
|
harbor 8 on 90 53
|
|
harbor 8 off 100 27
|
|
harbor 9 on 87 53
|
|
harbor 9 off 98 26
|
|
harbor 10 on 86 55
|
|
harbor 10 off 97 26
|
|
harbor 11 on 79 56
|
|
harbor 11 off 84 25
|
|
harbor 12 on 102 52
|
|
harbor 12 off 103 26
|
|
harbor 13 on 93 53
|
|
harbor 13 off 98 24
|
|
harbor 14 on 77 53
|
|
harbor 14 off 83 25
|
|
harbor 15 on 76 53
|
|
harbor 15 off 75 26
|
|
harbor 16 on 102 56
|
|
harbor 16 off 87 26
|
|
harbor 17 on 78 55
|
|
harbor 17 off 89 27
|
|
harbor 18 on 89 54
|
|
harbor 18 off 83 26
|
|
harbor 19 on 102 53
|
|
harbor 19 off 82 24
|
|
harbor 20 on 72 51
|
|
harbor 20 off 74 26
|
|
harbor 21 on 89 54
|
|
harbor 21 off 82 24
|
|
harbor 22 on 84 54
|
|
harbor 22 off 80 25
|
|
harbor 23 on 91 53
|
|
harbor 23 off 90 26
|
|
harbor 24 on 81 53
|
|
harbor 24 off 98 27
|