bruno/homemaker-layout
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Record the ecx A/B: directionally positive, not resolvable at N=6

Browse source 
Harbor and maple, three 500k plateau starts x two RNG seeds, 8000 evals, all
arms scored under stock. Run against the pre-39.19 defaults, so "stock" is the
old objective.

  harbor  geomean          39.17 -> 39.00  1W/0L/5T  diff +0.167  MDD 0.428
  harbor  geomean+daylight 39.17 -> 39.00  1W/0L/5T  diff +0.167  MDD 0.428
  maple   geomean          60.50 -> 60.17  1W/1L/4T  diff +0.333  MDD 1.434
  maple   geomean+daylight 60.50 -> 60.00  1W/0L/5T  diff +0.500  MDD 1.285

Every margin sits below what N=6 can resolve (N ~= 26-67 needed). Unlike
39.13's ramp, which was an exact null with sd=0 and identical trajectories, the
arms here do diverge and all four land on the favourable side of zero -- worth
exactly what four coin-flips landing the same way is worth. Recorded so a later
run cannot cite it as evidence.

Also recorded: none of 39.13, 39.14 or 39.18 produced a measurable search
improvement at this budget. All were shipped on correctness grounds and 39.19
then required two of them outright. If the 39 objective changes pay off in
search terms, this corpus has not shown it yet.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
This commit is contained in:
Claude 2026-09-06 07:38:23 +00:00
parent 7047a782e5
commit f47dfe60af
No known key found for this signature in database
2 changed files with 65 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

28
DESIGN.md
View file

@ -7597,6 +7597,34 @@ programmes where those reached two:
| room | 0.34 | **0.66** |
| circulation | 0.02 | **0.07** |
**A/B verdict: directionally positive, not resolvable at this budget.** Same
protocol as §39.13/§39.14 — harbor and maple, three 500 k plateau starts × two
RNG seeds, 8000 evals, all arms scored under stock (valid here by
construction, since no aggregation can move a fail). Run against the
*pre-§39.19* defaults, so `stock` really is the old objective:
```
harbor geomean vs stock N=6 39.17 -> 39.00 1W/0L/5T diff +0.167 MDD 0.428
harbor geomean+daylight vs stock N=6 39.17 -> 39.00 1W/0L/5T diff +0.167 MDD 0.428
maple geomean vs stock N=6 60.50 -> 60.17 1W/1L/4T diff +0.333 MDD 1.434
maple geomean+daylight vs stock N=6 60.50 -> 60.00 1W/0L/5T diff +0.500 MDD 1.285
```
Every margin is below what N=6 can resolve; N≈26–67 would be needed. Unlike
§39.13's ramp — which was an *exact* null, 0W/0L/6T with sd=0 and identical
trajectories — the arms here genuinely diverge and every one of the four lands
on the favourable side of zero. That is worth exactly as much as it sounds:
four coin-flips landing the same way. It is not evidence, and it is recorded
here so that a later run cannot cite it as if it were.
**The honest summary of this section's three A/Bs**: none of §39.13, §39.14 or
§39.18 produced a measurable search improvement at an 8000-eval plateau budget.
All three were shipped on correctness grounds — an unrankable tail, a
double-charge, an exemption that bought quality — and §39.19 then required two
of them outright. If the objective changes in §39 pay off in search terms, this
corpus has not yet shown it, and the powered protocol in
`experiments/ab_9gj_crinkliness.py` is what would.
**What it does not fix, deliberately.** A terrace still out-earns a room 4:1.
That residue is the rates, not the aggregation: `value_supported` is 300, the
same as `value_inside`, while an upper outside leaf costs 110 against a room's

37
experiments/results/ab_ecx.csv Normal file
View file

@ -0,0 +1,37 @@
programme,start,seed,tail,hard,soft,total,score,elapsed_s
harbor-house,coldstart-500000-s0.dom,0,stock,8,25,33,1.85176240526207e-11,406.1
harbor-house,coldstart-500000-s0.dom,0,geomean,8,25,33,1.3331368428113424e-11,408.8
harbor-house,coldstart-500000-s0.dom,0,geomean+daylight,8,25,33,1.3331474034308951e-11,486.4
harbor-house,coldstart-500000-s2.dom,0,stock,12,30,42,2.0876504874272412e-14,434.0
harbor-house,coldstart-500000-s2.dom,0,geomean,12,29,41,4.905855471687939e-14,360.9
harbor-house,coldstart-500000-s2.dom,0,geomean+daylight,12,30,42,2.0270861698600025e-14,356.8
maple-court,coldstart-500000-s1.dom,0,stock,17,56,73,3.549637383437798e-25,438.8
maple-court,coldstart-500000-s1.dom,0,geomean,17,56,73,3.4927712102261773e-25,451.9
maple-court,coldstart-500000-s1.dom,0,geomean+daylight,17,56,73,3.490922156298438e-25,473.2
harbor-house,coldstart-500000-s0.dom,1,stock,8,24,32,3.749216704165142e-11,414.2
harbor-house,coldstart-500000-s0.dom,1,geomean,8,24,32,2.7696350036240933e-11,426.3
harbor-house,coldstart-500000-s0.dom,1,geomean+daylight,8,24,32,2.6179276610885845e-11,496.2
harbor-house,coldstart-500000-s2.dom,1,stock,12,30,42,2.0876504874272412e-14,446.5
harbor-house,coldstart-500000-s2.dom,1,geomean,12,30,42,1.987950485765416e-14,365.1
harbor-house,coldstart-500000-s2.dom,1,geomean+daylight,12,29,41,3.0629342889555544e-14,354.0
maple-court,coldstart-500000-s1.dom,1,stock,17,56,73,3.5495919599065193e-25,438.9
maple-court,coldstart-500000-s1.dom,1,geomean,17,56,73,3.5024500858428627e-25,451.5
maple-court,coldstart-500000-s1.dom,1,geomean+daylight,17,56,73,3.491084124317304e-25,447.6
harbor-house,coldstart-500000-s1.dom,0,stock,7,36,43,1.4676873187056738e-15,368.6
harbor-house,coldstart-500000-s1.dom,0,geomean,7,36,43,1.4665664078808208e-15,372.1
harbor-house,coldstart-500000-s1.dom,0,geomean+daylight,7,36,43,1.2492266264690038e-15,462.0
maple-court,coldstart-500000-s0.dom,0,stock,18,35,53,9.987013209179406e-18,546.9
maple-court,coldstart-500000-s0.dom,0,geomean,18,35,53,9.60106526960439e-18,462.1
maple-court,coldstart-500000-s0.dom,0,geomean+daylight,18,35,53,9.763084974756856e-18,460.8
maple-court,coldstart-500000-s2.dom,0,stock,12,43,55,2.9415325131166865e-18,479.3
maple-court,coldstart-500000-s2.dom,0,geomean,12,44,56,1.496080486892727e-18,481.9
maple-court,coldstart-500000-s2.dom,0,geomean+daylight,11,44,55,3.1185722410760797e-18,479.8
harbor-house,coldstart-500000-s1.dom,1,stock,7,36,43,1.4676873187056738e-15,381.1
harbor-house,coldstart-500000-s1.dom,1,geomean,7,36,43,1.4465496601047535e-15,377.1
harbor-house,coldstart-500000-s1.dom,1,geomean+daylight,7,36,43,1.2455994055175617e-15,465.2
maple-court,coldstart-500000-s0.dom,1,stock,17,37,54,5.878387691540865e-18,547.7
maple-court,coldstart-500000-s0.dom,1,geomean,18,36,54,5.1384167541013474e-18,487.6
maple-court,coldstart-500000-s0.dom,1,geomean+daylight,17,37,54,5.537746091126898e-18,503.0
maple-court,coldstart-500000-s2.dom,1,stock,12,43,55,2.931601349417379e-18,506.2
maple-court,coldstart-500000-s2.dom,1,geomean,12,40,52,2.12145776572736e-17,493.1
maple-court,coldstart-500000-s2.dom,1,geomean+daylight,12,40,52,2.288105258404665e-17,493.3
1 programme start seed tail hard soft total score elapsed_s
2 harbor-house coldstart-500000-s0.dom 0 stock 8 25 33 1.85176240526207e-11 406.1
3 harbor-house coldstart-500000-s0.dom 0 geomean 8 25 33 1.3331368428113424e-11 408.8
4 harbor-house coldstart-500000-s0.dom 0 geomean+daylight 8 25 33 1.3331474034308951e-11 486.4
5 harbor-house coldstart-500000-s2.dom 0 stock 12 30 42 2.0876504874272412e-14 434.0
6 harbor-house coldstart-500000-s2.dom 0 geomean 12 29 41 4.905855471687939e-14 360.9
7 harbor-house coldstart-500000-s2.dom 0 geomean+daylight 12 30 42 2.0270861698600025e-14 356.8
8 maple-court coldstart-500000-s1.dom 0 stock 17 56 73 3.549637383437798e-25 438.8
9 maple-court coldstart-500000-s1.dom 0 geomean 17 56 73 3.4927712102261773e-25 451.9
10 maple-court coldstart-500000-s1.dom 0 geomean+daylight 17 56 73 3.490922156298438e-25 473.2
11 harbor-house coldstart-500000-s0.dom 1 stock 8 24 32 3.749216704165142e-11 414.2
12 harbor-house coldstart-500000-s0.dom 1 geomean 8 24 32 2.7696350036240933e-11 426.3
13 harbor-house coldstart-500000-s0.dom 1 geomean+daylight 8 24 32 2.6179276610885845e-11 496.2
14 harbor-house coldstart-500000-s2.dom 1 stock 12 30 42 2.0876504874272412e-14 446.5
15 harbor-house coldstart-500000-s2.dom 1 geomean 12 30 42 1.987950485765416e-14 365.1
16 harbor-house coldstart-500000-s2.dom 1 geomean+daylight 12 29 41 3.0629342889555544e-14 354.0
17 maple-court coldstart-500000-s1.dom 1 stock 17 56 73 3.5495919599065193e-25 438.9
18 maple-court coldstart-500000-s1.dom 1 geomean 17 56 73 3.5024500858428627e-25 451.5
19 maple-court coldstart-500000-s1.dom 1 geomean+daylight 17 56 73 3.491084124317304e-25 447.6
20 harbor-house coldstart-500000-s1.dom 0 stock 7 36 43 1.4676873187056738e-15 368.6
21 harbor-house coldstart-500000-s1.dom 0 geomean 7 36 43 1.4665664078808208e-15 372.1
22 harbor-house coldstart-500000-s1.dom 0 geomean+daylight 7 36 43 1.2492266264690038e-15 462.0
23 maple-court coldstart-500000-s0.dom 0 stock 18 35 53 9.987013209179406e-18 546.9
24 maple-court coldstart-500000-s0.dom 0 geomean 18 35 53 9.60106526960439e-18 462.1
25 maple-court coldstart-500000-s0.dom 0 geomean+daylight 18 35 53 9.763084974756856e-18 460.8
26 maple-court coldstart-500000-s2.dom 0 stock 12 43 55 2.9415325131166865e-18 479.3
27 maple-court coldstart-500000-s2.dom 0 geomean 12 44 56 1.496080486892727e-18 481.9
28 maple-court coldstart-500000-s2.dom 0 geomean+daylight 11 44 55 3.1185722410760797e-18 479.8
29 harbor-house coldstart-500000-s1.dom 1 stock 7 36 43 1.4676873187056738e-15 381.1
30 harbor-house coldstart-500000-s1.dom 1 geomean 7 36 43 1.4465496601047535e-15 377.1
31 harbor-house coldstart-500000-s1.dom 1 geomean+daylight 7 36 43 1.2455994055175617e-15 465.2
32 maple-court coldstart-500000-s0.dom 1 stock 17 37 54 5.878387691540865e-18 547.7
33 maple-court coldstart-500000-s0.dom 1 geomean 18 36 54 5.1384167541013474e-18 487.6
34 maple-court coldstart-500000-s0.dom 1 geomean+daylight 17 37 54 5.537746091126898e-18 503.0
35 maple-court coldstart-500000-s2.dom 1 stock 12 43 55 2.931601349417379e-18 506.2
36 maple-court coldstart-500000-s2.dom 1 geomean 12 40 52 2.12145776572736e-17 493.1
37 maple-court coldstart-500000-s2.dom 1 geomean+daylight 12 40 52 2.288105258404665e-17 493.3