6405ca1a59
Fixed-budget A/B (3000 evals, 3 seeds, harbor + maple, every arm scored
under stock urb so the permissive modes cannot win by deleting a fail
category).
usage_daylight's paired hard-fail deltas are harbor [0,-1,-10] and maple
[0,+2,-10]. The means (-3.7, -2.7) flatter it: the whole signal is seed 2
in both programmes, and seed 2 is the seed where stock itself does worst.
Two seeds in three are flat or slightly worse. On that seed soft rises as
much as hard falls (harbor -10h/+9s, maple -10h/+15s), so totals go
62->61 on harbor and 120->125 on maple.
Because the scoring is stock, that is a genuine trade of hard failures for
soft ones, not a relabelling -- progress under the tiered comparator, where
n_hard is primary, but a fail against this issue's acceptance criterion
("without inflating soft"). Which yardstick is right is now the live
question, and it is gvb's question as much as ssz's.
usage_daylight stays default off; undecided, not refuted. Higher-power run
(urb vs usage_daylight, 10 seeds) is running.
The diagnostic half stands independent of the search A/B: the objective
demands daylight of two thirds of the buried population that does not want
it, and 38.6's contrary null was an artefact of three modes that never
touched those leaves.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
31 lines
1.6 KiB
CSV
31 lines
1.6 KiB
CSV
programme,seed,mode,urb_hard,urb_soft,urb_total,own_hard,own_soft,own_total,elapsed_s
|
|
harbor-house,0,urb,16,39,55,16,39,55,80.2
|
|
harbor-house,1,urb,22,45,67,22,45,67,88.7
|
|
harbor-house,2,urb,22,40,62,22,40,62,79.8
|
|
harbor-house,0,floor,16,38,54,16,38,54,78.9
|
|
harbor-house,1,floor,21,46,67,21,46,67,82.9
|
|
harbor-house,2,floor,20,41,61,20,41,61,79.0
|
|
harbor-house,0,compact_ok,16,39,55,16,22,38,78.7
|
|
harbor-house,1,compact_ok,20,44,64,20,20,40,82.6
|
|
harbor-house,2,compact_ok,10,55,65,10,34,44,81.4
|
|
harbor-house,0,exempt_circulation,18,37,55,18,34,52,77.2
|
|
harbor-house,1,exempt_circulation,19,49,68,19,42,61,81.0
|
|
harbor-house,2,exempt_circulation,21,42,63,21,37,58,79.0
|
|
harbor-house,0,usage_daylight,16,39,55,16,30,46,79.9
|
|
harbor-house,1,usage_daylight,21,51,72,21,30,51,82.3
|
|
harbor-house,2,usage_daylight,12,49,61,12,35,47,81.8
|
|
maple-court,0,urb,30,77,107,30,77,107,108.5
|
|
maple-court,1,urb,23,83,106,23,83,106,109.0
|
|
maple-court,2,urb,51,69,120,51,69,120,110.1
|
|
maple-court,0,floor,35,74,109,35,74,109,109.1
|
|
maple-court,1,floor,21,81,102,21,81,102,107.9
|
|
maple-court,2,floor,50,69,119,50,69,119,110.3
|
|
maple-court,0,compact_ok,30,78,108,30,47,77,109.8
|
|
maple-court,1,compact_ok,25,82,107,25,43,68,108.6
|
|
maple-court,2,compact_ok,52,70,122,52,33,85,105.9
|
|
maple-court,0,exempt_circulation,30,72,102,30,66,96,105.1
|
|
maple-court,1,exempt_circulation,25,81,106,25,70,95,106.6
|
|
maple-court,2,exempt_circulation,52,70,122,52,61,113,112.6
|
|
maple-court,0,usage_daylight,30,69,99,30,52,82,110.3
|
|
maple-court,1,usage_daylight,25,73,98,25,56,81,107.9
|
|
maple-court,2,usage_daylight,41,84,125,41,59,100,113.2
|