ssz: record the A/B result honestly -- not a pass at n=3

Fixed-budget A/B (3000 evals, 3 seeds, harbor + maple, every arm scored
under stock urb so the permissive modes cannot win by deleting a fail
category).

usage_daylight's paired hard-fail deltas are harbor [0,-1,-10] and maple
[0,+2,-10]. The means (-3.7, -2.7) flatter it: the whole signal is seed 2
in both programmes, and seed 2 is the seed where stock itself does worst.
Two seeds in three are flat or slightly worse. On that seed soft rises as
much as hard falls (harbor -10h/+9s, maple -10h/+15s), so totals go
62->61 on harbor and 120->125 on maple.

Because the scoring is stock, that is a genuine trade of hard failures for
soft ones, not a relabelling -- progress under the tiered comparator, where
n_hard is primary, but a fail against this issue's acceptance criterion
("without inflating soft"). Which yardstick is right is now the live
question, and it is gvb's question as much as ssz's.

usage_daylight stays default off; undecided, not refuted. Higher-power run
(urb vs usage_daylight, 10 seeds) is running.

The diagnostic half stands independent of the search A/B: the objective
demands daylight of two thirds of the buried population that does not want
it, and 38.6's contrary null was an artefact of three modes that never
touched those leaves.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB

DESIGN.md

@@ -5192,6 +5192,44 @@ steer the search to a better building?* The arm's own-objective count is
 reported alongside solely to show the size of the definitional discount. A mode
 passes on the stock column. `experiments/ab_ssz_search.py`.
 
+**The result: NOT A PASS at this budget, and n=3 cannot decide it.** Budget
+3000, 3 seeds, paired per-seed deltas against `urb` (negative = fewer fails):
+
+| mode | harbor hard Δ | mean | maple hard Δ | mean |
+|---|---|---|---|---|
+| `floor` | [0, −1, −2] | −1.0 | [+5, −2, −1] | +0.7 |
+| `compact_ok` | [0, −2, **−12**] | −4.7 | [0, +2, +1] | +1.0 |
+| `exempt_circulation` | [+2, −3, −1] | −0.7 | [0, +2, +1] | +1.0 |
+| `usage_daylight` | [0, −1, **−10**] | −3.7 | [0, +2, **−10**] | −2.7 |
+
+The means flatter every mode. **The whole signal is seed 2**, in both
+programmes, and seed 2 is the seed where stock itself does worst (harbor 22
+hard against 16/22; maple 51 against 30/23). Two seeds in three are flat or
+slightly worse. What this says is "on a bad run, the permissive modes do less
+badly" — which is not nothing, but it is not the claim.
+
+And on that seed the soft count rises by as much as the hard count falls:
+harbor seed 2 −10 hard / +9 soft, maple seed 2 −10 hard / +15 soft. **Totals**:
+harbor 62 → 61, maple 120 → **125**. Because every arm is scored under stock
+`urb`, this is a real change of layout and not a relabelling — the search
+genuinely traded hard failures for soft ones. Under the project's tiered
+comparator, where `n_hard` is the primary key, that trade is progress. Under
+`ssz`'s acceptance criterion — *lowers hard without inflating soft* — it is
+not. The criterion is stricter than the comparator the search actually uses;
+which of the two is the right yardstick is now the live question, and it is
+`homemaker-py-gvb`'s question as much as this one.
+
+`usage_daylight` stays **default off** pending a higher-powered run
+(`urb` vs `usage_daylight` only, more seeds, both programmes). Nothing here
+justifies shipping it as a default, and nothing here refutes it either: at
+n=3 with one dominant seed, the honest reading is *undecided*.
+
+**What is decided** is the diagnostic half, which does not depend on the search
+A/B: the daylight requirement is applied to rooms that architecturally do not
+want daylight, in two thirds of the buried population, and §38.6's contrary
+null was an artefact of three modes that never touched those leaves.
+
+
 ## 39. Config audit: requirements that actively fight the engine (`homemaker-py-ju3`) — measured 2026-08-25
 
 The corpus `patterns.config` targets and `costs.config` values were estimated

experiments/results/ab_ssz_search.csv

@@ -5,3 +5,27 @@ harbor-house,2,urb,22,40,62,22,40,62,79.8
 harbor-house,0,floor,16,38,54,16,38,54,78.9
 harbor-house,1,floor,21,46,67,21,46,67,82.9
 harbor-house,2,floor,20,41,61,20,41,61,79.0
+harbor-house,0,compact_ok,16,39,55,16,22,38,78.7
+harbor-house,1,compact_ok,20,44,64,20,20,40,82.6
+harbor-house,2,compact_ok,10,55,65,10,34,44,81.4
+harbor-house,0,exempt_circulation,18,37,55,18,34,52,77.2
+harbor-house,1,exempt_circulation,19,49,68,19,42,61,81.0
+harbor-house,2,exempt_circulation,21,42,63,21,37,58,79.0
+harbor-house,0,usage_daylight,16,39,55,16,30,46,79.9
+harbor-house,1,usage_daylight,21,51,72,21,30,51,82.3
+harbor-house,2,usage_daylight,12,49,61,12,35,47,81.8
+maple-court,0,urb,30,77,107,30,77,107,108.5
+maple-court,1,urb,23,83,106,23,83,106,109.0
+maple-court,2,urb,51,69,120,51,69,120,110.1
+maple-court,0,floor,35,74,109,35,74,109,109.1
+maple-court,1,floor,21,81,102,21,81,102,107.9
+maple-court,2,floor,50,69,119,50,69,119,110.3
+maple-court,0,compact_ok,30,78,108,30,47,77,109.8
+maple-court,1,compact_ok,25,82,107,25,43,68,108.6
+maple-court,2,compact_ok,52,70,122,52,33,85,105.9
+maple-court,0,exempt_circulation,30,72,102,30,66,96,105.1
+maple-court,1,exempt_circulation,25,81,106,25,70,95,106.6
+maple-court,2,exempt_circulation,52,70,122,52,61,113,112.6
+maple-court,0,usage_daylight,30,69,99,30,52,82,110.3
+maple-court,1,usage_daylight,25,73,98,25,56,81,107.9
+maple-court,2,usage_daylight,41,84,125,41,59,100,113.2