diff --git a/DESIGN.md b/DESIGN.md index 7ccd300..0b03789 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -5192,6 +5192,44 @@ steer the search to a better building?* The arm's own-objective count is reported alongside solely to show the size of the definitional discount. A mode passes on the stock column. `experiments/ab_ssz_search.py`. +**The result: NOT A PASS at this budget, and n=3 cannot decide it.** Budget +3000, 3 seeds, paired per-seed deltas against `urb` (negative = fewer fails): + +| mode | harbor hard Δ | mean | maple hard Δ | mean | +|---|---|---|---|---| +| `floor` | [0, −1, −2] | −1.0 | [+5, −2, −1] | +0.7 | +| `compact_ok` | [0, −2, **−12**] | −4.7 | [0, +2, +1] | +1.0 | +| `exempt_circulation` | [+2, −3, −1] | −0.7 | [0, +2, +1] | +1.0 | +| `usage_daylight` | [0, −1, **−10**] | −3.7 | [0, +2, **−10**] | −2.7 | + +The means flatter every mode. **The whole signal is seed 2**, in both +programmes, and seed 2 is the seed where stock itself does worst (harbor 22 +hard against 16/22; maple 51 against 30/23). Two seeds in three are flat or +slightly worse. What this says is "on a bad run, the permissive modes do less +badly" — which is not nothing, but it is not the claim. + +And on that seed the soft count rises by as much as the hard count falls: +harbor seed 2 −10 hard / +9 soft, maple seed 2 −10 hard / +15 soft. **Totals**: +harbor 62 → 61, maple 120 → **125**. Because every arm is scored under stock +`urb`, this is a real change of layout and not a relabelling — the search +genuinely traded hard failures for soft ones. Under the project's tiered +comparator, where `n_hard` is the primary key, that trade is progress. Under +`ssz`'s acceptance criterion — *lowers hard without inflating soft* — it is +not. The criterion is stricter than the comparator the search actually uses; +which of the two is the right yardstick is now the live question, and it is +`homemaker-py-gvb`'s question as much as this one. + +`usage_daylight` stays **default off** pending a higher-powered run +(`urb` vs `usage_daylight` only, more seeds, both programmes). Nothing here +justifies shipping it as a default, and nothing here refutes it either: at +n=3 with one dominant seed, the honest reading is *undecided*. + +**What is decided** is the diagnostic half, which does not depend on the search +A/B: the daylight requirement is applied to rooms that architecturally do not +want daylight, in two thirds of the buried population, and §38.6's contrary +null was an artefact of three modes that never touched those leaves. + + ## 39. Config audit: requirements that actively fight the engine (`homemaker-py-ju3`) — measured 2026-08-25 The corpus `patterns.config` targets and `costs.config` values were estimated diff --git a/experiments/results/ab_ssz_search.csv b/experiments/results/ab_ssz_search.csv index 62833c4..8f6124d 100644 --- a/experiments/results/ab_ssz_search.csv +++ b/experiments/results/ab_ssz_search.csv @@ -5,3 +5,27 @@ harbor-house,2,urb,22,40,62,22,40,62,79.8 harbor-house,0,floor,16,38,54,16,38,54,78.9 harbor-house,1,floor,21,46,67,21,46,67,82.9 harbor-house,2,floor,20,41,61,20,41,61,79.0 +harbor-house,0,compact_ok,16,39,55,16,22,38,78.7 +harbor-house,1,compact_ok,20,44,64,20,20,40,82.6 +harbor-house,2,compact_ok,10,55,65,10,34,44,81.4 +harbor-house,0,exempt_circulation,18,37,55,18,34,52,77.2 +harbor-house,1,exempt_circulation,19,49,68,19,42,61,81.0 +harbor-house,2,exempt_circulation,21,42,63,21,37,58,79.0 +harbor-house,0,usage_daylight,16,39,55,16,30,46,79.9 +harbor-house,1,usage_daylight,21,51,72,21,30,51,82.3 +harbor-house,2,usage_daylight,12,49,61,12,35,47,81.8 +maple-court,0,urb,30,77,107,30,77,107,108.5 +maple-court,1,urb,23,83,106,23,83,106,109.0 +maple-court,2,urb,51,69,120,51,69,120,110.1 +maple-court,0,floor,35,74,109,35,74,109,109.1 +maple-court,1,floor,21,81,102,21,81,102,107.9 +maple-court,2,floor,50,69,119,50,69,119,110.3 +maple-court,0,compact_ok,30,78,108,30,47,77,109.8 +maple-court,1,compact_ok,25,82,107,25,43,68,108.6 +maple-court,2,compact_ok,52,70,122,52,33,85,105.9 +maple-court,0,exempt_circulation,30,72,102,30,66,96,105.1 +maple-court,1,exempt_circulation,25,81,106,25,70,95,106.6 +maple-court,2,exempt_circulation,52,70,122,52,61,113,112.6 +maple-court,0,usage_daylight,30,69,99,30,52,82,110.3 +maple-court,1,usage_daylight,25,73,98,25,56,81,107.9 +maple-court,2,usage_daylight,41,84,125,41,59,100,113.2