Three times in this log a verdict rested on a sample that could not have
produced it: 38.19 (programme-house claimed at N=20, resolves at N=60),
38.21 (harbor at n=3 resolves nothing finer than ~15 fails, yet every
recorded margin is smaller), 39.5/38.20 (a 10-fail cpsat margin inside a
+-23-fail noise band). Each was found years later.
experiments/ab_report.py makes it visible when the verdict is made:
minimum detectable difference = t_crit(0.975, N-1) * sd / sqrt(N)
A margin below the MDD is not a weak result but an absent one -- the
experiment could not have distinguished it from zero however it came out.
The report flags that, refuses to endorse a winner, and states the N needed.
Validated against both datasets measured this session, reproducing the
hand-computed figures exactly:
programme-house N=60 +0.567 p=0.017 MDD 0.462 verdict supported
programme-house N=20 +0.700 p=0.085 MDD 0.805 UNDERPOWERED, N~=26
harbor N=24 +1.208 p=0.502 MDD 3.669 UNDERPOWERED, N~=202
Harbor needing ~200 seeds means it cannot answer the collapse_insearch
question at any N this project would realistically run.
Fixed a defect in my own first version: with all-ties (sd=0) the MDD
collapses to zero and the naive abs(mean) < mdd reported "margin exceeds
the MDD -- verdict supported" for a margin of 0.000, with t=nan. A reporter
that endorses a zero margin is worse than none. Degenerate cases are now
explicit and distinguish all-ties from a constant non-zero difference.
Separate correction found while validating: 38.19's published p-values for
N=20 and N=40 were 0.069 and 0.045, from a normal approximation. The exact
paired t-test gives 0.085 and 0.052 -- so N=40 did NOT reach significance
either; it took N=60. The approximation was anti-conservative, making
results look more significant than they are, the same direction of error
this thread is about. Corrected at all four citing sites, and the "needs
N >= 40" guidance raised to N >= 60.
Closes homemaker-py-tco.
Lint at parity (46); tests 384 passed, 0 failed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
Owner's ruling, and it corrects the design not just the classification: the
daylight requirement is already defined in the crinkliness. The gaussian's
compact side IS "too little exposed wall per unit floor"; its exposed side
is envelope cost. 38.9's proposed daylight: axis was redundant, and keying
it off usage: was worse than redundant.
What was actually missing: crinkliness is the only leaf quality factor with
no per-space target. size, width and proportion are all declared by the
space; crinkliness was one global number for every room in every building.
crinkliness: none -> no minimum-exposure requirement, may be buried
crinkliness: [t, s] -> this space's own target
key absent -> the global uncrinkliness target, as today
`none` clips the factor on the compact side, it does not switch it off:
over-exposure is still penalised, because a crinkly leaf costs envelope
whatever it holds. A store may be buried; a store may not be a starfish.
The mechanism is backward compatible -- an absent key resolves to the
global target, so shipping it changes no score. Behaviour changes only
where a config declares something, which keeps the objective change
visible per programme in config rather than hidden in a default.
Owner's classification: everything a person occupies wants a window, WCs
and reception/waiting/foyer included; only stores, plant, records and
laundry do not. migrate_crinkliness_key.py declared crinkliness: none on 18
corpus spaces. Crinkliness fails 271 -> 243, of which not-defects 136 (50%)
-> 108 (44%); the 28 that went are exactly the utility fails.
usage_daylight and needs_daylight are removed as mis-keyed, and
DAYLIGHT_USAGES with them -- a vocabulary value should exist only where the
engine treats it differently. The historical crinkliness_mode modes stay,
default off, so 38.6/38.8 remain reproducible.
uncrinkliness_circulation is now settable to none like any space, but its
default is left unchanged pending a ruling: corridors were not among the
groups ruled on and are 63% of the remaining phantom fails.
Lint at parity (46); tests 364 passed, same 7 pre-existing fixture failures.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
DESIGN.md 38.6 concluded the three crinkliness modes were inert against the
circulation-deletion incentive. Two things were wrong with that measurement.
Its premise, 38.2, is retracted. And its script selected leaves with the
pre-39.4 prefix rule `type[:1].upper() in ("C","O")`, which sweeps every
programme room starting with c or o -- cr1, of1 -- in as circulation.
The simpler problem is that none of the three modes ever touched the leaves
ssz is about. quality_uncrinkliness reaches `if not crink` before any mode
logic that matters, so for a zero-exposure leaf: floor returns 0.01 (one
percent of a unit quality, multiplied into a product and weighed against a
whole leaf's cost -- inert); compact_ok is self-contradictory, announcing
that compact is not a defect and then returning the floor for the most
compact case of all; exempt_circulation reaches at most a third of them.
Measured: 0% / 0% / 0% / 21-33% of buried leaves rescued.
What the buried leaves are, now that 39.7 gives every space a usage: two
thirds of them are spaces that architecturally do not want a window --
stores, WCs, plant, corridors, covered courtyards -- scored identically
with a windowless bedroom. harbor 22/33, maple 33/46, health 9/18.
- crinkliness_mode="usage_daylight": daylight required of the uses a
person occupies (programme.DAYLIGHT_USAGES) and nothing else. Elsewhere
the factor is clipped on the compact side only, so being buried stops
being a defect while over-exposure still costs -- a crinkly leaf costs
envelope whatever it is used for. A windowless bedroom stays the hard
zero it is under stock: 11/11, 13/13, 9/9 still failing.
- compact_ok repaired to score the buried limit as compact, the behaviour
its name always claimed. It now rescues 100% including bedrooms, and is
kept as the upper-bound control, not a candidate.
- ab_ssz_search.py: the fixed-budget search A/B ssz's acceptance criteria
actually asks for. Every arm is optimised under its own objective and
re-scored under stock urb, because the permissive modes return 1.0
where stock fails and would otherwise win by deleting a fail category.
- ab_crinkliness_mode_ssz.py: prefix rule fixed, retracted premise
flagged in its docstring.
- 38.7's remaining claims from the retracted 38.2/38.3 corrected.
Default is unchanged ("urb"), byte-identical to all prior runs. Lint at
parity (46 pre-existing); tests 366 passed, 10 new, same 7 pre-existing
fixture failures (homemaker-py-bdf).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
Claude