Cold-start re-baseline after the DESIGN.md 38.10/38.11 objective
change. Single worker (avoids homemaker-py-b8g), scored by the
shipped scorer from the programme directory.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
Cold-start re-baseline after the DESIGN.md 38.10/38.11 objective
change. Single worker (avoids homemaker-py-b8g), scored by the
shipped scorer from the programme directory.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
Cold-start re-baseline after the DESIGN.md 38.10/38.11 objective
change. Single worker (avoids homemaker-py-b8g), scored by the
shipped scorer from the programme directory.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
Cold-start re-baseline after the DESIGN.md 38.10/38.11 objective
change. Single worker (avoids homemaker-py-b8g), scored by the
shipped scorer from the programme directory.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
Cold-start re-baseline after the DESIGN.md 38.10/38.11 objective
change. Single worker (avoids homemaker-py-b8g), scored by the
shipped scorer from the programme directory.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
Cold-start re-baseline after the DESIGN.md 38.10/38.11 objective
change. Single worker (avoids homemaker-py-b8g), scored by the
shipped scorer from the programme directory.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
0wr asked which harbor A/Bs were decided by a narrow margin before 39.4.
Measuring harbor's variance makes the margin-by-margin triage moot.
Harbor's paired seed-to-seed sd is 6.19 fails (24 paired ON/OFF runs,
budget 2500), so n=3 resolves nothing finer than 15.4 fails. Every recorded
harbor margin is below that: 13.9 share_edge_cap 3.7, 20 qpk 8.3, 23 f1d
mixed, 37.1 tiering 6.3. So it is not that SOME harbor results were narrow
-- no harbor A/B run at three seeds could resolve the margin it reported,
independently of what 39.4 did to the programme. Of the 220 possible
3-seed subsets of the 24 runs, 56 (25%) show a clean 3/3 sweep for ON.
Re-measured 20's harbor arm, the one backing a live default:
N=3 2W/1L/0T +2.67 p=0.560
N=12 8W/3L/1T +3.50 p=0.076
N=24 13W/10L/1T +1.21 p=0.502 CI [-2.46,+4.88]
Null. The published "harbor: ON wins 3/3, 80.3 -> 72.0" was a lucky draw --
even seeds 1-3 measured here give 2W/1L, not a sweep.
So 20's claim that the qpk verdict "holds at both example scales tested" is
withdrawn and annotated in place. collapse_insearch's default rests on
programme-house alone (38.19, N=60, +0.57, p=0.017). It is not refuted on
harbor -- direction positive but indistinguishable from zero -- but harbor
must not be cited as corroboration.
Harness generalised (PROG/BUDGET/WORKERS) and results kept.
Filed homemaker-py-... : A/B harnesses should report the minimum detectable
difference for the N they run, so an underpowered verdict is visible when
it is made rather than years later.
Closes homemaker-py-0wr.
Lint at parity (46).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
Cold-start re-baseline after the DESIGN.md 38.10/38.11 objective
change. Single worker (avoids homemaker-py-b8g), scored by the
shipped scorer from the programme directory.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
38.18 confirmed the 1ph default-flip was sound for its own era, but that
measurement predates three changes to the objective it was measured
against -- 39.4's namespace fix, 38.10/38.11's per-space crinkliness, and
38.12's missing-space cascade -- and collapse_insearch runs collapse_global
inside every eval, valued against exactly the factors those touched. A
default carried on a superseded measurement is an assumption, not a result.
Re-ran the 1ph protocol as published on the current codebase:
N OFF ON W/L/T diff t p
published 1ph 20 7.95 7.10 11/6/3 +0.85 2.38 0.028
current objective 20 7.85 7.15 10/7/3 +0.70 1.82 0.069
current objective 40 7.60 7.03 21/14/5 +0.57 2.01 0.045
current objective 60 7.58 7.02 29/19/12 +0.57 2.45 0.017
Verdict: the default STANDS. At N=60, mean diff +0.567 fails/seed, paired
t=2.454 (df=59), p=0.0171 exact, 95% CI [+0.105, +1.029] excluding zero;
Wilcoxon signed-rank cross-check agrees (p=0.0138), which matters because
fail counts are small integers and normality is not obvious.
Two caveats. The effect is about a third smaller than published (+0.57 vs
+0.85) -- partly regression from a lucky N=20 draw, partly plausible real
erosion, since several fails collapse_global used to clear have been
redefined out of existence or made harder.
More usefully: the published N=20 can no longer detect its own effect. At
exactly that sample size the current answer is p ~= 0.069, a null by the
conventional threshold. Had I stopped at N=20 the honest report would have
been "the 1ph verdict no longer reproduces" and the default would have
looked unjustified. It took N=60 to resolve. That is the 8sh/1ph/qi6/lj3
pattern this log warns about, now biting the flagship result itself: any
future re-validation of this default needs N >= 40.
20 annotated in place so a reader of the original claim sees the current
figure. Harness takes a seed range now (APPEND=1 to extend a sweep).
Closes homemaker-py-ioe.
Lint at parity (46); tests 384 passed, 0 failed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
d86 asked for the rigorous version of 35's spot-check -- check out the
codebase at the 1ph commit, backport the iio stale-leaf-share fix, re-run
the historical seed sets.
One constraint had to be worked around: this repository's history begins
2026-07-30, six days AFTER the 1ph commit (2026-07-24), so that checkout
does not exist here. Closest reachable stand-in is 391f510 (2026-07-30), a
genuine ancestor of the iio fix and therefore pre-iio, carrying that era's
examples/ and objective. Measured there, not at the true 1ph commit, and
that is a real limitation of the reproduction.
Protocol as published: programme-house, budget 3000, 4 workers, seeds 1-20,
ON vs OFF, both arms finished with --collapse. Run twice over one worktree,
as-is then with the 22-line iio hunk applied.
OFF ON W/L/T mean diff t(df=19)
published 1ph 7.95 7.10 11/6/3 +0.85 2.38
pre-iio 391f510 8.05 7.10 11/6/3 +0.95 2.59
same + iio fix 8.05 7.10 11/6/3 +0.95 2.59
The published verdict reproduces, and the iio fix changes nothing: 0 of 40
(seed, arm) cells differ, per-seed counts equal cell by cell.
It could not have been otherwise. The bug needs a leaf carrying a STALE
share/share_type, and leaf-sharing only stamps a share where a code has
count > 1 -- programme-house declares count: 1 for all six codes. Over 8
constructed seeds at that commit, programme-house had share_type set on 0
of 56 leaves; harbor-house on 24 of 128. _collapse_value reads
leaf.share_type, so the bug is structurally unreachable on the 1ph
protocol. That is why 35 saw 2/3 harbor seeds diverge by 5-8 fails while
programme-house at N=20 moves not one cell: harbor has codes at counts 10,
6 and 5.
20's retroactive caveat is discharged for the 1ph section and stays live
for harbor-house/qpk, where shares exist and divergence was measured.
Kept experiments/rerun_1ph_protocol.sh and both result TSVs.
Closes homemaker-py-d86.
Lint at parity (46); tests 384 passed, 0 failed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
All four 500k runs died about 10 minutes in when the container was
reclaimed. No SIGTERM fired, so no .dom was written and 0 of 12 runs
completed. My plan committed results per finished run, which protected
nothing because no run reached its commit point. The bad assumption was
reading "reclaimed after inactivity" as CPU inactivity; it is conversation
inactivity, and background compute does not hold the box open.
Progress reached before the loss (from the tracked logs): harbor 24,960
evals / 40 fails, maple 14,880 / 79, health-centre 25,920 / 33,
programme-house 138,800 / 2.
The underlying gap is not environmental: a search's only output lands at
the very end or on SIGTERM, so ANY abrupt loss -- reclaimed container, OOM,
power cut -- takes the whole run with it. On a 3M-eval search that is 2.4
days of compute with no recoverable artefact.
- driver.search gains checkpoint=/checkpoint_every=: the current best is
handed to a callback at most every N evals. Rate-limited by evals, not
improvements, which come in bursts early. A failing checkpoint is logged
and swallowed -- losing a checkpoint is bad, losing the search because a
checkpoint failed is worse.
- homemaker-evolve --checkpoint-every N writes <out>.dom.checkpoint via
mkstemp + os.replace, so a crash can never catch it half-written. It is
deliberately NOT the output path: a checkpoint is a leaf-sharing run's
internal best, dishonest under the canonical scorer until the finish
stage unfolds it (homemaker-py-3l6), and must not be mistaken for the
finished article.
- Verified the written checkpoint re-loads as a valid .dom.
Default off, so behaviour is unchanged without the flag.
Lint at parity (46); tests 372 passed (3 new), same 2 pre-existing failures.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
Cold-start re-baseline after the DESIGN.md 38.10/38.11 objective
change. Single worker (avoids homemaker-py-b8g), scored by the
shipped scorer from the programme directory.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
Correction 1: 38.8's A/B scores every arm under stock urb, which is only
sound if stock is ground truth -- and stock is what this section shows is
miscalibrated. Scoring the repair under the objective it repairs penalises
it for repairing. That result is not evidence against usage_daylight.
The measurement that matters needs no search: are the emitted failures
true? audit_crinkliness_truth.py classifies every stock crinkliness fail by
the leaf's declared usage. 164 of 271 (61%) are reported against spaces
that want no daylight -- stores, WCs, plant, corridors, covered courtyards
-- and it holds on evolved artefacts (maple generated.dom: 64%), not just
constructed seeds. Under value *= 0.5 ** len(failures) each one halves the
fitness of a design that did nothing wrong.
Correction 2: usage: is the wrong key, and usage_daylight as written
mis-keys it. 39.7 established usage: as an ACCESS-requirement class.
"Needs no special access" and "needs no window" are different questions,
and the corpus separates them: usage: none is Waiting Room, Reception,
Reception Office, Entrance Foyer -- a waiting room plainly wants daylight
and DAYLIGHT_USAGES exempts it. The usage: bedroom half is right, but by
luck of where the access axis happened to fall.
The audit is robust to that error (61% -> 57% if none is reclassified), so
the finding stands; the design does not. Daylight needs its own declared
per-space axis, decided by the programme author as usage: was.
usage_daylight stays default off and is not the shipping fix. The
compact-side clip is the right shape for the factor, so the mechanism is
retained pending a daylight: attribute to key it on.
Higher-power A/B stopped: it was chasing a score against a flawed objective
with a mis-keyed fix.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
Fixed-budget A/B (3000 evals, 3 seeds, harbor + maple, every arm scored
under stock urb so the permissive modes cannot win by deleting a fail
category).
usage_daylight's paired hard-fail deltas are harbor [0,-1,-10] and maple
[0,+2,-10]. The means (-3.7, -2.7) flatter it: the whole signal is seed 2
in both programmes, and seed 2 is the seed where stock itself does worst.
Two seeds in three are flat or slightly worse. On that seed soft rises as
much as hard falls (harbor -10h/+9s, maple -10h/+15s), so totals go
62->61 on harbor and 120->125 on maple.
Because the scoring is stock, that is a genuine trade of hard failures for
soft ones, not a relabelling -- progress under the tiered comparator, where
n_hard is primary, but a fail against this issue's acceptance criterion
("without inflating soft"). Which yardstick is right is now the live
question, and it is gvb's question as much as ssz's.
usage_daylight stays default off; undecided, not refuted. Higher-power run
(urb vs usage_daylight, 10 seeds) is running.
The diagnostic half stands independent of the search A/B: the objective
demands daylight of two thirds of the buried population that does not want
it, and 38.6's contrary null was an artefact of three modes that never
touched those leaves.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
fitness.py already imports the module as `_programme` at the top and uses
that idiom elsewhere (`_programme.SOCIABLE_USAGES`). The local re-import
sat in the per-leaf hot path for no reason.
Also snapshots the in-progress ab_ssz_search.csv; the full run is still
going and will supersede it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
DESIGN.md 38.6 concluded the three crinkliness modes were inert against the
circulation-deletion incentive. Two things were wrong with that measurement.
Its premise, 38.2, is retracted. And its script selected leaves with the
pre-39.4 prefix rule `type[:1].upper() in ("C","O")`, which sweeps every
programme room starting with c or o -- cr1, of1 -- in as circulation.
The simpler problem is that none of the three modes ever touched the leaves
ssz is about. quality_uncrinkliness reaches `if not crink` before any mode
logic that matters, so for a zero-exposure leaf: floor returns 0.01 (one
percent of a unit quality, multiplied into a product and weighed against a
whole leaf's cost -- inert); compact_ok is self-contradictory, announcing
that compact is not a defect and then returning the floor for the most
compact case of all; exempt_circulation reaches at most a third of them.
Measured: 0% / 0% / 0% / 21-33% of buried leaves rescued.
What the buried leaves are, now that 39.7 gives every space a usage: two
thirds of them are spaces that architecturally do not want a window --
stores, WCs, plant, corridors, covered courtyards -- scored identically
with a windowless bedroom. harbor 22/33, maple 33/46, health 9/18.
- crinkliness_mode="usage_daylight": daylight required of the uses a
person occupies (programme.DAYLIGHT_USAGES) and nothing else. Elsewhere
the factor is clipped on the compact side only, so being buried stops
being a defect while over-exposure still costs -- a crinkly leaf costs
envelope whatever it is used for. A windowless bedroom stays the hard
zero it is under stock: 11/11, 13/13, 9/9 still failing.
- compact_ok repaired to score the buried limit as compact, the behaviour
its name always claimed. It now rescues 100% including bedrooms, and is
kept as the upper-bound control, not a candidate.
- ab_ssz_search.py: the fixed-budget search A/B ssz's acceptance criteria
actually asks for. Every arm is optimised under its own objective and
re-scored under stock urb, because the permissive modes return 1.0
where stock fails and would otherwise win by deleting a fail category.
- ab_crinkliness_mode_ssz.py: prefix rule fixed, retracted premise
flagged in its docstring.
- 38.7's remaining claims from the retracted 38.2/38.3 corrected.
Default is unchanged ("urb"), byte-identical to all prior runs. Lint at
parity (46 pre-existing); tests 366 passed, 10 new, same 7 pre-existing
fixture failures (homemaker-py-bdf).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
CP-SAT assignment does not clearly beat greedy end-to-end: worse on
harbor-house hard fails, roughly a wash on maple-court, and the reassign
operator fired in only 1 of 18 runs. Keeps assign_solver="greedy" and
enable_reassign=False as defaults; closes the bead's remaining acceptance
criterion. Raw results in experiments/results/.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LSwQwpEaHFBkeVSDDWd75S
