39.24's sweep listed two entries as DEAD rather than suspect -- inert code that
reads as live. Neither changes a score or a failure on any corpus artefact, and
that is verified rather than asserted: every artefact scores identically to its
39.25 measurement.
ratio_public_outside and ratio_private_outside. evaluate_building read both and
multiplied a gaussian into the building factor for each. Neither key exists in
CONF_DEFAULTS and no patterns.config in the repository declares either, so both
branches were guarded and never ran. Removing them also retires what fed them:
the four public_length_*/private_length_* tracking keys accumulated per leaf in
process_storey, and the _public_length/_private_length helpers, which had no
other caller.
NOT removed, because they are live: _public_access, _public_access_outside,
_public_access_pins and the has_public_access_* tracking flags, which drive real
checks and collapse_global's preserve_public_access. Only the length-ratio
machinery was dead.
The daylight quality factor. evaluate_leaf set factors["daylight"] = 1.0
unconditionally -- pinned since the URB_NO_OCCLUSION descope (6) and unable to
be anything else. It was never in _GRADED_FACTORS, so it contributed nothing to
the graded signal, and 39.18's geometric mean then had to special-case it in
factor_is_asked as a factor that is never asked. A constant that exists only to
be excluded is worth deleting. If 2g5 rebuilds occlusion it reintroduces a real
daylight factor, which would need factor_is_asked to say True anyway.
Two tests referenced the removed factor. test_leaf_grade_ignores_non_graded_keys
now names a key that genuinely does not exist; the aggregate underflow test
dropped its daylight entry, which would otherwise have been counted as asked and
changed the expected geometric mean.
Worth doing despite changing no number: 39.20 and 39.25 were both cases where
something inert looked live -- a parity test that never ran, a per-level rule
switched off in every config -- and in both the misreading cost real time and
produced a wrong conclusion. An objective with fewer things in it that do
nothing is one where "this term does nothing" is informative rather than
routine.
Still open on dpt, each needing a ruling or a rate change rather than a
measurement: quality_size's upper side, the minimum-internal-area factor as a
third statement of "build the rooms", and the 0.5**n_fails curve.
426 passed.
Refs homemaker-py-dpt.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
Eight memories stored via bd remember for the next session:
config-declares-over-defaults a CONF_DEFAULTS value is not the live one
absence-of-fails-means-nothing zero fails can mean the check is disabled
crinkliness-is-alexander-159 1/crink is depth PER WINDOW WALL
score-is-already-a-ratio value/cost makes per-m2 pricing proportional
fails-are-emitted-before-aggregation
objective-state-after-39.25 every live setting after this session
ab-protocol-and-mdd what 8000 evals at n=6 can and cannot resolve
none-of-39s-changes-are-search-proven
bk9 retitled to post-39.25 and given the full brief: the seven objective
changes to re-baseline, the two left default-OFF and why, what the old
artefacts now score (and why that is not the baseline), and three quantities
39.12 did not capture that now have predictions attached -- per-storey daylit
wall supply/demand, which storey the open space lands on, and the outdoor
fraction, which is the one unmeasured risk in 39.22-39.25.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
Memories stored via bd remember (injected at bd prime, so they survive into the
re-baseline session):
config-declares-over-defaults a CONF_DEFAULTS value is not the live one;
the corpus configs declare their own
absence-of-fails-means-nothing zero fails can mean the check is disabled
crinkliness-is-alexander-159 1/crink is depth PER WINDOW WALL
score-is-already-a-ratio value/cost makes per-m2 pricing
proportional already
fails-are-emitted-before-aggregation why aggregation and rate changes cannot
move the fail set
objective-state-after-39.25 every default now live, and what is still
default-OFF
ab-protocol-and-mdd the harness, and what 8000 evals can and
cannot resolve
none-of-39s-changes-are-search-proven
bk9 re-scoped: it was filed after 39.19 and four more changes have landed since
(39.22-39.25). Its comment now lists all six, names the two still default-OFF,
gives the rescored baseline layouts, and asks for four things beyond 39.12's
mean/sd/MDD -- per-storey daylit-wall supply/demand, the outdoor fraction (the
one unmeasured risk in 39.25), open space by storey, and corridor geometry.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
Owner: "maybe plot_ratio is having unintended consequences, Alexander simply
says that all levels should have accessible outside space, he doesn't say how
much."
Both halves are right, and the codebase had the two rules the wrong way round.
ratio_outside is a gaussian on the outdoor FRACTION, applied as a whole-building
multiplier, and its declared targets have no stated basis and contradict each
other: health-centre targets 0.06 and sits at 0.096-0.129, so it is penalised
x0.39-x0.77 for having too MUCH; programme-house targets 0.30 and sits at
0.098-0.293, penalised x0.40-x0.999 for having too LITTLE. Penalties as large as
the ratio_circulation ones 39.24 removed, pulling two programmes in opposite
directions on the same quantity.
force_roof_garden already implements the rule Alexander actually states -- per
level, no outdoor space at all is a hard fail, no quantity attached. It has
existed all along and was switched OFF in every corpus config.
Near-miss worth recording: measuring first, I found zero "no outside space"
fails across the twelve baseline runs and briefly read that as the requirement
being met everywhere. It meant the check never ran. Same shape as 39.20's parity
tests -- no failures from a test that is not executing looks exactly like no
failures from a test that passes, and the tell was again the config, not the
code.
Enabled, it bites on 4 of 25 baseline levels: maple s0 level 1, and
programme-house level 0 in all three seeds. A house with no outdoor space on its
own ground floor is a fair criticism of the layout, and exactly what a
building-level fraction cannot catch, since 22% outdoor concentrated on one
storey satisfies it perfectly.
The upper side ratio_outside used to provide is covered in a better currency by
the minimum-internal-area factor (internal area >= 1.2x the programme's declared
room area), which is live -- binding on harbor s0 x0.920 and programme-house s0
x0.787.
This is the one change in 39.22-39.25 whose risk is NOT measured: nothing here
proves the outdoor fraction will not drift up once the search is free to raise
it, and outdoor space is profitable (1.64 return against a room's 0.66). The
re-baseline (bk9) is what shows it, and 39.25 asks for the fraction to be
recorded there.
4 hard fails added, none removed. 426 passed.
Refs homemaker-py-hxi.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
ssz's three candidate repairs were all tried and recorded (38.1/38.6/38.8,
superseded by 38.10/38.11), and what it was pointing at was resolved in 39.13
(the tail underflows across its whole range, rescaled, NULL), 39.14 (two-sided
gaussian double-charged surplus daylight, made one-sided) and 39.16 (the
constant is Alexander 159 and is not miscalibrated). What has substance left is
k54, and 39.17 relocated even that to per-storey frontage starvation.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
Owner: "I think a corridor could be worth a sixth of a room, this is ok. maybe
we should dump the ratio_circulation altogether if there is already a pressure
in circulation caused by the cost benefit ratio per msq. this is the kind of
thing we want to root out of the scoring model: anything that is double
counting, or using a gaussian where a linear ramp is appropriate, etc."
value_circulation = 50 stands; hxi's rate question is closed.
The duplication argument is stronger than it first looks. score = value/cost is
already a ratio, so the per-m2 economics (50 against a build cost of 200) is not
merely an absolute pressure -- adding corridor moves value/cost by an amount
that depends on how much of the building is already corridor. It is ALREADY
proportional. ratio_circulation said the same thing again as a whole-building
multiplier, on a curve where twice the corridor is far more than twice as bad.
Correction to my own first measurement: I overrode ratio_circulation and got
scores going DOWN when a <=1 multiplier was removed, which is impossible. All
four corpus programmes DECLARE ratio_circulation, so the CONF_DEFAULTS value I
had changed was never in play and the two arms differed only in sigma. Same
trap as value_supported in 39.19.
Against the keep-it case, recorded because it is the one real argument: three of
four declare a POSITIVE target (harbor/maple 0.08, health-centre 0.10), making
the term formally two-sided rather than "less is better". It does not survive
the numbers -- the lower side is worth at most 13.3% on the large programmes
against 99% on the upper side, and "a building needs some circulation" is
enforced structurally by access and connectivity, which no amount of value can
buy off.
Disabled in CONF_DEFAULTS and the four corpus configs, each with the reason
inline and a note that a [target, sigma] pair re-enables it. Fail sets
unchanged; it was always a value multiplier. Scores +42% to +7712%.
39.24 also sweeps every remaining term against the owner's two tests. Verdicts:
perpendicular, proportion, width, crinkliness, access, size's lower side,
ratio_outside, staircase volume and the count/limit fails are all sound.
Filed as homemaker-py-dpt: size's UPPER side (cost already charges area; 82% of
size fails are over-target), the minimum-internal-area factor (a third
statement of "build the rooms"), the 0.5**n_fails curve (a ruling, not a
measurement), and two dead paths -- ratio_public/private_outside, which no
config declares, and the daylight factor pinned to 1.0 since the descope.
Also updated a test I added last turn which asserted ratio_circulation was the
second charge; it now pins that the linear ramp is the ONLY one.
419 passed.
Closes homemaker-py-hxi.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
Owner's ruling: "as long as circulation is more expensive to build than it has
value then we have a linear ramp. a gaussian ramp is probably not appropriate
here as double the amount of corridor is simply twice as bad, so it should
score the same as two half size corridors".
Both halves check out. The linear ramp is already there -- value_circulation 50
against a build cost of 200, so every m2 of corridor is worth -150 and the
objective pushes for less of it without needing a cap. And the AMOUNT of
circulation is separately governed at building level by ratio_circulation
[0.00, 0.20], a gaussian on the circulation fraction, which is where that
question belongs. The per-leaf size gaussian was a third charge on the same
thing.
It was also the only one of the three that depended on how the corridor was cut
up. One 20 m2 corridor scored gaussian(20,0,14) = 0.360 and contributed 360;
two 10 m2 halves scored 0.775 each and contributed 775 between them. Splitting a
corridor in half multiplied its value by 2.15x -- an artefact of where the tree
happened to cut, rewarding the search for fragmenting its own spine. The
ruling's test (one 2A leaf must score as two A leaves) is exactly what a
gaussian on an amount cannot satisfy, and is now a test.
size_circulation = None; quality_size returns 1.0 for circulation and
shapecurve gives amin, amax = 0, inf.
BUG this exposed: get_space_params falls through to a habitable default when a
generic family key is missing and could not tell "missing" from "present but
null", so a corridor silently inherited a room's 16 m2 size target.
_generic_param now returns (found, value); pinned by a test. The same trap
applied to 39.22's proportion_circulation.
Fail-set effect of 39.22 and 39.23 together: 16 corridor size fails and 7
proportion fails removed, none added. harbor 33/43/42 -> 32/40/38, maple
54/73/55 -> 51/65/52, health-centre 4/9/5 -> 3/9/5, programme-house unchanged.
The layouts are identical -- these are failures the objective should never have
been reporting.
Two shape-curve tests moved fixture: both built an infeasible upper storey from
a 'C' leaf, infeasible precisely because of the bounds now removed. The fixture
is a cr1 leaf, whose infeasibility is a contradiction between two of its own
bounds (needs >= 180 m2 for its aspect bound, <= 101.5 m2 for its size bound
across the box's fixed 23.52 m span) rather than a tight fit. The invariants
they test are unchanged.
Left open on hxi: the rate gap, value_circulation 50 against value_inside 300
on identical build cost. Whether a corridor is worth a sixth of a room per m2
is a design judgement, and the linear ramp is only as steep as that number.
419 passed.
Refs homemaker-py-hxi.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
Owner's ruling: "there should be no cap on the proportion of a corridor,
especially for big buildings, the crinkliness rule is there to prevent these
becoming unpleasant spaces."
proportion_circulation was [1.5, 0.5], failing a corridor above aspect 2.57 --
at the 1.97 m minimum width the width factor allows, a corridor 5.1 m long; at
the 2.4 m target width, 6.2 m. The corpus shows the consequence: median
circulation leaf 14.3 m2 at aspect 1.67, a stubby room rather than a corridor.
The justification checks out arithmetically. The unpleasant space the cap was
standing in for is a long BURIED corridor -- and crinkliness already sends that
to zero, since a buried leaf has crink == 0. A long corridor along a facade
scores 0.90. Aspect cannot tell those two apart; crinkliness can, so the cap was
duplicating a rule that already exists and does the job better.
Shipped: proportion_circulation = None (no aspect requirement).
quality_proportion returns 1.0 for circulation, and shapecurve.leaf_constraints
yields rmax = inf so the DP agrees with the fitness instead of pruning
topologies the objective would accept. A habitable room's aspect target is
untouched. The narrow side still holds -- width_circulation keeps a corridor
>= 1.97 m, and "edge too long" still caps a single wall at 8 m.
Unlike 39.14/39.18/39.19 this DOES change the fail set, which is the point.
Across the twelve baseline artefacts it removes exactly 7 corridor proportion
fails and adds none: harbor 33/43/42 -> 33/42/40, maple 54/73/55 -> 54/71/54,
health-centre 4/9/5 -> 3/9/5, programme-house unchanged.
Also recorded in 39.22, and a retraction: hxi was titled "search is rewarded
for deleting the circulation spine", which 39.8 had already measured and
refuted -- 0 of the 7 connectivity-breaking deletions sampled were rewarded. My
own earlier comment on the bead restated that retracted claim; corrected, and
the bead retitled.
What binds next: removing the cap roughly doubles a corridor leaf's reach, from
proportion at ~6.2 m to size_circulation at 12.5 m (2.4 m wide hits the 30 m2
fail edge there). size_circulation's target area is ZERO, the other half of the
double-charge -- circulation priced as overhead once in value_circulation = 50
and again in a size factor whose optimum is non-existence. Not changed: it is a
distinct parameter with its own rationale, unruled, and 39.16 is a standing
reminder about inherited constants.
415 passed.
Refs homemaker-py-hxi.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
Owner's decision: "we need to abandon the perl oracle, this was only useful
when initially porting, but I suspect many of the remaining problems have been
carried in from the perl (such as the weird scoring of outdoor and circulation
space, which definitely needs fixing)".
39 supports that second clause. Every defect the section found is inherited,
not introduced: the two-sided crinkliness gaussian that double-charges surplus
daylight (39.14), quality as a product over a variable number of factors
(39.18), value_supported priced as value_inside so a terrace was worth more per
m2 than a room (39.19), and circulation returning 0.07 per unit cost (hxi).
So parity with the oracle was never a safety net -- it was a commitment to
reproduce those defects. Each of 39.14, 39.18 and 39.19 would have been a
parity failure had parity ever been checked, and keeping the tests would have
meant reverting the fixes or explaining the failures away.
Removed: oracle.py, test_oracle.py, the two parity tests and their fixture
machinery in test_dom_corpus.py, innerloop.OracleEvaluator with its use_native
and urb_root plumbing, the same plumbing through driver, and fourteen
experiments/ scripts that could only run against Perl. Several of those are
cited in earlier DESIGN sections; the citations now point into git history,
which is the honest state -- they had been unrunnable since the oracle root
(/home/bruno/src/urb) stopped being present. run_search is superseded by
run_search_scaled, which does the same job natively.
Kept: dump_areas.pl/.py, which validate GEOMETRY against Urb (4.1) rather than
fitness, and the prose in fitness_cmd.py and dom.py explaining why the
.score/.fails formats are shaped as they are. Provenance is worth keeping; a
dead code path is not.
CLAUDE.md updated: fitness.py is the only evaluator, and "Urb did it this way"
is no longer an argument that a constant is right. 39.16 is the standing
counterweight in the other direction -- the crinkliness target WAS right and
twice looked wrong only because the code reading it was misunderstood.
Inheritance is neither evidence for nor against.
410 passed. The 69 removed cases account exactly: 64 parity (all skipped, since
no oracle .score was ever committed), 4 in test_oracle.py, and the guard test
39.20 added as a stopgap.
Closes homemaker-py-118. Files homemaker-py-bk9 for the re-baseline that 39.19
made necessary.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
Harbor and maple, three 500k plateau starts x two RNG seeds, 8000 evals, all
arms scored under stock. Run against the pre-39.19 defaults, so "stock" is the
old objective.
harbor geomean 39.17 -> 39.00 1W/0L/5T diff +0.167 MDD 0.428
harbor geomean+daylight 39.17 -> 39.00 1W/0L/5T diff +0.167 MDD 0.428
maple geomean 60.50 -> 60.17 1W/1L/4T diff +0.333 MDD 1.434
maple geomean+daylight 60.50 -> 60.00 1W/0L/5T diff +0.500 MDD 1.285
Every margin sits below what N=6 can resolve (N ~= 26-67 needed). Unlike
39.13's ramp, which was an exact null with sd=0 and identical trajectories, the
arms here do diverge and all four land on the favourable side of zero -- worth
exactly what four coin-flips landing the same way is worth. Recorded so a later
run cannot cite it as evidence.
Also recorded: none of 39.13, 39.14 or 39.18 produced a measurable search
improvement at this budget. All were shipped on correctness grounds and 39.19
then required two of them outright. If the 39 objective changes pay off in
search terms, this corpus has not shown it yet.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
The 39.19 objective change broke three tests. They were the right ones to
break, and how they broke matters more than the change.
test_native_fitness_score_parity and test_native_fitness_fail_set_parity read a
cached .score/.fails beside each corpus .dom and assert the native Python
fitness agrees. They are the ONLY check that the native evaluator still agrees
with the Urb oracle it was ported from, and CLAUDE.md still describes oracle.py
and the Perl tool as kept for cross-validation.
But .gitignore lines 10-11 exclude *.dom.score and *.dom.fails, and
git log --diff-filter=A over those patterns finds zero files ever added on any
branch. No oracle cache has ever existed here, so on a clean checkout all 64
parametrised cases skip.
Worse than skipping is what happens when they do not. Nothing in a .score file
records who wrote it, so a .dom left in that directory by a search run -- with
a .score written by homemaker-fitness, the NATIVE scorer -- silently becomes a
parity fixture, and the test compares the native scorer with itself. That
passes by construction whatever the native scorer says.
Three such cases were live and green: the coldstart-500000-s*.dom artefacts
committed to examples/programme-house during 39.12 and scored natively this
session. They surfaced only because 39.19 made the native scorer disagree with
its own stale output; absent an objective change, a green "native matches
oracle" would have been reported indefinitely.
Stopgap: parametrisation restricted to the Perl corpus's MD5-named files so a
session artefact cannot become a fixture again; the skip message now says
parity is UNVERIFIED rather than reading like an optional missing cache; a
guard test asserts the restriction. All 64 cases skip honestly. Regenerating
the caches with the native scorer would not have been a fix -- it would have
re-cemented the self-comparison.
Filed as homemaker-py-118 (P1): regenerate fixtures from the Perl oracle,
narrow the ignore rules so fixture caches can be tracked, and find out whether
parity still holds -- it may not, since 39.14, 39.18 and 39.19 all changed the
native objective and the oracle has none of them. If parity is being abandoned
deliberately the tests should be deleted with a note. What must not survive is
a test that looks like a guarantee and is not one.
415 passed, 64 skipped.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
Owner's ruling. Measured over the twelve baseline layouts as realised value per
m2 (rate x quality, not the rate alone):
as shipped before this room 67.0 terrace 294.6 violates, 4.39x
value_supported=100 only room 67.0 terrace 98.2 violates, 1.46x
geometric mean only room 132.9 terrace 296.8 violates, 2.23x
both room 132.9 terrace 98.9 satisfies
The 4.4x is roughly 2.2x aggregation and 2.0x rate, so neither half alone is
enough. That is why 39.18's geometric-mean aggregation moves from default-OFF
to default-ON here rather than waiting on its own A/B: it is not an optional
improvement, it is half of a ruling.
value_supported 300 -> 100, and set to value_outside rather than to a number
that makes the inequality come out -- back-solving from the corpus's measured
mean room quality would rot the moment either changed. Outdoor space is worth
the same to an occupant whatever level it sits on; the real difference between
a ground garden and a roof terrace is what it takes to BUILD, and cost already
says that (outside 10.0 vs outside_supported 110.0). Value describes worth,
cost describes structure, and the level belongs in the second.
Changed in CONF_DEFAULTS and the four corpus patterns.config files, which all
declared 300.0 explicitly. NOT changed in harbor-house-l0 (a shape-curve test
fixture) or y51-sweep-* (historical fixtures that exist to reproduce past
measurements) -- repricing those would destroy what they are for.
Neither change can move a fail, structurally rather than luckily: value rates
never enter fail emission, and evaluate_leaf emits each fail from its factor
before anything is combined. Verified corpus-wide: identical fail sets, scores
+11% to +169% (and -5% once, on a layout that is mostly terrace).
tests/test_terrace_value_ruling.py pins the ruling as an invariant of the
objective, and asserts that reverting the aggregation breaks it again, so
neither half can be quietly dropped.
The 500k cold-start baseline (39.12) is superseded -- this changes what "good"
means. The layouts stay valid and their fail counts are unchanged, but a fresh
corpus run is needed before any new number is compared with them.
Still untouched: circulation returns 0.07 per unit cost against a room's 0.66,
by far the worst thing a building can contain. That is homemaker-py-hxi.
Refs homemaker-py-ecx.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
39.17 left the search's storey choice unexplained and blamed value_rate. It is
not the rate, or not only.
Measured over the twelve baseline runs, value/cost by leaf kind: outside ground
7.40, roof terrace 2.69, room 0.34, circulation 0.02. A terrace returns 2.7x
its cost where a room returns a third of it, so filling upper storeys with
terrace is not the search leaving value on the table -- it is by a wide margin
the most profitable thing the objective offers. 7% of the corpus area produces
32% of its value.
Most of that gap is mean quality: 0.986 for a terrace against 0.223 for a
room. Quality is a PRODUCT of factors and the kinds are not asked the same
number of questions -- an outside leaf is exempt from size, crinkliness and
access, so 3 of 7 factors can ever bite it against a room's 6. Each exemption
is individually right (no programme size target; uncovered outside is lit by
definition; ground-level outside needs no access). The consequence is not: a
leaf exempt from the two harshest factors out-scores one judged on them and
doing well, purely by not being asked, and quality multiplies the value rate.
Stated generally, and this is not about outside space: under a product, adding
any new quality criterion mechanically devalues every leaf it applies to,
including leaves that score 1.0 on it. The objective's scale should not depend
on how many things it measures.
quality_aggregate="geometric_mean" (default OFF, "product" is stock) divides
that out. Computed in log space so six small factors cannot underflow the
product before the root is taken; a zero factor still gives zero, so a fully
buried leaf is worth nothing either way.
Telling "exempt" from "asked and scored 1.0" needs factor_is_asked, which
restates conditions that live inside the quality_* methods. That duplication
can drift, so tests/test_fitness_aggregate.py pins it against every leaf in the
corpus: wherever the predicate says exempt, the factor really is 1.0.
Fail set byte-identical everywhere, and for a stronger reason than 39.13/39.14
had: evaluate_leaf emits each fail from the factor itself before anything is
combined, so no aggregation can move one. Score effect +37% to +169%, reaching
all four programmes where the crinkliness changes reached two; room value/cost
0.34 -> 0.66, circulation 0.02 -> 0.07.
Deliberately not fixed: a terrace still out-earns a room 4:1, which is the
rates (value_supported = value_inside = 300 against costs of 110 and 200), not
the aggregation. That is a design judgement for the programme author, and
39.16 is a standing reminder that "this inherited constant looks wrong" has
been wrong twice already in this section. Left open on ecx with the numbers.
A/B running; verdict to follow.
Refs homemaker-py-ecx.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
39.16 relocated the crinkliness residual to a plan-form question. Four
measurements answer it, and two of them refute the premises 773 was filed on.
The search DOES build courtyards -- harbor 8 (273 m2), maple 16 (404 m2),
health-centre 13 (107 m2) over three seeds -- and they work: of 524 lit edges
44% come from the plot wall, 30% from a courtyard, 26% from a perimeter void,
and a courtyard supplies at least one side for 53% of the two-aspect leaves.
No operator is missing. (The shape-curve DP does NOT model exposure --
shapecurve.py:25 -- but per 38.24 it fires ~8 times in 500k evals, so that gap
is not what is costing anything.)
The answer is per-storey. Comparing each storey's demand, sum A_i/(1.6202*h),
with the lit wall its leaves actually hold: every harbor and maple ground floor
is below 1.0 and every top floor above 1.2, and the ratio predicts the fail
rate almost exactly -- above ~1.2 near-zero fails, below 1.0 40-55% of the
storey. health-centre and programme-house sit at 1.6-4.2 throughout and fail
essentially nothing.
That corrects 39.11, which divided demand evenly across storeys and concluded
harbor and maple were frontage-feasible "with room to spare". Programmes pin
rooms to level 0 and the ground floor cannot set itself back to buy perimeter:
harbor's pinned 347 m2 needs 71.4 m against the plot's 53.0 m, maple's 414 m2
needs 85.2 m against 55.0 m, while health-centre and programme-house have 51.0
and 14.2 m spare. Same ordering as the corpus fail counts, and fixed before any
search runs. The averaged check is not just weaker: on maple it asks for a
22 m2 courtyard where the ground floor needs 57 m2.
New third _preflight check, advisory like the others, silent on the two
programmes with slack. tests/test_evolve_preflight.py covers all three checks
and asserts the ground-floor figure exceeds the averaged one -- if they ever
agree, one has stopped earning its place. 39.11 annotated in place.
Also recorded, not acted on: the open space is on the wrong storey (harbor puts
50 m2 of courtyard on the starved ground floor and 223 m2 on the surplus first
floor), because value_rate pays an outside leaf above ground value_supported =
300 -- a room's rate -- against a cost of 110, with nothing tying its value to
whether it illuminates anything. Filed as homemaker-py-ecx.
411 passed, 72 skipped.
Refs homemaker-py-773.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
The owner supplied the provenance the analysis was missing. The constant is
Christopher Alexander, A Pattern Language 159, "Light on Two Sides of Every
Room", and it changes what the numbers mean.
1/crink = A/(L*h) is floor area per metre of ILLUMINATED wall over storey
height. It is not room depth -- it equals depth only for a room lit on one
side. So 5/6 * h = 2.5 m is 2.5 m of room depth PER WINDOW WALL: one side
allows 2.5 m at the peak and 4.86 m at the fail edge, two opposite sides allow
5.00 m and 9.72 m. A 4 m room scores 0.395 lit on one side and 0.902 lit on
two. The factor is the pattern stated as a ratio, and it is not
miscalibrated.
WITHDRAWN from 39.14: the "2.5 m absurd optimum" reading, and "the corpus's
realised median depth is 2.95 m, so the search built what it was paid for" --
2.95 was the median A/L, while the corpus's single-aspect leaves are a median
3.46 m deep and its two-opposite leaves 4.42 m. Ordinary rooms. Section
retitled, passage struck in place.
WITHDRAWN from 39.15: calling six specs "self-contradictory". They are large
rooms, and under Alexander a large room is supposed to need two aspects; the
audit's new column reports the pattern working, not a mis-specification. What
is real is the tension between that demand and what the plan form supplies.
SURVIVES, on a better argument: crinkliness_shape="daylight". 159 states a
MINIMUM, and a two-sided gaussian turns a minimum into a target -- 68% of the
133 leaves in the clipped region are lit on two or more sides, mean quality
0.770, docked for satisfying the pattern well, on top of the
exterior_wall/boundary_wall charge those windows already carry in cost.
New 39.16 records this and relocates the residual. Over the 430 graded
baseline leaves: unlit 77 (100% fail), one side 208 (15%), two-corner 87 (2%),
two-opposite 34 (0%), three/four 24 (4%). Light on two sides all but
guarantees a pass and only 33.7% of leaves get it, so the open question is why
a binary slicing tree on a convex plot can only give a third of its leaves two
aspects -- a plan-form question, not a scoring one. Filed as
homemaker-py-773; 39.11's courtyard finding is the same question from the
other side.
Both errors came from reading a dimensionless ratio as a length, so the
provenance and the interpretation now sit next to the constant in fitness.py,
not only in DESIGN.md.
405 passed, 72 skipped.
Refs homemaker-py-u5q.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
39.13 -- the tail rescale, and its verdict. NULL, and not for want of power:
12 of 12 pairs byte-identical, both programmes, same trajectories. The failing
tail is 0.034% of corpus value, so making it orderable cannot move a search.
Kept (free, and k54 needs the region orderable) but recorded as correct and
inert, not as a fix. It also corrects gvb's premise: zero exposure is NOT
beyond the inner loop's reach -- perturbing division ratios alone moves the
zero-exposure set on 6-12 of 12 trials at +-25%, and in the direction wanted
(harbor s1 8 -> 6 buried leaves).
39.14 -- what the factor actually rewards. 1/crink is the room's depth from
its daylit wall in storey-heights, so the variable is sound and its fail
boundary (1.62h = 4.86 m) agrees with 38.3's independently-derived frontage
bound. The two-sided gaussian on it is not: the near side penalises surplus
daylight that edge_cost and outside_edge_cost already bill at 100 and 133.3
per m2, it has never once produced a fail (it needs crink > 21.5; corpus max
is 3.95), and its peak sits at a 2.5 m deep room -- an ordinary 4 m room
scores 0.395 and the corpus's realised median depth is 2.95 m. The search
built what it was paid for. A/B at pilot budget is underpowered rather than
null: the arms reach different layouts but the same fail counts.
39.15 -- the magic numbers. A sigma is not a preference, it is an acceptance
interval target +- 2.1460*sigma, so it decides failures. The blanket
hypothesis does not survive -- programme-house reaches 1 fail, structural on
two of three seeds. The specific one does, and it shows 39.1's CLEAN verdict
answered a weaker question: sweeping a spec's tolerance box asks whether SOME
shape is feasible, and all 67 pass, but at the DECLARED target area and aspect
harbor needs 7 corner rooms and maple 6, while health-centre and
programme-house need none. Within-programme, those codes fail 62% and 78% of
their instances against 26% and 31% for all others. Three declared quantities
are jointly contradictory and nothing said so; the resolution is an author
decision, not a retuned constant.
Also recorded: 82% of size fails are rooms larger than target, which is the
same shape of double-charge but explicitly NOT the same case -- size's upper
bound is the main brake on growth and must not be removed on the analogy.
405 passed, 72 skipped.
Closes homemaker-py-9gj, homemaker-py-u5q.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
A fail is `quality < FAIL_THRESHOLD`, and every factor is a gaussian, so a
(target, sigma) pair does not express a soft preference -- it DEFINES an
acceptance interval, target +- 2.1460*sigma. Sigma is the tolerance that
decides failures, and these sigmas were inherited from Urb without a recorded
derivation.
audit_programme_config.py already swept each spec's whole tolerance box and
asked "is SOME shape in here feasible?". Every corpus spec passes that, which
is what 39.1 recorded as CLEAN. But a tolerance is not a design intent: the
author declared a target area and a target aspect, and that is the room they
asked for. Asking whether THAT room is feasible is a different question.
Six specs answer it differently -- harbor's cr1/da1/n and maple's da1/lr1/n.
Built as declared they need two exposed sides, a corner: harbor's common room
is 80 m2 at aspect 2.0, so 6.32 x 12.65 m, and 6.32 m is deeper than the
4.86 m single-aspect daylight limit. They are "feasible" in the box only at
the bottom of their area tolerance and the top of their aspect one -- the
search can satisfy them only by building something other than what was asked
for.
New `at declared target` column, plus a count of instances needing >=2 sides
as declared: harbor 7, maple 6, health-centre 0, programme-house 0. The
corner budget is reported as an inequality against storey count rather than a
fixed number, since init.dom is one storey for every corpus programme and the
search grows the rest.
_multi_aspect() replaces the duplicated substring test, and _audit_at_target
reuses the live Gaussians rather than reimplementing them.
Refs homemaker-py-u5q.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
The tail rescale shipped in cd392e7 is a measured NULL as a search
intervention -- 12 of 12 pairs byte-identical on harbor and maple, 8000 evals
from a plateau, not merely underpowered. Of course it is: the whole failing
tail is 0.034% of corpus value. Looking at the rest of the factor, prompted by
the owner, found something much larger above the threshold.
crink = area_outside/area = (L*h)/A, so 1/crink = A/(L*h) is the room's mean
depth from its daylit wall in storey-heights. That is the right variable for a
daylight rule, and the fail boundary it implies (1/crink = 1.62, i.e. 4.86 m at
h=3) is a sensible one that agrees with 38.3's frontage bound derived
independently. What is wrong is hanging a TWO-sided gaussian on it:
* The near side penalises a room for having MORE daylit wall than target --
while leaf_cost's siblings edge_cost and outside_edge_cost already charge
that same wall at exterior_wall=100 and boundary_wall=133.3 per m2. The wall
is billed once in cost and again as lost value.
* It never earns its keep as a failure either: the over-exposed branch only
reaches FAIL_THRESHOLD above crinkliness 21.5, and the corpus maximum is
3.95. It has never produced a single fail; it only removes value.
* 133 of the 318 passing graded leaves in the 500k baseline (42%) sit on that
side, mean quality 0.810.
crinkliness_shape="daylight" (default OFF, "gaussian" is stock) clips it: a
room shallower than the gaussian's peak scores 1.0, because daylight is a
sufficiency requirement and surplus is the cost model's business, not this
factor's. Clipping at the PEAK rather than at FAIL_THRESHOLD is deliberate --
it keeps the factor continuous and preserves the graded approach to the
daylight limit, where clipping at the threshold would put a 10x cliff on the
exact boundary the 0.5**n fail multiplier already steps on.
Fail set byte-identical on all 21 corpus artefacts for all four
shape/tail combinations, so stock stays a valid yardstick for every arm.
Area-weighted crinkliness quality 0.480 -> 0.513, leaf quality product
0.2722 -> 0.2831; per-artefact score +0.2%..+19.6%, and unlike the ramp it
reaches health-centre and programme-house, where the tail change was 0.000%.
Note "daylight" clips the OPPOSITE side from 38.1's superseded compact_ok,
which forgives being buried; composing either with those modes is refused.
ab_9gj_ramp.py becomes ab_9gj_crinkliness.py and takes named arms, since it
now covers both changes; its first arm is the baseline and the yardstick.
Refs homemaker-py-9gj.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
Only the merged <out>.csv is worth keeping; the shards are intermediate by
construction and one per worker process.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
quality_uncrinkliness evaluates a gaussian at x = 1/crink, so its exponent
grows like 1/crink^2 and underflows a double to exactly zero below crink ~
1/15. Measured over the twelve 500k cold-start runs (39.12): 430 leaves carry
a minimum-exposure requirement, 112 fail it, and those 112 span quality
1e-300..1e-1 while contributing 0.034% of total value on 23% of the floor
area. Every value in that range is numerically zero beside a passing leaf's
~1, so the search cannot rank two layouts that differ only in how exposed
their under-lit rooms are.
This is wider than the bead's diagnosis (a flat 0.0 for zero-exposure leaves)
and it explains why 38.1's `floor` mode measured as a no-op: max(q, 0.01) maps
110 of the 112 onto one constant, replacing a flat zero with a flat 0.01.
crinkliness_tail="ramp" (default OFF, "gaussian" is stock) replaces the tail --
only the tail, only below FAIL_THRESHOLD, only on the compact side -- with a
straight line in crinkliness meeting the gaussian exactly at the crossing.
_crink_at_fail_threshold inverts the gaussian there using the same truncated
_E the factor is evaluated with.
Deliberately conservative: nothing at or above FAIL_THRESHOLD moves, so no
calibration changes and no leaf crosses the threshold. The fail set is
byte-identical on all 21 committed corpus artefacts, the four init.dom seeds
included -- asserted in tests/test_fitness_crinkliness_tail.py, not assumed.
That invariance is also what makes it legal to score both arms of the A/B
under stock (the 38.9 trap's one exemption). A fully buried leaf still scores
exactly 0; this restores an ordering within the failing region, it does not
forgive it. Composing with 38.1's superseded modes is refused, since both
rewrite the same tail.
Score effect on the baseline artefacts: +0.3%..+2.8% on harbor and maple,
exactly +0.000% on health-centre, programme-house, and every init.dom -- a
programme with no partially-exposed failing rooms has nothing to grade, and
neither does any starting layout. The ramp is a mid-search signal by
construction, so experiments/ab_9gj_ramp.py defaults to seeding each run from
a 500k plateau artefact rather than cold.
The module-level math import replaces a now-redundant local one.
DESIGN.md 39.13 and the A/B verdict follow in a separate commit.
Refs homemaker-py-9gj.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
DESIGN.md 38.7 pinned Phase 9's acceptance test to "harbor-house reaches its
known 15-fail floor". That figure was measured before 39.4, against harbor's
*effective* programme of 32 instances -- cr1/of/st1/st2 were being read as
generic circulation and silently dropped. Against the real 37-instance
programme the same artefact scores 89, so the target is not measurable; and
2v1, the fix it was the acceptance test for, closed NULL (39.8), so there is
no combined fix left to accept.
New 39.12 records:
* the five evolved-3M*.dom artefacts rescored under the current objective
(69/85/87/89/145), and why they are not a floor -- they were evolved under
one objective and are scored under another;
* that the bead's migration premise is stale: experiments/migrate_ju3_rename.py
does not exist, because ju3 tightened the matching rule at source (39.3)
instead, so the old artefacts parse correctly with no migration;
* the 4x3-seed 500k cold-start baseline (~430 h) as the reference from here,
with each programme's sd and its minimum detectable difference at n=3 --
harbor mean 39.3, sd 5.5, MDD 13.7;
* zero missing-space fails in all twelve runs: the dominant term in the 3M
artefacts is not one the live search still fails on;
* crinkliness at 112 of the 321 corpus fails (35%), all soft, all in the
regime 9gj says quality_uncrinkliness returns a flat 0.0 for -- the largest
single component of the residual is one the objective cannot descend;
* 66 of the 84 hard fails as one access-topology family (not-adjacent-to,
inaccessible usable space, not connected), mechanism in 39.9.
38.7's acceptance paragraph is annotated in place rather than rewritten. The
connectivity clause is demoted to a separately tracked standing defect: it
appears in 10 of the 12 baseline runs.
tests/test_collapse_insearch.py carried its own stale "82 -> 58"; the same
layout now scores 89 -> 64, so the docstring dates the figure instead of
asserting a current one.
Closes homemaker-py-ut5.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
Cold-start re-baseline after the DESIGN.md 38.10/38.11 objective
change. Single worker (avoids homemaker-py-b8g), scored by the
shipped scorer from the programme directory.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
Cold-start re-baseline after the DESIGN.md 38.10/38.11 objective
change. Single worker (avoids homemaker-py-b8g), scored by the
shipped scorer from the programme directory.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
Cold-start re-baseline after the DESIGN.md 38.10/38.11 objective
change. Single worker (avoids homemaker-py-b8g), scored by the
shipped scorer from the programme directory.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
Cold-start re-baseline after the DESIGN.md 38.10/38.11 objective
change. Single worker (avoids homemaker-py-b8g), scored by the
shipped scorer from the programme directory.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
Cold-start re-baseline after the DESIGN.md 38.10/38.11 objective
change. Single worker (avoids homemaker-py-b8g), scored by the
shipped scorer from the programme directory.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
Cold-start re-baseline after the DESIGN.md 38.10/38.11 objective
change. Single worker (avoids homemaker-py-b8g), scored by the
shipped scorer from the programme directory.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
Cold-start re-baseline after the DESIGN.md 38.10/38.11 objective
change. Single worker (avoids homemaker-py-b8g), scored by the
shipped scorer from the programme directory.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
Cold-start re-baseline after the DESIGN.md 38.10/38.11 objective
change. Single worker (avoids homemaker-py-b8g), scored by the
shipped scorer from the programme directory.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
Cold-start re-baseline after the DESIGN.md 38.10/38.11 objective
change. Single worker (avoids homemaker-py-b8g), scored by the
shipped scorer from the programme directory.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
Cold-start re-baseline after the DESIGN.md 38.10/38.11 objective
change. Single worker (avoids homemaker-py-b8g), scored by the
shipped scorer from the programme directory.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
Cold-start re-baseline after the DESIGN.md 38.10/38.11 objective
change. Single worker (avoids homemaker-py-b8g), scored by the
shipped scorer from the programme directory.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
With tym landed the DP finally runs on real leaf-sharing searches, so v4s
asked for the search-level A/B. No A/B was run: two structural facts make
the payoff zero before any seed is drawn, and measuring a no-op would have
produced a null that reads like a measurement rather than a fact.
Reach. driver._evaluate gates the warm-start on `x0 is None`, and every
child gets x0 = innerloop.warm_x0(...) from its parent (driver.py:764), so
it fires only for bootstrap-population individuals. Instrumented over a
4000-eval run: 8 DP solves, exactly pop_size. At the corpus baseline's 500k
budget that is 8 evaluations out of 500,000.
Applicability. Constructed-seed feasibility over 6 seeds: harbor-house 0/6
with leaf_sharing on AND off, programme-house 0/6, harbor-house-l0 4-5/6
(75% over a real search). Even those 8 bootstrap individuals get no warm
start on the real programmes; only l0, the reduced programme the DP was
validated on, is feasible.
The infeasibility is correct, checked because it is the dangerous
direction: on 4 harbor topologies the DP rejects, an NM polish minimising
the shape-fail family reaches 14/16/17/16 fails, never 0 -- 0/4 false
negatives. Full harbor is genuinely shape-infeasible per topology where l0
is not. Per topology only: programme-house reaches 0 hard / 1 soft in the
500k baseline while its constructed seeds are DP-infeasible.
shapecurve_prune is separately inert -- it acts only inside the
feasibility_max_shape_fails branch and its exact arm needs best_n_fails<=0.
tym was still worth doing: the DP now models leaf-sharing exactly (38.23)
and fires on real runs, which is what turned an untestable question into a
structural answer.
Filed homemaker-py-ldj for what would have to change: give the DP reach
beyond the bootstrap, and/or use infeasibility as a ranking signal (an
infeasible topology still has a shape-fail floor, which is what a
pre-filter wants) rather than a precondition that discards it.
Closes homemaker-py-v4s.
Lint at parity (46).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
shapecurve.leaf_constraints derived each leaf's feasible area from its own
type's base (target, sigma). quality_size does not: a leaf holding k
same-code rooms is centred on k*target with sigma*k, and a co-typed leaf
adds both codes' targets. The DP modelled neither, so eligible() excluded
leaf_sharing/max_share/multi_use -- and leaf_sharing defaults True in
driver.search, so the guard excluded essentially every real run. The DP was
correct and unreachable.
Why the guard could not just be dropped, measured before touching it: on 6
harbor constructed seeds, 24 of 24 shared leaves (100%) have a real area
outside the unscaled single-room bounds. Relaxing eligible without
modelling k would have made the DP call every one of those topologies
infeasible -- false negatives that prune feasible topologies and misdirect
the NM warm-start. The guard was load-bearing.
Fix: mirror quality_size by asking the SAME Fitness object -- k =
graph.leaf_share(leaf, fit._max_share) when fit._leaf_sharing, then
target*k / sigma*k, else fit._leaf_co_type for the additive case. Same
object, same flags, same branch order, deliberately not re-derived: 39.5's
cpsat._matches bug was a solver optimising a relation the scorer had moved,
and this is the same hazard class.
Verified as an exact inversion: for every shared leaf in a real seed,
quality_size evaluated at the DP's amin and amax returns FAIL_THRESHOLD to
1e-9 (k=3 n-leaf: bounds [128.50, 231.50], both 0.100000).
superpose stays excluded for a different reason than the others: it does
not rescale a target, it changes which type the leaf is scored as, and the
collapse happens after the DP has read leaf.type.
shapecurve_warmstart/shapecurve_prune remain default off, so no current run
changes -- including the cold-start baseline in progress. They are now
applicable, which unblocks homemaker-py-v4s.
Closes homemaker-py-tym.
Lint at parity (46); tests 387 passed (3 new, 1 legacy rewritten to the new
contract rather than deleted), 0 failed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
Three times in this log a verdict rested on a sample that could not have
produced it: 38.19 (programme-house claimed at N=20, resolves at N=60),
38.21 (harbor at n=3 resolves nothing finer than ~15 fails, yet every
recorded margin is smaller), 39.5/38.20 (a 10-fail cpsat margin inside a
+-23-fail noise band). Each was found years later.
experiments/ab_report.py makes it visible when the verdict is made:
minimum detectable difference = t_crit(0.975, N-1) * sd / sqrt(N)
A margin below the MDD is not a weak result but an absent one -- the
experiment could not have distinguished it from zero however it came out.
The report flags that, refuses to endorse a winner, and states the N needed.
Validated against both datasets measured this session, reproducing the
hand-computed figures exactly:
programme-house N=60 +0.567 p=0.017 MDD 0.462 verdict supported
programme-house N=20 +0.700 p=0.085 MDD 0.805 UNDERPOWERED, N~=26
harbor N=24 +1.208 p=0.502 MDD 3.669 UNDERPOWERED, N~=202
Harbor needing ~200 seeds means it cannot answer the collapse_insearch
question at any N this project would realistically run.
Fixed a defect in my own first version: with all-ties (sd=0) the MDD
collapses to zero and the naive abs(mean) < mdd reported "margin exceeds
the MDD -- verdict supported" for a margin of 0.000, with t=nan. A reporter
that endorses a zero margin is worse than none. Degenerate cases are now
explicit and distinguish all-ties from a constant non-zero difference.
Separate correction found while validating: 38.19's published p-values for
N=20 and N=40 were 0.069 and 0.045, from a normal approximation. The exact
paired t-test gives 0.085 and 0.052 -- so N=40 did NOT reach significance
either; it took N=60. The approximation was anti-conservative, making
results look more significant than they are, the same direction of error
this thread is about. Corrected at all four citing sites, and the "needs
N >= 40" guidance raised to N >= 60.
Closes homemaker-py-tco.
Lint at parity (46); tests 384 passed, 0 failed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
record_and_push used `git add -A`, so each completion committed the entire
working tree. On the live local run that meant one commit carrying 49 files
and ~1.97M insertions -- three still-running programmes' partial
coldstart-*.dom, plus unrelated evolved-*.dom -- under a message naming
only programme-house seed 0. In-flight artefacts were being recorded as if
they were results, attributed to the wrong run.
Now stages and commits exactly this run's .dom/.log/.score/.fails plus the
results TSV, via `git commit --only <paths>` so it holds regardless of what
else is staged and a concurrent edit elsewhere cannot ride along.
Note for the run currently in progress: it has the old code loaded, so it
will keep sweeping until restarted. The committed .dom files for
harbor-house, maple-court and health-centre are mid-run snapshots, not
results -- their logs show 75k/87k of 500k evals -- and should be
disregarded until those runs report their own rows.
First real result is in: programme-house seed 0 @ 500k, finish collapse
28 -> 1 fails, final 1 fail (0 hard / 1 soft), 19400s.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
The 0wr commit referenced the follow-up as 'homemaker-py-...' because the
ID did not exist yet; it is homemaker-py-tco. Its description also lost the
formula to a shell backtick -- restored via stdin:
minimum detectable difference = t_crit(0.975, N-1) * sd / sqrt(N)
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
0wr asked which harbor A/Bs were decided by a narrow margin before 39.4.
Measuring harbor's variance makes the margin-by-margin triage moot.
Harbor's paired seed-to-seed sd is 6.19 fails (24 paired ON/OFF runs,
budget 2500), so n=3 resolves nothing finer than 15.4 fails. Every recorded
harbor margin is below that: 13.9 share_edge_cap 3.7, 20 qpk 8.3, 23 f1d
mixed, 37.1 tiering 6.3. So it is not that SOME harbor results were narrow
-- no harbor A/B run at three seeds could resolve the margin it reported,
independently of what 39.4 did to the programme. Of the 220 possible
3-seed subsets of the 24 runs, 56 (25%) show a clean 3/3 sweep for ON.
Re-measured 20's harbor arm, the one backing a live default:
N=3 2W/1L/0T +2.67 p=0.560
N=12 8W/3L/1T +3.50 p=0.076
N=24 13W/10L/1T +1.21 p=0.502 CI [-2.46,+4.88]
Null. The published "harbor: ON wins 3/3, 80.3 -> 72.0" was a lucky draw --
even seeds 1-3 measured here give 2W/1L, not a sweep.
So 20's claim that the qpk verdict "holds at both example scales tested" is
withdrawn and annotated in place. collapse_insearch's default rests on
programme-house alone (38.19, N=60, +0.57, p=0.017). It is not refuted on
harbor -- direction positive but indistinguishable from zero -- but harbor
must not be cited as corroboration.
Harness generalised (PROG/BUDGET/WORKERS) and results kept.
Filed homemaker-py-... : A/B harnesses should report the minimum detectable
difference for the N they run, so an underpowered verdict is visible when
it is made rather than years later.
Closes homemaker-py-0wr.
Lint at parity (46).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
38.20's cap fix took the suite from ~4.5 to ~10 min because the
assign_cpsat tests now solve to optimality. Recovered to ~6.8 min.
The bigger win was not the threading. The secondary-adjacency test ran the
cpsat arm THREE times and averaged, and its own comment says why: the cpsat
path "is not yet bit-reproducible (homemaker-py-fdp)", so one 10-seed
aggregate could straddle greedy's deterministic value and the test was
flaky by construction. fdp is fixed (38.15), so one pass says exactly what
three did -- that was work spent papering over a bug that no longer exists.
constructive_topology and _assign_adjacency_aware now forward an optional
cpsat_limits=(time_limit_s, deterministic_limit); default None keeps
solve_room_labels' defaults, so production is unchanged -- verified 24/24
harbor solves still OPTIMAL at the defaults. It is not a tuning knob: it
exists so a test whose claim does not depend on optimality can economise.
test_construction_assign_cpsat_yields_valid_seed asserts invariants only
and uses it, 91s -> 53s.
That test now also guards a real trap: too small a budget makes
solve_room_labels return None, _assign_adjacency_aware falls back to
greedy, and the test would pass while exercising nothing. It counts
fallbacks and fails if any occur.
The two quality comparisons keep the full budget deliberately -- their
claims are about the optimum, and cheapening them would weaken what they
assert. That is why the suite does not return to 4.5 min; the residue is
the honest price of optimal deterministic solves.
Also corrected a stale claim in the secondary-adjacency comment: it
measures only "not adjacent to" fails and is not a claim that cpsat seeds
better overall, which 38.20 measured markedly worse.
Closes homemaker-py-7t1.
Lint at parity (46); tests 384 passed, 0 failed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
Cold-start re-baseline after the DESIGN.md 38.10/38.11 objective
change. Single worker (avoids homemaker-py-b8g), scored by the
shipped scorer from the programme directory.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
The follow-up issue was cited by an ID written before the issue was
created. The real one is homemaker-py-7t1.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
