All four 500k runs died about 10 minutes in when the container was
reclaimed. No SIGTERM fired, so no .dom was written and 0 of 12 runs
completed. My plan committed results per finished run, which protected
nothing because no run reached its commit point. The bad assumption was
reading "reclaimed after inactivity" as CPU inactivity; it is conversation
inactivity, and background compute does not hold the box open.
Progress reached before the loss (from the tracked logs): harbor 24,960
evals / 40 fails, maple 14,880 / 79, health-centre 25,920 / 33,
programme-house 138,800 / 2.
The underlying gap is not environmental: a search's only output lands at
the very end or on SIGTERM, so ANY abrupt loss -- reclaimed container, OOM,
power cut -- takes the whole run with it. On a 3M-eval search that is 2.4
days of compute with no recoverable artefact.
- driver.search gains checkpoint=/checkpoint_every=: the current best is
handed to a callback at most every N evals. Rate-limited by evals, not
improvements, which come in bursts early. A failing checkpoint is logged
and swallowed -- losing a checkpoint is bad, losing the search because a
checkpoint failed is worse.
- homemaker-evolve --checkpoint-every N writes <out>.dom.checkpoint via
mkstemp + os.replace, so a crash can never catch it half-written. It is
deliberately NOT the output path: a checkpoint is a leaf-sharing run's
internal best, dishonest under the canonical scorer until the finish
stage unfolds it (homemaker-py-3l6), and must not be mistaken for the
finished article.
- Verified the written checkpoint re-loads as a valid .dom.
Default off, so behaviour is unchanged without the flag.
Lint at parity (46); tests 372 passed (3 new), same 2 pre-existing failures.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
Adds src/homemaker_layout/cpsat.py (OR-Tools CP-SAT) as an exact alternative
to operators._assign_adjacency_aware's greedy/beam room-code placement,
wired in as assign_solver="greedy"|"cpsat" (EXPERIMENTAL, default "greedy",
byte-identical to before) through constructive_topology/lift_base_to_storeys/
driver.search, plus a new operators.mutate_reassign in-search repair
operator (driver.search's enable_reassign=False default, mirrors
enable_ruin_recreate). Both found and fixed a resize-fragility bug (a
second CP-SAT pass against settled geometry, operators._cpsat_relabel_settled)
and a CP-SAT symmetry-blowup stall (explicit interchangeable-code grouping).
Seeder-level A/B on harbor-house is a solid, low-noise positive (~13% fewer
real fitness-scored secondary-adjacency fails, 10 seeds). Full driver.search
A/B is only pilot-scale (budget=3000 vs the bead's own 20k target) and
inconclusive -- both flags stay default-off pending a larger-N confirmation.
Full writeup: DESIGN.md §37.7. Bead left in_progress (own acceptance
criteria not fully met); homemaker-py-5bv tracks the deferred post-collapse
repair item.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LSwQwpEaHFBkeVSDDWd75S
Generalise shapecurve.py's DP to process dom.levels(root) bottom-up per
storey instead of assuming a single free tree. A divided node's split is
free only per solver.free_branches' own criterion (below is None or
undivided there) -- geometry.coordinate always mirrors a below-linked
node's corners from the storey below regardless of whether that storey's
counterpart is divided, so every free region at any storey reduces to the
exact same single-region problem the pre-existing _check/realise already
solved. New _region_roots finds below-fixed leaves (checked directly,
gridless) and below-fixed-box/free-split fringe nodes per storey;
_solve_all_levels realises each storey before checking the one above and
snapshots+restores on any infeasibility, preserving solve()'s all-or-nothing
and is_feasible()'s never-writes contracts across the whole tree.
eligible() now allows any storey count.
Validated on the real (non-de-risked) examples/harbor-house: 200 random
2-storey topologies, DP-vs-NM agreement 99.5%, 0 false negatives, 117.7x
speedup (DESIGN.md §37.6). Full suite 397 passed.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LSwQwpEaHFBkeVSDDWd75S
Adds shapecurve.is_feasible() (a non-mutating refactor of solve()'s check
phase) and a shapecurve_prune flag composing the DP's exact feasible/
infeasible verdict with operators.predicted_shape_fails' existing heuristic
prune: DP-feasible vetoes a heuristic prune outright; DP-infeasible only
hard-prunes when the incumbent already has zero total fails (exact, since
infeasible proves the shape-fail floor is >=1); otherwise defers unchanged
to today's heuristic threshold. Conservative by design since a wrong prune
is unrecoverable.
Validated 0/400 false negatives across two structurally distinct plots
(harbor-house-l0 + a newly-added programme-house sweep, the first genuinely
non-rectangular plot this DP has been checked against). The real
driver.search A/B on harbor-house-l0 measured NULL (byte-identical off/on)
for a root-caused, pre-existing reason: predicted_shape_fails rarely
triggers organically at this scale, so neither new branch had an opening to
fire -- not a defect in this change. Full writeup: DESIGN.md §37.5.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LSwQwpEaHFBkeVSDDWd75S
Promotes the validated shape-curve DP (experiments/shapecurve_spike.py,
2g7.4, DESIGN.md §37.2) from a reference-only spike into
src/homemaker_layout/shapecurve.py, and wires it into driver._evaluate as a
warm-start for innerloop.optimise: when eligible (single storey, no
leaf_sharing/superpose/max_share/multi_use) and no caller-supplied x0, the
DP's exact shape-feasible ratio point is written onto the tree before NM
runs, off by default (shapecurve_warmstart=/--shapecurve-warmstart).
Caught and fixed a latent bug promoting the spike: realise() could leave
numpy.float64 in `division`, which yaml.safe_dump can't serialise — the
original spike never round-tripped through dom.dumps so this was never hit.
A/B on harbor-house-l0 (experiments/ab_shapecurve_warmstart.py, budget=2000,
5 seeds): mean total fails 16.6 (on) vs 19.6 (off), ~3.5x mean fitness
improvement; mean hard-fail count alone was a noise-level wash at this
sample size. Full writeup in DESIGN.md §37.4.
Deliberately deferred to new tracked beads (children of 2g7): DP-exact hard
pre-filter (wkh), multi-storey below-link support (koo), leaf_sharing/
co_type modelling (tym), true skew-quad polygon algebra (ekc) — 6xh stays
in_progress pending those.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LSwQwpEaHFBkeVSDDWd75S
Composer half of the ground-truth bead: examples/harbor-house/drawings/
harbor-house 1.svg turned out to be a Bonsai/Blender render of 3m.dom's own
IFC (32 IfcSpace paths == 3m.dom's upper-storey leaf count), not a human
trace, so no usable reference exists yet -- this builds and tests the
pipeline that will consume one once traced. compose.py parses storey-N
Inkscape layers of cut-lines + labels against a boundary-stub .dom (plot/
height/elevation only, no room shapes to keep aligned across storeys) and
recursively detects guillotine cuts, mirroring geometry.py's own
division-line algebra; non-slicible regions and label mismatches are
reported by location rather than guessed at. homemaker-compose CLI added.
Renamed dom._link to public dom.link since compose.py needs to re-link from
outside dom.py. Full design writeup in DESIGN.md sec 37.3; actual human
tracing of harbor-house/programme-house is tracked as follow-up under
2g7.1, still open.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LSwQwpEaHFBkeVSDDWd75S
Splits the flat outer-search comparator (-n_fails, fitness) into a tiered
(-n_hard, -n_soft, fitness) so search budget stops being spent polishing
SOFT shape fails (crinkliness/proportion/size/width/edge-too-long/
staircase-volume) while HARD structural fails (missing space, wrong/
required level, level/circulation/vertical connectivity, adjacency,
stairs, covered-outside, storey limits, public access) remain unfixed.
fitness.classify_fail_tier/tier_counts classify every fail string emitted
across fitness.py and graph.py, raising on anything unrecognised so new
fail sites must declare a tier. Validated against all real fail strings in
the checked-in corpus plus every fail-emission call site read from source.
driver.Individual gains n_hard/n_soft (populated from innerloop.Result.
fail_lines); search(use_tiers=...) swaps the comparator when set (default
off, so existing runs are unaffected — inner-loop 0.5^n cliff untouched).
evolve.py exposes --use-tiers / HOMEMAKER_USE_TIERS.
experiments/tier_ab_2g7_3.py runs the acceptance A/B (harbor+maple, 3
seeds, 20k evals) in the background; results pending.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LSwQwpEaHFBkeVSDDWd75S
Adds operators.mutate_bridge_circulation: retypes the cheapest path between
two disconnected circulation components to circulation, directly clearing a
'level N not connected' fail instead of relying on the qi6 graded comparator
key (measured negative, DESIGN.md §18). Gated off by default via
driver.search's enable_bridge_circulation flag and evolve.py
--bridge-circulation, mirroring enable_reassociate's clean-toggle pattern.
qi6/qpk-protocol A/B (DESIGN.md §21) is directionally positive but mixed at
N=3/N=5 (never worse on total fails; clears 2/5 baseline not-connected fails
vs qi6's 0/4; one seed's RNG-trajectory divergence adds 2 new not-connected
fails) — kept default off pending a larger-N confirmation sweep
(homemaker-py-qjg) and a mutation-weight bump experiment (homemaker-py-lj3).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GDZjAATDWW1xFfc7xnJqSt
20-seed programme-house sweep (vs the original 5) resolves the qpk A/B's
mixed 3/5 result as small-sample noise around a true small positive: mean
fails 7.95->7.10 (~10.7%), 11W/6L/3T, paired t-test p~0.028. Flips
collapse_insearch's default from OFF to ON in evolve.py and driver.py
(_overrides_for/_fitness_for/_evaluate/search/polish_finish); opt out with
--no-collapse-insearch. fitness.Fitness itself is unchanged.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GDZjAATDWW1xFfc7xnJqSt
driver.search()/search_staged() gain enable_shape_repair (default off),
mirroring the enable_reassociate clean-toggle pattern: only builds a
fitness.Fitness instance and passes it to operators.mutate() when
enabled, so shape_rotate/deslim (7fm) can actually be selected mid-GA
instead of always no-opping on fit=None.
Full A/B sweep (harbor-house, budget=1M, 4 seeds) shows no improvement:
mean fails 14.50 (off) vs 14.75 (on), within seed noise. Confirms 7fm's
finish-time finding at in-search scale — these operators don't rescue
harbor-house's residual fails even with GA selection pressure.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GDZjAATDWW1xFfc7xnJqSt
Ramp the leaf-share grain down within one continuous run (e.g. 4->3->2->off),
carrying the whole population across each step — graduated non-convexity over
the single hard sharing->off transition of the §15 finish.
- operators.unfold_shared_leaves(above=cap): unfold only leaves whose share
exceeds the new grain cap, leaving smaller-share leaves collapsed for the
next step. above=1 (default) keeps the full-unfold §15 behaviour.
- driver: max_share override threaded through _overrides_for/_fitness_for/
_evaluate so a phase can rebuild the evaluator at a lower leaf_share_max cap;
search(seed_pop=) evaluates an explicit initial population so a phase hands
its whole population to the next instead of restarting from a single best.
- driver.search_annealed: one phase per descending grain then a de-share
polish; unfold-above-cap between steps; cumulative accounting + grain-tagged
history; honest canonical best (byte-for-byte verified vs homemaker-fitness).
- evolve: --anneal-grain LADDER CLI (self-finishing; §15 finish not applied).
8iv settled the primitive (grid unfold beat the circulation-aware slice), so
the ramp reuses the plain balanced-grid unfold at every step.
Tests: unfold above-cap selectivity, seed_pop seeding, search_annealed phase
stitching / honest finish / degenerate-ladder fallback. 258 pass. DESIGN §16.
Head-to-head A/B on harbor-house still to run; verdict pending (issue open).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M8566xAxTnwtJTkpXjYNZm
Leaf-sharing evolve runs optimised a sharing-credited objective (a shared
leaf of code X with share=k counts as k programme rooms, size re-centred on
k*target) but wrote the un-materialised genome, which the canonical
homemaker-fitness rates catastrophically worse (harbor-house: internal
1.03e-05 vs canonical 6.73e-29, 15 critical missing-room fails).
Fix: before write, driver.polish_finish() unfolds every live shared leaf
into k distinct rooms (operators.unfold_shared_leaves — pays down the count
deficit) then warm-starts a leaf_sharing=False polish search from the
unfolded genome so the materialised rooms get proportion/width/size cleanup.
Returned best.fitness is the canonical score (leaf_sharing off => internal
== canonical). This is yaa's proven unfold-then-polish path, made automatic.
evolve.py: new --polish-budget (env HOMEMAKER_POLISH_BUDGET; -1=auto=
budget//2, 0=unfold+rescore only). Interrupt forces polish_budget=0 for a
fast honest output. Default stays --leaf-sharing on (its topology-search
speed retained; output made honest by the finish). Schedule B in-run
annealing remains homemaker-py-kpu.
Verified (harbor-house 3000+1500): reported polish fitness 4.79788e-27
matches canonical homemaker-fitness exactly, 0 critical fails. Tests: 254
pass (+3 polish_finish).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M8566xAxTnwtJTkpXjYNZm
The constructive seeder was never nondeterministic: _assign_adjacency_aware
ends every max/min with a unique leaf-idx tiebreak and uses set unions only
for membership, so iteration order never leaks. constructive_topology(seed=0)
is byte-identical across processes for every example programme. The cited
"sig 4480 vs 16064" was a measurement artifact — Python's builtin hash() of a
str is salted per process (PYTHONHASHSEED), so an identical signature hashes to
different ints run-to-run.
The real run-to-run noise was parallel-only: driver._run_batch admitted futures
via as_completed (completion order), and admit() is order-sensitive (accrues
n_evals per result; keeps the first individual of an equal-key tie as best). A
long parallel run diverged 167 vs 161 fails (maple seed 0). Fix: admit futures
in submission order (block on each result in turn; all still run concurrently),
reproducing the serial admission sequence. Two workers=4 runs are now
byte-identical. Serial (workers=1) was already byte-for-byte reproducible.
Per-seed numbers are reproducible only at a fixed worker count; serial != parallel
is expected (children/iteration 1 vs n_workers changes batch granularity).
- driver: iterate futs in submission order, not as_completed
- test: test_search_parallel_is_reproducible (fails on pre-fix, passes on fix)
- DESIGN.md §12.4: corrected the reproducibility note
Closes homemaker-py-xcy
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Land the two evidence-supported parts of the re-scoped 9gp capstone as
operators on the existing decoded Node tree (no Polish-expression rewrite),
each default-OFF and measured against the §12.2 leu.2 baseline.
9gp.1 shape-feasibility pre-filter: operators.predicted_shape_fails lays a
topology out at its proportion-aware target geometry and counts shape fails
(size/width/proportion/crinkliness); driver._evaluate prunes clearly-infeasible
topologies before the inner loop (1 eval vs ~80), guarded so nothing that could
beat the incumbent is discarded. search/search_staged feasibility_filter,
feasibility_max_shape_fails (env FEAS/MAXSHAPE), default OFF.
9gp.2 M3 Wong-Liu reassociate: operators.mutate_reassociate adds associativity
(a|b)|c <-> a|(b|c) on same-orientation live cuts — the canonical-slicing move
missing from swap(M1)/rotate(M2), attacking the §11.4/§11.5 reachability
bottleneck. enable_reassociate (env REASSOC), default OFF (weight 0 -> baseline
byte-identical).
Unit tests (operators + driver) green, full suite 211 passed; maple-court smoke
run clean under native fitness. A/B sweep handed off per the plan; DESIGN.md
§12.3 documents the design and the pending measurement.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
genome.signature: ratio-invariant structural topology hash (per-storey tree
shape + cut orientation + leaf types), the cheap stand-in for the 9gp canonical
encoding. driver gains niche_by_signature (one individual per topology, replaces
the fitness-scalar dedup) and restart_patience (soft restart: keep elites,
refill with fresh seeds); SearchResult gains n_distinct_signatures /
diversity_history / n_restarts.
Diversity criterion MET (final-pop distinct ~5/16 -> 16/16). Gate NOT met:
blank-slate programme-house mean fails 12.3(legacy)/12.7(niche)/13.0(restart)
over 3 seeds at 20000 evals; harbor staged 95/94/108. Niching is a tie within
seed noise, restarts strictly worse — falsifies the premise that the
fitness-scalar dedup causes premature convergence. Both flags default-off,
kept for reuse. Epic c4c complete.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Outer search now ranks individuals by (-n_fails, fitness) instead of raw
fitness scalar. This prevents high-score 3-fail designs from displacing
2-fail designs in tournament selection and population replacement — the
root cause of the §4.8 pathology where flag count dominates geometry.
Inner loop is unchanged: it still optimises against the raw 0.5^n fitness
scalar, so the cliff that prevents trading into new failures remains intact
(0/9 regressions in experiments/penalty_reshape.py).
Also removes stale _CHILD_INNER_KW = {"sigmas": (0.05,)}: this was left
over from the CMA-ES era; the NM inner loop default (homemaker-py-d6d)
does not accept a sigmas parameter.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add n_workers parameter to driver.search(). When n_workers > 1, a
ProcessPoolExecutor evaluates the bootstrap batch and main-loop children
in parallel, giving near-linear speedup with core count. The geometry
module-level cache is cleared in each worker after fork to prevent stale
id-keyed entries. Serial behaviour (n_workers=1, default) is unchanged.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Copy programme-house corpus (36 .dom + .score + .fails + patterns.config)
into examples/ and update all 5 test files to use project-relative paths.
Native Python fitness (use_native=True) was already the default; tests now
run without /home/bruno/src/urb present.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When the seed is an undivided bare plot (init.dom), auto-generate pop_size
random topologies before the memetic loop starts, each evaluated at
child_budget. This crosses the zero-feasibility region that single-seed
chaining cannot escape — the programme-house cold start was stalling at 18
fails after 2000 evals vs urb-evolve's 6.
Auto-detection via seed_root.divided preserves the existing single-seed
path for warm starts from existing designs; all previous tests pass unchanged.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
driver.py (homemaker-py-b39): tournament selection, operators.mutate (storey
ops down-weighted) + area-matched crossover, every child's geometry
delegated to the warm-started inner loop (Lamarckian write-back; children
use a single local CMA phase - the exploratory ladder phase exists for cold
projections children never face). Budget stated and accounted in oracle
evaluations; near-duplicate fitness guard against population collapse
(neutral mutations are common, per 8cs).
free_with_keys/ratio_map/warm_x0 promoted from the 8cs experiment into
innerloop.py as the Lamarckian inheritance API; alignment with
solver.free_branches asserted across the corpus.
tests/test_driver.py fakes the inner loop: budget accounting, monotone
improvement history, warm-start + sigma plumbing, valid .dom output.
31 tests pass. experiments/run_search.py is the end-to-end acceptance run.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
