e5eb397b52
All four 500k runs died about 10 minutes in when the container was
reclaimed. No SIGTERM fired, so no .dom was written and 0 of 12 runs
completed. My plan committed results per finished run, which protected
nothing because no run reached its commit point. The bad assumption was
reading "reclaimed after inactivity" as CPU inactivity; it is conversation
inactivity, and background compute does not hold the box open.
Progress reached before the loss (from the tracked logs): harbor 24,960
evals / 40 fails, maple 14,880 / 79, health-centre 25,920 / 33,
programme-house 138,800 / 2.
The underlying gap is not environmental: a search's only output lands at
the very end or on SIGTERM, so ANY abrupt loss -- reclaimed container, OOM,
power cut -- takes the whole run with it. On a 3M-eval search that is 2.4
days of compute with no recoverable artefact.
- driver.search gains checkpoint=/checkpoint_every=: the current best is
handed to a callback at most every N evals. Rate-limited by evals, not
improvements, which come in bursts early. A failing checkpoint is logged
and swallowed -- losing a checkpoint is bad, losing the search because a
checkpoint failed is worse.
- homemaker-evolve --checkpoint-every N writes <out>.dom.checkpoint via
mkstemp + os.replace, so a crash can never catch it half-written. It is
deliberately NOT the output path: a checkpoint is a leaf-sharing run's
internal best, dishonest under the canonical scorer until the finish
stage unfolds it (homemaker-py-3l6), and must not be mistaken for the
finished article.
- Verified the written checkpoint re-loads as a valid .dom.
Default off, so behaviour is unchanged without the flag.
Lint at parity (46); tests 372 passed (3 new), same 2 pre-existing failures.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
58 lines
3.4 KiB
Text
58 lines
3.4 KiB
Text
WARNING: programme demands 240 m2 per storey on a 183 m2 plot (131%). Every room will be squeezed below its target however long the search runs. (DESIGN.md §39.11)
|
|
WARNING: 240 m2 per storey needs ~49 m of daylit wall; the plot's non-private perimeter gives 41 m. Roughly 12 m2 of courtyard closes the gap, but only -57 m2 of plot is spare. (DESIGN.md §38.3)
|
|
seed : /home/user/homemaker-layout/examples/health-centre/init.dom
|
|
programme : health-centre
|
|
budget : 500000
|
|
pop : 16
|
|
child_budget : 80
|
|
workers : 1
|
|
rng seed : 0
|
|
leaf sharing : True (factor=3)
|
|
superpose : False
|
|
multi_use : False
|
|
conn grade : False
|
|
use tiers : False
|
|
bridge circulation : False
|
|
ruin recreate : False
|
|
collapse in-search : True
|
|
shapecurve warmstart : False
|
|
shapecurve prune : False
|
|
output : /home/user/homemaker-layout/examples/health-centre/coldstart-500000-s0.dom
|
|
[ 80 evals] best 7.05564e-20 (fails 53) via construct/0
|
|
[ 400 evals] best 1.03209e-17 (fails 47) via construct/4
|
|
[ 2160 evals] best 8.5085e-17 (fails 44) via level_compound_fix noop
|
|
[ 2800 evals] best 8.71761e-17 (fails 44) via level_retype noop
|
|
[ 3280 evals] best 1.01747e-16 (fails 44) via place_missing t9 -> llrrr
|
|
[ 3760 evals] best 1.07254e-16 (fails 44) via core_divide noop
|
|
[ 3840 evals] best 1.94754e-16 (fails 43) via retype 0/llrrl->ph1
|
|
[ 4800 evals] best 2.09073e-16 (fails 42) via core_divide noop
|
|
[ 4960 evals] best 2.709e-16 (fails 42) via crossover lrlll<->rrrrr
|
|
[ 5840 evals] best 2.7669e-16 (fails 42) via crossover lllll<->rllll
|
|
[ 5920 evals] best 9.22724e-16 (fails 41) via place_missing t9 -> rrrlr
|
|
[ 6480 evals] best 9.45485e-16 (fails 41) via place_missing noop
|
|
[ 6960 evals] best 3.7158e-15 (fails 39) via place_missing t9 -> rrrlr
|
|
[ 8320 evals] best 4.98363e-15 (fails 39) via level_compound_fix noop
|
|
[ 9680 evals] best 5.20512e-15 (fails 39) via crossover lrlll<->rlrrl
|
|
[ 9840 evals] best 5.48608e-15 (fails 39) via place_missing noop
|
|
[ 10080 evals] best 5.67651e-15 (fails 39) via crossover rrrrr<->rllrl
|
|
[ 10640 evals] best 6.58735e-15 (fails 38) via retype 0/rrllr->O
|
|
[ 12320 evals] best 7.02146e-15 (fails 38) via level_fix noop
|
|
[ 12880 evals] best 7.53327e-15 (fails 38) via level_compound_fix noop
|
|
[ 14240 evals] best 8.06389e-15 (fails 38) via level_compound_fix noop
|
|
[ 14480 evals] best 1.56759e-14 (fails 37) via swap 0/llrl
|
|
[ 14560 evals] best 1.59668e-14 (fails 37) via crossover rrlrl<->llrlr
|
|
[ 14880 evals] best 1.62722e-14 (fails 37) via core_undivide noop
|
|
[ 15360 evals] best 3.83618e-14 (fails 36) via level_retype noop
|
|
[ 16320 evals] best 3.97387e-14 (fails 36) via level_fix noop
|
|
[ 16560 evals] best 4.10139e-14 (fails 36) via level_fix noop
|
|
[ 17040 evals] best 4.70405e-14 (fails 35) via swap 0/llrl
|
|
[ 17600 evals] best 5.08607e-14 (fails 35) via level_compound_fix noop
|
|
[ 20320 evals] best 9.45373e-14 (fails 35) via crossover rllr<->rlrll
|
|
[ 21040 evals] best 9.66604e-14 (fails 35) via level_compound_fix noop
|
|
[ 21600 evals] best 9.92079e-14 (fails 35) via level_fix noop
|
|
[ 21840 evals] best 1.24031e-13 (fails 35) via level_retype noop
|
|
[ 22000 evals] best 1.45873e-13 (fails 35) via crossover lrrl<->lrlrl
|
|
[ 23440 evals] best 1.48115e-13 (fails 35) via retype 0/rrlrr->pt1
|
|
[ 23840 evals] best 1.51872e-13 (fails 35) via level_compound_fix noop
|
|
[ 25200 evals] best 5.3306e-13 (fails 33) via undivide 0/llll
|
|
[ 25920 evals] best 5.46749e-13 (fails 33) via level_fix noop
|