Commit graph

11 commits

Author SHA1 Message Date
51782047f3 coldstart harbor-house seed 2 @ 500000: 42 fails (12h/30s)
Cold-start re-baseline after the DESIGN.md 38.10/38.11 objective
change. Single worker (avoids homemaker-py-b8g), scored by the
shipped scorer from the programme directory.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
2026-09-02 17:53:41 +01:00
b885bd3dbd coldstart maple-court seed 1 @ 500000: 73 fails (17h/56s)
Cold-start re-baseline after the DESIGN.md 38.10/38.11 objective
change. Single worker (avoids homemaker-py-b8g), scored by the
shipped scorer from the programme directory.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
2026-09-02 10:52:55 +01:00
d7ac57d131 coldstart health-centre seed 1 @ 500000: 9 fails (5h/4s)
Cold-start re-baseline after the DESIGN.md 38.10/38.11 objective
change. Single worker (avoids homemaker-py-b8g), scored by the
shipped scorer from the programme directory.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
2026-09-01 20:28:37 +01:00
b39b467662 coldstart maple-court seed 0 @ 500000: 54 fails (18h/36s)
Cold-start re-baseline after the DESIGN.md 38.10/38.11 objective
change. Single worker (avoids homemaker-py-b8g), scored by the
shipped scorer from the programme directory.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
2026-09-01 03:21:09 +01:00
9d99cf34f7 coldstart programme-house seed 1 @ 500000: 1 fails (1h/0s)
Cold-start re-baseline after the DESIGN.md 38.10/38.11 objective
change. Single worker (avoids homemaker-py-b8g), scored by the
shipped scorer from the programme directory.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
2026-08-31 21:50:30 +01:00
2415ad6504 coldstart harbor-house seed 1 @ 500000: 43 fails (7h/36s)
Cold-start re-baseline after the DESIGN.md 38.10/38.11 objective
change. Single worker (avoids homemaker-py-b8g), scored by the
shipped scorer from the programme directory.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
2026-08-31 16:50:54 +01:00
d5d99dd4f6 coldstart harbor-house seed 0 @ 500000: 33 fails (8h/25s)
Cold-start re-baseline after the DESIGN.md 38.10/38.11 objective
change. Single worker (avoids homemaker-py-b8g), scored by the
shipped scorer from the programme directory.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
2026-08-31 09:46:46 +01:00
6d1eb26088 coldstart health-centre seed 0 @ 500000: 4 fails (2h/2s)
Cold-start re-baseline after the DESIGN.md 38.10/38.11 objective
change. Single worker (avoids homemaker-py-b8g), scored by the
shipped scorer from the programme directory.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
2026-08-31 01:20:51 +01:00
ac5913151a coldstart programme-house seed 0 @ 500000: 1 fails (0h/1s)
Cold-start re-baseline after the DESIGN.md 38.10/38.11 objective
change. Single worker (avoids homemaker-py-b8g), scored by the
shipped scorer from the programme directory.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
2026-08-29 18:27:40 +01:00
Claude
e5eb397b52
Checkpoint long searches; the cold-start runs were lost to a reclaimed box
All four 500k runs died about 10 minutes in when the container was
reclaimed. No SIGTERM fired, so no .dom was written and 0 of 12 runs
completed. My plan committed results per finished run, which protected
nothing because no run reached its commit point. The bad assumption was
reading "reclaimed after inactivity" as CPU inactivity; it is conversation
inactivity, and background compute does not hold the box open.

Progress reached before the loss (from the tracked logs): harbor 24,960
evals / 40 fails, maple 14,880 / 79, health-centre 25,920 / 33,
programme-house 138,800 / 2.

The underlying gap is not environmental: a search's only output lands at
the very end or on SIGTERM, so ANY abrupt loss -- reclaimed container, OOM,
power cut -- takes the whole run with it. On a 3M-eval search that is 2.4
days of compute with no recoverable artefact.

  - driver.search gains checkpoint=/checkpoint_every=: the current best is
    handed to a callback at most every N evals. Rate-limited by evals, not
    improvements, which come in bursts early. A failing checkpoint is logged
    and swallowed -- losing a checkpoint is bad, losing the search because a
    checkpoint failed is worse.
  - homemaker-evolve --checkpoint-every N writes <out>.dom.checkpoint via
    mkstemp + os.replace, so a crash can never catch it half-written. It is
    deliberately NOT the output path: a checkpoint is a leaf-sharing run's
    internal best, dishonest under the canonical scorer until the finish
    stage unfolds it (homemaker-py-3l6), and must not be mistaken for the
    finished article.
  - Verified the written checkpoint re-loads as a valid .dom.

Default off, so behaviour is unchanged without the flag.

Lint at parity (46); tests 372 passed (3 new), same 2 pre-existing failures.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
2026-08-29 05:43:58 +00:00
Claude
e2d5838433
coldstart programme-house seed 0 @ 1000: 7 fails (3h/4s)
Cold-start re-baseline after the DESIGN.md 38.10/38.11 objective
change. Single worker (avoids homemaker-py-b8g), scored by the
shipped scorer from the programme directory.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
2026-08-28 23:29:01 +00:00