Coldstart runner: time runs with a clock that stops when the machine does

The runner measured elapsed_s with time.time(), i.e. CLOCK_REALTIME, which
keeps advancing while a machine is suspended. The box running bk9 was suspended
for three days, so any run spanning that would have reported elapsed_s inflated
by ~72 h -- silently, since the fail count and score are unaffected and nothing
else would look wrong.

time.monotonic() is CLOCK_MONOTONIC on Linux, which stops during suspend
(CLOCK_BOOTTIME is the variant that does not). That measures the time a run
actually had a CPU, which is what the column is for.

Nothing is corrupted yet -- no run has completed since the suspend, so the
timing column is still only the 39.12 baseline, measured on a box that stayed
awake. Noted in the module docstring so that caveat travels with the number.

Refs homemaker-py-bk9.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
This commit is contained in:
Claude 2026-09-09 07:00:30 +00:00
parent 3ccbba7284
commit d04e585d38
No known key found for this signature in database

View file

@ -14,6 +14,11 @@ Design notes, all of which matter for the result being trustworthy:
* **Seed-major order.** The queue runs seed 0 of every programme, then seed 1, * **Seed-major order.** The queue runs seed 0 of every programme, then seed 1,
then seed 2 -- so if the box is lost half way we have all four programmes at then seed 2 -- so if the box is lost half way we have all four programmes at
fewer seeds, rather than one programme at three seeds and nothing else. fewer seeds, rather than one programme at three seeds and nothing else.
* **Timings exclude suspend.** ``elapsed_s`` is measured with
``time.monotonic()``, so a run that spans a suspended machine reports the
time it actually had a CPU rather than wall time. The 39.12 baseline's
~430 h total was measured with ``time.time()`` and is only trustworthy
because that box stayed awake.
* **Commit and push after every finished run.** This is an ephemeral container; * **Commit and push after every finished run.** This is an ephemeral container;
it is reclaimed on inactivity or session end. Anything not pushed is gone. Git it is reclaimed on inactivity or session end. Anything not pushed is gone. Git
calls are serialised under a lock file so the runner cannot race a human (or calls are serialised under a lock file so the runner cannot race a human (or
@ -159,7 +164,12 @@ def main() -> None:
"--seed", str(seed), "--workers", "1", "--output", str(out), "--seed", str(seed), "--workers", "1", "--output", str(out),
"--checkpoint-every", str(checkpoint_every)], "--checkpoint-every", str(checkpoint_every)],
cwd=d, stdout=subprocess.DEVNULL, stderr=fh) cwd=d, stdout=subprocess.DEVNULL, stderr=fh)
running[proc.pid] = (proc, prog, seed, out, fh, time.time()) # monotonic, NOT time.time(): CLOCK_REALTIME advances while the
# machine is suspended, so a run spanning an overnight suspend
# would report elapsed_s inflated by the suspend. CLOCK_MONOTONIC
# stops (that is what CLOCK_BOOTTIME is for), so this measures the
# time the run actually had a CPU.
running[proc.pid] = (proc, prog, seed, out, fh, time.monotonic())
print(f" start {prog} seed {seed} -> {out.name}", flush=True) print(f" start {prog} seed {seed} -> {out.name}", flush=True)
time.sleep(10) time.sleep(10)
@ -168,7 +178,7 @@ def main() -> None:
continue continue
fh.close() fh.close()
del running[pid] del running[pid]
elapsed = round(time.time() - t0, 1) elapsed = round(time.monotonic() - t0, 1)
if not out.exists(): if not out.exists():
print(f" FAILED {prog} seed {seed} (rc={proc.returncode}, " print(f" FAILED {prog} seed {seed} (rc={proc.returncode}, "
f"{elapsed}s) -- see the .log", flush=True) f"{elapsed}s) -- see the .log", flush=True)