From d04e585d38c37724a2813c2d171fe25051f84b0c Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 9 Sep 2026 07:00:30 +0000 Subject: [PATCH] Coldstart runner: time runs with a clock that stops when the machine does The runner measured elapsed_s with time.time(), i.e. CLOCK_REALTIME, which keeps advancing while a machine is suspended. The box running bk9 was suspended for three days, so any run spanning that would have reported elapsed_s inflated by ~72 h -- silently, since the fail count and score are unaffected and nothing else would look wrong. time.monotonic() is CLOCK_MONOTONIC on Linux, which stops during suspend (CLOCK_BOOTTIME is the variant that does not). That measures the time a run actually had a CPU, which is what the column is for. Nothing is corrupted yet -- no run has completed since the suspend, so the timing column is still only the 39.12 baseline, measured on a box that stayed awake. Noted in the module docstring so that caveat travels with the number. Refs homemaker-py-bk9. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB --- experiments/run_coldstart_baseline.py | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/experiments/run_coldstart_baseline.py b/experiments/run_coldstart_baseline.py index 7130761..a9fa39f 100644 --- a/experiments/run_coldstart_baseline.py +++ b/experiments/run_coldstart_baseline.py @@ -14,6 +14,11 @@ Design notes, all of which matter for the result being trustworthy: * **Seed-major order.** The queue runs seed 0 of every programme, then seed 1, then seed 2 -- so if the box is lost half way we have all four programmes at fewer seeds, rather than one programme at three seeds and nothing else. +* **Timings exclude suspend.** ``elapsed_s`` is measured with + ``time.monotonic()``, so a run that spans a suspended machine reports the + time it actually had a CPU rather than wall time. The 39.12 baseline's + ~430 h total was measured with ``time.time()`` and is only trustworthy + because that box stayed awake. * **Commit and push after every finished run.** This is an ephemeral container; it is reclaimed on inactivity or session end. Anything not pushed is gone. Git calls are serialised under a lock file so the runner cannot race a human (or @@ -159,7 +164,12 @@ def main() -> None: "--seed", str(seed), "--workers", "1", "--output", str(out), "--checkpoint-every", str(checkpoint_every)], cwd=d, stdout=subprocess.DEVNULL, stderr=fh) - running[proc.pid] = (proc, prog, seed, out, fh, time.time()) + # monotonic, NOT time.time(): CLOCK_REALTIME advances while the + # machine is suspended, so a run spanning an overnight suspend + # would report elapsed_s inflated by the suspend. CLOCK_MONOTONIC + # stops (that is what CLOCK_BOOTTIME is for), so this measures the + # time the run actually had a CPU. + running[proc.pid] = (proc, prog, seed, out, fh, time.monotonic()) print(f" start {prog} seed {seed} -> {out.name}", flush=True) time.sleep(10) @@ -168,7 +178,7 @@ def main() -> None: continue fh.close() del running[pid] - elapsed = round(time.time() - t0, 1) + elapsed = round(time.monotonic() - t0, 1) if not out.exists(): print(f" FAILED {prog} seed {seed} (rc={proc.returncode}, " f"{elapsed}s) -- see the .log", flush=True)