bruno/homemaker-layout
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Coldstart runner: time runs with a clock that stops when the machine does

Browse source 
The runner measured elapsed_s with time.time(), i.e. CLOCK_REALTIME, which
keeps advancing while a machine is suspended. The box running bk9 was suspended
for three days, so any run spanning that would have reported elapsed_s inflated
by ~72 h -- silently, since the fail count and score are unaffected and nothing
else would look wrong.

time.monotonic() is CLOCK_MONOTONIC on Linux, which stops during suspend
(CLOCK_BOOTTIME is the variant that does not). That measures the time a run
actually had a CPU, which is what the column is for.

Nothing is corrupted yet -- no run has completed since the suspend, so the
timing column is still only the 39.12 baseline, measured on a box that stayed
awake. Noted in the module docstring so that caveat travels with the number.

Refs homemaker-py-bk9.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
This commit is contained in:
Claude 2026-09-09 07:00:30 +00:00
parent 3ccbba7284
commit d04e585d38
No known key found for this signature in database
1 changed files with 12 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

14
experiments/run_coldstart_baseline.py
View file

@ -14,6 +14,11 @@ Design notes, all of which matter for the result being trustworthy:
* **Seed-major order.** The queue runs seed 0 of every programme, then seed 1,
then seed 2 -- so if the box is lost half way we have all four programmes at
fewer seeds, rather than one programme at three seeds and nothing else.
* **Timings exclude suspend.** ``elapsed_s`` is measured with
``time.monotonic()``, so a run that spans a suspended machine reports the
time it actually had a CPU rather than wall time. The 39.12 baseline's
~430 h total was measured with ``time.time()`` and is only trustworthy
because that box stayed awake.
* **Commit and push after every finished run.** This is an ephemeral container;
it is reclaimed on inactivity or session end. Anything not pushed is gone. Git
calls are serialised under a lock file so the runner cannot race a human (or
@ -159,7 +164,12 @@ def main() -> None:
"--seed", str(seed), "--workers", "1", "--output", str(out),
"--checkpoint-every", str(checkpoint_every)],
cwd=d, stdout=subprocess.DEVNULL, stderr=fh)
running[proc.pid] = (proc, prog, seed, out, fh, time.time())
# monotonic, NOT time.time(): CLOCK_REALTIME advances while the
# machine is suspended, so a run spanning an overnight suspend
# would report elapsed_s inflated by the suspend. CLOCK_MONOTONIC
# stops (that is what CLOCK_BOOTTIME is for), so this measures the
# time the run actually had a CPU.
running[proc.pid] = (proc, prog, seed, out, fh, time.monotonic())
print(f" start {prog} seed {seed} -> {out.name}", flush=True)
time.sleep(10)
@ -168,7 +178,7 @@ def main() -> None:
continue
fh.close()
del running[pid]
elapsed = round(time.time() - t0, 1)
elapsed = round(time.monotonic() - t0, 1)
if not out.exists():
print(f" FAILED {prog} seed {seed} (rc={proc.returncode}, "
f"{elapsed}s) -- see the .log", flush=True)