bruno/homemaker-layout
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

coldstart runner: commit only its own artefacts, not the whole tree

Browse source 
record_and_push used `git add -A`, so each completion committed the entire
working tree. On the live local run that meant one commit carrying 49 files
and ~1.97M insertions -- three still-running programmes' partial
coldstart-*.dom, plus unrelated evolved-*.dom -- under a message naming
only programme-house seed 0. In-flight artefacts were being recorded as if
they were results, attributed to the wrong run.

Now stages and commits exactly this run's .dom/.log/.score/.fails plus the
results TSV, via `git commit --only <paths>` so it holds regardless of what
else is staged and a concurrent edit elsewhere cannot ride along.

Note for the run currently in progress: it has the old code loaded, so it
will keep sweeping until restarted. The committed .dom files for
harbor-house, maple-court and health-centre are mid-run snapshots, not
results -- their logs show 75k/87k of 500k evals -- and should be
disregarded until those runs report their own rows.

First real result is in: programme-house seed 0 @ 500k, finish collapse
28 -> 1 fails, final 1 fail (0 hard / 1 soft), 19400s.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
This commit is contained in:
Claude 2026-08-29 19:40:00 +00:00
parent f973d5c020
commit c747e55abe
No known key found for this signature in database
1 changed files with 17 additions and 6 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

23
experiments/run_coldstart_baseline.py
View file

@ -72,7 +72,7 @@ def score(dom_path: Path) -> tuple[int, int, int, float]:
return len(lines), hard, len(lines) - hard, val
def record_and_push(row: dict) -> None:
def record_and_push(row: dict, artefacts: "list[Path]") -> None:
rows = []
if RESULTS.exists():
rows = list(csv.DictReader(RESULTS.open(), delimiter="\t"))
@ -85,10 +85,19 @@ def record_and_push(row: dict) -> None:
msg = (f"coldstart {row['programme']} seed {row['seed']} @ {row['budget']}: "
f"{row['fails']} fails ({row['hard']}h/{row['soft']}s)")
# Commit ONLY this run's artefacts plus the results table. `git add -A` here
# swept the whole working tree: one completion committed 49 files and ~2M
# insertions, including three still-running programmes' partial .dom files
# and unrelated evolved-*.dom, all under a message naming a different
# programme. `--only` commits exactly the listed paths regardless of what
# else is staged, so a concurrent edit elsewhere in the tree cannot ride
# along and in-flight runs are never committed as if they were results.
paths = [str(a.relative_to(REPO)) for a in artefacts if a.exists()]
paths.append(str(RESULTS.relative_to(REPO)))
with git_lock():
subprocess.run(["git", "add", "-A"], cwd=REPO, capture_output=True)
subprocess.run(["git", "add", "--", *paths], cwd=REPO, capture_output=True)
subprocess.run(
["git", "commit", "-q", "-m", msg + "\n\n"
["git", "commit", "-q", "--only", *paths, "-m", msg + "\n\n"
"Cold-start re-baseline after the DESIGN.md 38.10/38.11 objective\n"
"change. Single worker (avoids homemaker-py-b8g), scored by the\n"
"shipped scorer from the programme directory.\n\n"
@ -156,9 +165,11 @@ def main() -> None:
n, hard, soft, val = score(out)
print(f" done {prog} seed {seed}: {n} fails "
f"({hard}h/{soft}s) score {val:.4g} in {elapsed}s", flush=True)
record_and_push(dict(programme=prog, seed=seed, budget=args.budget,
fails=n, hard=hard, soft=soft, score=f"{val:.6g}",
elapsed_s=elapsed, dom=out.name))
record_and_push(
dict(programme=prog, seed=seed, budget=args.budget,
fails=n, hard=hard, soft=soft, score=f"{val:.6g}",
elapsed_s=elapsed, dom=out.name),
[out, log, out.with_suffix(".dom.score"), out.with_suffix(".dom.fails")])
print("\n=== all runs complete ===", flush=True)