diff --git a/.beads/issues.jsonl b/.beads/issues.jsonl index e4a01db..3d3b932 100644 --- a/.beads/issues.jsonl +++ b/.beads/issues.jsonl @@ -36,7 +36,7 @@ {"_type":"issue","id":"homemaker-py-1p0","title":"Geometry inner loop: full-objective equal-offset ratio optimiser","description":"DESIGN.md §5.1, §7 Phase 1. Productionise experiments/optimize_fullfitness.py into homemaker: optimise(topology, x0=None) -\u003e (geometry, fitness). DOF = equal-offset division ratios of free branches (solver.free_branches, lowest-storey cut ownership), clipped to [eps, 1-eps]. Objective = full oracle fitness (never a proxy — §4.2 falsified). Must support warm-start x0 (§5.6) and a population/batch evaluation mode so each iteration scores via one batched oracle call (§4.6).","acceptance_criteria":"Reproduces or exceeds §4.5 gains (x1.24–x1.67, no new failures) on 2f45907, candidate-002, c964435; works as a library call on any corpus .dom","status":"closed","priority":1,"issue_type":"feature","assignee":"Bruno Postle","owner":"bruno@postle.net","created_at":"2026-06-11T23:36:58Z","created_by":"Bruno Postle","updated_at":"2026-06-12T08:46:31Z","started_at":"2026-06-12T00:14:19Z","closed_at":"2026-06-12T08:46:31Z","close_reason":"innerloop.optimise() lands: batched CMA-ES sigma ladder (0.05/0.15, IPOP popsize doubling, deterministic seeding) over equal-offset free-branch ratios vs full oracle fitness; warm-start x0 supported. Acceptance vs unprojected originals: x1.65/x1.66/x1.58 against bars x1.24/x1.67/x1.59, no new failures, 46 oracle calls vs NM's 200. Two near-bar results accepted as reproduced-within-noise (1% tol) — draw spread brackets the single-NM-draw bars; approved by Bruno 2026-06-12. Gotchas: equal-offset projection of legacy unequal cuts loses fitness/adds failures (midpoint projection used); pycma seed=0 means clock-seeded.","dependencies":[{"issue_id":"homemaker-py-1p0","depends_on_id":"homemaker-py-av5","type":"blocks","created_at":"2026-06-12T00:39:33Z","created_by":"Bruno Postle","metadata":"{}"}],"dependency_count":1,"dependent_count":3,"comment_count":0} {"_type":"issue","id":"homemaker-py-8cs","title":"Experiment: warm-vs-cold start of inner loop (Lamarckian inheritance)","description":"DESIGN.md §5.6, §4.6. Warm-starting a child topology's inner loop from the parent's optimised ratios is the main lever for cutting per-topology cost (~3 min/topology cold). Apply single topology mutations to optimised corpus designs, re-optimise warm (surviving cuts keep values, new cuts get heuristic defaults) vs cold, compare oracle-call counts to convergence at equal final fitness.","acceptance_criteria":"Speedup factor measured across \u003e=10 mutated topologies; decision recorded (expect order-of-magnitude; if \u003c2x, revisit §4.6 Phase-2 scoping)","notes":"Experiment script committed (experiments/warm_vs_cold.py, 1cc86c8) and machinery validated oracle-free; one mutated child scored through the oracle OK. Waiting on homemaker-py-gp2 reference run to finish, then execute under URB_NO_OCCLUSION=1 (3 parents x 400 evals + 12 children x 2 x 200 evals, ~1.5-2 h oracle time). Default budgets: parent 400, child 200; target = evals to 95% of best final.","status":"closed","priority":1,"issue_type":"task","owner":"bruno@postle.net","created_at":"2026-06-11T23:36:58Z","created_by":"Bruno Postle","updated_at":"2026-06-12T11:44:45Z","closed_at":"2026-06-12T11:44:45Z","close_reason":"Measured (URB_NO_OCCLUSION=1, parent budget 400, child 200, 12 single mutations across 3 designs): cold start reached 95% of warm final in 0/12 cases within budget — speedup unbounded at practical budgets; warm finals beat cold finals x1.2-x4 in 12/12; 6/12 warm starts were within 95% at 1 eval (near-neutral mutations). Decision: Lamarckian warm-starting is MANDATORY in the memetic driver (homemaker-py-b39), not an optimisation; cold starts produce strictly worse geometry at equal budget. Note: 2 undivides were exactly fitness-neutral (same-type merge == Merge_Divided equivalence) — locality datum for homemaker-py-nyb.","dependencies":[{"issue_id":"homemaker-py-8cs","depends_on_id":"homemaker-py-1p0","type":"blocks","created_at":"2026-06-12T00:39:34Z","created_by":"Bruno Postle","metadata":"{}"}],"dependency_count":1,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"homemaker-py-av5","title":"Batched oracle: score many .dom files per invocation","description":"oracle.py currently scores one .dom per urb-fitness.pl call (~1.65 s/dom). DESIGN.md §4.6: batching amortises Perl startup to ~0.99 s/dom and is required so population/batch optimisers can score a whole generation in one oracle call. Extend oracle.py with a batch API: write N .dom files, one perl invocation, parse N .score/.fails pairs. Keep the single-file path for compatibility.","acceptance_criteria":"Batch of 35 corpus files scores in one perl invocation; per-file results identical to single-file calls; measured s/dom reported","status":"closed","priority":1,"issue_type":"task","assignee":"Bruno Postle","owner":"bruno@postle.net","created_at":"2026-06-11T23:36:56Z","created_by":"Bruno Postle","updated_at":"2026-06-12T00:14:06Z","started_at":"2026-06-11T23:50:40Z","closed_at":"2026-06-12T00:14:06Z","close_reason":"score_batch() lands in oracle.py; 35-file corpus parity verified single-vs-batch (1e-12 rel fitness, exact fail sets); 0.98 s/dom batched vs 1.27 single, x1.30","dependency_count":0,"dependent_count":1,"comment_count":0} -{"_type":"issue","id":"homemaker-py-tco","title":"A/B harnesses should report the minimum detectable difference for the N they run","description":"Two independent findings this session say the project's A/B protocols are routinely underpowered for the margins they report:\\n\\n §38.19 programme-house needed N=60 to resolve an effect its own protocol claimed at N=20 (p ~= 0.069 at N=20, 0.017 at N=60)\\n §38.21 harbor's paired sd is 6.19 fails, so n=3 resolves nothing finer than ~15.4 fails -- yet every recorded harbor margin is below that, and 25% of 3-seed subsets show a clean 3/3 sweep by chance\\n\\nThe fix is procedural and cheap. The A/B harnesses (run_*_ab.sh, ab_*.py, rerun_1ph_protocol.sh) should print, alongside the result, the minimum difference their N and observed sd could have detected -- so an underpowered verdict is visible AT THE POINT IT IS MADE rather than years later.\\n\\nA one-line addition to each harness's summary: given the paired diffs it already computes, report and flag when the observed margin is below it. Optionally refuse to declare a winner in that case.\\n\\nThis is not about re-running old A/Bs (§38.21 covers harbor, §38.19 programme-house); it is about not generating more of them.","acceptance_criteria":"The shared A/B summary path reports the minimum detectable difference for the N actually run, and flags a verdict whose margin falls below it; applied to at least rerun_1ph_protocol.sh and ab_ssz_search.py.","status":"open","priority":2,"issue_type":"task","owner":"noreply@anthropic.com","created_at":"2026-08-29T19:34:30Z","created_by":"Claude","updated_at":"2026-08-29T19:34:30Z","dependency_count":0,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"homemaker-py-tco","title":"A/B harnesses should report the minimum detectable difference for the N they run","description":"Two independent findings this session say the project's A/B protocols are\nroutinely underpowered for the margins they report:\n\n §38.19 programme-house needed N=60 to resolve an effect its own protocol\n claimed at N=20 (p ~= 0.069 at N=20, 0.017 at N=60)\n §38.21 harbor's paired sd is 6.19 fails, so n=3 resolves nothing finer than\n ~15.4 fails -- yet every recorded harbor margin is below that, and 25%\n of 3-seed subsets show a clean 3/3 sweep by chance\n\nThe fix is procedural and cheap. The A/B harnesses (run_*_ab.sh, ab_*.py,\nrerun_1ph_protocol.sh) should print, alongside the result, the minimum\ndifference their N and observed sd could have detected -- so an underpowered\nverdict is visible AT THE POINT IT IS MADE rather than years later.\n\nA one-line addition to each harness's summary: from the paired diffs it already\ncomputes, report\n\n minimum detectable difference = t_crit(0.975, N-1) * sd / sqrt(N)\n\nand flag when the observed margin falls below it. Optionally refuse to declare a\nwinner in that case.\n\nThis is not about re-running old A/Bs (§38.21 covers harbor, §38.19\nprogramme-house); it is about not generating more of them.\n","acceptance_criteria":"The shared A/B summary path reports the minimum detectable difference for the N actually run, and flags a verdict whose margin falls below it; applied to at least rerun_1ph_protocol.sh and ab_ssz_search.py.","status":"open","priority":2,"issue_type":"task","owner":"noreply@anthropic.com","created_at":"2026-08-29T19:34:30Z","created_by":"Claude","updated_at":"2026-08-29T19:35:08Z","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"homemaker-py-ioe","title":"Is collapse_insearch=True still the right default under the current objective?","description":"The default was flipped OFF -\u003e ON by homemaker-py-1ph (DESIGN.md §20, 2026-07-24) on the strength of a programme-house N=20 sweep: mean 7.95 -\u003e 7.10, 11W/6L/3T, paired t p ~= 0.028. homemaker-py-d86 (§38.18) has now confirmed that verdict was sound FOR ITS OWN ERA -- it reproduces on a pre-iio commit, and the iio stale-share bug is structurally unreachable on that protocol because programme-house declares count: 1 for every code, so no leaf ever carries a share.\n\nBut the objective has changed substantially since, three times over, and all of it after 1ph:\n\n §39.4 the generic-namespace fix -- codes like cr1 were being read as generic\n circulation, so 14% of harbor's programme was silently optional\n §38.10 / §38.11 crinkliness declared per space; 14 corpus spaces now declare\n crinkliness: none\n §38.12 the missing-space cascade no longer weighted by YAML verbosity, a fixed\n 5 fails per missing instance instead of 3-5\n\ncollapse_insearch runs collapse_global inside every fitness eval, and collapse_global's assignment is valued against exactly the quality factors those changes touched. So the ON-beats-OFF margin was measured against an objective that no longer exists. The direction is plausibly unchanged -- but it is currently an assumption carried on a superseded measurement, and it is a DEFAULT, so every run inherits it.\n\nThe protocol and harness already exist: experiments/rerun_1ph_protocol.sh runs programme-house, budget 3000, 4 workers, seeds 1-20, ON vs OFF, both arms finished with --collapse, and takes about 6 minutes.\n\nNote when re-running: n_workers is an algorithm parameter (§38.17), so keep 4 workers to stay comparable with the published protocol, and record it with the result.","acceptance_criteria":"The 1ph protocol re-run at N=20 on the current codebase and objective, with the ON-vs-OFF verdict either reconfirmed or restated; if the margin has moved materially, DESIGN.md §20's default-flip rationale is updated to say so and the default is reconsidered on the new numbers.","status":"closed","priority":2,"issue_type":"task","assignee":"Claude","owner":"noreply@anthropic.com","created_at":"2026-08-29T13:36:18Z","created_by":"Claude","updated_at":"2026-08-29T13:57:45Z","started_at":"2026-08-29T13:36:32Z","closed_at":"2026-08-29T13:57:45Z","close_reason":"Re-validated: the default STANDS, but with two caveats worth carrying\n(DESIGN.md §38.19).\n\nRe-ran the 1ph protocol as published on the current codebase and objective --\nprogramme-house, budget 3000, 4 workers, ON vs OFF, both arms finished with\n--collapse.\n\n N OFF ON W/L/T diff t p\n published 1ph (07-24) 20 7.95 7.10 11/6/3 +0.85 2.38 0.028\n historical re-run (§38.18) 20 8.05 7.10 11/6/3 +0.95 2.59 --\n current objective 20 7.85 7.15 10/7/3 +0.70 1.82 0.069\n current objective 40 7.60 7.03 21/14/5 +0.57 2.01 0.045\n current objective 60 7.58 7.02 29/19/12 +0.57 2.45 0.017\n\nAt N=60: mean diff +0.567 fails/seed, paired t=2.454 (df=59), p=0.0171 exact,\n95% CI [+0.105, +1.029] excluding zero. Wilcoxon signed-rank cross-check agrees\n(p=0.0138), which matters because fail counts are small integers and normality\nis not obvious.\n\nCaveat 1: the effect is about a third smaller than published (+0.57 vs +0.85).\nPartly regression from a slightly lucky N=20 draw, partly plausible real erosion\n-- several fails collapse_global used to clear have been redefined out of\nexistence or made harder by §39.4 / §38.10-12.\n\nCaveat 2, the more useful one: THE PUBLISHED N=20 CAN NO LONGER DETECT ITS OWN\nEFFECT. At exactly the published sample size the current answer is p ~= 0.069, a\nnull by the conventional threshold. Had I run N=20 and stopped, the honest report\nwould have been \"the 1ph verdict no longer reproduces\" and the default would have\nlooked unjustified. It took N=60 to resolve. That is the \"8sh/1ph/qi6/lj3\npattern\" this log already warns about, now biting the flagship result itself.\nAny future re-validation of this default needs N \u003e= 40; N=20 should not be\ntrusted to settle it either way.\n\n§20 annotated in place so a reader of the original claim sees the current figure.\nHarness now takes a seed range (APPEND=1 to extend a sweep); results in\nexperiments/results/ioe_1ph_current_objective.tsv.\n","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"homemaker-py-vjd","title":"cpsat assignment ~7.5x slower after the 3qj adjacency: re-check §39.5's cpsat-vs-greedy verdict","description":"Declaring harbor's t -\u003e n adjacency (homemaker-py-3qj, DESIGN.md §38.14) made the CP-SAT room-labelling model markedly harder. constructive_topology seeding, 3-seed average:\\n\\n harbor greedy 0.06s -\u003e 0.06s (unchanged) cpsat 0.28s -\u003e 2.11s (7.5x)\\n maple greedy 0.03s -\u003e 0.03s (unchanged) cpsat 0.47s -\u003e 1.37s (2.9x)\\n\\nassign_solver defaults to greedy so ordinary runs pay nothing, and mutate_reassign/enable_reassign are opt-in too. But §39.5 concluded cpsat beats greedy on both programmes, and that was measured on a cheaper problem than the corpus now poses. The verdict needs re-checking on quality-per-second, not just quality.\\n\\nAlso worth checking whether cpsat.max_deterministic_time is now being hit, which would mean it is returning early rather than solving -- that would change the quality side of the comparison too, silently.","acceptance_criteria":"cpsat vs greedy re-measured on the current corpus for both solution quality AND wall time; §39.5's verdict either reconfirmed or restated; if max_deterministic_time is being hit, that is recorded and the limit reconsidered.","status":"closed","priority":2,"issue_type":"task","assignee":"Claude","owner":"noreply@anthropic.com","created_at":"2026-08-29T10:57:26Z","created_by":"Claude","updated_at":"2026-08-29T14:47:36Z","started_at":"2026-08-29T14:20:49Z","closed_at":"2026-08-29T14:47:36Z","close_reason":"Re-measured. Three findings (DESIGN.md §38.20).\n\n1. A LIVE BUG IN THE CAP, found on the way. solve_room_labels sets a\ndeterministic work-unit budget (max_deterministic_time=4.0) and a wall-clock\nbackstop, with the comment that the wall clock is \"a pathological-case backstop\nonly\". At its 2.0s value it had become THE BINDING CONSTRAINT: on harbor, 2 of\n24 solves returned FEASIBLE not OPTIMAL, wall time hit exactly 2010 ms, and the\ndeterministic budget was never reached (max 2.483 of 4.0). Those labellings were\nboth suboptimal AND load-dependent -- the wall clock is precisely the cap §39.5\nadded the deterministic one to escape. Cause: §38.14's `t -\u003e n` adjacency makes\nthe model much harder, and the 2s value dated from when solves took ~124 ms.\nRaised to 30s; now 24/24 harbor and 36/36 maple solves are OPTIMAL with the\ndeterministic budget still in headroom (max 3.569/4.0).\n\n2. THE VERDICT REVERSES. Re-measured deterministically (fdp's id()-ordering fix\nmeans the arms no longer differ by memory layout), 12 constructed seeds, scored\ncanonically:\n\n harbor greedy 722h/601s = 1323 0.079 s/seed\n harbor cpsat 908h/640s = 1548 1.623 s/seed\n maple greedy 777h/987s = 1764 0.063 s/seed\n maple cpsat 1213h/1043s= 2256 1.327 s/seed\n\ncpsat LOSES on both, +225 and +492 fails, at ~21x the seeding time, concentrated\nin hard fails.\n\n3. TIME AND QUALITY HAVE DIFFERENT CAUSES. Removing §38.14's `t -\u003e n` from\nharbor: cpsat goes 1.623 -\u003e 0.193 s/seed (8.4x faster) but still +205 vs greedy\n(was +225). So the adjacency explains the time blow-up and ~9% of the quality\ngap; the regression is otherwise pre-existing.\n\nSquaring with §39.5: that section records cpsat returning 194/180/171/182 over\nfour identical 10-seed aggregates before the determinism work. Its 10-fail\nharbor margin (102 vs 92) sits well inside a noise band that wide, and was\nmeasured with fdp's id()-ordered room_slots still live. So the seeder-level\n\"cpsat wins\" claim was never established rather than being overturned. §39.5\nannotated in place.\n\nCaveat stated in the write-up: absolute totals are ~6x §39.5's because the\nobjective has changed (§39.4, §38.10-12), so they are not directly comparable to\nthat table. The greedy-vs-cpsat comparison within this measurement is\nlike-for-like and is what the verdict rests on.\n\nNo default changes -- assign_solver was already greedy for §37.7's independent\nreason, and this reinforces it. What changes is that \"cpsat wins the seeder A/B\"\nshould no longer be cited as a reason to pursue it.\n\nCost recorded and filed as homemaker-py-2xk: the cap fix takes the test suite\nfrom ~4.5 to ~10 min, and the tests cannot opt out because constructive_topology\ndoes not thread the solver limits through.\n","comments":[{"id":"01a04dfd-ca76-7a21-b18d-d2af9ccd9ef4","issue_id":"homemaker-py-vjd","author":"Claude","text":"Correction: the close note above cites the follow-up as homemaker-py-2xk. That\nID does not exist -- I wrote it before creating the issue. The real one is\nhomemaker-py-7t1 (\"cpsat solver limits are not threadable from\nconstructive_topology, so tests pay full solve cost\"). DESIGN.md §38.20 has been\ncorrected to match.","created_at":"2026-08-29T14:47:53Z"}],"dependency_count":0,"dependent_count":0,"comment_count":1} {"_type":"issue","id":"homemaker-py-9gj","title":"quality_uncrinkliness returns a flat hard 0.0, so the objective cannot rank two equally-buried layouts","description":"Narrowed remnant of homemaker-py-ssz after the owner's daylight ruling (DESIGN.md §38.11). A buried leaf usually IS a defect -- corridors and WCs included -- so scoring it badly is correct. The residual complaint is not that the value is low, it is that it is FLAT: quality_uncrinkliness returns exactly 0.0 for every zero-exposure leaf, and since evaluate_leaf multiplies factors into quality and process_storey accumulates value += quality * rate * area, two layouts that differ only in how badly buried their rooms are score identically.\n\nSo the objective gives the search no gradient to descend in precisely the region it most needs to escape. This is a search-mechanics problem, not a calibration one, and it should be judged on whether it helps the search escape -- NOT on fail counts, which by construction it will not move (the fails are real and should stay).\n\nNote the trap recorded in §38.9: an arm optimised under a modified objective must not be scored under the objective it modifies, and equally must not be scored under its own. For a pure gradient change that emits the same fail set, stock scoring IS valid -- that is the one case where the yardstick is sound.","acceptance_criteria":"A variant that keeps the fail set byte-identical to stock (every currently-failing leaf still fails) but is monotone in how buried a leaf is; A/B at fixed budget on harbor + maple with enough seeds to see past the one-seed variance that made the §38.8 n=3 result undecidable.","status":"open","priority":2,"issue_type":"bug","owner":"noreply@anthropic.com","created_at":"2026-08-28T23:14:04Z","created_by":"Claude","updated_at":"2026-08-28T23:14:04Z","dependency_count":0,"dependent_count":0,"comment_count":0}