diff --git a/.beads/issues.jsonl b/.beads/issues.jsonl index 97ecda8..5ef0404 100644 --- a/.beads/issues.jsonl +++ b/.beads/issues.jsonl @@ -110,7 +110,7 @@ {"_type":"issue","id":"homemaker-py-2g7.8","title":"LLM operator synthesis (AlphaEvolve-style): evolve mutation-operator code against the A/B harness","description":"Second LLM role, after the repair operator proves the plumbing: let the LLM propose new OPERATOR CODE (python functions with the mutate_* signature) and evaluate candidates with the exact experiment discipline DESIGN.md already enforces (control reproduces baseline, 3 seeds, 20k evals, verdict). The project's ledger of 20+ operator experiments with verdicts is unusually good few-shot material: feed it the §11-§13 history so it learns what already failed (niching, grading, annealing...) and why. Sandbox the generated code; acceptance purely empirical via the harness. This is compute-hungry — schedule after the shape-curve DP lands so each A/B is cheap.","acceptance_criteria":"one synthesized operator survives the standard 3-seed A/B gate on harbor or maple (mean fails strictly better, control reproduces baseline)","status":"open","priority":3,"issue_type":"feature","owner":"bruno@postle.net","created_at":"2026-08-02T09:15:56Z","created_by":"Bruno Postle","updated_at":"2026-08-02T09:15:56Z","dependencies":[{"issue_id":"homemaker-py-2g7.8","depends_on_id":"homemaker-py-2g7.7","type":"blocks","created_at":"2026-08-02T10:15:56Z","created_by":"Bruno Postle","metadata":"{}"},{"issue_id":"homemaker-py-2g7.8","depends_on_id":"homemaker-py-2g7","type":"parent-child","created_at":"2026-08-02T10:15:56Z","created_by":"Bruno Postle","metadata":"{}"}],"dependency_count":1,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"homemaker-py-pek","title":"fitness.py: delete the dead first process_storey definition (silently shadowed)","description":"Found by the homemaker-py-zrx expert review. class Fitness defines process_storey TWICE: the original gnw-scope version at fitness.py:1146 and the extended hgg version at fitness.py:1452. Python keeps only the second; the first ~45 lines are dead code that still reads as live. This is a silent-bug vector: an edit to the first definition (e.g. a fix to the covered-outside failure emission, which is duplicated verbatim in both) changes nothing at runtime and no test would notice. Delete the first definition (its docstring notes are preserved in the second). No behaviour change; run the suite to confirm 337 pass.","status":"closed","priority":3,"issue_type":"task","assignee":"Bruno Postle","owner":"bruno@postle.net","created_at":"2026-08-02T08:19:56Z","created_by":"Bruno Postle","updated_at":"2026-08-05T15:57:28Z","started_at":"2026-08-05T14:43:30Z","closed_at":"2026-08-05T15:57:28Z","close_reason":"Deleted dead first process_storey definition; verified second is a strict superset; 405 tests pass (5 pre-existing unrelated failures confirmed present on main before this change)","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"homemaker-py-sd3","title":"driver.collapse_best bakes collapse_insearch=True into its finish evaluator, making the 94g keep-better guard vacuous","description":"Found by the homemaker-py-zrx expert review; same family as homemaker-py-7ua but in the PRODUCT (driver.py), not the experiment script. driver.collapse_best builds its evaluator as _fitness_for(str(programme_dir), leaf_sharing, superpose, multi_use=multi_use) — so collapse_insearch silently takes _fitness_for's default True. collapse_best has no collapse_insearch parameter, so evolve.py cannot thread the run's --collapse-insearch flag through even if it wanted to.\n\nConsequences, verified on 5 evolved harbor-house trees today: (1) the keep-better guard of collapse_finish is VACUOUS — base_fails is measured on a deepcopy that _evaluate_full re-collapses in-eval, so base == collapsed on 5/5 files (e.g. evolved-3M-nols-3.dom logs '12 -\u003e 12 (applied)' where the canonical evaluator shows the collapse actually did 15 -\u003e 12). The 94g safety property 'kept only if the fail count does not increase' is therefore not being checked against the true pre-collapse tree: a canonically fail-INCREASING collapse would be silently applied (collapse_global is 'monotone on harbor-house but not proven in general' per its own docstring — the guard exists precisely for that case). (2) The '[finish] collapse: N -\u003e M' log line under-reports the collapse's real effect (experiment logs quoting it understate 94g's contribution). (3) In a --no-collapse-insearch run the finish evaluator contradicts the run's objective outright — the deterministic 7ua mechanism, now in the default pipeline. (4) Minor: max_share and conn_grade are also not forwarded (matters for kpu/anneal and qi6 runs). Same pattern in search_annealed's final rescore branch: _evaluate(..., leaf_sharing=False, superpose=superpose) leaves _evaluate's collapse_insearch default True, and search_annealed has no way to pass the flag to it.\n\nOn the 5 probed files the returned tree's canonical fails happened to equal the reported number (the tree is a collapse fixpoint after iters=6 + 2-opt, so the extra in-eval collapse found nothing) — but that is not guaranteed, and the vacuous guard + misleading log line are unconditional.\n\nRecommended fix: add a collapse_insearch (and max_share/conn_grade) parameter to collapse_best, thread it from evolve.py, and make collapse_finish's keep-better measurement use a CANONICAL (collapse_insearch=False) evaluator regardless — the guard's job is to protect the canonical fail count of the written .dom, which homemaker-fitness scores with the on-disk config (no insearch override). Decide explicitly which objective the final 'best: N fails' report should quote (canonical is what the .dom.fails sidecar will say).","notes":"Fixed. Two changes: (1) fitness.collapse_finish now forces collapse_insearch=False (canonical) for its own base_fails/cand_fails measurement, saving/restoring self._collapse_insearch around the two score_with_fails calls -- regardless of how the Fitness instance itself was configured, so the guard can never again compare a pre-collapsed base against a pre-collapsed candidate. (2) driver.collapse_best now builds its evaluator with collapse_insearch=False explicitly (hardcoded, not threaded -- canonical is always the right objective for the 94g guard and the final reported fail count, matching what homemaker-fitness reports for the written .dom with no override), and threads max_share/conn_grade through to _fitness_for for config parity. Also fixed the same-family bug in search_annealed's no-polish-budget final rescore branch (was silently defaulting to collapse_insearch=True via _evaluate's default; now reads collapse_insearch/multi_use from search_kw). evolve.py forwards conn_grade to collapse_best. Added a regression test (test_collapse_finish_guard_is_canonical_even_with_insearch_collapse_on) that builds a Fitness with collapse_insearch=True in conf and asserts base_f still reflects the true pre-collapse fail count. Full suite: 405 passed (same 5 pre-existing CP-SAT/reassign failures, confirmed present on main before this change, unrelated).","status":"closed","priority":3,"issue_type":"bug","assignee":"Bruno Postle","owner":"bruno@postle.net","created_at":"2026-08-02T08:19:41Z","created_by":"Bruno Postle","updated_at":"2026-08-05T06:47:15Z","started_at":"2026-08-04T23:41:55Z","closed_at":"2026-08-05T06:47:15Z","close_reason":"94g keep-better guard is now non-vacuous; canonical scoring enforced","dependency_count":0,"dependent_count":0,"comment_count":0} -{"_type":"issue","id":"homemaker-py-d86","title":"Rigorously re-verify qpk/1ph historical numbers against the homemaker-py-iio fix","description":"homemaker-py-iio (fixed 2026-08-02) found a stale-leaf-share metadata leak\nin Fitness._collapse_value/_usage_quality that could corrupt one cell of\ncollapse_global's Hungarian assignment during any leaf_sharing+collapse\nrun -- i.e. essentially the entire \"full default stack\" used from\nhomemaker-py-x3b (leaf_sharing default-on) onward, including the very\nstudies that justified defaulting collapse_insearch on (94g, qpk/1ph, 8sh).\n\nA same-codebase fix-vs-no-fix re-run of the qpk protocol (harbor-house,\nbudget 2500, seeds 1-3) confirmed the bug demonstrably perturbs real\nper-seed outcomes under collapse_insearch=ON (2/3 seeds diverged by 5-8\nfails, non-directionally) -- see DESIGN.md §35 for full details. That\nre-run used TODAY's codebase, not the actual historical commit, and only 3\nharbor-house seeds, not the original seed sets -- so it establishes the bug\nwas real and non-trivial but does NOT establish whether 1ph's aggregate\nN=20 programme-house verdict (mean 7.95-\u003e7.10, paired t-test p~=0.028)\nwould have changed under the fix.\n\nThis issue is to do the rigorous version: check out the codebase near the\n1ph commit (~2026-07-24, \"post-qpk commits through 161\"), backport the iio\nfix there in an isolated worktree, and re-run the ACTUAL historical seed\nsets (programme-house N=20 seeds 1-20, harbor-house N=3 seeds 1-3) at the\n1ph protocol's exact parameters, comparing per-seed and aggregate results\nagainst the published numbers. Low priority: the qualitative direction of\nthe qpk/1ph conclusion is probably still right (noise is non-directional\nand the N=20 statistical margin is comfortably above the observed per-seed\nswing), this is about tightening confidence, not expecting a reversal.","notes":"homemaker-py-r5a (fixed 2026-08-02) also affects this: it is the COMMIT-door companion to iio (a leaf relabelled back to its own stale share_type resurrects a stale multiplicity credit). Any re-verification run here should use the codebase state after BOTH iio and r5a, not iio alone.","status":"open","priority":3,"issue_type":"task","owner":"bruno@postle.net","created_at":"2026-08-02T06:53:51Z","created_by":"Bruno Postle","updated_at":"2026-08-02T09:44:35Z","dependency_count":0,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"homemaker-py-d86","title":"Rigorously re-verify qpk/1ph historical numbers against the homemaker-py-iio fix","description":"homemaker-py-iio (fixed 2026-08-02) found a stale-leaf-share metadata leak\nin Fitness._collapse_value/_usage_quality that could corrupt one cell of\ncollapse_global's Hungarian assignment during any leaf_sharing+collapse\nrun -- i.e. essentially the entire \"full default stack\" used from\nhomemaker-py-x3b (leaf_sharing default-on) onward, including the very\nstudies that justified defaulting collapse_insearch on (94g, qpk/1ph, 8sh).\n\nA same-codebase fix-vs-no-fix re-run of the qpk protocol (harbor-house,\nbudget 2500, seeds 1-3) confirmed the bug demonstrably perturbs real\nper-seed outcomes under collapse_insearch=ON (2/3 seeds diverged by 5-8\nfails, non-directionally) -- see DESIGN.md §35 for full details. That\nre-run used TODAY's codebase, not the actual historical commit, and only 3\nharbor-house seeds, not the original seed sets -- so it establishes the bug\nwas real and non-trivial but does NOT establish whether 1ph's aggregate\nN=20 programme-house verdict (mean 7.95-\u003e7.10, paired t-test p~=0.028)\nwould have changed under the fix.\n\nThis issue is to do the rigorous version: check out the codebase near the\n1ph commit (~2026-07-24, \"post-qpk commits through 161\"), backport the iio\nfix there in an isolated worktree, and re-run the ACTUAL historical seed\nsets (programme-house N=20 seeds 1-20, harbor-house N=3 seeds 1-3) at the\n1ph protocol's exact parameters, comparing per-seed and aggregate results\nagainst the published numbers. Low priority: the qualitative direction of\nthe qpk/1ph conclusion is probably still right (noise is non-directional\nand the N=20 statistical margin is comfortably above the observed per-seed\nswing), this is about tightening confidence, not expecting a reversal.","notes":"homemaker-py-r5a (fixed 2026-08-02) also affects this: it is the COMMIT-door companion to iio (a leaf relabelled back to its own stale share_type resurrects a stale multiplicity credit). Any re-verification run here should use the codebase state after BOTH iio and r5a, not iio alone.","status":"closed","priority":3,"issue_type":"task","assignee":"Claude","owner":"bruno@postle.net","created_at":"2026-08-02T06:53:51Z","created_by":"Bruno Postle","updated_at":"2026-08-29T13:21:29Z","started_at":"2026-08-29T13:03:07Z","closed_at":"2026-08-29T13:21:29Z","close_reason":"Done, with one constraint worked around and a stronger answer than expected\n(DESIGN.md §38.18).\n\nCONSTRAINT: this repository's history begins 2026-07-30, six days AFTER the 1ph\ncommit (2026-07-24), so the checkout this issue asks for does not exist here.\nClosest reachable stand-in is 391f510 (2026-07-30) -- a genuine ancestor of the\niio fix 929be5b, therefore pre-iio, and carrying that era's examples/ and\nobjective. Everything below is measured there, not at the true 1ph commit. That\nis a real limitation of the reproduction.\n\nProtocol as published: programme-house init.dom, budget 3000, 4 workers, seeds\n1-20, ON vs OFF, both arms finished with --collapse. Run twice over one\nworktree -- as-is, then with the 22-line iio fitness.py hunk applied.\n\n OFF ON W/L/T mean diff t (df=19)\n published 1ph (07-24) 7.95 7.10 11/6/3 +0.85 2.38\n pre-iio 391f510 8.05 7.10 11/6/3 +0.95 2.59\n same + iio fix backported 8.05 7.10 11/6/3 +0.95 2.59\n\n1. The published verdict REPRODUCES -- ON beats OFF, significant at N=20, same\n 11/6/3 split.\n2. The iio fix changes NOTHING: 0 of 40 (seed, arm) cells differ. Not a\n coincidence of means -- per-seed counts are equal cell by cell.\n\nAnd it could not have been otherwise. The bug needs a leaf carrying a STALE\nshare/share_type, and leaf-sharing only stamps a share where a code has\ncount \u003e 1. programme-house declares count: 1 for all six codes. Measured over 8\nconstructed seeds at that commit:\n\n programme-house 56 leaves share\u003e1 on 0 share_type set on 0\n harbor-house 128 leaves share\u003e1 on 24 share_type set on 24\n\n_collapse_value reads leaf.share_type; on programme-house it is never set, so\nthe bug is STRUCTURALLY UNREACHABLE on the 1ph protocol. That is why §35 saw 2/3\nharbor seeds diverge by 5-8 fails while programme-house at N=20 moves not one\ncell -- harbor has codes at counts 10, 6 and 5.\n\n§20's retroactive caveat is discharged FOR THE 1ph SECTION and stays live for\nharbor-house/qpk, where shares exist and divergence was measured.\n\nKept: experiments/rerun_1ph_protocol.sh and the two result TSVs.\n\nMethod note for future archaeology: \"was measurement X affected by bug Y\" was\nanswerable here from the programme's structure (all counts == 1) without running\nanything. Worth checking that first.\n","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"homemaker-py-7ua","title":"run_staged_search.py final rescore omits collapse_insearch override, causing false MISMATCH under leaf-sharing","description":"experiments/run_staged_search.py's _native_score() (used for the final 're-scored (native): ... -\u003e OK/MISMATCH' sanity line) calls fitness.load_config(programme_dir) with NO overrides, but driver.search_staged's internal evaluator always runs with collapse_insearch=True (baked into driver.search's default, search_staged has no param to disable it). The script's monkeypatched fitness.load_config only injects leaf_sharing/share_edge_cap/multi_use, not collapse_insearch, so the final rescore conf silently diverges from the search-time conf whenever leaf_sharing is on (the current default stack). Observed during homemaker-py-91f: a WORKERS=4 budget=2000 harbor-house run reported best fails=38 during search but re-scored fails=34 -\u003e MISMATCH (partly parallel non-determinism per homemaker-py-b8g, but the missing collapse_insearch override is a separate, deterministic contributor). Fix: add collapse_insearch=True to the monkeypatched conf alongside leaf_sharing/share_edge_cap.","status":"closed","priority":3,"issue_type":"bug","assignee":"Bruno Postle","owner":"bruno@postle.net","created_at":"2026-08-01T11:32:58Z","created_by":"Bruno Postle","updated_at":"2026-08-05T09:04:05Z","started_at":"2026-08-05T07:33:29Z","closed_at":"2026-08-05T09:04:05Z","close_reason":"Fixed: added collapse_insearch=True to the monkeypatched load_config conf in run_staged_search.py's leaf_share/multi_use branch. Verified with a smoke run (programme-house, LEAFSHARE=1, budget=150): pre-fix reported MISMATCH (1.51708e-08 vs 1.56663e-08), post-fix reports OK with identical values. Full pytest suite: 405 passed, 5 pre-existing failures in test_cpsat.py/test_operators.py unrelated to this change (confirmed failing on main before this fix too).","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"homemaker-py-b8g","title":"Investigate parallel/BLAS non-determinism noise source in n_workers\u003e1 runs","description":"DESIGN.md §14 (psk, island-model experiment) flagged a real, uninvestigated noise source: 'Phase A is unaffected by the probe, yet harbor seed 2 scored 71 then 73 on byte-identical re-runs -- parallel/BLAS non-determinism, the same +/-2-3 effect §12.4 flagged.' This is DISTINCT from the homemaker-py-xcy bug (ProcessPoolExecutor as_completed ordering), which was fixed and made same-worker-count parallel runs reproducible for the SEARCH TRAJECTORY. This remaining noise is at the SCORING level (a single fitness eval on a fixed genome apparently returning different fail counts across runs), plausibly numpy/scipy BLAS thread nondeterminism in the geometry/inner-loop math. It was never root-caused or fixed, and it widens the error bars on every A/B in this log run at n_workers\u003e1 (the great majority of them, since serial sweeps are expensive). Investigate: reproduce minimally (score the same frozen .dom N times under workers\u003e1), bisect whether it's BLAS threading (try OMP_NUM_THREADS=1/OPENBLAS_NUM_THREADS=1), floating-point summation order, or something else; fix or document a mitigation (e.g. pin thread count in worker processes).","design":"Reference: DESIGN.md §14 'Noise caveat (carry forward)', §12.4 (homemaker-py-xcy, the related-but-distinct trajectory-ordering bug already fixed). If the cause is BLAS thread count, the fix is likely a one-line env pin in the worker pool initializer (driver.py's ProcessPoolExecutor setup).","notes":"homemaker-py-zrx review (2026-08-02) found a concrete, non-BLAS candidate mechanism for part of this noise in PARALLEL STAGED runs: homemaker-py-cvw — substrate_readiness in the parent process reads stale id()-keyed geometry cache entries (24/300 corrupted in a churn probe, worst error ~1.0), perturbing stage-1 selection address-dependently across byte-identical re-runs. Does not explain fixed-genome single-eval divergence (if that was ever actually isolated); re-test after cvw lands before chasing BLAS.","status":"closed","priority":3,"issue_type":"bug","assignee":"Claude","owner":"bruno@postle.net","created_at":"2026-08-01T10:07:45Z","created_by":"Bruno Postle","updated_at":"2026-08-29T12:52:33Z","started_at":"2026-08-29T12:19:37Z","closed_at":"2026-08-29T12:52:33Z","close_reason":"Investigated. The premise does not survive measurement: there is no\nnon-determinism (DESIGN.md §38.17).\n\n score a frozen .dom, 20 repeats in one process bit-identical\n same .dom, 8 processes, varied PYTHONHASHSEED bit-identical\n full search, harbor seeds 0-3, n_workers 1..4,\n repeated across processes bit-identical PER worker count\n the same with OMP/OPENBLAS/MKL_NUM_THREADS=1 IDENTICAL to unpinned\n\nThat last line is the important one. This issue's proposed remedy was \"likely a\none-line env pin in the worker pool initializer\". Pinning the BLAS thread count\nchanges nothing bit-for-bit, so shipping it would have looked like a fix, done\nnothing, and retired the issue. BLAS is not implicated. The bead's other\nhypothesis -- \"a single fitness eval on a fixed genome returning different fail\ncounts\" -- is also false.\n\nWhat IS real, and is not noise: the trajectory is a deterministic FUNCTION of\nn_workers. harbor seed 3, budget 1500:\n\n w=1 64 fails 1.6264880162149419e-22\n w=2 64 fails, same bits\n w=3 64 fails, same bits\n w=4 65 fails 7.685882216045091e-23\n\nEach stable across processes. Mechanism, driver.py's batch loop:\n batch_n = min(n_workers, max(1, (budget - n_evals + child_budget - 1) // child_budget))\nbatch_n children are bred from ONE population snapshot before any is admitted,\nand the shared rng is consumed in a different pattern. At w=1 each child sees\nthe population its predecessor updated. So a 4-worker run is partly generational\nand a 1-worker run is steady-state -- same seed, different search. Divergence is\noccasional (seeds 0/1/2 agreed across worker counts, seed 3 did not), which is\nexactly how it reads as \"noise\" when sampled.\n\n§14's \"71 then 73 on byte-identical re-runs\" was most likely homemaker-py-xcy,\nthe as_completed admission-ordering bug, which WAS genuinely non-deterministic\nand is fixed. No residual noise behind it.\n\nShipped instead of a no-op env pin:\n - driver.search's docstring states n_workers is an ALGORITHM parameter and\n that results from different worker counts are not comparable.\n - test_search_is_reproducible_at_a_fixed_worker_count, parametrised over\n 2/3/4 workers -- asserts each is internally stable, and deliberately does\n NOT assert they agree with each other.\n - test_scoring_a_frozen_design_is_deterministic guards the floor.\n\nPractical exposure: the run_*_ab.sh harnesses already pin WORKERS=4 within a\nrun, so arms inside one harness are sound. The risk is comparing across\nharnesses, or against a historical figure whose worker count was not recorded.\n","dependency_count":0,"dependent_count":1,"comment_count":0} {"_type":"issue","id":"homemaker-py-7xb","title":"Validate full winning construction stack generalises to health-centre","description":"The whole positive construction-quality stack (adjacency-aware + proportion-aware seeding, depth-balanced growth, leaf-sharing factor 3, interior-O odiv=3, share-aware edge cap) has only ever been measured end-to-end on harbor-house and maple-court (DESIGN.md §11-§13, cumulative -54%/-41% vs the leu.2 baseline per §13.7). examples/health-centre exists (built for homemaker-py-9yx, a non-synthetic ~20-room programme of a different building type -- primary care, not house/co-housing) but has only ever been used to NULL-test ruin_recreate; the positive stack itself has never been run there. Run the current default full stack (staged search, matching the §13.9/§13.10 default config) on health-centre at a comparable budget/seed count to harbor/maple's Phase-8 measurements, and report whether the fail-count reduction pattern (dominated by leaf-sharing, then depth-balance synergy, then interior-O) holds on a structurally different programme mix, or whether health-centre's room-type diversity (19 distinct codes, mostly single-instance, per §32) changes which lever dominates.","design":"Reference: DESIGN.md §13.3/§13.5/§13.6/§13.9 (the levers to validate), §32 (9yx, health-centre's construction and room-code tiering). No new code expected -- this is a measurement run with the existing default-on stack, comparable to the leu.1/§12.1 benchmark-establishment style.","status":"open","priority":3,"issue_type":"task","owner":"bruno@postle.net","created_at":"2026-08-01T10:07:29Z","created_by":"Bruno Postle","updated_at":"2026-08-01T10:07:29Z","dependencies":[{"issue_id":"homemaker-py-7xb","depends_on_id":"homemaker-py-7b7","type":"blocks","created_at":"2026-08-26T16:20:35Z","created_by":"Claude","metadata":"{}"}],"dependency_count":1,"dependent_count":0,"comment_count":0} diff --git a/DESIGN.md b/DESIGN.md index 8b419f3..0eb18bb 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -2784,6 +2784,13 @@ is real per-seed noise (not a directional bias) that does not appear to overturn verdict above, but the exact historical per-seed numbers quoted in this section were not re-measured under the fix. See §35 for the mechanism and what was (and wasn't) re-confirmed. +**Caveat DISCHARGED for this section (`homemaker-py-d86`, §38.18).** The 1ph protocol has now been +re-run at N=20 on a pre-iio commit with and without the fix backported: all 40 (seed, arm) cells are +identical, and the reason is structural rather than lucky — programme-house declares `count: 1` for +every code, so no leaf ever acquires a share and the stale-share bug cannot fire on this programme +at all. The numbers above stand. The caveat remains live for harbor-house/qpk, where shares do exist +and §35 measured real divergence. + ## 21. Insert/relocate-circulation repair operator (`homemaker-py-8sh`) — DONE (mixed, kept off) **Motivation.** qi6's remaining candidate (§18): mechanism (a), an explicit search-time @@ -5778,6 +5785,59 @@ over 2/3/4 workers, asserting each is internally stable and deliberately not asserting they agree with each other) and `test_scoring_a_frozen_design_is_deterministic`. +### 38.18 The 1ph verdict re-verified: the iio bug could never have touched it (`homemaker-py-d86`) + +`d86` asked for the rigorous version of §35's spot-check: take the codebase at the +`1ph` commit, backport the `iio` stale-leaf-share fix, and re-run the actual +historical seed sets to see whether the published verdict would have changed. + +**One constraint had to be worked around.** This repository's history begins +**2026-07-30**, six days *after* the 1ph commit (2026-07-24) — the historical +checkout the issue asks for does not exist here. The closest reachable stand-in +is `391f510` (2026-07-30), which is a genuine ancestor of the iio fix +(`929be5b`, 2026-08-01) and therefore pre-iio, and which carries that era's +`examples/` and objective. Everything below is measured there, not at the true +1ph commit, and that is a real limitation of the reproduction. + +**Protocol as published:** programme-house `init.dom`, budget 3000, 4 workers, +seeds 1–20, ON vs OFF, both arms finished with `--collapse`. Run twice over the +same worktree — once as-is, once with the 22-line `iio` `fitness.py` hunk applied. + +| codebase | OFF | ON | W/L/T | mean diff | t (df=19) | +|---|---|---|---|---|---| +| published 1ph (2026-07-24) | 7.95 | 7.10 | 11/6/3 | +0.85 | 2.38 | +| pre-iio `391f510` | **8.05** | **7.10** | **11/6/3** | **+0.95** | **2.59** | +| the same, + iio fix backported | 8.05 | 7.10 | 11/6/3 | +0.95 | 2.59 | + +Two results: + +1. **The published verdict reproduces.** ON beats OFF, significant at N=20, with + a win/loss/tie split identical to the published 11/6/3. +2. **The iio fix changes nothing — 0 of 40 (seed, arm) cells differ.** Not a + coincidence of means: the per-seed fail counts are equal cell by cell. + +**And it could not have been otherwise.** The iio bug needs a leaf carrying a +*stale* `share`/`share_type` — leftover multiplicity from a code the leaf has +since been retyped away from. Leaf-sharing only ever stamps a share when a code +has `count > 1`, and **programme-house declares `count: 1` for all six of its +codes**. Measured directly over 8 constructed seeds at that commit: + +| programme | leaves | `share > 1` | `share_type` set | +|---|---|---|---| +| programme-house | 56 | **0** | **0** | +| harbor-house | 128 | 24 | 24 | + +`_collapse_value` reads `leaf.share_type`; on programme-house it is never set, +so the bug is **structurally unreachable on the 1ph protocol**. That is why §35 +saw 2 of 3 harbor seeds diverge by 5–8 fails while programme-house at N=20 moves +not one cell — harbor has codes at counts 10, 6 and 5. + +§20's retroactive caveat is therefore **discharged for the 1ph section** and +stays live for harbor-house/qpk, where shares exist and the divergence was +measured. Worth noting for future archaeology: "was this measurement affected by +bug X" is often answerable from the programme's structure without re-running +anything. + ## 39. Config audit: requirements that actively fight the engine (`homemaker-py-ju3`) — measured 2026-08-25 The corpus `patterns.config` targets and `costs.config` values were estimated diff --git a/experiments/rerun_1ph_protocol.sh b/experiments/rerun_1ph_protocol.sh new file mode 100755 index 0000000..30e07d0 --- /dev/null +++ b/experiments/rerun_1ph_protocol.sh @@ -0,0 +1,26 @@ +#!/usr/bin/env bash +# Re-run the 1ph protocol (DESIGN.md §20): programme-house init.dom, budget 3000, +# 4 workers, seeds 1-20, ON vs OFF, BOTH arms finished with --collapse. +set -u +W="$1"; TAG="$2"; OUT="$3" +cd "$W/examples/programme-house" +: > "$OUT" +for seed in $(seq 1 20); do + for arm in on off; do + flag=""; [ "$arm" = "off" ] && flag="--no-collapse-insearch" + t0=$(date +%s) + PYTHONPATH="$W/src" timeout 600 python -m homemaker_layout.evolve init.dom \ + --budget 3000 --seed "$seed" --workers 4 --collapse $flag \ + --output "$OUT.$arm.dom" > "$OUT.$arm.log" 2>&1 + t1=$(date +%s) + n=$(PYTHONPATH="$W/src" python - "$OUT.$arm.dom" <<'PY' +import copy, sys +from homemaker_layout import dom, fitness +conf, cost = fitness.load_config(".") +_, f = fitness.Fitness(conf, cost).score_with_fails(copy.deepcopy(dom.load(sys.argv[1]))) +print(len(f)) +PY +) + echo -e "$TAG\t$seed\t$arm\t$n\t$((t1-t0))" >> "$OUT" + done +done diff --git a/experiments/results/d86_1ph_iiofix.tsv b/experiments/results/d86_1ph_iiofix.tsv new file mode 100644 index 0000000..b5e04c4 --- /dev/null +++ b/experiments/results/d86_1ph_iiofix.tsv @@ -0,0 +1,40 @@ +iiofix 1 on 5 8 +iiofix 1 off 8 6 +iiofix 2 on 7 8 +iiofix 2 off 8 6 +iiofix 3 on 9 8 +iiofix 3 off 8 6 +iiofix 4 on 9 7 +iiofix 4 off 11 5 +iiofix 5 on 9 8 +iiofix 5 off 8 7 +iiofix 6 on 8 7 +iiofix 6 off 11 5 +iiofix 7 on 9 8 +iiofix 7 off 11 6 +iiofix 8 on 8 8 +iiofix 8 off 7 7 +iiofix 9 on 6 7 +iiofix 9 off 9 6 +iiofix 10 on 4 8 +iiofix 10 off 7 7 +iiofix 11 on 5 7 +iiofix 11 off 5 6 +iiofix 12 on 7 8 +iiofix 12 off 6 7 +iiofix 13 on 6 7 +iiofix 13 off 9 6 +iiofix 14 on 8 8 +iiofix 14 off 7 6 +iiofix 15 on 6 8 +iiofix 15 off 8 6 +iiofix 16 on 8 7 +iiofix 16 off 8 6 +iiofix 17 on 6 7 +iiofix 17 off 8 6 +iiofix 18 on 7 7 +iiofix 18 off 8 6 +iiofix 19 on 8 7 +iiofix 19 off 7 6 +iiofix 20 on 7 7 +iiofix 20 off 7 6 diff --git a/experiments/results/d86_1ph_preiio.tsv b/experiments/results/d86_1ph_preiio.tsv new file mode 100644 index 0000000..a99d71e --- /dev/null +++ b/experiments/results/d86_1ph_preiio.tsv @@ -0,0 +1,40 @@ +preiio 1 on 5 8 +preiio 1 off 8 6 +preiio 2 on 7 7 +preiio 2 off 8 6 +preiio 3 on 9 7 +preiio 3 off 8 7 +preiio 4 on 9 8 +preiio 4 off 11 6 +preiio 5 on 9 8 +preiio 5 off 8 7 +preiio 6 on 8 8 +preiio 6 off 11 6 +preiio 7 on 9 7 +preiio 7 off 11 6 +preiio 8 on 8 7 +preiio 8 off 7 6 +preiio 9 on 6 8 +preiio 9 off 9 5 +preiio 10 on 4 7 +preiio 10 off 7 7 +preiio 11 on 5 7 +preiio 11 off 5 6 +preiio 12 on 7 8 +preiio 12 off 6 7 +preiio 13 on 6 8 +preiio 13 off 9 6 +preiio 14 on 8 8 +preiio 14 off 7 6 +preiio 15 on 6 8 +preiio 15 off 8 6 +preiio 16 on 8 7 +preiio 16 off 8 5 +preiio 17 on 6 7 +preiio 17 off 8 6 +preiio 18 on 7 7 +preiio 18 off 8 5 +preiio 19 on 8 7 +preiio 19 off 7 5 +preiio 20 on 7 7 +preiio 20 off 7 6