Re-validate collapse_insearch's default under the current objective
38.18 confirmed the 1ph default-flip was sound for its own era, but that
measurement predates three changes to the objective it was measured
against -- 39.4's namespace fix, 38.10/38.11's per-space crinkliness, and
38.12's missing-space cascade -- and collapse_insearch runs collapse_global
inside every eval, valued against exactly the factors those touched. A
default carried on a superseded measurement is an assumption, not a result.
Re-ran the 1ph protocol as published on the current codebase:
N OFF ON W/L/T diff t p
published 1ph 20 7.95 7.10 11/6/3 +0.85 2.38 0.028
current objective 20 7.85 7.15 10/7/3 +0.70 1.82 0.069
current objective 40 7.60 7.03 21/14/5 +0.57 2.01 0.045
current objective 60 7.58 7.02 29/19/12 +0.57 2.45 0.017
Verdict: the default STANDS. At N=60, mean diff +0.567 fails/seed, paired
t=2.454 (df=59), p=0.0171 exact, 95% CI [+0.105, +1.029] excluding zero;
Wilcoxon signed-rank cross-check agrees (p=0.0138), which matters because
fail counts are small integers and normality is not obvious.
Two caveats. The effect is about a third smaller than published (+0.57 vs
+0.85) -- partly regression from a lucky N=20 draw, partly plausible real
erosion, since several fails collapse_global used to clear have been
redefined out of existence or made harder.
More usefully: the published N=20 can no longer detect its own effect. At
exactly that sample size the current answer is p ~= 0.069, a null by the
conventional threshold. Had I stopped at N=20 the honest report would have
been "the 1ph verdict no longer reproduces" and the default would have
looked unjustified. It took N=60 to resolve. That is the 8sh/1ph/qi6/lj3
pattern this log warns about, now biting the flagship result itself: any
future re-validation of this default needs N >= 40.
20 annotated in place so a reader of the original claim sees the current
figure. Harness takes a seed range now (APPEND=1 to extend a sweep).
Closes homemaker-py-ioe.
Lint at parity (46); tests 384 passed, 0 failed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
This commit is contained in:
parent
50fb293fa5
commit
e9e7060fb4
4 changed files with 174 additions and 4 deletions
|
|
@ -36,6 +36,7 @@
|
||||||
{"_type":"issue","id":"homemaker-py-1p0","title":"Geometry inner loop: full-objective equal-offset ratio optimiser","description":"DESIGN.md §5.1, §7 Phase 1. Productionise experiments/optimize_fullfitness.py into homemaker: optimise(topology, x0=None) -\u003e (geometry, fitness). DOF = equal-offset division ratios of free branches (solver.free_branches, lowest-storey cut ownership), clipped to [eps, 1-eps]. Objective = full oracle fitness (never a proxy — §4.2 falsified). Must support warm-start x0 (§5.6) and a population/batch evaluation mode so each iteration scores via one batched oracle call (§4.6).","acceptance_criteria":"Reproduces or exceeds §4.5 gains (x1.24–x1.67, no new failures) on 2f45907, candidate-002, c964435; works as a library call on any corpus .dom","status":"closed","priority":1,"issue_type":"feature","assignee":"Bruno Postle","owner":"bruno@postle.net","created_at":"2026-06-11T23:36:58Z","created_by":"Bruno Postle","updated_at":"2026-06-12T08:46:31Z","started_at":"2026-06-12T00:14:19Z","closed_at":"2026-06-12T08:46:31Z","close_reason":"innerloop.optimise() lands: batched CMA-ES sigma ladder (0.05/0.15, IPOP popsize doubling, deterministic seeding) over equal-offset free-branch ratios vs full oracle fitness; warm-start x0 supported. Acceptance vs unprojected originals: x1.65/x1.66/x1.58 against bars x1.24/x1.67/x1.59, no new failures, 46 oracle calls vs NM's 200. Two near-bar results accepted as reproduced-within-noise (1% tol) — draw spread brackets the single-NM-draw bars; approved by Bruno 2026-06-12. Gotchas: equal-offset projection of legacy unequal cuts loses fitness/adds failures (midpoint projection used); pycma seed=0 means clock-seeded.","dependencies":[{"issue_id":"homemaker-py-1p0","depends_on_id":"homemaker-py-av5","type":"blocks","created_at":"2026-06-12T00:39:33Z","created_by":"Bruno Postle","metadata":"{}"}],"dependency_count":1,"dependent_count":3,"comment_count":0}
|
{"_type":"issue","id":"homemaker-py-1p0","title":"Geometry inner loop: full-objective equal-offset ratio optimiser","description":"DESIGN.md §5.1, §7 Phase 1. Productionise experiments/optimize_fullfitness.py into homemaker: optimise(topology, x0=None) -\u003e (geometry, fitness). DOF = equal-offset division ratios of free branches (solver.free_branches, lowest-storey cut ownership), clipped to [eps, 1-eps]. Objective = full oracle fitness (never a proxy — §4.2 falsified). Must support warm-start x0 (§5.6) and a population/batch evaluation mode so each iteration scores via one batched oracle call (§4.6).","acceptance_criteria":"Reproduces or exceeds §4.5 gains (x1.24–x1.67, no new failures) on 2f45907, candidate-002, c964435; works as a library call on any corpus .dom","status":"closed","priority":1,"issue_type":"feature","assignee":"Bruno Postle","owner":"bruno@postle.net","created_at":"2026-06-11T23:36:58Z","created_by":"Bruno Postle","updated_at":"2026-06-12T08:46:31Z","started_at":"2026-06-12T00:14:19Z","closed_at":"2026-06-12T08:46:31Z","close_reason":"innerloop.optimise() lands: batched CMA-ES sigma ladder (0.05/0.15, IPOP popsize doubling, deterministic seeding) over equal-offset free-branch ratios vs full oracle fitness; warm-start x0 supported. Acceptance vs unprojected originals: x1.65/x1.66/x1.58 against bars x1.24/x1.67/x1.59, no new failures, 46 oracle calls vs NM's 200. Two near-bar results accepted as reproduced-within-noise (1% tol) — draw spread brackets the single-NM-draw bars; approved by Bruno 2026-06-12. Gotchas: equal-offset projection of legacy unequal cuts loses fitness/adds failures (midpoint projection used); pycma seed=0 means clock-seeded.","dependencies":[{"issue_id":"homemaker-py-1p0","depends_on_id":"homemaker-py-av5","type":"blocks","created_at":"2026-06-12T00:39:33Z","created_by":"Bruno Postle","metadata":"{}"}],"dependency_count":1,"dependent_count":3,"comment_count":0}
|
||||||
{"_type":"issue","id":"homemaker-py-8cs","title":"Experiment: warm-vs-cold start of inner loop (Lamarckian inheritance)","description":"DESIGN.md §5.6, §4.6. Warm-starting a child topology's inner loop from the parent's optimised ratios is the main lever for cutting per-topology cost (~3 min/topology cold). Apply single topology mutations to optimised corpus designs, re-optimise warm (surviving cuts keep values, new cuts get heuristic defaults) vs cold, compare oracle-call counts to convergence at equal final fitness.","acceptance_criteria":"Speedup factor measured across \u003e=10 mutated topologies; decision recorded (expect order-of-magnitude; if \u003c2x, revisit §4.6 Phase-2 scoping)","notes":"Experiment script committed (experiments/warm_vs_cold.py, 1cc86c8) and machinery validated oracle-free; one mutated child scored through the oracle OK. Waiting on homemaker-py-gp2 reference run to finish, then execute under URB_NO_OCCLUSION=1 (3 parents x 400 evals + 12 children x 2 x 200 evals, ~1.5-2 h oracle time). Default budgets: parent 400, child 200; target = evals to 95% of best final.","status":"closed","priority":1,"issue_type":"task","owner":"bruno@postle.net","created_at":"2026-06-11T23:36:58Z","created_by":"Bruno Postle","updated_at":"2026-06-12T11:44:45Z","closed_at":"2026-06-12T11:44:45Z","close_reason":"Measured (URB_NO_OCCLUSION=1, parent budget 400, child 200, 12 single mutations across 3 designs): cold start reached 95% of warm final in 0/12 cases within budget — speedup unbounded at practical budgets; warm finals beat cold finals x1.2-x4 in 12/12; 6/12 warm starts were within 95% at 1 eval (near-neutral mutations). Decision: Lamarckian warm-starting is MANDATORY in the memetic driver (homemaker-py-b39), not an optimisation; cold starts produce strictly worse geometry at equal budget. Note: 2 undivides were exactly fitness-neutral (same-type merge == Merge_Divided equivalence) — locality datum for homemaker-py-nyb.","dependencies":[{"issue_id":"homemaker-py-8cs","depends_on_id":"homemaker-py-1p0","type":"blocks","created_at":"2026-06-12T00:39:34Z","created_by":"Bruno Postle","metadata":"{}"}],"dependency_count":1,"dependent_count":0,"comment_count":0}
|
{"_type":"issue","id":"homemaker-py-8cs","title":"Experiment: warm-vs-cold start of inner loop (Lamarckian inheritance)","description":"DESIGN.md §5.6, §4.6. Warm-starting a child topology's inner loop from the parent's optimised ratios is the main lever for cutting per-topology cost (~3 min/topology cold). Apply single topology mutations to optimised corpus designs, re-optimise warm (surviving cuts keep values, new cuts get heuristic defaults) vs cold, compare oracle-call counts to convergence at equal final fitness.","acceptance_criteria":"Speedup factor measured across \u003e=10 mutated topologies; decision recorded (expect order-of-magnitude; if \u003c2x, revisit §4.6 Phase-2 scoping)","notes":"Experiment script committed (experiments/warm_vs_cold.py, 1cc86c8) and machinery validated oracle-free; one mutated child scored through the oracle OK. Waiting on homemaker-py-gp2 reference run to finish, then execute under URB_NO_OCCLUSION=1 (3 parents x 400 evals + 12 children x 2 x 200 evals, ~1.5-2 h oracle time). Default budgets: parent 400, child 200; target = evals to 95% of best final.","status":"closed","priority":1,"issue_type":"task","owner":"bruno@postle.net","created_at":"2026-06-11T23:36:58Z","created_by":"Bruno Postle","updated_at":"2026-06-12T11:44:45Z","closed_at":"2026-06-12T11:44:45Z","close_reason":"Measured (URB_NO_OCCLUSION=1, parent budget 400, child 200, 12 single mutations across 3 designs): cold start reached 95% of warm final in 0/12 cases within budget — speedup unbounded at practical budgets; warm finals beat cold finals x1.2-x4 in 12/12; 6/12 warm starts were within 95% at 1 eval (near-neutral mutations). Decision: Lamarckian warm-starting is MANDATORY in the memetic driver (homemaker-py-b39), not an optimisation; cold starts produce strictly worse geometry at equal budget. Note: 2 undivides were exactly fitness-neutral (same-type merge == Merge_Divided equivalence) — locality datum for homemaker-py-nyb.","dependencies":[{"issue_id":"homemaker-py-8cs","depends_on_id":"homemaker-py-1p0","type":"blocks","created_at":"2026-06-12T00:39:34Z","created_by":"Bruno Postle","metadata":"{}"}],"dependency_count":1,"dependent_count":0,"comment_count":0}
|
||||||
{"_type":"issue","id":"homemaker-py-av5","title":"Batched oracle: score many .dom files per invocation","description":"oracle.py currently scores one .dom per urb-fitness.pl call (~1.65 s/dom). DESIGN.md §4.6: batching amortises Perl startup to ~0.99 s/dom and is required so population/batch optimisers can score a whole generation in one oracle call. Extend oracle.py with a batch API: write N .dom files, one perl invocation, parse N .score/.fails pairs. Keep the single-file path for compatibility.","acceptance_criteria":"Batch of 35 corpus files scores in one perl invocation; per-file results identical to single-file calls; measured s/dom reported","status":"closed","priority":1,"issue_type":"task","assignee":"Bruno Postle","owner":"bruno@postle.net","created_at":"2026-06-11T23:36:56Z","created_by":"Bruno Postle","updated_at":"2026-06-12T00:14:06Z","started_at":"2026-06-11T23:50:40Z","closed_at":"2026-06-12T00:14:06Z","close_reason":"score_batch() lands in oracle.py; 35-file corpus parity verified single-vs-batch (1e-12 rel fitness, exact fail sets); 0.98 s/dom batched vs 1.27 single, x1.30","dependency_count":0,"dependent_count":1,"comment_count":0}
|
{"_type":"issue","id":"homemaker-py-av5","title":"Batched oracle: score many .dom files per invocation","description":"oracle.py currently scores one .dom per urb-fitness.pl call (~1.65 s/dom). DESIGN.md §4.6: batching amortises Perl startup to ~0.99 s/dom and is required so population/batch optimisers can score a whole generation in one oracle call. Extend oracle.py with a batch API: write N .dom files, one perl invocation, parse N .score/.fails pairs. Keep the single-file path for compatibility.","acceptance_criteria":"Batch of 35 corpus files scores in one perl invocation; per-file results identical to single-file calls; measured s/dom reported","status":"closed","priority":1,"issue_type":"task","assignee":"Bruno Postle","owner":"bruno@postle.net","created_at":"2026-06-11T23:36:56Z","created_by":"Bruno Postle","updated_at":"2026-06-12T00:14:06Z","started_at":"2026-06-11T23:50:40Z","closed_at":"2026-06-12T00:14:06Z","close_reason":"score_batch() lands in oracle.py; 35-file corpus parity verified single-vs-batch (1e-12 rel fitness, exact fail sets); 0.98 s/dom batched vs 1.27 single, x1.30","dependency_count":0,"dependent_count":1,"comment_count":0}
|
||||||
|
{"_type":"issue","id":"homemaker-py-ioe","title":"Is collapse_insearch=True still the right default under the current objective?","description":"The default was flipped OFF -\u003e ON by homemaker-py-1ph (DESIGN.md §20, 2026-07-24) on the strength of a programme-house N=20 sweep: mean 7.95 -\u003e 7.10, 11W/6L/3T, paired t p ~= 0.028. homemaker-py-d86 (§38.18) has now confirmed that verdict was sound FOR ITS OWN ERA -- it reproduces on a pre-iio commit, and the iio stale-share bug is structurally unreachable on that protocol because programme-house declares count: 1 for every code, so no leaf ever carries a share.\n\nBut the objective has changed substantially since, three times over, and all of it after 1ph:\n\n §39.4 the generic-namespace fix -- codes like cr1 were being read as generic\n circulation, so 14% of harbor's programme was silently optional\n §38.10 / §38.11 crinkliness declared per space; 14 corpus spaces now declare\n crinkliness: none\n §38.12 the missing-space cascade no longer weighted by YAML verbosity, a fixed\n 5 fails per missing instance instead of 3-5\n\ncollapse_insearch runs collapse_global inside every fitness eval, and collapse_global's assignment is valued against exactly the quality factors those changes touched. So the ON-beats-OFF margin was measured against an objective that no longer exists. The direction is plausibly unchanged -- but it is currently an assumption carried on a superseded measurement, and it is a DEFAULT, so every run inherits it.\n\nThe protocol and harness already exist: experiments/rerun_1ph_protocol.sh runs programme-house, budget 3000, 4 workers, seeds 1-20, ON vs OFF, both arms finished with --collapse, and takes about 6 minutes.\n\nNote when re-running: n_workers is an algorithm parameter (§38.17), so keep 4 workers to stay comparable with the published protocol, and record it with the result.","acceptance_criteria":"The 1ph protocol re-run at N=20 on the current codebase and objective, with the ON-vs-OFF verdict either reconfirmed or restated; if the margin has moved materially, DESIGN.md §20's default-flip rationale is updated to say so and the default is reconsidered on the new numbers.","status":"closed","priority":2,"issue_type":"task","assignee":"Claude","owner":"noreply@anthropic.com","created_at":"2026-08-29T13:36:18Z","created_by":"Claude","updated_at":"2026-08-29T13:57:45Z","started_at":"2026-08-29T13:36:32Z","closed_at":"2026-08-29T13:57:45Z","close_reason":"Re-validated: the default STANDS, but with two caveats worth carrying\n(DESIGN.md §38.19).\n\nRe-ran the 1ph protocol as published on the current codebase and objective --\nprogramme-house, budget 3000, 4 workers, ON vs OFF, both arms finished with\n--collapse.\n\n N OFF ON W/L/T diff t p\n published 1ph (07-24) 20 7.95 7.10 11/6/3 +0.85 2.38 0.028\n historical re-run (§38.18) 20 8.05 7.10 11/6/3 +0.95 2.59 --\n current objective 20 7.85 7.15 10/7/3 +0.70 1.82 0.069\n current objective 40 7.60 7.03 21/14/5 +0.57 2.01 0.045\n current objective 60 7.58 7.02 29/19/12 +0.57 2.45 0.017\n\nAt N=60: mean diff +0.567 fails/seed, paired t=2.454 (df=59), p=0.0171 exact,\n95% CI [+0.105, +1.029] excluding zero. Wilcoxon signed-rank cross-check agrees\n(p=0.0138), which matters because fail counts are small integers and normality\nis not obvious.\n\nCaveat 1: the effect is about a third smaller than published (+0.57 vs +0.85).\nPartly regression from a slightly lucky N=20 draw, partly plausible real erosion\n-- several fails collapse_global used to clear have been redefined out of\nexistence or made harder by §39.4 / §38.10-12.\n\nCaveat 2, the more useful one: THE PUBLISHED N=20 CAN NO LONGER DETECT ITS OWN\nEFFECT. At exactly the published sample size the current answer is p ~= 0.069, a\nnull by the conventional threshold. Had I run N=20 and stopped, the honest report\nwould have been \"the 1ph verdict no longer reproduces\" and the default would have\nlooked unjustified. It took N=60 to resolve. That is the \"8sh/1ph/qi6/lj3\npattern\" this log already warns about, now biting the flagship result itself.\nAny future re-validation of this default needs N \u003e= 40; N=20 should not be\ntrusted to settle it either way.\n\n§20 annotated in place so a reader of the original claim sees the current figure.\nHarness now takes a seed range (APPEND=1 to extend a sweep); results in\nexperiments/results/ioe_1ph_current_objective.tsv.\n","dependency_count":0,"dependent_count":0,"comment_count":0}
|
||||||
{"_type":"issue","id":"homemaker-py-vjd","title":"cpsat assignment ~7.5x slower after the 3qj adjacency: re-check §39.5's cpsat-vs-greedy verdict","description":"Declaring harbor's t -\u003e n adjacency (homemaker-py-3qj, DESIGN.md §38.14) made the CP-SAT room-labelling model markedly harder. constructive_topology seeding, 3-seed average:\\n\\n harbor greedy 0.06s -\u003e 0.06s (unchanged) cpsat 0.28s -\u003e 2.11s (7.5x)\\n maple greedy 0.03s -\u003e 0.03s (unchanged) cpsat 0.47s -\u003e 1.37s (2.9x)\\n\\nassign_solver defaults to greedy so ordinary runs pay nothing, and mutate_reassign/enable_reassign are opt-in too. But §39.5 concluded cpsat beats greedy on both programmes, and that was measured on a cheaper problem than the corpus now poses. The verdict needs re-checking on quality-per-second, not just quality.\\n\\nAlso worth checking whether cpsat.max_deterministic_time is now being hit, which would mean it is returning early rather than solving -- that would change the quality side of the comparison too, silently.","acceptance_criteria":"cpsat vs greedy re-measured on the current corpus for both solution quality AND wall time; §39.5's verdict either reconfirmed or restated; if max_deterministic_time is being hit, that is recorded and the limit reconsidered.","status":"open","priority":2,"issue_type":"task","owner":"noreply@anthropic.com","created_at":"2026-08-29T10:57:26Z","created_by":"Claude","updated_at":"2026-08-29T10:57:26Z","dependency_count":0,"dependent_count":0,"comment_count":0}
|
{"_type":"issue","id":"homemaker-py-vjd","title":"cpsat assignment ~7.5x slower after the 3qj adjacency: re-check §39.5's cpsat-vs-greedy verdict","description":"Declaring harbor's t -\u003e n adjacency (homemaker-py-3qj, DESIGN.md §38.14) made the CP-SAT room-labelling model markedly harder. constructive_topology seeding, 3-seed average:\\n\\n harbor greedy 0.06s -\u003e 0.06s (unchanged) cpsat 0.28s -\u003e 2.11s (7.5x)\\n maple greedy 0.03s -\u003e 0.03s (unchanged) cpsat 0.47s -\u003e 1.37s (2.9x)\\n\\nassign_solver defaults to greedy so ordinary runs pay nothing, and mutate_reassign/enable_reassign are opt-in too. But §39.5 concluded cpsat beats greedy on both programmes, and that was measured on a cheaper problem than the corpus now poses. The verdict needs re-checking on quality-per-second, not just quality.\\n\\nAlso worth checking whether cpsat.max_deterministic_time is now being hit, which would mean it is returning early rather than solving -- that would change the quality side of the comparison too, silently.","acceptance_criteria":"cpsat vs greedy re-measured on the current corpus for both solution quality AND wall time; §39.5's verdict either reconfirmed or restated; if max_deterministic_time is being hit, that is recorded and the limit reconsidered.","status":"open","priority":2,"issue_type":"task","owner":"noreply@anthropic.com","created_at":"2026-08-29T10:57:26Z","created_by":"Claude","updated_at":"2026-08-29T10:57:26Z","dependency_count":0,"dependent_count":0,"comment_count":0}
|
||||||
{"_type":"issue","id":"homemaker-py-9gj","title":"quality_uncrinkliness returns a flat hard 0.0, so the objective cannot rank two equally-buried layouts","description":"Narrowed remnant of homemaker-py-ssz after the owner's daylight ruling (DESIGN.md §38.11). A buried leaf usually IS a defect -- corridors and WCs included -- so scoring it badly is correct. The residual complaint is not that the value is low, it is that it is FLAT: quality_uncrinkliness returns exactly 0.0 for every zero-exposure leaf, and since evaluate_leaf multiplies factors into quality and process_storey accumulates value += quality * rate * area, two layouts that differ only in how badly buried their rooms are score identically.\n\nSo the objective gives the search no gradient to descend in precisely the region it most needs to escape. This is a search-mechanics problem, not a calibration one, and it should be judged on whether it helps the search escape -- NOT on fail counts, which by construction it will not move (the fails are real and should stay).\n\nNote the trap recorded in §38.9: an arm optimised under a modified objective must not be scored under the objective it modifies, and equally must not be scored under its own. For a pure gradient change that emits the same fail set, stock scoring IS valid -- that is the one case where the yardstick is sound.","acceptance_criteria":"A variant that keeps the fail set byte-identical to stock (every currently-failing leaf still fails) but is monotone in how buried a leaf is; A/B at fixed budget on harbor + maple with enough seeds to see past the one-seed variance that made the §38.8 n=3 result undecidable.","status":"open","priority":2,"issue_type":"bug","owner":"noreply@anthropic.com","created_at":"2026-08-28T23:14:04Z","created_by":"Claude","updated_at":"2026-08-28T23:14:04Z","dependency_count":0,"dependent_count":0,"comment_count":0}
|
{"_type":"issue","id":"homemaker-py-9gj","title":"quality_uncrinkliness returns a flat hard 0.0, so the objective cannot rank two equally-buried layouts","description":"Narrowed remnant of homemaker-py-ssz after the owner's daylight ruling (DESIGN.md §38.11). A buried leaf usually IS a defect -- corridors and WCs included -- so scoring it badly is correct. The residual complaint is not that the value is low, it is that it is FLAT: quality_uncrinkliness returns exactly 0.0 for every zero-exposure leaf, and since evaluate_leaf multiplies factors into quality and process_storey accumulates value += quality * rate * area, two layouts that differ only in how badly buried their rooms are score identically.\n\nSo the objective gives the search no gradient to descend in precisely the region it most needs to escape. This is a search-mechanics problem, not a calibration one, and it should be judged on whether it helps the search escape -- NOT on fail counts, which by construction it will not move (the fails are real and should stay).\n\nNote the trap recorded in §38.9: an arm optimised under a modified objective must not be scored under the objective it modifies, and equally must not be scored under its own. For a pure gradient change that emits the same fail set, stock scoring IS valid -- that is the one case where the yardstick is sound.","acceptance_criteria":"A variant that keeps the fail set byte-identical to stock (every currently-failing leaf still fails) but is monotone in how buried a leaf is; A/B at fixed budget on harbor + maple with enough seeds to see past the one-seed variance that made the §38.8 n=3 result undecidable.","status":"open","priority":2,"issue_type":"bug","owner":"noreply@anthropic.com","created_at":"2026-08-28T23:14:04Z","created_by":"Claude","updated_at":"2026-08-28T23:14:04Z","dependency_count":0,"dependent_count":0,"comment_count":0}
|
||||||
{"_type":"issue","id":"homemaker-py-7b7","title":"health-centre demands 131% of its plot: the programme is unsatisfiable as written","description":"`evolve._preflight` (homemaker-py-tdp, DESIGN.md 39.11) reports health-centre demanding 240 m2 of floor per storey on a 183 m2 plot -- 131%. It is 1 storey (n_storeys_required=1, storey_minimum=1), so there is nowhere for the overflow to go.\n\nThe consequence is visible in every evaluated layout: 100% of rooms come out undersized, uniformly, at a median 0.60x their declared target. No search can fix this -- quality_size is being asked for something the plot cannot supply, so the size gaussian is pinned near-zero across the whole design and contributes no useful gradient. Contrast harbor-house and maple-court, where the seeder hits targets at median 1.01x.\n\nThis is a corpus data problem, not an engine problem. Either the plot is too small or the programme's sizes/counts were estimated too generously (the CLAUDE.md note that all corpus targets were 'estimated many years ago and never changed' applies).\n\nOptions, roughly in order of honesty:\n 1. enlarge the plot in examples/health-centre/init.dom to ~1.4x\n 2. scale down size/count in patterns.config until demand fits with headroom\n 3. raise the storey count so the demand spreads over 2 levels\n 4. leave it and treat health-centre as a deliberate over-tight brief, documented as such\n\nWhichever is chosen, note it in DESIGN.md 39.11 so the pre-flight warning is not read as a bug in the check.","acceptance_criteria":"health-centre's demanded area per storey fits its plot with headroom, OR the programme is explicitly documented as a deliberately over-tight brief; `evolve._preflight` no longer reports it as an unexplained anomaly; median room area / (target * share) on a constructed seed is materially above 0.60x.","status":"closed","priority":2,"issue_type":"bug","owner":"noreply@anthropic.com","created_at":"2026-08-26T16:20:31Z","created_by":"Claude","updated_at":"2026-08-29T10:20:00Z","closed_at":"2026-08-29T10:20:00Z","close_reason":"Fixed per the owner's ruling: enlarge the plot, sized assuming a courtyard\n(DESIGN.md §38.13).\n\nThe plot is a quadrilateral in this engine, so the courtyard is interior space\nthe search carves out, not a hole in the site. Sizing therefore has to allow for\nthree things, only one of which the programme declares: rooms 240 m2, circulation\n~65 m2 (~27%, NOT in demand -- the search creates C leaves), courtyard ~36 m2.\n\nScaled about the polygon centroid by k = 1.4606, chosen so the INSET area (what\nleaves actually get, after wall_outer) lands on 400 m2. Centroid scaling keeps\nthe site's irregular shape and its one private edge -- same site, larger.\n\n plot 183.2 -\u003e 400.0 m2\n rooms / plot 131% -\u003e 60% (harbor's ratio exactly, and harbor works)\n daylit perim 41.5 -\u003e 61.3 m (49.4 needed)\n median room area / (target x share) 0.60x -\u003e 1.00x\n\nThat last figure is the one that matters: 0.60x with 100% of rooms undersized\nwas the geometric signature of the infeasibility, and it is now level with\nharbor/maple at 1.01x.\n\nThe courtyard remains REQUIRED by the geometry rather than merely permitted: a\nroom is at most 1.6202*h = 4.86 m deep before failing crinkliness, so a ~20x20 m\nplot with a daylit ring leaves a 10.3 x 10.3 m, 106 m2 core that cannot reach an\nexternal wall. Enlarging did not remove the pressure that produces a courtyard,\nit made room for one.\n\nevolve._preflight is now silent on health-centre. Unblocks homemaker-py-7xb.\n","dependency_count":0,"dependent_count":1,"comment_count":0}
|
{"_type":"issue","id":"homemaker-py-7b7","title":"health-centre demands 131% of its plot: the programme is unsatisfiable as written","description":"`evolve._preflight` (homemaker-py-tdp, DESIGN.md 39.11) reports health-centre demanding 240 m2 of floor per storey on a 183 m2 plot -- 131%. It is 1 storey (n_storeys_required=1, storey_minimum=1), so there is nowhere for the overflow to go.\n\nThe consequence is visible in every evaluated layout: 100% of rooms come out undersized, uniformly, at a median 0.60x their declared target. No search can fix this -- quality_size is being asked for something the plot cannot supply, so the size gaussian is pinned near-zero across the whole design and contributes no useful gradient. Contrast harbor-house and maple-court, where the seeder hits targets at median 1.01x.\n\nThis is a corpus data problem, not an engine problem. Either the plot is too small or the programme's sizes/counts were estimated too generously (the CLAUDE.md note that all corpus targets were 'estimated many years ago and never changed' applies).\n\nOptions, roughly in order of honesty:\n 1. enlarge the plot in examples/health-centre/init.dom to ~1.4x\n 2. scale down size/count in patterns.config until demand fits with headroom\n 3. raise the storey count so the demand spreads over 2 levels\n 4. leave it and treat health-centre as a deliberate over-tight brief, documented as such\n\nWhichever is chosen, note it in DESIGN.md 39.11 so the pre-flight warning is not read as a bug in the check.","acceptance_criteria":"health-centre's demanded area per storey fits its plot with headroom, OR the programme is explicitly documented as a deliberately over-tight brief; `evolve._preflight` no longer reports it as an unexplained anomaly; median room area / (target * share) on a constructed seed is materially above 0.60x.","status":"closed","priority":2,"issue_type":"bug","owner":"noreply@anthropic.com","created_at":"2026-08-26T16:20:31Z","created_by":"Claude","updated_at":"2026-08-29T10:20:00Z","closed_at":"2026-08-29T10:20:00Z","close_reason":"Fixed per the owner's ruling: enlarge the plot, sized assuming a courtyard\n(DESIGN.md §38.13).\n\nThe plot is a quadrilateral in this engine, so the courtyard is interior space\nthe search carves out, not a hole in the site. Sizing therefore has to allow for\nthree things, only one of which the programme declares: rooms 240 m2, circulation\n~65 m2 (~27%, NOT in demand -- the search creates C leaves), courtyard ~36 m2.\n\nScaled about the polygon centroid by k = 1.4606, chosen so the INSET area (what\nleaves actually get, after wall_outer) lands on 400 m2. Centroid scaling keeps\nthe site's irregular shape and its one private edge -- same site, larger.\n\n plot 183.2 -\u003e 400.0 m2\n rooms / plot 131% -\u003e 60% (harbor's ratio exactly, and harbor works)\n daylit perim 41.5 -\u003e 61.3 m (49.4 needed)\n median room area / (target x share) 0.60x -\u003e 1.00x\n\nThat last figure is the one that matters: 0.60x with 100% of rooms undersized\nwas the geometric signature of the infeasibility, and it is now level with\nharbor/maple at 1.01x.\n\nThe courtyard remains REQUIRED by the geometry rather than merely permitted: a\nroom is at most 1.6202*h = 4.86 m deep before failing crinkliness, so a ~20x20 m\nplot with a daylit ring leaves a 10.3 x 10.3 m, 106 m2 core that cannot reach an\nexternal wall. Enlarging did not remove the pressure that produces a courtyard,\nit made room for one.\n\nevolve._preflight is now silent on health-centre. Unblocks homemaker-py-7xb.\n","dependency_count":0,"dependent_count":1,"comment_count":0}
|
||||||
|
|
|
||||||
48
DESIGN.md
48
DESIGN.md
|
|
@ -2768,6 +2768,11 @@ path):
|
||||||
- Cost: ON still ~1.2–1.3× OFF wall-clock at this size (20.9s mean OFF → 26.4s mean ON), same order
|
- Cost: ON still ~1.2–1.3× OFF wall-clock at this size (20.9s mean OFF → 26.4s mean ON), same order
|
||||||
as the original measurement.
|
as the original measurement.
|
||||||
|
|
||||||
|
**Re-validated under the current objective (`homemaker-py-ioe`, §38.19):** the default still
|
||||||
|
stands, but the margin is about a third smaller (+0.57 fails/seed against the +0.85 below) and is
|
||||||
|
**no longer detectable at this section's N=20** (p ≈ 0.069 there); it takes N=60 to reach
|
||||||
|
p = 0.017. Do not re-check this default at N=20.
|
||||||
|
|
||||||
Confirms the qpk verdict holds at both example scales tested. `collapse_insearch` default flipped
|
Confirms the qpk verdict holds at both example scales tested. `collapse_insearch` default flipped
|
||||||
**OFF → ON** in `evolve.py` (`--collapse-insearch`/`--no-collapse-insearch`,
|
**OFF → ON** in `evolve.py` (`--collapse-insearch`/`--no-collapse-insearch`,
|
||||||
`HOMEMAKER_COLLAPSE_INSEARCH`) and `driver.py` (`_overrides_for`, `_fitness_for`, `_evaluate`,
|
`HOMEMAKER_COLLAPSE_INSEARCH`) and `driver.py` (`_overrides_for`, `_fitness_for`, `_evaluate`,
|
||||||
|
|
@ -5838,6 +5843,49 @@ measured. Worth noting for future archaeology: "was this measurement affected by
|
||||||
bug X" is often answerable from the programme's structure without re-running
|
bug X" is often answerable from the programme's structure without re-running
|
||||||
anything.
|
anything.
|
||||||
|
|
||||||
|
### 38.19 `collapse_insearch=True` re-validated under the current objective (`homemaker-py-ioe`)
|
||||||
|
|
||||||
|
§38.18 confirmed the `1ph` default-flip was sound *for its own era*. But that
|
||||||
|
measurement predates three changes to the objective it was measured against —
|
||||||
|
§39.4's generic-namespace fix, §38.10/§38.11's per-space crinkliness, and
|
||||||
|
§38.12's missing-space cascade — and `collapse_insearch` runs `collapse_global`
|
||||||
|
inside every eval, valued against exactly the quality factors those changes
|
||||||
|
touched. A default carried on a superseded measurement is an assumption, not a
|
||||||
|
result, so the `1ph` protocol was re-run as published on the current codebase:
|
||||||
|
programme-house, budget 3000, 4 workers, ON vs OFF, both arms finished with
|
||||||
|
`--collapse` (`experiments/rerun_1ph_protocol.sh`).
|
||||||
|
|
||||||
|
| | N | OFF | ON | W/L/T | mean diff | t | p |
|
||||||
|
|---|---|---|---|---|---|---|---|
|
||||||
|
| published `1ph` (2026-07-24) | 20 | 7.95 | 7.10 | 11/6/3 | +0.85 | 2.38 | 0.028 |
|
||||||
|
| historical re-run (§38.18) | 20 | 8.05 | 7.10 | 11/6/3 | +0.95 | 2.59 | — |
|
||||||
|
| **current objective** | 20 | 7.85 | 7.15 | 10/7/3 | +0.70 | 1.82 | **0.069** |
|
||||||
|
| current objective | 40 | 7.60 | 7.03 | 21/14/5 | +0.57 | 2.01 | 0.045 |
|
||||||
|
| **current objective** | **60** | **7.58** | **7.02** | **29/19/12** | **+0.57** | **2.45** | **0.017** |
|
||||||
|
|
||||||
|
**Verdict: the default stands.** At N=60, mean diff **+0.567 fails/seed**,
|
||||||
|
paired t = 2.454 (df=59), **p = 0.0171** exact, 95% CI **[+0.105, +1.029]**
|
||||||
|
excluding zero. A Wilcoxon signed-rank cross-check agrees (p = 0.0138), which
|
||||||
|
matters here because fail counts are small integers and normality is not
|
||||||
|
obvious.
|
||||||
|
|
||||||
|
**Two things worth recording beyond the verdict.**
|
||||||
|
|
||||||
|
First, **the effect is about a third smaller than published** — +0.57 against
|
||||||
|
+0.85. Some of that is regression from a slightly lucky N=20 draw, and some is
|
||||||
|
plausibly real erosion: several of the fails `collapse_global` used to clear
|
||||||
|
have been redefined out of existence or made harder by the objective work.
|
||||||
|
|
||||||
|
Second, and more usefully: **the published protocol's N=20 can no longer detect
|
||||||
|
its own effect.** At exactly the published sample size the current answer is
|
||||||
|
p ≈ 0.069 — a null by the conventional threshold. Had this been re-run at N=20
|
||||||
|
and stopped there, the honest report would have been "the 1ph verdict no longer
|
||||||
|
reproduces", and the default would have looked unjustified. It took N=60 to
|
||||||
|
resolve. That is precisely the "8sh/1ph/qi6/lj3 pattern" this document already
|
||||||
|
warns about, now biting the flagship result itself: **any future re-validation
|
||||||
|
of this default needs N ≥ 40, and N=20 should not be trusted to settle it either
|
||||||
|
way.**
|
||||||
|
|
||||||
## 39. Config audit: requirements that actively fight the engine (`homemaker-py-ju3`) — measured 2026-08-25
|
## 39. Config audit: requirements that actively fight the engine (`homemaker-py-ju3`) — measured 2026-08-25
|
||||||
|
|
||||||
The corpus `patterns.config` targets and `costs.config` values were estimated
|
The corpus `patterns.config` targets and `costs.config` values were estimated
|
||||||
|
|
|
||||||
|
|
@ -1,11 +1,12 @@
|
||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# Re-run the 1ph protocol (DESIGN.md §20): programme-house init.dom, budget 3000,
|
# Re-run the 1ph protocol (DESIGN.md §20): programme-house init.dom, budget 3000,
|
||||||
# 4 workers, seeds 1-20, ON vs OFF, BOTH arms finished with --collapse.
|
# 4 workers, ON vs OFF, BOTH arms finished with --collapse.
|
||||||
|
# Usage: rerun_1ph_protocol.sh <worktree> <tag> <out.tsv> [lo] [hi] (APPEND=1 to add)
|
||||||
set -u
|
set -u
|
||||||
W="$1"; TAG="$2"; OUT="$3"
|
W="$1"; TAG="$2"; OUT="$3"; LO="${4:-1}"; HI="${5:-20}"
|
||||||
cd "$W/examples/programme-house"
|
cd "$W/examples/programme-house"
|
||||||
: > "$OUT"
|
[ "${APPEND:-0}" = "1" ] || : > "$OUT"
|
||||||
for seed in $(seq 1 20); do
|
for seed in $(seq "$LO" "$HI"); do
|
||||||
for arm in on off; do
|
for arm in on off; do
|
||||||
flag=""; [ "$arm" = "off" ] && flag="--no-collapse-insearch"
|
flag=""; [ "$arm" = "off" ] && flag="--no-collapse-insearch"
|
||||||
t0=$(date +%s)
|
t0=$(date +%s)
|
||||||
|
|
|
||||||
120
experiments/results/ioe_1ph_current_objective.tsv
Normal file
120
experiments/results/ioe_1ph_current_objective.tsv
Normal file
|
|
@ -0,0 +1,120 @@
|
||||||
|
current 1 on 5 9
|
||||||
|
current 1 off 8 6
|
||||||
|
current 2 on 5 8
|
||||||
|
current 2 off 8 5
|
||||||
|
current 3 on 9 7
|
||||||
|
current 3 off 8 6
|
||||||
|
current 4 on 9 8
|
||||||
|
current 4 off 10 6
|
||||||
|
current 5 on 9 8
|
||||||
|
current 5 off 8 6
|
||||||
|
current 6 on 8 8
|
||||||
|
current 6 off 11 6
|
||||||
|
current 7 on 8 7
|
||||||
|
current 7 off 9 6
|
||||||
|
current 8 on 8 8
|
||||||
|
current 8 off 7 7
|
||||||
|
current 9 on 6 7
|
||||||
|
current 9 off 9 6
|
||||||
|
current 10 on 4 8
|
||||||
|
current 10 off 7 6
|
||||||
|
current 11 on 5 7
|
||||||
|
current 11 off 5 6
|
||||||
|
current 12 on 7 9
|
||||||
|
current 12 off 6 6
|
||||||
|
current 13 on 6 8
|
||||||
|
current 13 off 8 6
|
||||||
|
current 14 on 8 8
|
||||||
|
current 14 off 7 6
|
||||||
|
current 15 on 6 7
|
||||||
|
current 15 off 8 6
|
||||||
|
current 16 on 8 7
|
||||||
|
current 16 off 8 6
|
||||||
|
current 17 on 9 8
|
||||||
|
current 17 off 7 6
|
||||||
|
current 18 on 7 7
|
||||||
|
current 18 off 8 6
|
||||||
|
current 19 on 9 7
|
||||||
|
current 19 off 8 5
|
||||||
|
current 20 on 7 8
|
||||||
|
current 20 off 7 6
|
||||||
|
current 21 on 5 8
|
||||||
|
current 21 off 5 7
|
||||||
|
current 22 on 8 8
|
||||||
|
current 22 off 9 6
|
||||||
|
current 23 on 8 8
|
||||||
|
current 23 off 9 6
|
||||||
|
current 24 on 7 8
|
||||||
|
current 24 off 8 6
|
||||||
|
current 25 on 7 9
|
||||||
|
current 25 off 9 6
|
||||||
|
current 26 on 8 8
|
||||||
|
current 26 off 10 6
|
||||||
|
current 27 on 9 8
|
||||||
|
current 27 off 10 6
|
||||||
|
current 28 on 5 8
|
||||||
|
current 28 off 8 6
|
||||||
|
current 29 on 6 8
|
||||||
|
current 29 off 8 6
|
||||||
|
current 30 on 5 8
|
||||||
|
current 30 off 7 6
|
||||||
|
current 31 on 10 8
|
||||||
|
current 31 off 9 6
|
||||||
|
current 32 on 10 8
|
||||||
|
current 32 off 7 6
|
||||||
|
current 33 on 3 8
|
||||||
|
current 33 off 7 6
|
||||||
|
current 34 on 6 8
|
||||||
|
current 34 off 8 6
|
||||||
|
current 35 on 8 8
|
||||||
|
current 35 off 5 6
|
||||||
|
current 36 on 8 8
|
||||||
|
current 36 off 7 6
|
||||||
|
current 37 on 5 7
|
||||||
|
current 37 off 4 5
|
||||||
|
current 38 on 5 8
|
||||||
|
current 38 off 5 6
|
||||||
|
current 39 on 8 8
|
||||||
|
current 39 off 7 7
|
||||||
|
current 40 on 7 7
|
||||||
|
current 40 off 5 6
|
||||||
|
current 41 on 10 8
|
||||||
|
current 41 off 7 6
|
||||||
|
current 42 on 8 7
|
||||||
|
current 42 off 8 5
|
||||||
|
current 43 on 7 7
|
||||||
|
current 43 off 8 6
|
||||||
|
current 44 on 7 8
|
||||||
|
current 44 off 7 6
|
||||||
|
current 45 on 6 7
|
||||||
|
current 45 off 5 5
|
||||||
|
current 46 on 8 7
|
||||||
|
current 46 off 7 6
|
||||||
|
current 47 on 6 9
|
||||||
|
current 47 off 9 6
|
||||||
|
current 48 on 8 8
|
||||||
|
current 48 off 10 6
|
||||||
|
current 49 on 8 8
|
||||||
|
current 49 off 10 6
|
||||||
|
current 50 on 7 7
|
||||||
|
current 50 off 7 6
|
||||||
|
current 51 on 7 8
|
||||||
|
current 51 off 7 6
|
||||||
|
current 52 on 7 8
|
||||||
|
current 52 off 9 6
|
||||||
|
current 53 on 10 8
|
||||||
|
current 53 off 8 6
|
||||||
|
current 54 on 5 7
|
||||||
|
current 54 off 9 6
|
||||||
|
current 55 on 7 8
|
||||||
|
current 55 off 9 6
|
||||||
|
current 56 on 7 7
|
||||||
|
current 56 off 7 6
|
||||||
|
current 57 on 2 8
|
||||||
|
current 57 off 5 6
|
||||||
|
current 58 on 7 7
|
||||||
|
current 58 off 7 6
|
||||||
|
current 59 on 8 7
|
||||||
|
current 59 off 7 6
|
||||||
|
current 60 on 5 8
|
||||||
|
current 60 off 5 6
|
||||||
|
Loading…
Add table
Reference in a new issue