bruno/homemaker-layout
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

DESIGN.md sec 37.7: update CP-SAT assignment writeup with final A/B result

Browse source 
Full harbor+maple/3-seed/20k-budget run superseded the earlier inconclusive
pilot table; section now records the closed verdict (no clear win, both
flags stay default off) instead of "driver-level INCONCLUSIVE at pilot
scale".

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LSwQwpEaHFBkeVSDDWd75S
This commit is contained in:
Bruno Postle 2026-08-05 00:06:03 +01:00
parent eb1248d3a5
commit 9692c912ed
1 changed files with 35 additions and 35 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

70
DESIGN.md
View file

@ -4582,7 +4582,7 @@ test renamed and inverted (`test_shapecurve_warmstart_handles_multistorey`
now asserts `shapecurve.solve` **is** called on a multi-storey child, where now asserts `shapecurve.solve` **is** called on a multi-storey child, where
it previously asserted the opposite). Full suite: 397 passed. it previously asserted the opposite). Full suite: 397 passed.
## 37.7 CP-SAT type assignment for a fixed tree (`homemaker-py-2g7.5`) — PARTIAL, seeder-level positive, driver-level INCONCLUSIVE at pilot scale ## 37.7 CP-SAT type assignment for a fixed tree (`homemaker-py-2g7.5`) — CLOSED: seeder-level positive in isolation, does NOT survive a full driver.search run; both flags stay default off
`§11.6`/`§11.7`'s greedy connected-dominating-set + hardest-constrained- `§11.6`/`§11.7`'s greedy connected-dominating-set + hardest-constrained-
code-first room placement (`operators._assign_adjacency_aware`) was Phase code-first room placement (`operators._assign_adjacency_aware`) was Phase
@ -4664,43 +4664,43 @@ proxy for what `homemaker-fitness` actually scores. Wired in as:
coefficient range — reverted in favour of the explicit grouping coefficient range — reverted in favour of the explicit grouping
constraint above. constraint above.
**`driver.search`-level A/B: INCONCLUSIVE at pilot scale, NOT the bead's own **`driver.search`-level A/B: RUN TO COMPLETION at the bead's own acceptance
20k-budget/harbor+maple/3-seed acceptance protocol.** Wall-clock budget for protocol (harbor+maple, 3 seeds, budget=20000) — result does NOT clear the
this session did not stretch to the bead's own acceptance criteria (~28min bar.** An earlier pilot (harbor-house only, budget=3000) was inconclusive
per arm at budget=20000 on real harbor-house, ×3 arms ×3 seeds ×2 because `reassign` never fired in the small child-count that budget
programmes ≈ 8+ hours). `experiments/ab_cpsat_assign.py`, harbor-house only, produces. The full run (`experiments/ab_cpsat_assign.py 20000 3
budget=3000, 3 seeds (greedy / cpsat-seed-only / cpsat+`enable_reassign`): <programme>`, ~12h wall clock, 18 `driver.search` runs total; raw log
`experiments/results/ab_cpsat_assign_20k_harbor_maple.log`) settles it:
| arm | mean hard | mean soft | mean fitness | mean wall | | programme | greedy hard/soft | cpsat hard/soft | reassign hard/soft |
|---|---|---|---|---| |---|---|---|---|
| greedy | 21.000 | 39.000 | 9.308e-19 | 237.2s | | harbor-house | 9.3 / 35.3 | 13.0 / 32.0 | 8.3 / 35.3 |
| cpsat | 24.000 | 36.667 | 5.313e-18 | 245.6s | | maple-court | 22.7 / 72.3 | 22.3 / 68.3 | 35.3 / 77.7 |
| reassign | 19.667 | 44.667 | 3.930e-19 | 240.0s |
Mixed: `cpsat` is worse on mean HARD fails but better on soft fails and `reassign_fired` (mean over 3 seeds): harbor-house 0.0, maple-court 0.3 —
~5.7x the mean fitness; `reassign` has the best mean hard fails but the i.e. it fired in only 1 of 18 runs total (1 of 6 `reassign`-arm runs), even
worst soft fails. The `reassign` arm's own mechanism was **never observed at 20k budget. None of the bead's three acceptance criteria hold: `cpsat`
to fire** in any of the 3 seeds (`mean_reassign_fired=0.0`) — at is WORSE than greedy on harbor-house hard fails (13.0 vs 9.3) and only
budget=3000 the loop only generates ~15-20 children total, and `reassign` roughly tied on maple-court (22.3 vs 22.7) — no consistent "strictly lower"
carries the implicit uniform mutation weight (no boost was added — DESIGN.md win; `reassign` is worse than greedy end-to-end on maple-court (35.3 vs
§22's `bridge_circulation` precedent measured that an un-A/B-tested weight 22.7 hard); and the operator does not reliably fire even once per run.
boost can backfire, so none was assumed here without evidence) — so the Likely explanation: CP-SAT's exact optimum at seed geometry (or after
`reassign` arm's difference from `cpsat` at this budget is attributable to `_cpsat_relabel_settled`) doesn't stay optimal once the inner loop keeps
RNG/exploration noise from the changed weight-normalisation denominator, moving ratios across thousands of further evals — the bead's own §11.2
not to the operator's own effect. `mutate_reassign` firing-and-being- lesson, but the "re-run after geometry settles" fix only re-solves once,
accepted IS independently confirmed at the operator level not continuously, and any seeder-level gain gets swamped by ordinary search
(`tests/test_operators.py::test_reassign_fires_and_preserves_room_multiset`, noise over a 20k-eval run. Not pursuing a higher budget or more seeds to
20/20 direct trials on a real seeded harbor-house design) — the pilot budget chase this further — the direction (no clear win, one programme regresses)
was simply too small to give it enough draws in the full search loop. is consistent enough between the pilot and the full run to close it out.
**Verdict: ship as opt-in EXPERIMENTAL, both default off** (matching every **Verdict: ship as opt-in EXPERIMENTAL, both default off, and leave it
other flag in this codebase) — the seeder-level win (item (a)) is real and there** (matching every other flag in this codebase) — the seeder-level win
measured with low noise; the full-search-level payoff (matching the bead's in isolation (item (a), measured via
own acceptance criteria) is unconfirmed at pilot scale and needs a proper `test_assign_cpsat_matches_or_beats_greedy_secondary_adjacency`) is real and
larger-budget/larger-N run, tracked as follow-up work on `2g7.5` itself low-noise, but does not survive contact with a full `driver.search` run
(left `in_progress`, not closed — same pattern `6xh` used when its own across two programmes at the bead's own acceptance budget. `2g7.5` is
acceptance bar wasn't fully met). `homemaker-py-5bv` (child of `2g7`/`2g7.5`) CLOSED on this basis. `homemaker-py-5bv` (child of `2g7`/`2g7.5`) remains
tracks the deferred item (c). open, tracking the deferred item (c).
**Verification.** `tests/test_cpsat.py` (5 tests): a hand-built **Verification.** `tests/test_cpsat.py` (5 tests): a hand-built
counter-example graph (hub + one non-hub edge) where the beam/greedy counter-example graph (hub + one non-hub edge) where the beam/greedy