7fm: targeted shape-repair operators (shape_rotate/deslim), negative finish-time result

Diagnosed the geometry-intrinsic residual from 94g's collapse: ratio
re-optimisation isn't the bottleneck (1500-eval NM makes zero difference on
the 12-fail collapsed best layout); the causes are upstream area starvation
and cut-orientation mismatch. Added mutate_shape_rotate/mutate_deslim
targeting each, gated on a Fitness instance like the existing reqs-gated
repair ops.

Evaluated as a finish-time exhaustive hill-climb on the same 6-layout
harbor-house sweep 94g used: zero improving moves found anywhere — every
candidate move traded the shape fail for a new adjacency/access fail on the
co-evolved layout (§4.2's lesson, now confirmed for topology repair). Closes
homemaker-py-7fm; spun homemaker-py-161 for the open in-search-GA question.

See DESIGN.md §19 for the full writeup.

DESIGN.md

@@ -2465,3 +2465,73 @@ toward connected circulation and clear `not connected` fails — needs full-budg
 pending (short 60-eval smoke run confirms the plumbing only). If the graded key alone is
 insufficient, the follow-on is an insert/relocate-circulation mutation operator (mechanism (a),
 still `homemaker-py-qi6`) that now has a gradient to climb. 276 tests pass.
+
+## 19. Geometry/topology repair for shape-intrinsic fails (`homemaker-py-7fm`) — DONE (negative)
+
+**Motivation.** §17 established that ~12 of the harbor-house best layout's 15 residual fails
+survive the label-only collapse — long-thin cells (`width`/`proportion`/`crinkliness`) whose
+geometry, not room assignment, is wrong. `bd memory collapse-global-94g-and-any-label-usage-
+optimisation` spun this out as its own problem: a mechanism that moves *geometry*, evaluated for
+net fail-count effect on the same 6-layout sweep §17 used.
+
+**Diagnosis (rules out mechanism (a)).** Re-ran the full-fitness ratio inner loop
+(`innerloop.optimise`, Nelder-Mead, 1500 evals, warm-started from the evolved ratios — far above
+the ~80-200/child budget search actually spends) on the 12-fail collapsed best layout: **zero
+change**, byte-identical fail lines. These are not local optima of the ratio search reachable
+with more budget. Tracing two representative fails back through the tree found two distinct
+structural causes, neither fixable by re-solving ratios on the existing cuts: (1) **area
+starvation** — a leaf's *defining branch* (several levels up) was allocated too little total
+area for what it has to share with its siblings (a storage leaf wanting 18m² sat in a 6.4m²
+branch whose sibling got 52.8m² of outside space); (2) **orientation mismatch** — a leaf is the
+correctly-area-sized-but-thin remainder of a cut whose *rotation* runs parallel to its parent
+rectangle's long axis, so no ratio value on that axis avoids a sliver.
+
+**Mechanism (`operators.mutate_shape_rotate`, `operators.mutate_deslim`).** Two targeted repair
+operators addressing each cause, in the `mutate_level_fix` style (structural, not blind-random):
+`_shape_failing(leaf, fit)` identifies a named-room leaf whose width or proportion factor
+actually fails (`< FAIL_THRESHOLD` under `Fitness.quality_width`/`quality_proportion` — not a
+geometric proxy, which over-flags leaves the Gaussian tail still passes). `mutate_shape_rotate`
+re-orients the live cut that produced a failing leaf (targets cause 2); `mutate_deslim` merges a
+failing leaf into its sibling, undoing the division that starved it (targets cause 1), leaving
+the displaced room for `mutate_place_missing` (already in `MUTATIONS`) to re-insert elsewhere.
+Both are registered in `MUTATIONS`/`mutate()`, gated on a `fit` argument (a new `fit_ops` class
+alongside the existing `reqs_ops`) so they no-op — and are excluded from the outer search's
+`weights` — wherever a `Fitness` instance isn't threaded through, exactly as `place_missing` etc.
+gate on `reqs`. `driver.search`/`evolve.py` do **not** yet pass `fit` through (see Status below),
+so the operators exist but are currently unreachable from the GA — they were evaluated instead
+as a finish-time greedy hill-climb (below).
+
+**Verification (measured, negative).** A finish-time hill-climb applied both operators
+exhaustively — for every live cut driving a shape fail, all 3 alternate rotations were tried
+(not just `mutate_shape_rotate`'s single random draw) alongside a `deslim` + `place_missing` +
+ratio-resolve, keeping the best only if it did not increase the fail count — on the same 6
+harbor-house evolved layouts as §17 (total fails 187): **0 improving moves found on any layout,
+on any candidate cut, under any of the 4 tried variants.** Manually inspecting the rejected
+candidates for the representative case (harbor-house evolved-3M-nols-3, leaf `0/rlrlr` "la1",
+the proportion fail traced above) shows why: every one of the 3 rotations and the deslim+
+reinsert produced a **worse** layout — new `no outside public access`, `not adjacent to c`,
+`access`, or `edge too long` fails, in every trial. This is §4.2's core lesson (proxy/partial-
+objective repair of a co-evolved local optimum "is structurally unable to win" — every cut
+position is *simultaneously* a size/shape knob **and** an adjacency/access/circulation knob) now
+confirmed for structural topology repair, not just ratio-solving: on a tightly co-evolved
+layout, the cut that makes a leaf thin is *also* the cut providing some other leaf's public-
+access or adjacency, so straightening it elsewhere is not free. The residual geometry-intrinsic
+fails on the harbor-house best layout appear to be close to a genuine Pareto floor for this
+topology, not a repairable inefficiency — consistent with §17's own framing ("geometry-/
+building-bound").
+
+**Status / next.** `mutate_shape_rotate`/`mutate_deslim` land in `operators.py`, default-excluded
+from `mutate()` (no `fit` threaded through the outer search yet), with dedicated tests
+(`tests/test_operators.py`: fail detection, noop-without-`fit`, targeted-cut selection, merge +
+`place_missing` repairability) plus automatic coverage via the existing
+`test_mutations_yield_canonical_genomes` parametrisation. 282 tests pass. The finish-time
+hill-climb script is **not** productionised (unlike §17's `collapse_cmd.py`) because it never
+found an improving move to apply — there is nothing to wire up. Not tested: whether these
+operators help as *in-search* GA moves (mechanism (c)) — a full multi-generation run gives
+selection pressure and population diversity a chance to accept a locally-worse move that a later
+step or recombination completes, a fundamentally different regime from single-step greedy
+hill-climbing on an already-finished layout. That A/B (thread `fit` through `driver.search`,
+gate with an `enable_shape_repair`-style flag as §12.3 did for `reassociate`, run full-budget
+with/without) is the remaining open question and would need to be its own measured experiment
+before further code changes — this session's finding is that the *finish-time* half of the
+issue's candidate mechanisms is a dead end, not that geometry repair is impossible in general.

src/homemaker_layout/operators.py

@@ -373,6 +373,94 @@ def mutate_place_missing(root: dom.Node, rng: np.random.Generator,
     return _finalise(child), f"place_missing {code} -> {host_id}"
 
 
+def _shape_failing(leaf: dom.Node, fit) -> bool:
+    """A named-room leaf whose width or proportion factor actually fails
+    (``< fitness.FAIL_THRESHOLD``) under ``fit``, the same Gaussian quality
+    functions the scorer uses (``Fitness.quality_width``/``quality_proportion``)
+    — not a geometric proxy, which over-flags leaves the gaussian tail still
+    passes. Generic circulation/outside/sahn leaves are never candidates —
+    they absorb slack by design (solver.py ``min_width_generic``), not a
+    repair target."""
+    if not leaf.type or leaf.type[0].lower() in "cos":
+        return False
+    from . import fitness as _fit_mod
+
+    return (fit.quality_width(leaf) < _fit_mod.FAIL_THRESHOLD
+            or fit.quality_proportion(leaf) < _fit_mod.FAIL_THRESHOLD)
+
+
+def mutate_shape_rotate(root: dom.Node, rng: np.random.Generator,
+                        types: list[str], fit=None) -> tuple[dom.Node, str]:
+    """Repair operator (homemaker-py-7fm): re-orient the cut that produced a
+    shape-failing (long-thin) leaf.
+
+    Diagnosis (bd memory, 7fm): re-running the full-fitness ratio inner loop
+    with a large budget does not clear these fails — they are not local optima
+    of the ratio, because the offending leaf is the *thin* side of a cut whose
+    orientation runs parallel to its parent rectangle's long axis, so any ratio
+    value on that axis yields a thin sliver. Rotating the defining (live) cut
+    changes which axis the ratio divides; the inner loop then re-tunes the
+    ratio on the new axis. Targets only the cut that actually produced a
+    failing leaf, unlike the untargeted ``mutate_rotate``. Requires ``fit``
+    (a ``fitness.Fitness``) to identify genuinely failing leaves.
+    """
+    if fit is None:
+        return _finalise(copy.deepcopy(root)), "shape_rotate noop"
+    child = copy.deepcopy(root)
+    cands: list[tuple[int, dom.Node, dom.Node]] = []
+    for li, n in _owned_branches(child):
+        if n.below is not None:
+            continue
+        for side in ("l", "r"):
+            leaf = n.left if side == "l" else n.right
+            if not leaf.divided and _shape_failing(leaf, fit):
+                cands.append((li, n, leaf))
+    if not cands:
+        return _finalise(child), "shape_rotate noop"
+    li, n, leaf = _pick(rng, cands)
+    n.rotation = (n.rotation + int(rng.integers(1, 4))) % 4
+    return _finalise(child), f"shape_rotate {li}/{n.id or 'root'} (fixing {leaf.id})"
+
+
+def mutate_deslim(root: dom.Node, rng: np.random.Generator,
+                  types: list[str], fit=None) -> tuple[dom.Node, str]:
+    """Repair operator (homemaker-py-7fm): merge a shape-failing (long-thin)
+    leaf into its sibling, undoing the division that starved it.
+
+    Unlike ``mutate_shape_rotate`` this addresses cuts whose *area* share is
+    wrong (an upstream branch several levels up gave the whole subtree too
+    little area to satisfy every leaf inside it — no ratio or rotation on the
+    local cut can fix that, bd memory 7fm), not just its orientation. The
+    displaced room becomes a missing-space fail that ``mutate_place_missing``
+    (already in ``MUTATIONS``) re-inserts elsewhere on a later step. Requires
+    ``fit`` (a ``fitness.Fitness``) to identify genuinely failing leaves.
+    """
+    if fit is None:
+        return _finalise(copy.deepcopy(root)), "deslim noop"
+    from . import geometry as _geo
+
+    child = copy.deepcopy(root)
+    cands = [
+        (li, n) for li, n in _owned_branches(child)
+        if not n.left.divided and not n.right.divided
+        and (_shape_failing(n.left, fit) or _shape_failing(n.right, fit))
+    ]
+    if not cands:
+        return _finalise(child), "deslim noop"
+    li, n = _pick(rng, cands)
+    l_fail, r_fail = _shape_failing(n.left, fit), _shape_failing(n.right, fit)
+    if l_fail and not r_fail:
+        survivor = n.right
+    elif r_fail and not l_fail:
+        survivor = n.left
+    else:
+        survivor = max((n.left, n.right), key=_geo.area)
+    n.type = survivor.type if survivor.type and survivor.type[0].lower() not in "cos" else "C"
+    n.division = None
+    n.left = n.right = None
+    return _finalise(child), f"deslim {li}/{n.id or 'root'} (kept {n.type})"
+
+
 def _leaves_with_depth(n: dom.Node, d: int = 0) -> list[tuple[dom.Node, int]]:
     """Every leaf under ``n`` paired with its depth below ``n``."""
     if not n.divided:
@@ -1239,6 +1327,8 @@ MUTATIONS = {
     "level_retype": mutate_level_retype,
     "level_add": mutate_level_add,
     "level_delete": mutate_level_delete,
+    "shape_rotate": mutate_shape_rotate,
+    "deslim": mutate_deslim,
 }
 
 
@@ -1251,20 +1341,27 @@ _BASE_P_OPS = ("divide", "undivide", "retype", "swap", "rotate")
 
 def mutate(root: dom.Node, rng: np.random.Generator, types: list[str],
            weights: dict[str, float] | None = None,
-           reqs=None, base_p: float = 1.0) -> tuple[dom.Node, str]:
+           reqs=None, base_p: float = 1.0, fit=None) -> tuple[dom.Node, str]:
     """Apply one random mutation drawn from MUTATIONS."""
     names = sorted(MUTATIONS)
     p = np.array([(weights or {}).get(n, 1.0) for n in names], dtype=float)
     # these operators need programme reqs; disable them when not available
     reqs_ops = ("level_fix", "level_compound_fix", "place_missing")
+    # these need a Fitness instance to identify genuinely shape-failing leaves
+    fit_ops = ("shape_rotate", "deslim")
     if reqs is None:
         for op in reqs_ops:
             p[names.index(op)] = 0.0
+    if fit is None:
+        for op in fit_ops:
+            p[names.index(op)] = 0.0
     if p.sum() == 0:
         p[:] = 1.0
     name = str(rng.choice(names, p=p / p.sum()))
     if name in reqs_ops:
         return MUTATIONS[name](root, rng, types, reqs=reqs)
+    if name in fit_ops:
+        return MUTATIONS[name](root, rng, types, fit=fit)
     if name in _BASE_P_OPS:
         return MUTATIONS[name](root, rng, types, base_p=base_p)
     return MUTATIONS[name](root, rng, types)

tests/test_operators.py

@@ -1,5 +1,6 @@
 """Operator tests (oracle-free): every child is a valid, canonical genome."""
 
+import copy
 from pathlib import Path
 
 import numpy as np
@@ -512,3 +513,97 @@ def test_predicted_shape_fails_is_nonneg_and_pure():
     assert sum(len(lvl.leaves()) for lvl in dom.levels(root)) == n_leaves
     # deterministic
     assert operators.predicted_shape_fails(root, reqs, fit) == pred
+
+
+# --------------------------------------------------------------------------- #
+# 7fm — targeted shape repair (shape_rotate / deslim)
+# --------------------------------------------------------------------------- #
+@pytest.mark.skipif(not HARBOR.is_dir(), reason="harbor-house not available")
+def test_shape_failing_flags_known_fail_only():
+    from homemaker_layout import fitness, programme
+
+    conf, cost = fitness.load_config(str(HARBOR))
+    fit = fitness.Fitness(conf, cost)
+    root = dom.load(str(HARBOR / "generated.dom"))
+    lvl0 = dom.levels(root)[0]
+
+    # generated.dom/0/rr (type "r") has a real proportion fail (fixture,
+    # verified via homemaker-fitness); an outside leaf is never a candidate
+    # regardless of its geometry.
+    assert operators._shape_failing(lvl0.by_id("rr"), fit)
+    assert not operators._shape_failing(lvl0.by_id("lllrl"), fit)  # type O
+
+
+@pytest.mark.skipif(not HARBOR.is_dir(), reason="harbor-house not available")
+def test_mutate_shape_rotate_noop_without_fit():
+    root = dom.load(str(HARBOR / "generated.dom"))
+    child, desc = operators.mutate_shape_rotate(root, np.random.default_rng(0), TYPES)
+    assert "noop" in desc
+    canonical(child)
+
+
+def _with_forced_slim_leaf(root: dom.Node, code: str = "r") -> tuple[dom.Node, str]:
+    """Force a real, deterministic shape fail: divide the largest outside leaf
+    95/5 into (``code``, "C"). The 5% side is narrow/high-aspect on any real
+    plot, and both sides are fresh leaves (a valid deslim candidate too),
+    unlike the fixture's organic fails which may not have a mergeable sibling."""
+    from homemaker_layout import geometry
+
+    child = copy.deepcopy(root)
+    lvl0 = dom.levels(child)[0]
+    host = max((lf for lf in lvl0.leaves() if lf.type == "O"), key=geometry.area)
+    host_id = host.id
+    host.division = [0.05, 0.05]
+    host.rotation = 0
+    host.left = dom.Node(type=code)
+    host.right = dom.Node(type="C")
+    host.type = None
+    child = operators._finalise(child)
+    leaf_id = (host_id + "l") if host_id else "l"
+    return child, leaf_id
+
+
+@pytest.mark.skipif(not HARBOR.is_dir(), reason="harbor-house not available")
+def test_mutate_shape_rotate_targets_a_failing_cut():
+    from homemaker_layout import fitness, programme
+
+    reqs = programme.load_programme_dir(str(HARBOR))
+    types = sorted(reqs) + ["C", "O"]
+    conf, cost = fitness.load_config(str(HARBOR))
+    fit = fitness.Fitness(conf, cost)
+    root, leaf_id = _with_forced_slim_leaf(dom.load(str(HARBOR / "generated.dom")))
+    assert operators._shape_failing(dom.levels(root)[0].by_id(leaf_id), fit)
+
+    child, desc = operators.mutate_shape_rotate(root, np.random.default_rng(0), types, fit=fit)
+    assert "noop" not in desc
+    canonical(child)
+    # only the rotation of the targeted cut changes; leaf multiset preserved
+    assert _leaf_types(child) == _leaf_types(root)
+
+
+@pytest.mark.skipif(not HARBOR.is_dir(), reason="harbor-house not available")
+def test_mutate_deslim_merges_failing_leaf_and_is_repairable():
+    from homemaker_layout import fitness, graph, programme
+
+    reqs = programme.load_programme_dir(str(HARBOR))
+    types = sorted(reqs) + ["C", "O"]
+    conf, cost = fitness.load_config(str(HARBOR))
+    fit = fitness.Fitness(conf, cost)
+    root, _leaf_id = _with_forced_slim_leaf(dom.load(str(HARBOR / "generated.dom")))
+    n_leaves = sum(len(lvl.leaves()) for lvl in dom.levels(root))
+
+    child, desc = operators.mutate_deslim(root, np.random.default_rng(0), types, fit=fit)
+    assert "noop" not in desc
+    canonical(child)
+    # a merge strictly reduces the leaf count...
+    assert sum(len(lvl.leaves()) for lvl in dom.levels(child)) == n_leaves - 1
+    # ...and the displaced room is repairable by the existing place_missing op
+    _, missing = graph.check_space_counts(child, reqs)
+    assert missing
+    rng = np.random.default_rng(0)
+    for _ in range(len(missing) + 5):
+        child, _ = operators.mutate_place_missing(child, rng, types, reqs=reqs)
+        _, missing = graph.check_space_counts(child, reqs)
+        if not missing:
+            break
+    assert missing == []