diff --git a/DESIGN.md b/DESIGN.md index a9cb36d..7cd342b 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -4834,3 +4834,161 @@ connectivity/level-placement/geometry problem, not an adjacency-graph one — future effort on that plateau (`homemaker-py-2g7.7`'s LLM repair operator, or a level-connectivity-targeted operator) is better aimed than a graph-dual construction pass would have been. + +## 38. The plateau is an objective-gradient problem, not a search problem (`homemaker-py-ssz`/`hxi`/`tdp`/`gvb`/`1i8`) — measured 2026-08-25 + +Independent review of why the search "finds solutions that are clearly not the +best and gets stuck in local minima", prompted by the §37 scoreboard: *every* +fail-count win of Phases 6–8 was a construction/objective-honesty lever and +*every* search-machinery lever (§11.4 grade, §11.5 niching/restarts, §11.8 +tournament-k, §14 islands, §16 annealing, §29/§30 beam, §27 bubble, §34 +autodiff, §37.7 CP-SAT) was null or negative. That pattern is itself the +finding: eight independent attempts to improve the *search* all failed, which +is what you would expect if the search is working correctly and the +**objective's gradient points away from good buildings**. + +Reproduce everything below with `experiments/diag_exposure_frontage.py` +(`frontage` / `exposure` / `value` reports; no search run required). + +### 38.1 Zero-exposure leaves score a hard quality of 0 (`homemaker-py-ssz`) + +`fitness.quality_uncrinkliness` computes `crink = area_outside / area` and +returns a hard `0.0` when `area_outside == 0` — a leaf with no daylit wall +(no non-`private`/`fortified` external edge, no adjacent uncovered outside +leaf). This is the mathematically consistent limit of the formula +(`1/crink → ∞`, and `gaussian(∞, …) → 0`), so it is a **faithful port, not a +porting bug** — but its consequences were never traced: + +- `evaluate_leaf` **multiplies** factors into `quality`, and `process_storey` + accumulates `value += quality * rate * area`. A buried leaf therefore + contributes **exactly zero value** while still adding cost. +- So the objective cannot distinguish a buried room that is perfectly sized + from one that is absurd. Both score zero. The only thing the objective can + still see about a buried room is that it costs money. + +Measured share of interior/covered leaves that are zero-exposure, under the +driver's real default stack (`leaf_sharing`, `depth_balanced`, +`interior_outside`, `collapse_insearch`), 3 constructed seeds each: + +| programme | zero-exposure | wrong-ratio | ok | +|---|---|---|---| +| harbor-house | **36 (46%)** | 24 | 18 | +| health-centre | **35 (45%)** | 0 | 43 | +| maple-court | **83 (56%)** | 44 | 22 | + +On a converged run (`homemaker-evolve init.dom --budget 20000 --seed 1`, +harbor-house, 57 fails) **14 of 17 crinkliness fails are zero-exposure**, and +roughly 470 m² of the 721 m² ground-floor plate sits at zero value. + +### 38.2 Buried circulation and outside space are negative-value (`homemaker-py-hxi`) + +Programme rooms are pinned in place by the missing-space fail cascade — but +nothing pins circulation (`C`) or outside (`O`) leaves, which carry no +`count:` requirement. Deleting a buried one is therefore a pure win. +Measured on a constructed harbor-house seed (`value` report): + +| deleted leaf | score change | fail change | +|---|---|---| +| buried `O`, 45.8 m² | **×85.6 BETTER** | 92 → 85 | +| buried `C`, 46.6 m² | **×61.6 BETTER** | 92 → 86 | +| buried `k1`, 30.3 m² | ×0.00 worse | 92 → 107 | +| buried `da1`, 61.7 m² | ×0.00 worse | 92 → 107 | + +**The search is rewarded, by roughly two orders of magnitude, for deleting the +circulation spine.** Observed live: in the 20 000-eval harbor-house run above, +`undivide`/`core_undivide` appear 16 times in the improvement log. + +This retro-explains three prior results as one mechanism, and suggests two of +them were measuring a broken gradient rather than a bad idea: + +- **§18 graded circulation-connectivity — NEGATIVE.** A secondary comparator + key cannot beat a ×60 primary-scalar gradient pulling the other way. +- **§21/§22 `bridge_circulation` — mixed/null.** The operator inserts exactly + the corridor leaves the objective then pays to delete. +- **The 3M-eval run's `level 0/1 not connected` hard fails surviving >1M + evals.** Not a stubborn search; a correctly-followed gradient. + +### 38.3 The binding constraint is a frontage budget (`homemaker-py-tdp`) + +Closed form, no search needed. Crinkliness fails when `1/crink > 1.6202` +(solving `gaussian(x, 1, 5/6, 1.1/3) = FAIL_THRESHOLD`), and `crink = L·h/A`, +so every interior leaf needs exposed wall `L ≥ A/(1.6202·h)` — per storey, +`A_storey/4.86` metres at `h = 3`. `area_outside` skips `private` and +`fortified` perimeter edges, and harbor-house/maple-court mark **half their +plot perimeter `private`**: + +| programme | daylit frontage | needed per built storey | verdict | observed floor | +|---|---|---|---|---| +| harbor-house | 54 m | 148 m | **2.7× short** | plateaus 30–40 fails | +| maple-court | 56 m | 162 m | **2.9× short** | plateaus 74–84 fails | +| health-centre | 43 m | 41 m | feasible | §32 clean null | +| programme-house | 24 m | 12 m | 2× surplus | **1 fail** (12k evals) | + +**The corpus fail-count plateau is predicted by frontage deficit alone.** The +two programmes that are frontage-short are exactly the two that plateau; the +two with surplus are the two that effectively solve. Causal check +(`experiments/diag_exposure_frontage.py`, 6 seeds): relabelling harbor's two +`private` edges as open — identical geometry, identical programme, perimeter +labels only — cuts zero-exposure leaves **52% → 19%** and seeder crinkliness +fails 16.5 → 12.0. + +The deficit *is* closable: ~108 m² per storey (≈15% of the plate) given over +to ~3 m courtyard slots would satisfy harbor's budget while still leaving +1226 m² of floor against an 835 m² programme demand. **But that is precisely +the move the objective punishes en route** — a small new `O` leaf is itself +buried, hence zero-value, hence worth ×85 to delete. The payoff only arrives +once a slot is wide enough and long enough to serve many rooms at once. Every +intermediate step is punished; the reward is behind a coordinated multi-leaf +move. That is the valley, and no amount of population diversity crosses it +when the gradient opposes you the whole way — which is why §11.5, §14, §16 and +§37.10-style diversity levers were always going to be null here. + +### 38.4 Crinkliness is mis-tiered as SOFT (`homemaker-py-gvb`) + +`fitness._SOFT_FAIL_MARKERS` lists `" crinkliness"` as SOFT, defined in §37.1 +as "a continuous per-leaf shape metric the inner-loop ratio solve can improve +without changing the tree". **False for the zero-exposure case**: no ratio +assignment can give a buried leaf a wall, so by this document's own definition +it is HARD. Zero-exposure share of crinkliness fails: harbor-house 60%, +maple-court 65%, health-centre 100%, and 82% (14/17) on the converged run. +Since crinkliness is the single largest fail category (48% of the residual, +§13.11), **the §37.1 tiered comparator is mis-informed about the largest block +of fails it sorts** — `n_soft` is not the polish-budget signal it was designed +to be. Fix: emit a distinct fail string for the zero-exposure case (which also +makes the condition visible in `.fails` output, where today it is +indistinguishable from an ordinary shape miss) and tier it HARD. + +### 38.5 The missing-space cascade is weighted by YAML verbosity (`homemaker-py-1i8`) + +`graph.check_space_counts` emits, per missing room instance, 2 base fails +(`missing required space: X` + `(critical)`) plus one `would need ` +placeholder for **each of `size`/`width`/`proportion` the programme happens to +declare** — `has_size` is literally `"size" in c` from the YAML. So a missing +room costs 3–5 fails depending only on how many optional keys the author +typed, and under `value *= 0.5 ** len(failures)` that is a **4× difference in +fitness weight between two single rooms**. In programme-house, missing `b1` +(declares all three) = 5 fails = 1/32; missing `t2` (declares `size` only) = +3 fails = 1/8. The tiered comparator inherits it: `n_hard` is dominated by +these cascades, so the primary search key is weighted by config verbosity. + +### 38.6 Consequences for the Phase 9 plan + +§37 track 1 ("no ground truth … the residual taxonomy may be miscalibrated +rather than unmet") was aimed at the right target, and §38.3 supplies a cheap +way to test it that does **not** need `2g7.1`'s traced human plans: the +frontage bound is a pre-flight feasibility check computable from a plot and a +programme alone. Two of the four corpus programmes fail it by ~3×, which means +a share of the residual those runs are being judged on **is not reachable at +all** — and any A/B measured against that residual has been measuring, in +part, an unsatisfiable constraint. + +Tracks 2 and 3 (cheaper evaluation, exact sub-solvers) remain sound but are +orthogonal: making an evaluation 97× faster, or a labelling exact, does not +change which direction the objective points. Recommended ordering is now +`ssz` → `hxi`/`gvb` (restore a value gradient for interior space, re-tier), +then `tdp` (ship the pre-flight bound and re-baseline the corpus), and only +then resume `2g7.9`/`2g7.10`. In particular `2g7.7` (LLM repair operator at +stagnation) is worth deferring until after `ssz`: an LLM asked to propose a +valley-crossing multi-edit against an objective that pays ×85 to delete the +corridor it just inserted will have its work reverted by the next selection +step. diff --git a/experiments/diag_exposure_frontage.py b/experiments/diag_exposure_frontage.py new file mode 100644 index 0000000..5edcec7 --- /dev/null +++ b/experiments/diag_exposure_frontage.py @@ -0,0 +1,237 @@ +"""Exposure / frontage diagnostic for the crinkliness residual. + +Evidence for `homemaker-py-ssz` / `hxi` / `tdp` (DESIGN.md §38). Three reports, +none of which needs a search run: + +1. ``exposure`` — decompose crinkliness failures into ZERO-EXPOSURE (the leaf + has no daylit wall at all, so ``quality_uncrinkliness`` returns a hard 0.0 + and the leaf's whole quality product collapses to zero) vs. wrong-ratio + (exposed, but off the uncrinkliness target). Run on constructed seeds under + the driver's real default stack, or on a ``.dom`` from disk. + +2. ``value`` — measure what a buried leaf is worth to the objective, by + deleting one (undividing its parent) and re-scoring. Circulation/outside + leaves carry no missing-space requirement, so nothing offsets their removal. + +3. ``frontage`` — the closed-form feasibility bound. Crinkliness fails when + ``1/crink > X`` with ``crink = L*h/A``, so every interior leaf needs exposed + wall ``L >= A/(X*h)``; per storey that is ``A_storey/(X*h)`` metres. Compare + against the daylit plot perimeter (``area_outside`` skips ``private`` and + ``fortified`` edges) to get the deficit. + +Usage:: + + python experiments/diag_exposure_frontage.py frontage + python experiments/diag_exposure_frontage.py exposure examples/harbor-house + python experiments/diag_exposure_frontage.py exposure --dom out.dom examples/harbor-house + python experiments/diag_exposure_frontage.py value examples/harbor-house +""" + +from __future__ import annotations + +import argparse +import copy +import math +from pathlib import Path + +import numpy as np +import yaml + +from homemaker_layout import dom as dom_mod +from homemaker_layout import driver, fitness, geometry +from homemaker_layout import graph as graph_mod +from homemaker_layout import operators, programme + +CORPUS = ["examples/harbor-house", "examples/maple-court", + "examples/health-centre", "examples/programme-house"] + + +def fail_bounds() -> tuple[float, float]: + """Solve ``gaussian(x, 1, 5/6, 1.1/3) == FAIL_THRESHOLD`` for x. + + Returns ``(buried_above, exposed_below)``: a leaf fails crinkliness when + ``1/crink`` exceeds the first (too buried) or falls below the second (too + exposed). Derived from the real constants, never hard-coded. + """ + b, c = fitness.CONF_DEFAULTS["uncrinkliness"] + k = math.sqrt(-2 * c * c * math.log(fitness.FAIL_THRESHOLD) / math.log(fitness._E)) + return b + k, b - k + + +def _fit(progdir: str) -> fitness.Fitness: + """Evaluator configured exactly as ``driver.search`` builds it by default.""" + ov = driver._overrides_for(leaf_sharing=True, superpose=False, max_share=None, + conn_grade=False, collapse_insearch=True, + multi_use=False) + conf, cost = fitness.load_config(progdir, overrides=ov) + return fitness.Fitness(conf, cost) + + +def _seed(progdir: str, seed: int) -> dom_mod.Node: + """One constructed seed under ``driver.search``'s own default arguments.""" + reqs = programme.load_programme_dir(progdir) + return operators.constructive_topology( + dom_mod.load(f"{progdir}/init.dom"), reqs, np.random.default_rng(seed), + sorted(reqs) + ["C", "O"], + min_storeys=programme.storey_minimum(progdir), + adjacency_aware=True, proportion_aware=True, circ_divisor=3, + leaf_sharing=True, leaf_share_factor=3, depth_balanced=True, + interior_outside=True, outside_divisor=3) + + +def _exposure_rows(fit: fitness.Fitness, root: dom_mod.Node) -> list[tuple]: + """Per interior/covered leaf: (level, id, type, area, exposed_area, quality). + + Reproduces the state ``_evaluate_full`` reaches at storey processing — + in-search collapse, preprocess, merge — so the numbers match a real eval. + """ + tree = copy.deepcopy(root) + geometry.clear_cache() + dom_mod.canonicalize_shares(tree) + if fit._collapse_insearch: + fit.collapse_global(tree, adjacency=True, objective="threshold", + preserve_public_access=True, iters=3) + fit.preprocess_building(tree) + dom_mod.merge_divided(tree) + geometry.clear_cache() + graphs = graph_mod.build_graphs(tree, fit.conf("door_width") or 1.2) + + rows = [] + for li, lvl in enumerate(dom_mod.levels(tree)): + for leaf in lvl.leaves(): + if dom_mod.is_outside(leaf) and not dom_mod.is_covered(leaf): + continue # exempt by quality_uncrinkliness + rows.append((li, leaf.id, leaf.type, geometry.area(leaf), + fit.area_outside(leaf, graphs[li], {}), + fit.quality_uncrinkliness(leaf, graphs[li], {}))) + return rows + + +def report_exposure(progdir: str, seeds: range, dom_path: str | None) -> None: + fit = _fit(progdir) + roots = ([dom_mod.load(dom_path)] if dom_path + else [_seed(progdir, s) for s in seeds]) + ok = zero = ratio = 0 + n_fails = n_crink = 0 + for root in roots: + _, fails = fit.score_with_fails(copy.deepcopy(root)) + n_fails += len(fails) + n_crink += sum("crinkliness" in f for f in fails) + for _, _, _, _, exposed, q in _exposure_rows(fit, root): + if exposed == 0: + zero += 1 + elif q < fitness.FAIL_THRESHOLD: + ratio += 1 + else: + ok += 1 + total = ok + zero + ratio + label = dom_path or f"{len(roots)} constructed seeds" + print(f"=== {progdir} ({label})") + print(f" mean fails/design {n_fails / len(roots):.1f}, of which crinkliness " + f"{n_crink / len(roots):.1f}") + print(f" interior leaves: ok={ok} ZERO-EXPOSURE={zero} wrong-ratio={ratio}") + if total: + print(f" -> {100 * zero / total:.0f}% of interior leaves have NO daylit wall: " + f"hard quality=0, unreachable by any ratio assignment") + + +def report_value(progdir: str, seed: int, limit: int) -> None: + """What is a buried leaf worth? Delete one and re-score.""" + fit = _fit(progdir) + root = _seed(progdir, seed) + base_score, base_fails = fit.score_with_fails(copy.deepcopy(root)) + print(f"=== {progdir} seed {seed}: baseline score {base_score:.4g}, " + f"{len(base_fails)} fails") + + buried = [(li, lid, typ) for li, lid, typ, _, exposed, _ + in _exposure_rows(fit, root) if exposed == 0] + print(f" {len(buried)} buried leaves; deleting each (undivide its parent):") + + tried = 0 + for li, lid, typ in buried: + if tried >= limit: + break + cand = copy.deepcopy(root) + lvls = dom_mod.levels(cand) + if li >= len(lvls): + continue + node = lvls[li].by_id(lid) + if node is None or node.parent is None: + continue + parent = node.parent + if parent.below is not None and parent.below.divided: + continue # inherited cut, not owned here + sibling = parent.right if parent.left is node else parent.left + if sibling is None or sibling.divided: + continue + parent.division = None + parent.left = parent.right = None + parent.type = sibling.type + dom_mod.link(cand) + geometry.clear_cache() + score, fails = fit.score_with_fails(copy.deepcopy(cand)) + tried += 1 + verdict = "BETTER" if score > base_score else "worse" + print(f" delete {lid:<8s} type={typ:<5s}: score x{score / base_score:>8.2f} " + f"({verdict}), fails {len(base_fails)} -> {len(fails)}") + + +def report_frontage(progdirs: list[str]) -> None: + x_buried, x_exposed = fail_bounds() + print(f"crinkliness fails when 1/crink > {x_buried:.4f} (buried) " + f"or < {x_exposed:.4f} (over-exposed)") + print(f"=> every interior leaf needs exposed wall L >= A / ({x_buried:.4f} * h)\n") + + for progdir in progdirs: + seed = yaml.safe_load(open(f"{progdir}/init.dom")) + corners, per = seed["node"], (seed.get("perimeter") or {}) + height = seed.get("height") or 3.0 + n = len(corners) + edges = [math.hypot(corners[(i + 1) % n][0] - corners[i][0], + corners[(i + 1) % n][1] - corners[i][1]) for i in range(n)] + daylit = sum(e for k, e in zip("abcd", edges) + if (per.get(k) or "").lower() not in ("private", "fortified")) + area = abs(sum(corners[i][0] * corners[(i + 1) % n][1] + - corners[(i + 1) % n][0] * corners[i][1] + for i in range(n))) / 2 + reqs = programme.load_programme_dir(progdir) + n_storeys = max(programme.n_storeys_required(reqs), + programme.storey_minimum(progdir)) + demand = sum(r.size * r.count for r in reqs.values()) + needed = area / (x_buried * height) + + print(f"=== {Path(progdir).name}") + print(f" plot {area:.0f} m2, perimeter {sum(edges):.0f} m, " + f"{n_storeys} storeys, h={height}") + print(f" perimeter {per} -> daylit frontage {daylit:.0f} m") + print(f" a fully built storey needs {needed:.0f} m exposed wall; " + f"plot supplies {daylit:.0f} m " + f"-> {needed / max(daylit, 1e-9):.1f}x short" + if needed > daylit else + f" a fully built storey needs {needed:.0f} m exposed wall; " + f"plot supplies {daylit:.0f} m -> FEASIBLE") + print(f" programme demands {demand:.0f} m2 over {n_storeys} storeys " + f"({demand / n_storeys:.0f} m2/storey of {area:.0f} m2 plot)\n") + + +def main() -> None: + ap = argparse.ArgumentParser(description=__doc__, + formatter_class=argparse.RawDescriptionHelpFormatter) + ap.add_argument("report", choices=("exposure", "value", "frontage")) + ap.add_argument("progdir", nargs="?", default=None) + ap.add_argument("--dom", default=None, help="score this .dom instead of seeds") + ap.add_argument("--seeds", type=int, default=3) + ap.add_argument("--limit", type=int, default=5, help="deletions to try (value)") + args = ap.parse_args() + + if args.report == "frontage": + report_frontage([args.progdir] if args.progdir else CORPUS) + elif args.report == "exposure": + for d in ([args.progdir] if args.progdir else CORPUS): + report_exposure(d, range(args.seeds), args.dom) + else: + report_value(args.progdir or CORPUS[0], seed=0, limit=args.limit) + + +if __name__ == "__main__": + main()