homemaker-layout/experiments/ab_crinkliness_mode_ssz.py

168 lines
6.8 KiB
Python
Raw Permalink Normal View History

§38.2 refinement: connectivity is under-priced ~3x, not just a crinkliness bug Follow-up measurement corrects the first draft of §38 in two ways. 1. Harbor-house's floor is 15 fails (evolved-3M-nols-3, 1.7M evals), not the 30-40 I quoted from §13.11's 20k-budget runs. Frontage deficit predicts the COST of solving, not impossibility: ~150x budget gap between a frontage-short and a frontage-surplus programme. Table corrected. 2. Zero-exposure is only half the mechanism, and not the dominant half. Splitting the deletion test by lit vs buried shows a WELL-DAYLIT corridor (q_crink=0.736) is still worth x4.06 to delete. Cause: value_circulation=50 vs value_inside=300, so merging corridor into room is a flat x6 gain, while 'level N not connected' costs only x0.5. Break-even needs 0.5^k < 50/300, i.e. k > 2.58 -- severing must cost at least 3 fails and costs 1. Net x3.0 predicted, x4.06 measured. The objective is net-positive on severing the spine even when the circulation is perfectly lit, which explains why both 'level N not connected' fails survive in the best layout after 1.7M evals. Adds fitness.quality_uncrinkliness crinkliness_mode (EXPERIMENTAL, default "urb" = stock hard 0.0, byte-identical: 336 passed vs 331 before, same 7 pre-existing fixture failures). A/B harness ab_crinkliness_mode_ssz.py shows none of the three modes removes the incentive, and the lit column is 3/8 under every mode including stock -- clean isolation of the two mechanisms. Filed homemaker-py-2v1 (P0) for the pricing fix; ssz/hxi now depend on it. Acceptance test recorded up front: harbor must reach 15 fails in materially fewer than 1.7M evals AND without either not-connected fail. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
2026-08-26 07:40:37 +00:00
"""A/B the `crinkliness_mode` repairs against the circulation-deletion incentive.
Evidence for `homemaker-py-ssz` / `homemaker-py-2v1` (DESIGN.md §38.6). The
test: on a constructed layout, delete each unpinned circulation/outside leaf
(by undividing its parent, which merges it into its sibling) and re-score. A
healthy objective should not pay you to do that.
Reports the count of deletions that still improve the score, and the median
score ratio, per mode. Splitting the rows by whether the deleted leaf was
buried or lit is what separates the two mechanisms:
* buried leaves are rewarded because ``quality_uncrinkliness`` returns a hard
0.0, so they contribute zero value (§38.1);
* **lit** leaves are rewarded because ``value_circulation`` (50) is one sixth
of ``value_inside`` (300), so merging corridor into room is a flat ×6 gain
against a ×0.5 connectivity penalty (§38.2 refinement) which no
``crinkliness_mode`` can touch.
ssz: daylight is required of rooms that do not need it DESIGN.md 38.6 concluded the three crinkliness modes were inert against the circulation-deletion incentive. Two things were wrong with that measurement. Its premise, 38.2, is retracted. And its script selected leaves with the pre-39.4 prefix rule `type[:1].upper() in ("C","O")`, which sweeps every programme room starting with c or o -- cr1, of1 -- in as circulation. The simpler problem is that none of the three modes ever touched the leaves ssz is about. quality_uncrinkliness reaches `if not crink` before any mode logic that matters, so for a zero-exposure leaf: floor returns 0.01 (one percent of a unit quality, multiplied into a product and weighed against a whole leaf's cost -- inert); compact_ok is self-contradictory, announcing that compact is not a defect and then returning the floor for the most compact case of all; exempt_circulation reaches at most a third of them. Measured: 0% / 0% / 0% / 21-33% of buried leaves rescued. What the buried leaves are, now that 39.7 gives every space a usage: two thirds of them are spaces that architecturally do not want a window -- stores, WCs, plant, corridors, covered courtyards -- scored identically with a windowless bedroom. harbor 22/33, maple 33/46, health 9/18. - crinkliness_mode="usage_daylight": daylight required of the uses a person occupies (programme.DAYLIGHT_USAGES) and nothing else. Elsewhere the factor is clipped on the compact side only, so being buried stops being a defect while over-exposure still costs -- a crinkly leaf costs envelope whatever it is used for. A windowless bedroom stays the hard zero it is under stock: 11/11, 13/13, 9/9 still failing. - compact_ok repaired to score the buried limit as compact, the behaviour its name always claimed. It now rescues 100% including bedrooms, and is kept as the upper-bound control, not a candidate. - ab_ssz_search.py: the fixed-budget search A/B ssz's acceptance criteria actually asks for. Every arm is optimised under its own objective and re-scored under stock urb, because the permissive modes return 1.0 where stock fails and would otherwise win by deleting a fail category. - ab_crinkliness_mode_ssz.py: prefix rule fixed, retracted premise flagged in its docstring. - 38.7's remaining claims from the retracted 38.2/38.3 corrected. Default is unchanged ("urb"), byte-identical to all prior runs. Lint at parity (46 pre-existing); tests 366 passed, 10 new, same 7 pre-existing fixture failures (homemaker-py-bdf). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
2026-08-26 16:45:10 +00:00
.. warning::
**The premise of this test is RETRACTED see §38.2 and §38.8.** The ×6-vs-
×0.5 arithmetic assumed severing the spine costs one connectivity fail;
measured, the connectivity count is unchanged in every rewarded deletion, so
this script is not measuring what its docstring says. It is kept because the
per-mode buried/lit split is still a useful description of what each mode
touches, but a mode does **not** pass or fail on these numbers. The A/B that
decides `ssz` is ``experiments/ab_ssz_search.py`` (fixed-budget search,
scored under the stock objective).
§38.2 refinement: connectivity is under-priced ~3x, not just a crinkliness bug Follow-up measurement corrects the first draft of §38 in two ways. 1. Harbor-house's floor is 15 fails (evolved-3M-nols-3, 1.7M evals), not the 30-40 I quoted from §13.11's 20k-budget runs. Frontage deficit predicts the COST of solving, not impossibility: ~150x budget gap between a frontage-short and a frontage-surplus programme. Table corrected. 2. Zero-exposure is only half the mechanism, and not the dominant half. Splitting the deletion test by lit vs buried shows a WELL-DAYLIT corridor (q_crink=0.736) is still worth x4.06 to delete. Cause: value_circulation=50 vs value_inside=300, so merging corridor into room is a flat x6 gain, while 'level N not connected' costs only x0.5. Break-even needs 0.5^k < 50/300, i.e. k > 2.58 -- severing must cost at least 3 fails and costs 1. Net x3.0 predicted, x4.06 measured. The objective is net-positive on severing the spine even when the circulation is perfectly lit, which explains why both 'level N not connected' fails survive in the best layout after 1.7M evals. Adds fitness.quality_uncrinkliness crinkliness_mode (EXPERIMENTAL, default "urb" = stock hard 0.0, byte-identical: 336 passed vs 331 before, same 7 pre-existing fixture failures). A/B harness ab_crinkliness_mode_ssz.py shows none of the three modes removes the incentive, and the lit column is 3/8 under every mode including stock -- clean isolation of the two mechanisms. Filed homemaker-py-2v1 (P0) for the pricing fix; ssz/hxi now depend on it. Acceptance test recorded up front: harbor must reach 15 fails in materially fewer than 1.7M evals AND without either not-connected fail. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
2026-08-26 07:40:37 +00:00
Usage::
python experiments/ab_crinkliness_mode_ssz.py
python experiments/ab_crinkliness_mode_ssz.py --seeds 5 --progdir examples/maple-court
"""
from __future__ import annotations
import argparse
import copy
import numpy as np
from homemaker_layout import dom as dom_mod
from homemaker_layout import driver, fitness, geometry
from homemaker_layout import graph as graph_mod
from homemaker_layout import operators, programme
ssz: crinkliness is declared per space; there is no daylight attribute Owner's ruling, and it corrects the design not just the classification: the daylight requirement is already defined in the crinkliness. The gaussian's compact side IS "too little exposed wall per unit floor"; its exposed side is envelope cost. 38.9's proposed daylight: axis was redundant, and keying it off usage: was worse than redundant. What was actually missing: crinkliness is the only leaf quality factor with no per-space target. size, width and proportion are all declared by the space; crinkliness was one global number for every room in every building. crinkliness: none -> no minimum-exposure requirement, may be buried crinkliness: [t, s] -> this space's own target key absent -> the global uncrinkliness target, as today `none` clips the factor on the compact side, it does not switch it off: over-exposure is still penalised, because a crinkly leaf costs envelope whatever it holds. A store may be buried; a store may not be a starfish. The mechanism is backward compatible -- an absent key resolves to the global target, so shipping it changes no score. Behaviour changes only where a config declares something, which keeps the objective change visible per programme in config rather than hidden in a default. Owner's classification: everything a person occupies wants a window, WCs and reception/waiting/foyer included; only stores, plant, records and laundry do not. migrate_crinkliness_key.py declared crinkliness: none on 18 corpus spaces. Crinkliness fails 271 -> 243, of which not-defects 136 (50%) -> 108 (44%); the 28 that went are exactly the utility fails. usage_daylight and needs_daylight are removed as mis-keyed, and DAYLIGHT_USAGES with them -- a vocabulary value should exist only where the engine treats it differently. The historical crinkliness_mode modes stay, default off, so 38.6/38.8 remain reproducible. uncrinkliness_circulation is now settable to none like any space, but its default is left unchanged pending a ruling: corridors were not among the groups ruled on and are 63% of the remaining phantom fails. Lint at parity (46); tests 364 passed, same 7 pre-existing fixture failures. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
2026-08-28 22:38:54 +00:00
MODES = ("urb", "floor", "compact_ok", "exempt_circulation")
§38.2 refinement: connectivity is under-priced ~3x, not just a crinkliness bug Follow-up measurement corrects the first draft of §38 in two ways. 1. Harbor-house's floor is 15 fails (evolved-3M-nols-3, 1.7M evals), not the 30-40 I quoted from §13.11's 20k-budget runs. Frontage deficit predicts the COST of solving, not impossibility: ~150x budget gap between a frontage-short and a frontage-surplus programme. Table corrected. 2. Zero-exposure is only half the mechanism, and not the dominant half. Splitting the deletion test by lit vs buried shows a WELL-DAYLIT corridor (q_crink=0.736) is still worth x4.06 to delete. Cause: value_circulation=50 vs value_inside=300, so merging corridor into room is a flat x6 gain, while 'level N not connected' costs only x0.5. Break-even needs 0.5^k < 50/300, i.e. k > 2.58 -- severing must cost at least 3 fails and costs 1. Net x3.0 predicted, x4.06 measured. The objective is net-positive on severing the spine even when the circulation is perfectly lit, which explains why both 'level N not connected' fails survive in the best layout after 1.7M evals. Adds fitness.quality_uncrinkliness crinkliness_mode (EXPERIMENTAL, default "urb" = stock hard 0.0, byte-identical: 336 passed vs 331 before, same 7 pre-existing fixture failures). A/B harness ab_crinkliness_mode_ssz.py shows none of the three modes removes the incentive, and the lit column is 3/8 under every mode including stock -- clean isolation of the two mechanisms. Filed homemaker-py-2v1 (P0) for the pricing fix; ssz/hxi now depend on it. Acceptance test recorded up front: harbor must reach 15 fails in materially fewer than 1.7M evals AND without either not-connected fail. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
2026-08-26 07:40:37 +00:00
def make_fitness(progdir: str, mode: str) -> fitness.Fitness:
"""Evaluator matching ``driver.search``'s defaults, with one mode override."""
overrides = dict(driver._overrides_for(
leaf_sharing=True, superpose=False, max_share=None, conn_grade=False,
collapse_insearch=True, multi_use=False) or {})
overrides["crinkliness_mode"] = mode
conf, cost = fitness.load_config(progdir, overrides=overrides)
return fitness.Fitness(conf, cost)
def constructed_seed(progdir: str, seed: int) -> dom_mod.Node:
reqs = programme.load_programme_dir(progdir)
return operators.constructive_topology(
dom_mod.load(f"{progdir}/init.dom"), reqs, np.random.default_rng(seed),
sorted(reqs) + ["C", "O"],
min_storeys=programme.storey_minimum(progdir),
adjacency_aware=True, proportion_aware=True, circ_divisor=3,
leaf_sharing=True, leaf_share_factor=3, depth_balanced=True,
interior_outside=True, outside_divisor=3)
def unpinned_leaves(fit: fitness.Fitness, root: dom_mod.Node) -> list[tuple]:
"""(level, id, type, exposed_area) for circulation/outside leaves — the ones
no missing-space cascade pins in place."""
tree = copy.deepcopy(root)
geometry.clear_cache()
dom_mod.canonicalize_shares(tree)
fit.preprocess_building(tree)
dom_mod.merge_divided(tree)
geometry.clear_cache()
graphs = graph_mod.build_graphs(tree, fit.conf("door_width") or 1.2)
out = []
for li, lvl in enumerate(dom_mod.levels(tree)):
for leaf in lvl.leaves():
if dom_mod.is_outside(leaf) and not dom_mod.is_covered(leaf):
continue
ssz: daylight is required of rooms that do not need it DESIGN.md 38.6 concluded the three crinkliness modes were inert against the circulation-deletion incentive. Two things were wrong with that measurement. Its premise, 38.2, is retracted. And its script selected leaves with the pre-39.4 prefix rule `type[:1].upper() in ("C","O")`, which sweeps every programme room starting with c or o -- cr1, of1 -- in as circulation. The simpler problem is that none of the three modes ever touched the leaves ssz is about. quality_uncrinkliness reaches `if not crink` before any mode logic that matters, so for a zero-exposure leaf: floor returns 0.01 (one percent of a unit quality, multiplied into a product and weighed against a whole leaf's cost -- inert); compact_ok is self-contradictory, announcing that compact is not a defect and then returning the floor for the most compact case of all; exempt_circulation reaches at most a third of them. Measured: 0% / 0% / 0% / 21-33% of buried leaves rescued. What the buried leaves are, now that 39.7 gives every space a usage: two thirds of them are spaces that architecturally do not want a window -- stores, WCs, plant, corridors, covered courtyards -- scored identically with a windowless bedroom. harbor 22/33, maple 33/46, health 9/18. - crinkliness_mode="usage_daylight": daylight required of the uses a person occupies (programme.DAYLIGHT_USAGES) and nothing else. Elsewhere the factor is clipped on the compact side only, so being buried stops being a defect while over-exposure still costs -- a crinkly leaf costs envelope whatever it is used for. A windowless bedroom stays the hard zero it is under stock: 11/11, 13/13, 9/9 still failing. - compact_ok repaired to score the buried limit as compact, the behaviour its name always claimed. It now rescues 100% including bedrooms, and is kept as the upper-bound control, not a candidate. - ab_ssz_search.py: the fixed-budget search A/B ssz's acceptance criteria actually asks for. Every arm is optimised under its own objective and re-scored under stock urb, because the permissive modes return 1.0 where stock fails and would otherwise win by deleting a fail category. - ab_crinkliness_mode_ssz.py: prefix rule fixed, retracted premise flagged in its docstring. - 38.7's remaining claims from the retracted 38.2/38.3 corrected. Default is unchanged ("urb"), byte-identical to all prior runs. Lint at parity (46 pre-existing); tests 366 passed, 10 new, same 7 pre-existing fixture failures (homemaker-py-bdf). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
2026-08-26 16:45:10 +00:00
# §39.4: generic structural types are EXACT `C`/`O`/`S`, never a
# first-character prefix -- `cr1` is a programme room. This script
# predates that rule and its original `type[:1].upper() in ("C","O")`
# test swept programme rooms into the "unpinned" set, which is one
# reason the §38.6 numbers do not reproduce.
if not dom_mod.is_generic(leaf.type):
§38.2 refinement: connectivity is under-priced ~3x, not just a crinkliness bug Follow-up measurement corrects the first draft of §38 in two ways. 1. Harbor-house's floor is 15 fails (evolved-3M-nols-3, 1.7M evals), not the 30-40 I quoted from §13.11's 20k-budget runs. Frontage deficit predicts the COST of solving, not impossibility: ~150x budget gap between a frontage-short and a frontage-surplus programme. Table corrected. 2. Zero-exposure is only half the mechanism, and not the dominant half. Splitting the deletion test by lit vs buried shows a WELL-DAYLIT corridor (q_crink=0.736) is still worth x4.06 to delete. Cause: value_circulation=50 vs value_inside=300, so merging corridor into room is a flat x6 gain, while 'level N not connected' costs only x0.5. Break-even needs 0.5^k < 50/300, i.e. k > 2.58 -- severing must cost at least 3 fails and costs 1. Net x3.0 predicted, x4.06 measured. The objective is net-positive on severing the spine even when the circulation is perfectly lit, which explains why both 'level N not connected' fails survive in the best layout after 1.7M evals. Adds fitness.quality_uncrinkliness crinkliness_mode (EXPERIMENTAL, default "urb" = stock hard 0.0, byte-identical: 336 passed vs 331 before, same 7 pre-existing fixture failures). A/B harness ab_crinkliness_mode_ssz.py shows none of the three modes removes the incentive, and the lit column is 3/8 under every mode including stock -- clean isolation of the two mechanisms. Filed homemaker-py-2v1 (P0) for the pricing fix; ssz/hxi now depend on it. Acceptance test recorded up front: harbor must reach 15 fails in materially fewer than 1.7M evals AND without either not-connected fail. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
2026-08-26 07:40:37 +00:00
continue
out.append((li, leaf.id, leaf.type,
fit.area_outside(leaf, graphs[li], {})))
return out
def delete_leaf(root: dom_mod.Node, li: int, lid: str) -> "dom_mod.Node | None":
"""Undivide the leaf's parent, merging it into its sibling. None if the cut
is inherited (not owned at this storey) or the sibling is itself divided."""
cand = copy.deepcopy(root)
lvls = dom_mod.levels(cand)
if li >= len(lvls):
return None
node = lvls[li].by_id(lid)
if node is None or node.parent is None:
return None
parent = node.parent
if parent.below is not None and parent.below.divided:
return None
sibling = parent.right if parent.left is node else parent.left
if sibling is None or sibling.divided:
return None
parent.division = None
parent.left = parent.right = None
parent.type = sibling.type
dom_mod.link(cand)
geometry.clear_cache()
return cand
def run(progdir: str, seeds: int) -> None:
print(f"programme: {progdir}, {seeds} constructed seeds")
print("a healthy objective rewards NO deletions\n")
header = f"{'mode':<20}{'buried':<14}{'lit':<14}{'all':<14}median x"
print(header)
print("-" * len(header))
for mode in MODES:
fit = make_fitness(progdir, mode)
counts = {"buried": [0, 0], "lit": [0, 0]}
ratios = []
for s in range(seeds):
root = constructed_seed(progdir, s)
base, _ = fit.score_with_fails(copy.deepcopy(root))
for li, lid, _typ, exposed in unpinned_leaves(fit, root):
cand = delete_leaf(root, li, lid)
if cand is None:
continue
score, _ = fit.score_with_fails(copy.deepcopy(cand))
bucket = "buried" if exposed == 0 else "lit"
counts[bucket][1] += 1
ratios.append(score / base)
if score > base:
counts[bucket][0] += 1
tot = [counts["buried"][0] + counts["lit"][0],
counts["buried"][1] + counts["lit"][1]]
median = sorted(ratios)[len(ratios) // 2] if ratios else float("nan")
buried = "%d/%d" % tuple(counts["buried"])
lit = "%d/%d" % tuple(counts["lit"])
both = "%d/%d" % tuple(tot)
print(f"{mode:<20}{buried:<14}{lit:<14}{both:<14}x{median:.2f}")
def main() -> None:
ap = argparse.ArgumentParser(description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument("--progdir", default="examples/harbor-house")
ap.add_argument("--seeds", type=int, default=3)
args = ap.parse_args()
run(args.progdir, args.seeds)
if __name__ == "__main__":
main()