homemaker-layout/experiments/audit_crinkliness_truth.py

142 lines
5.8 KiB
Python
Raw Normal View History

ssz: 61% of reported crinkliness fails are not defects; two corrections Correction 1: 38.8's A/B scores every arm under stock urb, which is only sound if stock is ground truth -- and stock is what this section shows is miscalibrated. Scoring the repair under the objective it repairs penalises it for repairing. That result is not evidence against usage_daylight. The measurement that matters needs no search: are the emitted failures true? audit_crinkliness_truth.py classifies every stock crinkliness fail by the leaf's declared usage. 164 of 271 (61%) are reported against spaces that want no daylight -- stores, WCs, plant, corridors, covered courtyards -- and it holds on evolved artefacts (maple generated.dom: 64%), not just constructed seeds. Under value *= 0.5 ** len(failures) each one halves the fitness of a design that did nothing wrong. Correction 2: usage: is the wrong key, and usage_daylight as written mis-keys it. 39.7 established usage: as an ACCESS-requirement class. "Needs no special access" and "needs no window" are different questions, and the corpus separates them: usage: none is Waiting Room, Reception, Reception Office, Entrance Foyer -- a waiting room plainly wants daylight and DAYLIGHT_USAGES exempts it. The usage: bedroom half is right, but by luck of where the access axis happened to fall. The audit is robust to that error (61% -> 57% if none is reclassified), so the finding stands; the design does not. Daylight needs its own declared per-space axis, decided by the programme author as usage: was. usage_daylight stays default off and is not the shipping fix. The compact-side clip is the right shape for the factor, so the mechanism is retained pending a daylight: attribute to key it on. Higher-power A/B stopped: it was chasing a score against a flawed objective with a mis-keyed fix. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
2026-08-26 18:07:01 +00:00
"""Are the crinkliness failures the objective emits real defects? (`homemaker-py-ssz`)
Not an A/B. This asks a correctness question the search cannot answer: of the
`crinkliness` failures the STOCK objective reports, how many are on a space
that architecturally wants daylight at all?
A `crinkliness` fail says "this leaf has too little exposed wall for its area".
For a bedroom or a living room that is a real defect. For a broom cupboard, a
WC, a plant room, an internal corridor or a covered courtyard it is not -- those
are ordinary buried architecture, and the fail is an artefact of applying one
daylight requirement to every space regardless of use (DESIGN.md §38.8).
Every fail is classified by the leaf's DECLARED `usage:` (§39.7), so nothing
here rests on how a code is spelled.
Usage::
python experiments/audit_crinkliness_truth.py
python experiments/audit_crinkliness_truth.py --seeds 5
python experiments/audit_crinkliness_truth.py --dom examples/harbor-house/generated.dom
"""
from __future__ import annotations
import argparse
import collections
import copy
from pathlib import Path
import numpy as np
from homemaker_layout import dom as dom_mod
from homemaker_layout import driver, fitness, geometry
from homemaker_layout import graph as graph_mod
from homemaker_layout import operators, programme
CORPUS = ["examples/harbor-house", "examples/maple-court", "examples/health-centre"]
ssz: crinkliness is declared per space; there is no daylight attribute Owner's ruling, and it corrects the design not just the classification: the daylight requirement is already defined in the crinkliness. The gaussian's compact side IS "too little exposed wall per unit floor"; its exposed side is envelope cost. 38.9's proposed daylight: axis was redundant, and keying it off usage: was worse than redundant. What was actually missing: crinkliness is the only leaf quality factor with no per-space target. size, width and proportion are all declared by the space; crinkliness was one global number for every room in every building. crinkliness: none -> no minimum-exposure requirement, may be buried crinkliness: [t, s] -> this space's own target key absent -> the global uncrinkliness target, as today `none` clips the factor on the compact side, it does not switch it off: over-exposure is still penalised, because a crinkly leaf costs envelope whatever it holds. A store may be buried; a store may not be a starfish. The mechanism is backward compatible -- an absent key resolves to the global target, so shipping it changes no score. Behaviour changes only where a config declares something, which keeps the objective change visible per programme in config rather than hidden in a default. Owner's classification: everything a person occupies wants a window, WCs and reception/waiting/foyer included; only stores, plant, records and laundry do not. migrate_crinkliness_key.py declared crinkliness: none on 18 corpus spaces. Crinkliness fails 271 -> 243, of which not-defects 136 (50%) -> 108 (44%); the 28 that went are exactly the utility fails. usage_daylight and needs_daylight are removed as mis-keyed, and DAYLIGHT_USAGES with them -- a vocabulary value should exist only where the engine treats it differently. The historical crinkliness_mode modes stay, default off, so 38.6/38.8 remain reproducible. uncrinkliness_circulation is now settable to none like any space, but its default is left unchanged pending a ruling: corridors were not among the groups ruled on and are 63% of the remaining phantom fails. Lint at parity (46); tests 364 passed, same 7 pre-existing fixture failures. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
2026-08-28 22:38:54 +00:00
# Ruled by the project owner: everything a person occupies wants a window --
# WCs and bathrooms included, reception/waiting/foyer included, offices and
# consulting rooms included. Only stores, plant, records and laundry do not,
# together with the generic structural types (a corridor has its own
# `uncrinkliness_circulation` target; a courtyard is not a room).
NO_DAYLIGHT = {"utility"}
ssz: 61% of reported crinkliness fails are not defects; two corrections Correction 1: 38.8's A/B scores every arm under stock urb, which is only sound if stock is ground truth -- and stock is what this section shows is miscalibrated. Scoring the repair under the objective it repairs penalises it for repairing. That result is not evidence against usage_daylight. The measurement that matters needs no search: are the emitted failures true? audit_crinkliness_truth.py classifies every stock crinkliness fail by the leaf's declared usage. 164 of 271 (61%) are reported against spaces that want no daylight -- stores, WCs, plant, corridors, covered courtyards -- and it holds on evolved artefacts (maple generated.dom: 64%), not just constructed seeds. Under value *= 0.5 ** len(failures) each one halves the fitness of a design that did nothing wrong. Correction 2: usage: is the wrong key, and usage_daylight as written mis-keys it. 39.7 established usage: as an ACCESS-requirement class. "Needs no special access" and "needs no window" are different questions, and the corpus separates them: usage: none is Waiting Room, Reception, Reception Office, Entrance Foyer -- a waiting room plainly wants daylight and DAYLIGHT_USAGES exempts it. The usage: bedroom half is right, but by luck of where the access axis happened to fall. The audit is robust to that error (61% -> 57% if none is reclassified), so the finding stands; the design does not. Daylight needs its own declared per-space axis, decided by the programme author as usage: was. usage_daylight stays default off and is not the shipping fix. The compact-side clip is the right shape for the factor, so the mechanism is retained pending a daylight: attribute to key it on. Higher-power A/B stopped: it was chasing a score against a flawed objective with a mis-keyed fix. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
2026-08-26 18:07:01 +00:00
def stock_fitness(progdir: str) -> fitness.Fitness:
"""Stock objective -- `crinkliness_mode` left at its "urb" default."""
ov = dict(driver._overrides_for(
leaf_sharing=True, superpose=False, max_share=None, conn_grade=False,
collapse_insearch=True, multi_use=False) or {})
conf, cost = fitness.load_config(progdir, overrides=ov)
return fitness.Fitness(conf, cost)
def constructed(progdir: str, s: int) -> dom_mod.Node:
reqs = programme.load_programme_dir(progdir)
return operators.constructive_topology(
dom_mod.load(f"{progdir}/init.dom"), reqs, np.random.default_rng(s),
sorted(reqs) + ["C", "O"], min_storeys=programme.storey_minimum(progdir),
adjacency_aware=True, proportion_aware=True, circ_divisor=3,
leaf_sharing=True, leaf_share_factor=3, depth_balanced=True,
interior_outside=True, outside_divisor=3)
def audit(fit: fitness.Fitness, root: dom_mod.Node) -> collections.Counter:
"""usage -> count, over the leaves that emit a stock `crinkliness` fail."""
tree = copy.deepcopy(root)
geometry.clear_cache()
dom_mod.canonicalize_shares(tree)
fit.preprocess_building(tree)
dom_mod.merge_divided(tree)
geometry.clear_cache()
graphs = graph_mod.build_graphs(tree, fit.conf("door_width") or 1.2)
out: collections.Counter = collections.Counter()
for li, lvl in enumerate(dom_mod.levels(tree)):
groups = geometry.boundary_groups(lvl)
for leaf in lvl.leaves():
if dom_mod.is_outside(leaf) and not dom_mod.is_covered(leaf):
continue
if fit.quality_uncrinkliness(leaf, graphs[li], groups) >= fitness.FAIL_THRESHOLD:
continue # not a failure
out[fit.usage_of(leaf) or f"<generic {leaf.type}>"] += 1
return out
def report(label: str, tally: collections.Counter) -> tuple[int, int]:
total = sum(tally.values())
ssz: crinkliness is declared per space; there is no daylight attribute Owner's ruling, and it corrects the design not just the classification: the daylight requirement is already defined in the crinkliness. The gaussian's compact side IS "too little exposed wall per unit floor"; its exposed side is envelope cost. 38.9's proposed daylight: axis was redundant, and keying it off usage: was worse than redundant. What was actually missing: crinkliness is the only leaf quality factor with no per-space target. size, width and proportion are all declared by the space; crinkliness was one global number for every room in every building. crinkliness: none -> no minimum-exposure requirement, may be buried crinkliness: [t, s] -> this space's own target key absent -> the global uncrinkliness target, as today `none` clips the factor on the compact side, it does not switch it off: over-exposure is still penalised, because a crinkly leaf costs envelope whatever it holds. A store may be buried; a store may not be a starfish. The mechanism is backward compatible -- an absent key resolves to the global target, so shipping it changes no score. Behaviour changes only where a config declares something, which keeps the objective change visible per programme in config rather than hidden in a default. Owner's classification: everything a person occupies wants a window, WCs and reception/waiting/foyer included; only stores, plant, records and laundry do not. migrate_crinkliness_key.py declared crinkliness: none on 18 corpus spaces. Crinkliness fails 271 -> 243, of which not-defects 136 (50%) -> 108 (44%); the 28 that went are exactly the utility fails. usage_daylight and needs_daylight are removed as mis-keyed, and DAYLIGHT_USAGES with them -- a vocabulary value should exist only where the engine treats it differently. The historical crinkliness_mode modes stay, default off, so 38.6/38.8 remain reproducible. uncrinkliness_circulation is now settable to none like any space, but its default is left unchanged pending a ruling: corridors were not among the groups ruled on and are 63% of the remaining phantom fails. Lint at parity (46); tests 364 passed, same 7 pre-existing fixture failures. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
2026-08-28 22:38:54 +00:00
real = sum(n for u, n in tally.items() if u not in NO_DAYLIGHT
and not u.startswith("<generic"))
ssz: 61% of reported crinkliness fails are not defects; two corrections Correction 1: 38.8's A/B scores every arm under stock urb, which is only sound if stock is ground truth -- and stock is what this section shows is miscalibrated. Scoring the repair under the objective it repairs penalises it for repairing. That result is not evidence against usage_daylight. The measurement that matters needs no search: are the emitted failures true? audit_crinkliness_truth.py classifies every stock crinkliness fail by the leaf's declared usage. 164 of 271 (61%) are reported against spaces that want no daylight -- stores, WCs, plant, corridors, covered courtyards -- and it holds on evolved artefacts (maple generated.dom: 64%), not just constructed seeds. Under value *= 0.5 ** len(failures) each one halves the fitness of a design that did nothing wrong. Correction 2: usage: is the wrong key, and usage_daylight as written mis-keys it. 39.7 established usage: as an ACCESS-requirement class. "Needs no special access" and "needs no window" are different questions, and the corpus separates them: usage: none is Waiting Room, Reception, Reception Office, Entrance Foyer -- a waiting room plainly wants daylight and DAYLIGHT_USAGES exempts it. The usage: bedroom half is right, but by luck of where the access axis happened to fall. The audit is robust to that error (61% -> 57% if none is reclassified), so the finding stands; the design does not. Daylight needs its own declared per-space axis, decided by the programme author as usage: was. usage_daylight stays default off and is not the shipping fix. The compact-side clip is the right shape for the factor, so the mechanism is retained pending a daylight: attribute to key it on. Higher-power A/B stopped: it was chasing a score against a flawed objective with a mis-keyed fix. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
2026-08-26 18:07:01 +00:00
print(f"=== {label}: {total} crinkliness fails")
if not total:
print(" none\n")
return 0, 0
for usage, n in tally.most_common():
ssz: crinkliness is declared per space; there is no daylight attribute Owner's ruling, and it corrects the design not just the classification: the daylight requirement is already defined in the crinkliness. The gaussian's compact side IS "too little exposed wall per unit floor"; its exposed side is envelope cost. 38.9's proposed daylight: axis was redundant, and keying it off usage: was worse than redundant. What was actually missing: crinkliness is the only leaf quality factor with no per-space target. size, width and proportion are all declared by the space; crinkliness was one global number for every room in every building. crinkliness: none -> no minimum-exposure requirement, may be buried crinkliness: [t, s] -> this space's own target key absent -> the global uncrinkliness target, as today `none` clips the factor on the compact side, it does not switch it off: over-exposure is still penalised, because a crinkly leaf costs envelope whatever it holds. A store may be buried; a store may not be a starfish. The mechanism is backward compatible -- an absent key resolves to the global target, so shipping it changes no score. Behaviour changes only where a config declares something, which keeps the objective change visible per programme in config rather than hidden in a default. Owner's classification: everything a person occupies wants a window, WCs and reception/waiting/foyer included; only stores, plant, records and laundry do not. migrate_crinkliness_key.py declared crinkliness: none on 18 corpus spaces. Crinkliness fails 271 -> 243, of which not-defects 136 (50%) -> 108 (44%); the 28 that went are exactly the utility fails. usage_daylight and needs_daylight are removed as mis-keyed, and DAYLIGHT_USAGES with them -- a vocabulary value should exist only where the engine treats it differently. The historical crinkliness_mode modes stay, default off, so 38.6/38.8 remain reproducible. uncrinkliness_circulation is now settable to none like any space, but its default is left unchanged pending a ruling: corridors were not among the groups ruled on and are 63% of the remaining phantom fails. Lint at parity (46); tests 364 passed, same 7 pre-existing fixture failures. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
2026-08-28 22:38:54 +00:00
exempt = usage in NO_DAYLIGHT or usage.startswith("<generic")
verdict = ("not a defect -- no daylight wanted" if exempt
else "REAL DEFECT")
ssz: 61% of reported crinkliness fails are not defects; two corrections Correction 1: 38.8's A/B scores every arm under stock urb, which is only sound if stock is ground truth -- and stock is what this section shows is miscalibrated. Scoring the repair under the objective it repairs penalises it for repairing. That result is not evidence against usage_daylight. The measurement that matters needs no search: are the emitted failures true? audit_crinkliness_truth.py classifies every stock crinkliness fail by the leaf's declared usage. 164 of 271 (61%) are reported against spaces that want no daylight -- stores, WCs, plant, corridors, covered courtyards -- and it holds on evolved artefacts (maple generated.dom: 64%), not just constructed seeds. Under value *= 0.5 ** len(failures) each one halves the fitness of a design that did nothing wrong. Correction 2: usage: is the wrong key, and usage_daylight as written mis-keys it. 39.7 established usage: as an ACCESS-requirement class. "Needs no special access" and "needs no window" are different questions, and the corpus separates them: usage: none is Waiting Room, Reception, Reception Office, Entrance Foyer -- a waiting room plainly wants daylight and DAYLIGHT_USAGES exempts it. The usage: bedroom half is right, but by luck of where the access axis happened to fall. The audit is robust to that error (61% -> 57% if none is reclassified), so the finding stands; the design does not. Daylight needs its own declared per-space axis, decided by the programme author as usage: was. usage_daylight stays default off and is not the shipping fix. The compact-side clip is the right shape for the factor, so the mechanism is retained pending a daylight: attribute to key it on. Higher-power A/B stopped: it was chasing a score against a flawed objective with a mis-keyed fix. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
2026-08-26 18:07:01 +00:00
print(f" {usage:<18}{n:>4} {verdict}")
print(f" -> {total - real}/{total} ({100 * (total - real) / total:.0f}%) "
f"are reported against spaces that do not want daylight\n")
return real, total
def main() -> None:
ap = argparse.ArgumentParser(description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument("--seeds", type=int, default=3)
ap.add_argument("--corpus", nargs="+", default=CORPUS)
ap.add_argument("--dom", nargs="*", default=[],
help="also audit these evolved .dom files (programme dir inferred)")
args = ap.parse_args()
print("Stock objective. Every fail classified by the leaf's declared usage.\n")
grand_real = grand_total = 0
for progdir in args.corpus:
fit = stock_fitness(progdir)
tally: collections.Counter = collections.Counter()
for s in range(args.seeds):
tally += audit(fit, constructed(progdir, s))
r, t = report(f"{Path(progdir).name} ({args.seeds} constructed seeds)", tally)
grand_real += r
grand_total += t
for dom_path in args.dom:
progdir = str(Path(dom_path).parent)
fit = stock_fitness(progdir)
r, t = report(f"{dom_path} (evolved)", audit(fit, dom_mod.load(dom_path)))
grand_real += r
grand_total += t
if grand_total:
print(f"OVERALL: {grand_total - grand_real}/{grand_total} "
f"({100 * (grand_total - grand_real) / grand_total:.0f}%) of the "
f"crinkliness failures the objective reports are not defects.")
if __name__ == "__main__":
main()