From e09221051c154e44b3d633bef197d7bf0f74821e Mon Sep 17 00:00:00 2001 From: Bruno Postle Date: Mon, 29 Jun 2026 22:56:23 +0100 Subject: [PATCH] =?UTF-8?q?6zy/=C2=A711.8:=20co-tune=20topology=20diversit?= =?UTF-8?q?y=20=C3=97=20tournament=20pressure=20=E2=80=94=20null=20robust?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Expose tournament_k (default 2) on search()/search_staged(), threaded into both _tournament call sites and the staged path's internal search() calls; HOMEMAKER_TOURNAMENT_K env knob in the scaled/staged harnesses; run_6zy_ab.sh joint niche×k grid (RESUME-able). Result (negative, acceptable): no (niche,k) cell beats the legacy (off,k=2) baseline. Blank-slate programme-house (5 seeds) baseline mean 4.80 fails is the best of the 6-cell grid; every k>2 and every niche=on cell is 6.0-7.0. Niching bites (pop_distinct 16/16 vs 4-11) but sharper pressure does not convert it to lower fails — §11.5 'diffuses effort' null is robust to selection pressure; plateau stays reachability-bound (confirms §11.4/§11.5). Co-Authored-By: Claude Opus 4.8 --- DESIGN.md | 75 +++++++++++++++++++++++++++++ experiments/run_6zy_ab.sh | 81 ++++++++++++++++++++++++++++++++ experiments/run_search_scaled.py | 4 ++ experiments/run_staged_search.py | 3 ++ src/homemaker_layout/driver.py | 14 ++++-- 5 files changed, 173 insertions(+), 4 deletions(-) create mode 100755 experiments/run_6zy_ab.sh diff --git a/DESIGN.md b/DESIGN.md index fa8f5ad..0125119 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -1108,6 +1108,81 @@ core as `fixed_circ`; threaded through `search_staged(seed_adjacency_aware=True) denser, more-circulation layouts), which is the canonical-encoding / shape-feasibility territory of `homemaker-py-9gp`. +### 11.8 Topology diversity × selection pressure, co-tuned (`homemaker-py-6zy`) — DONE (negative) + +Premise (loose end from §11.5): structural niching was A/B'd against the legacy +fitness-scalar dedup with selection pressure **held fixed** at a binary tournament +(`driver._tournament`, `k=2`). §11.5's own mechanism note named the coupling as +the reason for its null — "Maximal diversity (16/16) with the fixed tournament +pressure just **diffuses** effort" — i.e. diversity and pressure are coupled but +were varied as if independent: niching widens the population, but `k` was never +**sharpened** to convert the extra exploration back into exploitation. This issue +isolates that coupling — sweep tournament size jointly with niching to test +whether sharper selection turns the 16/16 structural diversity into lower fails. +The project had already pivoted to the canonical encoding (`homemaker-py-9gp`); +this is a falsification check so the lever is not silently lost, not an expected +win (§11.4/§11.5 both located the plateau in **reachability**). + +**Implementation (knob only; default-off behaviour unchanged).** Exposed +`tournament_k: int = 2` on `search` / `search_staged`, threaded into both +`_tournament` call sites (crossover pair + mutation parent) and all three internal +`search()` calls of the staged path; reuses the §11.5 `genome.signature` / +`niche_by_signature` machinery unchanged. The experiments harness reads +`HOMEMAKER_TOURNAMENT_K` (mirrors `NICHE`) in `run_search_scaled.py` / +`run_staged_search.py`; `experiments/run_6zy_ab.sh` runs the joint grid (RESUME-able). + +- *Commands (reproduce, `URB_NO_OCCLUSION=1`, 20000 evals; blank-slate seed + `init.dom` to match §11.5):* + ```bash + # grid: NICHE ∈ {0,1} × HOMEMAKER_TOURNAMENT_K ∈ {2,3,4} + NICHE=0 HOMEMAKER_TOURNAMENT_K=2 python3 experiments/run_search_scaled.py \ + examples/programme-house 20000 examples/programme-house/init.dom scratch/out.dom + # harbor (staged): run_staged_search.py, seed examples/harbor-house/init.dom + bash experiments/run_6zy_ab.sh # full grid → scratch/6zy/summary.tsv + ``` + +- *Diversity (mechanism check) — confirmed biting.* `niche=on` holds the final + population at **16/16** distinct topologies at every `k`; `niche=off` sits at + **4–11/16**. The pressure knob is genuinely varied (`k`=2,3,4). So both levers + are live — the null below is not a machinery artefact. + +- *Fail count (the gate) — no cell beats the baseline.* Blank-slate + programme-house, total fails at budget over **5 seeds** (0–4), mean (sd): + + | niche \ k | k=2 | k=3 | k=4 | + |:---------:|:-----------:|:-----------:|:-----------:| + | **off** | **4.80** (1.60) | 6.40 (2.50) | 6.00 (2.00) | + | **on** | 6.20 (1.72) | 7.00 (1.41) | 6.60 (1.85) | + + The legacy `(off, k=2)` cell is the **best** of the six (4.80); every + higher-pressure row and every `niche=on` row is equal-or-worse (6.0–7.0). All + differences sit within ~1 sd at 5 seeds, so the grid is a wash — but the central + tendency is unambiguous: sharpening `k` and adding niching both *slightly hurt*, + the opposite of the rescue the premise hypothesised. Harbor-house (staged, seed + 0) reinforces it: `niche=on` is uniformly worse than `off` at every `k` + (k2 72→83, k3 77→82, k4 67→75); within the `niche=on` row higher `k` helps + monotonically (83→82→75) but never catches the `niche=off` row, and the best + cell overall (`off, k=4` = 67) is a single-seed wiggle within noise of the + `off, k=2` = 72 baseline. + +- *Why it fails — the coupling is real but points the wrong way.* Sharper + selection does **not** convert the extra structural diversity into lower fails; + if anything the 16/16 niched population at high `k` over-commits the + larger spread to a handful of basins and loses the occasional lucky low-fail + draw the smaller fitness-scalar population stumbles into. §11.5's "diffuses + effort" diagnosis survives co-tuning: the bottleneck is **reachability** + (operators + encoding cannot reach the low-fail basins), so reshaping + selection/population pressure cannot recover what the search space does not + expose — the same conclusion §11.4 reached from the comparator side and §11.5 + from the diversity side. + +- *Verdict: §11.5 null is robust to selection pressure — reject `k>2` and niching + as defaults; binary tournament + fitness-scalar dedup stand.* `tournament_k` is + kept (default-2) as a reusable knob alongside `niche_by_signature`. With + §11.4/§11.5/§11.8 all negative on the outer loop, the residual is confirmed + structural: the principled lever is the canonical encoding + richer topology + operators (`homemaker-py-9gp`), not selection or population management. + ## 12. Phase 7 — scaling validation & residual reduction (post-c4c) **Epic:** `homemaker-py-leu`. **Status:** opened 2026-06-19. Continuation of the diff --git a/experiments/run_6zy_ab.sh b/experiments/run_6zy_ab.sh new file mode 100755 index 0000000..4bb04b5 --- /dev/null +++ b/experiments/run_6zy_ab.sh @@ -0,0 +1,81 @@ +#!/usr/bin/env bash +# 6zy joint A/B: topology diversity (structural niching) x selection pressure +# (tournament size k). §11.5 raised diversity to 16/16 but held pressure FIXED at +# k=2; this sweep co-tunes them to test whether sharper selection converts the +# extra structural diversity into lower fails rather than diffusing it. +# +# Grid: NICHE in {0,1} x HOMEMAKER_TOURNAMENT_K in {2,3,4}. +# (niche=0,k=2) is the legacy baseline; (niche=1,k=2) reproduces §11.5's niche +# column; the k in {3,4} rows are the new higher-pressure cells. +# Seeds: programme-house 0..4 (>=5, widened from §11.5's thin 3 to clear seed +# noise); harbor-house staged seed 0. +# Equal native-fitness budget, URB_NO_OCCLUSION=1. Results -> scratch/6zy/summary.tsv +set -u +cd "$(dirname "$0")/.." +OUT=scratch/6zy +mkdir -p "$OUT" +SUMMARY="$OUT/summary.tsv" +# RESUME=1 keeps an existing summary and skips cells already recorded (non-ERR), +# so a killed sweep can finish only its missing cells. Default starts fresh. +if [ "${RESUME:-0}" = 1 ] && [ -s "$SUMMARY" ]; then + echo "RESUME: keeping $(($(wc -l < "$SUMMARY") - 1)) existing rows" +else + : > "$SUMMARY" + printf 'prog\tniche\tk\tseed\tfails\tseen\tpop_distinct\trestarts\ttag\n' >> "$SUMMARY" +fi +BUDGET=${BUDGET:-20000} +PH_SEEDS=${PH_SEEDS:-"0 1 2 3 4"} +HARBOR_SEEDS=${HARBOR_SEEDS:-"0"} +MAXJOBS=${MAXJOBS:-2} +# §11.5 reproduce-commands seed programme-house from init.dom (a bare undivided +# plot) so the run is a true BLANK-SLATE topology search; c964…dom is a finished +# design that would warm-start and floor trivially. Must match §11.5 for the +# direct comparison the acceptance criteria require. +PH_SEED_FILE=examples/programme-house/init.dom + +run_one() { + local prog=$1 harness=$2 niche=$3 k=$4 seed=$5 seedfile=$6 + local tag="${prog}_n${niche}_k${k}_s${seed}" + local log="$OUT/${tag}.log" + if [ "${RESUME:-0}" = 1 ] && grep -qP "\t${tag}\$" "$SUMMARY" 2>/dev/null \ + && ! grep -qP "\tERR\t.*\t${tag}\$" "$SUMMARY" 2>/dev/null; then + echo "SKIP $tag (already recorded)" + return + fi + URB_NO_OCCLUSION=1 NICHE=$niche HOMEMAKER_TOURNAMENT_K=$k \ + python3 "experiments/$harness" \ + "examples/$prog" "$BUDGET" "$seed" "$seedfile" \ + "$OUT/${tag}.dom" > "$log" 2>&1 + local fails seen popd restarts + fails=$(grep -oE 'best *: [0-9.eE+-]+ \([0-9]+ fails\)' "$log" | grep -oE '[0-9]+ fails' | grep -oE '[0-9]+' | tail -1) + seen=$(grep -oE 'diversity : [0-9]+ distinct topologies seen' "$log" | grep -oE '[0-9]+' | tail -1) + popd=$(grep -oE '[0-9]+/[0-9]+ distinct in final population' "$log" | head -1 | grep -oE '^[0-9]+') + restarts=$(grep -oE '[0-9]+ restarts' "$log" | grep -oE '[0-9]+' | tail -1) + printf '%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\n' \ + "$prog" "$niche" "$k" "$seed" "${fails:-ERR}" "${seen:-?}" "${popd:-?}" "${restarts:-?}" "$tag" >> "$SUMMARY" + echo "DONE $tag -> ${fails:-ERR} fails (seen=${seen:-?}, popd=${popd:-?})" +} + +JOBS=() +for niche in 0 1; do + for k in 2 3 4; do + for seed in $PH_SEEDS; do + JOBS+=("programme-house|run_search_scaled.py|$niche|$k|$seed|$PH_SEED_FILE") + done + for seed in $HARBOR_SEEDS; do + JOBS+=("harbor-house|run_staged_search.py|$niche|$k|$seed|examples/harbor-house/init.dom") + done + done +done + +echo "queued ${#JOBS[@]} jobs, budget=$BUDGET, maxjobs=$MAXJOBS" +for job in "${JOBS[@]}"; do + IFS='|' read -r prog harness niche k seed seedfile <<< "$job" + while [ "$(jobs -rp | wc -l)" -ge "$MAXJOBS" ]; do wait -n; done + run_one "$prog" "$harness" "$niche" "$k" "$seed" "$seedfile" & +done +wait +echo "ALL DONE" +# keep header first, sort the rest +{ head -1 "$SUMMARY"; tail -n +2 "$SUMMARY" | sort; } > "$SUMMARY.tmp" && mv "$SUMMARY.tmp" "$SUMMARY" +cat "$SUMMARY" diff --git a/experiments/run_search_scaled.py b/experiments/run_search_scaled.py index 4e06edd..d517faa 100644 --- a/experiments/run_search_scaled.py +++ b/experiments/run_search_scaled.py @@ -71,6 +71,8 @@ def main() -> int: # fitness-scalar dedup (the "before"); RESTART_PATIENCE= enables soft # restarts (default off). niche = os.environ.get("NICHE", "0") == "1" + # 6zy: tournament size (selection pressure), default k=2 (legacy binary). + tournament_k = int(os.environ.get("HOMEMAKER_TOURNAMENT_K", "2")) rp = os.environ.get("RESTART_PATIENCE") restart_patience = int(rp) if rp else None adj = os.environ.get("ADJ", "1") == "1" # s44 adjacency-aware seeding @@ -82,6 +84,7 @@ def main() -> int: print(f"rng seed : {rng_seed}") print(f"use_grade : {use_grade}") print(f"niche : {niche}") + print(f"tourn_k : {tournament_k}") print(f"restart_p : {restart_patience}") print(f"adj_aware : {adj}") print(f"prop_aware: {prop}") @@ -101,6 +104,7 @@ def main() -> int: seed=rng_seed, log=lambda m: print(m, flush=True), use_grade=use_grade, + tournament_k=tournament_k, niche_by_signature=niche, restart_patience=restart_patience, seed_adjacency_aware=adj, diff --git a/experiments/run_staged_search.py b/experiments/run_staged_search.py index 79d8d28..c758e84 100644 --- a/experiments/run_staged_search.py +++ b/experiments/run_staged_search.py @@ -53,6 +53,7 @@ def main() -> int: use_grade = os.environ.get("USE_GRADE") == "1" # §11.4 graded objective A/B niche = os.environ.get("NICHE", "0") == "1" # §11.5 structural niching A/B + tournament_k = int(os.environ.get("HOMEMAKER_TOURNAMENT_K", "2")) # 6zy pressure knob rp = os.environ.get("RESTART_PATIENCE") restart_patience = int(rp) if rp else None adj = os.environ.get("ADJ", "1") == "1" # s44/ld5 adjacency-aware seeding A/B @@ -96,6 +97,7 @@ def main() -> int: print(f"rng seed : {rng_seed}") print(f"use_grade : {use_grade}") print(f"niche : {niche}") + print(f"tourn_k : {tournament_k}") print(f"restart_p : {restart_patience}") print(f"adj_aware : {adj}") print(f"prop_aware: {prop}") @@ -123,6 +125,7 @@ def main() -> int: seed=rng_seed, log=lambda m: print(m, flush=True), use_grade=use_grade, + tournament_k=tournament_k, niche_by_signature=niche, restart_patience=restart_patience, seed_adjacency_aware=adj, diff --git a/src/homemaker_layout/driver.py b/src/homemaker_layout/driver.py index b13d8a1..c26a65f 100644 --- a/src/homemaker_layout/driver.py +++ b/src/homemaker_layout/driver.py @@ -187,6 +187,7 @@ def search( base_p: float = 1.0, child_probe=None, use_grade: bool = False, + tournament_k: int = 2, niche_by_signature: bool = False, restart_patience: int | None = None, restart_elite: int = 1, @@ -478,13 +479,14 @@ def search( tasks = [] for _ in range(batch_n): if len(pop) >= 2 and rng.random() < p_crossover: - a, b = _tournament(pop, rng, _key), _tournament(pop, rng, _key) + a, b = (_tournament(pop, rng, _key, k=tournament_k), + _tournament(pop, rng, _key, k=tournament_k)) child_root, _, desc = operators.crossover(a.root, b.root, rng) if child_probe is not None: desc = f"{desc}|pf={a.n_fails},{b.n_fails}" ratios = {**b.ratios, **a.ratios} # primary parent wins else: - parent = _tournament(pop, rng, _key) + parent = _tournament(pop, rng, _key, k=tournament_k) child_root, desc = operators.mutate(parent.root, rng, types, weights=mutation_weights, reqs=reqs, base_p=base_p) @@ -537,6 +539,7 @@ def search_staged( log=None, n_workers: int = 1, use_grade: bool = False, + tournament_k: int = 2, niche_by_signature: bool = False, restart_patience: int | None = None, restart_elite: int = 1, @@ -591,7 +594,8 @@ def search_staged( child_budget=child_budget, seed_budget=seed_budget, p_crossover=p_crossover, seed=seed, types=types, inner_kw=inner_kw, log=log, n_workers=n_workers, - use_grade=use_grade, niche_by_signature=niche_by_signature, + use_grade=use_grade, tournament_k=tournament_k, + niche_by_signature=niche_by_signature, restart_patience=restart_patience, restart_elite=restart_elite, seed_adjacency_aware=seed_adjacency_aware, seed_proportion_aware=seed_proportion_aware, @@ -625,6 +629,7 @@ def search_staged( inner_kw=inner_kw, log=log, n_workers=n_workers, rank_bonus_fn=lambda root: graph.substrate_readiness(root, reqs, n_storeys), rank_bonus_weight=rank_bonus_weight, + tournament_k=tournament_k, niche_by_signature=niche_by_signature, restart_patience=restart_patience, restart_elite=restart_elite, seed_adjacency_aware=seed_adjacency_aware, @@ -670,7 +675,8 @@ def search_staged( # §11.4: the graded objective targets the dense two-floor quality-fail # regime, which is Stage 2. Stage 1 keeps its readiness-biased key so the # substrate-selection semantics (§11.3) are unchanged. - use_grade=use_grade, niche_by_signature=niche_by_signature, + use_grade=use_grade, tournament_k=tournament_k, + niche_by_signature=niche_by_signature, restart_patience=restart_patience, restart_elite=restart_elite, enable_reassociate=enable_reassociate, feasibility_filter=feasibility_filter,