bd: sync issues.jsonl export after 1s3 final close

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R8agJBT2ZpmF3ErW7wi2wY

DESIGN.md

@@ -3459,7 +3459,7 @@ on every real (non-duplicated-code) programme tested at any scale from 6 rooms (
 suggests was measuring the wrong thing. No further follow-up is filed — the room-count hypothesis from
 `f1d` (§23) is now addressed on the diversity axis `xyu` (§31) could not reach.
 
-## 33. Multi-use leaves as a permanent design goal (`homemaker-py-1s3`, §26 path b) — DONE (promising, not a clean win)
+## 33. Multi-use leaves as a permanent design goal (`homemaker-py-1s3`, §26 path b) — DONE (NULL, N=3 signal did not replicate)
 
 **Motivation.** §26 scoped two readings of "multi-use leaves" — a leaf legitimately serving several
 DIFFERENT compatible programme codes at once (study+guest bedroom, kitchen+dining, Stewart Brand's
@@ -3536,24 +3536,50 @@ discarded entirely, see the bead's history):
 (Baseline drifts slightly run-to-run — the staged search's own within-seed run-to-run noise at this
 budget/worker-count, not a bug; each combination's Δ is against its own paired baseline row.)
 
+Among the three, the precision-weighted single-compromise-peak model was the only one to improve BOTH
+programmes at N=3, so it is the one landed in the shipped code (`_clipped_gaussian`/mixture kept in
+`fitness.py`, documented and unit-tested, as a recorded negative alternative). But per the confirmations
+below, this N=3 comparison — used to pick a combination strategy — turned out to be too small a sample to
+trust for the multi_use verdict itself.
+
+**Larger-N confirmation — the N=3 signal did not replicate.** N=3 is a thin sample (§31/§32's own standard
+is N=15), so the precision-weighted result was checked two ways before considering any default-flip
+recommendation:
+
+| test | conditions | harbor-house Δ | health-centre Δ |
+|---|---|---:|---:|
+| original | N=3, staged search, budget 20 000 | **−1.4%** (2W/1L) | **−13.9%** (2W/1L) |
+| confirm #1 | N=15, plain search, budget 3 000 (mirrors `xyu`/`9yx`'s own protocol exactly) | +6.1% worse (5W/10L, p=0.30) | +6.6% worse (3W/11L/1T, Wilcoxon p=**0.044**) |
+| confirm #2 | N=15, staged search, budget 20 000 (**same conditions as the original**, more seeds) | +6.6% worse (4W/11L, p=0.15) | +4.7% worse (6W/9L, p=0.48) |
+
+Confirm #1 uses a cheaper protocol (budget 3000, and for the multi-storey `harbor-house`, plain search
+rather than staged — `search_staged` only falls through to plain search on single-storey programmes) so it
+answers a related but distinct question. Confirm #2 is the true same-conditions replication — identical to
+the original A/B except 15 seeds instead of 3 — and it **also trends negative on both programmes**, though
+neither reaches significance at this N. Two of the three measurements, including the one that actually
+matches the original protocol, disagree with the original finding's *direction*. The honest read: the N=3
+positive result was very likely sampling noise from an unlucky (or lucky) three-seed draw, not a real
+effect — `harbor-house`'s original 2W/1L was already a coin-flip-sized sample, and `health-centre`'s dramatic
+−13.9% at N=3 (driven substantially by one seed swinging from 71→43 fails) did not hold up at N=15 (mean
+Δ flipped to +4.7%, p=0.48 — indistinguishable from no effect).
+
 **Diagnosis.** Leaf-sharing's k×target scaling never changes the SHAPE constraint: k identical rooms share
 one identical width/proportion target, so a shared leaf is exactly as easy or hard to satisfy geometrically
-as any single instance of that code, just bigger. Multi-use fusion is different — combining two
+as any single instance of that code, just bigger. Multi-use fusion is different — the combined leaf's
-potentially-DIFFERENT codes' shape targets is a real modelling choice, and it matters a lot: the naive
+larger area target competes with every other room for the same limited plot area, and (whichever shape
-stricter-of-both hack over-penalises (health-centre +24.5%), the mixture under-constrains (`max()` lets a
+combination is used) the fused leaf's shape constraint is at best as forgiving as either code alone, never
-leaf score 1.0 by satisfying only the WEAKER of the two codes' targets, health-centre +20.4%), and only the
+more so. The mechanism does not appear to reliably pay for this cost the way leaf-sharing's pure count
-precision-weighted single-compromise-peak model improved BOTH programmes. `harbor-house` (fewer, larger
+relaxation does — consistent with the broader pattern in this log (§11.4/11.5, §14, §16, §21, §22, §26,
-rooms, more slack per leaf) tolerates all three combinations reasonably; `health-centre` (19 distinct codes
+§27, §30) that search-machinery/fitness-shaping-adjacent levers rarely move the needle, and that small-N
-packed into a small footprint, the exact stress case §32 was built to probe) is where the combination choice
+results in this problem class need real confirmation before being trusted (the same lesson `y51`/`xyu`/`9yx`,
-swings the result by ~38 points of relative fail count (+24.5% to −13.9%) — the shape-combination model is
+§31/§32, already taught once).
-not a minor implementation detail here, it is close to the whole story.
 
-**Status.** Landed with the precision-weighted combination (best of the three tried); `_clipped_gaussian` and
+**Status.** `multi_use` stays default **OFF** and is not recommended even as a promising candidate — the
-the mixture pattern are kept in `fitness.py`, documented and unit-tested, as a recorded negative alternative.
+larger-N evidence points toward NULL-to-mildly-negative rather than positive. The mechanism itself (declared
-`multi_use` stays default **OFF**: the precision-weighted result is genuinely promising — both programmes
+`co_locate` pairs, `graph.leaf_codes()` resolver, precision-weighted shape combination, construction-time
-improve on average (−1.4% / −13.9%) — but it is not the clean sweep §13.3's default-flip bar set (leaf-sharing:
+fusion) is complete, fully tested (335/335 passing, `tests/test_multi_use.py`), gated OFF by default and
-*every* share run beat *every* baseline run). Here harbor-house loses on 1 of 3 seeds (101→117) and only 3
+bit-identical when off, so it remains available if a future architect wants to opt a specific programme into
-seeds per arm were run, so this is a real but statistically thin signal, closer to §31's "weak but not
+it manually despite the null aggregate result — but no further investment (default flip, additional
-evaporated" than to §13.3's "total separation". Not filed as an immediate follow-up bead given the compute
+combination strategies, or a larger sweep) is planned. This closes out `homemaker-py-1s3` and, with it, both
-cost of another A/B (~2h per combination tried), but a promising candidate for a larger-N confirmation
+halves of §26's original multi-use-leaves question: path (a) (search relaxation) was NULL/NEGATIVE, path (b)
-(mirroring `xyu`/`9yx`'s N=15 pattern) if revisited — the mechanism itself is complete, tested, and ready.
+(permanent fusion) is NULL after replication.

experiments/run_1s3_confirm.sh

@@ -0,0 +1,51 @@
+#!/usr/bin/env bash
+# homemaker-py-1s3 larger-N confirmation (DESIGN.md §33 follow-up): the full
+# 20k-eval staged A/B (N=3 seeds) found the precision-weighted multi_use
+# combination improves both example programmes (harbor-house -1.4%,
+# health-centre -13.9%) but wasn't the clean sweep needed for a default
+# flip -- harbor-house lost 1/3 seeds. Mirrors xyu/9yx's own larger-N
+# confirmation protocol EXACTLY (DESIGN.md §31/§32,
+# experiments/run_xyu_sweep.sh / run_9yx_sweep.sh): plain `homemaker-evolve`
+# (not staged search), budget=3000, 4 workers, N=15 seeds, finish-time
+# --collapse default. Cheaper per-run than the staged A/B so N=15 is
+# tractable in a fraction of the time.
+#
+# Usage: experiments/run_1s3_confirm.sh
+set -u
+cd "$(dirname "$0")/.."
+
+WORKERS=4
+BUDGET=3000
+SEEDS="1 2 3 4 5 6 7 8 9 10 11 12 13 14 15"
+OUT=scratch/1s3_confirm; mkdir -p "$OUT"
+TSV=scratch/1s3_confirm_results.tsv
+[ -f "$TSV" ] || printf 'programme\tseed\tmulti_use\tbudget\tfails\tfitness\telapsed_s\n' > "$TSV"
+
+run() {  # programme seed multi_use(0|1)
+  local prog="$1" seed="$2" mu="$3"
+  local tag="mu${mu}"
+  local dom="$OUT/${prog}_${tag}_s${seed}.dom"
+  local log="$OUT/${prog}_${tag}_s${seed}.log"
+  local flag="--no-multi-use"; [ "$mu" = 1 ] && flag="--multi-use"
+  echo ">>> $prog seed=$seed multi_use=$mu budget=$BUDGET"
+  local t0; t0=$(date +%s)
+  homemaker-evolve "examples/$prog/init.dom" \
+    --budget "$BUDGET" --workers "$WORKERS" --seed "$seed" \
+    $flag --output "$dom" > "$log" 2>&1
+  local t1; t1=$(date +%s)
+  local fitness fails
+  fitness=$(sed -n 's/^best *: \([0-9.e+-]*\) .*/\1/p' "$log")
+  fails=$(sed -n 's/^best *: [0-9.e+-]* (\([0-9]*\) fails).*/\1/p' "$log")
+  ( cd "examples/$prog" && homemaker-fitness "$(realpath "../../$dom")" > /dev/null 2>&1 )
+  printf '%s\t%s\t%s\t%s\t%s\t%s\t%s\n' \
+    "$prog" "$seed" "$mu" "$BUDGET" "${fails:-ERR}" "${fitness:-ERR}" "$((t1-t0))" >> "$TSV"
+  echo "    -> ${fails:-ERR} fails, fitness=${fitness:-ERR}, $((t1-t0))s"
+}
+
+for prog in harbor-house health-centre; do
+  for seed in $SEEDS; do run "$prog" "$seed" 0; done
+  for seed in $SEEDS; do run "$prog" "$seed" 1; done
+done
+
+echo "=== 1s3 multi_use N=15 confirmation complete ==="
+column -t -s $'\t' "$TSV"

experiments/run_1s3_confirm_staged.sh

@@ -0,0 +1,46 @@
+#!/usr/bin/env bash
+# homemaker-py-1s3 larger-N confirmation, SAME conditions (DESIGN.md §33):
+# the N=15/budget=3000/plain-search confirmation (run_1s3_confirm.sh) came
+# back negative on both programmes, but that protocol differs from the one
+# the original positive result was measured under (staged search, budget
+# 20000 -- run_multiuse_ab.sh, N=3) in both budget AND, for the multi-storey
+# harbor-house, search algorithm (search_staged falls back to plain search
+# only on single-storey programmes). This script is the true same-conditions
+# confirmation: identical to run_multiuse_ab.sh (staged, 20000, 4 workers)
+# but N=15 seeds instead of 3, so it can actually confirm or refute the
+# original finding rather than answering a different question. Expensive
+# (~10h sequential at this budget/seed count) -- run in background.
+#
+# Usage: experiments/run_1s3_confirm_staged.sh
+set -u
+cd "$(dirname "$0")/.."
+BUDGET="${1:-20000}"
+SEEDS="${SEEDS:-0 1 2 3 4 5 6 7 8 9 10 11 12 13 14}"
+OUT=scratch/1s3_confirm_staged; mkdir -p "$OUT"
+TSV=scratch/1s3_confirm_staged_results.tsv
+[ -f "$TSV" ] || printf 'programme\tseed\tmulti_use\tfails\ttopologies\telapsed_s\n' > "$TSV"
+
+run() {  # programme seed multi_use(0|1)
+  local prog="$1" seed="$2" mu="$3"
+  local tag="mu${mu}"
+  local log="$OUT/${prog}_${tag}_s${seed}.log"
+  echo ">>> $prog seed=$seed multi_use=$mu"
+  local t0; t0=$(date +%s)
+  env URB_NO_OCCLUSION=1 MULTIUSE="$mu" WORKERS="${WORKERS:-4}" \
+    python3 experiments/run_staged_search.py "examples/$prog" "$BUDGET" "$seed" \
+      "examples/$prog/init.dom" "$OUT/${prog}_${tag}_s${seed}.dom" > "$log" 2>&1
+  local t1; t1=$(date +%s)
+  local fails topos
+  fails=$(grep 're-scored (native)' "$log" | tail -1 | sed -n 's/.*(\([0-9]*\) fails).*/\1/p')
+  topos=$(grep -m1 '^evals' "$log" | sed -n 's/.*across \([0-9]*\) topologies.*/\1/p')
+  printf '%s\t%s\t%s\t%s\t%s\t%s\n' "$prog" "$seed" "$mu" "${fails:-ERR}" "${topos:-?}" "$((t1-t0))" >> "$TSV"
+  echo "    -> ${fails:-ERR} fails, ${topos:-?} topologies, $((t1-t0))s"
+}
+
+for prog in harbor-house health-centre; do
+  for seed in $SEEDS; do run "$prog" "$seed" 0; done
+  for seed in $SEEDS; do run "$prog" "$seed" 1; done
+done
+
+echo "=== 1s3 staged-conditions N=15 confirmation complete ==="
+column -t -s $'\t' "$TSV"