Compare commits
No commits in common. "d125d2f19ca8e7df2f6c52629c187aabff89874b" and "baa9109c6758a8870729e375050759501ee70175" have entirely different histories.
d125d2f19c
...
baa9109c67
3 changed files with 27 additions and 114 deletions
File diff suppressed because one or more lines are too long
42
DESIGN.md
42
DESIGN.md
|
|
@ -2415,7 +2415,7 @@ gated on the 9o5 landscape-flattening risk (§13 / `homemaker-py-xi7`) and its o
|
||||||
Tests: `tests/test_collapse_global.py` ×6 (demand-set relabel, level hard constraint, c/o/s
|
Tests: `tests/test_collapse_global.py` ×6 (demand-set relabel, level hard constraint, c/o/s
|
||||||
exclusion, no-op safety, keep-better/unmerged); 267 pass.
|
exclusion, no-op safety, keep-better/unmerged); 267 pass.
|
||||||
|
|
||||||
## 18. Graded circulation-connectivity signal (`homemaker-py-qi6`) — DONE (negative)
|
## 18. Graded circulation-connectivity signal (`homemaker-py-qi6`) — in progress
|
||||||
|
|
||||||
**Motivation — the binary fail is flat.** After the §17 collapse, the residual fails on the
|
**Motivation — the binary fail is flat.** After the §17 collapse, the residual fails on the
|
||||||
harbor-house set are dominated by `level N not connected` (2 of the best layout's 12; also on
|
harbor-house set are dominated by `level N not connected` (2 of the best layout's 12; also on
|
||||||
|
|
@ -2458,39 +2458,13 @@ the gradient toward connected topologies.
|
||||||
(env `HOMEMAKER_CONN_GRADE`, default OFF); the grade is read off the optimised tree, one extra
|
(env `HOMEMAKER_CONN_GRADE`, default OFF); the grade is read off the optimised tree, one extra
|
||||||
native eval per child.
|
native eval per child.
|
||||||
|
|
||||||
**Build.** Signal, fitness wiring, CLI, and 9 tests landed (`tests/test_conn_grade.py`: pure-graph
|
**Status / next.** Signal, fitness wiring, CLI, and 9 tests landed (`tests/test_conn_grade.py`:
|
||||||
fraction contract, non-circ cells ignored, monotone under (dis)connection, and the score/fail-
|
pure-graph fraction contract, non-circ cells ignored, monotone under (dis)connection, and the
|
||||||
count-invariance of the flag). 276 tests pass.
|
score/fail-count-invariance of the flag). The A/B — does the gradient actually pull evolve runs
|
||||||
|
toward connected circulation and clear `not connected` fails — needs full-budget runs and is
|
||||||
**A/B verdict (measured, 2026-07-22, qpk protocol, `experiments/run_qi6_ab.sh`) — NEGATIVE.**
|
pending (short 60-eval smoke run confirms the plumbing only). If the graded key alone is
|
||||||
Equal-budget `conn_grade` ON vs OFF, both arms finished with the standard finish-time `--collapse`
|
insufficient, the follow-on is an insert/relocate-circulation mutation operator (mechanism (a),
|
||||||
(94g), 4 workers, canonical `homemaker-fitness` re-score for the `.fails` breakdown:
|
still `homemaker-py-qi6`) that now has a gradient to climb. 276 tests pass.
|
||||||
|
|
||||||
- **harbor-house** (`init.dom`, budget 2500, seeds 1–3): **byte-identical output** in every seed
|
|
||||||
(dom, fail list, fitness all diff-clean ON vs OFF) — the secondary comparator key never fired,
|
|
||||||
i.e. the search trajectory never actually hit a tie at fail-count that the grade could break.
|
|
||||||
This is the programme §18 was motivated on (2 of 15 fails on the best layout are `not
|
|
||||||
connected`), and the signal moved nothing.
|
|
||||||
- **programme-house** (`init.dom`, budget 3000, seeds 1–5): 3/5 seeds tie exactly (byte-identical
|
|
||||||
`.fails`); seeds 1 and 2 diverge to a **different topology** with one fewer total fail (8→7
|
|
||||||
each) — but the diff is entirely adjacency/crinkliness/width/access/size fails, not
|
|
||||||
connectivity. In all 4 seed-arms across both programmes where a `not connected` fail was
|
|
||||||
actually present (harbor 1&3, programme 3&4), the fail is **unchanged** in both arms — zero
|
|
||||||
cases of the grade clearing one.
|
|
||||||
- **Conclusion: the grade does not do what §18 designed it to do.** It occasionally perturbs
|
|
||||||
tie-breaking among equal-fail-count neighbours (programme-house seeds 1/2), which can
|
|
||||||
incidentally shift the total fail count, but that perturbation never targets circulation
|
|
||||||
connectivity specifically — consistent with a comparator key that is either too weak relative
|
|
||||||
to the primary `(-n_fails, fitness)` keys to steer topology choice, or whose grade values are
|
|
||||||
rarely distinct enough between the actual neighbours the search compares to break a tie in the
|
|
||||||
intended direction.
|
|
||||||
|
|
||||||
**Status / next.** Kept default OFF (already was). Mechanism (b) (graded proximity as a tertiary
|
|
||||||
key) is falsified by this A/B, not just unconfirmed — do not re-attempt without a different
|
|
||||||
mechanism. The remaining candidate from the original issue is mechanism (a): an explicit
|
|
||||||
insert/relocate-circulation mutation/repair operator, which does not depend on the search
|
|
||||||
stumbling onto a fail-count tie to act. Not started; low priority per DISCOVERED-FROM epic
|
|
||||||
`homemaker-py-94g`'s framing (fitness fidelity, not search capability).
|
|
||||||
|
|
||||||
## 19. Geometry/topology repair for shape-intrinsic fails (`homemaker-py-7fm`) — DONE (negative)
|
## 19. Geometry/topology repair for shape-intrinsic fails (`homemaker-py-7fm`) — DONE (negative)
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,60 +0,0 @@
|
||||||
#!/usr/bin/env bash
|
|
||||||
# qi6 A/B (DESIGN.md §18): does the graded circulation-connectivity signal
|
|
||||||
# actually pull full-budget evolve runs toward connected circulation and clear
|
|
||||||
# 'level N not connected' fails, or is the secondary-comparator gradient too
|
|
||||||
# weak to matter (mirrors the qpk §20 protocol: equal-budget ON vs OFF, both
|
|
||||||
# arms finished with the standard finish-time --collapse (94g) so the
|
|
||||||
# comparison is apples-to-apples on the final collapsed score)?
|
|
||||||
#
|
|
||||||
# Authoritative metrics: total fail count and 'not connected' fail count, read
|
|
||||||
# from the .fails file homemaker-fitness writes (evolve.py itself only prints
|
|
||||||
# n_fails, not the fail list). Each run appends one TSV row so partial results
|
|
||||||
# survive an interrupt.
|
|
||||||
#
|
|
||||||
# Usage: experiments/run_qi6_ab.sh
|
|
||||||
set -u
|
|
||||||
cd "$(dirname "$0")/.."
|
|
||||||
|
|
||||||
WORKERS=4
|
|
||||||
OUT=scratch/qi6_ab; mkdir -p "$OUT"
|
|
||||||
TSV=scratch/qi6_ab_results.tsv
|
|
||||||
[ -f "$TSV" ] || printf 'programme\tseed\tconn_grade\tbudget\tfails\tnot_connected\tfitness\telapsed_s\n' > "$TSV"
|
|
||||||
|
|
||||||
run() { # programme seed conn_grade(0|1) budget
|
|
||||||
local prog="$1" seed="$2" cg="$3" budget="$4"
|
|
||||||
local tag="cg${cg}"
|
|
||||||
local dom="$OUT/${prog}_${tag}_s${seed}.dom"
|
|
||||||
local log="$OUT/${prog}_${tag}_s${seed}.log"
|
|
||||||
local flag="--no-conn-grade"; [ "$cg" = 1 ] && flag="--conn-grade"
|
|
||||||
echo ">>> $prog seed=$seed conn_grade=$cg budget=$budget"
|
|
||||||
local t0; t0=$(date +%s)
|
|
||||||
homemaker-evolve "examples/$prog/init.dom" \
|
|
||||||
--budget "$budget" --workers "$WORKERS" --seed "$seed" \
|
|
||||||
$flag --output "$dom" > "$log" 2>&1
|
|
||||||
local t1; t1=$(date +%s)
|
|
||||||
local fitness fails notconn
|
|
||||||
fitness=$(sed -n 's/^best *: \([0-9.e+-]*\) .*/\1/p' "$log")
|
|
||||||
fails=$(sed -n 's/^best *: [0-9.e+-]* (\([0-9]*\) fails).*/\1/p' "$log")
|
|
||||||
# re-score with the canonical scorer to get the .fails breakdown (evolve.py
|
|
||||||
# only prints the count, not the list); dom lives outside examples/$prog so
|
|
||||||
# pass an absolute path while cd'd there for patterns.config resolution
|
|
||||||
( cd "examples/$prog" && homemaker-fitness "$(realpath "../../$dom")" > /dev/null 2>&1 )
|
|
||||||
notconn=0
|
|
||||||
if [ -f "${dom}.fails" ]; then
|
|
||||||
notconn=$(grep -c 'not connected' "${dom}.fails")
|
|
||||||
fi
|
|
||||||
printf '%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\n' \
|
|
||||||
"$prog" "$seed" "$cg" "$budget" "${fails:-ERR}" "$notconn" "${fitness:-ERR}" "$((t1-t0))" >> "$TSV"
|
|
||||||
echo " -> ${fails:-ERR} fails (${notconn} not-connected), fitness=${fitness:-ERR}, $((t1-t0))s"
|
|
||||||
}
|
|
||||||
|
|
||||||
# harbor-house: budget 2500, seeds 1-3 (qpk protocol)
|
|
||||||
for seed in 1 2 3; do run harbor-house "$seed" 0 2500; done
|
|
||||||
for seed in 1 2 3; do run harbor-house "$seed" 1 2500; done
|
|
||||||
|
|
||||||
# programme-house: budget 3000, seeds 1-5 (qpk protocol)
|
|
||||||
for seed in 1 2 3 4 5; do run programme-house "$seed" 0 3000; done
|
|
||||||
for seed in 1 2 3 4 5; do run programme-house "$seed" 1 3000; done
|
|
||||||
|
|
||||||
echo "=== qi6 conn_grade A/B complete ==="
|
|
||||||
column -t -s $'\t' "$TSV"
|
|
||||||
Loading…
Add table
Reference in a new issue