d0567d7a74
Owner's decision: "we need to abandon the perl oracle, this was only useful when initially porting, but I suspect many of the remaining problems have been carried in from the perl (such as the weird scoring of outdoor and circulation space, which definitely needs fixing)". 39 supports that second clause. Every defect the section found is inherited, not introduced: the two-sided crinkliness gaussian that double-charges surplus daylight (39.14), quality as a product over a variable number of factors (39.18), value_supported priced as value_inside so a terrace was worth more per m2 than a room (39.19), and circulation returning 0.07 per unit cost (hxi). So parity with the oracle was never a safety net -- it was a commitment to reproduce those defects. Each of 39.14, 39.18 and 39.19 would have been a parity failure had parity ever been checked, and keeping the tests would have meant reverting the fixes or explaining the failures away. Removed: oracle.py, test_oracle.py, the two parity tests and their fixture machinery in test_dom_corpus.py, innerloop.OracleEvaluator with its use_native and urb_root plumbing, the same plumbing through driver, and fourteen experiments/ scripts that could only run against Perl. Several of those are cited in earlier DESIGN sections; the citations now point into git history, which is the honest state -- they had been unrunnable since the oracle root (/home/bruno/src/urb) stopped being present. run_search is superseded by run_search_scaled, which does the same job natively. Kept: dump_areas.pl/.py, which validate GEOMETRY against Urb (4.1) rather than fitness, and the prose in fitness_cmd.py and dom.py explaining why the .score/.fails formats are shaped as they are. Provenance is worth keeping; a dead code path is not. CLAUDE.md updated: fitness.py is the only evaluator, and "Urb did it this way" is no longer an argument that a constant is right. 39.16 is the standing counterweight in the other direction -- the crinkliness target WAS right and twice looked wrong only because the code reading it was misunderstood. Inheritance is neither evidence for nor against. 410 passed. The 69 removed cases account exactly: 64 parity (all skipped, since no oracle .score was ever committed), 4 in test_oracle.py, and the guard test 39.20 added as a stopgap. Closes homemaker-py-118. Files homemaker-py-bk9 for the re-baseline that 39.19 made necessary. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB
83 lines
2.4 KiB
Python
83 lines
2.4 KiB
Python
"""Inner-loop search tests against a fake evaluator (no perl, no oracle)."""
|
|
|
|
from dataclasses import dataclass
|
|
|
|
import numpy as np
|
|
import pytest
|
|
|
|
from homemaker_layout import innerloop
|
|
|
|
|
|
@dataclass
|
|
class _Score:
|
|
"""Minimal evaluator result: what the optimisers actually read.
|
|
|
|
Was `oracle.Score`; the Perl oracle is gone (DESIGN.md §39.21) and these
|
|
tests only ever needed a fitness with an empty failure set.
|
|
"""
|
|
|
|
fitness: float
|
|
fail_lines: tuple = ()
|
|
|
|
@property
|
|
def n_fails(self) -> int:
|
|
return len(self.fail_lines)
|
|
|
|
|
|
class FakeEvaluator:
|
|
"""Duck-typed OracleEvaluator over an analytic objective."""
|
|
|
|
def __init__(self, fn):
|
|
self.fn = fn
|
|
self.n_evals = 0
|
|
self.n_oracle_calls = 0
|
|
|
|
def evaluate(self, xs):
|
|
self.n_evals += len(xs)
|
|
self.n_oracle_calls += 1
|
|
return [_Score(self.fn(np.asarray(x))) for x in xs]
|
|
|
|
|
|
def concave(x):
|
|
# maximum 1.0 at 0.3 in every coordinate
|
|
return float(1.0 - np.sum((x - 0.3) ** 2))
|
|
|
|
|
|
@pytest.mark.parametrize("search", [innerloop.nm_search, innerloop.compass_search, innerloop.cma_search])
|
|
def test_search_converges_on_concave(search):
|
|
# the production configs trade final-digit polish for basin coverage
|
|
# (multi-start sigma ladder), so assert basin convergence, not precision
|
|
ev = FakeEvaluator(concave)
|
|
r = search(ev, np.full(4, 0.7), budget=400)
|
|
assert r.fitness > 0.99
|
|
assert np.allclose(r.x, 0.3, atol=0.1)
|
|
assert r.x0_fitness == pytest.approx(concave(np.full(4, 0.7)))
|
|
|
|
|
|
@pytest.mark.parametrize("search", [innerloop.nm_search, innerloop.compass_search, innerloop.cma_search])
|
|
def test_search_respects_budget_and_bounds(search):
|
|
seen = []
|
|
|
|
def spy(x):
|
|
seen.append(x.copy())
|
|
return concave(x)
|
|
|
|
ev = FakeEvaluator(spy)
|
|
r = search(ev, np.full(3, 0.5), budget=60)
|
|
# NM may slightly overshoot on the final call; others batch so allow one extra cycle
|
|
assert r.n_evals == ev.n_evals <= 60 + 3 * 10
|
|
assert all((x >= innerloop._EPS - 1e-12).all() and (x <= 1 - innerloop._EPS + 1e-12).all()
|
|
for x in seen)
|
|
|
|
|
|
def test_compass_never_returns_worse_than_start():
|
|
# a hostile objective: best at the start, everything else worse
|
|
x0 = np.full(3, 0.5)
|
|
|
|
def hostile(x):
|
|
return -float(np.sum(np.abs(x - x0)))
|
|
|
|
ev = FakeEvaluator(hostile)
|
|
r = innerloop.compass_search(ev, x0, budget=100)
|
|
assert r.fitness == pytest.approx(0.0)
|
|
assert np.allclose(r.x, x0)
|