104ba12237
Three times in this log a verdict rested on a sample that could not have produced it: 38.19 (programme-house claimed at N=20, resolves at N=60), 38.21 (harbor at n=3 resolves nothing finer than ~15 fails, yet every recorded margin is smaller), 39.5/38.20 (a 10-fail cpsat margin inside a +-23-fail noise band). Each was found years later. experiments/ab_report.py makes it visible when the verdict is made: minimum detectable difference = t_crit(0.975, N-1) * sd / sqrt(N) A margin below the MDD is not a weak result but an absent one -- the experiment could not have distinguished it from zero however it came out. The report flags that, refuses to endorse a winner, and states the N needed. Validated against both datasets measured this session, reproducing the hand-computed figures exactly: programme-house N=60 +0.567 p=0.017 MDD 0.462 verdict supported programme-house N=20 +0.700 p=0.085 MDD 0.805 UNDERPOWERED, N~=26 harbor N=24 +1.208 p=0.502 MDD 3.669 UNDERPOWERED, N~=202 Harbor needing ~200 seeds means it cannot answer the collapse_insearch question at any N this project would realistically run. Fixed a defect in my own first version: with all-ties (sd=0) the MDD collapses to zero and the naive abs(mean) < mdd reported "margin exceeds the MDD -- verdict supported" for a margin of 0.000, with t=nan. A reporter that endorses a zero margin is worse than none. Degenerate cases are now explicit and distinguish all-ties from a constant non-zero difference. Separate correction found while validating: 38.19's published p-values for N=20 and N=40 were 0.069 and 0.045, from a normal approximation. The exact paired t-test gives 0.085 and 0.052 -- so N=40 did NOT reach significance either; it took N=60. The approximation was anti-conservative, making results look more significant than they are, the same direction of error this thread is about. Corrected at all four citing sites, and the "needs N >= 40" guidance raised to N >= 60. Closes homemaker-py-tco. Lint at parity (46); tests 384 passed, 0 failed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MJ84Feep79Hhm3E4zZJmnB |
||
|---|---|---|
| .. | ||
| hooks | ||
| .gitignore | ||
| config.yaml | ||
| issues.jsonl | ||
| metadata.json | ||
| README.md | ||
Beads - AI-Native Issue Tracking
Welcome to Beads! This repository uses Beads for issue tracking - a modern, AI-native tool designed to live directly in your codebase alongside your code.
What is Beads?
Beads is issue tracking that lives in your repo, making it perfect for AI coding agents and developers who want their issues close to their code. No web UI required - everything works through the CLI and integrates seamlessly with git.
Learn more: github.com/steveyegge/beads
Quick Start
Essential Commands
# Create new issues
bd create "Add user authentication"
# View all issues
bd list
# View issue details
bd show <issue-id>
# Update issue status
bd update <issue-id> --claim
bd update <issue-id> --status done
# Sync with Dolt remote
bd dolt push
Working with Issues
Issues in Beads are:
- Git-native: Stored in Dolt database with version control and branching
- AI-friendly: CLI-first design works perfectly with AI coding agents
- Branch-aware: Issues can follow your branch workflow
- Always in sync: Auto-syncs with your commits
Why Beads?
✨ AI-Native Design
- Built specifically for AI-assisted development workflows
- CLI-first interface works seamlessly with AI coding agents
- No context switching to web UIs
🚀 Developer Focused
- Issues live in your repo, right next to your code
- Works offline, syncs when you push
- Fast, lightweight, and stays out of your way
🔧 Git Integration
- Automatic sync with git commits
- Branch-aware issue tracking
- Dolt-native three-way merge resolution
Get Started with Beads
Try Beads in your own projects:
# Install Beads
curl -sSL https://raw.githubusercontent.com/steveyegge/beads/main/scripts/install.sh | bash
# Initialize in your repo
bd init
# Create your first issue
bd create "Try out Beads"
Learn More
- Documentation: github.com/steveyegge/beads/docs
- Quick Start Guide: Run
bd quickstart - Examples: github.com/steveyegge/beads/examples
Beads: Issue tracking that moves at the speed of thought ⚡