D81 — Named regression board sets with work budgets¶
Decision¶
Regression runs use named board sets in bench/sets/ (quick, mid, planes) with deterministic work budgets (run.py --work, no knowledge base); plane boards are judged after a KiCad refill (--refill) with the board's project and rules beside the output; bench/compare_runs.py checks the gates of doc 10 §5
Alternatives¶
Manifest files and a report generator as first planned; keep sampling by --limit with work budgets passed through TM_ROUTE_ARGS; judge planes on stale fills
Reason¶
Board lists are the smallest thing that names a tier and survives sampling changes. Work budgets make two runs of one binary byte-identical, so "unchanged" can be checked, not estimated. Stale fills and default rules gave false results on the plane set (hundreds of false errors without the project files). The planned statistics and dashboard are not needed for paired regression checks (doc 10 §4)