compare-report: median-of-reps grouping, --exclude, --results

--group NAME=t1,t2 collapses repetitions into one median column per device,
--exclude drops bad runs, and --results reads a collected results directory.
This commit is contained in:
smill 2026-09-29 23:56:54 -04:00
commit 531ed18082
2 changed files with 84 additions and 14 deletions

View file

@ -383,6 +383,17 @@ are back-filled from the raw results (`fio` JSON, `openssl`, `7z`,
`sysbench-mem`, `glmark2`/`vkmark`), so existing runs compare without re-running.
Needs `matplotlib`; use `--no-graphs` for the table alone.
With **median-of-3**, collapse repetitions into one column per device and drop
any bad run:
```bash
python3 compare-report.py --results ~/Documents/results \
--group "desktop=before1-headless,before2-headless,before3-headless" \
--group "server=server1-headless,server2-headless,server3-headless" \
--group "t14=t14-2,t14-3" # t14-1 dropped (low-battery run)
```
`--group NAME=t1,t2,...` makes one median column; `--exclude t1 t2` drops tags.
`--results DIR` reads another machine's collected `results/`.
Fill the results table in `benchmarks.txt` with the medians after each side.
---