smill/basic-benchmark
Watch
1
0
Fork
You've already forked basic-benchmark
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Remove Phoronix Test Suite

Browse source 
Drops the optional RUN_PTS group, the pts-core phase, and every Phoronix
reference from the scripts and docs. The suite is now just PassMark, the
bundled llama.cpp, the app workloads, and the quick tools.
This commit is contained in:
smill 2026-09-29 21:21:06 -04:00
commit 13da8e986a
5 changed files with 22 additions and 57 deletions
Download patch file Download diff file Expand all files Collapse all files

24
README.md
View file

@ -137,9 +137,9 @@ tools/llama/bin/llama-bench \
Override with `LLAMA_MODEL=` (any GGUF), or `LLAMA_NGL=` to offload to GPU. Override with `LLAMA_MODEL=` (any GGUF), or `LLAMA_NGL=` to offload to GPU.
### App workloads — LibreOffice / GEGL / Inkscape / GIMP (optional) ### App workloads — LibreOffice / GEGL / Inkscape / GIMP (optional)
Stock distro applications are driven over a fixed sample corpus. **No Phoronix Stock distro applications are driven over a fixed sample corpus. Each is the
Test Suite is involved** — each is the app's own CLI, timed, named after the app's own CLI, timed, named after the benchmark. Install the ones you want to
benchmark. Install the ones you want to measure: measure:
```bash ```bash
sudo dnf install -y libreoffice-writer libreoffice-calc # documents -> PDF sudo dnf install -y libreoffice-writer libreoffice-calc # documents -> PDF
sudo dnf install -y gegl04-tools # GEGL operations sudo dnf install -y gegl04-tools # GEGL operations
@ -153,14 +153,6 @@ bytes. Missing apps are skipped; skip the whole group with `RUN_APPS=0`. GIMP 3
reshaped the Script-Fu API, so the GIMP step picks a version-appropriate script reshaped the Script-Fu API, so the GIMP step picks a version-appropriate script
and is best-effort — check `results/gimp-<tag>.txt` if a number looks wrong. and is best-effort — check `results/gimp-<tag>.txt` if a number looks wrong.
### Phoronix Test Suite (optional, short core only)
Already installed (`phoronix-test-suite` 10.8.6). To use it:
```bash
sudo phoronix-test-suite make-download-cache # once, needs internet
```
Only the short core (`build-linux-kernel`, `c-ray`, `stream`) is used, behind
`RUN_PTS=1`. The long suite is intentionally **not** included.
> `run-benchmarks.sh` finds PassMark at `tools/pt/pt_linux_x64` **or** > `run-benchmarks.sh` finds PassMark at `tools/pt/pt_linux_x64` **or**
> `tools/pt/PerformanceTest/PerformanceTest_Linux_x86-64`. Override the location > `tools/pt/PerformanceTest/PerformanceTest_Linux_x86-64`. Override the location
> with `TOOLS=/path` (or set `PT=` directly). > with `TOOLS=/path` (or set `PT=` directly).
@ -238,7 +230,6 @@ Headless part (`~25-30 min` per pass):
| 7 | `fio` 4×30 s | 3-4 min | same SSD, control; runs as your user | | 7 | `fio` 4×30 s | 3-4 min | same SSD, control; runs as your user |
| 8 | `stress-ng` 5 min + `turbostat` | ~8 min | thermals/power | | 8 | `stress-ng` 5 min + `turbostat` | ~8 min | thermals/power |
| 9 | `systemd-analyze` | <1 min | informational | | 9 | `systemd-analyze` | <1 min | informational |
| + | PTS short core (`RUN_PTS=1`) | +30-45 min | kernel build, c-ray, stream |
Headed part (`~1 min` per pass): Headed part (`~1 min` per pass):
@ -251,7 +242,6 @@ Every step writes its own `results/<name>-<tag>.txt/json`.
Options (env): Options (env):
```bash ```bash
RUN_PTS=1 ./run-benchmarks.sh before headless # add the 3 short Phoronix tests
RUN_APPS=0 ./run-benchmarks.sh before headless # skip the app workloads RUN_APPS=0 ./run-benchmarks.sh before headless # skip the app workloads
CORPUS=/path ./run-benchmarks.sh before headless # app corpus cache location CORPUS=/path ./run-benchmarks.sh before headless # app corpus cache location
GOVERNOR=powersave ./run-benchmarks.sh after headless # pin a governor (see §7) GOVERNOR=powersave ./run-benchmarks.sh after headless # pin a governor (see §7)
@ -287,8 +277,8 @@ FIOFILE=/data/fio.bin ./run-benchmarks.sh before headless # fio scratch file
6. Warm up once, then take the **median of 3** runs. Run both parts back to back 6. Warm up once, then take the **median of 3** runs. Run both parts back to back
on the same side. on the same side.
7. Note ambient/room temperature next to the results. 7. Note ambient/room temperature next to the results.
8. **Back up** `results/` (and `~/.phoronix-test-suite` if used) before 8. **Back up** `results/` before reinstalling or re-imaging — copy them
reinstalling or re-imaging — copy them somewhere off the machine. somewhere off the machine.
--- ---
@ -345,7 +335,7 @@ env idle PassMark-cpu PassMark-mem 7zip 7zip-1t openssl
sysbench-1t sysbench-nt sysbench-mem llama sysbench-1t sysbench-nt sysbench-mem llama
fio-seqread fio-seqwrite fio-randread fio-randwrite fio-seqread fio-seqwrite fio-randread fio-randwrite
thermal-load systemd libreoffice gegl inkscape gimp thermal-load systemd libreoffice gegl inkscape gimp
glmark2 vkmark pts-core glmark2 vkmark
``` ```
Example `results/scores-before-headless.csv`: Example `results/scores-before-headless.csv`:
@ -407,7 +397,6 @@ Before any reinstall or re-image, copy the kit and its results somewhere safe:
```bash ```bash
cp -a ~/basic-benchmark /path/to/backup/basic-benchmark-$(date +%F) cp -a ~/basic-benchmark /path/to/backup/basic-benchmark-$(date +%F)
cp -a ~/.phoronix-test-suite /path/to/backup/pts-backup # if PTS was used
``` ```
After the new build: put the kit back, run `INSTALL_DEPS=1 ./run-benchmarks.sh After the new build: put the kit back, run `INSTALL_DEPS=1 ./run-benchmarks.sh
@ -447,6 +436,5 @@ before-headed`.
**headed** for the display-bound GPU tests. **headed** for the display-bound GPU tests.
- **PassMark** is the primary cross-platform score. The suite is - **PassMark** is the primary cross-platform score. The suite is
self-contained and offline: no uploads and no license keys. self-contained and offline: no uploads and no license keys.
- Phoronix is optional and **short only** — no long suite.
- `powerlog.py`/`power-report.py` are plain Python 3; the only Python dep is - `powerlog.py`/`power-report.py` are plain Python 3; the only Python dep is
`matplotlib` (installed on demand; see §3). `matplotlib` (installed on demand; see §3).

32
benchmarks.txt
View file

@ -15,7 +15,7 @@
# This file is the reference for what that script runs, plus extras. # This file is the reference for what that script runs, plus extras.
# #
# Primary cross-platform score: PassMark PerformanceTest (cpubenchmark.net). # Primary cross-platform score: PassMark PerformanceTest (cpubenchmark.net).
# Technical depth: Phoronix Test Suite + the quick tools below. # Technical depth: the quick tools below.
############################################################################ ############################################################################
############################################################################ ############################################################################
@ -36,10 +36,9 @@
6 fio 4x30s headless script 3-4 min runs as your user 6 fio 4x30s headless script 3-4 min runs as your user
7 stress-ng 5 min + turbostat headless script ~8 min thermals/power 7 stress-ng 5 min + turbostat headless script ~8 min thermals/power
8 systemd-analyze headless script <1 min 8 systemd-analyze headless script <1 min
glmark2 + vkmark headed script 3-4 min needs desktop session glmark2 + vkmark headed script ~1 min needs desktop session
---------- headless total ---------- ~25-30 min ---------- headless total ---------- ~25-30 min
---------- headed total ------------ ~4-5 min ---------- headed total ------------ ~1 min
+ PTS core: build-linux-kernel, c-ray, stream +30-45 min RUN_PTS=1
=> ground rule wants MEDIAN OF 3 runs: x3 the above => ground rule wants MEDIAN OF 3 runs: x3 the above
* powerlog.py samples power/temp ~1/s for the WHOLE part -> * powerlog.py samples power/temp ~1/s for the WHOLE part ->
results/power-<tag>.csv + results/phases-<tag>.csv results/power-<tag>.csv + results/phases-<tag>.csv
@ -63,16 +62,16 @@
6. Warm up once, then take the MEDIAN of 3 runs. Throw away the first. 6. Warm up once, then take the MEDIAN of 3 runs. Throw away the first.
7. Note room/ambient temperature next to thermal results; thermals are noisy. 7. Note room/ambient temperature next to thermal results; thermals are noisy.
8. Use the SAME tool versions on both sides (with their (point) releases: 8. Use the SAME tool versions on both sides (with their (point) releases:
PassMark PT build, PTS version). Record them in env-$TAG.txt. PassMark PT build). Record them in env-$TAG.txt.
9. BACK UP before any reinstall / reimage: 9. BACK UP before any reinstall / reimage:
~/.phoronix-test-suite/ and ~/basic-benchmark/results/ ~/basic-benchmark/results/
copy both somewhere safe (off the machine) before reinstalling. copy it somewhere safe (off the machine) before reinstalling.
== 0. ENV CAPTURE (run first, each side) == == 0. ENV CAPTURE (run first, each side) ==
export TAG=before # or: after export TAG=before # or: after
mkdir -p ~/basic-benchmark/results && cd ~/basic-benchmark/results mkdir -p ~/basic-benchmark/results && cd ~/basic-benchmark/results
{ date; uname -a; lscpu; free -h; lsblk; { date; uname -a; lscpu; free -h; lsblk;
phoronix-test-suite version; stress-ng --version; stress-ng --version;
glxinfo -B 2>/dev/null; vulkaninfo --summary 2>/dev/null | head -40; glxinfo -B 2>/dev/null; vulkaninfo --summary 2>/dev/null | head -40;
sha256sum ~/basic-benchmark/tools/pt/PerformanceTest/PerformanceTest_Linux_x86-64 2>/dev/null; sha256sum ~/basic-benchmark/tools/pt/PerformanceTest/PerformanceTest_Linux_x86-64 2>/dev/null;
} > env-$TAG.txt 2>&1 } > env-$TAG.txt 2>&1
@ -117,7 +116,6 @@
== 5. MEMORY — BANDWIDTH == == 5. MEMORY — BANDWIDTH ==
sysbench memory --threads=$(nproc) run | tee sysbench-mem-$TAG.txt sysbench memory --threads=$(nproc) run | tee sysbench-mem-$TAG.txt
# (pts/stream below gives TRIAD/COPY/SCALE/ADD MB/s)
# Compare bandwidth between the two builds. # Compare bandwidth between the two builds.
== 5b. LLM INFERENCE — token generation on CPU/RAM (bundled llama.cpp) == == 5b. LLM INFERENCE — token generation on CPU/RAM (bundled llama.cpp) ==
@ -128,7 +126,7 @@
| tee llama-$TAG.txt | tee llama-$TAG.txt
# -ngl 0 forces CPU/RAM (no GPU offload). Record the tg t/s number. # -ngl 0 forces CPU/RAM (no GPU offload). Record the tg t/s number.
== 5c. APP WORKLOADS (LibreOffice / GEGL / Inkscape / GIMP — no Phoronix) == == 5c. APP WORKLOADS (LibreOffice / GEGL / Inkscape / GIMP) ==
# Stock distro apps run over a fixed sample corpus; run-benchmarks.sh fetches # Stock distro apps run over a fixed sample corpus; run-benchmarks.sh fetches
# the corpus ONCE into tools/corpus/ (checksum-pinned) and times the app's own # the corpus ONCE into tools/corpus/ (checksum-pinned) and times the app's own
# CLI, reporting a rate (items/s, higher is better). Missing apps are skipped. # CLI, reporting a rate (items/s, higher is better). Missing apps are skipped.
@ -138,17 +136,6 @@
# Phases / scores are named after the benchmark: libreoffice gegl inkscape gimp. # Phases / scores are named after the benchmark: libreoffice gegl inkscape gimp.
# GIMP 3 reshaped Script-Fu, so the GIMP step is best-effort. # GIMP 3 reshaped Script-Fu, so the GIMP step is best-effort.
== 6. REAL-WORLD BUILD / RENDER (short Phoronix core) ==
# Phoronix Test Suite pins versions/counts and stores results you can diff.
# Installed here: 10.8.6
sudo phoronix-test-suite make-download-cache # once, before going offline
phoronix-test-suite benchmark \
pts/build-linux-kernel \
pts/c-ray \
pts/stream
# verify names if any differ:
phoronix-test-suite list-available-tests | grep -Ei 'build|c-ray|stream'
== 7. GPU — DISCRETE RX 6700 XT (same card both runs; HEADED part only) == == 7. GPU — DISCRETE RX 6700 XT (same card both runs; HEADED part only) ==
sudo dnf install -y glmark2 vkmark vulkan-tools mesa-demos sudo dnf install -y glmark2 vkmark vulkan-tools mesa-demos
# Monitors hang off the Cezanne iGPU, so PIN the dGPU or these tests would # Monitors hang off the Cezanne iGPU, so PIN the dGPU or these tests would
@ -162,7 +149,6 @@
# VKMARK_DURATION and VKMARK_BENCH (texture|shading|vertex). # VKMARK_DURATION and VKMARK_BENCH (texture|shading|vertex).
glmark2 -b terrain:duration=10 2>&1 | tail -25 | tee glmark2-$TAG.txt glmark2 -b terrain:duration=10 2>&1 | tail -25 | tee glmark2-$TAG.txt
vkmark -b texture:duration=10 2>&1 | tail -30 | tee vkmark-$TAG.txt vkmark -b texture:duration=10 2>&1 | tail -30 | tee vkmark-$TAG.txt
phoronix-test-suite benchmark pts/vkmark # if available in cache
# Confirm which GPU rendered: grep -i 'device\|renderer' glxinfo-$TAG.txt # Confirm which GPU rendered: grep -i 'device\|renderer' glxinfo-$TAG.txt
== 8. STORAGE / IO (same drive — mostly a control; watch for regressions) == == 8. STORAGE / IO (same drive — mostly a control; watch for regressions) ==
@ -225,7 +211,7 @@
# env PassMark 7zip openssl sysbench # env PassMark 7zip openssl sysbench
# Example: # Example:
# PassMark,24500 # PassMark,24500
# pts-core,420 <- kernel-build seconds; energy = avg_w * seconds # libreoffice,3.9 <- docs/s; energy = avg_w * seconds
# Metrics reported: avg/p95/peak W, avg/peak °C, energy (kJ) per phase, # Metrics reported: avg/p95/peak W, avg/peak °C, energy (kJ) per phase,
# and score-per-watt (higher = better; lower energy = better). # and score-per-watt (higher = better; lower energy = better).

2
capture-specs.sh
View file

@ -53,7 +53,7 @@ disks() {
} }
tool_versions() { tool_versions() {
for c in 7z openssl fio stress-ng sysbench sensors turbostat cpupower \ for c in 7z openssl fio stress-ng sysbench sensors turbostat cpupower \
glmark2 vkmark glxinfo vulkaninfo phoronix-test-suite python3 \ glmark2 vkmark glxinfo vulkaninfo python3 \
libreoffice inkscape gegl gimp; do libreoffice inkscape gegl gimp; do
have "$c" && printf ' %-20s %s\n' "$c" "$("$c" --version 2>&1 | head -1)" have "$c" && printf ' %-20s %s\n' "$c" "$("$c" --version 2>&1 | head -1)"
done done

2
power-report.py
View file

@ -11,7 +11,7 @@ per-phase power/thermal summary.
scores-<tag>.csv format (one benchmark per line, name must match a phase): scores-<tag>.csv format (one benchmark per line, name must match a phase):
PassMark-cpu,24500 PassMark-cpu,24500
PassMark-mem,2500 PassMark-mem,2500
pts-core,420 libreoffice,3.9
""" """
import argparse import argparse
import csv import csv

19
run-benchmarks.sh
View file

@ -24,7 +24,6 @@
# Env options: # Env options:
# INSTALL_DEPS=1 auto-install missing tools via sudo dnf (fresh Fedora) # INSTALL_DEPS=1 auto-install missing tools via sudo dnf (fresh Fedora)
# default is 'ask' (prompts); 0 disables # default is 'ask' (prompts); 0 disables
# RUN_PTS=1 add the 3 short Phoronix tests (needs download cache)
# RUN_GPU=0 skip the headed GPU tests in 'all' mode # RUN_GPU=0 skip the headed GPU tests in 'all' mode
# TOOLS=/path where PassMark lives (default ./tools) # TOOLS=/path where PassMark lives (default ./tools)
# FIOFILE=/path fio scratch file (must be on the target filesystem) # FIOFILE=/path fio scratch file (must be on the target filesystem)
@ -99,7 +98,6 @@ TAG="$TAG_BASE"
[[ "$PART" != "all" ]] && TAG="$TAG-$PART" [[ "$PART" != "all" ]] && TAG="$TAG-$PART"
RUN_GPU="${RUN_GPU:-1}" RUN_GPU="${RUN_GPU:-1}"
RUN_PTS="${RUN_PTS:-0}"
TOOLS="${TOOLS:-$DIR/tools}" TOOLS="${TOOLS:-$DIR/tools}"
if [[ -z "${PT:-}" ]]; then if [[ -z "${PT:-}" ]]; then
for cand in "$TOOLS/pt/pt_linux_x64" \ for cand in "$TOOLS/pt/pt_linux_x64" \
@ -115,12 +113,12 @@ LLAMA_MODEL="${LLAMA_MODEL:-$LLAMA_DIR/models/MiniCPM5-2B-Q8_0.gguf}"
LLAMA_NGL="${LLAMA_NGL:-0}" # 0 = CPU/RAM only (no GPU offload) LLAMA_NGL="${LLAMA_NGL:-0}" # 0 = CPU/RAM only (no GPU offload)
LLAMA_TG="${LLAMA_TG:-128}" # generated tokens LLAMA_TG="${LLAMA_TG:-128}" # generated tokens
LLAMA_REPS="${LLAMA_REPS:-2}" LLAMA_REPS="${LLAMA_REPS:-2}"
# App workloads (no Phoronix Test Suite): stock distro applications run over a # App workloads: stock distro applications run over a
# fixed sample corpus, fetched once and checksum-pinned, then cached under # fixed sample corpus, fetched once and checksum-pinned, then cached under
# CORPUS and reused by every run so both sides process identical input bytes. # CORPUS and reused by every run so both sides process identical input bytes.
RUN_APPS="${RUN_APPS:-1}" RUN_APPS="${RUN_APPS:-1}"
CORPUS="${CORPUS:-$TOOLS/corpus}" CORPUS="${CORPUS:-$TOOLS/corpus}"
CORPUS_URL="${CORPUS_URL:-http://phoronix-test-suite.com/benchmark-files}" CORPUS_URL="${CORPUS_URL:-http://phoronix-test-suite.com/benchmark-files}" # sample-file CDN
SAMPLER="$DIR/powerlog.py" SAMPLER="$DIR/powerlog.py"
REPORT="$DIR/power-report.py" REPORT="$DIR/power-report.py"
PHASES="$RES/phases-$TAG.csv" PHASES="$RES/phases-$TAG.csv"
@ -258,7 +256,7 @@ parse_passmark() { # $1 = yml, $2 = phase name, $3 = suite ("cpu"|"mem")
}' "$yml" }' "$yml"
} }
# --- app workloads: fixed corpora + timing (no Phoronix Test Suite) ----------- # --- app workloads: fixed corpora + timing ------------------------------------
# Each app benchmark runs the stock command-line tool over a fixed input set and # Each app benchmark runs the stock command-line tool over a fixed input set and
# reports a rate (items/s, higher is better) so the score/W table stays meaningful. # reports a rate (items/s, higher is better) so the score/W table stays meaningful.
fetch_corpus() { # $1=file $2=url $3=sha256 -> ensures $CORPUS/<file> is present fetch_corpus() { # $1=file $2=url $3=sha256 -> ensures $CORPUS/<file> is present
@ -390,7 +388,7 @@ doctor() {
printf 'model: %s\n' "$([ -f "$LLAMA_MODEL" ] && echo "$LLAMA_MODEL ($(du -h "$LLAMA_MODEL" 2>/dev/null | cut -f1))" || echo "MISSING ($LLAMA_MODEL)")" printf 'model: %s\n' "$([ -f "$LLAMA_MODEL" ] && echo "$LLAMA_MODEL ($(du -h "$LLAMA_MODEL" 2>/dev/null | cut -f1))" || echo "MISSING ($LLAMA_MODEL)")"
printf 'tools:\n' printf 'tools:\n'
for t in 7z openssl sysbench fio stress-ng sensors turbostat cpupower \ for t in 7z openssl sysbench fio stress-ng sensors turbostat cpupower \
glmark2 vkmark glxinfo vulkaninfo phoronix-test-suite nproc; do glmark2 vkmark glxinfo vulkaninfo nproc; do
printf ' %-22s %s\n' "$t" "$(command -v "$t" || echo MISSING)" printf ' %-22s %s\n' "$t" "$(command -v "$t" || echo MISSING)"
done done
printf 'apps:\n' printf 'apps:\n'
@ -454,7 +452,6 @@ run_env() {
lscpu lscpu
free -h free -h
lsblk lsblk
have phoronix-test-suite && phoronix-test-suite version
have stress-ng && stress-ng --version have stress-ng && stress-ng --version
have glxinfo && glxinfo -B 2>/dev/null have glxinfo && glxinfo -B 2>/dev/null
have vulkaninfo && vulkaninfo --summary 2>/dev/null | head -70 have vulkaninfo && vulkaninfo --summary 2>/dev/null | head -70
@ -501,7 +498,7 @@ run_llama() {
reg_score "llama-tg" "$tg" reg_score "llama-tg" "$tg"
} }
# --- app workloads (stock apps + fixed corpora; no Phoronix Test Suite) ------- # --- app workloads (stock apps + fixed corpora) -------------------------------
# Phases and score names are the benchmark names: libreoffice gegl inkscape gimp. # Phases and score names are the benchmark names: libreoffice gegl inkscape gimp.
# Each registers a rate (items/s) so higher stays better for the score/W table. # Each registers a rate (items/s) so higher stays better for the score/W table.
@ -810,12 +807,6 @@ run_headless() {
systemd-analyze | tee "systemd-analyze-$TAG.txt" systemd-analyze | tee "systemd-analyze-$TAG.txt"
systemd-analyze blame | head -20 >> "systemd-analyze-$TAG.txt" systemd-analyze blame | head -20 >> "systemd-analyze-$TAG.txt"
fi fi
if [[ "$RUN_PTS" == "1" ]] && have phoronix-test-suite; then
section "PHORONIX TEST SUITE (core)"
phase "pts-core"
phoronix-test-suite benchmark pts/build-linux-kernel pts/c-ray pts/stream
fi
} }
run_headed() { run_headed() {