otko/specifications/15-rebuild-adoption-plan.md

15 — Rebuild Adoption Plan

Status: proposed. Source of truth for intent: the greenfield rebuild specs at ~/Sync/otko-development/specifications (00–14). This document plans how to bring that rebuild's good ideas into this repo without breaking the working app.

Requirement IDs are shown without brackets for brevity (e.g. PKG-032 = [PKG-032] in the rebuild set) so they stay grep-able.

---

1. Context

	This repo (~/Sync/otko)	Rebuild (~/Sync/otko-development)
Role	Working app — the primary deliverable	Greenfield spec-conformant attempt
VCS	Not a git repo in this checkout	rebuild branch, commit 725040e
License	AGPL-3.0	MIT (+ NOTICE)
Python	3.10–3.12	3.12 only
Project file	.osmodel	.otko
Scope	Superset: static, modal, transient, pushover, response-spectrum, quad/shell-adjacent elements, HDF5	Narrow v1: static + modal; reports, combinations, self-weight, units display
src LOC	~37k	~25k
Gate	deps not installed here; 57 collection errors locally	ruff passes; mypy/pytest deps missing locally

The rebuild is ahead on: reporting/Typst, load combinations, self-weight, display-unit conversion, closed-form diagrams, modal mass participation, progress/cancel, local-axis editing, viewmodel separation, and quality gates. It is behind on everything this repo already ships. Therefore the plan is a selective harvest, not a merge.

2. Goal and non-goals

Goal. Adopt the rebuild's good capabilities into this repo, incrementally, each phase independently shippable, with the existing app and tests staying green.

Non-goals.

• Do not rewrite this repo to the rebuild's architecture (ProjectStore, frozen Project, CommandFactory, services/scene.py, services/diagram_data.py substrates).
• Do not replace the runner/export/results surface — this repo's is a strict superset.
• Do not change .osmodel, the AGPL license, or the Python 3.10–3.12 support range as part of this plan.

3. Principles

1. Additive, adapt, don't copy. Port algorithms/patterns; re-type to this repo's APIs (StaticResults/ModalResults, mutable Project, 4-value UnitSystem, element_forces.DiagramData).
2. Protect the working app. Every phase keeps pytest -m "not slow" and the GUI smoke tests green. New optional deps stay lazy so headless installs are unaffected.
3. One phase = one branch = one shippable PR, with tests and a gate.
4. Trace every item to a rebuild spec ID so "is it done?" stays mechanical.
5. License hygiene. MIT→AGPL-3.0 is one-way compatible: preserve the MIT copyright/permission notice for ported files (NOTICE + per-file header). Never copy this repo's AGPL code back into the MIT rebuild.
6. Establish version control first. This checkout has no .git. Before any work, create a repo/branch per AGENTS.md (origin is the self-hosted Forgejo; work on feat/<topic>, PR against main).

4. Adoption matrix

Verdicts: PORT (near-verbatim), ADAPT (re-type/re-wire), REIMPL (reimplement against this repo's APIs), SKIP (intentional divergence). Size: S ≈ hours · M ≈ a few days · L ≈ 1–2+ weeks.

4.1 Quality / foundations

#	Capability	Rebuild source	Verdict	Phase	Size	Spec IDs
Q1	Single-source version + solver pin via _const.py	src/otko/_const.py, pyproject.toml [tool.hatch.version]	ADAPT	0	S	PKG-032/033, RUN-085
Q2	Layering/import-discipline AST tests	tests/unit/test_architecture.py (246)	ADAPT (paths)	0	M	ARC-001..003/006/011/060..064
Q3	CI split into 5 jobs incl. missing type (mypy) job; mark integration slow	.github/workflows/ci.yml, tests/unit/test_ci_config.py, test_meta_gates.py	ADAPT	0	M	TST-004/031/040/041
Q4	conftest hardening (offscreen default; ops.wipe() in teardown)	tests/conftest.py	ADAPT	0	S	TST §7
Q5	tests/services/ split from tests/unit/	tests layout	ADAPT	0	M	TST-002
Q6	Standalone docs + docs test	docs/QUICK_GUIDE.md, docs/README.md, tests/unit/test_docs.py	ADAPT	0	M	—
Q7	NOTICE + attribution for ported MIT code	NOTICE	REIMPL (AGPL wording)	0	S	PKG-042
Q8	Drop phantom scipy/pandas from [gui]	pyproject.toml, tests/unit/test_packaging.py	ADAPT	0	S	PKG-011
Q9	Scoped mypy overrides + coverage fail_under	pyproject.toml	ADAPT	0	S	TST-031/032
Q10	Packaging config tests subset (entrypoint, line length, single-source)	tests/unit/test_packaging.py (117)	ADAPT	0	S	PKG-030..033
Q11	Spec-ID traceability harness	tests/unit/test_spec_traceability.py (343)	REIMPL	8	L	TST-013/024/025
Q12	Performance probes	tests/performance/*	ADAPT	8	L	OVR-010..014

4.2 Services / engine

#	Capability	Rebuild source	Verdict	Phase	Size	Spec IDs
S1	SolverSession Protocol + Real/Recording sessions	services/solver_session.py (154)	PORT	1	S	RUN-003/010/011/012
S2	Persistence refuses to save an invalid model + resolved path	services/persistence.py (63)	ADAPT	1	S	PER-021/022
S3	Result fields: StaticResults.time, modal ModeParticipation record	services/results.py (138)	ADAPT	1	S	RES-001/003/005, ANL-012
S4	Component-less element forces = absent, not zeros	runner/run.py	ADAPT	1	S	RUN-053
S5	progress(int) + cancelled + isInterruptionRequested() + cancel(); real Cancel UX	services/qt_workers.py, viewmodels/analysis_vm.py, views/dialogs/run.py	ADAPT	2	M	ANL-031..036, ARC-032/033, UX-081
S6	Modal mass participation unified with response-spectrum path	runner/modal.py (466), services/results.py	ADAPT	2	M	RUN-062/070..072, ANL-013
S7	Display-unit layer + GRAVITY + ProjectMeta.display (adapter over 4-system enum)	core/units.py (298)	ADAPT	3	M	UNT-003/004/010/020..024/030..032
S8	Closed-form diagram math (N/V/M/T shapes + extrema)	core/diagrams.py (436)	PORT	4	S	RES-010/011/012
S9	Backend-neutral diagram_data service + render_matplotlib	services/diagram_data.py (400)	REIMPL	4	M	RES-020..024/030..032, CAN-084
S10	Report pipeline CSV / PNG(≥300 dpi) / SVG / Typst (+optional PDF) + case-report action	services/report/* (1141)	PORT + ADAPT deps	5	M	RES-040..047/050/051/061
S11	Local axis: LocalAxis, element field, geomTransf dedup by (type,vecxz,roll), editor + triad	core/geometry/local_axis.py, runner/emit.py	ADAPT	6	M	GEO-060..064, DOM-040
S12	Load combinations entity + commands + runner materialisation	core/combinations.py (44), commands/combinations.py (146)	PORT + ADAPT	7	M	LOD-050..058, ANL-001, RUN-050
S13	Self-weight service + regenerate command	services/self_weight.py (364)	REIMPL	7	M	UNT-040..045, LOD-011/040..043
S14	Material rho coverage for self-weight	core/materials	ADAPT	7	S	UNT-041

4.3 UI / UX

#	Capability	Rebuild source	Verdict	Phase	Size	Spec IDs
U1	Pure formatting helpers (DOF/MASS/RESTRAINT_LABELS, number/float parse)	views/formatting.py (84)	PORT	0	S	UX-033/062
U2	Timestamped, severity-tagged console	views/docks/console.py (103)	PORT	0	S	UX-042
U3	ValidatedDialog base (help line, inline errors, OK-gating, unit suffix)	views/dialogs/base.py (191)	PORT + staged	8	M	UX-060..064
U4	ResultsVM per-case handle cache; CanvasVM selection ownership	viewmodels/results_vm.py, canvas_vm.py	REIMPL	8	M	RES-005, CAN-042/041
U5	Overlay helpers: ghost undeformed, unit-labelled diagram extremes, zero hint, pattern-filtered loads	views/canvas/overlays.py (684)	REIMPL	8	M	CAN-052/062/082/083, LOD-063
U6	QSettings layout persistence + closeEvent save prompt	views/main_window.py	PORT	0	S	UX-013, PER-032/033, ARC-050
U7	ThemeManager + dark icon set	views/resources/theme.py, icons.py	ADAPT (verify icon provenance)	8	S	UX-070/071/072
U8	Thicken ProjectViewModel (move action_handlers logic into VM methods)	viewmodels/project_vm.py (713)	REIMPL	8	L	UX-001/002/005, ARC-005/020..022
U9	Analysis VM status + Simple/Advanced control fields	viewmodels/analysis_vm.py	ADAPT	8	M	ANL-021/050..054, UX-037

4.4 Explicitly deferred / skipped

Item	Verdict	Why
Polygon-first sections + services/sections_mesh.py (opstool GPLv3)	DEFER (Phase 9, conditional)	Schema-breaking redesign; adds GPLv3/runtime deps; not report-critical
Frozen Project/entities + ProjectStore/CommandFactory rewrite	DEFER (not recommended now)	Large coordinated rewrite of 43 commands + all call sites; low user-visible value vs risk
Canvas backend Protocol + plotly_backend.py + services/scene.py	DEFER (Phase 9)	Requires core + views surgery; plotly pulls QtWebEngine (known teardown SIGSEGV)
.otko file suffix / greenfield Project schema (PER-001/006)	SKIP	Breaks .osmodel corpus and compatibility; do only as an explicit user-approved migration
Rebuild runner/export.py	SKIP	This repo's export.py is a superset (5 case types, .py+.tcl)
Rebuild static/modal HDF5	SKIP	Rebuild has none; this repo's transient HDF5 is already better
Rebuild slim materials/analysis/loads unions	SKIP	Dropping HystereticSM, Transient/Pushover/ResponseSpectrum, Path/Imposed patterns would regress shipped features
MIT license / py3.12-only / single-OS CI assertions	SKIP	Intentional differences (AGPL, 3.10–3.12, 3-OS matrix)

5. Phases

Phase 0 — Foundations, safety net, quick wins

Why first: makes later phases verifiable and cheap; all items are additive and low risk. Also unblocks the mpy gate that currently does nothing.

• Prep: initialise git and a feat/rebuild-adoption branch (§3.6).
• Q1 Create src/otko/_const.py (__version__, OPENSEESPY_VERSION), switch pyproject.toml to dynamic = ["version"] + [tool.hatch.version], and import the pin in services/export.py (currently duplicated at export.py:52). Spec: PKG-032/033, RUN-085.
• Q2 Port tests/unit/test_architecture.py; adapt CANVAS = views/canvas3d path and services/{_emit,_run}.py paths. Expect it to flag three existing offenders — decide per item: OSS_PICK_DEBUG env var in views/canvas3d/model_canvas.py:29 (ARC-052), QUndoStack outside VMs, and the canvas path. Either fix or add a documented, narrow allowlist.
• Q3 Split CI: lint (+ruff format --check), type (mypy scopes), test-headless (pytest tests/unit tests/services -m "not slow"), test-gui (xvfb-run pytest tests/gui), test-integration (pytest tests/integration -m slow). Add pytestmark = pytest.mark.slow to integration modules. Keep the existing 3-OS × 3.10–3.12 matrix and the long Linux Qt apt list.
• Q4 tests/conftest.py: QT_QPA_PLATFORM=offscreen default; move ops.wipe() to teardown (this repo currently wipes before each test).
• Q5 Move service-level tests (runner_translation, persistence, results, export, …) into tests/services/; update CI and AGENTS.md.
• Q6 Add docs/QUICK_GUIDE.md + docs/README.md (rewrite .otko → .osmodel, rebuild-only APIs → this repo's), update CONTRIBUTING.md with the 5-layer table + install/run/test commands; port test_docs.py.
• Q7 Add NOTICE crediting MIT otko-development for ported files; add a per-file header to ported files (# Ported from otko-development (MIT), (c) 2026 OTKO contributors).
• Q8/Q9/Q10 pyproject.toml: drop scipy/pandas from [gui]; convert mypy to scoped strict overrides (core/services/viewmodels); add [tool.coverage.run] fail_under; port the test_packaging.py subset that applies (entrypoint, version single-source, line length).
• U1/U2/U6 Port views/formatting.py; timestamp the console (views/docks/console.py into the existing dock); add save_layout/ restore_layout (QSettings) and a closeEvent unsaved-changes prompt.

Gate: ruff check src tests, ruff format --check src tests, mypy src/otko/core src/otko/services src/otko/viewmodels, pytest -m "not slow", arch test, and the new packaging/docs tests pass.

Phase 1 — Cheap, safe services wins

• S1 Port services/solver_session.py (SolverSession, RealSolverSession, RecordingSolverSession). Inject into OpenSeesRunner while keeping the existing ops_module= shim; consolidate the two wipe paths (_emit.py:105 → session.reset()). Tests: tests/services/test_solver_session.py.
• S2 In services/persistence.py::save_project, call validate_references() first and refuse to write an invalid model, reporting the problem list (PER-022); return the resolved path (PER-021).
• S3 Extend StaticResults with time and add a ModeParticipation record to ModalResults (keep the existing dataclass names — consumers in views/docks/results_panel.py dispatch on them).
• S4 RUN-053: return an absent (not zero-filled) force entry when a component is missing; guard the static eleForce fallback.

Gate: new service tests + existing runner/persistence tests green; runner integration suite unchanged.

Phase 2 — Progress & cancellation

• S5 Add to services/qt_workers.py: progress(int), cancelled, an _interruption_requested() check between steps, and a bounded teardown wait. Add AnalysisRunner.cancel() (calls QThread.requestInterruption()). Wire progress callbacks through static (RUN-054) and modal (RUN-063) runs. Replace the indeterminate bar in views/dialogs/run_analysis.py with a real progress bar + working Cancel (UX-081). Add per-case status to the case manager (ANL-050).
• S6 Unify modal mass participation: put the rebuild's participation math into one service and have both modal results and the existing response-spectrum path (services/spectrum.py) consume it. Add the ANL-013 no-mass warning.

Gate: cancel leaves no partial handle; progress reaches 100; participation sums to 1.0 against a hand-checked example.

Phase 3 — Units display layer + GRAVITY

• S7 Port the display machinery from core/units.py as an adapter: keep this repo's 4-value UnitSystem and map families (SI_M_N|SI_MM_N → METRIC, US_FT_KIP|US_IN_KIP → IMPERIAL); add GRAVITY, display(value, system, quantity, pref), DisplayPrefs; add ProjectMeta.display (persisted). Retarget the status-bar/menu unit picker so it writes display prefs, not the stored meta.units (fixes UNT-031).
• Watch-outs: rebuild uses StrEnum (3.11+) — do not adopt it (this repo targets 3.10). export.py currently embeds meta.units.value; keep that but source labels via the new API.

Gate: switching display units changes every label and leaves project.model_dump() byte-identical; old 4-system .osmodel files still load.

Phase 4 — Closed-form diagrams

• S8 Port core/diagrams.py near-verbatim (pure numpy, self-contained).
• S9 Build a new services/diagram_data.py (REIMPL) on top of this repo's StaticResults + load patterns, reusing core/diagrams for interior shapes/extrema. Run it alongside services/element_forces.py, migrate views/canvas3d/diagram_renderer.py and views/dock_manager.py behind a flag, then retire the old extractor once the on-screen output matches.

Gate: port test_diagram_data.py (end-values, Vz/My plane, axial, all-six components, shared data, auto-scale); on-screen diagrams unchanged.

Phase 5 — Reporting pipeline ★ (the headline v1 capability)

• S10 Port services/report/{__init__,csv,figures,typst,pdf}.py. Add a [reports] extra (matplotlib, imageio-ffmpeg; Typst CLI optional, probed via shutil.which, graceful degradation per RES-046). Expose a "case report" action that produces the full set: CSV tables, PNG (≥300 dpi), SVG, a Typst document (PDF optional), conditional static/modal sections.
• Dependencies: Phase 3 (units) and Phase 4 (diagrams) must land first; reuse Phase 1 result fields.

Gate: the reference test — cantilever moment 50 kN·m at the fixed end (RES-061) — plus report generation off the GUI thread and preservation of the original model file. Port test_report_{csv,figures,typst,modal}.py and the test_full_report.py / test_report_reference.py integration tests.

Phase 6 — Local axis (user-editable)

• S11 Port core/geometry/local_axis.py; add local_axis: LocalAxis to the frame elements (ElasticBeamColumn, ForceBeamColumn, DispBeamColumn, BeamWithHinges). Change services/_emit.py geomTransf to read the explicit axis and dedup by (type, vecxz, roll_deg) instead of auto-deriving. Update the renderer triad (its current getattr(el, "vecxz") at model_renderer.py:727 is dead code) and add an editor (property dock / dialog).
• Critical migration risk: a blind default vecxz=(0,0,1) is parallel to a vertical member's axis and would break existing vertical models. Use a safe default rule (e.g. derive the default from geometry exactly as the runner does today, then only override when the user sets it) so existing .osmodel files keep loading and producing identical results.

Gate: existing frame/eigen integration tests unchanged; new test_local_axis_parallel_rejected + test_geom_transf_dedup_by_combination.

Phase 7 — Load combinations + self-weight

• S12 Port core/combinations.py and commands/combinations.py; add Project.combinations; extend validate_references; add StaticCase.combination_id with the XOR invariant (combination or patterns+factors, ANL-001); materialise scaled loads in the runner (LOD-054) without regressing the existing per-pattern factor path.
• S13 Reimplement services/self_weight.py against this repo's types: w = ρ·A·g distributed local for frames, equivalent nodal loads for trusses (UNT-042/043), a ConstantTimeSeries + PlainLoadPattern, regeneration that replaces the prior pattern (LOD-041), no-density/unresolved reporting (LOD-042), and the gravity constant recorded in the description (UNT-045). Add a RegenerateSelfWeightCommand (undoable).
• S14 ElasticIsotropic currently is the only material with rho (materials/__init__.py:28); add rho to the other self-weight-capable materials, or report them as missing density.
• Depends on Phase 3 (GRAVITY) and Phase 6 (correct local projection).

Gate: port test_loads_combinations.py and test_self_weight.py (minus the combination assertion until S12 lands); verify superposition matches a hand calculation.

Phase 8 — UI/UX adoption (larger, optional but valuable)

• U3 ValidatedDialog base + migrate dialogs incrementally (unit suffix via the Phase 3 API).
• U4 Extract ResultsVM (per-case handle cache) and CanvasVM (selection ownership) as supersets of the current state holders so existing call sites keep working.
• U5 Reimplement overlay pure helpers against this repo's deformation/element_forces: ghost undeformed (CAN-062), unit-labelled diagram extremes (CAN-082), zero-component hint (CAN-083), and pattern-filtered load display (CAN-052/LOD-063). Unit-test them display-free.
• U7 Adopt ThemeManager + a dark icon set — verify SVG provenance and license first before copying any icon assets.
• U8/U9 Thicken ProjectViewModel (move action_handlers.py logic into VM methods, one command family at a time) and add analysis-case status / Simple-vs-Advanced controls.
• Q11/Q12 Spec-ID traceability harness and performance probes (only if the team commits to the [AREA-NNN] convention and a perf budget).

Gate: GUI tests under xvfb-run; VM logic unit-tested without a display.

Phase 9 — Deferred, only on explicit decision

• Polygon-first sections + sections_mesh (opstool GPLv3, lazy import, optional [sections] extra; ensure AGPL compatibility and attribution).
• Frozen Project/ProjectStore/CommandFactory migration.
• Canvas backend Protocol + plotly backend + services/scene.py.

Revisit only after Phases 0–8 are stable and if the value justifies the blast radius documented in the dossiers.

6. Dependency graph

Phase 0 (foundations)
   │
   ├─► Phase 1 (sessions, persistence, result fields)
   │        │
   │        ├─► Phase 2 (progress/cancel, mass participation)
   │        │
   │        └─► Phase 3 (units display + GRAVITY) ──┐
   │                                                │
   │             Phase 4 (diagrams) ────────────────┤
   │                                                ▼
   │                                   Phase 5 (REPORTING) ★
   │
   ├─► Phase 6 (local axis) ──┐
   │                          ▼
   └─►                 Phase 7 (combinations + self-weight)
                              │
                              ▼
                       Phase 8 (UI/UX) ──► Phase 9 (deferred)

Critical path to the v1 promise (a printable case report): Phase 3 → Phase 4 → Phase 5, with Phase 1 as a cheap prerequisite and Phase 2 interleavable.

7. Licensing & attribution

• This repo is AGPL-3.0; the rebuild is MIT. MIT code may be incorporated into an AGPL work — one-way. Preserve the MIT notice: add a NOTICE (Phase 0) and a short provenance header to each ported file.
• Never copy this repo's AGPL code back into the MIT rebuild.
• opstool is GPLv3 (only relevant to deferred sections_mesh); GPLv3 ↔ AGPLv3 are compatible. Keep it lazy/optional and do not vendor its source.
• Verify the provenance/license of any copied icon/SVG assets before adopting the dark icon set (U7).

8. Risks and mitigations

Risk	Mitigation
Big-bang port destabilises the working app	Strict additive phases; feature flags for diagram/report migration; keep old paths until parity is proven
Freezing/command rewrite stalls progress	Explicitly deferred (Phase 9); combinations/self-weight do not require it
Units enum change corrupts stored models	Keep 4-value enum; add a family mapping + DisplayPrefs; byte-identical dump test
Local-axis default breaks vertical members	Derive default per-element as today; override only on explicit user set; regression tests on vertical frames
opstool private-API coupling / GPL	Deferred; pin version + thin adapter; lazy import; attribution
Report pipeline pulls GUI deps into headless	Keep matplotlib/Typst optional and imported lazily; headless CI must pass without [reports]
StrEnum / 3.11-only syntax	Target py3.10: avoid StrEnum, zip(strict=), etc.
No VCS in this checkout	Initialise git + branch before any code changes
Tests can't run locally (deps missing)	Phase 0 fixes env expectations; run gates in CI

9. Tracking

Adopt the rebuild's .otko-build/tasks.json pattern (schema, gates, {id, phase, layer, specs, title, tests, status, deps}), seeded from this plan. Each task names the phase, the spec IDs it implements, and its tests (the rebuild's tasks.json is a working reference). Keep build state out of versioned source (.gitignore it) or commit it deliberately — team choice.

10. Recommended first step

Execute Phase 0 in one branch. It is almost entirely additive, it turns the currently-inert mypy/CI gate into a real one, and it establishes the attribution and docs baseline the later ports depend on. Then proceed 1 → 2 → 3 → 4 → 5, pulling Phase 6/7 forward only if self-weight is needed before reporting.