otko/docs/adr/ADR-0001-gidopensees-schema-import.md

# ADR-0001 — Import gidopensees Schemas into OTKO

| Field | Value |
|---|---|
| **Status** | Proposed |
| **Date** | 2026-05-22 |
| **Author** | ogunc |
| **Deciders** | Core maintainers |
| **Source project** | [gidopensees](https://github.com/rclab-auth/gidopensees) — AUTh Lab of R/C and Masonry Structures |

---

## 1. Context

### Why gidopensees?

gidopensees is the most complete published schema inventory for the OpenSees
material/element/condition set. It covers 60+ material types, 30+ element
types, and 31 boundary-condition and load types — all expressed as GiD
preprocessor `.mat` / `.cnd` BOOK definitions, with DEPENDENCIES (field
visibility rules), `#UNITS#` annotations, and TKWIDGET hooks for
auto-fill presets and Wiki links.

OTKO currently supports ~34 of these objects (see
`docs/gap-analysis-gidopensees.md`). Importing gidopensees schemas would
close 23 P1 gaps (Phase 8 targets) and 21 P2 gaps without requiring us to
reverse-engineer OpenSeesPy docs for each type.

### What we are NOT doing

This ADR covers **schema definitions only** — Pydantic model fields, field
metadata, and default values. It does not cover:

- The `bas/` Python code-generation templates (separate ADR).
- The `tcl/` TKWIDGET Tcl implementations (separate ADR).
- Any runtime OpenSeesPy command emission — that lives in
  `services/opensees_runner.py` and follows naturally once a schema
  is accepted.

---

## 2. Decision

### 2.1 New package: `core/catalog/`

A new package `src/otko/core/catalog/` will hold
gidopensees-derived schema definitions alongside curated additions.

```
src/otko/core/catalog/
├── __init__.py          # re-exports curated + generated public symbols
├── generated/           # output of the codegen tool — DO NOT edit by hand
│   ├── _header.py       # shared attribution header (inserted by codegen)
│   ├── steel.py
│   ├── concrete.py
│   ├── other_uniaxial.py
│   ├── nd_materials.py
│   ├── sections.py
│   └── elements.py
└── curated/             # human-reviewed, hand-edited overrides and additions
    ├── README.md        # explains the curated/ contract
    └── …
```

**Existing modules are untouched.** The objects in `core/materials/`,
`core/sections/`, `core/geometry/elements.py` remain authoritative for
every type already supported. Catalog objects enter the UI only after
equivalence is established (see §2.7).

### 2.2 Codegen tool: `tools/gidopensees_import/`

A build-time parser + code-generator lives in
`tools/gidopensees_import/`, outside `src/`:

```
tools/gidopensees_import/
├── README.md
├── parse_mat.py         # parses OpenSees.mat into an intermediate IR
├── parse_cnd.py         # parses OpenSees.cnd into an intermediate IR
├── codegen.py           # renders IR → Pydantic v2 model source files
├── ir.py                # intermediate representation dataclasses
└── tests/               # unit tests for the parser and codegen
```

The tool runs once per gidopensees update. Its output (`generated/`) is
committed so CI never requires gidopensees to be present. The tool is
invoked manually by a maintainer:

```bash
python tools/gidopensees_import/codegen.py \
  --mat path/to/OpenSees.mat \
  --cnd path/to/OpenSees.cnd \
  --out src/otko/core/catalog/generated/
```

**Rationale:** keeping the parser outside `src/` prevents it from being
imported at runtime, avoids adding GiD file parsing as a dependency of
the installable package, and makes the "run once, commit output" contract
explicit to contributors.

### 2.3 Attribution

Every file under `core/catalog/generated/` carries this header comment
(inserted by `codegen.py`):

```python
# This file is derived from gidopensees.
# Source: https://github.com/rclab-auth/gidopensees
# Authors: AUTh Lab of R/C and Masonry Structures
#          (https://rclab.civil.auth.gr/)
# Modifications: generated by tools/gidopensees_import/codegen.py
```

Files under `curated/` carry a similar header when they derive from
gidopensees content.

### 2.4 `.osmodel` format backward-compatibility

The `.osmodel` JSON format uses Pydantic discriminated unions keyed on a
`"type"` field. New material types from `core/catalog/` appear as new
discriminator values. The union in `core/materials/__init__.py` is
extended only when a catalog type is promoted to stable (see §2.7).

Old project files that do not contain the new discriminator values load
cleanly: Pydantic ignores unknown items in lists when
`model_config = ConfigDict(extra="ignore")`, and the `Project` validator
will log (not raise) on unknown type strings if we add a graceful
fallback.

**Migration path:**
1. Catalog type is added with a temporary discriminator value
   (e.g. `"catalog.Concrete04"`).
2. After verification (§2.7) it is promoted to a stable value
   (e.g. `"Concrete04"`) and the temporary value is kept as an alias for
   one minor version.
3. A `migrate_osmodel.py` script in `tools/` handles the rename if needed.

No existing `.osmodel` file ever breaks on open.

### 2.5 Field metadata for TKWIDGET hooks

gidopensees BOOK definitions contain TKWIDGET directives that fire
auto-fill presets (e.g. `SteelUniaxMaterial::GenerateValues`), Wiki links
(`TK_MaterialWikiInfo`), and the Material Tester dialog
(`TK_MaterialTester`). These are deferred from this ADR.

Each field or model where a TKWIDGET hook is relevant carries a
`json_schema_extra` annotation recording the hook name:

```python
class Concrete04(CatalogEntity):
    ...
    class model_config(ConfigDict):
        json_schema_extra = {
            "tkwidget_hooks": [
                "ConcreteUniaxMaterial::GenerateValues",
                "TK_MaterialWikiInfo",
                "TK_MaterialTester",
            ]
        }
```

This metadata is visible to future UI layers without coupling `core/` to
Qt. Implementation of the actual preset dialogs and Wiki-link buttons
comes in a later UI phase.

### 2.6 DEPENDENCIES → `dependent_schemas` metadata (not validators)

gidopensees BOOK DEPENDENCIES express field-visibility rules:

```
(1, RESTORE, Gap_length, #CURRENT#), (0, HIDE, Gap_length, #CURRENT#)
```

These are **viewmodel / UI concerns**, not data-validity rules, and
therefore must not become Pydantic validators in `core/`.

Each Pydantic model that has visibility dependencies stores them as
`json_schema_extra["dependencies"]` — a list of dicts describing the
trigger field, trigger value, and affected fields. The viewmodel layer
reads these at dialog-construction time to wire up the show/hide logic.

**Example:**

```python
class ViscousDamper(CatalogEntity):
    activate_gap: Literal[0, 1] = 0
    gap_length: float | None = None

    model_config = ConfigDict(
        json_schema_extra={
            "dependencies": [
                {"trigger": "activate_gap", "value": 1, "restore": ["gap_length"]},
                {"trigger": "activate_gap", "value": 0, "hide": ["gap_length"]},
            ]
        }
    )
```

### 2.7 Unit-annotated fields (`#UNITS#`)

gidopensees marks numeric fields with `#UNITS#` where the value's
interpretation is unit-system–dependent (forces in kN, lengths in m, etc.).

OTKO already has `core/units.py` with `UnitSystem` and
`UnitLabels`. It does **not** currently attach unit metadata to individual
model fields — the unit system is a project-level property and all numeric
values are stored in the project's native unit system, with `UnitLabels`
used only for display.

**Intended design:** Generated catalog models will mark unit-annotated fields
using a `Field` `metadata` entry (Pydantic v2 `Annotated` style):

```python
from typing import Annotated
from pydantic import Field

class UnitTag:
    """Marker for fields whose display label depends on UnitSystem."""
    def __init__(self, quantity: str):
        self.quantity = quantity  # e.g. "force", "length", "stress"

ForceMagnitude = Annotated[float, UnitTag("force")]
LengthValue    = Annotated[float, UnitTag("length")]
StressValue    = Annotated[float, UnitTag("stress")]
```

These type aliases would live in `core/catalog/_units.py`. No conversion logic
is added to `core/`; the viewmodel layer reads `UnitTag.quantity` to
select the right `UnitLabels` field for axis labels and input hints.

**Implementation status (2026-05-22) — DEFERRED:** The codegen tool does not
yet emit `UnitTag` annotations. Fields corresponding to `#UNITS#` entries in
the gidopensees source are currently emitted as `str` with a
`# TODO: unit-aware type` comment preserving the gidopensees default string
(e.g. `yield_stress_fy: str = '500 MPa'  # TODO: unit-aware type`). This is
a conscious deferral: the `str` placeholder keeps the field present and
round-trippable without binding the codebase to a unit-system convention that
is not yet finalised. A dedicated unit-system layer — covering `UnitTag`,
`_units.py`, and viewmodel wiring — is tracked as future work and will be
addressed in a follow-up ADR before any `#UNITS#` field is promoted to stable.

**If this convention is inadequate** (e.g. if we need per-field unit
conversion in the future), a follow-up ADR should address it before
the convention is applied beyond `catalog/`.

### 2.8 Verification plan

A catalog type is promoted from generated → stable only when **all three**
of the following are satisfied:

1. **Schema equivalence test** (`tests/unit/catalog/test_<name>_schema.py`):
   Constructs a model instance with the same arguments as the
   corresponding Tcl example from the OpenSees Wiki and asserts that
   `model.model_dump()` produces the expected dict. No OpenSeesPy import.

2. **Round-trip test** (`tests/unit/catalog/test_<name>_roundtrip.py`):
   Serialises the model to JSON (`.osmodel` fragment), deserialises it
   back, and asserts equality. Confirms the discriminator and all field
   aliases survive the round-trip.

3. **Integration smoke test** (`tests/integration/catalog/test_<name>.py`):
   Builds a minimal project using the new type, runs it through
   `OpenSeesRunner`, and checks that the runner does not raise and that at
   least one result quantity (reaction, displacement, or force) is finite.
   Tagged `@pytest.mark.slow` and skipped if `openseespy` is not installed.

Manual review checklist (for the PR that promotes a type):

- [ ] Attribution header present in the generated file.
- [ ] `json_schema_extra["tkwidget_hooks"]` populated where applicable.
- [ ] `json_schema_extra["dependencies"]` populated for every DEPENDENCY
      in the source BOOK.
- [ ] `#UNITS#` fields use the correct `UnitTag` quantity string.
- [ ] The type discriminator value does not collide with any existing type
      in `core/materials/__init__.py`, `core/sections/__init__.py`, or
      `core/geometry/elements.py`.
- [ ] The integration smoke test result has been spot-checked against the
      gidopensees wiki reference or an independent OpenSees Tcl run.

---

## 3. Alternatives considered

### 3A: Extend existing `core/materials/__init__.py` directly

Rejected. The existing module is small and well-tested; adding 60+
unverified types creates noise and makes equivalence tracking harder.
A separate `catalog/` namespace keeps the boundary clear.

### 3B: Use gidopensees at runtime (import `.mat` on startup)

Rejected. The GiD BOOK format is proprietary and requires the GiD parser.
Depending on gidopensees at runtime adds a third-party dependency to the
installed package and makes offline / air-gapped installs harder. The
codegen + committed-output approach keeps the package dependency-clean.

### 3C: Hand-write every new type without the codegen tool

Would work but loses the systematic relationship between gidopensees
DEPENDENCIES / TKWIDGET metadata and the Pydantic model. The codegen
pipeline preserves that metadata structurally so UI implementors can
reference it rather than re-reading `.mat` files.

---

## 4. Risks

| Risk | Likelihood | Impact | Mitigation |
|---|---|---|---|
| Namespace collision between `catalog/` and existing `core/` types | Medium | Medium | Verify discriminator values before promotion; CI check added to the merge checklist |
| gidopensees schema drift (upstream changes `.mat`) | Low | Medium | `generated/` is committed; only re-run codegen intentionally; diff the output and add a CHANGELOG entry |
| Testing surface explosion (60+ new types × 3 test tiers) | High | Low | Only promoted types get full test coverage; generated-but-not-yet-promoted types have schema + round-trip tests only |
| Attribution omission | Low | High | Codegen always inserts the header; a pre-commit hook (`grep -r "rclab-auth/gidopensees" core/catalog/generated/` must pass) enforces it |
| `#UNITS#` convention inadequacy | Medium | Medium | Convention is isolated to `core/catalog/_units.py`; a follow-up ADR can replace it without touching existing `core/` code |

---

## 5. Out of scope for this ADR

- `bas/` Python template parsing and code generation (separate ADR).
- `tcl/` TKWIDGET Tcl implementations (separate ADR).
- Seismic isolator element schemas (`elastomericBearing*`,
  `frictionPendulumBearing`, etc.) — those are new elements, not directly
  in the gidopensees BOOK format; they get their own ADR.
- IDA batch runner (analysis feature, not schema).
- Fiber-section visual editor UI polish (UI feature, not schema).
- The OpenSeesPy command emission side — `services/opensees_runner.py`
  will need updates for each promoted type, but those changes follow
  naturally from the schema and are reviewed in the same PR as the
  integration smoke test.