otko/vis_improvement.md

# Plotly visualisation improvements — handoff / working brief

Status: **implemented and verified; uncommitted** on branch `fix/plotly-orbit-lag`.
This file is a self-contained brief so a fresh LLM (or human) can continue the
work without re-deriving context. Read it top to bottom before touching code.

---

## 1. Objective

Make otko's **Plotly canvas** (`src/otko/views/canvas_plotly/`) feel and look
like opstool's plotly visualisation, which the user considers the gold
standard. Earlier in this session we already fixed the two foundational bugs:

1. **Camera framing** — `layout.scene.camera` is *not* in data units; it lives
   in gl-plot3d's normalized scene space. We now compute a normalized eye
   distance (`framed_camera_distance`) and frame on the **model** bounds
   (grid excluded).
2. **Orbit / horizon** — scene uses `dragmode: "turntable"` (Z-up), and a JS
   gesture layer matches the PyVista/VTK mouse bindings.

With those done, the user asked for 5 opstool-inspired improvements. **All 5
were selected and implemented in this session.** This document records what
landed, design decisions, and what still needs doing (verification, docs,
possible polish).

---

## 2. Reference: opstool source (read-only clone)

Local clone (do **not** edit):

```
.slim/clonedeps/repos/yexiang92__opstool/opstool/vis/plotly/
```

Key files and what to read:

| File | Relevant content |
| --- | --- |
| `plot_utils.py` | `PLOT_ARGS_DEFAULT` (colours, sizes), `_plot_points_cmap` (~679), `_plot_lines_cmap` (~747), `_plot_unstru_cmap` (~837), `_make_lines_plotly` (~628) |
| `plot_resp_base.py` | `_get_plotly_dim_scene` (~220, camera/eye/2D recipe), `_make_lines_arrows` (~361), `_plot_bc` (~305), `_get_bc_points_3d/_2d` (~438–527) |
| `vis_model.py` | `plot_node_load` (~462), `plot_ele_load` (~515), `plot_beam_local_axes` (~406), `update_fig` (~633, theme/title) |
| `vis_nodal_resp.py` | `_create_mesh` (scalar contour + `show_origin` undeformed ghost ~57–169), `plot_anim` (~248), `_update_antimate_layout` / `_update_slider_layout` (~94, ~140) |
| `vis_frame_resp.py` | frame force contour + animation methods |

Important facts learned from the opstool source (do not re-litigate):

- plotly `scatter3d.marker` **and** `scatter3d.line` accept `coloraxis`,
  `colorscale`, `cmin`, `cmax`, `showscale`, `colorbar`, and array `color`.
  `layout.coloraxis` exists (plotly.py 7.1.0; it is a valid layout key even
  though `Layout()._valid_props` doesn't list it directly).
- `mesh3d` supports `intensity`/`colorscale` but **not** `coloraxis`.
- opstool's contour colouring uses a shared `coloraxis` + `cmin/cmax` from the
  response peak, with a `colorbar` carrying the component/unit title.
- opstool scales load arrows by `|F| * (min+max bound)/20 / max|F|` and tints
  each load pattern (matplotlib `winter` / `turbo_r`).
- opstool draws supports as oriented line loops (plates/circles/triangles) per
  restrained translation DOF.

---

## 3. What was implemented (the 5 changes)

### #1 Scalar contour colouring + colorbar

**Where:** `src/otko/views/canvas_plotly/trace_builder.py`

- `SceneOptions` gained `scalars`, `scalar_label`, `scalar_clim`.
- `build()` auto-derives scalars when a deformation source exposes
  `magnitudes(node_ids)` (see `DeformationSource.magnitudes` in
  `src/otko/services/deformation.py`), label `"|u|"`.
- `_build_nodes` colours `marker.color` by scalar array with
  `colorscale=self._style.response_colorscale()`, `cmin/cmax`,
  `showscale=True`, and a `colorbar` titled `scalar_label`; hover shows the
  value.
- `_build_frames` colours `line.color` per endpoint by scalar array.
- Helper `_scalar_clim()`.

**Effect:** deformed/modal views now show a displacement-magnitude contour with
a colourbar (verified by screenshot).

### #2 Undeformed "ghost" reference overlay

**Where:** `trace_builder.py` (`_build_undeformed_reference`),
`SceneOptions.show_undeformed`; `plotly_canvas.py`
(`set_show_undeformed`, `_show_undeformed`, applied in `_push_scene`);
`render_controls.py`; `menu_builder.py` (`_act_show_reference`, checkable,
Display menu); `main_window.py` (wiring + re-apply on canvas swap);
`canvas_base.py` (protocol method).

- Plotly: faint grey line trace behind the deformed shape.
- PyVista: `model_renderer.py` `_refresh_undeformed_overlay()` builds a grey
  `pv.PolyData` wireframe from `_node_original_points` + frame connectivity;
  `model_canvas.py` `set_show_undeformed`. (This made the previously duplicated
  `DeformationSource` class in `model_renderer.py` redundant — it now imports
  the one from `otko.services.deformation`.)
- **Display → Show Undeformed Reference** is enabled only in a post view and is
  auto-cleared when returning to the model (`render_controls._reset_reference_overlay`).

### #3 Load magnitude scaling + per-pattern colours + hover

**Where:** `trace_builder.py` `_build_loads`, `_pattern_colors`,
`_PATTERN_PALETTE`, `_cone_trace` (now takes `customdata` + hovertemplate).

- Arrow length is proportional to `|F| / max|F|`, clamped to `[0.15·scale, scale]`
  where `scale = 0.05 · model diagonal`. Previously all arrows were the same
  length regardless of magnitude.
- One cone trace per load pattern, tinted from `_PATTERN_PALETTE`. With a
  single pattern (nothing to distinguish) the per-type style tint is kept
  (`nodal_load_color` / `element_load_color`).
- Hover: `"<pattern> · <N/E id><br>|F| = <mag>"` via `customdata`.
- Cone sizing uses `sizemode: "scaled"` with a small unitless `sizeref` (~0.12).
  `"absolute"` interprets `sizeref` against the vector norm in *normalized*
  scene units, which rendered oversized "fins" that hid the model on large
  models (see the `snip4.png` report). `_cone_trace` docstring records this.

### #4 DOF-accurate support glyphs

**Where:** `trace_builder.py` `_build_supports`, `_support_loops`.

- Ported/modified from opstool `_get_bc_points_3d` / `_get_bc_points_2d`
  (GPL-3.0). Each restrained translation axis → a plate perpendicular to it;
  2D roller → circle; 2D ux&uy → triangle. Rotation-only supports fall back to
  a square marker.
- Emits one `scatter3d` lines trace named `"supports"`.
- **NOTICE updated** to record this GPL port (already done).

### #5 In-page plotly frame animation

**Where:**
- `html.py`: `window.otkoAnimate(payloadJson, durationMs, loops, preserveView)`
  and `window.otkoStopAnimation()`. Builds a name list, repeats it `loops`
  times, `Plotly.react` → `addFrames` → `animate` (with `mergeView` to preserve
  camera/ranges).
- `plotly_canvas.py`: `animate(options, duration_ms=40, loops=100)`,
  `stop_animation()`, `base_scene_options()`;
  `CanvasCapabilities(in_page_animation=True)`.
- `canvas_base.py`: `CanvasCapabilities.in_page_animation` (default False).
- `docks/mode_shape_animator.py`: new `playToggled(bool)` signal,
  `period_seconds()`, `set_in_page_mode(bool)`; `_on_tick` updates the scrubber
  but does **not** emit frames while `_in_page`.
- `dock_manager.py`: `_animate_mode_in_page()` builds 36 phase frames from
  `modal_to_deformation` and calls `canvas.animate(...)` when the capability is
  present; play/pause map to animate/stop.
- `render_controls._on_show_undeformed` calls `stop_animation()` if present.

---

## 4. Files changed (all uncommitted)

```
 M AGENTS.md
 M NOTICE
 M docs/QUICK_GUIDE.md
 M src/otko/services/deformation.py                 # magnitudes()
 M src/otko/views/action_handlers.py                # (earlier) Mouse Controls dialog
 M src/otko/views/canvas3d/model_canvas.py          # set_show_undeformed
 M src/otko/views/canvas3d/model_renderer.py        # ghost overlay; import DeformationSource
 M src/otko/views/canvas_base.py                    # protocol + capability
 M src/otko/views/canvas_plotly/html.py             # gestures, otkoAnimate
 M src/otko/views/canvas_plotly/plotly_canvas.py    # show_undeformed, animate
 M src/otko/views/canvas_plotly/trace_builder.py    # #1–#4
 M src/otko/views/dialogs/__init__.py               # MouseControlsDialog export
 M src/otko/views/dialogs/quick_guide.py            # nav hint
 M src/otko/views/dock_manager.py                   # in-page animation
 M src/otko/views/docks/mode_shape_animator.py      # playToggled / in-page
 M src/otko/views/main_window.py                    # reference toggle wiring
 M src/otko/views/menu_builder.py                   # _act_show_reference
 M src/otko/views/render_controls.py                # enable/reset reference
 M tests/gui/test_main_window.py
 M tests/gui/test_plotly_hover.py
 M tests/gui/test_plotly_view_preservation.py       # animation push + deformed push tests
 M tests/unit/test_plotly_trace_builder.py          # #1–#4 tests
?? src/otko/views/dialogs/mouse_controls.py         # earlier: Help dialog
?? tests/gui/test_plotly_gestures.py                # earlier: gestures
?? tests/gui/test_undeformed_reference.py           # PyVista ghost overlay
?? vis_improvement.md                               # this brief
```

---

## 5. Tests already added and passing

- `tests/unit/test_plotly_trace_builder.py`
  - `test_loads_scale_with_magnitude_and_colour_by_pattern`
  - `test_supports_use_dof_glyphs_not_markers`
  - `test_scalar_colouring_adds_arrays_and_colorbar`
  - `test_deformation_auto_colours_by_magnitude`
  - `test_undeformed_reference_overlay_is_opt_in`
  - (`_FakeDeformation` stand-in keeps the unit test free of pyvista/Qt)
- `tests/gui/test_plotly_view_preservation.py`
  - `test_animation_push_carries_prebuilt_frames` — parses the emitted
    `window.otkoAnimate(...)` payload with `json.JSONDecoder().raw_decode`.
- `tests/gui/test_plotly_gestures.py` — asserts `window.otkoAnimate` /
  `window.otkoStopAnimation` exist in the page.

Last full gate (after all of the above, except the final two tests below):
`pytest -m "not slow"` → **100% pass, only pre-existing skips**; `ruff check`
→ the pre-existing 15 findings only; `mypy core/services` → the 3 pre-existing
environment errors.

Additional tests added during the follow-up verification pass:

- `tests/gui/test_undeformed_reference.py` — exercises the **PyVista** ghost
  overlay (`_refresh_undeformed_overlay`) in MODEL vs DEFORMED mode and toggles
  it off.
- `tests/gui/test_plotly_view_preservation.py::test_deformed_push_colours_by_scalar_and_ghosts_the_reference`
  — canvas wiring: a DEFORMED push with `show_undeformed` carries the
  `undeformed-reference` trace and a node colourbar titled `|u|`.

---

## 6. What remains / known risks (DO THIS NEXT)

1. ~~Run the full gate.~~ **Done** — all green (see §5). Note: `pytest`
   segfaults at interpreter teardown in this env (QtWebEngine "profile still
   not deleted") **on the clean baseline too**; it is pre-existing, not caused
   by these changes. CI (xvfb) should be fine.
2. **Manually eyeball in-page animation on a real display (only remaining
   item).** Offscreen QtWebEngine throttles `requestAnimationFrame`, so
   `Plotly.animate`'s promise may not resolve in tests. A dispatch probe
   confirmed `addFrames(n)` and `animate(sequence)` are called with the right
   sizes, but end-to-end playback (loop, pause, camera preservation) has **not**
   been seen on-screen. Launch `python -m otko`, open a model with a modal
   result, **Display → Animate Mode Shape → Play** on the Plotly backend and
   confirm looping + Pause + Back-to-model. `loops=100` × 36 frames = 3600
   frame names; reduce if it stutters.
3. **Animation stop path — verified from source.** `Plotly.animate(gd, [],
   {mode:'immediate'})` calls `discardExistingFrames()` (plotly.js
   `plot_api.js` `animate`), so `otkoStopAnimation` does interrupt. Interrupt
   rejects the previous animate promise with no reason; the JS error handler
   only logs truthy errors to avoid console noise.
4. **PyVista ghost overlay — tested** (see §5). Still worth a glance that it
   does not interfere with node picking on-screen, but the actor is
   `pickable=False`.
5. **Docs — done** for the new features (`docs/QUICK_GUIDE.md` §2/§3,
   `AGENTS.md`, `NOTICE`).
6. **Optional polish (opstool parity not yet done)**
   - scene title with model stats + `font.family` (opstool `update_fig`);
   - **2D auto-scene for planar models — evaluated and deliberately skipped.**
     Turntable's `updateFx` forces `camera.up = [0,0,1]`, so a top view's
     screen-up is arbitrary (a planar XY truss renders rotated 90°: verified
     via screenshot). Fixing it needs per-preset up handling that fights
     turntable; not worth the risk.
   - Max/Min response annotations (`show_max_min`);
   - response component selector (currently only `|u|` magnitude);
   - smoothing/interpolated beam displacement (opstool `interpolate_beam_disp`);
   - colourbar units (currently label only, no unit string).

---

## 7. Architecture / constraints to respect

- Layering: `views → viewmodels → services → core`. `core` = stdlib+numpy+
  pydantic; `services` may use h5py/openseespy; `views` = Qt/PyVista/plotly.
  `trace_builder.py` is **pure** (no Qt, no pyvista, no plotly import) and is
  unit-tested headless — keep it that way.
- Both canvas backends implement the `CanvasBackend` protocol
  (`views/canvas_base.py`). If you add a consumer-facing method, add it to
  **both** canvases and (ideally) the protocol.
- opstool is GPL-3.0; any ported code must be recorded in `NOTICE` (already
  done for `_support_loops` and the colour/load recipes).
- Never commit `.osmodel` by hand (not relevant here).
- Do not commit changes unless the user asks. Conventional Commits style.

---

## 8. Useful commands / environment

```bash
# Python lives in the repo venv (system pytest lacks otko on sys.path)
.venv/bin/python -c "import otko"
.venv/bin/pytest tests/unit/test_plotly_trace_builder.py -q
.venv/bin/pytest tests/gui/test_plotly_gestures.py -q      # opens QtWebEngine
```

Rendering screenshots for quick visual checks (used during development):
use in-page `Plotly.toImage('plot', {format:'png', width:900, height:640})`
(a promise; poll `window.__img` from Python and base64-decode). Offscreen
`qtbot` widgets do not need `show()`; WebGL works but rAF is throttled.

Quick smoke to inspect trace shapes without Qt:

```python
from otko.services import load_project
from otko.views.canvas_plotly.trace_builder import PlotlyTraceBuilder, SceneOptions
scene = PlotlyTraceBuilder().build(load_project("examples/space_frame_3d.osmodel"), SceneOptions())
for t in scene.data:
    print(t.get("name"), t["type"])
```

---

## 9. One-paragraph recap for a fresh LLM

The Plotly canvas now frames correctly (normalized camera, turntable, model
bounds), has VTK-parity mouse gestures, and five opstool-inspired upgrades:
scalar response colouring with a colourbar, an undeformed reference overlay,
magnitude-scaled per-pattern load arrows with hover, DOF-accurate support
glyphs, and in-page plotly frame animation. Everything is implemented and
covered by focused unit/GUI tests; the remaining work is running the full gate,
eyeballing the animation and PyVista ghost on a real display, and finishing
docs. Start by reading `trace_builder.py` (`SceneOptions`, `build`,
`_build_nodes`, `_build_frames`, `_build_supports`, `_build_loads`),
`canvas_plotly/html.py` (gestures + `otkoAnimate`), and
`canvas_plotly/plotly_canvas.py` (`_push_scene`, `_camera_dict`, `animate`).