essim/DESIGN.md

# essim — notes for Claude

System digital twin: simulator for the interconnections between cards/boards. C++17, FTXUI for the TUI, importers for external netlist formats.

## Build

```sh
cmake -S . -B build
cmake --build build -j
./build/essim
```

- CMake **3.14+** required (uses `FetchContent_MakeAvailable`).
- FTXUI is fetched at configure time from GitHub (`v6.1.9`, shallow clone). First configure pays ~20 s for the clone; subsequent ones are cached in `build/_deps/`.
- **System dependencies** (resolved via `find_package`): `libzip` (target `libzip::zip`) and `pugixml` (target `pugixml::pugixml`). Used by the ODS importer. Available on Arch via `pacman -S libzip pugixml`.
- Sources are collected with `file(GLOB_RECURSE ALL_SOURCES "src/*.cpp")`. **After adding a new `.cpp`, re-run `cmake -S . -B build`** — CMake does not re-glob automatically and link will fail with "undefined reference".

## Layout

```
src/
  main.cpp               -- launches Tui
  system/                -- domain model
    syselmts.hpp           SystemElement + SystemElementContainer<T> (templated, get/merge/iterate)
    modules.{hpp,cpp}      Module, Modules
    parts.{hpp,cpp}        Part (carries `kind` + `connector_type`), Parts
    pins.{hpp,cpp}         Pin, Pins
    signals.{hpp,cpp}      Signal, Signals
    signal_type.hpp        SignalType + helpers
    component_kind.{hpp,cpp} ComponentKind enum + infer_component_kind(name)
    pin_name.{hpp,cpp}     canonical_pin_name(s) — zero-pad digit suffix to 3
    connect.{hpp,cpp}      Connection, Connections
    transform.{hpp,cpp}    Transform / IdentityTransform / TransformRegistry +
                           CheckIdentityCompatible + FillIdentityNCs
    pin_role.{hpp,cpp}     pin_role(kind, name), pin_layout(kind),
                           FillPartFromLayout(part, kind)
    nets.{hpp,cpp}         find_net / compute_all_nets / net_type_consistent
    persist.{hpp,cpp}      save / restore (tab-delimited)
    system.{hpp,cpp}       System: owns Modules + Connections, exposes Load()
  imports/               -- adapters that populate the domain
    import_base.hpp        ImportBase interface
    import_mentor.{hpp,cpp} Mentor Graphics netlist parser
    import_altium.{hpp,cpp} Altium netlist parser (`[ ]` parts, `( )` signals)
    import_ods.{hpp,cpp}   ODS spreadsheet pinout (libzip + pugixml)
  tui/                   -- FTXUI shell, split by responsibility
    tui.{hpp,cpp}          Class Tui (state + Run() orchestrator + screen-mode event dispatcher)
    tui_helpers.{hpp,cpp}  Free helpers: ToLower, NaturalLess, LongestCommonPrefix
    shell.cpp              Print, Submit, Dispatch, Finalize, Tokenize, history persistence
    completion.cpp         CompleteCommand, CompletePath, CompleteInline
    commands.cpp           RegisterCommands (all built-in commands declared here)
    screen_main.cpp        BuildMainScreen (visualisation area + bottom input)
    screen_search.cpp      BuildSearchScreen
    screen_connect.cpp     BuildConnectScreen + shared RefreshFilteredPartList helper
    screen_settype.cpp     BuildSettypeScreen
    screen_explore.cpp     BuildExploreScreen (browse modules → parts/signals/connections → details; not scriptable)
    screen_net.cpp         BuildNetScreen (BFS over connections from a starting (module, signal))
doc/classes.puml         -- PlantUML class diagram
```

`include/` and `lib/` are kept empty by design — FTXUI used to live there as precompiled `.a` + headers, now it comes through FetchContent.

## Domain conventions

- Everything in `system/` is **pointer-based** (commit `d8122d1`: "everything is pointer"). Containers store `T*`, ownership lives with the container.
- `SystemElementContainer<T>::merge(name)` is the get-or-create primitive — call it instead of `add` when you don't know whether the element already exists. `add` throws on duplicate names or empty names.
- `using namespace std;` is present in `syselmts.hpp` — pre-existing, don't add more `using namespace` in headers.
- Include guards `_NAME_HPP_` *and* `#pragma once` are both used. Match the existing style.

## TUI

`Tui::Run()` enters an FTXUI fullscreen loop:

- Top: scrollable visualisation area (`window` + `vscroll_indicator | yframe`, last line gets `focus` so it auto-scrolls).
- Bottom: single-line `Input` inside a border. Label switches between `> ` (normal) and `<question>? ` (during a multi-step prompt).
- `CatchEvent` is wrapped **outside** the `Renderer` (not between `Renderer` and `Input`). Pattern: `Renderer(input_component, lambda)` then `CatchEvent(renderer, handler)`. Wrapping `CatchEvent(input, …)` inside a `Container::Vertical({…})` and then `Renderer(container, …)` was found to break the `Input`'s content rendering on at least one terminal — typed characters didn't show even though `on_enter` fired correctly.
- `InputOption::transform` is overridden to avoid hard-coded colors (default sets `White on Black` when focused, which is unreadable on light terminal themes). The custom transform only applies `dim` to the placeholder; everything else inherits terminal default fg/bg, so the UI adapts to any theme.
- `InputOption::multiline` **must** be set to `false` for the command prompt. Default is `true`, and FTXUI's `HandleReturn()` then both inserts `\n` into the content **and** fires `on_enter` — so `Submit()` would receive `"help\n"` instead of `"help"` and command lookups would all fall through to "unknown command".
- Commands live in a `std::map<std::string, CommandSpec>` registry built in `RegisterCommands()`. Each `CommandSpec` carries `params` (list of `Param{name, completion}` where `completion` is the `Tui::Completion` enum: `None | Path | Command`), an `action(args)` lambda, a `prompt_for_missing` bool (default `true`), a one-line `description` string, a `scriptable` bool (default `true`), and an `interactive` bool (default `false`) that flags screen-opening commands. Setting `scriptable = false` (e.g. `explore`) excludes the command from the script-save buffer; if a non-scriptable command is invoked from a sourced file the `Source` loop still aborts via the `screen_idx != 0` check, since these commands are typically screen-openers. `interactive = true` is consumed only by `help`: the listing splits commands into two sections (`Interactive (open a full-screen mode)` and `Other`), and `help <name>` adds an `[interactive]` tag plus a note about the bare-vs-inline duality. `Dispatch(raw)` tokenises the input (whitespace split with `"…"` quoting), takes inline args first, and (only when `prompt_for_missing`) pushes a `Prompt` onto the queue for each missing param. The last prompt's callback calls `Finalize()`. Set `prompt_for_missing = false` for commands that accept either zero args or a full arg list (e.g. `search`: bare → interactive, full → inline) — the action gets called immediately with whatever it received and decides what to do.
- `Tab` completes by `Param::completion`: `Path` triggers `CompletePath()` (filesystem listing with `~/` expansion), `Command` triggers `CompleteCommand()` (matches against the registry), `None` does nothing. Both work inline (`CompleteInline()` figures out the arg position) and inside a multi-step prompt.
- `help` (no args) iterates the registry and prints `name — description` for every command. `help <command>` describes a single command, including each `Param`'s name. The argument has `Completion::Command`, so `help <Tab>` lists registered commands.
- `Finalize()` rebuilds the **canonical inline form** (`name arg1 "arg with spaces" arg3`) and writes that to history — so a `load` command answered via interactive prompts still shows up in `history` (and on disk) as a single inline line, ready to be replayed via ↑.
- Multi-step prompts work via a `std::deque<Prompt>` queue. `Submit()` pops them one by one before falling back to dispatch. Adding a new command = one entry in `RegisterCommands()`; the prompt-flow and inline-flow are both handled automatically.
- Tab completion: at the top-level prompt (no `pending`), completes built-in command names. Inside a prompt with `path_completion = true` (e.g. the `filename` step of `load`), completes file paths via `std::filesystem::directory_iterator` (handles `~/`, dirs get a trailing `/`). Logic: 1 match → replace; multiple with progress on the longest common prefix → extend; multiple stuck at LCP → list candidates in the visualisation area.

Built-in commands: `new`, `set`, `load`, `duplicate`, `save`, `restore`, `source`, `script-save`, `connect`, `set-type`, `set-signal-type`, `search`, `explore`, `verify`, `net`, `clear`, `help`, `quit`/`exit`. `Esc` cancels an in-progress multi-step prompt.

`set <name> <value>` declares a session-scoped variable. Subsequent commands expand `$name` and `${name}` in their args (substitution happens in `Finalize` between canonical-form recording and `spec.action(args)` — so `history` and `script-save` keep the **unexpanded** form, while the action sees resolved values). Unknown variables are left literal. `vars` is reset by `new`. Validation: `[A-Za-z_][A-Za-z0-9_]*`.

`duplicate <source> <newname>` deep-copies a module: signals (with type overrides), parts (with `connector_type` and `kind`), pins (with `expected_signal_type`), and rewires each pin to the equivalent same-named signal in the new module. Connections are NOT copied (they're cross-module topology).

`script-save <file>` writes a replay-ready script of every command issued since the last `new` (the `recorded` buffer is cleared inside the `new` action). The buffer is appended to inside `Finalize()` after the action runs, with a denylist of commands that aren't useful in a replay: `clear`, `help`, `quit`, `exit`, `source`, `script-save`. Note the `source` exclusion: when a script is sourced, the *individual lines inside* the script are recorded (because each goes through `Finalize`), so the saved script reproduces the same end state without the indirection.

`source <file>` reads a script line by line and feeds each line through `Submit()`. While the script is running, `in_source = true` is set on the `Tui` and:
- `Dispatch` / `Finalize` skip writing to memory + on-disk history.
- After each `Submit`, if `screen_idx != 0` (a screen was opened by an "interactive" command like bare `connect`/`search`/`set-type`), the script is aborted with an error message and `screen_idx` is reset to 0 — interactive screen-opening commands are explicitly disallowed in scripts.

Pending prompts (from incomplete inline commands) are NOT considered interactive and are filled by subsequent script lines, the way you'd expect. Lines starting with `#` and blank lines are skipped; leading/trailing whitespace is trimmed; `~/` is expanded.

`save` / `restore` (`src/system/persist.{hpp,cpp}`): tab-delimited line format, no extra deps. Tags: `M` (module), `P` (part with `connector_type`), `N` (pin → signal name; empty = NC), `S` (signal → type override; only emitted for non-default), `C` (connection header with endpoints + `transform_name`), `W` (wire pair within the current connection).

**Signals** carry a `type` (`SignalType::Power | GndShield | Other`) auto-inferred from the name in `Signal::Signal` via `infer_signal_type` (heuristic: GND/GROUND/SHIELD/CHASSIS → GndShield; PWR/VCC/VDD/VEE/VSS/VBAT/VS_/VS3_*/+/- prefixes → Power; else Other). Override with `set-signal-type <module> <signal> <power|gnd|other>`. The explore screen shows the type in the signal detail header.

**Pin role expectations**: every Pin carries an `expected_signal_type` populated by `set-type` from a per-(connector_type, pin_name) lookup (`src/system/pin_role.{hpp,cpp}`). The framework is wired end-to-end; the actual VPX 3U lookup table is currently a stub returning Other for all positions — fill in `vpx_3u_role(col, row, idx)` with the real VITA 46 layout when needed.

**Connector pin layout (preparation)**: `pin_layout(connector_type)` returns the canonical full pin-name list for a known connector kind, and `FillPartFromLayout(part, kind)` materialises NC pins for any layout position absent from the imported netlist. `set-type` calls it after setting `connector_type` (no-op today since `pin_layout` is a stub returning `{}` for everything — populate alongside `vpx_3u_role`). End-to-end chain in place: `set-type → FillPartFromLayout → pin_role`.

**`verify` (two passes)**: walks all typed pins and reports local mismatches between `expected_signal_type` and the actual signal type, AND walks all bridged nets reporting Power↔GndShield inconsistencies. `net <module> <signal>` prints the BFS-reached `(module, signal)` set with types and an `[INCONSISTENT]` flag.

**Component classification**: every `Part` carries a `ComponentKind kind` (`Passive | Semiconductor | IntegratedCircuit | Connector | TestPoint | Switch | Crystal | Mechanical | Other`) inferred at construction by `infer_component_kind(name)` from the leading reference-designator letter(s) (longest-match: `LED/TP/SW/FB/MK/MP/MH/HS/RA/RN/RP/RV` first, then single-letter R/C/L/F/D/Q/U/J/P/Y/X/S). Recomputed on `restore` (no persistence tag). Not yet exposed in TUI commands — branchpoints will be `search` filter, `set-type` guard, and `explore` header.

`SignalType` lives in its own header `src/system/signal_type.hpp` (extracted from signals to avoid a pins↔signals include cycle).

**Pins** are either NC (`signal() == nullptr`) or connected to exactly one signal. The ODS importer creates a Pin for every row that has a non-empty pin name, even when the signal column is empty or `"NC"` — the pin stays in the Part as NC. `restore` replaces `Tui::sys` entirely (`unique_ptr::reset`). Names are stored as-is — must not contain TAB or newline (true for the EE netlists we ingest). Format is versioned by the `# essim system snapshot v1` header for future compatibility.

**Connector types & transforms**: every `Part` carries a `connector_type` string (default `""`, set via the `set-type` command — inline `set-type m p kind` or bare which opens a TUI screen with module menu, part filter+menu, type input, list of types already in use, and an Apply button). When `connect` validates a pair, it consults `TransformRegistry::lookup(p1->connector_type, p2->connector_type)` (defined in `src/system/transform.{hpp,cpp}`) — both directions of the pair are tried. If neither is registered, an `IdentityTransform` fallback wires each pin of A to the canonical-equivalent pin of B (when present). The resulting `(Pin*, Pin*)` list and the transform's name are stored on the `Connection` (`pin_map`, `transform_name`). To register a real transform: define a `Transform` subclass in `transform.cpp` and call `TransformRegistry::get().add("kindA", "kindB", new MyTransform())` at init — there's no startup hook for this yet, so a small `RegisterBuiltinTransforms()` helper is the natural place to add when more types appear.

**Identity wiring uses canonical names**: `IdentityTransform::apply` builds `unordered_map<canonical, Pin*>` for side B and looks up each side-A pin by its canonical form. So `A1` (one card) auto-pairs with `A001` (the other) thanks to `canonical_pin_name` (`pre + zero-padded(3) digit suffix`; mixed/non-numeric returns the original). Same canonicalisation in `CheckIdentityCompatible`. **`pin_role` doesn't need canonicalisation** because `parse_pin` extracts `(col, row)` via `stoi` which already strips leading zeros.

**Subset wiring + NC backfill**: `CheckIdentityCompatible(a, b, info=&s)` accepts the case where one side's canonical pin set is a subset of the other's — typical when one importer drops NC pins (Altium) and the other doesn't (Mentor). It populates `info` with a non-fatal "N pin(s) only on '<part>'" message. Bidirectional mismatch (both sides have orphans) is still refused. After acceptance, `connect` calls `FillIdentityNCs(p1, p2)` which materialises the orphan canonical positions on the missing side as NC pins (`new Pin(other_side_name)`) — so `Connection::pin_map.size()` matches the larger side's count. Idempotent.

`screen_idx` mapping: 0 = main TUI, 1 = search, 2 = connect, 3 = set-type, 4 = explore, 5 = net. Adding a new screen mode is the same recipe each time: state members, `Container::Vertical` of focusable components, a `Renderer` lambda that recomputes derived state per frame (e.g. filtered part lists), an entry in `Container::Tab`, and Tab/Esc handling in the outer `CatchEvent`.

**Screen titles** (shared idiom): every interactive screen renders a top bar in the form `" essim "` (bold) + `"→ "` (dim) + `"<screen-name>"` (bold) + `"  — <short description>"` (dim), followed by a `separator()`. The main screen has its own variant that adds a live `N module(s), M connection(s)` counter on the right. Aim is to make the breadcrumb between essim and the current mode visible at all times.

**Focus highlighting**: each interactive screen reuses `FocusLabel(elem, focused)` from `tui_helpers.hpp` (inline helper, `focused ? e | inverted : e`) on the label of the currently-focused field so the user sees at a glance where the next keystroke lands. Indices match the `Container::Vertical` order — e.g. `connect` has 7 (m1, p1-filter, p1-menu, m2, p2-filter, p2-menu, Button), `net` has 3 (filter, module, signal). Buttons (`Connect`, `Apply`) get the highlight on the button itself, not a separate label.

**Main-screen title bar**: top of `BuildMainScreen` renders `" essim "` (bold) + `"— system digital twin"` (dim) on the left and a live `"N module(s), M connection(s)"` (or `"no system loaded"`) on the right, then a `separator()`. Built from `sys->modules()->size()` / `sys->connections()->size()` each frame — cheap. `screen_main.cpp` therefore needs to include `system/system.hpp`, `system/modules.hpp`, `system/connect.hpp` directly (forward-declared in `tui.hpp`).

**Main-screen scrollback**: the visualisation area lets you scroll through past output. State is `Tui::scroll_offset` (0 = follow tail). Keys (default branch of the outer `CatchEvent` in `Run()`): `PageUp` / `PageDown` step 10 lines, `Home` jumps to top, `End` returns to tail. `Print()` resets `scroll_offset = 0` so any new output snaps the view back to the tail (otherwise late errors would be hidden). Render: instead of `focusPositionRelative(0, 1)` always anchoring to the bottom, the screen places `| focus` on `output[size - 1 - scroll_offset]` and shows a `[scroll: -N / PgUp PgDn Home End]` indicator next to the prompt when offset > 0. `vscroll_indicator | yframe` for the FTXUI-side scroll bar.

`connect` is dual-mode (`prompt_for_missing = false`):
- Inline: `connect m1 p1 m2 p2`. Each part argument is resolved as exact name first, then case-insensitive substring; ambiguous → lists candidates and aborts.
- Bare: opens a dedicated full-screen layout (`screen_idx = 2`) with two columns (`endpoint 1` / `endpoint 2`). Each column has a module `Menu`, a part filter `Input`, and a filtered part `Menu`. A `Button(" Connect ")` at the bottom commits. `Tab` cycles focus across the 7 components, `Esc` leaves.

The connect screen rebuilds the filtered part lists inside the `Renderer` lambda each frame (cheap; lets the part menus react live to module changes and filter typing). Selection indices are clamped if a list shrinks below the previous index.

The Connection record stores `Module*`/`Part*` for both endpoints (added to `Connection` for this — minimal struct fields, no behaviour change).

`search` switches to a second full-screen layout (handled by `Container::Tab({main, search, …}, &screen_idx)`). Layout:
- Left column: `Menu` for the module list and `Menu` for the type (`parts` / `signals`).
- Right column: `Input` for the live filter query, plus a results panel rebuilt every frame.
- `Tab` cycles focus between the query input and the menus. **Implemented manually** in the outer `CatchEvent`: `Menu::OnEvent` consumes `Event::Tab` to cycle its own entries and returns `true`, which prevents `Container::Vertical` from ever seeing the event (Container only cycles between children when the active child returns `false`). So we short-circuit Tab/TabReverse upstream and mutate `search_focus_idx` directly.
- `Esc` exits the search mode (flips `screen_idx` back to 0). The search state (selected module/type, query) is preserved across re-entries until `search` is run again.

`net` is dual-mode (`prompt_for_missing = false`, `interactive = true`):
- Inline: `net <module> <signal>` — prints the BFS-reached `(module, signal)` set in the visualisation area (with types and an `[INCONSISTENT]` flag).
- Bare: opens `screen_idx = 5`. Three columns: module `Menu` (left), filter `Input` + filtered signal `Menu` of the selected module (middle), and a read-only panel (right) that recomputes the net on every frame and lists `(module, signal, type)` for each member plus a header summarising count + dominant type + inconsistency flag. The signal list is sorted with `NaturalLess`; `net_sig_idx` is clamped if the filter shrinks it. `Tab` cycles 3 fields (filter → module → signal); `Esc` leaves.

**Long-running scripts (`source`)**: `Source(file)` is event-paced. It reads all lines, sets `loading = true`, then spawns a detached pacing thread that posts `Event::Special("\x02tick")` every ~30 ms — **but only one tick at a time**: the main thread acks each one (`tick_in_flight = false` at the end of `ProcessNextSourceLine`) before the ticker sleeps and posts the next. Without this, FTXUI batches all queued events into one render pass; a long line (e.g. a heavy Mentor parse) would let the ticker queue many ticks and the modal would freeze. Each tick triggers `ProcessNextSourceLine`, which processes one effective line (skipping comments/blanks) via `Submit()`. A centred `borderDouble` modal (`" Computing… "` + filename + `N / M lines`) is overlaid via `dbox` while `loading` is true.

Adding a new screen mode = drop a `screen_<name>.cpp` defining `Component Tui::BuildXxxScreen()`, add corresponding state members in `tui.hpp`, register a `BuildXxxScreen` declaration in the screen-builders block, then in `tui.cpp::Run()` add a child to `Container::Tab` and a case in the screen-mode `switch` of the outer `CatchEvent` (Tab cycling + Esc-leave). Commands lived in `commands.cpp` set `screen_idx` to enter the new mode.

Command history is persisted on disk and loaded on startup. Path resolution is platform-aware:
- Linux/macOS: `$XDG_DATA_HOME/essim/history`, falling back to `~/.local/share/essim/history`.
- Windows: `%LOCALAPPDATA%\essim\history`, falling back to `%APPDATA%\…` then `%USERPROFILE%\AppData\Local\…`.

Each successful submission appends a single line to the file (so a crash doesn't lose history). Multi-step prompt answers are NOT persisted — only top-level commands.

## Gotchas

- All three importers (`IMPORT_MENTOR`, `IMPORT_ALTIUM`, `IMPORT_ODS`) are wired in `System::Load`. Wrap calls in `try/catch` (the TUI does).
- **Altium importer drops NC pins entirely**: the source format only enumerates pins inside `(signal …)` blocks, so positions not connected to any signal on this card never become `Pin`s. Mentor (via `Explicit Pin:`) and ODS (one row per pin) materialise NC. This is the asymmetry that motivates `FillIdentityNCs` at `connect` time and (eventually) `FillPartFromLayout` at `set-type` time.
- ODS importer: each spreadsheet sheet becomes a `Part` (sheet name = part name). Rows are pin/signal pairs; the **first non-empty row of each sheet is dropped as a header** (no validation of header content). Empty cells skip the row; `"NC"` keeps the pin in the part but doesn't connect it to a signal. Pins or parts whose name collides (rare in well-formed sheets) are silently dropped.
- `System::Load` throws `std::runtime_error("Unknown import type")` for any value outside the three enum cases.
- `Modules`/`Parts`/etc. have no const-correct iteration on the `*` accessor; iterators on `SystemElementContainer<T>` are available but `begin()`/`end()` are non-const-safe in some places.

## Documentation

`doc/` hosts:
- `api/` — auto-generated Markdown API reference, browseable directly in gitea.
- `Doxyfile.in` — CMake-templated Doxygen config (XML-only output to `build/doc/xml/`).
- `gen_api_md.py` — custom XML → Markdown emitter (~330 lines, stdlib-only Python).
- `README.md`, `classes.puml` — meta + diagram.

Pipeline: `doxygen` extracts the XML from `src/**`, then `gen_api_md.py` walks `index.xml` plus each `class*.xml` / `*_8hpp.xml` and emits one Markdown file per class/struct/file plus an index. Source-code references use relative paths (`../../../../src/...#L42`) so gitea's renderer turns them into clickable line-anchored links. We picked a custom emitter over `doxybook2` (not in Arch/AUR) and `moxygen` (Node dep) to keep the toolchain pure-C++/Python and trivially modifiable.

Regenerate with `cmake --build build --target doc` after substantive code changes — the target is auto-created when both Doxygen and Python 3 are present at configure time; otherwise the regular build target is unaffected and a status line records which tool is missing. The Markdown output is committed (`doc/api/`), so gitea readers see fresh docs without building.

## Memory layout for sessions

Persistent notes live in `~/.claude/projects/-home-francois-Projets-essim/memory/`. Index: `MEMORY.md`. Add project/feedback/user/reference entries there when relevant — see top-level Claude Code memory rules.

This file is `DESIGN.md` (renamed from the original `CLAUDE.md`).