docs: add 0.2.1 release note (load-time optimisations + fix)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

DESIGN.md

@@ -224,27 +224,66 @@ Four distribution channels coexist, all sharing the single `src/testium/` packag
 The `.deb` work-in-progress lives in `package/deb/`:
 - `test_distro.sh debian:bookworm | debian:trixie | ubuntu:24.04` spins up a Docker/Podman container, reports system package availability, falls back to pip for what's missing (`pyside6` on bookworm/ubuntu, `telnetlib3`, `junit_xml`), runs the validation suite. Currently green on the three targets.
 
+### Building all channels (`build_all.sh`)
+
+`build_all.sh` builds every artifact into `dist/` (manual PDF, wheel, PyInstaller binary, Flatpak bundle, AppImage). It reuses `scripts/build_env.sh` + `set_env.sh` so the venv at `test/tmp/.venv` stays the single source of Python deps; `build`/`pyinstaller`/`sphinx`/`linuxdoc` (and `pygls`, via the `[lsp]` extra) are installed there on demand. A step is skipped if its artifact already exists; `--clean` forces a rebuild.
+
+- **Parallelism (default).** A serial *prep* phase does everything that writes the shared venv (the `pip install`s) plus the Flatpak runtime install and the wheel (the AppImage installs it). Then manual + PyInstaller + Flatpak + AppImage build concurrently — they only *read* the venv, so there is no concurrent-pip race. Per-step output goes to `dist/.build-logs/<step>.log`; results print in completion order (`wait -n`), and a failing step's log is dumped at the end. `--serial` builds one at a time. Ctrl+C is trapped to kill each job's whole process tree (subshell + grandchildren: podman container, flatpak-builder, pyinstaller), so no orphans survive.
+- **`--ram` (slow/flash storage).** Redirects the build scratch to `/dev/shm` and skips UPX, a large win when building from a USB stick / SD card (I/O-bound on flash): `TMPDIR` + `PIP_CACHE_DIR`, the PyInstaller `--workpath` (`PYI_WORKPATH`), and a tmpfs bind-mount at the in-container AppImage AppDir (`APPIMAGE_APPDIR_TMPFS`); UPX is disabled via `TESTIUM_NO_UPX` (read by the `.spec`). **Flatpak is excluded** — `flatpak-builder` mounts its state dir with `rofiles-fuse` and FUSE cannot mount on `/dev/shm` (`fusermount: Permission denied`), so it builds on disk. Each `package/*/build.sh` honours these env vars with on-disk defaults, so behaviour is unchanged without `--ram`; the tmpfs scratch is freed on exit. On a RAM-limited machine combine with `--serial`.
+
 ### Host-only py_func / lua_func in sandboxed bundles (Flatpak, AppImage)
 
-The bundled Python (Flatpak's runtime python, AppImage's `python3.11`) is reserved for the **main process only**. Subprocesses (`py_func`, `lua_func`, `git`) must use the host's interpreters and tools so user-installed modules (pyserial, junit_xml, …) are visible. This is enforced by `interpreter/utils/bins.py`:
+The bundled Python (Flatpak's runtime python, AppImage's `python3.11`) is reserved for the **main process only**. Subprocesses (`py_func`, `lua_func`, `git`, the `run` item's sub-instance) must use the host's interpreters and tools so user-installed modules (pyserial, junit_xml, …) are visible. This is enforced by `interpreter/utils/bins.py`:
 
 - `_in_flatpak()` (checks `/.flatpak-info`) and `_in_appimage()` (checks `APPIMAGE` env var) detect the sandbox.
-- `_which(name)` probes only host bin dirs in those modes:
-  - Flatpak: `/run/host/usr/{local/,}bin`, `/run/host/bin` (host mounted via `--filesystem=host-os`).
-  - AppImage: `/usr/local/bin`, `/usr/bin`, `/bin` (we are directly on the host filesystem).
-  - If the host has no python3/lua, `ensure()` raises `ETUMRuntimeError` at test load with the candidate list — no silent fallback to a bundled interpreter.
-- User overrides (`python_bin`/`lua_bin` in globdict): bare names are resolved through `_which()` (host-only), absolute paths are accepted as-is.
-- `apply_host_libs(env)` is called by `py_process.py` / `lua_process.py` on the env passed to Popen:
-  - Flatpak: prepends host lib dirs to `LD_LIBRARY_PATH` so the dynamic linker finds host `.so`'s.
-  - AppImage: strips `$APPDIR`-prefixed entries from `LD_LIBRARY_PATH` / `PYTHONPATH` / `PATH` and drops `PYTHONHOME`, so the host Python doesn't try to load the bundled stdlib/site-packages.
-- `apply_host_lua_paths(env)` (Flatpak only) prepends `/run/host/usr/{lib,share}/lua/X.Y` to `LUA_PATH` / `LUA_CPATH` so `cjson`, `socket`, etc. resolve. Must be called **after** user `lua_env` overrides so host paths win. AppImage relies on host Lua's compiled-in defaults.
+- **Flatpak**: the sandbox glibc/ABI is incompatible with arbitrary host shared libraries, so we **cannot** run host binaries inside the Flatpak runtime — `LD_LIBRARY_PATH` injection trips a `_dl_call_libc_early_init` assertion. The supported way out is `flatpak-spawn --host`, a stub on `$PATH` inside every Flatpak that proxies an `exec` over D-Bus to the host's `org.freedesktop.Flatpak` service. The manifest grants `--talk-name=org.freedesktop.Flatpak` so the call is allowed. Helpers:
+  - `flatpak_host_spawn(interp, args, host_cwd, extra_env=…)` builds the spawn command vector with a curated set of forwarded env vars (`HOME`, `USER`, `DISPLAY`, `DBUS_SESSION_BUS_ADDRESS`, …) plus any explicit overrides.
+  - `_get_host_testium_path()` returns a path to the testium package the host can read. In Flatpak the package lives under `/app/lib/testium` which the host cannot see, so the package is staged once per process under `/tmp/testium_host_*` (`/tmp` is shared) and reused. In source / wheel / PyInstaller installs under `$HOME` the original path is returned untouched.
+  - `_which_host_flatpak(name)` resolves a binary by spawning `command -v` on the host (or `test -x` for absolute paths) — sandbox-visible probing under `/run/host/...` is unreliable (only `host-os` is mounted; user paths like `/scratch` aren't there).
+  - `_python_version()` and `_lua_version()` go through `_run_probe()` which dispatches to `flatpak-spawn` in Flatpak so validation happens against the actual host interpreter.
+  - `py_process.py` / `lua_process.py` `start()` use `flatpak_host_spawn` with `host_cwd = _get_host_testium_path()[+/lua_func]` and forward `PYTHONPATH` / `LUA_PATH` / `LUA_CPATH` / `PATH` as `--env=` arguments.
+  - The `run` item's `_testium_launch_cmd()` prefixes `flatpak run org.testium.Testium` with `flatpak-spawn --host` so the sub-instance is launched by the host's `flatpak` CLI, not by an unworkable in-sandbox `flatpak` binary.
+- **AppImage**: we are directly on the host filesystem, so the regular discovery on `/usr/local/bin`, `/usr/bin`, `/bin` suffices. `apply_host_libs(env)` strips `$APPDIR`-prefixed entries from `LD_LIBRARY_PATH` / `PYTHONPATH` / `PATH` and drops `PYTHONHOME` so the host Python doesn't try to load the bundled stdlib/site-packages.
+- User overrides (`python_bin`/`lua_bin` in globdict): in Flatpak, both bare names and absolute paths go through `_which()` so they are validated on the host side (the sandbox can't see e.g. `/scratch/...`). Outside Flatpak, absolute paths are accepted as-is and bare names go through PATH discovery.
+- If the host has no python3/lua, `ensure()` raises `ETUMRuntimeError` at test load with the candidate list — no silent fallback to a bundled interpreter.
 - `py_process.py` additionally pops `PYTHONUSERBASE` (set to `/var/data/python` by the Flatpak runtime, which would hide `~/.local/lib/...`).
 
+### Declarative test item parameters
+
+Each `TestItem` subclass declares its accepted parameters as a class attribute `PARAMS = ParamSet(Param(...), ...)` (`interpreter/utils/param_decl.py`). The descriptor carries the parameter name, *kind* (`SCALAR` — the default and may be omitted; `LIST`; `BLOCK`; `Enum("a", "b", ...)`), `required` flag, `default`, and free-form `doc`. There is **no Python type** in the descriptor on purpose: most parameter values are expressions (`$(...)` / `<| ... |>`) whose effective type is only known after expansion, so a static type would be misleading. Post-expansion `validate=lambda v: ...` callbacks are available as an opt-in for the rare cases where a runtime check is warranted (e.g. a specific format).
+
+`TestItem.COMMON_PARAMS` (in `test_item.py`) declares the 14 parameters accepted by every item: `name`, `doc`, `skipped`, `key`, `stop_on_failure`, `execute_on_stop`, `process_result`, `store_result`, `expected_result`, `no_fail`, `report`, `condition`, `steps`, and the internal `seq_filename` injected by the loader. The base class concatenates `COMMON_PARAMS + subclass.PARAMS` in `_validate_declared_params()` and:
+
+- emits a `tm.print_warn(...)` listing the accepted names when an unknown key appears in the user YAML (catches typos like `param_filee`);
+- raises `ETUMSyntaxError` (with the `.tum` source as context) when a `required=True` param is missing.
+
+Validation is **opt-in per subclass**: while a subclass keeps `PARAMS = None` (the base-class default), the check is skipped entirely. This kept the migration incremental — items can be visited one by one without forcing a big-bang change. All structured items have been migrated; only the "unstructured-body" classes (`TestItemConsoleWrite`/`WriteLn` which carry the message as the raw value, `TestItemPlotActionAdd`/`Export` which take arbitrary plot-data keys, `TestItemUnittestElement` which is internally instantiated with `dict_item=None`) intentionally remain unvalidated.
+
+Diagnostics are currently **warnings** for unknown params so an out-of-tree `.tum` with a pre-existing typo doesn't suddenly fail. The flip to a hard error is a one-line change in `_validate_declared_params()` once the user is comfortable.
+
+Action items follow the same declarative principle. A `TestItemActions` parent (`console`, `plot`, `json_rpc`) declares its nested actions as a class attribute `ACTIONS = {yaml_key: action_class}` (e.g. `{"open": TestItemConsoleOpen, "write": …}`), mirroring `PARAMS`. The base `TestItemActions.__init__` seeds `self.action_classes` from `type(self).ACTIONS`; the imperative `register_actions(**…)` method is retained only as an escape hatch for actions that can't be known at class-definition time (none today). Because the action classes are always defined above their parent in the module, the class-level dict resolves without forward-reference gymnastics.
+
+The schema is the realized source of truth for the LSP server (`testium lsp`), the `testium schema` CLI dump, and future auto-generated manual sections: `ParamSet.to_schema()` returns the JSON-Schema-shaped representation, and `lsp/schema.py` reads both `PARAMS` and `ACTIONS` **purely from class attributes** — no `inspect.getsource`/AST parsing. This is what lets the full schema (including nested actions) survive a frozen PyInstaller build where the `.py` source isn't on disk.
+
+### Language server (`testium lsp`) across channels
+
+The `testium_assist` editor extension is a thin LSP client that spawns `testium lsp` and talks JSON-RPC over stdio, so the language server must work from *every* distribution channel. Two requirements:
+
+1. **`pygls` (+ `lsprotocol`, `cattrs`, `attrs`, `typing_extensions`) must be bundled.** It is the pyproject `[lsp]` extra (kept optional so a plain `pip install testium` stays lean), wired into each full-app channel: `build_env.sh` installs it into the shared `test/tmp/.venv` (covers **source run** and the **PyInstaller** build env); the **AppImage** installs the wheel as `…whl[lsp]`; the **Flatpak** adds a `python3-lsp` pip module (network-at-build, consistent with the manifest's global `--share=network`); the **PyInstaller** `.spec` force-collects the submodules via `collect_submodules` + explicit `hiddenimports` (including the lazily-imported `lsp`, `lsp.server`, `lsp.schema`).
+2. **The schema must build without source** — handled by the declarative `PARAMS`/`ACTIONS` above; PyInstaller is the only channel that strips `.py` source, and it no longer matters.
+
+`test/validation/lsp_check.py` enforces both per channel: `run.sh` calls it before launching the suite, asserting that `<channel> schema` returns JSON whose `console`/`plot`/`json_rpc` items still carry their actions, and that `<channel> lsp` answers an `initialize` request with capabilities (and never reports the pygls dependency missing). So `./test/validation/run.sh --mode flatpak|pyinstaller|appimage` now fails loudly if a channel ships a broken or pygls-less language server.
+
 ### Version reporting (`interpreter/utils/version.py`)
 
 Both Flatpak and AppImage export `TESTIUM_VERSION` from a launcher (Flatpak: launcher script in `org.testium.Testium.yaml`; AppImage: `runtime.env` in `AppImageBuilder.yml`). `get_testium_version()` checks `/.flatpak-info` / `APPIMAGE` and reads `TESTIUM_VERSION` rather than relying on package metadata or repo introspection.
 
 ## Recent fixes / notable changes
+- `build_all.sh`: builds the four heavy channels in parallel (serial prep for the shared venv + wheel), results in completion order, Ctrl+C kills the whole job tree; `--ram` puts the build scratch on tmpfs (`/dev/shm`) + skips UPX for fast builds on USB/SD storage (Flatpak excluded — rofiles-fuse can't mount tmpfs). See the "Building all channels" section.
+- LSP across packaging channels: `testium lsp` (and the `testium_assist` editor extension that spawns it) now works from source, wheel, PyInstaller, Flatpak and AppImage. Two enablers — (1) action items declare a class-level `ACTIONS = {key: class}` registry (like `PARAMS`), so `lsp/schema.py` builds the full schema from class attributes with no `inspect.getsource`/AST (which broke under frozen PyInstaller); (2) the `[lsp]` extra (pygls) is wired into every full-app channel. `test/validation/lsp_check.py`, run by `run.sh` before the suite, asserts per-channel that `schema` keeps its actions and `lsp` answers `initialize`. See the matching architecture sections.
+- Declarative test item parameters (v0.2): each `TestItem` subclass exposes a `PARAMS = ParamSet(...)` class attribute consumed by the base `__init__`. Catches unknown YAML keys (typo warnings listing the accepted names) and missing required params (load-time errors with `.tum` context). Lays the schema foundation for a future LSP server and auto-generated manual sections. See the matching architecture section.
+- Flatpak: `py_func` / `lua_func` / `run` sub-instance now execute on the host via `flatpak-spawn --host`. The previous attempt to inject host lib dirs into the sandbox's `LD_LIBRARY_PATH` was abandoned — host shared libs are ABI-incompatible with the Flatpak runtime's glibc and would trip `_dl_call_libc_early_init`. The manifest gained `--talk-name=org.freedesktop.Flatpak` so the spawn proxy call is allowed. The testium package is staged once per process under `/tmp` (shared with the host) so the host interpreter can locate `py_func` / `lua_func`.
+- Validation suite: single entry point with `--mode source|wheel|pyinstaller|flatpak|appimage` to validate every packaging channel against the same items. Per-mode report filenames prevent clobbering.
 - Restructure: single `src/testium/` Python package (was 4 sibling top-levels: `testium`, `lib`, `py_func`, `lua_func`). `lib/` → `runtime/`, `libs/` → `api/`. `pip install` now produces a clean `site-packages/testium/` with no top-level pollution; `.lua` files travel via `package_data`.
 - `bins.py`: centralised resolution + cache of external `python3` / `lua` binaries. Replaces the scattered `tm.gd("python_bin")`/`tm.gd("lua_bin")` dance and the duplicated discovery logic in `py_process.py`/`lua_process.py`. Validates at test load via `TestSet._validate_runtime_deps()` so missing interpreters fail fast.
 - Subprocess API contract: user scripts in `py_func`/`lua_func` use the JSON-RPC bridge (`py_func.tm` / Lua `tm`) — never `api.testium` / `interpreter.*` directly. `SUPPORTED_API` extended with `OS`, `get_main_dir`, `init_timestamp`, `timestamp`, `timestamp_as_sec` so subprocess scripts have the same surface as main-process code.
@@ -275,10 +314,12 @@ Both Flatpak and AppImage export `TESTIUM_VERSION` from a launcher (Flatpak: lau
 ## Validation tests
 Located in `test/validation/`. Two entry points:
 ```
-./test/validation/run.sh        # wrapper — uses a dedicated venv (see below)
-./run.sh -b -- test/validation/main.tum   # direct — testium's own python is used for test execution
+./test/validation/run.sh [clean] [--mode MODE] [extra args]   # wrapper — uses a dedicated venv (see below)
+./run.sh -b -- test/validation/main.tum                       # direct — testium's own python is used for test execution
 ```
-The `run.sh` / `run.bat` wrappers create a dedicated Python venv at `${TMPDIR:-/tmp}/testium-validation-venv` (Linux) or `%TEMP%\testium-validation-venv` (Windows), with `--system-site-packages` + `pip install junit-xml`, and run the suite with `-d python_bin=…` so every test-execution subprocess (eval_proc, py_func, cycle, post_exec) runs inside the venv. testium itself keeps running in the project's own environment. `clean` as the first argument recreates the venv.
+The same item set is reused across every packaging channel — `--mode source|wheel|pyinstaller|flatpak|appimage` selects which testium binary launches the suite (`source` is the default, invoking the project's `run.sh`). Each mode stamps its results into a distinct report file (`validation-<mode>.sqlite`, `validation-<mode>-<item>.xml`) so successive runs in different modes don't clobber each other. Prerequisites (PyInstaller binary built, Flatpak bundle installed, …) are checked before launch with a hint pointing at `build_all.sh`. On Windows only `source`, `wheel`, `pyinstaller` are supported.
+
+The `run.sh` / `run.bat` wrappers create a dedicated **host** Python venv at `${TMPDIR:-/tmp}/testium-validation-venv` (Linux) or `%TEMP%\testium-validation-venv` (Windows), with `--system-site-packages` + `pip install junit-xml`, and run the suite with `-d python_bin=…` so every test-execution subprocess (eval_proc, py_func, cycle, post_exec) runs inside that venv. testium itself keeps running in its own environment for the chosen mode. The venv is shared across modes because every test-execution subprocess ends up on the host either directly (source/wheel/pyinstaller/appimage) or via `flatpak-spawn --host` (flatpak). `clean` as the first argument recreates the venv. `wheel` mode also creates a separate `testium-wheel-venv-<v>` to hold the installed package.
 
 The `venv` item (`test/validation/items/venv/`) asserts that the override actually took effect: `python_bin` is set, `sys.executable` matches it, `sys.prefix == dirname(dirname(python_bin))`, and `sys.prefix != sys.base_prefix` (the last marker catches the case where `python_bin` happens to be a system interpreter, which path-equality alone would miss because the venv's `bin/python3` is a symlink to the host). Both `eval_proc` (inline `<| … |>`) and `py_func` paths are exercised.
 

README.md

@@ -27,6 +27,27 @@ Pre-built artifacts are published at
   runnable directly, no Python installation required on the host. Lua
   support still needs a system `lua` interpreter and the `lua-socket` /
   `lua-cjson` modules.
+* **AppImage** (`Testium-<version>-x86_64.AppImage`) — single-file
+  Linux binary, runnable directly:
+
+  ```sh
+  chmod +x Testium-*-x86_64.AppImage
+  ./Testium-*-x86_64.AppImage
+  ```
+
+  Requires `libfuse2` on the host (FUSE 2 — distinct from `fuse3`, which
+  most distros now ship by default):
+
+  | Distro | Package |
+  |--------|---------|
+  | Arch / CachyOS / Manjaro | `fuse2` |
+  | Debian trixie / Ubuntu 24.04+ | `libfuse2t64` |
+  | Debian bookworm / Ubuntu 22.04 | `libfuse2` |
+  | Fedora | `fuse-libs` |
+
+  If you can't install libfuse2 (e.g. minimal container), prefix the
+  invocation with `APPIMAGE_EXTRACT_AND_RUN=1` — the AppImage will
+  self-extract to `/tmp` on each run instead of FUSE-mounting.
 * **Flatpak bundle** (`testium.flatpak`) — install with:
 
   ```sh
@@ -41,6 +62,9 @@ Pre-built artifacts are published at
   `testium` command is available in the terminal (requires `~/.local/bin` in
   `PATH`, which most modern distributions provide by default).
 
+Every channel ships the language server, so `testium lsp` (see
+[Editor support](#editor-support)) works out of the box from any of them.
+
 ## Quick start
 
 From a checkout of the repository:
@@ -82,6 +106,29 @@ python3 src/testium               # GUI
 python3 src/testium -b mytest.tum # batch
 ```
 
+## Editor support
+
+testium ships a Language Server Protocol (LSP) server that gives `.tum` files
+completion of item types, hover documentation, and an outline view in any
+LSP-capable editor:
+
+```sh
+testium lsp        # speaks LSP over stdio; an editor's LSP client drives it
+testium schema     # dumps the item/parameter schema as JSON (what the LSP serves)
+```
+
+The server is bundled in every pre-built release (wheel, binary, Flatpak,
+AppImage). For a source / wheel install, pull the language-server extra:
+
+```sh
+pip install 'testium[lsp]'                 # from PyPI / a wheel
+pip install -e /path/to/testium/src[lsp]   # from a source checkout
+```
+
+A VSCode / VSCodium client extension (`testium_assist`) wraps `testium lsp`;
+the schema is built from testium itself, so new item types and parameters
+appear in the editor on the next testium upgrade with no client change.
+
 ## Troubleshooting
 
 ### `wl_proxy_marshal_flags` symbol error

build_all.sh

@@ -1,5 +1,5 @@
 #!/bin/bash
-# Build every distribution channel of testium, in order:
+# Build every distribution channel of testium:
 #   1. Manual PDF         -> dist/testium-manual-<v>.pdf
 #   2. Wheel              -> dist/testium-<v>-py3-none-any.whl   (PEP 427 name)
 #   3. PyInstaller binary -> dist/testium-<v>
@@ -11,6 +11,21 @@
 # By default, a step is skipped if its artifact already exists in dist/.
 # Pass --clean to remove existing dist/ artifacts and rebuild everything.
 #
+# Parallelism: the wheel is built first (the AppImage installs it), then the
+# manual, PyInstaller, Flatpak and AppImage builds run concurrently. The shared
+# venv at test/tmp/.venv is only WRITTEN during the serial prep phase (the
+# `pip install` of build/sphinx/pyinstaller); the parallel builds only read it,
+# so there is no concurrent-pip race. Pass --serial to build one step at a time
+# (useful when debugging or on a resource-constrained machine). Per-step output
+# of the parallel phase is captured under dist/.build-logs/<step>.log and the
+# log of any failing step is printed at the end.
+#
+# Pass --ram to redirect the per-channel build scratch (PyInstaller workpath,
+# AppImage AppDir) and TMPDIR/PIP_CACHE_DIR to /dev/shm, and skip UPX. Big
+# speedup on slow/flash storage. Flatpak is excluded (its rofiles-fuse can't
+# mount on /dev/shm), so it still builds on disk. On a RAM-limited machine
+# combine with --serial (e.g. ./build_all.sh --ram --serial).
+#
 # All artifacts are collected (copied) under <repo>/dist/. Original outputs in
 # src/dist/, package/*/dist/, doc/manual/ are left in place. Wheel and AppImage
 # keep their original names (which already contain the version); manual,
@@ -26,9 +41,13 @@
 set -e
 
 CLEAN=0
+SERIAL=0
+RAM=0
 for arg in "$@"; do
     case "$arg" in
         --clean|-c) CLEAN=1 ;;
+        --serial)   SERIAL=1 ;;
+        --ram)      RAM=1 ;;
         *) echo "Unknown option: $arg" >&2; exit 1 ;;
     esac
 done
@@ -63,6 +82,31 @@ export REQ_PATH="$SCRIPT_DIR/src/requirements.txt"
 bash "$SCRIPT_DIR/scripts/build_env.sh"
 source "$SCRIPT_DIR/scripts/set_env.sh"
 
+# ---------- RAM mode: put build scratch on tmpfs (--ram) ----------------------
+# On slow storage (USB stick, SD card) the per-channel build dirs and temp
+# churn dominate. --ram redirects the PyInstaller workpath, the AppImage AppDir
+# and TMPDIR/PIP_CACHE_DIR to /dev/shm, and skips UPX. Flatpak is intentionally
+# NOT moved: flatpak-builder mounts its state dir with rofiles-fuse, and FUSE
+# can't mount on /dev/shm (fusermount: Permission denied) — so it builds on
+# disk. The tmpfs scratch is freed on exit.
+if [ "$RAM" -eq 1 ]; then
+    RAMROOT="/dev/shm/testium-build-${VERSION}"
+    echo "-- RAM mode: build scratch under $RAMROOT (tmpfs), freed on exit"
+    echo "   (flatpak builds on disk — rofiles-fuse can't mount on /dev/shm)"
+    rm -rf "$RAMROOT"
+    mkdir -p "$RAMROOT"/{tmp,pip,pyi-work,appdir}
+    export TMPDIR="$RAMROOT/tmp"
+    export PIP_CACHE_DIR="$RAMROOT/pip"
+    export PYI_WORKPATH="$RAMROOT/pyi-work"            # pyinstaller --workpath
+    export APPIMAGE_APPDIR_TMPFS="$RAMROOT/appdir"     # AppDir bind-mount
+    export TESTIUM_NO_UPX=1                            # skip slow UPX in the spec
+    trap 'rm -rf "$RAMROOT"' EXIT
+    if [ "$SERIAL" -ne 1 ]; then
+        echo "   note: with --ram, prefer adding --serial so each step gets the"
+        echo "         full tmpfs and you don't risk OOM (flatpak+appimage are ~1 GB each)."
+    fi
+fi
+
 step() {
     echo
     echo "================================================================"
@@ -70,50 +114,100 @@ step() {
     echo "================================================================"
 }
 
-skip() { echo "  (already built — skipping)"; }
+# Kill a process and its whole descendant tree (children first) — used by the
+# interrupt handler so SIGINT also stops grandchildren the parallel jobs spawned
+# (podman container, flatpak-builder, pyinstaller …), not just the subshells.
+_kill_tree() {
+    local pid=$1 c
+    for c in $(pgrep -P "$pid" 2>/dev/null); do
+        _kill_tree "$c"
+    done
+    kill -TERM "$pid" 2>/dev/null || true
+}
+
+# Set as INT/TERM handler around the parallel wait. Stops every running build
+# tree, then exits — the EXIT trap (set under --ram) frees the tmpfs scratch.
+_interrupt() {
+    echo >&2
+    echo "-- interrupted: stopping running builds…" >&2
+    local pid
+    for pid in "${!PID2NAME[@]}"; do
+        _kill_tree "$pid"
+    done
+    exit 130
+}
+
+# ---------- artifact paths ----------------------------------------------------
 
-# 1. Manual PDF
 MANUAL="$DIST_DIR/testium-manual-${VERSION}.pdf"
-step "1/5  Manual PDF (version $VERSION)"
-if [ ! -f "$MANUAL" ]; then
-    python -m pip install --quiet --upgrade sphinx linuxdoc
-    bash "$SCRIPT_DIR/doc/manual/sphinx/build_doc.sh"
-    cp -f "$SCRIPT_DIR/doc/manual/testium_manual.pdf" "$MANUAL"
-else
-    skip
-fi
+PYI_BIN="$DIST_DIR/testium-${VERSION}"
+FLATPAK_BUNDLE="$DIST_DIR/testium-${VERSION}.flatpak"
+wheel_in_dist() { ls -1t "$DIST_DIR"/testium-${VERSION}-*.whl 2>/dev/null | head -1; }
+appimage_in_dist() { ls -1t "$DIST_DIR"/Testium-${VERSION}-*.AppImage 2>/dev/null | head -1; }
 
-# 2. Wheel — PEP 427 name kept (already contains version)
-step "2/5  Wheel (version $VERSION)"
-WHEEL=$(ls -1t "$DIST_DIR"/testium-${VERSION}-*.whl 2>/dev/null | head -1)
-if [ -z "$WHEEL" ]; then
-    python -m pip install --quiet --upgrade build
+# ---------- per-step build functions (assume tools are installed) -------------
+
+build_wheel() {
+    if [ -n "$(wheel_in_dist)" ]; then echo "wheel: already built — skipping"; return 0; fi
+    echo "wheel: building"
     (
         cd "$SCRIPT_DIR/src"
         rm -rf dist build *.egg-info
         python -m build --wheel
     )
-    WHEEL_SRC=$(ls -1t "$SCRIPT_DIR/src/dist"/*.whl | head -1)
-    WHEEL="$DIST_DIR/$(basename "$WHEEL_SRC")"
-    cp -f "$WHEEL_SRC" "$WHEEL"
-else
-    skip
-fi
+    local src; src=$(ls -1t "$SCRIPT_DIR/src/dist"/*.whl | head -1)
+    cp -f "$src" "$DIST_DIR/$(basename "$src")"
+    echo "wheel: done"
+}
 
-# 3. PyInstaller binary
-PYI_BIN="$DIST_DIR/testium-${VERSION}"
-step "3/5  PyInstaller binary (version $VERSION)"
-if [ ! -f "$PYI_BIN" ]; then
-    python -m pip install --quiet --upgrade pyinstaller
+build_manual() {
+    if [ -f "$MANUAL" ]; then echo "manual: already built — skipping"; return 0; fi
+    echo "manual: building"
+    bash "$SCRIPT_DIR/doc/manual/sphinx/build_doc.sh"
+    cp -f "$SCRIPT_DIR/doc/manual/testium_manual.pdf" "$MANUAL"
+    echo "manual: done"
+}
+
+build_pyinstaller() {
+    if [ -f "$PYI_BIN" ]; then echo "pyinstaller: already built — skipping"; return 0; fi
+    echo "pyinstaller: building"
     bash "$SCRIPT_DIR/package/pyinstaller/build.sh"
     cp -f "$SCRIPT_DIR/package/pyinstaller/dist/testium" "$PYI_BIN"
-else
-    skip
-fi
+    echo "pyinstaller: done"
+}
+
+build_flatpak() {
+    if [ -f "$FLATPAK_BUNDLE" ]; then echo "flatpak: already built — skipping"; return 0; fi
+    echo "flatpak: building"
+    (
+        cd "$SCRIPT_DIR/package/flatpak"
+        bash build.sh
+    )
+    cp -f "$SCRIPT_DIR/package/flatpak/testium.flatpak" "$FLATPAK_BUNDLE"
+    echo "flatpak: done"
+}
+
+build_appimage() {
+    if [ -n "$(appimage_in_dist)" ]; then echo "appimage: already built — skipping"; return 0; fi
+    echo "appimage: building"
+    (
+        cd "$SCRIPT_DIR/package/appimage"
+        bash build.sh
+    )
+    local src; src=$(ls -1t "$SCRIPT_DIR/package/appimage"/*.AppImage 2>/dev/null | head -1)
+    cp -f "$src" "$DIST_DIR/$(basename "$src")"
+    chmod +x "$DIST_DIR/$(basename "$src")"
+    echo "appimage: done"
+}
+
+# ---------- serial prep: tool installs (shared venv) + flatpak runtimes -------
+
+step "Prep: build tools + runtimes (serial — shared venv)"
+
+[ -f "$MANUAL" ]   || python -m pip install --quiet --upgrade sphinx linuxdoc
+[ -n "$(wheel_in_dist)" ] || python -m pip install --quiet --upgrade build
+[ -f "$PYI_BIN" ]  || python -m pip install --quiet --upgrade pyinstaller
 
-# 4. Flatpak bundle
-FLATPAK_BUNDLE="$DIST_DIR/testium-${VERSION}.flatpak"
-step "4/5  Flatpak bundle (version $VERSION)"
 if [ ! -f "$FLATPAK_BUNDLE" ]; then
     FLATPAK_DEPS=(
         "org.kde.Platform//6.10"
@@ -130,35 +224,76 @@ if [ ! -f "$FLATPAK_BUNDLE" ]; then
             flatpak install --user --noninteractive flathub "$dep"
         fi
     done
-    (
-        cd "$SCRIPT_DIR/package/flatpak"
-        bash build.sh
-    )
-    cp -f "$SCRIPT_DIR/package/flatpak/testium.flatpak" "$FLATPAK_BUNDLE"
-else
-    skip
 fi
 
-# 5. AppImage
-step "5/5  AppImage (version $VERSION)"
-APPIMAGE=$(ls -1t "$DIST_DIR"/Testium-${VERSION}-*.AppImage 2>/dev/null | head -1)
-if [ -z "$APPIMAGE" ]; then
-    (
-        cd "$SCRIPT_DIR/package/appimage"
-        bash build.sh
-    )
-    APPIMAGE_SRC=$(ls -1t "$SCRIPT_DIR/package/appimage"/*.AppImage 2>/dev/null | head -1)
-    APPIMAGE="$DIST_DIR/$(basename "$APPIMAGE_SRC")"
-    cp -f "$APPIMAGE_SRC" "$APPIMAGE"
-    chmod +x "$APPIMAGE"
+# ---------- serial: wheel (the AppImage installs it) --------------------------
+
+step "1/5  Wheel (version $VERSION)"
+build_wheel
+
+# ---------- build the rest --------------------------------------------------
+
+REST=(manual pyinstaller flatpak appimage)
+
+if [ "$SERIAL" -eq 1 ]; then
+    n=2
+    for name in "${REST[@]}"; do
+        step "$n/5  $name (version $VERSION)"
+        "build_$name"
+        n=$((n + 1))
+    done
 else
-    skip
+    step "2-5/5  manual + pyinstaller + flatpak + appimage (parallel)"
+    LOGDIR="$DIST_DIR/.build-logs"
+    mkdir -p "$LOGDIR"
+    declare -A PID2NAME
+    for name in "${REST[@]}"; do
+        log="$LOGDIR/$name.log"
+        echo "  -> launching $name (log: $log)"
+        ( "build_$name" ) >"$log" 2>&1 &
+        PID2NAME[$!]="$name"
+    done
+
+    # From here until all jobs are reaped, Ctrl+C stops every build tree.
+    trap _interrupt INT TERM
+
+    # Reap in completion order (wait -n) so each result prints the moment that
+    # build finishes, not when its slot comes up in the array.
+    FAILED=()
+    remaining=${#PID2NAME[@]}
+    while [ "$remaining" -gt 0 ]; do
+        if wait -n -p donepid; then rc=0; else rc=$?; fi
+        name="${PID2NAME[$donepid]:-}"
+        [ -z "$name" ] && continue
+        if [ "$rc" -eq 0 ]; then
+            echo "  -> $name: OK"
+        else
+            echo "  -> $name: FAILED (rc=$rc)"
+            FAILED+=("$name")
+        fi
+        remaining=$((remaining - 1))
+    done
+
+    trap - INT TERM
+
+    if [ "${#FAILED[@]}" -gt 0 ]; then
+        for name in "${FAILED[@]}"; do
+            echo
+            echo "===================== $name log ====================="
+            cat "$LOGDIR/$name.log"
+        done
+        echo >&2
+        echo "BUILD FAILED: ${FAILED[*]} (logs under $LOGDIR)" >&2
+        exit 1
+    fi
 fi
 
+# ---------- summary -----------------------------------------------------------
+
 step "All packages built"
 printf "  manual       : %s\n" "$MANUAL"
-printf "  wheel        : %s\n" "$WHEEL"
+printf "  wheel        : %s\n" "$(wheel_in_dist)"
 printf "  pyinstaller  : %s\n" "$PYI_BIN"
 printf "  flatpak      : %s\n" "$FLATPAK_BUNDLE"
-printf "  appimage     : %s\n" "$APPIMAGE"
+printf "  appimage     : %s\n" "$(appimage_in_dist)"
 printf "  release_note : %s\n" "$RELEASE_NOTE"

doc/manual/sphinx/source/modes.rst

@@ -23,3 +23,47 @@ graphical interface.
     :caption: call a test in batch mode
 
     testium -b test/my_test/main.tum
+
+.. _sec_language_server:
+
+Language server (editor support)
+--------------------------------
+
+*testium* ships a `Language Server Protocol
+<https://microsoft.github.io/language-server-protocol/>`_ server so that
+``.tum`` files get editor assistance — completion of test item types, hover
+documentation of their parameters, and an outline view — in any LSP-capable
+editor.
+
+The server speaks LSP over standard input/output and is started with:
+
+.. code-block:: text
+    :caption: start the language server
+
+    testium lsp
+
+It is not meant to be launched directly by the user: an editor's LSP client
+spawns it and drives the exchange. A VSCode / VSCodium client extension,
+*testium_assist*, is provided for that purpose; any other LSP client (Neovim,
+Emacs ``lsp-mode``, …) can be pointed at ``testium lsp`` as well.
+
+The information the server exposes is the test item schema, which can also be
+dumped as JSON for inspection or tooling:
+
+.. code-block:: text
+    :caption: dump the item / parameter schema
+
+    testium schema
+
+Because the schema is built from *testium* itself, every new item type or
+parameter becomes available in the editor on the next *testium* upgrade, with
+no change to the client.
+
+The language server is included in the pre-built binary, Flatpak and AppImage
+releases. For a source or wheel installation, pull the optional ``lsp``
+dependencies:
+
+.. code-block:: text
+    :caption: enable the language server for a wheel / source install
+
+    pip install 'testium[lsp]'

doc/manual/sphinx/source/tum_syntax.rst

@@ -232,6 +232,15 @@ list of the main test item (and eventually of the loop test item).
 TUM file ``main`` item is itself a variant of test items with a name and an
 step list attributes.
 
+.. note::
+
+   Each test item declares the parameters it accepts. When a ``.tum`` file
+   uses a key the item does not know, *testium* emits a warning listing the
+   accepted parameter names (catching typos such as ``param_filee`` for
+   ``param_file``); a missing **required** parameter aborts loading with an
+   error pointing at the source ``.tum`` file. Valid existing tests are
+   unaffected.
+
 
 
 .. toctree::

package/appimage/AppImageBuilder.yml

@@ -77,7 +77,10 @@ AppDir:
         python3.11 -m pip install --break-system-packages --upgrade --isolated --no-input --ignore-installed --prefix=$TARGET_APPDIR/usr -r ../../src/requirements.txt
 
         export PIP_CONFIG_FILE=$HOME/.pip/pip.conf
-        python3.11 -m pip install --break-system-packages --upgrade --isolated --no-input --ignore-installed --prefix=$TARGET_APPDIR/usr ../../src/dist/testium-{{APP_VERSION}}-py3-none-any.whl
+        # Install the wheel with the [lsp] extra so `testium lsp` (pygls) works
+        # from the AppImage. The extra pulls pygls/lsprotocol/cattrs/attrs from
+        # the index (network is available at build time, see get-pip above).
+        python3.11 -m pip install --break-system-packages --upgrade --isolated --no-input --ignore-installed --prefix=$TARGET_APPDIR/usr "../../src/dist/testium-{{APP_VERSION}}-py3-none-any.whl[lsp]"
 
 
 AppImage:

package/appimage/build.sh

@@ -17,11 +17,20 @@ else
 fi
 echo "Using $RUNTIME — building testium $APP_VERSION AppImage..."
 
+# APPIMAGE_APPDIR_TMPFS (set by build_all --ram) bind-mounts a host tmpfs dir at
+# the AppDir build path, keeping the ~1 GB AppDir churn off slow storage.
+APPDIR_MOUNT=""
+if [ -n "$APPIMAGE_APPDIR_TMPFS" ]; then
+    mkdir -p "$APPIMAGE_APPDIR_TMPFS"
+    APPDIR_MOUNT="-v $APPIMAGE_APPDIR_TMPFS:/work/package/appimage/AppDir"
+fi
+
 # APPIMAGE_EXTRACT_AND_RUN=1 lets appimagetool run without FUSE in the container.
 $RUNTIME run --rm \
     --privileged \
     -e APPIMAGE_EXTRACT_AND_RUN=1 \
     -v "$REPO_ROOT:/work" \
+    $APPDIR_MOUNT \
     -w /work/package/appimage \
     debian:bookworm bash -c "
         set -e

package/flatpak/build.sh

@@ -7,11 +7,19 @@
 
 set -e
 
-# Build + install local
-flatpak-builder --user --verbose --force-clean --install --repo=repo build org.testium.Testium.yaml
+# Build + install local. FLATPAK_BUILDDIR / FLATPAK_STATEDIR / FLATPAK_REPODIR
+# (set by build_all --ram) redirect the build dir, the state dir
+# (.flatpak-builder) and the ostree repo to tmpfs. flatpak-builder hardlinks
+# between the state dir and the build dir, so they MUST be on the same
+# filesystem — hence the state dir moves to tmpfs too (its download cache then
+# doesn't persist across --ram runs).
+BUILDDIR="${FLATPAK_BUILDDIR:-build}"
+STATEDIR="${FLATPAK_STATEDIR:-.flatpak-builder}"
+REPODIR="${FLATPAK_REPODIR:-repo}"
+flatpak-builder --user --verbose --force-clean --install --state-dir="$STATEDIR" --repo="$REPODIR" "$BUILDDIR" org.testium.Testium.yaml
 
 # Génère le bundle distribuable
-flatpak build-bundle repo testium.flatpak org.testium.Testium
+flatpak build-bundle "$REPODIR" testium.flatpak org.testium.Testium
 echo "Bundle généré : $(pwd)/testium.flatpak"
 
 # Crée ~/.local/bin/testium pour pouvoir taper "testium" en console

package/flatpak/org.testium.Testium.yaml

@@ -28,6 +28,23 @@ build-options:
 
 modules:
   - python3-requirements.json
+
+  # Language-server deps for `testium lsp` (pygls + lsprotocol + cattrs + attrs
+  # + typing_extensions). Installed from PyPI at build time — the build already
+  # runs with --share=network (see build-options). The core runtime deps stay
+  # offline-pinned in python3-requirements.json; these are pure-python wheels,
+  # hence --only-binary=:all: (no compilation, deterministic).
+  - name: python3-lsp
+    buildsystem: simple
+    build-options:
+      build-args:
+        - --share=network
+    build-commands:
+      # Whole command single-quoted: the ':all: ' colon-space would otherwise
+      # make YAML parse this list item as a mapping, silently dropping the
+      # command (flatpak-builder then runs an empty module — installs nothing).
+      - 'pip3 install --prefix=${FLATPAK_DEST} --only-binary=:all: "pygls>=1.3"'
+
   # 1. Dépendances Python tierces (HORS PySide6)
   # Utilisez flatpak-pip-generator pour vos autres libs (ex: pyserial, requests, etc.)
   # - name: python3-requirements

package/pyinstaller/build.sh

@@ -2,11 +2,15 @@
 
 SCRIPT_DIR=$(realpath $( dirname "$0"))
 
-rm -r "${SCRIPT_DIR}/build" "${SCRIPT_DIR}/dist"
+rm -rf "${SCRIPT_DIR}/build" "${SCRIPT_DIR}/dist"
 
 pwd=$(pwd)
 cd ${SCRIPT_DIR}
-pyinstaller testium.spec
+# PYI_WORKPATH (set by build_all --ram) puts the big intermediate build tree on
+# tmpfs; dist/ stays local so build_all can collect the binary.
+WORKARG=""
+[ -n "$PYI_WORKPATH" ] && WORKARG="--workpath $PYI_WORKPATH"
+pyinstaller $WORKARG testium.spec
 RESULT=$?
 if [ -n "$1" ] && [ "$1" = "install" ]; then
     if [ $RESULT -eq 0 ]; then

package/pyinstaller/testium.spec

@@ -1,5 +1,21 @@
 # -*- mode: python ; coding: utf-8 -*-
 import os
+from PyInstaller.utils.hooks import collect_submodules
+
+# Language-server dependencies for `testium lsp`. pygls/lsprotocol register
+# converters and features dynamically, so we collect their submodules wholesale
+# and force-import their pure-python deps (cattrs/attrs/typing_extensions).
+# The testium lsp modules are imported lazily by the CLI dispatch
+# (`from lsp.server import serve`), which PyInstaller's static analysis misses —
+# hence the explicit names. No source files need bundling: the schema export is
+# now fully declarative (PARAMS + ACTIONS class attributes), so it no longer
+# reads .py source via inspect.getsource (which fails in a frozen build).
+_LSP_HIDDEN = (
+    collect_submodules("pygls")
+    + collect_submodules("lsprotocol")
+    + ["cattrs", "attr", "attrs", "typing_extensions",
+       "lsp", "lsp.server", "lsp.schema"]
+)
 
 # junit_xml is imported by post_exec scripts running under the *host* Python,
 # not the frozen interpreter — so bundling it via hiddenimports alone is not
@@ -54,7 +70,7 @@ a = Analysis(
                    "colorama",
                    "matplotlib",
                    "junit_xml",
-                   "lxml"],
+                   "lxml"] + _LSP_HIDDEN,
     hookspath=[],
     hooksconfig={},
     runtime_hooks=[],
@@ -73,7 +89,9 @@ exe = EXE(
     debug=False,
     bootloader_ignore_signals=False,
     strip=False,
-    upx=True,
+    # UPX is CPU+IO heavy for a marginal size gain — build_all --ram sets
+    # TESTIUM_NO_UPX=1 to skip it (much faster on slow/flash storage).
+    upx=not os.environ.get("TESTIUM_NO_UPX"),
     upx_exclude=[],
     runtime_tmpdir=None,
     console=True,

release_note.txt

@@ -1,3 +1,16 @@
+version 0.2.1
+==============
+- Faster test loading, especially for large tests built from jinja
+  templates and ``!include``: compiled jinja templates are cached and
+  reused (a file included many times is compiled once), rendering happens
+  in memory instead of through a temporary file, and YAML is parsed with
+  the libyaml C loader when available. Typical load time is 3-6x lower on
+  include / template-heavy tests; behaviour is unchanged.
+- Fix: a nested list holding more than one step under ``steps`` no longer
+  duplicates its entries while the step tree is built.
+- New load-time benchmark under ``test/benchmark/`` (synthetic-tree
+  generator + in-process timing harness) to measure the load pipeline.
+
 version 0.2
 ==============
 - Test items: each item type now declares its accepted parameters
@@ -5,6 +18,17 @@ version 0.2
   WARN listing the accepted names instead of being silently ignored;
   missing required parameters error out at load time with the source
   ``.tum`` file as context. No change to valid existing tests.
+- Editor support: testium now ships a language server. ``testium lsp``
+  gives ``.tum`` files item-type completion, hover documentation and an
+  outline view in any LSP-capable editor (a VSCode / VSCodium client is
+  provided separately). ``testium schema`` dumps the item/parameter
+  schema as JSON. The server works from every channel — bundled in the
+  binary / Flatpak / AppImage, and pulled by ``pip install testium[lsp]``
+  for wheel installs.
+- build_all.sh: the four heavy channels now build in parallel (results
+  reported as each finishes; Ctrl+C stops them cleanly). New ``--ram``
+  option builds on a tmpfs (``/dev/shm``) and skips UPX for much faster
+  packaging on USB-stick / SD-card storage.
 
 version 0.1.3
 ==============

scripts/build_env.sh

@@ -33,6 +33,11 @@ if [ ! -d "$PY_VENV_DIR" ]; then
     python3 -m venv "$PY_VENV_DIR"
     source "$PY_VENV_DIR/bin/activate"
     pip install --extra-index-url https://pypi.python.org/pypi -r $REQ_PATH
+    # Language-server deps (the pyproject [lsp] extra). Installed here so the
+    # source run AND the PyInstaller build — both of which use this venv — can
+    # start / collect the `testium lsp` server. pip-installed wheel users get
+    # them via `pip install testium[lsp]` instead.
+    pip install --extra-index-url https://pypi.python.org/pypi "pygls>=1.3"
     # Validation suite plugin used to verify the report-exporter
     # entry-points discovery end-to-end.
     FAKE_EXPORTER_DIR="$(dirname "$REQ_PATH")/../test/validation/fake_exporter"

src/pyproject.toml

@@ -30,6 +30,12 @@ dependencies = [
 ]
 dynamic = ["version"]
 
+[project.optional-dependencies]
+# `pip install testium[lsp]` adds the language-server dependencies. The
+# stdio-only LSP server (`testium lsp`) reuses the schema export from the
+# core install; pygls is the only marginal cost.
+lsp = ["pygls>=1.3"]
+
 [project.scripts]
 testium = "testium:main"
 

src/testium/__init__.py

@@ -11,6 +11,30 @@ sys.path.append(os.path.abspath(ourpath.parent))
 import interpreter.utils.constants as cst
 
 def main():
+    # Subcommand dispatch (must run *before* argparse so neither 'schema' nor
+    # 'lsp' has to share the GUI/batch flag surface). The subcommands also
+    # skip the multiprocessing 'spawn' setup which is only meaningful for the
+    # main runtime — schema is a pure stdout dump and lsp speaks JSON-RPC
+    # over stdio without ever forking a test process.
+    if len(sys.argv) >= 2 and sys.argv[1] in ("schema", "lsp"):
+        sub = sys.argv[1]
+        if sub == "schema":
+            from lsp.schema import dump_all_schemas_json
+            print(dump_all_schemas_json())
+            return
+        # lsp
+        try:
+            from lsp.server import serve
+        except ImportError as e:
+            print(
+                f"testium lsp: language server dependencies missing ({e.name}). "
+                "Install with: pip install 'testium[lsp]'",
+                file=sys.stderr,
+            )
+            sys.exit(2)
+        serve()
+        return
+
     # This line sets the method for the "Process" function. It is required for Linux
     # support of the test dialogs.
     multiprocessing.set_start_method('spawn')

src/testium/interpreter/test_items/item_actions/__init__.py

@@ -5,6 +5,13 @@ from interpreter.test_items.item_actions.action import TestItemAction
 
 
 class TestItemActions(TestItem):
+    # Declarative action registry: subclasses set ``ACTIONS = {yaml_key: class}``
+    # as a class attribute (mirroring ``PARAMS``). It is read here to populate
+    # the runtime registry, and read identically by the schema export — no
+    # instantiation or source inspection required. ``register_actions()`` stays
+    # available as an imperative escape hatch for dynamic/conditional cases.
+    ACTIONS = {}
+
     def __init__(
         self, item_type, dict_actions, parent=None, status_queue=None, filename=""
     ):
@@ -12,7 +19,7 @@ class TestItemActions(TestItem):
         super().__init__(dict_actions, parent, status_queue, filename=filename)
         self._type = item_type
         self.is_container = False
-        self.action_classes = {}
+        self.action_classes = dict(type(self).ACTIONS)
         self.actions_token = None
         self.actions = []
         try:
@@ -24,6 +31,9 @@ class TestItemActions(TestItem):
             )
 
     def register_actions(self, **args: TestItemAction):
+        # Imperative escape hatch. The declarative ``ACTIONS`` class attribute
+        # covers every current subclass; use this only to add actions that
+        # can't be known at class-definition time (e.g. platform-conditional).
         for action_name, action_class in args.items():
             self.action_classes.update({action_name: action_class})
 

src/testium/interpreter/test_items/test_item_console.py

@@ -388,18 +388,19 @@ class TestItemConsole(TestItemActions):
                   "as long as their names differ."),
     )
 
+    ACTIONS = {
+        "open": TestItemConsoleOpen,
+        "close": TestItemConsoleClose,
+        "write": TestItemConsoleWrite,
+        "writeln": TestItemConsoleWriteLn,
+        "read_until": TestItemConsoleReadUntil,
+    }
+
     def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
         super().__init__(
             cst.TYPE_CONSOLE, dict_item, parent, status_queue, filename=filename
         )
 
-        self.register_actions(
-            open=TestItemConsoleOpen,
-            close=TestItemConsoleClose,
-            write=TestItemConsoleWrite,
-            writeln=TestItemConsoleWriteLn,
-            read_until=TestItemConsoleReadUntil,
-        )
         self.actions_token = {}
 
         global console

src/testium/interpreter/test_items/test_item_json_rpc/__init__.py

@@ -210,6 +210,13 @@ class TestItemJSON_RPC(TestItemActions):
               doc="If true, don't echo wire traffic to the log."),
     )
 
+    ACTIONS = {
+        "open": TestItemJSRPCActionOpen,
+        "close": TestItemJSRPCActionClose,
+        "query": TestItemJSRPCActionQuery,
+        "receive": TestItemJSRPCActionReceive,
+    }
+
     def __init__(
         self, dict_item: dict, parent: TestItem = None, status_queue=None, filename=""
     ):
@@ -217,13 +224,6 @@ class TestItemJSON_RPC(TestItemActions):
             cst.TYPE_JSON_RPC, dict_item, parent, status_queue, filename=filename
         )
 
-        self.register_actions(
-            open=TestItemJSRPCActionOpen,
-            close=TestItemJSRPCActionClose,
-            query=TestItemJSRPCActionQuery,
-            receive=TestItemJSRPCActionReceive,
-        )
-
         # Console specific params
         self._console = self._prms.getParam("console", required=False)
         # UDP specific params

src/testium/interpreter/test_items/test_item_runtime_plot.py

@@ -263,18 +263,18 @@ class TestItemPlot(TestItemActions):
                   "action and by $(plv_<plot_name>) for last-values output."),
     )
 
+    ACTIONS = {
+        "open": TestItemPlotActionOpen,
+        "close": TestItemPlotActionClose,
+        "periodic": TestItemPlotActionPeriodic,
+        "add": TestItemPlotActionAdd,
+        "last_value": TestItemPlotActionLastValues,
+        "export": TestItemPlotActionExport,
+    }
+
     def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
         super().__init__(
             cst.TYPE_GRAPH, dict_item, parent, status_queue, filename=filename
         )
 
-        self.register_actions(
-            open=TestItemPlotActionOpen,
-            close=TestItemPlotActionClose,
-            periodic=TestItemPlotActionPeriodic,
-            add=TestItemPlotActionAdd,
-            last_value=TestItemPlotActionLastValues,
-            export=TestItemPlotActionExport,
-        )
-
         self.actions_token = self._prms.getParam("plot_name", required=True)

src/testium/interpreter/test_set.py

@@ -29,6 +29,51 @@ def _build_item_path(item) -> str:
     return " > ".join(reversed(parts))
 
 
+def _flatten_actions(actions, out, parent_seq_name):
+    """Expand nested lists and included ``sequence`` entries into ``out`` as a
+    flat list of single test-item dicts, propagating each sequence's source
+    filename onto its items.
+
+    Replaces the previous approach, which spliced each entry into the step
+    list and rebuilt the whole list every time (O(n^2) over the step list, and
+    a rebuild that duplicated entries when a nested list held more than one
+    element). This single forward pass is linear.
+    """
+    for idx, action in enumerate(actions):
+        # a bare list raises its elements to the same level
+        if isinstance(action, (list, tuple)):
+            _flatten_actions(action, out, parent_seq_name)
+            continue
+        # a NoneType (e.g. pointing at an unused alias) contributes nothing
+        if action is None:
+            continue
+        # a 'sequence' (an included file) is spliced in, with its filename
+        # propagated onto each of its items
+        if isinstance(action, dict) and "sequence" in action:
+            sequence = action["sequence"]["data"]
+            f = action["sequence"]["filename"]
+            if isinstance(sequence, dict):
+                sequence = [{k: v} for k, v in sequence.items()]
+            # Case of an empty sequence
+            elif sequence is None:
+                tm.print_info(
+                    f"An empty sequence is loaded in '{parent_seq_name}'."
+                )
+                sequence = []
+            elif not isinstance(sequence, list):
+                raise ETUMSyntaxError(
+                    f"Syntax error in '{parent_seq_name}' step number {idx+1}. Sequence definition: '{str(action)}'",
+                    f
+                )
+            for s in sequence:
+                if isinstance(s, dict) and s:
+                    s[list(s.keys())[0]]["seq_filename"] = f
+            _flatten_actions(sequence, out, parent_seq_name)
+            continue
+
+        out.append(action)
+
+
 class TestSet:
     def __init__(
         self,
@@ -434,56 +479,16 @@ class TestSet:
                 f"No valid list of actions in sequence {parent_seq_name}",
                 file_name
             )
-        # first we merged to the same level 'sequence dict entries and list within the list
-        counter = 0
         test_dir = tm.gd("test_directory")
-        la = len(parent_seq_actions)
-        while counter < la:
-            action = parent_seq_actions[counter]
-            # if action is a list raise up to the the same level,
-            # ie insert action element into the parent_seq_actions
-            if isinstance(action, (list, tuple)):
-                parent_seq_actions[counter : counter + 1] = action
-                parent_seq_actions = (
-                    parent_seq_actions[:counter]
-                    + action
-                    + parent_seq_actions[counter + 1 :]
-                )
-                la = len(parent_seq_actions)
-                continue
-            # if action is a NoneType skip and continue
-            # (when pointing to an unused alias for instance)
-            if action is None:
-                counter += 1
-                continue
-            # if action is a sequence we insert its entry into the action list
-            if "sequence" in action:
-                sequence = action["sequence"]["data"]
-                f = action["sequence"]["filename"]
-                if isinstance(sequence, dict):
-                    sequence = [{k: v} for k, v in sequence.items()]
-                # Case of an empty sequence
-                elif sequence is None:
-                    tm.print_info(
-                        f"An empty sequence is loaded in '{parent_seq_name}'."
-                    )
-                    sequence = []
-                elif not isinstance(sequence, list):
-                    raise ETUMSyntaxError(
-                        f"Syntax error in '{parent_seq_name}' step number {counter+1}. Sequence definition: '{str(action)}'",
-                        f
-                    )
-                for s in sequence:
-                    s[list(s.keys())[0]]["seq_filename"] = f
-                parent_seq_actions = (
-                    parent_seq_actions[:counter]
-                    + sequence
-                    + parent_seq_actions[counter + 1 :]
-                )
-                la = len(parent_seq_actions)
-                continue
 
-            # Action is now for sure a list of dict of length 1
+        # Flatten nested lists and included 'sequence' entries to the same level
+        # in one linear pass (was an in-place splice + full list rebuild per
+        # entry: O(n^2) over the step list).
+        flat_actions = []
+        _flatten_actions(parent_seq_actions, flat_actions, parent_seq_name)
+
+        for action in flat_actions:
+            # Action is now for sure a dict of length 1
             k = list(action.keys())[0]
             if action[k].get("seq_filename", None) is None:
                 action[k]["seq_filename"] = file_name
@@ -546,8 +551,6 @@ class TestSet:
                     action[k]["seq_filename"]
                 )
 
-            counter += 1
-
         return ret
 
     def tree(self):

src/testium/interpreter/utils/include.py

@@ -6,10 +6,10 @@ from runtime.tum_except import ETUMFileError
 from interpreter.utils.template import template_to_test
 from copy import copy
 from interpreter.utils.globdict import global_dict
-from interpreter.utils.yaml_load import yaml_load
+from interpreter.utils.yaml_load import yaml_load, YAML_BASE_LOADER
 
 
-class TUMLoaderNoIncludes(yaml.Loader):
+class TUMLoaderNoIncludes(YAML_BASE_LOADER):
 
     def __init__(self, stream):
 

src/testium/interpreter/utils/template.py

@@ -1,33 +1,74 @@
+import io
 import os
 from sys import exc_info
-from jinja2 import Template
+from jinja2 import Environment
 from jinja2.exceptions import TemplateSyntaxError, TemplateError, UndefinedError
-from tempfile import TemporaryFile
 from interpreter.utils.yaml_load import print_yaml
 from runtime.tum_except import ETUMSyntaxError
 
 
+# One Environment reused for every render (default settings, i.e. identical
+# behaviour to jinja2.Template), plus a compiled-template cache so a file that
+# is included many times — or a test that is reloaded — is compiled only once.
+# Jinja compilation is the expensive step; render (variable substitution) stays
+# per-call. Cache is keyed on path + mtime + size so an edited file recompiles.
+_ENV = Environment()
+_template_cache = {}  # abspath -> (mtime_ns, size, compiled_template)
+
+
+class _RenderedStream(io.StringIO):
+    """A rendered template kept in memory.
+
+    Carries ``root`` (and ``name``) so the YAML loader resolves ``!include``
+    paths exactly as it did from the on-disk temp file this replaces — without
+    the write + seek + read round-trip (one temp file per included file). That
+    round-trip is pure overhead, and especially costly on slow storage.
+    """
+
+
+def _compiled_template(filename: str):
+    """Return the compiled jinja template for *filename*, reusing the cached
+    one when the file is unchanged (path + mtime + size)."""
+    key = os.path.abspath(filename)
+    try:
+        st = os.stat(filename)
+    except OSError:
+        st = None
+    if st is not None:
+        cached = _template_cache.get(key)
+        if (cached is not None
+                and cached[0] == st.st_mtime_ns
+                and cached[1] == st.st_size):
+            return cached[2]
+    with open(filename, "r") as f:
+        source = f.read()
+    template = _ENV.from_string(source)  # compile (may raise TemplateSyntaxError)
+    if st is not None:
+        _template_cache[key] = (st.st_mtime_ns, st.st_size, template)
+    return template
+
+
 def template_to_test(filename: str, params: list):
     """ Function which processes an eventual jinja2 template to a test file
     """
-    # Temporary file created to receive the processed include
-    # file
-    tmpf = TemporaryFile('w+t')
-    with open(filename, 'r') as f:
-        try:
-            j2_template = Template(f.read())
-        except TemplateError as e:
+    # Compile (cached) — a syntax error in the template surfaces here.
+    try:
+        j2_template = _compiled_template(filename)
+    except TemplateError as e:
+        with open(filename, "r") as f:
             print_yaml(f, filename)
-            type, value, tb = exc_info()
-            msg = "Template error"
-            if hasattr(value, 'lineno'):
-                msg = msg + f" on line {value.lineno}: "
-            else:
-                msg += ": "
-            raise ETUMSyntaxError(msg + str(e), filename)
+        type, value, tb = exc_info()
+        msg = "Template error"
+        if hasattr(value, 'lineno'):
+            msg = msg + f" on line {value.lineno}: "
+        else:
+            msg += ": "
+        raise ETUMSyntaxError(msg + str(e), filename)
+
+    # Render into memory (no temp file).
     try:
         params["include_directory"] = os.path.dirname(os.path.abspath(filename))
-        tmpf.write(j2_template.render(params))
+        rendered = j2_template.render(params)
     except TemplateSyntaxError as e:
         raise ETUMSyntaxError(f"""Template loading of file '{filename}' with following parameters '{str(params)}'
 Syntax error in template: {e.message}""")
@@ -42,8 +83,7 @@ Template rendering error: {e.message}""")
         raise ETUMSyntaxError(f"""Template loading of file '{filename}' with following parameters '{str(params)}'
 Unexpected error: {str(e)}""")
 
-    # return to begining of the temp file
-    tmpf.seek(0, os.SEEK_SET)
-    tmpf.root = os.path.dirname(filename)
-
-    return tmpf
+    stream = _RenderedStream(rendered)
+    stream.root = os.path.dirname(filename)
+    stream.name = filename
+    return stream

src/testium/interpreter/utils/test_init.py

@@ -11,7 +11,7 @@ import api.testium as tm
 import interpreter.utils.globdict as globdict
 import interpreter.utils.settings as prefs
 from interpreter.utils.paths import testium_path
-from interpreter.utils.yaml_load import yaml_load
+from interpreter.utils.yaml_load import yaml_load, YAML_BASE_LOADER
 from interpreter.utils import clear_recursively
 from runtime.tum_except import ETUMSyntaxError
 from interpreter.utils.params import expanse, eval_func_init
@@ -89,7 +89,7 @@ def locate_report_file(rep_file):
 def yamltodict(param_file, silent=True):
     # load of the file
     with open(param_file, "r") as fd:
-        dp = yaml_load(fd, param_file, yaml.Loader)
+        dp = yaml_load(fd, param_file, YAML_BASE_LOADER)
 
     if dp is None:
         tm.print_info(f"The YAML file '{param_file}' is empty.")

src/testium/interpreter/utils/version.py

@@ -1,10 +1,54 @@
+import atexit
 import os
+import stat
 import sys
+import tempfile
 from importlib import import_module
 
 import interpreter.utils.settings as prefs
 import api.testium as tm
 
+
+# When running inside a Flatpak, the host /usr/bin/git is reachable at
+# /run/host/usr/bin/git but linked against host glibc/zlib, which the
+# sandbox can't load (``libz-ng.so.2`` not found). gitpython resolves git
+# eagerly on import and would crash the whole test run. We install a
+# tiny shell wrapper under /tmp that forwards to ``flatpak-spawn --host
+# git``, and point gitpython at it via ``GIT_PYTHON_GIT_EXECUTABLE``.
+_HOST_GIT_WRAPPER = None
+
+
+def _setup_flatpak_git():
+    global _HOST_GIT_WRAPPER
+    if not os.path.isfile("/.flatpak-info"):
+        return
+    if _HOST_GIT_WRAPPER is not None:
+        return
+    fd, path = tempfile.mkstemp(prefix="testium-git-host-", suffix=".sh", dir="/tmp")
+    with os.fdopen(fd, "w") as f:
+        f.write('#!/bin/sh\nexec flatpak-spawn --host git "$@"\n')
+    os.chmod(path, stat.S_IRWXU)
+    _HOST_GIT_WRAPPER = path
+    atexit.register(_cleanup_flatpak_git)
+    os.environ["GIT_PYTHON_GIT_EXECUTABLE"] = path
+    # Silence gitpython's warning if its refresh probe ever still fails;
+    # the wrapper itself should make the probe succeed.
+    os.environ.setdefault("GIT_PYTHON_REFRESH", "quiet")
+
+
+def _cleanup_flatpak_git():
+    global _HOST_GIT_WRAPPER
+    if _HOST_GIT_WRAPPER and os.path.isfile(_HOST_GIT_WRAPPER):
+        try:
+            os.unlink(_HOST_GIT_WRAPPER)
+        except OSError:
+            pass
+        _HOST_GIT_WRAPPER = None
+
+
+_setup_flatpak_git()
+
+
 _cached_versions = {}
 
 def repo_rev(path):

src/testium/interpreter/utils/yaml_load.py

@@ -1,3 +1,4 @@
+import yaml
 from yaml.parser import ParserError
 from yaml import load, Loader
 from yaml.scanner import ScannerError
@@ -5,6 +6,12 @@ from api.testium import print_debug
 from runtime.tum_except import ETUMSyntaxError
 import io
 
+# Use the libyaml-backed loader (much faster parsing) when PyYAML was built
+# with it, falling back to the pure-Python loader otherwise. The C loader
+# raises the same ParserError/ScannerError and supports the same custom
+# constructors (!include) and construct_* helpers the TUM loaders rely on.
+YAML_BASE_LOADER = yaml.CLoader if getattr(yaml, "__with_libyaml__", False) else yaml.Loader
+
 
 def print_yaml(file: io.TextIOWrapper, file_name):
     """ Prints YAML file if debug mode is activated.
@@ -21,10 +28,10 @@ def yaml_load(file, real_file_name: str, loader: Loader):
         return load(file, loader)
 
     except ParserError as e:
-        if isinstance(file, io.TextIOWrapper):
+        if isinstance(file, (io.TextIOWrapper, io.StringIO)):
             print_yaml(file, real_file_name)
         raise ETUMSyntaxError(f"yaml file parsing error: " + str(e), real_file_name)
     except ScannerError as e:
-        if isinstance(file, io.TextIOWrapper):
+        if isinstance(file, (io.TextIOWrapper, io.StringIO)):
             print_yaml(file, real_file_name)
         raise ETUMSyntaxError("yaml file scanning error: " + str(e), real_file_name)

src/testium/lsp/__init__.py

@@ -0,0 +1,16 @@
+"""testium language tooling.
+
+Hosts the JSON-Schema-style schema export of every test item type, and a
+``pygls`` language server that consumes the same schema to provide
+completion / hover / diagnostics for ``.tum`` files in any LSP-capable
+editor (VSCode, neovim, Helix, Emacs, …).
+
+Entry points (both surfaced through the ``testium`` CLI):
+
+- ``testium schema`` — dump the schema of every item type as JSON on stdout.
+  Zero runtime dependencies; can be used by editors that already speak the
+  YAML JSON Schema extension to get static completion immediately.
+
+- ``testium lsp`` — start the language server over stdio. Requires the
+  ``pygls`` optional dependency (``pip install testium[lsp]``).
+"""

src/testium/lsp/__main__.py

@@ -0,0 +1,6 @@
+"""Entry point for ``python -m testium.lsp`` (alternative to ``testium lsp``)."""
+
+from lsp.server import serve
+
+if __name__ == "__main__":
+    serve()

src/testium/lsp/schema.py

@@ -0,0 +1,122 @@
+"""Schema export of the test item registry.
+
+Walks every ``TestItemType`` entry (``interpreter/utils/constants.py``),
+combines its declared ``PARAMS`` with the common ones, and returns a
+serialisable structure keyed by ``item_cmd`` — the YAML key the user
+writes (e.g. ``sleep``, ``py_func``, ``dialog_message``).
+
+Items intentionally without ``PARAMS`` (the unstructured-body classes
+like console ``write``/``writeln`` or plot ``add``/``export``) are
+emitted as ``"params_declared": false`` so consumers know to suggest
+nothing for them rather than reporting a closed empty set.
+
+Action items (children of ``parallel``, ``console``, ``json_rpc``,
+``plot``) are registered separately under each parent's ``actions``
+entry — they're not top-level YAML keys, they live nested inside a
+parent's ``steps:``.
+"""
+
+import json
+
+from interpreter.utils.constants import TestItemType
+from interpreter.utils.test_init import _constants_init
+
+
+# Action class -> parent cmd (the action's parent in the YAML). Action classes
+# aren't first-class TestItemType entries (TYPE_*_ACTION is one generic bucket),
+# so we resolve their YAML key from the parent's declarative ``ACTIONS`` map.
+def _collect_action_classes(parent_class):
+    """Return {action_yaml_key: action_class} for a TestItemActions parent.
+
+    Each parent declares its actions as a class-level ``ACTIONS = {key: class}``
+    attribute (see ``item_actions/TestItemActions``). We read it directly — no
+    instantiation, no source inspection — so this works identically whether the
+    package runs from source, a wheel, or a frozen (PyInstaller) build where the
+    ``.py`` source isn't on disk.
+    """
+    return dict(getattr(parent_class, "ACTIONS", None) or {})
+
+
+def _params_to_schema(item_class, common_params):
+    """Return the params-portion of an item's schema entry.
+
+    Common params are flagged so consumers can render them differently
+    (an editor might show "common" parameters in a separate group).
+    """
+    own = getattr(item_class, "PARAMS", None)
+    if own is None:
+        return {"params_declared": False}
+    common_names = set(common_params.names())
+    params = []
+    for p in common_params:
+        d = p.to_schema()
+        d["common"] = True
+        params.append(d)
+    for p in own:
+        if p.name in common_names:
+            # Subclass overrode a common param (e.g. tightened doc).
+            for d in params:
+                if d["name"] == p.name:
+                    d.update(p.to_schema())
+                    break
+            continue
+        d = p.to_schema()
+        d["common"] = False
+        params.append(d)
+    return {"params_declared": True, "params": params}
+
+
+def dump_all_schemas():
+    """Return the full schema as a Python dict ready for json.dumps.
+
+    Shape:
+        {
+          "items": {
+            "sleep": {
+              "display_name": "Sleep",
+              "params_declared": true,
+              "params": [{name, kind, required, default?, doc, common}, ...],
+            },
+            "console": {
+              ...,
+              "actions": {"open": {...}, "close": {...}, ...},
+            },
+            ...
+          }
+        }
+    """
+    _constants_init()
+    # Imported lazily — pulls test_item.py which references constants.
+    from interpreter.test_items.test_item import COMMON_PARAMS
+
+    out = {"items": {}}
+    for tp in TestItemType:
+        cls = getattr(tp, "item_class", None)
+        if cls is None:
+            continue
+        # Action types (CONSOLE_ACTION, GRAPH_ACTION, JSON_RPC_ACTION) have no
+        # standalone YAML representation — skip them here, they show up under
+        # their parent's "actions" key.
+        cmd = tp.item_cmd
+        if cmd.endswith("_action"):
+            continue
+        entry = {"display_name": tp.item_name}
+        entry.update(_params_to_schema(cls, COMMON_PARAMS))
+
+        actions = _collect_action_classes(cls)
+        if actions:
+            entry["actions"] = {
+                name: _params_to_schema(acls, COMMON_PARAMS)
+                for name, acls in actions.items()
+            }
+            for name in entry["actions"]:
+                entry["actions"][name]["display_name"] = name
+
+        out["items"][cmd] = entry
+    return out
+
+
+def dump_all_schemas_json(indent=2):
+    """Same as ``dump_all_schemas`` but serialised to a JSON string."""
+    return json.dumps(dump_all_schemas(), indent=indent, sort_keys=False,
+                      default=str)

src/testium/lsp/server.py

@@ -0,0 +1,313 @@
+"""LSP server for ``.tum`` files.
+
+Features available so far:
+
+- **Completion** — when the user starts a new YAML step (``- <cursor>``),
+  the server proposes the full list of known item types. The completion
+  item carries a short hover-style description listing required and
+  optional parameters.
+- **Hover** — over a known item-type word (``sleep``, ``py_func``, …)
+  the server renders the same description in a popup.
+- **Document symbols (outline)** — every ``- <type>:`` line becomes an
+  entry in the editor's outline view. Nesting follows YAML indentation,
+  so containers (``group``, ``loop``, ``parallel``, ``console`` …)
+  display their children as a subtree.
+
+The server speaks LSP over stdio. Start it with::
+
+    testium lsp
+
+Editors invoke it through their LSP client; the connection layer
+(``vscode-languageclient``, ``nvim-lspconfig``, ``lsp-mode``, …) takes
+care of the JSON-RPC framing.
+
+Architecture notes
+------------------
+
+The schema is built once at server start (``dump_all_schemas()``) and
+kept in memory; an editor restart picks up upstream changes. The schema
+is the **only** source of truth — when testium adds a new item type or
+parameter, the LSP automatically exposes it without any change here.
+
+The current handlers stay deliberately heuristic on the parser side:
+completion uses a line-prefix regex, outline a per-line ``- <known>:``
+sweep with indentation tracking. A proper YAML+Jinja parsing pass is
+still pending and is the prerequisite for *parameter*-level completion
+and diagnostics.
+"""
+
+import re
+
+try:
+    # pygls 2.x moved LanguageServer under pygls.lsp.server. We pin >=1.3 in
+    # the optional dependency to stay open to either family, but the import
+    # path differs — try the new one first, then the legacy one.
+    try:
+        from pygls.lsp.server import LanguageServer
+    except ImportError:
+        from pygls.server import LanguageServer  # pygls < 2
+    from lsprotocol.types import (
+        TEXT_DOCUMENT_COMPLETION,
+        TEXT_DOCUMENT_DOCUMENT_SYMBOL,
+        TEXT_DOCUMENT_HOVER,
+        CompletionItem,
+        CompletionItemKind,
+        CompletionList,
+        CompletionOptions,
+        CompletionParams,
+        DocumentSymbol,
+        DocumentSymbolParams,
+        Hover,
+        HoverParams,
+        InsertTextFormat,
+        MarkupContent,
+        MarkupKind,
+        Position,
+        Range,
+        SymbolKind,
+    )
+except ImportError as exc:
+    # Surfaced by the CLI dispatcher with a friendly install hint.
+    raise
+
+
+from lsp.schema import dump_all_schemas
+
+
+_LINE_START_STEP = re.compile(r"^\s*-\s*([A-Za-z_][A-Za-z0-9_]*)?\s*:?\s*$")
+
+# Matches "- <identifier>:" for outline / hover purposes. Captures the start
+# column of the identifier and the identifier itself. Trailing tokens after
+# the colon (inline-form params, comments) are tolerated.
+_STEP_LINE = re.compile(r"^(?P<lead>\s*-\s*)(?P<ident>[A-Za-z_][A-Za-z0-9_]*)\s*:")
+
+# Matches a ``name: <value>`` line under an item — used by the outline pass
+# to surface the user's display name next to the item type.
+_NAME_FIELD = re.compile(r"^\s*name\s*:\s*(?P<value>.+?)\s*$")
+
+# Word boundary used by hover to extract the identifier under the cursor.
+_IDENT_AT = re.compile(r"[A-Za-z_][A-Za-z0-9_]*")
+
+
+def _render_item_markdown(cmd, entry):
+    """Render an item-type's schema entry as a Markdown hover string.
+
+    Reused by both the completion-item documentation and the hover
+    handler so the editor presents identical information regardless of
+    how the user reached it.
+    """
+    detail = entry.get("display_name", cmd)
+    lines = [f"**{cmd}** — {detail}", ""]
+    if entry.get("params_declared"):
+        non_common = [p for p in entry["params"] if not p["common"]]
+        required = [p for p in non_common if p["required"]]
+        optional = [p for p in non_common if not p["required"]]
+        if required:
+            lines.append("Required parameters:")
+            for p in required:
+                lines.append(f"- `{p['name']}` — {p['doc']}")
+            lines.append("")
+        if optional:
+            lines.append("Optional parameters:")
+            for p in optional:
+                lines.append(f"- `{p['name']}` — {p['doc']}")
+    else:
+        lines.append("(Parameter list is not described — this item's body is the "
+                     "raw user value.)")
+    return "\n".join(lines)
+
+
+def _build_item_completions(schema):
+    """Return a list of CompletionItem covering every top-level item type.
+
+    Each completion inserts ``<name>:`` with the cursor positioned after
+    the colon so the user can immediately start typing parameters.
+    """
+    items = []
+    for cmd, entry in schema["items"].items():
+        if cmd == "default":
+            # Root sentinel; never appears as a YAML key.
+            continue
+        items.append(
+            CompletionItem(
+                label=cmd,
+                kind=CompletionItemKind.Class,
+                detail=entry.get("display_name", cmd),
+                documentation=MarkupContent(
+                    kind=MarkupKind.Markdown,
+                    value=_render_item_markdown(cmd, entry),
+                ),
+                insert_text=f"{cmd}:",
+                insert_text_format=InsertTextFormat.PlainText,
+            )
+        )
+    items.sort(key=lambda it: it.label)
+    return items
+
+
+def _word_at(line, character):
+    """Return ``(start, end, text)`` of the identifier under ``character``.
+
+    Returns ``None`` when the cursor isn't on a word. Used by hover.
+    """
+    for m in _IDENT_AT.finditer(line):
+        if m.start() <= character <= m.end():
+            return m.start(), m.end(), m.group(0)
+    return None
+
+
+def _build_document_symbols(lines, item_cmds):
+    """Walk ``lines`` and produce a nested ``DocumentSymbol`` tree.
+
+    Heuristics (no YAML parsing yet):
+    - Each ``- <known_cmd>:`` line becomes a symbol.
+    - Nesting follows the indentation of the leading ``-``: a deeper-
+      indented step is treated as a child of the most recent shallower
+      step.
+    - The symbol's ``detail`` is the ``name: <value>`` field if found
+      within a small window after the step header (no YAML parsing —
+      we just look at indented lines that aren't another ``- …`` step).
+
+    The result is suitable for the LSP outline panel even when the
+    surrounding YAML is mid-edit and structurally invalid.
+    """
+    root_children = []
+    # Each stack entry: (indent_col, children_list_to_append_to,
+    #                    pending_parent_symbol or None).
+    stack = [(-1, root_children, None)]
+
+    def _attach_name(parent_symbol, start_line):
+        """Look for the nearest ``name:`` field in the children of ``parent``."""
+        if parent_symbol is None or start_line + 1 >= len(lines):
+            return
+        base_indent = len(lines[start_line]) - len(lines[start_line].lstrip(" "))
+        for j in range(start_line + 1, min(start_line + 10, len(lines))):
+            l = lines[j]
+            stripped = l.lstrip(" ")
+            indent = len(l) - len(stripped)
+            if indent <= base_indent and stripped.strip() != "":
+                break
+            m = _NAME_FIELD.match(l)
+            if m:
+                value = m.group("value").strip("\"' ")
+                parent_symbol.detail = value
+                return
+
+    for i, raw_line in enumerate(lines):
+        m = _STEP_LINE.match(raw_line)
+        if not m:
+            continue
+        cmd = m.group("ident")
+        if cmd not in item_cmds:
+            continue
+        indent = len(m.group("lead")) - len(m.group("lead").lstrip(" "))
+        # Pop the stack until we find a parent with strictly smaller indent.
+        while stack and stack[-1][0] >= indent:
+            stack.pop()
+        if not stack:
+            stack.append((-1, root_children, None))
+        parent_children = stack[-1][1]
+
+        ident_start = m.start("ident")
+        ident_end = m.end("ident")
+        symbol = DocumentSymbol(
+            name=cmd,
+            detail=None,
+            kind=SymbolKind.Function,
+            range=Range(
+                start=Position(line=i, character=0),
+                end=Position(line=i, character=len(raw_line.rstrip("\n"))),
+            ),
+            selection_range=Range(
+                start=Position(line=i, character=ident_start),
+                end=Position(line=i, character=ident_end),
+            ),
+            children=[],
+        )
+        parent_children.append(symbol)
+        stack.append((indent, symbol.children, symbol))
+        _attach_name(symbol, i)
+    return root_children
+
+
+def _make_server():
+    server = LanguageServer("testium-lsp", "0.1.0")
+    schema = dump_all_schemas()
+    item_completions = _build_item_completions(schema)
+    # Set of cmd names accepted by the outline / hover passes. We include
+    # action names (console open/close/…, plot open/close/…, …) too so they
+    # appear in the outline tree and respond to hover.
+    item_cmds = set()
+    for cmd, entry in schema["items"].items():
+        if cmd == "default":
+            continue
+        item_cmds.add(cmd)
+        item_cmds.update(entry.get("actions", {}).keys())
+
+    @server.feature(
+        TEXT_DOCUMENT_COMPLETION,
+        CompletionOptions(trigger_characters=["-", " "]),
+    )
+    def completion(params: CompletionParams):
+        doc = server.workspace.get_text_document(params.text_document.uri)
+        line_idx = params.position.line
+        if line_idx >= len(doc.lines):
+            return CompletionList(is_incomplete=False, items=[])
+        line = doc.lines[line_idx]
+        # Only look at what's left of the cursor.
+        prefix = line[: params.position.character]
+        if not _LINE_START_STEP.match(prefix):
+            return CompletionList(is_incomplete=False, items=[])
+        return CompletionList(is_incomplete=False, items=item_completions)
+
+    @server.feature(TEXT_DOCUMENT_HOVER)
+    def hover(params: HoverParams):
+        doc = server.workspace.get_text_document(params.text_document.uri)
+        line_idx = params.position.line
+        if line_idx >= len(doc.lines):
+            return None
+        line = doc.lines[line_idx]
+        # Only respond when the cursor is on the type part of a step line
+        # ("- sleep:") — never for arbitrary words in a string.
+        step_match = _STEP_LINE.match(line)
+        if not step_match:
+            return None
+        word = _word_at(line, params.position.character)
+        if word is None:
+            return None
+        start, end, text = word
+        if text != step_match.group("ident") or text not in item_cmds:
+            return None
+        # Resolve the entry: top-level item, or action of any parent.
+        entry = schema["items"].get(text)
+        if entry is None:
+            for parent_entry in schema["items"].values():
+                actions = parent_entry.get("actions") or {}
+                if text in actions:
+                    entry = actions[text]
+                    break
+        if entry is None:
+            return None
+        return Hover(
+            contents=MarkupContent(
+                kind=MarkupKind.Markdown,
+                value=_render_item_markdown(text, entry),
+            ),
+            range=Range(
+                start=Position(line=line_idx, character=start),
+                end=Position(line=line_idx, character=end),
+            ),
+        )
+
+    @server.feature(TEXT_DOCUMENT_DOCUMENT_SYMBOL)
+    def document_symbols(params: DocumentSymbolParams):
+        doc = server.workspace.get_text_document(params.text_document.uri)
+        return _build_document_symbols(doc.lines, item_cmds)
+
+    return server
+
+
+def serve():
+    """Start the LSP server on stdio. Blocks until the client disconnects."""
+    server = _make_server()
+    server.start_io()

test/benchmark/.gitignore

@@ -0,0 +1 @@
+cases/

test/benchmark/README.md

@@ -0,0 +1,116 @@
+# Load-time benchmark
+
+Measures how long *testium* takes to **load** a `.tum` test tree — template
+rendering (jinja) + YAML parsing + test-tree construction — *without* executing
+it. Purpose: get reproducible numbers before/after load-path optimisations, and
+attribute any gain to a specific part of the pipeline.
+
+It is meant for *very long* tests, the kind you can build with `jinja` loops and
+`!include`, where load time becomes noticeable.
+
+## Files
+
+| File | Role |
+|------|------|
+| `gen_bench_test.py` | Generates a synthetic `.tum` tree (the test input). |
+| `load_bench.py` | Drives the **real** loader in-process and times it. |
+| `run.sh` | Convenience: generate + time across profiles, using the project venv. |
+| `cases/` | Generated trees (git-ignored, recreated on demand). |
+
+The benchmark `.tum` files are **generated**, not committed — the generator is
+the artifact. They use only `let` leaves and `group` containers, so loading has
+no runtime side effect (no subprocess, no `<| |>` eval) and the timing reflects
+the parse/build pipeline alone.
+
+## Quick start
+
+```bash
+# default matrix (all profiles), 5 repeats each
+./test/benchmark/run.sh
+
+# one profile at one size
+./test/benchmark/run.sh repeat 2000
+
+# more repeats for a tighter min
+REPEAT=10 ./test/benchmark/run.sh includes 1000
+```
+
+`run.sh` uses the project venv at `test/tmp/.venv` (created by `./run.sh`). If it
+is missing, run `./run.sh` once first.
+
+To drive the harness directly on any `.tum` (not just generated ones):
+
+```bash
+test/tmp/.venv/bin/python3 test/benchmark/load_bench.py --repeat 5 --quiet path/to/main.tum
+```
+
+## Profiles
+
+Each profile isolates one cost. `--size` is the profile-specific count.
+
+| Profile | What it builds | Stresses |
+|---------|----------------|----------|
+| `flat` | one main file, N inline `let` steps | big YAML parse + linear object build |
+| `includes` | main `!include`s N **distinct** sub-files | per-include template+YAML+tempfile, `sequence` splice |
+| `repeat` | main `!include`s the **same** parametrised leaf N times | jinja **recompilation** of an identical template |
+| `jinja` | one main file, `{% for %}` emitting N steps | single large render + single large parse |
+| `deep` | nested includes, depth N | include recursion (see caveat) |
+| `mix` | groups + jinja loop + distinct + repeated includes | realistic blend |
+
+## Reading the output
+
+```
+phase              min      median
+initial         0.1131      0.1285   <- pass 1: discover config files (no includes)
+loadtest        1.0724      1.0900   <- config fixpoint loop + full recursive include load
+build           0.1850      0.1976   <- TestSet: load_test_recursively tree build
+total           1.3886      1.4227
+counters  (last run):
+  templates :    1003 calls   0.5247s  (exclusive: jinja compile+render+tempfile)
+  yaml      :    1004 parses   1.4696s  (inclusive of nested includes)
+```
+
+- **min** is the headline (least noisy); median is a sanity check.
+- **initial / loadtest / build** map to the three pipeline stages in
+  `interpreter/process.py` and `interpreter/test_set.py`. The main file is
+  rendered+parsed across `initial` *and* `loadtest` (the loader does ~3 passes).
+- **templates** = number of `template_to_test()` calls and their *exclusive*
+  wall time (one file render each — pure jinja compile+render+tempfile I/O).
+  A high count with the same source file = recompilation, the `repeat` case.
+- **yaml** = number of `yaml_load()` parses. Its time is *inclusive* of nested
+  includes, so use the **count** for attribution, not the seconds.
+
+## Mapping to the optimisation axes
+
+| Axis (see DESIGN / discussion) | Watch | Best profile to prove it |
+|--------------------------------|-------|--------------------------|
+| 1 — cache compiled jinja templates | `templates` time drops, count unchanged | `repeat` |
+| 2 — drop the tempfile round-trip | `templates` time drops | `includes`, `repeat`, `mix` |
+| 3 — C YAML loader (libyaml) | `yaml` time / `loadtest` drops | `flat`, `jinja` |
+| 6 — O(n²) sequence splice | `build` drops | `includes`, `mix` |
+
+## How to compare before/after a change
+
+1. Run the matrix on the current code, keep the output.
+2. Apply one axis.
+3. Re-run the **same** profiles/sizes; compare `min` per phase and the counters.
+
+Change one axis at a time so the attribution is clean. Run on an idle machine
+(and note the disk: on a USB stick the tempfile round-trip of axis 2 weighs
+more).
+
+## Caveat: deep includes
+
+The loader is recursive and spends ~10 stack frames per include level, so
+`deep` hits Python's `RecursionError` around ~90 nested levels. The harness
+reports this cleanly instead of crashing. Real tests are *wide* (many steps /
+many includes), not deep, so `includes`/`repeat`/`jinja`/`mix` are the
+representative "very long" cases.
+
+## Notes
+
+- No execution is triggered — timing stops where `Batch` would mark the test
+  *loaded*.
+- The profiles contain no `<| |>`, so the external eval process is not started.
+  Pass `--with-eval` to `load_bench.py` for trees that evaluate at load time.
+- Numbers are machine- and disk-specific; only compare runs from the same host.

test/benchmark/gen_bench_test.py

@@ -0,0 +1,179 @@
+#!/usr/bin/env python3
+"""Generate synthetic ``.tum`` test trees to benchmark *load* time.
+
+The generated trees are deliberately cheap to *build* (only ``let`` leaves and
+``group`` containers — no subprocess, no runtime side effect) so the load
+benchmark measures the parse / template / tree-build pipeline and nothing else.
+
+Profiles, each targeting a specific cost in the loader:
+
+  flat      one main file, N inline ``let`` steps, no include, no jinja.
+            Baseline: YAML parse of a big document + linear object build.
+
+  includes  main ``!include``s N *distinct* sub-files (a few steps each).
+            Stresses the per-include template+YAML+tempfile round-trip and the
+            ``sequence`` splice in test_set.load_test_recursively.
+
+  repeat    main ``!include``s the *same* parametrised leaf file N times.
+            Stresses jinja *recompilation*: the compiled template is identical
+            every time, only the render params (idx) differ -> the case a
+            template cache collapses.
+
+  jinja     one main file whose ``{% for %}`` loop emits N steps.
+            Stresses a single large jinja render + a single large YAML parse.
+
+  deep      nested includes, depth N (main -> d0 -> d1 -> ...).
+            Stresses include recursion and per-level template+YAML.
+
+  mix       a realistic blend: groups, a jinja loop, distinct includes and a
+            repeated parametrised include.
+
+Usage:
+    gen_bench_test.py --profile repeat --size 1000 --out cases/repeat_1000
+    -> writes <out>/main.tum (+ includes, + param.yaml) and prints the path.
+"""
+import argparse
+import os
+import shutil
+
+
+def _let(indent, i, name=None):
+    name = name if name is not None else f"s{i}"
+    pad = " " * indent
+    return (
+        f"{pad}- let:\n"
+        f"{pad}    name: {name}\n"
+        f"{pad}    values:\n"
+        f"{pad}        - k{i}: {i}\n"
+    )
+
+
+def gen_flat(out, n):
+    body = "".join(_let(8, i) for i in range(n))
+    main = f"main:\n    name: bench flat {n}\n    steps:\n{body}"
+    _write(out, "main.tum", main)
+
+
+def gen_includes(out, n):
+    steps = "".join(f"        - !include inc_{i}.tum\n" for i in range(n))
+    main = f"main:\n    name: bench includes {n}\n    steps:\n{steps}"
+    _write(out, "main.tum", main)
+    for i in range(n):
+        # each include is a YAML *sequence* (list of steps)
+        seq = "".join(_let(0, i * 3 + j, name=f"inc{i}_{j}") for j in range(3))
+        _write(out, f"inc_{i}.tum", seq)
+
+
+def gen_repeat(out, n):
+    steps = "".join(
+        f"        - !include {{file: leaf.tum, idx: {i}}}\n" for i in range(n)
+    )
+    main = f"main:\n    name: bench repeat {n}\n    steps:\n{steps}"
+    _write(out, "main.tum", main)
+    leaf = (
+        "- let:\n"
+        "    name: leaf_{{ idx }}\n"
+        "    values:\n"
+        "        - leaf_{{ idx }}: {{ idx }}\n"
+    )
+    _write(out, "leaf.tum", leaf)
+
+
+def gen_jinja(out, n):
+    main = (
+        f"main:\n    name: bench jinja {n}\n    steps:\n"
+        "{% for i in range(" + str(n) + ") %}\n"
+        "        - let:\n"
+        "            name: j{{ i }}\n"
+        "            values:\n"
+        "                - k{{ i }}: {{ i }}\n"
+        "{% endfor %}\n"
+    )
+    _write(out, "main.tum", main)
+
+
+def gen_deep(out, n):
+    main = (
+        f"main:\n    name: bench deep {n}\n    steps:\n"
+        "        - let:\n            name: top\n            values:\n                - a: 0\n"
+        "        - !include d_0.tum\n"
+    )
+    _write(out, "main.tum", main)
+    for i in range(n):
+        seq = _let(0, i, name=f"d{i}")
+        if i < n - 1:
+            seq += f"- !include d_{i + 1}.tum\n"
+        _write(out, f"d_{i}.tum", seq)
+
+
+def gen_mix(out, n):
+    # n groups, each: 2 inline lets, one distinct include, one repeated include,
+    # plus a small jinja loop. Roughly ~6*n steps.
+    per = max(1, n)
+    parts = [f"main:\n    name: bench mix {n}\n    steps:\n"]
+    for g in range(per):
+        parts.append(
+            f"        - group:\n"
+            f"            name: grp{g}\n"
+            f"            steps:\n"
+        )
+        parts.append(_let(16, g * 2, name=f"g{g}_a"))
+        parts.append(_let(16, g * 2 + 1, name=f"g{g}_b"))
+        parts.append(f"                - !include inc_{g}.tum\n")
+        parts.append(f"                - !include {{file: leaf.tum, idx: {g}}}\n")
+        parts.append(
+            "{% for i in range(3) %}\n"
+            f"                - let:\n"
+            f"                    name: g{g}_j{{{{ i }}}}\n"
+            f"                    values:\n"
+            f"                        - g{g}_k{{{{ i }}}}: {{{{ i }}}}\n"
+            "{% endfor %}\n"
+        )
+    _write(out, "main.tum", "".join(parts))
+    for g in range(per):
+        _write(out, f"inc_{g}.tum", _let(0, g, name=f"mixinc{g}"))
+    _write(
+        out,
+        "leaf.tum",
+        "- let:\n    name: mixleaf_{{ idx }}\n    values:\n        - mixleaf_{{ idx }}: {{ idx }}\n",
+    )
+
+
+PROFILES = {
+    "flat": gen_flat,
+    "includes": gen_includes,
+    "repeat": gen_repeat,
+    "jinja": gen_jinja,
+    "deep": gen_deep,
+    "mix": gen_mix,
+}
+
+
+def _write(out, name, content):
+    with open(os.path.join(out, name), "w") as f:
+        f.write(content)
+
+
+def main():
+    ap = argparse.ArgumentParser(description=__doc__,
+                                 formatter_class=argparse.RawDescriptionHelpFormatter)
+    ap.add_argument("--profile", required=True, choices=sorted(PROFILES))
+    ap.add_argument("--size", type=int, default=1000,
+                    help="profile-specific count (steps / includes / depth)")
+    ap.add_argument("--out", required=True, help="output directory (recreated)")
+    args = ap.parse_args()
+
+    out = os.path.abspath(args.out)
+    if os.path.isdir(out):
+        shutil.rmtree(out)
+    os.makedirs(out)
+
+    # minimal config file so the loader does not emit "no param file" noise
+    _write(out, "param.yaml", "bench_dummy: 1\n")
+
+    PROFILES[args.profile](out, args.size)
+    print(os.path.join(out, "main.tum"))
+
+
+if __name__ == "__main__":
+    main()

test/benchmark/load_bench.py

@@ -0,0 +1,200 @@
+#!/usr/bin/env python3
+"""Time the testium *load* pipeline on a given ``.tum`` tree.
+
+It drives the real loader code (``TestProcess._load_initial_params`` /
+``_load_test`` then ``TestSet(...)``) in-process, so the numbers track the
+production path and stay honest as the code evolves. Execution is never
+triggered — we stop exactly where ``Batch`` would report the test as *loaded*.
+
+Reported per run, over ``--repeat`` iterations (min is the headline, least
+noisy):
+
+    initial   first pass: discover config files (template+YAML, no includes)
+    loadtest  config-file fixpoint loop + full recursive include/template/YAML
+    build     TestSet construction: the load_test_recursively tree build
+    total     sum of the three
+
+Plus instrumentation counters (exact call counts, wall time) for the two
+hot leaves the optimisation axes target:
+
+    templates  jinja template_to_test() calls   (axis 1 compile cache, axis 2 tempfile)
+    yaml       yaml_load() parses               (axis 3 C loader)
+
+template time is exclusive (one file render); yaml time is wall-inclusive of
+nested includes, so lean on the *counts* for attribution.
+
+Must run inside the project venv (jinja2, pyyaml, telnetlib3, ...). The
+benchmark profiles contain no ``<| |>`` so the external eval process is not
+needed; pass --with-eval to start it for faithfulness on eval-heavy trees.
+
+Usage (see run.sh for the convenience wrapper):
+    test/tmp/.venv/bin/python3 test/benchmark/load_bench.py [--repeat 5] <main.tum>
+"""
+import argparse
+import os
+import statistics
+import sys
+from queue import Queue
+from time import perf_counter
+
+# --- bootstrap: src/testium for flat imports, src for `import testium` --------
+HERE = os.path.dirname(os.path.abspath(__file__))
+ROOT = os.path.abspath(os.path.join(HERE, "..", ".."))
+sys.path.insert(0, os.path.join(ROOT, "src"))
+sys.path.insert(0, os.path.join(ROOT, "src", "testium"))
+
+import api.testium as tm  # noqa: E402
+from interpreter.utils.test_init import env_init, apply_overrides  # noqa: E402
+from interpreter.utils.test_ctrl import TestSetController  # noqa: E402
+from interpreter.process import TestProcess  # noqa: E402
+from interpreter.test_set import TestSet  # noqa: E402
+from interpreter.utils.py_eval import eval_process_init  # noqa: E402
+from interpreter.utils.api_srv import api_request  # noqa: E402
+
+# --- instrumentation: count + time the two hot leaves -------------------------
+import interpreter.process as _proc  # noqa: E402
+import interpreter.utils.include as _inc  # noqa: E402
+import interpreter.utils.test_init as _ti  # noqa: E402
+import interpreter.utils.template as _tpl  # noqa: E402
+import interpreter.utils.yaml_load as _yl  # noqa: E402
+
+_C = {"tpl_n": 0, "tpl_t": 0.0, "yaml_n": 0, "yaml_t": 0.0}
+_orig_tpl = _tpl.template_to_test
+_orig_yaml = _yl.yaml_load
+
+
+def _wrap_tpl(*a, **k):
+    t = perf_counter()
+    try:
+        return _orig_tpl(*a, **k)
+    finally:
+        _C["tpl_t"] += perf_counter() - t
+        _C["tpl_n"] += 1
+
+
+def _wrap_yaml(*a, **k):
+    t = perf_counter()
+    try:
+        return _orig_yaml(*a, **k)
+    finally:
+        _C["yaml_t"] += perf_counter() - t
+        _C["yaml_n"] += 1
+
+
+# rebind in every module that did `from ... import template_to_test / yaml_load`
+for _m in (_proc, _inc):
+    _m.template_to_test = _wrap_tpl
+for _m in (_proc, _inc, _ti):
+    _m.yaml_load = _wrap_yaml
+
+
+def _reset_counters():
+    _C.update(tpl_n=0, tpl_t=0.0, yaml_n=0, yaml_t=0.0)
+
+
+def load_once(tp, fname, test_dir):
+    """One full load (no execution). Returns (initial, loadtest, build) seconds."""
+    t0 = perf_counter()
+    init_pf, gv = tp._load_initial_params(test_dir)
+    t1 = perf_counter()
+    test_dict, _pf = tp._load_test(init_pf, gv)
+    t2 = perf_counter()
+    TestSet(fname, test_dict, Queue())
+    t3 = perf_counter()
+    return (t1 - t0, t2 - t1, t3 - t2)
+
+
+def main():
+    ap = argparse.ArgumentParser(description=__doc__,
+                                 formatter_class=argparse.RawDescriptionHelpFormatter)
+    ap.add_argument("main_tum", help="path to the generated main.tum")
+    ap.add_argument("--repeat", type=int, default=5)
+    ap.add_argument("--with-eval", action="store_true",
+                    help="start the external eval process (needed only for <| |> at load)")
+    ap.add_argument("--quiet", action="store_true",
+                    help="silence the loader's INFO output during runs")
+    args = ap.parse_args()
+
+    fname = os.path.abspath(args.main_tum)
+    if not os.path.isfile(fname):
+        ap.error(f"not found: {fname}")
+    test_dir = os.path.dirname(fname)
+
+    env_init()
+    apply_overrides({}, {})
+
+    eval_proc = None
+    if args.with_eval:
+        eval_proc = eval_process_init(api_request, 10, test_dir)
+        eval_proc.start()
+        eval_proc.wait_ready(10)
+
+    if args.quiet:
+        # the loader prints a couple of INFO lines per config file; mute stdout
+        # around the measured section to avoid I/O skew.
+        devnull = open(os.devnull, "w")
+        real_stdout = sys.stdout
+
+    tp = TestProcess(fname, Queue(), TestSetController(),
+                     config_files=[], defines={}, gui_defaults={}, text_mode=True)
+
+    samples = []  # list of (initial, loadtest, build)
+    last_counters = None
+    try:
+        for r in range(args.repeat):
+            _reset_counters()
+            if args.quiet:
+                sys.stdout = devnull
+            try:
+                samples.append(load_once(tp, fname, test_dir))
+            except RecursionError:
+                if args.quiet:
+                    sys.stdout = real_stdout
+                print(f"file      : {fname}")
+                print("ERROR     : RecursionError during load — the include "
+                      "nesting is too deep for the recursive loader.\n"
+                      "            (each include level costs ~10 stack frames; "
+                      "raise sys.setrecursionlimit to probe further.)")
+                return 2
+            except Exception as e:  # noqa: BLE001 - report, don't crash the bench
+                if args.quiet:
+                    sys.stdout = real_stdout
+                print(f"file      : {fname}")
+                print(f"ERROR     : load failed: {type(e).__name__}: {e}")
+                return 2
+            finally:
+                if args.quiet:
+                    sys.stdout = real_stdout
+            last_counters = dict(_C)
+    finally:
+        if eval_proc is not None:
+            eval_proc.stop()
+            eval_proc.join()
+        if args.quiet:
+            devnull.close()
+
+    initial = [s[0] for s in samples]
+    loadtest = [s[1] for s in samples]
+    build = [s[2] for s in samples]
+    total = [sum(s) for s in samples]
+
+    def stat(xs):
+        return min(xs), statistics.median(xs)
+
+    print(f"file      : {fname}")
+    print(f"repeats   : {args.repeat}   (showing  min | median, seconds)")
+    print(f"{'phase':<10}{'min':>12}{'median':>12}")
+    for name, xs in (("initial", initial), ("loadtest", loadtest),
+                     ("build", build), ("total", total)):
+        mn, md = stat(xs)
+        print(f"{name:<10}{mn:>12.4f}{md:>12.4f}")
+    if last_counters:
+        print("counters  (last run):")
+        print(f"  templates : {last_counters['tpl_n']:>7d} calls   "
+              f"{last_counters['tpl_t']:>8.4f}s  (exclusive: jinja compile+render+tempfile)")
+        print(f"  yaml      : {last_counters['yaml_n']:>7d} parses  "
+              f"{last_counters['yaml_t']:>8.4f}s  (inclusive of nested includes)")
+
+
+if __name__ == "__main__":
+    sys.exit(main() or 0)

test/benchmark/run.sh

@@ -0,0 +1,49 @@
+#!/bin/bash
+# Load-time benchmark driver: generate synthetic .tum trees and time the
+# testium load pipeline on them, using the project venv.
+#
+# Usage:
+#   ./test/benchmark/run.sh                     # default matrix (all profiles)
+#   ./test/benchmark/run.sh <profile> <size>    # one profile at one size
+#   REPEAT=10 ./test/benchmark/run.sh repeat 2000
+#
+# Profiles: flat includes repeat jinja deep mix   (see gen_bench_test.py)
+#
+# Generated trees go under test/benchmark/cases/ (git-ignored). The numbers
+# are wall-clock; run on an otherwise idle machine and compare min values.
+set -e
+
+SCRIPT_DIR="$(realpath "$(dirname "$(readlink -f "$0")")")"
+PROJECT_DIR="$(realpath "$SCRIPT_DIR/../..")"
+VPY="$PROJECT_DIR/test/tmp/.venv/bin/python3"
+CASES="$SCRIPT_DIR/cases"
+REPEAT="${REPEAT:-5}"
+
+if [ ! -x "$VPY" ]; then
+    echo "ERROR: project venv not found at $VPY — run ./run.sh once to create it." >&2
+    exit 1
+fi
+
+bench() {
+    local profile="$1" size="$2"
+    local out="$CASES/${profile}_${size}"
+    local main
+    main="$("$VPY" "$SCRIPT_DIR/gen_bench_test.py" --profile "$profile" --size "$size" --out "$out")"
+    echo "===== profile=$profile size=$size ====="
+    "$VPY" "$SCRIPT_DIR/load_bench.py" --repeat "$REPEAT" --quiet "$main"
+    echo
+}
+
+if [ $# -eq 2 ]; then
+    bench "$1" "$2"
+    exit 0
+fi
+
+# Default matrix. 'deep' is kept small: the recursive loader hits Python's
+# recursion limit around ~90 nested include levels.
+bench flat     2000
+bench includes 1000
+bench repeat   1000
+bench jinja    2000
+bench deep     40
+bench mix      300

test/robustness/console serial/rsl_terminal_robustness.tum

@@ -1,69 +0,0 @@
-# Main
-################################################################################
-main:
-  name:                               Serial Terminal bug reproducer
-  version:                            0.1
-  steps:
-    - group:
-        name:                         Test preparation
-        steps:
-          - console:
-              name:             Open RSL Simulator Terminal
-              console_name:     RSL_simulator
-              steps:
-                  - open:
-                      protocol:   terminal
-                      terminal_path: $(rslsimulatorpath)
-                  - writeln: "pwd"
-                  - read_until: {expected: "$", timeout: 5}
-                  - writeln: "./RSverify $(rsTx)" # /dev/ttyMUE1
-                  - read_until: {expected: "RSL controller>", timeout: 5}
-                  - writeln: "setportconf 0 115200 none 8 1 1 255"
-                  - read_until: {expected: "RSL controller>", timeout: 5}
-                  - writeln: "send4ever 0 0"
-                  - read_until: {expected: "RSL controller>", timeout: 5}
-
-          - console:
-              name:                   Open the EUT console
-              console_name:           cons_target
-              doc:                    Initiates the console of the target in order
-                                      to be ready to capture its traces.
-              stop_on_failure:        True
-              steps:
-                - open:
-                    protocol:         serial
-                    serial_port:      $(rsRx) # /dev/ttyMUE2
-                    serial_baudrate:  115200
-
-    - loop:
-        name:                         Qualification loop
-        stop_on_failure:              False
-        steps:
-          - py_func:
-              name:                   Capture the RS serial output
-              file:                   $(test_directory)/terminal_bug_reproducer.py
-              func_name:              RetreiveData
-              param:
-                - cons_target
-
-          - sleep: {timeout: 1}
-
-# Cleanup sequence
-#-------------------------------------------------------------------------------
-    - group:
-        name:                         Cleanup
-        execute_on_stop:              True
-        steps:
-          - console:
-              name:                   Close the target console
-              console_name:           cons_target
-              execute_on_stop:        True
-              steps:
-                - close:
-
-          - console:
-              name:                   Close the RSL_simulator
-              console_name:           RSL_simulator
-              execute_on_stop:        True
-              steps:
-                - close:

test/robustness/console serial/terminal_bug_reproducer.py

@@ -1,26 +0,0 @@
-import api.testium as tm
-
-def RetreiveData(console_name):
-    print("--------------- retrieving data ---------------")
-    result = 0
-    cons   = tm.console(console_name)
-
-    if cons is None:
-        print("--------------- The console does not exist ---------------")
-    else:
-        try:
-            is_finished = False
-            while not is_finished:
-                status, d = cons.read_until('\n', timeout=0, return_data=True, mute=True)
-                if 0 == status:
-                    print("--------------- Data ---------------")
-                    print(d)
-                else:
-                    print("--------------- No data ---------------")
-                    print("Status: ", status)
-                    is_finished = True
-        except:
-            print("--------------- Error retrieving data ---------------")
-            result = -1
-
-    return result

test/robustness/console terminal/generate_char.sh

@@ -1,9 +0,0 @@
-chars='<=>| -,;:!/."()[]{}*\&#%+012345689abcdefghiklmnopqrstuvwxyzABCD'
-for j in {1..256} ;
-do
-    for i in {1..256} ; do
-        echo -n "${chars:RANDOM%${#chars}:1}"
-    done
-    echo
-    sleep 0.01
-done

test/robustness/console terminal/terminal_bug_reproducer.py

@@ -1,26 +0,0 @@
-import api.testium as tm
-
-def RetreiveData(console_name):
-    print("--------------- retrieving data ---------------")
-    result = 0
-    cons   = tm.console(console_name)
-
-    if cons is None:
-        print("--------------- The console does not exist ---------------")
-    else:
-        try:
-            is_finished = False
-            while not is_finished:
-                status, d = cons.read_until('\n', timeout=0, return_data=True, mute=True)
-                if 0 == status:
-                    print("--------------- Data ---------------")
-                    print(d)
-                else:
-                    print("--------------- No data ---------------")
-                    print("Status: ", status)
-                    is_finished = True
-        except:
-            print("--------------- Error retrieving data ---------------")
-            result = -1
-
-    return result

test/robustness/console terminal/terminal_robustness.tum

@@ -1,50 +0,0 @@
-# Main
-################################################################################
-main:
-  name:                               Terminal bug reproducer
-  version:                            0.1
-  steps:
-    - group:
-        name:                         Test preparation
-        steps:
-          - console:
-              name:                   Open the EUT console
-              console_name:           cons_target
-              doc:                    Initiates the console of the target in order
-                                      to be ready to capture its traces.
-              stop_on_failure:        True
-              steps:
-                - open:
-                    protocol: terminal
-
-    - loop:
-        name:                         Qualification loop
-        stop_on_failure:              False
-        steps:
-          - console:
-              name:                   write random data
-              console_name:           cons_target
-              steps:
-                  - writeln: bash $(test_directory)/generate_char.sh
-
-          - py_func:
-              name:                   Capture the terminal output
-              file:                   $(test_directory)/terminal_bug_reproducer.py
-              func_name:              RetreiveData
-              param:
-                - cons_target
-
-          - sleep: {timeout: 1}
-
-# Cleanup sequence
-#-------------------------------------------------------------------------------
-    - group:
-        name:                         Cleanup
-        execute_on_stop:              True
-        steps:
-          - console:
-              name:                   Close the target console
-              console_name:           cons_target
-              execute_on_stop:        True
-              steps:
-                - close:

test/validation/lsp_check.py

@@ -0,0 +1,102 @@
+#!/usr/bin/env python3
+"""Per-channel check of the testium language server.
+
+Given the channel's testium invocation as argv (e.g. ``flatpak run
+--command=testium org.testium.Testium``, a PyInstaller binary path, or
+``python -m testium``), verify two things end-to-end against that exact build:
+
+  1. ``<cmd> schema`` produces valid JSON whose item registry still includes the
+     nested action sets (console/plot/json_rpc). This catches a frozen build
+     that lost the actions — the failure mode the declarative ``ACTIONS``
+     refactor fixed (no more ``inspect.getsource`` at runtime).
+  2. ``<cmd> lsp`` starts a real language server: it must answer an LSP
+     ``initialize`` request with a capabilities result and must NOT report the
+     pygls dependency as missing. This catches a channel that forgot to bundle
+     the ``[lsp]`` extra.
+
+Exits non-zero (with a diagnostic) on the first failure so the validation run
+fails loudly. Used by ``run.sh`` before launching the main suite.
+"""
+import json
+import subprocess
+import sys
+
+EXPECTED_ACTION_PARENTS = ("console", "plot", "json_rpc")
+
+
+def fail(msg):
+    print(f"LSP CHECK: FAIL — {msg}", file=sys.stderr)
+    sys.exit(1)
+
+
+def _extract_json(raw):
+    """Parse JSON from ``raw`` bytes, tolerating leading non-JSON noise.
+
+    The source-mode launcher (run.sh) may print env-setup lines before the
+    schema JSON, so we fall back to parsing from the first ``{``.
+    """
+    try:
+        return json.loads(raw)
+    except json.JSONDecodeError:
+        start = raw.find(b"{")
+        if start < 0:
+            raise
+        return json.loads(raw[start:])
+
+
+def check_schema(cmd):
+    try:
+        out = subprocess.run(cmd + ["schema"], capture_output=True, timeout=120)
+    except Exception as e:  # noqa: BLE001
+        fail(f"`{' '.join(cmd)} schema` could not run: {e}")
+    if out.returncode != 0:
+        fail(f"`schema` exited {out.returncode}: {out.stderr.decode()[:300]}")
+    try:
+        data = _extract_json(out.stdout)
+    except json.JSONDecodeError as e:
+        fail(f"`schema` output is not valid JSON: {e}")
+    items = data.get("items", {})
+    for parent in EXPECTED_ACTION_PARENTS:
+        actions = (items.get(parent) or {}).get("actions") or {}
+        if not actions:
+            fail(f"schema item '{parent}' has no actions — a frozen build lost "
+                 f"the declarative ACTIONS (item keys: {sorted(items)[:8]}…)")
+    print(f"LSP CHECK: schema OK ({len(items)} items; actions present for "
+          f"{', '.join(EXPECTED_ACTION_PARENTS)})")
+
+
+def check_lsp(cmd):
+    body = json.dumps({
+        "jsonrpc": "2.0", "id": 1, "method": "initialize",
+        "params": {"processId": None, "rootUri": None, "capabilities": {}},
+    }).encode()
+    msg = b"Content-Length: %d\r\n\r\n%s" % (len(body), body)
+    try:
+        out = subprocess.run(cmd + ["lsp"], input=msg,
+                             capture_output=True, timeout=30)
+        stdout, stderr = out.stdout, out.stderr
+    except subprocess.TimeoutExpired as e:
+        # A server that stays alive past initialize is fine — it just never saw
+        # a shutdown. Use whatever it wrote so far as the response.
+        stdout, stderr = e.stdout or b"", e.stderr or b""
+    blob = stdout + stderr
+    if b"dependencies missing" in blob:
+        fail("`lsp` reports the pygls dependency missing — this channel did "
+             "not bundle the [lsp] extra.")
+    if b'"capabilities"' not in stdout:
+        fail("`lsp` did not return an initialize result. "
+             f"stdout[:200]={stdout[:200]!r} stderr[:200]={stderr[:200]!r}")
+    print("LSP CHECK: lsp initialize OK (server answered with capabilities)")
+
+
+def main():
+    cmd = sys.argv[1:]
+    if not cmd:
+        fail("usage: lsp_check.py <testium-invocation...>")
+    check_schema(cmd)
+    check_lsp(cmd)
+    print("LSP CHECK: PASS")
+
+
+if __name__ == "__main__":
+    main()

test/validation/run.sh

@@ -97,7 +97,10 @@ case "$MODE" in
             echo "Creating wheel venv at $WHEEL_VENV"
             python3 -m venv --system-site-packages "$WHEEL_VENV"
             "$WHEEL_VENV/bin/pip" install --quiet --upgrade pip
-            "$WHEEL_VENV/bin/pip" install --quiet "$WHEEL"
+            # Install with the [lsp] extra so the wheel channel is validated in
+            # its language-server-capable form (pulls pygls), matching how a
+            # user enables `testium lsp` from a wheel: pip install testium[lsp].
+            "$WHEEL_VENV/bin/pip" install --quiet "${WHEEL}[lsp]"
         fi
         CMD=("$WHEEL_VENV/bin/python" -m testium)
         ;;
@@ -137,6 +140,13 @@ esac
 echo "-- validation mode: $MODE"
 echo "-- launch: ${CMD[*]}"
 
+# ---------- LSP check (this exact channel) ------------------------------------
+# Verify `testium lsp` / `testium schema` work in the build under test before
+# running the suite: schema must keep its nested actions (declarative ACTIONS,
+# survives frozen builds) and the language server must start (pygls bundled).
+echo "-- LSP check ($MODE)"
+"$VENV_PYTHON" "$SCRIPT_DIR/lsp_check.py" "${CMD[@]}"
+
 exec "${CMD[@]}" -b \
     -d "python_bin=$VENV_PYTHON" \
     -d "validation_report_file=validation-$MODE" \