lsp: declarative action registry + cross-channel language server

Make `testium lsp` (and the testium_assist editor extension that spawns it)
work from every distribution channel: source, wheel, PyInstaller, Flatpak,
AppImage.

Two enablers:

1. Declarative ACTIONS registry. The TestItemActions parents (console, plot,
   json_rpc) now declare their nested actions as a class attribute
   `ACTIONS = {yaml_key: class}`, mirroring PARAMS. The base __init__ seeds
   action_classes from type(self).ACTIONS; register_actions() is kept only as
   an imperative escape hatch. lsp/schema.py reads ACTIONS directly, dropping
   the inspect.getsource/AST walk that returned no actions in a frozen
   PyInstaller build (no .py source on disk).

2. pygls bundled per channel. Kept as the pyproject [lsp] extra (lean
   `pip install testium`), layered into each full-app channel:
   - build_env.sh installs pygls into test/tmp/.venv (source run + PyInstaller
     build env)
   - AppImage installs the wheel as `…whl[lsp]`
   - Flatpak adds a python3-lsp network-pip module (matches the manifest's
     global --share=network)
   - PyInstaller .spec collect_submodules(pygls/lsprotocol) + hiddenimports for
     the lazily-imported lsp/lsp.server/lsp.schema

test/validation/lsp_smoke.py (run by run.sh before the suite) enforces both
per channel: `<channel> schema` must keep console/plot/json_rpc actions and
`<channel> lsp` must answer an initialize request without reporting pygls
missing. Verified for source mode; the other channels need a rebuild to verify.

DESIGN.md updated (declarative section + new "Language server across channels"
subsection + Recent fixes).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

DESIGN.md

@@ -254,13 +254,25 @@ Validation is **opt-in per subclass**: while a subclass keeps `PARAMS = None` (t
 
 Diagnostics are currently **warnings** for unknown params so an out-of-tree `.tum` with a pre-existing typo doesn't suddenly fail. The flip to a hard error is a one-line change in `_validate_declared_params()` once the user is comfortable.
 
-The schema is also designed to be the source of truth for a future LSP server and for auto-generated manual sections: `ParamSet.to_schema()` returns the JSON-Schema-shaped representation that those consumers will read.
+Action items follow the same declarative principle. A `TestItemActions` parent (`console`, `plot`, `json_rpc`) declares its nested actions as a class attribute `ACTIONS = {yaml_key: action_class}` (e.g. `{"open": TestItemConsoleOpen, "write": …}`), mirroring `PARAMS`. The base `TestItemActions.__init__` seeds `self.action_classes` from `type(self).ACTIONS`; the imperative `register_actions(**…)` method is retained only as an escape hatch for actions that can't be known at class-definition time (none today). Because the action classes are always defined above their parent in the module, the class-level dict resolves without forward-reference gymnastics.
+
+The schema is the realized source of truth for the LSP server (`testium lsp`), the `testium schema` CLI dump, and future auto-generated manual sections: `ParamSet.to_schema()` returns the JSON-Schema-shaped representation, and `lsp/schema.py` reads both `PARAMS` and `ACTIONS` **purely from class attributes** — no `inspect.getsource`/AST parsing. This is what lets the full schema (including nested actions) survive a frozen PyInstaller build where the `.py` source isn't on disk.
+
+### Language server (`testium lsp`) across channels
+
+The `testium_assist` editor extension is a thin LSP client that spawns `testium lsp` and talks JSON-RPC over stdio, so the language server must work from *every* distribution channel. Two requirements:
+
+1. **`pygls` (+ `lsprotocol`, `cattrs`, `attrs`, `typing_extensions`) must be bundled.** It is the pyproject `[lsp]` extra (kept optional so a plain `pip install testium` stays lean), wired into each full-app channel: `build_env.sh` installs it into the shared `test/tmp/.venv` (covers **source run** and the **PyInstaller** build env); the **AppImage** installs the wheel as `…whl[lsp]`; the **Flatpak** adds a `python3-lsp` pip module (network-at-build, consistent with the manifest's global `--share=network`); the **PyInstaller** `.spec` force-collects the submodules via `collect_submodules` + explicit `hiddenimports` (including the lazily-imported `lsp`, `lsp.server`, `lsp.schema`).
+2. **The schema must build without source** — handled by the declarative `PARAMS`/`ACTIONS` above; PyInstaller is the only channel that strips `.py` source, and it no longer matters.
+
+`test/validation/lsp_smoke.py` enforces both per channel: `run.sh` calls it before launching the suite, asserting that `<channel> schema` returns JSON whose `console`/`plot`/`json_rpc` items still carry their actions, and that `<channel> lsp` answers an `initialize` request with capabilities (and never reports the pygls dependency missing). So `./test/validation/run.sh --mode flatpak|pyinstaller|appimage` now fails loudly if a channel ships a broken or pygls-less language server.
 
 ### Version reporting (`interpreter/utils/version.py`)
 
 Both Flatpak and AppImage export `TESTIUM_VERSION` from a launcher (Flatpak: launcher script in `org.testium.Testium.yaml`; AppImage: `runtime.env` in `AppImageBuilder.yml`). `get_testium_version()` checks `/.flatpak-info` / `APPIMAGE` and reads `TESTIUM_VERSION` rather than relying on package metadata or repo introspection.
 
 ## Recent fixes / notable changes
+- LSP across packaging channels: `testium lsp` (and the `testium_assist` editor extension that spawns it) now works from source, wheel, PyInstaller, Flatpak and AppImage. Two enablers — (1) action items declare a class-level `ACTIONS = {key: class}` registry (like `PARAMS`), so `lsp/schema.py` builds the full schema from class attributes with no `inspect.getsource`/AST (which broke under frozen PyInstaller); (2) the `[lsp]` extra (pygls) is wired into every full-app channel. `test/validation/lsp_smoke.py`, run by `run.sh` before the suite, asserts per-channel that `schema` keeps its actions and `lsp` answers `initialize`. See the matching architecture sections.
 - Declarative test item parameters (v0.2): each `TestItem` subclass exposes a `PARAMS = ParamSet(...)` class attribute consumed by the base `__init__`. Catches unknown YAML keys (typo warnings listing the accepted names) and missing required params (load-time errors with `.tum` context). Lays the schema foundation for a future LSP server and auto-generated manual sections. See the matching architecture section.
 - Flatpak: `py_func` / `lua_func` / `run` sub-instance now execute on the host via `flatpak-spawn --host`. The previous attempt to inject host lib dirs into the sandbox's `LD_LIBRARY_PATH` was abandoned — host shared libs are ABI-incompatible with the Flatpak runtime's glibc and would trip `_dl_call_libc_early_init`. The manifest gained `--talk-name=org.freedesktop.Flatpak` so the spawn proxy call is allowed. The testium package is staged once per process under `/tmp` (shared with the host) so the host interpreter can locate `py_func` / `lua_func`.
 - Validation suite: single entry point with `--mode source|wheel|pyinstaller|flatpak|appimage` to validate every packaging channel against the same items. Per-mode report filenames prevent clobbering.

package/appimage/AppImageBuilder.yml

@@ -77,7 +77,10 @@ AppDir:
         python3.11 -m pip install --break-system-packages --upgrade --isolated --no-input --ignore-installed --prefix=$TARGET_APPDIR/usr -r ../../src/requirements.txt
 
         export PIP_CONFIG_FILE=$HOME/.pip/pip.conf
-        python3.11 -m pip install --break-system-packages --upgrade --isolated --no-input --ignore-installed --prefix=$TARGET_APPDIR/usr ../../src/dist/testium-{{APP_VERSION}}-py3-none-any.whl
+        # Install the wheel with the [lsp] extra so `testium lsp` (pygls) works
+        # from the AppImage. The extra pulls pygls/lsprotocol/cattrs/attrs from
+        # the index (network is available at build time, see get-pip above).
+        python3.11 -m pip install --break-system-packages --upgrade --isolated --no-input --ignore-installed --prefix=$TARGET_APPDIR/usr "../../src/dist/testium-{{APP_VERSION}}-py3-none-any.whl[lsp]"
 
 
 AppImage:

package/flatpak/org.testium.Testium.yaml

@@ -28,6 +28,20 @@ build-options:
 
 modules:
   - python3-requirements.json
+
+  # Language-server deps for `testium lsp` (pygls + lsprotocol + cattrs + attrs
+  # + typing_extensions). Installed from PyPI at build time — the build already
+  # runs with --share=network (see build-options). The core runtime deps stay
+  # offline-pinned in python3-requirements.json; these are pure-python wheels,
+  # hence --only-binary=:all: (no compilation, deterministic).
+  - name: python3-lsp
+    buildsystem: simple
+    build-options:
+      build-args:
+        - --share=network
+    build-commands:
+      - pip3 install --prefix=${FLATPAK_DEST} --only-binary=:all: "pygls>=1.3"
+
   # 1. Dépendances Python tierces (HORS PySide6)
   # Utilisez flatpak-pip-generator pour vos autres libs (ex: pyserial, requests, etc.)
   # - name: python3-requirements

package/pyinstaller/testium.spec

@@ -1,5 +1,21 @@
 # -*- mode: python ; coding: utf-8 -*-
 import os
+from PyInstaller.utils.hooks import collect_submodules
+
+# Language-server dependencies for `testium lsp`. pygls/lsprotocol register
+# converters and features dynamically, so we collect their submodules wholesale
+# and force-import their pure-python deps (cattrs/attrs/typing_extensions).
+# The testium lsp modules are imported lazily by the CLI dispatch
+# (`from lsp.server import serve`), which PyInstaller's static analysis misses —
+# hence the explicit names. No source files need bundling: the schema export is
+# now fully declarative (PARAMS + ACTIONS class attributes), so it no longer
+# reads .py source via inspect.getsource (which fails in a frozen build).
+_LSP_HIDDEN = (
+    collect_submodules("pygls")
+    + collect_submodules("lsprotocol")
+    + ["cattrs", "attr", "attrs", "typing_extensions",
+       "lsp", "lsp.server", "lsp.schema"]
+)
 
 # junit_xml is imported by post_exec scripts running under the *host* Python,
 # not the frozen interpreter — so bundling it via hiddenimports alone is not
@@ -54,7 +70,7 @@ a = Analysis(
                    "colorama",
                    "matplotlib",
                    "junit_xml",
-                   "lxml"],
+                   "lxml"] + _LSP_HIDDEN,
     hookspath=[],
     hooksconfig={},
     runtime_hooks=[],

scripts/build_env.sh

@@ -33,6 +33,11 @@ if [ ! -d "$PY_VENV_DIR" ]; then
     python3 -m venv "$PY_VENV_DIR"
     source "$PY_VENV_DIR/bin/activate"
     pip install --extra-index-url https://pypi.python.org/pypi -r $REQ_PATH
+    # Language-server deps (the pyproject [lsp] extra). Installed here so the
+    # source run AND the PyInstaller build — both of which use this venv — can
+    # start / collect the `testium lsp` server. pip-installed wheel users get
+    # them via `pip install testium[lsp]` instead.
+    pip install --extra-index-url https://pypi.python.org/pypi "pygls>=1.3"
     # Validation suite plugin used to verify the report-exporter
     # entry-points discovery end-to-end.
     FAKE_EXPORTER_DIR="$(dirname "$REQ_PATH")/../test/validation/fake_exporter"

src/testium/interpreter/test_items/item_actions/__init__.py

@@ -5,6 +5,13 @@ from interpreter.test_items.item_actions.action import TestItemAction
 
 
 class TestItemActions(TestItem):
+    # Declarative action registry: subclasses set ``ACTIONS = {yaml_key: class}``
+    # as a class attribute (mirroring ``PARAMS``). It is read here to populate
+    # the runtime registry, and read identically by the schema export — no
+    # instantiation or source inspection required. ``register_actions()`` stays
+    # available as an imperative escape hatch for dynamic/conditional cases.
+    ACTIONS = {}
+
     def __init__(
         self, item_type, dict_actions, parent=None, status_queue=None, filename=""
     ):
@@ -12,7 +19,7 @@ class TestItemActions(TestItem):
         super().__init__(dict_actions, parent, status_queue, filename=filename)
         self._type = item_type
         self.is_container = False
-        self.action_classes = {}
+        self.action_classes = dict(type(self).ACTIONS)
         self.actions_token = None
         self.actions = []
         try:
@@ -24,6 +31,9 @@ class TestItemActions(TestItem):
             )
 
     def register_actions(self, **args: TestItemAction):
+        # Imperative escape hatch. The declarative ``ACTIONS`` class attribute
+        # covers every current subclass; use this only to add actions that
+        # can't be known at class-definition time (e.g. platform-conditional).
         for action_name, action_class in args.items():
             self.action_classes.update({action_name: action_class})
 

src/testium/interpreter/test_items/test_item_console.py

@@ -388,18 +388,19 @@ class TestItemConsole(TestItemActions):
                   "as long as their names differ."),
     )
 
+    ACTIONS = {
+        "open": TestItemConsoleOpen,
+        "close": TestItemConsoleClose,
+        "write": TestItemConsoleWrite,
+        "writeln": TestItemConsoleWriteLn,
+        "read_until": TestItemConsoleReadUntil,
+    }
+
     def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
         super().__init__(
             cst.TYPE_CONSOLE, dict_item, parent, status_queue, filename=filename
         )
 
-        self.register_actions(
-            open=TestItemConsoleOpen,
-            close=TestItemConsoleClose,
-            write=TestItemConsoleWrite,
-            writeln=TestItemConsoleWriteLn,
-            read_until=TestItemConsoleReadUntil,
-        )
         self.actions_token = {}
 
         global console

src/testium/interpreter/test_items/test_item_json_rpc/__init__.py

@@ -210,6 +210,13 @@ class TestItemJSON_RPC(TestItemActions):
               doc="If true, don't echo wire traffic to the log."),
     )
 
+    ACTIONS = {
+        "open": TestItemJSRPCActionOpen,
+        "close": TestItemJSRPCActionClose,
+        "query": TestItemJSRPCActionQuery,
+        "receive": TestItemJSRPCActionReceive,
+    }
+
     def __init__(
         self, dict_item: dict, parent: TestItem = None, status_queue=None, filename=""
     ):
@@ -217,13 +224,6 @@ class TestItemJSON_RPC(TestItemActions):
             cst.TYPE_JSON_RPC, dict_item, parent, status_queue, filename=filename
         )
 
-        self.register_actions(
-            open=TestItemJSRPCActionOpen,
-            close=TestItemJSRPCActionClose,
-            query=TestItemJSRPCActionQuery,
-            receive=TestItemJSRPCActionReceive,
-        )
-
         # Console specific params
         self._console = self._prms.getParam("console", required=False)
         # UDP specific params

src/testium/interpreter/test_items/test_item_runtime_plot.py

@@ -263,18 +263,18 @@ class TestItemPlot(TestItemActions):
                   "action and by $(plv_<plot_name>) for last-values output."),
     )
 
+    ACTIONS = {
+        "open": TestItemPlotActionOpen,
+        "close": TestItemPlotActionClose,
+        "periodic": TestItemPlotActionPeriodic,
+        "add": TestItemPlotActionAdd,
+        "last_value": TestItemPlotActionLastValues,
+        "export": TestItemPlotActionExport,
+    }
+
     def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
         super().__init__(
             cst.TYPE_GRAPH, dict_item, parent, status_queue, filename=filename
         )
 
-        self.register_actions(
-            open=TestItemPlotActionOpen,
-            close=TestItemPlotActionClose,
-            periodic=TestItemPlotActionPeriodic,
-            add=TestItemPlotActionAdd,
-            last_value=TestItemPlotActionLastValues,
-            export=TestItemPlotActionExport,
-        )
-
         self.actions_token = self._prms.getParam("plot_name", required=True)

src/testium/lsp/schema.py

@@ -24,41 +24,17 @@ from interpreter.utils.test_init import _constants_init
 
 # Action class -> parent cmd (the action's parent in the YAML). Action classes
 # aren't first-class TestItemType entries (TYPE_*_ACTION is one generic bucket),
-# so we resolve their YAML key by looking at how each parent registers them.
+# so we resolve their YAML key from the parent's declarative ``ACTIONS`` map.
 def _collect_action_classes(parent_class):
     """Return {action_yaml_key: action_class} for a TestItemActions parent.
 
-    Each parent's ``__init__`` calls ``self.register_actions(name=class, ...)``
+    Each parent declares its actions as a class-level ``ACTIONS = {key: class}``
-    *during* construction, so we can't read the registry without instantiating
+    attribute (see ``item_actions/TestItemActions``). We read it directly — no
-    one. We work around it by parsing the source for the registration call —
+    instantiation, no source inspection — so this works identically whether the
-    cheap, no side effects, and the schema export is a CLI command anyway.
+    package runs from source, a wheel, or a frozen (PyInstaller) build where the
+    ``.py`` source isn't on disk.
     """
-    import ast
+    return dict(getattr(parent_class, "ACTIONS", None) or {})
-    import inspect
-
-    try:
-        src = inspect.getsource(parent_class)
-    except (OSError, TypeError):
-        return {}
-
-    actions = {}
-    tree = ast.parse(src)
-    for node in ast.walk(tree):
-        if not isinstance(node, ast.Call):
-            continue
-        fn = node.func
-        if not (isinstance(fn, ast.Attribute) and fn.attr == "register_actions"):
-            continue
-        for kw in node.keywords:
-            if kw.arg is None or not isinstance(kw.value, ast.Name):
-                continue
-            # ast.Name gives us only the bare identifier; resolve it through
-            # the parent class's defining module.
-            mod = __import__(parent_class.__module__, fromlist=[kw.value.id])
-            cls = getattr(mod, kw.value.id, None)
-            if cls is not None:
-                actions[kw.arg] = cls
-    return actions
 
 
 def _params_to_schema(item_class, common_params):

test/validation/lsp_smoke.py

@@ -0,0 +1,102 @@
+#!/usr/bin/env python3
+"""Per-channel smoke test for the testium language server.
+
+Given the channel's testium invocation as argv (e.g. ``flatpak run
+--command=testium org.testium.Testium``, a PyInstaller binary path, or
+``python -m testium``), verify two things end-to-end against that exact build:
+
+  1. ``<cmd> schema`` produces valid JSON whose item registry still includes the
+     nested action sets (console/plot/json_rpc). This catches a frozen build
+     that lost the actions — the failure mode the declarative ``ACTIONS``
+     refactor fixed (no more ``inspect.getsource`` at runtime).
+  2. ``<cmd> lsp`` starts a real language server: it must answer an LSP
+     ``initialize`` request with a capabilities result and must NOT report the
+     pygls dependency as missing. This catches a channel that forgot to bundle
+     the ``[lsp]`` extra.
+
+Exits non-zero (with a diagnostic) on the first failure so the validation run
+fails loudly. Used by ``run.sh`` before launching the main suite.
+"""
+import json
+import subprocess
+import sys
+
+EXPECTED_ACTION_PARENTS = ("console", "plot", "json_rpc")
+
+
+def fail(msg):
+    print(f"LSP SMOKE: FAIL — {msg}", file=sys.stderr)
+    sys.exit(1)
+
+
+def _extract_json(raw):
+    """Parse JSON from ``raw`` bytes, tolerating leading non-JSON noise.
+
+    The source-mode launcher (run.sh) may print env-setup lines before the
+    schema JSON, so we fall back to parsing from the first ``{``.
+    """
+    try:
+        return json.loads(raw)
+    except json.JSONDecodeError:
+        start = raw.find(b"{")
+        if start < 0:
+            raise
+        return json.loads(raw[start:])
+
+
+def check_schema(cmd):
+    try:
+        out = subprocess.run(cmd + ["schema"], capture_output=True, timeout=120)
+    except Exception as e:  # noqa: BLE001
+        fail(f"`{' '.join(cmd)} schema` could not run: {e}")
+    if out.returncode != 0:
+        fail(f"`schema` exited {out.returncode}: {out.stderr.decode()[:300]}")
+    try:
+        data = _extract_json(out.stdout)
+    except json.JSONDecodeError as e:
+        fail(f"`schema` output is not valid JSON: {e}")
+    items = data.get("items", {})
+    for parent in EXPECTED_ACTION_PARENTS:
+        actions = (items.get(parent) or {}).get("actions") or {}
+        if not actions:
+            fail(f"schema item '{parent}' has no actions — a frozen build lost "
+                 f"the declarative ACTIONS (item keys: {sorted(items)[:8]}…)")
+    print(f"LSP SMOKE: schema OK ({len(items)} items; actions present for "
+          f"{', '.join(EXPECTED_ACTION_PARENTS)})")
+
+
+def check_lsp(cmd):
+    body = json.dumps({
+        "jsonrpc": "2.0", "id": 1, "method": "initialize",
+        "params": {"processId": None, "rootUri": None, "capabilities": {}},
+    }).encode()
+    msg = b"Content-Length: %d\r\n\r\n%s" % (len(body), body)
+    try:
+        out = subprocess.run(cmd + ["lsp"], input=msg,
+                             capture_output=True, timeout=30)
+        stdout, stderr = out.stdout, out.stderr
+    except subprocess.TimeoutExpired as e:
+        # A server that stays alive past initialize is fine — it just never saw
+        # a shutdown. Use whatever it wrote so far as the response.
+        stdout, stderr = e.stdout or b"", e.stderr or b""
+    blob = stdout + stderr
+    if b"dependencies missing" in blob:
+        fail("`lsp` reports the pygls dependency missing — this channel did "
+             "not bundle the [lsp] extra.")
+    if b'"capabilities"' not in stdout:
+        fail("`lsp` did not return an initialize result. "
+             f"stdout[:200]={stdout[:200]!r} stderr[:200]={stderr[:200]!r}")
+    print("LSP SMOKE: lsp initialize OK (server answered with capabilities)")
+
+
+def main():
+    cmd = sys.argv[1:]
+    if not cmd:
+        fail("usage: lsp_smoke.py <testium-invocation...>")
+    check_schema(cmd)
+    check_lsp(cmd)
+    print("LSP SMOKE: PASS")
+
+
+if __name__ == "__main__":
+    main()

test/validation/run.sh

@@ -137,6 +137,13 @@ esac
 echo "-- validation mode: $MODE"
 echo "-- launch: ${CMD[*]}"
 
+# ---------- LSP smoke check (this exact channel) ------------------------------
+# Verify `testium lsp` / `testium schema` work in the build under test before
+# running the suite: schema must keep its nested actions (declarative ACTIONS,
+# survives frozen builds) and the language server must start (pygls bundled).
+echo "-- LSP smoke check ($MODE)"
+"$VENV_PYTHON" "$SCRIPT_DIR/lsp_smoke.py" "${CMD[@]}"
+
 exec "${CMD[@]}" -b \
     -d "python_bin=$VENV_PYTHON" \
     -d "validation_report_file=validation-$MODE" \