feat(windows): icon, windowed exe, no-admin installer

- PyInstaller exe built windowed (console=False) with package/testium.ico
  as the embedded icon (BMP entries for shell compatibility).
- Suppress stray subprocess console windows in the frozen Windows build via
  paths.no_window_kwargs() (CREATE_NO_WINDOW); wheel/source unchanged.
  Applied to py_process, lua_process, bins probes, sys_app_path_win.
- New per-user Inno Setup installer (package/innosetup/): no admin,
  version-scoped AppId/dir so versions install side-by-side, one Start
  Menu entry per version, .ico shipped for shortcut/uninstall icons.
- DESIGN.md + release_note.txt updated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

DESIGN.md

@@ -97,6 +97,15 @@ All dialog items (`dialog_image`, `dialog_question`, `dialog_references`, `dialo
 - For the live stream (terminal in batch / GUI panel), prefixes every line emitted from a branch's thread with `[<branch_name>] ` so concurrent branches stay readable.
 - Exposes `write` / `writeln` / `flush` (Python 3.14's `unittest` calls `stream.writeln()` directly without `_WritelnDecorator`).
 
+### Subprocess RPC startup handshake (py_func / lua_func / eval_proc)
+
+The parent ↔ subprocess JSON-RPC link runs over a localhost TCP socket. The **subprocess** owns the port: it binds `port 0` (OS-assigned), `listen()`s, then prints `__TESTIUM_RPC_PORT__=<port>` on stdout (constant `RPC_PORT_SENTINEL` in `runtime/jrpc.py`). The parent reads that line (`proc_drain.drain_and_read_port` + `wait_for_port`, deadline `gd("proc_start_timeout", 30)`) and only *then* connects — the server is guaranteed to be listening, so the connect succeeds on the first attempt.
+
+This replaced the previous fragile scheme (parent reserved a port via `bind(0)`+close, child re-bound the same port, parent connected on a timing guess) which broke intermittently on Windows: cold-start/antivirus variance pushed the worker past the connect deadline, and `connect()` to a not-yet-listening localhost port *times out* (≈1 s) instead of refusing, exhausting the retry budget. Notes:
+- The server no longer sets `SO_REUSEADDR` (a fresh ephemeral port needs no TIME_WAIT override; on Windows it would enable port hijacking).
+- `JsonRpcBase.wait_ready()` always settles (event set on success **and** failure) and returns the actual connection outcome — a connect failure no longer hangs a `wait_ready()` caller.
+- Non-sentinel subprocess stdout/stderr is still forwarded to the parent log (early-startup errors stay visible).
+
 ### Subprocess API contract (py_func / lua_func)
 
 User test scripts running inside a `py_func` or `lua_func` subprocess **must** use the JSON-RPC bridge to interact with testium state:
@@ -115,7 +124,7 @@ To add a new API call usable from subprocesses:
 `src/testium/interpreter/utils/bins.py` — single source of truth for the paths to the external Python and Lua interpreters used by subprocesses.
 
 - `python_bin()` / `lua_bin()` : resolve and cache. The cache is keyed by `(name, override)` so that a later change to `gd[python_bin]` (typically when a `param.yaml` sets the key) triggers a re-resolution on the next lookup instead of returning the stale auto-discovered path. Falls back to discovery on PATH (candidates: `python3`/`python` and `lua`/`lua5.5`/`lua5.4`/`lua5.3`/`lua5.2`/`lua5.1`).
-- `ensure(*names)` : called by `TestSet._validate_runtime_deps()` at test load. Always requires `python` (the eval engine always runs); requires `lua` only if a `lua_func` item is in the tree. Fails fast with a clear error citing tried candidates and override key.
+- `ensure(*names)` : called by `TestSet._validate_runtime_deps()` at test load. Always requires `python` (the eval engine always runs); requires `lua` only if a `lua_func` item is in the tree. Fails fast with a clear error citing tried candidates and override key. Also **publishes** each resolved path into gd (`python_bin` / `lua_bin`) when the key is unset, so test scripts can reference `$(python_bin)` / `$(lua_bin)` regardless of launch mode (e.g. GUI, where no `-d` override is passed). A user-provided value is never overwritten.
 
 Engines (`PyProcessBase`, `LuaProcessBase`, `EvalExecEngine`) call `bins.python_bin()`/`bins.lua_bin()` themselves — call sites never pass an explicit binary path.
 
@@ -217,7 +226,8 @@ Four distribution channels coexist, all sharing the single `src/testium/` packag
 | Channel | Where | Build | Notes |
 |---------|-------|-------|-------|
 | Wheel (`pip install`) | `src/pyproject.toml` | `python -m build` | Vanilla Python package; entry point `testium = "testium:main"`. |
-| PyInstaller binary | `package/pyinstaller/` | `build.sh` | Single ~130 MB binary. `py_func`, `runtime`, `lua_func` bundled at `_MEIPASS` root so the **host** Python can find them when launched as `python3 py_func`. `api`/`interpreter` are **not** exposed (subprocess isolation). |
+| PyInstaller binary | `package/pyinstaller/` | `build.sh` | Single ~130 MB binary. `py_func`, `runtime`, `lua_func` bundled at `_MEIPASS` root so the **host** Python can find them when launched as `python3 py_func`. `api`/`interpreter` are **not** exposed (subprocess isolation). Built windowed (`console=False`) with `package/testium.ico` as the exe icon — see "Windows frozen build". |
+| Windows installer | `package/innosetup/` | `build.ps1` (Inno Setup 6) | Wraps the PyInstaller exe. Per-user, **no admin** (`PrivilegesRequired=lowest`, installs under `%LOCALAPPDATA%`). Version-scoped `AppId` + install dir so versions coexist side-by-side; one Start Menu entry per version. |
 | Flatpak | `package/flatpak/` | `build.sh` (uses `flatpak-builder`) | KDE 6.10 runtime. The bundled Python runs only the main process; `py_func` / `lua_func` MUST run under the **host** interpreter (no Python/Lua bundled). Produces a distributable `.flatpak` bundle. |
 | AppImage | `package/appimage/` | `build.sh` (Debian Bookworm container via Podman/Docker) | Bundles Python 3.11 for the main process; `py_func` / `lua_func` MUST run under the **host** interpreter. Build runs in a container so it works on Arch / any non-Debian host. |
 
@@ -248,6 +258,19 @@ The bundled Python (Flatpak's runtime python, AppImage's `python3.11`) is reserv
 - If the host has no python3/lua, `ensure()` raises `ETUMRuntimeError` at test load with the candidate list — no silent fallback to a bundled interpreter.
 - `py_process.py` additionally pops `PYTHONUSERBASE` (set to `/var/data/python` by the Flatpak runtime, which would hide `~/.local/lib/...`).
 
+### Windows frozen build: no console, hidden subprocess windows
+
+The PyInstaller exe is built **windowed** (`console=False` in `testium.spec`) so a
+double-click doesn't open a console. The catch: a windowed process has **no console
+to inherit**, so every console subprocess it spawns (the `py_func`/`lua_func` host
+Python bridges — the otherwise-permanent window — plus the `where`/`which`/`--version`
+probes) opens its **own** console window. `paths.no_window_kwargs()` returns
+`{"creationflags": CREATE_NO_WINDOW}` on a frozen Windows build (and `{}` everywhere
+else, so the **wheel/source** keeps its console and child output stays visible). It is
+applied at every spawn site: `py_process.py`, `lua_process.py`, `bins._run_probe`,
+`paths.sys_app_path_win`. `termconsole.py` is intentionally exempt (it already hides
+`cmd.exe` via `STARTUPINFO`).
+
 ### Declarative test item parameters
 
 Each `TestItem` subclass declares its accepted parameters as a class attribute `PARAMS = ParamSet(Param(...), ...)` (`interpreter/utils/param_decl.py`). The descriptor carries the parameter name, *kind* (`SCALAR` — the default and may be omitted; `LIST`; `BLOCK`; `Enum("a", "b", ...)`), `required` flag, `default`, and free-form `doc`. There is **no Python type** in the descriptor on purpose: most parameter values are expressions (`$(...)` / `<| ... |>`) whose effective type is only known after expansion, so a static type would be misleading. Post-expansion `validate=lambda v: ...` callbacks are available as an opt-in for the rare cases where a runtime check is warranted (e.g. a specific format).
@@ -279,6 +302,7 @@ The `testium_assist` editor extension is a thin LSP client that spawns `testium
 Both Flatpak and AppImage export `TESTIUM_VERSION` from a launcher (Flatpak: launcher script in `org.testium.Testium.yaml`; AppImage: `runtime.env` in `AppImageBuilder.yml`). `get_testium_version()` checks `/.flatpak-info` / `APPIMAGE` and reads `TESTIUM_VERSION` rather than relying on package metadata or repo introspection.
 
 ## Recent fixes / notable changes
+- Subprocess RPC startup handshake: the `py_func`/`lua_func`/`eval_proc` worker now picks its own port (`bind 0`), announces it on stdout (`__TESTIUM_RPC_PORT__=`), and the parent connects only after reading it. Fixes intermittent Windows `failed to connect : timeout` and the matching `wait_ready()` hang; removes the reserve/close/rebind race and `SO_REUSEADDR`. See "Subprocess RPC startup handshake".
 - `build_all.sh`: builds the four heavy channels in parallel (serial prep for the shared venv + wheel), results in completion order, Ctrl+C kills the whole job tree; `--ram` puts the build scratch on tmpfs (`/dev/shm`) + skips UPX for fast builds on USB/SD storage (Flatpak excluded — rofiles-fuse can't mount tmpfs). See the "Building all channels" section.
 - LSP across packaging channels: `testium lsp` (and the `testium_assist` editor extension that spawns it) now works from source, wheel, PyInstaller, Flatpak and AppImage. Two enablers — (1) action items declare a class-level `ACTIONS = {key: class}` registry (like `PARAMS`), so `lsp/schema.py` builds the full schema from class attributes with no `inspect.getsource`/AST (which broke under frozen PyInstaller); (2) the `[lsp]` extra (pygls) is wired into every full-app channel. `test/validation/lsp_check.py`, run by `run.sh` before the suite, asserts per-channel that `schema` keeps its actions and `lsp` answers `initialize`. See the matching architecture sections.
 - Declarative test item parameters (v0.2): each `TestItem` subclass exposes a `PARAMS = ParamSet(...)` class attribute consumed by the base `__init__`. Catches unknown YAML keys (typo warnings listing the accepted names) and missing required params (load-time errors with `.tum` context). Lays the schema foundation for a future LSP server and auto-generated manual sections. See the matching architecture section.

README.md

@@ -129,6 +129,22 @@ A VSCode / VSCodium client extension (`testium_assist`) wraps `testium lsp`;
 the schema is built from testium itself, so new item types and parameters
 appear in the editor on the next testium upgrade with no client change.
 
+It is published on [Open VSX](https://open-vsx.org/extension/testium/testium-assist),
+so in **VSCodium, Cursor, Windsurf, Theia and code-server** it installs from the
+Extensions view (search `testium-assist`) or with
+`codium --install-extension testium.testium-assist`.
+
+**Microsoft VSCode** does not list Open VSX extensions, so install the `.vsix`
+by hand — download it from the Open VSX page above, then *Extensions → ⋯ →
+Install from VSIX…* or:
+
+```sh
+code --install-extension testium-assist-0.1.0.vsix
+```
+
+The extension runs `testium lsp`, so `testium` must be on the `PATH` (otherwise
+point the `testium.serverPath` setting at the binary/AppImage).
+
 ## Troubleshooting
 
 ### `wl_proxy_marshal_flags` symbol error

doc/manual/sphinx/source/modes.rst

@@ -67,3 +67,36 @@ dependencies:
     :caption: enable the language server for a wheel / source install
 
     pip install 'testium[lsp]'
+
+Installing the VSCode / VSCodium extension
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The *testium_assist* client extension is published on `Open VSX
+<https://open-vsx.org/extension/testium/testium-assist>`_, the registry used by
+VSCodium, Cursor, Windsurf, Eclipse Theia and code-server. In those editors,
+open the Extensions view and search ``testium-assist``, or install it from the
+command line:
+
+.. code-block:: text
+    :caption: install in VSCodium and other Open VSX editors
+
+    codium --install-extension testium.testium-assist
+
+Microsoft *VSCode* uses a different marketplace that does not list Open VSX
+extensions, so install the packaged ``.vsix`` by hand. Download it from the
+Open VSX page linked above, then either choose *Extensions* → *⋯* →
+*Install from VSIX…* in the UI, or run:
+
+.. code-block:: text
+    :caption: install the .vsix in Microsoft VSCode
+
+    code --install-extension testium-assist-0.1.0.vsix
+
+The extension launches ``testium lsp``, so the ``testium`` command must be on
+the ``PATH``. If *testium* is installed elsewhere — a specific binary or an
+AppImage — point the ``testium.serverPath`` setting at it instead.
+
+Once installed, open a ``.tum`` file: completion of item types, hover
+documentation and the outline view become available. If nothing happens, check
+that no ``files.associations`` entry forces ``*.tum`` to another language (it
+must stay the ``tum`` language the extension provides).

package/innosetup/build.ps1

@@ -0,0 +1,31 @@
+# Build the Testium installer from testium.iss (needs Inno Setup 6 / ISCC.exe).
+# Install ISCC without admin: winget install --id JRSoftware.InnoSetup -e
+
+$ErrorActionPreference = 'Stop'
+$scriptDir = Split-Path -Parent $MyInvocation.MyCommand.Path
+
+# The PyInstaller exe must exist first.
+$exe = Join-Path $scriptDir '..\pyinstaller\dist\testium.exe'
+if (-not (Test-Path $exe)) {
+    throw "PyInstaller build not found: $exe`nRun package\pyinstaller\build first."
+}
+
+# Locate ISCC.exe: PATH, then the usual install dirs.
+$iscc = (Get-Command ISCC.exe -ErrorAction SilentlyContinue).Source
+if (-not $iscc) {
+    foreach ($p in @(
+        "$env:LOCALAPPDATA\Programs\Inno Setup 6\ISCC.exe",
+        "${env:ProgramFiles(x86)}\Inno Setup 6\ISCC.exe",
+        "$env:ProgramFiles\Inno Setup 6\ISCC.exe")) {
+        if (Test-Path $p) { $iscc = $p; break }
+    }
+}
+if (-not $iscc) {
+    throw "ISCC.exe not found. Install Inno Setup 6:`n    winget install --id JRSoftware.InnoSetup -e"
+}
+
+Write-Host "Using ISCC: $iscc"
+& $iscc (Join-Path $scriptDir 'testium.iss')
+if ($LASTEXITCODE -ne 0) { throw "ISCC failed with exit code $LASTEXITCODE" }
+
+Write-Host "`nInstaller built in: $(Join-Path $scriptDir 'dist')"

package/innosetup/testium.iss

@@ -0,0 +1,127 @@
+; Inno Setup script: wraps the PyInstaller testium.exe into a per-user installer.
+; Build with Inno Setup 6: ISCC.exe testium.iss (or ./build.ps1).
+
+#define MyAppName "Testium"
+#define MyAppExeName "testium.exe"
+#define MyAppPublisher "Testium"
+#define MyAppURL "https://github.com/"
+
+; Read version from src/VERSION so the installer never drifts from the build.
+#define VerFile FileOpen("..\..\src\VERSION")
+#define MyAppVersion Trim(FileRead(VerFile))
+#expr FileClose(VerFile)
+#if MyAppVersion == ""
+  #error Could not read version from ..\..\src\VERSION
+#endif
+
+[Setup]
+; Version-scoped AppId: each version is a distinct app, installable side-by-side.
+AppId={{B7E6F1C2-9A4D-4E3B-8F71-7C2D5A6E0B14}_{#MyAppVersion}
+AppName={#MyAppName} {#MyAppVersion}
+AppVerName={#MyAppName} {#MyAppVersion}
+AppVersion={#MyAppVersion}
+AppPublisher={#MyAppPublisher}
+AppPublisherURL={#MyAppURL}
+UninstallDisplayName={#MyAppName} {#MyAppVersion}
+WizardStyle=modern
+; Per-version install dir so versions never overwrite each other.
+DefaultDirName={autopf}\{#MyAppName}\{#MyAppVersion}
+; Shared "Testium" Start Menu folder; shortcuts below are named per version.
+DefaultGroupName={#MyAppName}
+UninstallDisplayIcon={app}\testium.ico
+DisableProgramGroupPage=yes
+; Per-user install, no admin ever: installs under %LOCALAPPDATA%, no UAC prompt.
+PrivilegesRequired=lowest
+ArchitecturesInstallIn64BitMode=x64compatible
+OutputDir=dist
+OutputBaseFilename=testium-{#MyAppVersion}-setup
+SetupIconFile=..\testium.ico
+Compression=lzma2/max
+SolidCompression=yes
+; Tell Explorer to refresh the environment after a PATH change.
+ChangesEnvironment=yes
+
+[Languages]
+Name: "french"; MessagesFile: "compiler:Languages\French.isl"
+Name: "english"; MessagesFile: "compiler:Default.isl"
+
+[Tasks]
+Name: "desktopicon"; Description: "{cm:CreateDesktopIcon}"; GroupDescription: "{cm:AdditionalIcons}"; Flags: unchecked
+; PATH off by default: the exe is windowed (console=False), so CLI shows no output.
+Name: "addtopath"; Description: "Ajouter Testium au PATH (usage en ligne de commande)"; Flags: unchecked
+
+[Files]
+Source: "..\pyinstaller\dist\{#MyAppExeName}"; DestDir: "{app}"; Flags: ignoreversion
+; Ship the .ico so shortcuts/uninstall reference it directly, not the embedded one.
+Source: "..\testium.ico"; DestDir: "{app}"; Flags: ignoreversion
+
+[Icons]
+; Per-version names so each install shows separately in the Start Menu.
+Name: "{group}\{#MyAppName} {#MyAppVersion}"; Filename: "{app}\{#MyAppExeName}"; IconFilename: "{app}\testium.ico"
+Name: "{group}\{cm:UninstallProgram,{#MyAppName} {#MyAppVersion}}"; Filename: "{uninstallexe}"
+Name: "{autodesktop}\{#MyAppName} {#MyAppVersion}"; Filename: "{app}\{#MyAppExeName}"; IconFilename: "{app}\testium.ico"; Tasks: desktopicon
+
+[Run]
+Filename: "{app}\{#MyAppExeName}"; Description: "{cm:LaunchProgram,{#MyAppName}}"; Flags: nowait postinstall skipifsilent
+
+[Code]
+const
+  EnvKey = 'Environment';
+
+// True if Param is not already a full segment of the per-user PATH.
+function NeedsAddPath(Param: string): Boolean;
+var
+  OrigPath: string;
+begin
+  if not RegQueryStringValue(HKEY_CURRENT_USER, EnvKey, 'Path', OrigPath) then
+  begin
+    Result := True;
+    exit;
+  end;
+  Result := Pos(';' + Uppercase(Param) + ';', ';' + Uppercase(OrigPath) + ';') = 0;
+end;
+
+// On install: append {app} to the per-user PATH if the task is selected.
+procedure CurStepChanged(CurStep: TSetupStep);
+var
+  Path: string;
+begin
+  if CurStep = ssPostInstall then
+  begin
+    if WizardIsTaskSelected('addtopath') and NeedsAddPath(ExpandConstant('{app}')) then
+    begin
+      if not RegQueryStringValue(HKEY_CURRENT_USER, EnvKey, 'Path', Path) then
+        Path := '';
+      if (Path <> '') and (Copy(Path, Length(Path), 1) <> ';') then
+        Path := Path + ';';
+      Path := Path + ExpandConstant('{app}');
+      RegWriteStringValue(HKEY_CURRENT_USER, EnvKey, 'Path', Path);
+    end;
+  end;
+end;
+
+// On uninstall: strip {app} back out of the per-user PATH.
+procedure CurUninstallStepChanged(CurUninstallStep: TUninstallStep);
+var
+  Path, AppDir, Segment: string;
+  P: Integer;
+begin
+  if CurUninstallStep = usUninstall then
+  begin
+    if RegQueryStringValue(HKEY_CURRENT_USER, EnvKey, 'Path', Path) then
+    begin
+      AppDir := ExpandConstant('{app}');
+      Segment := ';' + AppDir;
+      P := Pos(Uppercase(Segment), Uppercase(Path));
+      if P > 0 then
+        Delete(Path, P, Length(Segment))
+      else
+      begin
+        P := Pos(Uppercase(AppDir) + ';', Uppercase(Path));
+        if P = 1 then
+          Delete(Path, 1, Length(AppDir) + 1);
+      end;
+      RegWriteStringValue(HKEY_CURRENT_USER, EnvKey, 'Path', Path);
+    end;
+  end;
+end;

package/pyinstaller/testium.spec

@@ -94,11 +94,11 @@ exe = EXE(
     upx=not os.environ.get("TESTIUM_NO_UPX"),
     upx_exclude=[],
     runtime_tmpdir=None,
-    console=True,
+    console=False,
     disable_windowed_traceback=False,
     argv_emulation=False,
     target_arch=None,
     codesign_identity=None,
     entitlements_file=None,
-    ico='../testium.png'
+    ico='../testium.ico'
 )

release_note.txt

@@ -1,3 +1,33 @@
+version 0.2.3
+=============
+- Windows version now working reliably. Fix of a problem of jrpc ports
+  handshakes between the py and lua processes and testium.
+  Beneficial to linux version too.
+- Windows: UTF-8 console output and a self-sufficient validation
+  wrapper (run.bat).
+- Resolved python_bin / lua_bin are now published into the global dict,
+  so test scripts can read them via $(python_bin) / $(lua_bin).
+- Windows: new per-user installer (no admin).
+
+version 0.2.2
+==============
+- Flatpak sandbox issue fixed for term console. Now a term console is
+  exactly like a host console.
+- Persistence fix of dialogs in case of flatpak.
+
+version 0.2.1
+==============
+- Faster test loading, especially for large tests built from jinja
+  templates and ``!include``: compiled jinja templates are cached and
+  reused (a file included many times is compiled once), rendering happens
+  in memory instead of through a temporary file, and YAML is parsed with
+  the libyaml C loader when available. Typical load time is 3-6x lower on
+  include / template-heavy tests; behaviour is unchanged.
+- Fix: a nested list holding more than one step under ``steps`` no longer
+  duplicates its entries while the step tree is built.
+- New load-time benchmark under ``test/benchmark/`` (synthetic-tree
+  generator + in-process timing harness) to measure the load pipeline.
+
 version 0.2
 ==============
 - Test items: each item type now declares its accepted parameters

src/VERSION

@@ -1 +1 @@
-0.2
+0.2.3

src/testium/__init__.py

@@ -11,6 +11,16 @@ sys.path.append(os.path.abspath(ourpath.parent))
 import interpreter.utils.constants as cst
 
 def main():
+    # Force UTF-8 on stdout/stderr so the runner's output survives a legacy
+    # console code page (Windows cp1252 can't encode box-drawing/accented
+    # chars). Only the stream encoders change; the locale default used for
+    # config files is untouched.
+    for _stream in (sys.stdout, sys.stderr):
+        try:
+            _stream.reconfigure(encoding="utf-8")
+        except (AttributeError, ValueError):
+            pass  # no stdout (frozen GUI) or non-reconfigurable stream
+
     # Subcommand dispatch (must run *before* argparse so neither 'schema' nor
     # 'lsp' has to share the GUI/batch flag surface). The subcommands also
     # skip the multiprocessing 'spawn' setup which is only meaningful for the

src/testium/api/termconsole.py

@@ -81,9 +81,13 @@ class TermConsole(Console):
                                    bufsize=0)
 
         else:
-            self.term = pexpect.spawn(  shell_cmd,
-                                        echo=False,
-                                        cwd=self.ppath)
+            # In Flatpak this returns a `flatpak-spawn --host` wrapper so the
+            # console behaves like a host shell (matching py_func / lua_func /
+            # run); elsewhere it's the chosen command unchanged.
+            from interpreter.utils import bins
+            argv = bins.host_console_command(shell_cmd, self.ppath)
+            self.term = pexpect.spawn(argv[0], args=argv[1:],
+                                      echo=False, cwd=self.ppath)
 
         self.q = BytesStore()
         self.t = threading.Thread(target=self.enqueue_output)

src/testium/interpreter/test_items/dialog_choices_files/choices_dialog.py

@@ -221,6 +221,11 @@ def main(args, conn=None):
 
     if conn:
         settings.setValue(SettingsLastChoices, result)
+        # Flush before sending: the parent terminates this subprocess as soon
+        # as it reads the result, so the QSettings destructor never runs and
+        # the write would race the kill (lost under Flatpak — see the
+        # tested-references dialog for the full rationale).
+        settings.sync()
         conn.send([result, success])
         conn.close()
     else:

src/testium/interpreter/test_items/test_item_py_func.py

@@ -90,7 +90,7 @@ class TestItemPyFunc(TestItem):
 
             if not engine.is_alive():
                 engine.start()
-                if not engine.wait_ready():
+                if not engine.wait_ready(10):
                     raise ETUMRuntimeError(
                         f"""Impossible to start the external python execution process.
 Is the python path correct ?

src/testium/interpreter/test_items/tested_references_files/tested_refs_dialog.py

@@ -76,6 +76,12 @@ def main(args, conn=None):
 
     if conn:
         settings.setValue(SettingsLastReference, result)
+        # Flush to disk *before* handing the result back: as soon as the parent
+        # receives it on the pipe it terminates this subprocess (SIGTERM, no
+        # handler), so the QSettings destructor never runs. Without sync() the
+        # write races the kill and is lost — reliably so under Flatpak, where
+        # the .conf is atomically renamed on the slower ~/.var/app overlay.
+        settings.sync()
         conn.send([result, success])
         conn.close()
     else:

src/testium/interpreter/test_report/report_export_html.py

@@ -14,7 +14,7 @@ class ReportExportHTML(rpe.ReportExport):
         self.prepareFile()
         self.create_base()
         self.process_tests()
-        with open(self._file_name, 'w') as f:
+        with open(self._file_name, 'w', encoding="utf-8") as f:
             f.write(lxml.html.tostring(self.root, pretty_print=True).decode())
 
     def testsIterate(self, row):

src/testium/interpreter/test_report/report_export_junit.py

@@ -20,7 +20,7 @@ class ReportExportJUnit(rpe.ReportExport):
 
         ts = TestSuite(repname, test_cases=self.test_cases,
                        hostname=tm.gd('host_ip'))
-        with open(self._file_name, 'w') as f:
+        with open(self._file_name, 'w', encoding="utf-8") as f:
             TestSuite.to_file(f, [ts])
 
     def testsIterate(self, row):

src/testium/interpreter/test_set.py

@@ -29,6 +29,51 @@ def _build_item_path(item) -> str:
     return " > ".join(reversed(parts))
 
 
+def _flatten_actions(actions, out, parent_seq_name):
+    """Expand nested lists and included ``sequence`` entries into ``out`` as a
+    flat list of single test-item dicts, propagating each sequence's source
+    filename onto its items.
+
+    Replaces the previous approach, which spliced each entry into the step
+    list and rebuilt the whole list every time (O(n^2) over the step list, and
+    a rebuild that duplicated entries when a nested list held more than one
+    element). This single forward pass is linear.
+    """
+    for idx, action in enumerate(actions):
+        # a bare list raises its elements to the same level
+        if isinstance(action, (list, tuple)):
+            _flatten_actions(action, out, parent_seq_name)
+            continue
+        # a NoneType (e.g. pointing at an unused alias) contributes nothing
+        if action is None:
+            continue
+        # a 'sequence' (an included file) is spliced in, with its filename
+        # propagated onto each of its items
+        if isinstance(action, dict) and "sequence" in action:
+            sequence = action["sequence"]["data"]
+            f = action["sequence"]["filename"]
+            if isinstance(sequence, dict):
+                sequence = [{k: v} for k, v in sequence.items()]
+            # Case of an empty sequence
+            elif sequence is None:
+                tm.print_info(
+                    f"An empty sequence is loaded in '{parent_seq_name}'."
+                )
+                sequence = []
+            elif not isinstance(sequence, list):
+                raise ETUMSyntaxError(
+                    f"Syntax error in '{parent_seq_name}' step number {idx+1}. Sequence definition: '{str(action)}'",
+                    f
+                )
+            for s in sequence:
+                if isinstance(s, dict) and s:
+                    s[list(s.keys())[0]]["seq_filename"] = f
+            _flatten_actions(sequence, out, parent_seq_name)
+            continue
+
+        out.append(action)
+
+
 class TestSet:
     def __init__(
         self,
@@ -434,56 +479,16 @@ class TestSet:
                 f"No valid list of actions in sequence {parent_seq_name}",
                 file_name
             )
-        # first we merged to the same level 'sequence dict entries and list within the list
-        counter = 0
         test_dir = tm.gd("test_directory")
-        la = len(parent_seq_actions)
-        while counter < la:
-            action = parent_seq_actions[counter]
-            # if action is a list raise up to the the same level,
-            # ie insert action element into the parent_seq_actions
-            if isinstance(action, (list, tuple)):
-                parent_seq_actions[counter : counter + 1] = action
-                parent_seq_actions = (
-                    parent_seq_actions[:counter]
-                    + action
-                    + parent_seq_actions[counter + 1 :]
-                )
-                la = len(parent_seq_actions)
-                continue
-            # if action is a NoneType skip and continue
-            # (when pointing to an unused alias for instance)
-            if action is None:
-                counter += 1
-                continue
-            # if action is a sequence we insert its entry into the action list
-            if "sequence" in action:
-                sequence = action["sequence"]["data"]
-                f = action["sequence"]["filename"]
-                if isinstance(sequence, dict):
-                    sequence = [{k: v} for k, v in sequence.items()]
-                # Case of an empty sequence
-                elif sequence is None:
-                    tm.print_info(
-                        f"An empty sequence is loaded in '{parent_seq_name}'."
-                    )
-                    sequence = []
-                elif not isinstance(sequence, list):
-                    raise ETUMSyntaxError(
-                        f"Syntax error in '{parent_seq_name}' step number {counter+1}. Sequence definition: '{str(action)}'",
-                        f
-                    )
-                for s in sequence:
-                    s[list(s.keys())[0]]["seq_filename"] = f
-                parent_seq_actions = (
-                    parent_seq_actions[:counter]
-                    + sequence
-                    + parent_seq_actions[counter + 1 :]
-                )
-                la = len(parent_seq_actions)
-                continue
 
-            # Action is now for sure a list of dict of length 1
+        # Flatten nested lists and included 'sequence' entries to the same level
+        # in one linear pass (was an in-place splice + full list rebuild per
+        # entry: O(n^2) over the step list).
+        flat_actions = []
+        _flatten_actions(parent_seq_actions, flat_actions, parent_seq_name)
+
+        for action in flat_actions:
+            # Action is now for sure a dict of length 1
             k = list(action.keys())[0]
             if action[k].get("seq_filename", None) is None:
                 action[k]["seq_filename"] = file_name
@@ -546,8 +551,6 @@ class TestSet:
                     action[k]["seq_filename"]
                 )
 
-            counter += 1
-
         return ret
 
     def tree(self):

src/testium/interpreter/utils/bins.py

@@ -19,12 +19,13 @@ Public API
 
 import atexit
 import os
+import shlex
 import shutil
 import subprocess
 import tempfile
 
 import api.testium as tm
-from interpreter.utils.paths import sys_app_path_lin, sys_app_path_win
+from interpreter.utils.paths import sys_app_path_lin, sys_app_path_win, no_window_kwargs
 from runtime.tum_except import ETUMRuntimeError
 
 
@@ -177,6 +178,27 @@ def flatpak_host_spawn(interp_bin, cmd_args, host_cwd, extra_env=None):
     return spawn
 
 
+def host_console_command(shell_cmd, cwd):
+    """Build the argv to start *shell_cmd* as an ordinary interactive console.
+
+    *shell_cmd* is the command the caller chose (a string — shell-split — or
+    an argv list); the choice is preserved verbatim.
+
+    Outside Flatpak the command is returned unchanged. Inside Flatpak a bare
+    spawn would run in the sandbox under the runtime python3, so a host venv
+    (``/path/venv/bin/python3 -m mod``) can't see its pip deps. We simply run
+    it on the host with ``flatpak-spawn --host`` so it behaves like any other
+    terminal: flatpak-spawn passes the current environment through unchanged
+    and the shell (sourced venv, profile, …) sets things up as the user wants.
+    No env forwarding or scrubbing — the launcher's leaked PYTHONPATH points at
+    /app paths absent on the host, so it's inert there.
+    """
+    argv = shlex.split(shell_cmd) if isinstance(shell_cmd, str) else list(shell_cmd)
+    if not _in_flatpak():
+        return argv
+    return ["flatpak-spawn", "--host", f"--directory={cwd}", *argv]
+
+
 def _which_host_flatpak(name):
     """Resolve a binary name (or absolute path) on the host via flatpak-spawn.
 
@@ -250,6 +272,7 @@ def _run_probe(cmd):
         r = subprocess.run(
             cmd, capture_output=True, text=True,
             encoding=tm.sys_encoding(), timeout=10, env=_probe_env(),
+            **no_window_kwargs(),
         )
     except (FileNotFoundError, PermissionError, subprocess.TimeoutExpired):
         return None
@@ -366,12 +389,16 @@ def ensure(*names):
     """
     missing = []
     for n in names:
-        if not _resolve(n):
+        path = _resolve(n)
         display, gd_key, candidates, _ = _SPECS[n]
+        if not path:
             missing.append(
                 f"  - {display}: tried {candidates} on PATH, none usable. "
                 f"Set '{gd_key}' in the YAML config to override."
             )
+        elif not tm.gd(gd_key):
+            # Publish resolved path so test scripts can use $(python_bin)/$(lua_bin).
+            tm.setgd(gd_key, path)
     if missing:
         raise ETUMRuntimeError(
             "Required external interpreter(s) not found:\n" + "\n".join(missing)

src/testium/interpreter/utils/include.py

@@ -6,10 +6,10 @@ from runtime.tum_except import ETUMFileError
 from interpreter.utils.template import template_to_test
 from copy import copy
 from interpreter.utils.globdict import global_dict
-from interpreter.utils.yaml_load import yaml_load
+from interpreter.utils.yaml_load import yaml_load, YAML_BASE_LOADER
 
 
-class TUMLoaderNoIncludes(yaml.Loader):
+class TUMLoaderNoIncludes(YAML_BASE_LOADER):
 
     def __init__(self, stream):
 

src/testium/interpreter/utils/lua_process.py

@@ -1,14 +1,13 @@
 import os
 import sys
 import subprocess
-import socket
 
 import api.testium as tm
 from runtime.jrpc import JsonRpcClient
-from interpreter.utils.paths import subproc_path
+from interpreter.utils.paths import subproc_path, no_window_kwargs
 from runtime.tum_except import ETUMRuntimeError
 from interpreter.utils import bins
-from interpreter.utils.proc_drain import drain_to_log
+from interpreter.utils.proc_drain import drain_and_read_port, wait_for_port
 
 
 class LuaProcessBase:
@@ -79,12 +78,7 @@ class LuaProcessBase:
                 else:
                     env[k] = e + ";" + env.get(k, "")
 
-        sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
-        sock.bind(("localhost", 0))
-        self._port = sock.getsockname()[1]
-        sock.close()
-
-        # POpen params
+        # POpen params (port 0 -> the Lua server picks a free port and reports it)
         cmd_args = [
             "main.lua",
             "--timeout",
@@ -92,7 +86,7 @@ class LuaProcessBase:
             "--host",
             "127.0.0.1",
             "--port",
-            f"{self._port}",
+            "0",
         ]
 
         if tm.debug_enabled() and tm.gd("debug_rpc", False):
@@ -120,12 +114,19 @@ class LuaProcessBase:
             stdout=subprocess.PIPE,
             stderr=subprocess.PIPE,
             restore_signals=False,
+            **no_window_kwargs(),
             **popen_kwargs,
         )
-        # Route subprocess stdout/stderr (lua require failures, syntax
-        # errors, anything written to fd 1/2 before the in-script
-        # remote_print is set up) into the parent's log.
-        drain_to_log(self._process, prefix="[lua_func] ")
+        # Forward subprocess output to the log and read the startup port sentinel.
+        holder = drain_and_read_port(self._process, prefix="[lua_func] ")
+        self._port = wait_for_port(
+            self._process, holder, tm.gd("proc_start_timeout", 30)
+        )
+        if self._port is None:
+            # Worker died before announcing its port: reset so a later start() retries clean.
+            self.stop()
+            self.join()
+            return
 
         self._rpc = JsonRpcClient(
             "localhost", self._port, req_handler=self._req_handler

src/testium/interpreter/utils/paths.py

@@ -8,6 +8,14 @@ import subprocess
 import api.testium as tm
 
 
+def no_window_kwargs():
+    # Hide stray child consoles in the frozen Windows GUI exe (console=False has
+    # no console to inherit). The wheel/source keeps its console, so leave it.
+    if sys.platform == "win32" and getattr(sys, "frozen", False):
+        return {"creationflags": subprocess.CREATE_NO_WINDOW}
+    return {}
+
+
 def testium_path():
 
     if getattr(sys, 'frozen', False):
@@ -54,6 +62,7 @@ def sys_app_path_win(app_name):
             text=True,
             encoding="oem",
             timeout=10,
+            **no_window_kwargs(),
         )
         data = result.stdout
     except (FileNotFoundError, PermissionError, subprocess.TimeoutExpired):

src/testium/interpreter/utils/proc_drain.py

@@ -8,6 +8,9 @@ exceptions before the in-process redirection kicks in, lua
 ``require`` failures, anything written to fd 1/2 directly).
 """
 import threading
+from time import monotonic
+
+from runtime.jrpc import RPC_PORT_SENTINEL
 
 
 def _drain_pipe(pipe, prefix):
@@ -46,3 +49,60 @@ def drain_to_log(process, prefix=""):
         t.start()
         threads.append(t)
     return threads
+
+
+def drain_and_read_port(process, prefix=""):
+    """Like :func:`drain_to_log`, but the stdout reader also watches for the
+    startup port sentinel. Returns a ``holder`` dict (passed to
+    :func:`wait_for_port`); all non-sentinel lines are still forwarded to the
+    log. stderr is drained as usual.
+    """
+    holder = {"port": None, "evt": threading.Event()}
+
+    def _read_stdout(pipe):
+        try:
+            for raw in iter(pipe.readline, b""):
+                line = raw.decode("utf-8", errors="replace").rstrip("\r\n")
+                if holder["port"] is None and line.startswith(RPC_PORT_SENTINEL):
+                    try:
+                        holder["port"] = int(line[len(RPC_PORT_SENTINEL):].strip())
+                    except ValueError:
+                        continue
+                    holder["evt"].set()
+                    continue
+                if line:
+                    print(f"{prefix}{line}" if prefix else line)
+        finally:
+            try:
+                pipe.close()
+            except Exception:
+                pass
+            holder["evt"].set()  # unblock waiter on EOF even without sentinel
+
+    if process.stdout is not None:
+        threading.Thread(
+            target=_read_stdout, args=(process.stdout,), daemon=True,
+        ).start()
+    if process.stderr is not None:
+        threading.Thread(
+            target=_drain_pipe, args=(process.stderr, prefix), daemon=True,
+        ).start()
+    return holder
+
+
+def wait_for_port(process, holder, deadline):
+    """Block until the port sentinel arrives, the process dies, or *deadline*
+    seconds elapse. Returns the port int or ``None``.
+    """
+    end = monotonic() + deadline
+    while holder["port"] is None:
+        remaining = end - monotonic()
+        if remaining <= 0:
+            break
+        holder["evt"].wait(min(remaining, 0.2))
+        if holder["port"] is not None:
+            break
+        if process.poll() is not None:
+            holder["evt"].wait(0.2)  # child exited; let the reader flush a trailing line
+            break
+    return holder["port"]

src/testium/interpreter/utils/py_process.py

@@ -1,13 +1,12 @@
 import os
 import sys
 import subprocess
-import socket
 from runtime.jrpc import JsonRpcClient
 import api.testium as tm
 from runtime.tum_except import ETUMRuntimeError
-from interpreter.utils.paths import testium_path, subproc_path
+from interpreter.utils.paths import testium_path, subproc_path, no_window_kwargs
 from interpreter.utils import bins
-from interpreter.utils.proc_drain import drain_to_log
+from interpreter.utils.proc_drain import drain_and_read_port, wait_for_port
 
 
 class PyProcessBase:
@@ -54,13 +53,6 @@ class PyProcessBase:
                 else:
                     env[k] = e + os.pathsep + env.get(k, "")
 
-        sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
-        sock.bind(("localhost", 0))
-        self._port = sock.getsockname()[1]
-        # Port was reserved until the sub-process is started. Now released.
-        if sock is not None:
-            sock.close()
-
         # In Flatpak the host can't see /app/lib/testium, so use a staged copy
         # under /tmp (shared between sandbox and host) for both cwd and as the
         # root in PYTHONPATH. Outside Flatpak the original paths are used.
@@ -75,7 +67,7 @@ class PyProcessBase:
         cmd_args = [
             "py_func",
             "-p",
-            f"{self._port}",
+            "0",
             "-t",
             f"{self._timeout}",
         ]
@@ -105,13 +97,19 @@ class PyProcessBase:
             stdout=subprocess.PIPE,
             stderr=subprocess.PIPE,
             restore_signals=False,
+            **no_window_kwargs(),
             **popen_kwargs,
         )
-        # Route subprocess stdout/stderr (early-startup errors,
-        # unhandled exceptions, anything written to fd 1/2 before the
-        # in-process JSON-RPC stdio_redir kicks in) into the parent's
-        # log.
-        drain_to_log(self._process, prefix="[py_func] ")
+        # Forward subprocess output to the log and read the startup port sentinel.
+        holder = drain_and_read_port(self._process, prefix="[py_func] ")
+        self._port = wait_for_port(
+            self._process, holder, tm.gd("proc_start_timeout", 30)
+        )
+        if self._port is None:
+            # Worker died before announcing its port: reset so a later start() retries clean.
+            self.stop()
+            self.join()
+            return
 
         self._rpc = JsonRpcClient(
             "localhost", self._port, req_handler=self._req_handler

src/testium/interpreter/utils/template.py

@@ -1,22 +1,61 @@
+import io
 import os
 from sys import exc_info
-from jinja2 import Template
+from jinja2 import Environment
 from jinja2.exceptions import TemplateSyntaxError, TemplateError, UndefinedError
-from tempfile import TemporaryFile
 from interpreter.utils.yaml_load import print_yaml
 from runtime.tum_except import ETUMSyntaxError
 
 
+# One Environment reused for every render (default settings, i.e. identical
+# behaviour to jinja2.Template), plus a compiled-template cache so a file that
+# is included many times — or a test that is reloaded — is compiled only once.
+# Jinja compilation is the expensive step; render (variable substitution) stays
+# per-call. Cache is keyed on path + mtime + size so an edited file recompiles.
+_ENV = Environment()
+_template_cache = {}  # abspath -> (mtime_ns, size, compiled_template)
+
+
+class _RenderedStream(io.StringIO):
+    """A rendered template kept in memory.
+
+    Carries ``root`` (and ``name``) so the YAML loader resolves ``!include``
+    paths exactly as it did from the on-disk temp file this replaces — without
+    the write + seek + read round-trip (one temp file per included file). That
+    round-trip is pure overhead, and especially costly on slow storage.
+    """
+
+
+def _compiled_template(filename: str):
+    """Return the compiled jinja template for *filename*, reusing the cached
+    one when the file is unchanged (path + mtime + size)."""
+    key = os.path.abspath(filename)
+    try:
+        st = os.stat(filename)
+    except OSError:
+        st = None
+    if st is not None:
+        cached = _template_cache.get(key)
+        if (cached is not None
+                and cached[0] == st.st_mtime_ns
+                and cached[1] == st.st_size):
+            return cached[2]
+    with open(filename, "r") as f:
+        source = f.read()
+    template = _ENV.from_string(source)  # compile (may raise TemplateSyntaxError)
+    if st is not None:
+        _template_cache[key] = (st.st_mtime_ns, st.st_size, template)
+    return template
+
+
 def template_to_test(filename: str, params: list):
     """ Function which processes an eventual jinja2 template to a test file
     """
-    # Temporary file created to receive the processed include
-    # file
-    tmpf = TemporaryFile('w+t')
-    with open(filename, 'r') as f:
+    # Compile (cached) — a syntax error in the template surfaces here.
     try:
-            j2_template = Template(f.read())
+        j2_template = _compiled_template(filename)
     except TemplateError as e:
+        with open(filename, "r") as f:
             print_yaml(f, filename)
         type, value, tb = exc_info()
         msg = "Template error"
@@ -25,9 +64,11 @@ def template_to_test(filename: str, params: list):
         else:
             msg += ": "
         raise ETUMSyntaxError(msg + str(e), filename)
+
+    # Render into memory (no temp file).
     try:
         params["include_directory"] = os.path.dirname(os.path.abspath(filename))
-        tmpf.write(j2_template.render(params))
+        rendered = j2_template.render(params)
     except TemplateSyntaxError as e:
         raise ETUMSyntaxError(f"""Template loading of file '{filename}' with following parameters '{str(params)}'
 Syntax error in template: {e.message}""")
@@ -42,8 +83,7 @@ Template rendering error: {e.message}""")
         raise ETUMSyntaxError(f"""Template loading of file '{filename}' with following parameters '{str(params)}'
 Unexpected error: {str(e)}""")
 
-    # return to begining of the temp file
-    tmpf.seek(0, os.SEEK_SET)
-    tmpf.root = os.path.dirname(filename)
-
-    return tmpf
+    stream = _RenderedStream(rendered)
+    stream.root = os.path.dirname(filename)
+    stream.name = filename
+    return stream

src/testium/interpreter/utils/test_init.py

@@ -11,7 +11,7 @@ import api.testium as tm
 import interpreter.utils.globdict as globdict
 import interpreter.utils.settings as prefs
 from interpreter.utils.paths import testium_path
-from interpreter.utils.yaml_load import yaml_load
+from interpreter.utils.yaml_load import yaml_load, YAML_BASE_LOADER
 from interpreter.utils import clear_recursively
 from runtime.tum_except import ETUMSyntaxError
 from interpreter.utils.params import expanse, eval_func_init
@@ -89,7 +89,7 @@ def locate_report_file(rep_file):
 def yamltodict(param_file, silent=True):
     # load of the file
     with open(param_file, "r") as fd:
-        dp = yaml_load(fd, param_file, yaml.Loader)
+        dp = yaml_load(fd, param_file, YAML_BASE_LOADER)
 
     if dp is None:
         tm.print_info(f"The YAML file '{param_file}' is empty.")

src/testium/interpreter/utils/yaml_load.py

@@ -1,3 +1,4 @@
+import yaml
 from yaml.parser import ParserError
 from yaml import load, Loader
 from yaml.scanner import ScannerError
@@ -5,6 +6,12 @@ from api.testium import print_debug
 from runtime.tum_except import ETUMSyntaxError
 import io
 
+# Use the libyaml-backed loader (much faster parsing) when PyYAML was built
+# with it, falling back to the pure-Python loader otherwise. The C loader
+# raises the same ParserError/ScannerError and supports the same custom
+# constructors (!include) and construct_* helpers the TUM loaders rely on.
+YAML_BASE_LOADER = yaml.CLoader if getattr(yaml, "__with_libyaml__", False) else yaml.Loader
+
 
 def print_yaml(file: io.TextIOWrapper, file_name):
     """ Prints YAML file if debug mode is activated.
@@ -21,10 +28,10 @@ def yaml_load(file, real_file_name: str, loader: Loader):
         return load(file, loader)
 
     except ParserError as e:
-        if isinstance(file, io.TextIOWrapper):
+        if isinstance(file, (io.TextIOWrapper, io.StringIO)):
             print_yaml(file, real_file_name)
         raise ETUMSyntaxError(f"yaml file parsing error: " + str(e), real_file_name)
     except ScannerError as e:
-        if isinstance(file, io.TextIOWrapper):
+        if isinstance(file, (io.TextIOWrapper, io.StringIO)):
             print_yaml(file, real_file_name)
         raise ETUMSyntaxError("yaml file scanning error: " + str(e), real_file_name)

src/testium/lua_func/main.lua

@@ -3,7 +3,7 @@
 -- =========================
 local config = {
     host = "0.0.0.0",
-    port = 9000,
+    port = 0,          -- 0 = OS-assigned; actual port is reported on stdout
     timeout = 60,
     verbose = false,
 }
@@ -76,6 +76,10 @@ server_sock:listen(1)
 local ip, port = server_sock:getsockname()
 utils.log("listening on %s:%d for %.1f secs", ip, port, config.timeout)
 
+-- Announce the actual bound port so the parent connects only once we listen.
+io.stdout:write("__TESTIUM_RPC_PORT__=" .. port .. "\n")
+io.stdout:flush()
+
 server_sock:settimeout(config.timeout) -- Prevents hanging on dead connections
 
 -- Main Server Loop

src/testium/main_win/desktop_integration.py

@@ -0,0 +1,95 @@
+"""Install a desktop entry + icon under the user's data dir so desktop shells
+show the testium icon in the task bar / dock.
+
+On a native Wayland session GNOME takes a window's task-bar icon from the
+``.desktop`` file whose name (or ``StartupWMClass``) matches the window
+``app_id`` — ``QGuiApplication.setWindowIcon`` is ignored there. The portable
+channels (source checkout, PyInstaller binary, AppImage) install no system
+desktop file, so we drop an idempotent one in ``~/.local/share``. The window
+``app_id`` is set to ``testium`` (see ``QApplication.setDesktopFileName`` in
+``testium_win``), which is exactly this file's base name.
+
+Flatpak ships its own ``org.testium.Testium.desktop`` and keeps its own app id,
+so the caller skips this integration there.
+"""
+import os
+import sys
+
+from PySide6.QtCore import Qt
+from PySide6.QtGui import QPixmap
+
+# Must match QApplication.setDesktopFileName(...) for the GUI, and is used as
+# both the desktop-file base name and the StartupWMClass.
+APP_ID = "testium"
+
+
+def _launch_command():
+    """Best-effort Exec= for the menu entry. Not needed for icon matching, but
+    makes the entry actually launchable when possible."""
+    appimage = os.environ.get("APPIMAGE")
+    if appimage:
+        return f'"{appimage}"'
+    if getattr(sys, "frozen", False):
+        return f'"{os.path.abspath(sys.executable)}"'
+    argv0 = os.path.abspath(sys.argv[0]) if sys.argv and sys.argv[0] else ""
+    if argv0 and os.path.exists(argv0):
+        return f'"{os.path.abspath(sys.executable)}" "{argv0}"'
+    return f'"{os.path.abspath(sys.executable)}" -m testium'
+
+
+def ensure_desktop_entry():
+    """Create (or refresh) ~/.local/share icon + desktop entry. Best-effort:
+    any failure is swallowed so it can never take the GUI down.
+
+    Freedesktop-only: a no-op off Linux (Windows / macOS use the window icon)."""
+    if not sys.platform.startswith("linux"):
+        return
+    try:
+        data_home = os.environ.get("XDG_DATA_HOME") or os.path.join(
+            os.path.expanduser("~"), ".local", "share"
+        )
+        icon_dir = os.path.join(data_home, "icons", "hicolor", "256x256", "apps")
+        app_dir = os.path.join(data_home, "applications")
+        icon_path = os.path.join(icon_dir, f"{APP_ID}.png")
+        desktop_path = os.path.join(app_dir, f"{APP_ID}.desktop")
+
+        os.makedirs(icon_dir, exist_ok=True)
+        os.makedirs(app_dir, exist_ok=True)
+
+        # Icon: render the bundled Qt resource to a PNG once. Requires a live
+        # QGuiApplication (the caller creates it before calling us).
+        if not os.path.isfile(icon_path):
+            pixmap = QPixmap(u":/black/testium_logo.png")
+            if not pixmap.isNull():
+                pixmap = pixmap.scaled(
+                    256, 256,
+                    Qt.AspectRatioMode.KeepAspectRatio,
+                    Qt.TransformationMode.SmoothTransformation,
+                )
+                pixmap.save(icon_path, "PNG")
+
+        # Absolute Icon= path so the shell resolves it without an icon-cache
+        # refresh; StartupWMClass lets X11 / XWayland match too.
+        desktop = (
+            "[Desktop Entry]\n"
+            "Type=Application\n"
+            "Name=Testium\n"
+            "Comment=Test sequencer\n"
+            f"Icon={icon_path}\n"
+            f"Exec={_launch_command()} %f\n"
+            "Terminal=false\n"
+            f"StartupWMClass={APP_ID}\n"
+            "Categories=Utility;Development;\n"
+        )
+
+        # Write only when missing or changed, to avoid needless menu churn.
+        current = None
+        if os.path.isfile(desktop_path):
+            with open(desktop_path, "r") as fh:
+                current = fh.read()
+        if current != desktop:
+            with open(desktop_path, "w") as fh:
+                fh.write(desktop)
+    except Exception:
+        # Desktop integration is a nicety, never a hard requirement.
+        pass

src/testium/main_win/testium_win.py

@@ -678,6 +678,24 @@ def MainWin(
     debug=False,
 ):
     app = QApplication(sys.argv)
+    # Application identity so desktop shells (GNOME, ...) show the testium
+    # icon in the task bar / dock instead of a generic one. On Wayland this
+    # sets the surface app_id; on X11/XWayland it sets WM_CLASS, so the window
+    # stops inheriting the launcher's class (e.g. "python3" under the AppImage,
+    # which is what GNOME was keying the wrong icon off) and the window icon
+    # below is used as the fallback. In Flatpak the id must be the Flatpak app
+    # id so it matches the installed desktop file.
+    app.setApplicationName("Testium")
+    app.setApplicationDisplayName("Testium")
+    app.setDesktopFileName(os.environ.get("FLATPAK_ID", "testium"))
+    app.setWindowIcon(QIcon(u":/black/testium_logo.png"))
+    # On native Wayland the task-bar icon comes from an installed desktop file
+    # matched to the app_id, not from setWindowIcon(). Flatpak ships its own;
+    # for the other Linux channels drop an idempotent one under ~/.local/share.
+    # Windows / macOS use the window icon set above, so this is Linux-only.
+    if sys.platform.startswith("linux") and not os.environ.get("FLATPAK_ID"):
+        from main_win.desktop_integration import ensure_desktop_entry
+        ensure_desktop_entry()
     ui = MainWindow(
         test_file,
         config_files,

src/testium/py_func/__init__.py

@@ -1,7 +1,9 @@
 #!/usr/bin/env python
+import sys
 import multiprocessing
 from py_func.tm import _init_api, _remote_print
 from runtime.stdout_redirect import stdio_redir
+from runtime.jrpc import RPC_PORT_SENTINEL
 
 
 class TcpStdOut:
@@ -24,21 +26,29 @@ def main():
     parser = argparse.ArgumentParser()
     parser.add_argument("-i", "--ip", type=str, help="Ip address or hostname to listen to",
                         default="localhost")
-    parser.add_argument("-p", "--port", type=int, help="port to listen to",
-                        default=9000)
+    parser.add_argument("-p", "--port", type=int, help="port to listen to (0 = OS-assigned)",
+                        default=0)
     parser.add_argument("-t", "--timeout", type=float, help="Timeout waiting for connection",
                         default=10)
     parser.add_argument("-v", "--verbose", action='store_true', help="port to listen to")
     args = parser.parse_args()
 
     thrd_api = _init_api(args.ip, args.port, args.timeout)
-    # redirect I/O
-    outstream = TcpStdOut()
-    stdio_redir.redirect(outstream)
     # debug the server
     if args.verbose:
         thrd_api.dbg_out = stdio_redir.ini_stdout
     thrd_api.start()
+
+    # Announce the bound port on real stdout (before redirection) so the parent connects.
+    port = thrd_api.wait_bound(args.timeout)
+    if port is None:
+        print("py_func: failed to bind a listening port", file=sys.stderr, flush=True)
+        return
+    print(f"{RPC_PORT_SENTINEL}{port}", flush=True)
+
+    # redirect I/O
+    outstream = TcpStdOut()
+    stdio_redir.redirect(outstream)
     try:
         while thrd_api.is_alive():
             thrd_api.join(1)

src/testium/runtime/jrpc.py

@@ -12,6 +12,9 @@ except:
 
 from runtime.tum_except import ETUMRuntimeError
 
+# Startup handshake: subprocess prints this + its bound port on stdout once listening.
+RPC_PORT_SENTINEL = "__TESTIUM_RPC_PORT__="
+
 """Lightweight JSON-RPC 2.0 helpers over TCP sockets.
 
 This module implements a minimal JSON-RPC 2.0 messaging layer using
@@ -279,6 +282,8 @@ class JsonRpcBase(threading.Thread):
         self._req_handler = req_handler
         self._dbg_out = dbg_out
         self._event_ready = threading.Event()
+        # Set on success AND failure so wait_ready() never hangs; outcome in _connected.
+        self._connected = False
 
     def handle_request(self, method, params):
         """Override to implement server-side request handling.
@@ -314,10 +319,12 @@ class JsonRpcBase(threading.Thread):
             self.name, sock, self.handle_request, dbg_out=self.dbg_out
         )
         self._rpc.wait_ready()
+        self._connected = True
         self._event_ready.set()
 
     def wait_ready(self, timeout=None):
-        return self._event_ready.wait(timeout)
+        self._event_ready.wait(timeout)
+        return self._connected
 
     @property
     def dbg_out(self):
@@ -348,20 +355,30 @@ class JsonRpcSrv(JsonRpcBase):
     def __init__(self, host, port, req_handler=None, timeout=10):
         super().__init__(host, port, req_handler, timeout)
         self.name = f"JsonRpcSvr_{port}"
+        self._bound_port = None
+        self._bound_evt = threading.Event()
+
+    @property
+    def bound_port(self):
+        return self._bound_port
+
+    def wait_bound(self, timeout=None):
+        self._bound_evt.wait(timeout)
+        return self._bound_port
 
     def run(self):
         # TCP/IP socket creation
         try:
             with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
 
-                sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
-
-                # Link of the socket at the configured port
+                # No SO_REUSEADDR: fresh ephemeral port; on Windows it enables hijacking.
                 sock.bind((self._host, self._port))
 
                 # Listens incoming connections
                 sock.listen(1)
-                self.print_info(f"listening on {self._host}:{self._port}")
+                self._bound_port = sock.getsockname()[1]
+                self._bound_evt.set()
+                self.print_info(f"listening on {self._host}:{self._bound_port}")
 
                 self.print_info(f"awaiting connection for {self._timeout} secs")
                 sock.settimeout(self._timeout)
@@ -382,6 +399,7 @@ class JsonRpcSrv(JsonRpcBase):
                         sleep(0.1)
 
         finally:
+            self._bound_evt.set()  # unblock wait_bound() even on failure
             if self._rpc is not None:
                 self._rpc.stop()
                 self._rpc.join()
@@ -407,35 +425,34 @@ class JsonRpcClient(JsonRpcBase):
         self.name = f"JsonRpcClt_{port}"
 
     def run(self):
+        try:
             if tm.OS() == "Windows":
                 self.run_win()
             else:
                 self.run_lin()
+        except Exception as e:
+            self.print_info(f"connection failed: {e}")
+        finally:
+            self._event_ready.set()  # settle wait_ready() whatever the outcome
 
     def run_win(self):
-        # TCP/IP socket creation
-        tslice = 1
-        t = self._timeout
+        # Server already listening (handshake); retry on refused/timeout until deadline.
+        deadline = monotonic() + self._timeout
         sock = None
         try:
-            while t >= 0:
+            while True:
                 sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
-                sock.settimeout(tslice)
-                # Link of the socket at the configured port
+                sock.settimeout(0.5)
                 try:
                     sock.connect((self._host, self._port))
                     break
-                except socket.timeout:
+                except OSError as e:
                     sock.close()
-                    t -= tslice
-                    if t < 0:
+                    if monotonic() >= deadline:
                         raise ETUMRuntimeError(
-                            f"{self.name}: failed to connect : timeout"
+                            f"{self.name}: failed to connect : {e}"
                         )
-                    else:
-                        sleep(tslice)
-                except socket.error as e:
-                    raise ETUMRuntimeError(f"{self.name}: failed to connect : {e}")
+                    sleep(0.1)
 
             self.print_info("Connected to server")
             self.connect(sock)

test/benchmark/.gitignore

@@ -0,0 +1 @@
+cases/

test/benchmark/README.md

@@ -0,0 +1,116 @@
+# Load-time benchmark
+
+Measures how long *testium* takes to **load** a `.tum` test tree — template
+rendering (jinja) + YAML parsing + test-tree construction — *without* executing
+it. Purpose: get reproducible numbers before/after load-path optimisations, and
+attribute any gain to a specific part of the pipeline.
+
+It is meant for *very long* tests, the kind you can build with `jinja` loops and
+`!include`, where load time becomes noticeable.
+
+## Files
+
+| File | Role |
+|------|------|
+| `gen_bench_test.py` | Generates a synthetic `.tum` tree (the test input). |
+| `load_bench.py` | Drives the **real** loader in-process and times it. |
+| `run.sh` | Convenience: generate + time across profiles, using the project venv. |
+| `cases/` | Generated trees (git-ignored, recreated on demand). |
+
+The benchmark `.tum` files are **generated**, not committed — the generator is
+the artifact. They use only `let` leaves and `group` containers, so loading has
+no runtime side effect (no subprocess, no `<| |>` eval) and the timing reflects
+the parse/build pipeline alone.
+
+## Quick start
+
+```bash
+# default matrix (all profiles), 5 repeats each
+./test/benchmark/run.sh
+
+# one profile at one size
+./test/benchmark/run.sh repeat 2000
+
+# more repeats for a tighter min
+REPEAT=10 ./test/benchmark/run.sh includes 1000
+```
+
+`run.sh` uses the project venv at `test/tmp/.venv` (created by `./run.sh`). If it
+is missing, run `./run.sh` once first.
+
+To drive the harness directly on any `.tum` (not just generated ones):
+
+```bash
+test/tmp/.venv/bin/python3 test/benchmark/load_bench.py --repeat 5 --quiet path/to/main.tum
+```
+
+## Profiles
+
+Each profile isolates one cost. `--size` is the profile-specific count.
+
+| Profile | What it builds | Stresses |
+|---------|----------------|----------|
+| `flat` | one main file, N inline `let` steps | big YAML parse + linear object build |
+| `includes` | main `!include`s N **distinct** sub-files | per-include template+YAML+tempfile, `sequence` splice |
+| `repeat` | main `!include`s the **same** parametrised leaf N times | jinja **recompilation** of an identical template |
+| `jinja` | one main file, `{% for %}` emitting N steps | single large render + single large parse |
+| `deep` | nested includes, depth N | include recursion (see caveat) |
+| `mix` | groups + jinja loop + distinct + repeated includes | realistic blend |
+
+## Reading the output
+
+```
+phase              min      median
+initial         0.1131      0.1285   <- pass 1: discover config files (no includes)
+loadtest        1.0724      1.0900   <- config fixpoint loop + full recursive include load
+build           0.1850      0.1976   <- TestSet: load_test_recursively tree build
+total           1.3886      1.4227
+counters  (last run):
+  templates :    1003 calls   0.5247s  (exclusive: jinja compile+render+tempfile)
+  yaml      :    1004 parses   1.4696s  (inclusive of nested includes)
+```
+
+- **min** is the headline (least noisy); median is a sanity check.
+- **initial / loadtest / build** map to the three pipeline stages in
+  `interpreter/process.py` and `interpreter/test_set.py`. The main file is
+  rendered+parsed across `initial` *and* `loadtest` (the loader does ~3 passes).
+- **templates** = number of `template_to_test()` calls and their *exclusive*
+  wall time (one file render each — pure jinja compile+render+tempfile I/O).
+  A high count with the same source file = recompilation, the `repeat` case.
+- **yaml** = number of `yaml_load()` parses. Its time is *inclusive* of nested
+  includes, so use the **count** for attribution, not the seconds.
+
+## Mapping to the optimisation axes
+
+| Axis (see DESIGN / discussion) | Watch | Best profile to prove it |
+|--------------------------------|-------|--------------------------|
+| 1 — cache compiled jinja templates | `templates` time drops, count unchanged | `repeat` |
+| 2 — drop the tempfile round-trip | `templates` time drops | `includes`, `repeat`, `mix` |
+| 3 — C YAML loader (libyaml) | `yaml` time / `loadtest` drops | `flat`, `jinja` |
+| 6 — O(n²) sequence splice | `build` drops | `includes`, `mix` |
+
+## How to compare before/after a change
+
+1. Run the matrix on the current code, keep the output.
+2. Apply one axis.
+3. Re-run the **same** profiles/sizes; compare `min` per phase and the counters.
+
+Change one axis at a time so the attribution is clean. Run on an idle machine
+(and note the disk: on a USB stick the tempfile round-trip of axis 2 weighs
+more).
+
+## Caveat: deep includes
+
+The loader is recursive and spends ~10 stack frames per include level, so
+`deep` hits Python's `RecursionError` around ~90 nested levels. The harness
+reports this cleanly instead of crashing. Real tests are *wide* (many steps /
+many includes), not deep, so `includes`/`repeat`/`jinja`/`mix` are the
+representative "very long" cases.
+
+## Notes
+
+- No execution is triggered — timing stops where `Batch` would mark the test
+  *loaded*.
+- The profiles contain no `<| |>`, so the external eval process is not started.
+  Pass `--with-eval` to `load_bench.py` for trees that evaluate at load time.
+- Numbers are machine- and disk-specific; only compare runs from the same host.

test/benchmark/gen_bench_test.py

@@ -0,0 +1,179 @@
+#!/usr/bin/env python3
+"""Generate synthetic ``.tum`` test trees to benchmark *load* time.
+
+The generated trees are deliberately cheap to *build* (only ``let`` leaves and
+``group`` containers — no subprocess, no runtime side effect) so the load
+benchmark measures the parse / template / tree-build pipeline and nothing else.
+
+Profiles, each targeting a specific cost in the loader:
+
+  flat      one main file, N inline ``let`` steps, no include, no jinja.
+            Baseline: YAML parse of a big document + linear object build.
+
+  includes  main ``!include``s N *distinct* sub-files (a few steps each).
+            Stresses the per-include template+YAML+tempfile round-trip and the
+            ``sequence`` splice in test_set.load_test_recursively.
+
+  repeat    main ``!include``s the *same* parametrised leaf file N times.
+            Stresses jinja *recompilation*: the compiled template is identical
+            every time, only the render params (idx) differ -> the case a
+            template cache collapses.
+
+  jinja     one main file whose ``{% for %}`` loop emits N steps.
+            Stresses a single large jinja render + a single large YAML parse.
+
+  deep      nested includes, depth N (main -> d0 -> d1 -> ...).
+            Stresses include recursion and per-level template+YAML.
+
+  mix       a realistic blend: groups, a jinja loop, distinct includes and a
+            repeated parametrised include.
+
+Usage:
+    gen_bench_test.py --profile repeat --size 1000 --out cases/repeat_1000
+    -> writes <out>/main.tum (+ includes, + param.yaml) and prints the path.
+"""
+import argparse
+import os
+import shutil
+
+
+def _let(indent, i, name=None):
+    name = name if name is not None else f"s{i}"
+    pad = " " * indent
+    return (
+        f"{pad}- let:\n"
+        f"{pad}    name: {name}\n"
+        f"{pad}    values:\n"
+        f"{pad}        - k{i}: {i}\n"
+    )
+
+
+def gen_flat(out, n):
+    body = "".join(_let(8, i) for i in range(n))
+    main = f"main:\n    name: bench flat {n}\n    steps:\n{body}"
+    _write(out, "main.tum", main)
+
+
+def gen_includes(out, n):
+    steps = "".join(f"        - !include inc_{i}.tum\n" for i in range(n))
+    main = f"main:\n    name: bench includes {n}\n    steps:\n{steps}"
+    _write(out, "main.tum", main)
+    for i in range(n):
+        # each include is a YAML *sequence* (list of steps)
+        seq = "".join(_let(0, i * 3 + j, name=f"inc{i}_{j}") for j in range(3))
+        _write(out, f"inc_{i}.tum", seq)
+
+
+def gen_repeat(out, n):
+    steps = "".join(
+        f"        - !include {{file: leaf.tum, idx: {i}}}\n" for i in range(n)
+    )
+    main = f"main:\n    name: bench repeat {n}\n    steps:\n{steps}"
+    _write(out, "main.tum", main)
+    leaf = (
+        "- let:\n"
+        "    name: leaf_{{ idx }}\n"
+        "    values:\n"
+        "        - leaf_{{ idx }}: {{ idx }}\n"
+    )
+    _write(out, "leaf.tum", leaf)
+
+
+def gen_jinja(out, n):
+    main = (
+        f"main:\n    name: bench jinja {n}\n    steps:\n"
+        "{% for i in range(" + str(n) + ") %}\n"
+        "        - let:\n"
+        "            name: j{{ i }}\n"
+        "            values:\n"
+        "                - k{{ i }}: {{ i }}\n"
+        "{% endfor %}\n"
+    )
+    _write(out, "main.tum", main)
+
+
+def gen_deep(out, n):
+    main = (
+        f"main:\n    name: bench deep {n}\n    steps:\n"
+        "        - let:\n            name: top\n            values:\n                - a: 0\n"
+        "        - !include d_0.tum\n"
+    )
+    _write(out, "main.tum", main)
+    for i in range(n):
+        seq = _let(0, i, name=f"d{i}")
+        if i < n - 1:
+            seq += f"- !include d_{i + 1}.tum\n"
+        _write(out, f"d_{i}.tum", seq)
+
+
+def gen_mix(out, n):
+    # n groups, each: 2 inline lets, one distinct include, one repeated include,
+    # plus a small jinja loop. Roughly ~6*n steps.
+    per = max(1, n)
+    parts = [f"main:\n    name: bench mix {n}\n    steps:\n"]
+    for g in range(per):
+        parts.append(
+            f"        - group:\n"
+            f"            name: grp{g}\n"
+            f"            steps:\n"
+        )
+        parts.append(_let(16, g * 2, name=f"g{g}_a"))
+        parts.append(_let(16, g * 2 + 1, name=f"g{g}_b"))
+        parts.append(f"                - !include inc_{g}.tum\n")
+        parts.append(f"                - !include {{file: leaf.tum, idx: {g}}}\n")
+        parts.append(
+            "{% for i in range(3) %}\n"
+            f"                - let:\n"
+            f"                    name: g{g}_j{{{{ i }}}}\n"
+            f"                    values:\n"
+            f"                        - g{g}_k{{{{ i }}}}: {{{{ i }}}}\n"
+            "{% endfor %}\n"
+        )
+    _write(out, "main.tum", "".join(parts))
+    for g in range(per):
+        _write(out, f"inc_{g}.tum", _let(0, g, name=f"mixinc{g}"))
+    _write(
+        out,
+        "leaf.tum",
+        "- let:\n    name: mixleaf_{{ idx }}\n    values:\n        - mixleaf_{{ idx }}: {{ idx }}\n",
+    )
+
+
+PROFILES = {
+    "flat": gen_flat,
+    "includes": gen_includes,
+    "repeat": gen_repeat,
+    "jinja": gen_jinja,
+    "deep": gen_deep,
+    "mix": gen_mix,
+}
+
+
+def _write(out, name, content):
+    with open(os.path.join(out, name), "w") as f:
+        f.write(content)
+
+
+def main():
+    ap = argparse.ArgumentParser(description=__doc__,
+                                 formatter_class=argparse.RawDescriptionHelpFormatter)
+    ap.add_argument("--profile", required=True, choices=sorted(PROFILES))
+    ap.add_argument("--size", type=int, default=1000,
+                    help="profile-specific count (steps / includes / depth)")
+    ap.add_argument("--out", required=True, help="output directory (recreated)")
+    args = ap.parse_args()
+
+    out = os.path.abspath(args.out)
+    if os.path.isdir(out):
+        shutil.rmtree(out)
+    os.makedirs(out)
+
+    # minimal config file so the loader does not emit "no param file" noise
+    _write(out, "param.yaml", "bench_dummy: 1\n")
+
+    PROFILES[args.profile](out, args.size)
+    print(os.path.join(out, "main.tum"))
+
+
+if __name__ == "__main__":
+    main()

test/benchmark/load_bench.py

@@ -0,0 +1,200 @@
+#!/usr/bin/env python3
+"""Time the testium *load* pipeline on a given ``.tum`` tree.
+
+It drives the real loader code (``TestProcess._load_initial_params`` /
+``_load_test`` then ``TestSet(...)``) in-process, so the numbers track the
+production path and stay honest as the code evolves. Execution is never
+triggered — we stop exactly where ``Batch`` would report the test as *loaded*.
+
+Reported per run, over ``--repeat`` iterations (min is the headline, least
+noisy):
+
+    initial   first pass: discover config files (template+YAML, no includes)
+    loadtest  config-file fixpoint loop + full recursive include/template/YAML
+    build     TestSet construction: the load_test_recursively tree build
+    total     sum of the three
+
+Plus instrumentation counters (exact call counts, wall time) for the two
+hot leaves the optimisation axes target:
+
+    templates  jinja template_to_test() calls   (axis 1 compile cache, axis 2 tempfile)
+    yaml       yaml_load() parses               (axis 3 C loader)
+
+template time is exclusive (one file render); yaml time is wall-inclusive of
+nested includes, so lean on the *counts* for attribution.
+
+Must run inside the project venv (jinja2, pyyaml, telnetlib3, ...). The
+benchmark profiles contain no ``<| |>`` so the external eval process is not
+needed; pass --with-eval to start it for faithfulness on eval-heavy trees.
+
+Usage (see run.sh for the convenience wrapper):
+    test/tmp/.venv/bin/python3 test/benchmark/load_bench.py [--repeat 5] <main.tum>
+"""
+import argparse
+import os
+import statistics
+import sys
+from queue import Queue
+from time import perf_counter
+
+# --- bootstrap: src/testium for flat imports, src for `import testium` --------
+HERE = os.path.dirname(os.path.abspath(__file__))
+ROOT = os.path.abspath(os.path.join(HERE, "..", ".."))
+sys.path.insert(0, os.path.join(ROOT, "src"))
+sys.path.insert(0, os.path.join(ROOT, "src", "testium"))
+
+import api.testium as tm  # noqa: E402
+from interpreter.utils.test_init import env_init, apply_overrides  # noqa: E402
+from interpreter.utils.test_ctrl import TestSetController  # noqa: E402
+from interpreter.process import TestProcess  # noqa: E402
+from interpreter.test_set import TestSet  # noqa: E402
+from interpreter.utils.py_eval import eval_process_init  # noqa: E402
+from interpreter.utils.api_srv import api_request  # noqa: E402
+
+# --- instrumentation: count + time the two hot leaves -------------------------
+import interpreter.process as _proc  # noqa: E402
+import interpreter.utils.include as _inc  # noqa: E402
+import interpreter.utils.test_init as _ti  # noqa: E402
+import interpreter.utils.template as _tpl  # noqa: E402
+import interpreter.utils.yaml_load as _yl  # noqa: E402
+
+_C = {"tpl_n": 0, "tpl_t": 0.0, "yaml_n": 0, "yaml_t": 0.0}
+_orig_tpl = _tpl.template_to_test
+_orig_yaml = _yl.yaml_load
+
+
+def _wrap_tpl(*a, **k):
+    t = perf_counter()
+    try:
+        return _orig_tpl(*a, **k)
+    finally:
+        _C["tpl_t"] += perf_counter() - t
+        _C["tpl_n"] += 1
+
+
+def _wrap_yaml(*a, **k):
+    t = perf_counter()
+    try:
+        return _orig_yaml(*a, **k)
+    finally:
+        _C["yaml_t"] += perf_counter() - t
+        _C["yaml_n"] += 1
+
+
+# rebind in every module that did `from ... import template_to_test / yaml_load`
+for _m in (_proc, _inc):
+    _m.template_to_test = _wrap_tpl
+for _m in (_proc, _inc, _ti):
+    _m.yaml_load = _wrap_yaml
+
+
+def _reset_counters():
+    _C.update(tpl_n=0, tpl_t=0.0, yaml_n=0, yaml_t=0.0)
+
+
+def load_once(tp, fname, test_dir):
+    """One full load (no execution). Returns (initial, loadtest, build) seconds."""
+    t0 = perf_counter()
+    init_pf, gv = tp._load_initial_params(test_dir)
+    t1 = perf_counter()
+    test_dict, _pf = tp._load_test(init_pf, gv)
+    t2 = perf_counter()
+    TestSet(fname, test_dict, Queue())
+    t3 = perf_counter()
+    return (t1 - t0, t2 - t1, t3 - t2)
+
+
+def main():
+    ap = argparse.ArgumentParser(description=__doc__,
+                                 formatter_class=argparse.RawDescriptionHelpFormatter)
+    ap.add_argument("main_tum", help="path to the generated main.tum")
+    ap.add_argument("--repeat", type=int, default=5)
+    ap.add_argument("--with-eval", action="store_true",
+                    help="start the external eval process (needed only for <| |> at load)")
+    ap.add_argument("--quiet", action="store_true",
+                    help="silence the loader's INFO output during runs")
+    args = ap.parse_args()
+
+    fname = os.path.abspath(args.main_tum)
+    if not os.path.isfile(fname):
+        ap.error(f"not found: {fname}")
+    test_dir = os.path.dirname(fname)
+
+    env_init()
+    apply_overrides({}, {})
+
+    eval_proc = None
+    if args.with_eval:
+        eval_proc = eval_process_init(api_request, 10, test_dir)
+        eval_proc.start()
+        eval_proc.wait_ready(10)
+
+    if args.quiet:
+        # the loader prints a couple of INFO lines per config file; mute stdout
+        # around the measured section to avoid I/O skew.
+        devnull = open(os.devnull, "w")
+        real_stdout = sys.stdout
+
+    tp = TestProcess(fname, Queue(), TestSetController(),
+                     config_files=[], defines={}, gui_defaults={}, text_mode=True)
+
+    samples = []  # list of (initial, loadtest, build)
+    last_counters = None
+    try:
+        for r in range(args.repeat):
+            _reset_counters()
+            if args.quiet:
+                sys.stdout = devnull
+            try:
+                samples.append(load_once(tp, fname, test_dir))
+            except RecursionError:
+                if args.quiet:
+                    sys.stdout = real_stdout
+                print(f"file      : {fname}")
+                print("ERROR     : RecursionError during load — the include "
+                      "nesting is too deep for the recursive loader.\n"
+                      "            (each include level costs ~10 stack frames; "
+                      "raise sys.setrecursionlimit to probe further.)")
+                return 2
+            except Exception as e:  # noqa: BLE001 - report, don't crash the bench
+                if args.quiet:
+                    sys.stdout = real_stdout
+                print(f"file      : {fname}")
+                print(f"ERROR     : load failed: {type(e).__name__}: {e}")
+                return 2
+            finally:
+                if args.quiet:
+                    sys.stdout = real_stdout
+            last_counters = dict(_C)
+    finally:
+        if eval_proc is not None:
+            eval_proc.stop()
+            eval_proc.join()
+        if args.quiet:
+            devnull.close()
+
+    initial = [s[0] for s in samples]
+    loadtest = [s[1] for s in samples]
+    build = [s[2] for s in samples]
+    total = [sum(s) for s in samples]
+
+    def stat(xs):
+        return min(xs), statistics.median(xs)
+
+    print(f"file      : {fname}")
+    print(f"repeats   : {args.repeat}   (showing  min | median, seconds)")
+    print(f"{'phase':<10}{'min':>12}{'median':>12}")
+    for name, xs in (("initial", initial), ("loadtest", loadtest),
+                     ("build", build), ("total", total)):
+        mn, md = stat(xs)
+        print(f"{name:<10}{mn:>12.4f}{md:>12.4f}")
+    if last_counters:
+        print("counters  (last run):")
+        print(f"  templates : {last_counters['tpl_n']:>7d} calls   "
+              f"{last_counters['tpl_t']:>8.4f}s  (exclusive: jinja compile+render+tempfile)")
+        print(f"  yaml      : {last_counters['yaml_n']:>7d} parses  "
+              f"{last_counters['yaml_t']:>8.4f}s  (inclusive of nested includes)")
+
+
+if __name__ == "__main__":
+    sys.exit(main() or 0)

test/benchmark/run.sh

@@ -0,0 +1,49 @@
+#!/bin/bash
+# Load-time benchmark driver: generate synthetic .tum trees and time the
+# testium load pipeline on them, using the project venv.
+#
+# Usage:
+#   ./test/benchmark/run.sh                     # default matrix (all profiles)
+#   ./test/benchmark/run.sh <profile> <size>    # one profile at one size
+#   REPEAT=10 ./test/benchmark/run.sh repeat 2000
+#
+# Profiles: flat includes repeat jinja deep mix   (see gen_bench_test.py)
+#
+# Generated trees go under test/benchmark/cases/ (git-ignored). The numbers
+# are wall-clock; run on an otherwise idle machine and compare min values.
+set -e
+
+SCRIPT_DIR="$(realpath "$(dirname "$(readlink -f "$0")")")"
+PROJECT_DIR="$(realpath "$SCRIPT_DIR/../..")"
+VPY="$PROJECT_DIR/test/tmp/.venv/bin/python3"
+CASES="$SCRIPT_DIR/cases"
+REPEAT="${REPEAT:-5}"
+
+if [ ! -x "$VPY" ]; then
+    echo "ERROR: project venv not found at $VPY — run ./run.sh once to create it." >&2
+    exit 1
+fi
+
+bench() {
+    local profile="$1" size="$2"
+    local out="$CASES/${profile}_${size}"
+    local main
+    main="$("$VPY" "$SCRIPT_DIR/gen_bench_test.py" --profile "$profile" --size "$size" --out "$out")"
+    echo "===== profile=$profile size=$size ====="
+    "$VPY" "$SCRIPT_DIR/load_bench.py" --repeat "$REPEAT" --quiet "$main"
+    echo
+}
+
+if [ $# -eq 2 ]; then
+    bench "$1" "$2"
+    exit 0
+fi
+
+# Default matrix. 'deep' is kept small: the recursive loader hits Python's
+# recursion limit around ~90 nested include levels.
+bench flat     2000
+bench includes 1000
+bench repeat   1000
+bench jinja    2000
+bench deep     40
+bench mix      300

test/validation/items/console/test.tum

@@ -94,6 +94,17 @@
 {% endif %}
         - read_until: {expected: endOfCmd, timeout: 1, process_result: "'Hello' in r'''$(result)''' and 'PASS' in r'''$(result)''' "}
 
+{% if os == "Linux" %}
+- console:
+    name: Console runs on host (not the Flatpak sandbox)
+    doc: Regression guard for the 0.2.1 Flatpak bug (term console spawned inside the sandbox instead of on the host). /.flatpak-info exists only inside the sandbox, so the host-only marker is emitted (and matched by read_until) ONLY when the shell really runs on the host. On a broken Flatpak the marker never appears, read_until times out and the item FAILS. The marker is built at runtime ($M) so it is never present in the command line itself. Passes on every other channel.
+    console_name: term
+    key: $(test)_PASS
+    steps:
+        - writeln: 'test -e /.flatpak-info && M=SANDBOX || M=HOST; echo "console_host_check_$M"'
+        - read_until: {expected: console_host_check_HOST, timeout: 5}
+{% endif %}
+
 - console:
     name: Console closure
     execute_on_stop: true

test/validation/items/jsonrpc/test.tum

@@ -20,7 +20,7 @@
             console_name: jrpces
             key: $(test)_PASS
             steps:
-                - writeln: python3 {{include_directory}}/jrpc_echo_server.py -c {{include_directory}}/jrpces.ini
+                - writeln: '"$(python_bin)" {{include_directory}}/jrpc_echo_server.py -c {{include_directory}}/jrpces.ini'
                 - read_until: {expected: ready, timeout: 5}
 
         - console:

test/validation/post_execution.py

@@ -89,7 +89,7 @@ def exec():
         junit_report = report.replace(".sqlite", f"-{test}.xml")
         print(junit_report)
         _prepare_file_to_save(junit_report)
-        with open(junit_report, "w") as f:
+        with open(junit_report, "w", encoding="utf-8") as f:
             f.write(TestSuite.to_xml_string([ts]))
 
     # cleanup

test/validation/run.bat

@@ -89,6 +89,13 @@ REM Reports are stamped with the mode so successive runs don't clobber each othe
 
 SET "TAIL=-b -d "python_bin=%VENV_PYTHON%" -d "validation_report_file=validation-%MODE%" -- "%SCRIPT_DIR%\main.tum"%EXTRA%"
 
+REM The report-exporter plugin (items\report_plugin) is a pip entry-point
+REM package. It must live in the *testium* environment, so it is installed into
+REM the source/wheel venvs below. A frozen PyInstaller binary cannot see
+REM externally-installed plugins, so report_plugin is expected to be skipped
+REM there (same as Linux pyinstaller mode).
+SET "FAKE_EXPORTER=%SCRIPT_DIR%\fake_exporter"
+
 REM ---------- per-mode launcher ----------------------------------------------
 
 echo -- validation mode: %MODE%
@@ -100,8 +107,25 @@ echo ERROR: unknown --mode '%MODE%'. Expected: source ^| wheel ^| pyinstaller.
 exit /b 1
 
 :MODE_SOURCE
-call "%PROJECT_DIR%\run.bat" %TAIL%
-exit /b %ERRORLEVEL%
+REM Run testium from src\ in a dedicated venv set up here. We do NOT delegate to
+REM the project's run.bat: that one launches the GUI and does not forward its
+REM arguments, so the suite would never run head-less.
+SET "TESTIUM_VENV=%PROJECT_DIR%\test\tmp\testium_venv"
+IF NOT EXIST "%TESTIUM_VENV%" (
+    echo Creating testium venv at %TESTIUM_VENV%
+    %PYTHON_EXE% -m venv "%TESTIUM_VENV%"
+    IF !ERRORLEVEL! NEQ 0 (
+        echo ERROR while creating the testium venv.
+        exit /b 1
+    )
+    call "%TESTIUM_VENV%\Scripts\pip" install --quiet --upgrade pip
+    call "%TESTIUM_VENV%\Scripts\pip" install --quiet -r "%PROJECT_DIR%\src\requirements.txt"
+    REM language-server extra so `testium lsp` works from source (lsp_check.py)
+    call "%TESTIUM_VENV%\Scripts\pip" install --quiet "pygls>=1.3"
+)
+call "%TESTIUM_VENV%\Scripts\pip" install --quiet -e "%FAKE_EXPORTER%"
+SET CMD="%TESTIUM_VENV%\Scripts\python.exe" "%PROJECT_DIR%\src\testium"
+GOTO LAUNCH
 
 :MODE_WHEEL
 SET "WHEEL=%PROJECT_DIR%\dist\testium-%VERSION%-py3-none-any.whl"
@@ -115,10 +139,13 @@ IF NOT EXIST "%WHEEL_VENV%" (
     echo Creating wheel venv at %WHEEL_VENV%
     %PYTHON_EXE% -m venv --system-site-packages "%WHEEL_VENV%"
     call "%WHEEL_VENV%\Scripts\pip" install --quiet --upgrade pip
-    call "%WHEEL_VENV%\Scripts\pip" install --quiet "%WHEEL%"
+    REM install with the [lsp] extra so the wheel channel is validated in its
+    REM language-server-capable form (pulls pygls), matching `pip install testium[lsp]`.
+    call "%WHEEL_VENV%\Scripts\pip" install --quiet "%WHEEL%[lsp]"
 )
-"%WHEEL_VENV%\Scripts\python.exe" -m testium %TAIL%
-exit /b %ERRORLEVEL%
+call "%WHEEL_VENV%\Scripts\pip" install --quiet -e "%FAKE_EXPORTER%"
+SET CMD="%WHEEL_VENV%\Scripts\python.exe" -m testium
+GOTO LAUNCH
 
 :MODE_PYI
 SET "PYI_BIN=%PROJECT_DIR%\dist\testium-%VERSION%.exe"
@@ -127,5 +154,22 @@ IF NOT EXIST "%PYI_BIN%" (
     echo ERROR: PyInstaller binary not found in %PROJECT_DIR%\dist -- run build_all.sh first.
     exit /b 1
 )
-"%PYI_BIN%" %TAIL%
+SET CMD="%PYI_BIN%"
+GOTO LAUNCH
+
+REM ---------- launch ----------------------------------------------------------
+
+:LAUNCH
+echo -- launch: %CMD%
+
+REM LSP check (this exact channel): `schema` must keep its nested actions and
+REM `lsp` must answer initialize. Mirrors run.sh; aborts the run on failure.
+echo -- LSP check (%MODE%)
+"%VENV_PYTHON%" "%SCRIPT_DIR%\lsp_check.py" %CMD%
+IF !ERRORLEVEL! NEQ 0 (
+    echo ERROR: LSP check failed for mode %MODE%.
+    exit /b 1
+)
+
+%CMD% %TAIL%
 exit /b %ERRORLEVEL%