Adding a some TUM template used to validate the JSON schema

DESIGN.md

@@ -114,20 +114,11 @@ To add a new API call usable from subprocesses:
 ### External interpreter resolution (`bins.py`)
 `src/testium/interpreter/utils/bins.py` — single source of truth for the paths to the external Python and Lua interpreters used by subprocesses.
 
-- `python_bin()` / `lua_bin()` : resolve and cache. The cache is keyed by `(name, override)` so that a later change to `gd[python_bin]` (typically when a `param.yaml` sets the key) triggers a re-resolution on the next lookup instead of returning the stale auto-discovered path. Falls back to discovery on PATH (candidates: `python3`/`python` and `lua`/`lua5.5`/`lua5.4`/`lua5.3`/`lua5.2`/`lua5.1`).
+- `python_bin()` / `lua_bin()` : resolve once, cache in memory. User can override via the `python_bin` / `lua_bin` global dict keys (typically populated from the YAML config). Falls back to discovery on PATH (candidates: `python3`/`python` and `lua`/`lua5.5`/`lua5.4`/`lua5.3`/`lua5.2`/`lua5.1`).
 - `ensure(*names)` : called by `TestSet._validate_runtime_deps()` at test load. Always requires `python` (the eval engine always runs); requires `lua` only if a `lua_func` item is in the tree. Fails fast with a clear error citing tried candidates and override key.
 
 Engines (`PyProcessBase`, `LuaProcessBase`, `EvalExecEngine`) call `bins.python_bin()`/`bins.lua_bin()` themselves — call sites never pass an explicit binary path.
 
-#### Override-timing contract (`apply_overrides`)
-`bins.python_bin()` is called for the **first** time inside `eval_process_init()` (the long-lived inline-`<| … |>` subprocess), which happens **before** the YAML param files are loaded. To make `-d python_bin=…` and the GUI `python_bin` preference take effect for `eval_proc` itself, `process.py:run()` applies them to gd **before** `eval_process_init()` via the `apply_overrides()` helper extracted from `update_global()`. The post-load `update_global()` call then re-applies the same overrides (after `prepare_global()` clears gd), keeping the gd value in sync with the cached resolution.
-
-| Override source | `eval_proc` | `py_func` / `cycle` / `post_exec` |
-|---|---|---|
-| `-d python_bin=…` (CLI) | ✅ | ✅ |
-| GUI `python_bin` preference | ✅ | ✅ |
-| `python_bin: …` in `param.yaml` | ❌ (eval_proc already started) | ✅ (cache re-resolves on key change) |
-
 ## Key files
 
 | Path | Role |
@@ -224,66 +215,27 @@ Four distribution channels coexist, all sharing the single `src/testium/` packag
 The `.deb` work-in-progress lives in `package/deb/`:
 - `test_distro.sh debian:bookworm | debian:trixie | ubuntu:24.04` spins up a Docker/Podman container, reports system package availability, falls back to pip for what's missing (`pyside6` on bookworm/ubuntu, `telnetlib3`, `junit_xml`), runs the validation suite. Currently green on the three targets.
 
-### Building all channels (`build_all.sh`)
-
-`build_all.sh` builds every artifact into `dist/` (manual PDF, wheel, PyInstaller binary, Flatpak bundle, AppImage). It reuses `scripts/build_env.sh` + `set_env.sh` so the venv at `test/tmp/.venv` stays the single source of Python deps; `build`/`pyinstaller`/`sphinx`/`linuxdoc` (and `pygls`, via the `[lsp]` extra) are installed there on demand. A step is skipped if its artifact already exists; `--clean` forces a rebuild.
-
-- **Parallelism (default).** A serial *prep* phase does everything that writes the shared venv (the `pip install`s) plus the Flatpak runtime install and the wheel (the AppImage installs it). Then manual + PyInstaller + Flatpak + AppImage build concurrently — they only *read* the venv, so there is no concurrent-pip race. Per-step output goes to `dist/.build-logs/<step>.log`; results print in completion order (`wait -n`), and a failing step's log is dumped at the end. `--serial` builds one at a time. Ctrl+C is trapped to kill each job's whole process tree (subshell + grandchildren: podman container, flatpak-builder, pyinstaller), so no orphans survive.
-- **`--ram` (slow/flash storage).** Redirects the build scratch to `/dev/shm` and skips UPX, a large win when building from a USB stick / SD card (I/O-bound on flash): `TMPDIR` + `PIP_CACHE_DIR`, the PyInstaller `--workpath` (`PYI_WORKPATH`), and a tmpfs bind-mount at the in-container AppImage AppDir (`APPIMAGE_APPDIR_TMPFS`); UPX is disabled via `TESTIUM_NO_UPX` (read by the `.spec`). **Flatpak is excluded** — `flatpak-builder` mounts its state dir with `rofiles-fuse` and FUSE cannot mount on `/dev/shm` (`fusermount: Permission denied`), so it builds on disk. Each `package/*/build.sh` honours these env vars with on-disk defaults, so behaviour is unchanged without `--ram`; the tmpfs scratch is freed on exit. On a RAM-limited machine combine with `--serial`.
-
 ### Host-only py_func / lua_func in sandboxed bundles (Flatpak, AppImage)
 
-The bundled Python (Flatpak's runtime python, AppImage's `python3.11`) is reserved for the **main process only**. Subprocesses (`py_func`, `lua_func`, `git`, the `run` item's sub-instance) must use the host's interpreters and tools so user-installed modules (pyserial, junit_xml, …) are visible. This is enforced by `interpreter/utils/bins.py`:
+The bundled Python (Flatpak's runtime python, AppImage's `python3.11`) is reserved for the **main process only**. Subprocesses (`py_func`, `lua_func`, `git`) must use the host's interpreters and tools so user-installed modules (pyserial, junit_xml, …) are visible. This is enforced by `interpreter/utils/bins.py`:
 
 - `_in_flatpak()` (checks `/.flatpak-info`) and `_in_appimage()` (checks `APPIMAGE` env var) detect the sandbox.
-- **Flatpak**: the sandbox glibc/ABI is incompatible with arbitrary host shared libraries, so we **cannot** run host binaries inside the Flatpak runtime — `LD_LIBRARY_PATH` injection trips a `_dl_call_libc_early_init` assertion. The supported way out is `flatpak-spawn --host`, a stub on `$PATH` inside every Flatpak that proxies an `exec` over D-Bus to the host's `org.freedesktop.Flatpak` service. The manifest grants `--talk-name=org.freedesktop.Flatpak` so the call is allowed. Helpers:
-  - `flatpak_host_spawn(interp, args, host_cwd, extra_env=…)` builds the spawn command vector with a curated set of forwarded env vars (`HOME`, `USER`, `DISPLAY`, `DBUS_SESSION_BUS_ADDRESS`, …) plus any explicit overrides.
-  - `_get_host_testium_path()` returns a path to the testium package the host can read. In Flatpak the package lives under `/app/lib/testium` which the host cannot see, so the package is staged once per process under `/tmp/testium_host_*` (`/tmp` is shared) and reused. In source / wheel / PyInstaller installs under `$HOME` the original path is returned untouched.
-  - `_which_host_flatpak(name)` resolves a binary by spawning `command -v` on the host (or `test -x` for absolute paths) — sandbox-visible probing under `/run/host/...` is unreliable (only `host-os` is mounted; user paths like `/scratch` aren't there).
-  - `_python_version()` and `_lua_version()` go through `_run_probe()` which dispatches to `flatpak-spawn` in Flatpak so validation happens against the actual host interpreter.
-  - `py_process.py` / `lua_process.py` `start()` use `flatpak_host_spawn` with `host_cwd = _get_host_testium_path()[+/lua_func]` and forward `PYTHONPATH` / `LUA_PATH` / `LUA_CPATH` / `PATH` as `--env=` arguments.
-  - The `run` item's `_testium_launch_cmd()` prefixes `flatpak run org.testium.Testium` with `flatpak-spawn --host` so the sub-instance is launched by the host's `flatpak` CLI, not by an unworkable in-sandbox `flatpak` binary.
-- **AppImage**: we are directly on the host filesystem, so the regular discovery on `/usr/local/bin`, `/usr/bin`, `/bin` suffices. `apply_host_libs(env)` strips `$APPDIR`-prefixed entries from `LD_LIBRARY_PATH` / `PYTHONPATH` / `PATH` and drops `PYTHONHOME` so the host Python doesn't try to load the bundled stdlib/site-packages.
-- User overrides (`python_bin`/`lua_bin` in globdict): in Flatpak, both bare names and absolute paths go through `_which()` so they are validated on the host side (the sandbox can't see e.g. `/scratch/...`). Outside Flatpak, absolute paths are accepted as-is and bare names go through PATH discovery.
-- If the host has no python3/lua, `ensure()` raises `ETUMRuntimeError` at test load with the candidate list — no silent fallback to a bundled interpreter.
+- `_which(name)` probes only host bin dirs in those modes:
+  - Flatpak: `/run/host/usr/{local/,}bin`, `/run/host/bin` (host mounted via `--filesystem=host-os`).
+  - AppImage: `/usr/local/bin`, `/usr/bin`, `/bin` (we are directly on the host filesystem).
+  - If the host has no python3/lua, `ensure()` raises `ETUMRuntimeError` at test load with the candidate list — no silent fallback to a bundled interpreter.
+- User overrides (`python_bin`/`lua_bin` in globdict): bare names are resolved through `_which()` (host-only), absolute paths are accepted as-is.
+- `apply_host_libs(env)` is called by `py_process.py` / `lua_process.py` on the env passed to Popen:
+  - Flatpak: prepends host lib dirs to `LD_LIBRARY_PATH` so the dynamic linker finds host `.so`'s.
+  - AppImage: strips `$APPDIR`-prefixed entries from `LD_LIBRARY_PATH` / `PYTHONPATH` / `PATH` and drops `PYTHONHOME`, so the host Python doesn't try to load the bundled stdlib/site-packages.
+- `apply_host_lua_paths(env)` (Flatpak only) prepends `/run/host/usr/{lib,share}/lua/X.Y` to `LUA_PATH` / `LUA_CPATH` so `cjson`, `socket`, etc. resolve. Must be called **after** user `lua_env` overrides so host paths win. AppImage relies on host Lua's compiled-in defaults.
 - `py_process.py` additionally pops `PYTHONUSERBASE` (set to `/var/data/python` by the Flatpak runtime, which would hide `~/.local/lib/...`).
 
-### Declarative test item parameters
-
-Each `TestItem` subclass declares its accepted parameters as a class attribute `PARAMS = ParamSet(Param(...), ...)` (`interpreter/utils/param_decl.py`). The descriptor carries the parameter name, *kind* (`SCALAR` — the default and may be omitted; `LIST`; `BLOCK`; `Enum("a", "b", ...)`), `required` flag, `default`, and free-form `doc`. There is **no Python type** in the descriptor on purpose: most parameter values are expressions (`$(...)` / `<| ... |>`) whose effective type is only known after expansion, so a static type would be misleading. Post-expansion `validate=lambda v: ...` callbacks are available as an opt-in for the rare cases where a runtime check is warranted (e.g. a specific format).
-
-`TestItem.COMMON_PARAMS` (in `test_item.py`) declares the 14 parameters accepted by every item: `name`, `doc`, `skipped`, `key`, `stop_on_failure`, `execute_on_stop`, `process_result`, `store_result`, `expected_result`, `no_fail`, `report`, `condition`, `steps`, and the internal `seq_filename` injected by the loader. The base class concatenates `COMMON_PARAMS + subclass.PARAMS` in `_validate_declared_params()` and:
-
-- emits a `tm.print_warn(...)` listing the accepted names when an unknown key appears in the user YAML (catches typos like `param_filee`);
-- raises `ETUMSyntaxError` (with the `.tum` source as context) when a `required=True` param is missing.
-
-Validation is **opt-in per subclass**: while a subclass keeps `PARAMS = None` (the base-class default), the check is skipped entirely. This kept the migration incremental — items can be visited one by one without forcing a big-bang change. All structured items have been migrated; only the "unstructured-body" classes (`TestItemConsoleWrite`/`WriteLn` which carry the message as the raw value, `TestItemPlotActionAdd`/`Export` which take arbitrary plot-data keys, `TestItemUnittestElement` which is internally instantiated with `dict_item=None`) intentionally remain unvalidated.
-
-Diagnostics are currently **warnings** for unknown params so an out-of-tree `.tum` with a pre-existing typo doesn't suddenly fail. The flip to a hard error is a one-line change in `_validate_declared_params()` once the user is comfortable.
-
-Action items follow the same declarative principle. A `TestItemActions` parent (`console`, `plot`, `json_rpc`) declares its nested actions as a class attribute `ACTIONS = {yaml_key: action_class}` (e.g. `{"open": TestItemConsoleOpen, "write": …}`), mirroring `PARAMS`. The base `TestItemActions.__init__` seeds `self.action_classes` from `type(self).ACTIONS`; the imperative `register_actions(**…)` method is retained only as an escape hatch for actions that can't be known at class-definition time (none today). Because the action classes are always defined above their parent in the module, the class-level dict resolves without forward-reference gymnastics.
-
-The schema is the realized source of truth for the LSP server (`testium lsp`), the `testium schema` CLI dump, and future auto-generated manual sections: `ParamSet.to_schema()` returns the JSON-Schema-shaped representation, and `lsp/schema.py` reads both `PARAMS` and `ACTIONS` **purely from class attributes** — no `inspect.getsource`/AST parsing. This is what lets the full schema (including nested actions) survive a frozen PyInstaller build where the `.py` source isn't on disk.
-
-### Language server (`testium lsp`) across channels
-
-The `testium_assist` editor extension is a thin LSP client that spawns `testium lsp` and talks JSON-RPC over stdio, so the language server must work from *every* distribution channel. Two requirements:
-
-1. **`pygls` (+ `lsprotocol`, `cattrs`, `attrs`, `typing_extensions`) must be bundled.** It is the pyproject `[lsp]` extra (kept optional so a plain `pip install testium` stays lean), wired into each full-app channel: `build_env.sh` installs it into the shared `test/tmp/.venv` (covers **source run** and the **PyInstaller** build env); the **AppImage** installs the wheel as `…whl[lsp]`; the **Flatpak** adds a `python3-lsp` pip module (network-at-build, consistent with the manifest's global `--share=network`); the **PyInstaller** `.spec` force-collects the submodules via `collect_submodules` + explicit `hiddenimports` (including the lazily-imported `lsp`, `lsp.server`, `lsp.schema`).
-2. **The schema must build without source** — handled by the declarative `PARAMS`/`ACTIONS` above; PyInstaller is the only channel that strips `.py` source, and it no longer matters.
-
-`test/validation/lsp_check.py` enforces both per channel: `run.sh` calls it before launching the suite, asserting that `<channel> schema` returns JSON whose `console`/`plot`/`json_rpc` items still carry their actions, and that `<channel> lsp` answers an `initialize` request with capabilities (and never reports the pygls dependency missing). So `./test/validation/run.sh --mode flatpak|pyinstaller|appimage` now fails loudly if a channel ships a broken or pygls-less language server.
-
 ### Version reporting (`interpreter/utils/version.py`)
 
 Both Flatpak and AppImage export `TESTIUM_VERSION` from a launcher (Flatpak: launcher script in `org.testium.Testium.yaml`; AppImage: `runtime.env` in `AppImageBuilder.yml`). `get_testium_version()` checks `/.flatpak-info` / `APPIMAGE` and reads `TESTIUM_VERSION` rather than relying on package metadata or repo introspection.
 
 ## Recent fixes / notable changes
-- `build_all.sh`: builds the four heavy channels in parallel (serial prep for the shared venv + wheel), results in completion order, Ctrl+C kills the whole job tree; `--ram` puts the build scratch on tmpfs (`/dev/shm`) + skips UPX for fast builds on USB/SD storage (Flatpak excluded — rofiles-fuse can't mount tmpfs). See the "Building all channels" section.
-- LSP across packaging channels: `testium lsp` (and the `testium_assist` editor extension that spawns it) now works from source, wheel, PyInstaller, Flatpak and AppImage. Two enablers — (1) action items declare a class-level `ACTIONS = {key: class}` registry (like `PARAMS`), so `lsp/schema.py` builds the full schema from class attributes with no `inspect.getsource`/AST (which broke under frozen PyInstaller); (2) the `[lsp]` extra (pygls) is wired into every full-app channel. `test/validation/lsp_check.py`, run by `run.sh` before the suite, asserts per-channel that `schema` keeps its actions and `lsp` answers `initialize`. See the matching architecture sections.
-- Declarative test item parameters (v0.2): each `TestItem` subclass exposes a `PARAMS = ParamSet(...)` class attribute consumed by the base `__init__`. Catches unknown YAML keys (typo warnings listing the accepted names) and missing required params (load-time errors with `.tum` context). Lays the schema foundation for a future LSP server and auto-generated manual sections. See the matching architecture section.
-- Flatpak: `py_func` / `lua_func` / `run` sub-instance now execute on the host via `flatpak-spawn --host`. The previous attempt to inject host lib dirs into the sandbox's `LD_LIBRARY_PATH` was abandoned — host shared libs are ABI-incompatible with the Flatpak runtime's glibc and would trip `_dl_call_libc_early_init`. The manifest gained `--talk-name=org.freedesktop.Flatpak` so the spawn proxy call is allowed. The testium package is staged once per process under `/tmp` (shared with the host) so the host interpreter can locate `py_func` / `lua_func`.
-- Validation suite: single entry point with `--mode source|wheel|pyinstaller|flatpak|appimage` to validate every packaging channel against the same items. Per-mode report filenames prevent clobbering.
 - Restructure: single `src/testium/` Python package (was 4 sibling top-levels: `testium`, `lib`, `py_func`, `lua_func`). `lib/` → `runtime/`, `libs/` → `api/`. `pip install` now produces a clean `site-packages/testium/` with no top-level pollution; `.lua` files travel via `package_data`.
 - `bins.py`: centralised resolution + cache of external `python3` / `lua` binaries. Replaces the scattered `tm.gd("python_bin")`/`tm.gd("lua_bin")` dance and the duplicated discovery logic in `py_process.py`/`lua_process.py`. Validates at test load via `TestSet._validate_runtime_deps()` so missing interpreters fail fast.
 - Subprocess API contract: user scripts in `py_func`/`lua_func` use the JSON-RPC bridge (`py_func.tm` / Lua `tm`) — never `api.testium` / `interpreter.*` directly. `SUPPORTED_API` extended with `OS`, `get_main_dir`, `init_timestamp`, `timestamp`, `timestamp_as_sec` so subprocess scripts have the same surface as main-process code.
@@ -309,20 +261,12 @@ Both Flatpak and AppImage export `TESTIUM_VERSION` from a launcher (Flatpak: lau
 - `unittest` item: renamed from `unittest_file`.
 - GUI test tree: check and fold state preserved across same-file reloads.
 - Licence: EUPL-1.2.
-- Interpreter override timing: `apply_overrides()` extracted from `update_global()` and called by `process.py:run()` before `eval_process_init()`, so `-d python_bin=…` / GUI prefs reach `bins.python_bin()` on its first lookup. `bins._resolve()` cache is now keyed by `(name, override)` so later `param.yaml` changes are picked up by subsequently constructed engines.
 
 ## Validation tests
-Located in `test/validation/`. Two entry points:
+Located in `test/validation/`. Run with `-b` flag:
 ```
-./test/validation/run.sh [clean] [--mode MODE] [extra args]   # wrapper — uses a dedicated venv (see below)
-./run.sh -b -- test/validation/main.tum                       # direct — testium's own python is used for test execution
+./run.sh -b -- test/validation/main.tum
 ```
-The same item set is reused across every packaging channel — `--mode source|wheel|pyinstaller|flatpak|appimage` selects which testium binary launches the suite (`source` is the default, invoking the project's `run.sh`). Each mode stamps its results into a distinct report file (`validation-<mode>.sqlite`, `validation-<mode>-<item>.xml`) so successive runs in different modes don't clobber each other. Prerequisites (PyInstaller binary built, Flatpak bundle installed, …) are checked before launch with a hint pointing at `build_all.sh`. On Windows only `source`, `wheel`, `pyinstaller` are supported.
-
-The `run.sh` / `run.bat` wrappers create a dedicated **host** Python venv at `${TMPDIR:-/tmp}/testium-validation-venv` (Linux) or `%TEMP%\testium-validation-venv` (Windows), with `--system-site-packages` + `pip install junit-xml`, and run the suite with `-d python_bin=…` so every test-execution subprocess (eval_proc, py_func, cycle, post_exec) runs inside that venv. testium itself keeps running in its own environment for the chosen mode. The venv is shared across modes because every test-execution subprocess ends up on the host either directly (source/wheel/pyinstaller/appimage) or via `flatpak-spawn --host` (flatpak). `clean` as the first argument recreates the venv. `wheel` mode also creates a separate `testium-wheel-venv-<v>` to hold the installed package.
-
-The `venv` item (`test/validation/items/venv/`) asserts that the override actually took effect: `python_bin` is set, `sys.executable` matches it, `sys.prefix == dirname(dirname(python_bin))`, and `sys.prefix != sys.base_prefix` (the last marker catches the case where `python_bin` happens to be a system interpreter, which path-equality alone would miss because the venv's `bin/python3` is a symlink to the host). Both `eval_proc` (inline `<| … |>`) and `py_func` paths are exercised.
-
 Parallel item tests: `test/validation/items/parallel/test.tum`
 
 ## Dependencies

README.md

@@ -27,27 +27,6 @@ Pre-built artifacts are published at
   runnable directly, no Python installation required on the host. Lua
   support still needs a system `lua` interpreter and the `lua-socket` /
   `lua-cjson` modules.
-* **AppImage** (`Testium-<version>-x86_64.AppImage`) — single-file
-  Linux binary, runnable directly:
-
-  ```sh
-  chmod +x Testium-*-x86_64.AppImage
-  ./Testium-*-x86_64.AppImage
-  ```
-
-  Requires `libfuse2` on the host (FUSE 2 — distinct from `fuse3`, which
-  most distros now ship by default):
-
-  | Distro | Package |
-  |--------|---------|
-  | Arch / CachyOS / Manjaro | `fuse2` |
-  | Debian trixie / Ubuntu 24.04+ | `libfuse2t64` |
-  | Debian bookworm / Ubuntu 22.04 | `libfuse2` |
-  | Fedora | `fuse-libs` |
-
-  If you can't install libfuse2 (e.g. minimal container), prefix the
-  invocation with `APPIMAGE_EXTRACT_AND_RUN=1` — the AppImage will
-  self-extract to `/tmp` on each run instead of FUSE-mounting.
 * **Flatpak bundle** (`testium.flatpak`) — install with:
 
   ```sh
@@ -62,9 +41,6 @@ Pre-built artifacts are published at
   `testium` command is available in the terminal (requires `~/.local/bin` in
   `PATH`, which most modern distributions provide by default).
 
-Every channel ships the language server, so `testium lsp` (see
-[Editor support](#editor-support)) works out of the box from any of them.
-
 ## Quick start
 
 From a checkout of the repository:
@@ -106,29 +82,6 @@ python3 src/testium               # GUI
 python3 src/testium -b mytest.tum # batch
 ```
 
-## Editor support
-
-testium ships a Language Server Protocol (LSP) server that gives `.tum` files
-completion of item types, hover documentation, and an outline view in any
-LSP-capable editor:
-
-```sh
-testium lsp        # speaks LSP over stdio; an editor's LSP client drives it
-testium schema     # dumps the item/parameter schema as JSON (what the LSP serves)
-```
-
-The server is bundled in every pre-built release (wheel, binary, Flatpak,
-AppImage). For a source / wheel install, pull the language-server extra:
-
-```sh
-pip install 'testium[lsp]'                 # from PyPI / a wheel
-pip install -e /path/to/testium/src[lsp]   # from a source checkout
-```
-
-A VSCode / VSCodium client extension (`testium_assist`) wraps `testium lsp`;
-the schema is built from testium itself, so new item types and parameters
-appear in the editor on the next testium upgrade with no client change.
-
 ## Troubleshooting
 
 ### `wl_proxy_marshal_flags` symbol error

build_all.sh

@@ -1,5 +1,5 @@
 #!/bin/bash
-# Build every distribution channel of testium:
+# Build every distribution channel of testium, in order:
 #   1. Manual PDF         -> dist/testium-manual-<v>.pdf
 #   2. Wheel              -> dist/testium-<v>-py3-none-any.whl   (PEP 427 name)
 #   3. PyInstaller binary -> dist/testium-<v>
@@ -8,24 +8,6 @@
 # release_note.txt is copied to dist/ up front (with a warning if it has no
 # entry for the current version).
 #
-# By default, a step is skipped if its artifact already exists in dist/.
-# Pass --clean to remove existing dist/ artifacts and rebuild everything.
-#
-# Parallelism: the wheel is built first (the AppImage installs it), then the
-# manual, PyInstaller, Flatpak and AppImage builds run concurrently. The shared
-# venv at test/tmp/.venv is only WRITTEN during the serial prep phase (the
-# `pip install` of build/sphinx/pyinstaller); the parallel builds only read it,
-# so there is no concurrent-pip race. Pass --serial to build one step at a time
-# (useful when debugging or on a resource-constrained machine). Per-step output
-# of the parallel phase is captured under dist/.build-logs/<step>.log and the
-# log of any failing step is printed at the end.
-#
-# Pass --ram to redirect the per-channel build scratch (PyInstaller workpath,
-# AppImage AppDir) and TMPDIR/PIP_CACHE_DIR to /dev/shm, and skip UPX. Big
-# speedup on slow/flash storage. Flatpak is excluded (its rofiles-fuse can't
-# mount on /dev/shm), so it still builds on disk. On a RAM-limited machine
-# combine with --serial (e.g. ./build_all.sh --ram --serial).
-#
 # All artifacts are collected (copied) under <repo>/dist/. Original outputs in
 # src/dist/, package/*/dist/, doc/manual/ are left in place. Wheel and AppImage
 # keep their original names (which already contain the version); manual,
@@ -33,39 +15,17 @@
 #
 # Re-uses scripts/build_env.sh and scripts/set_env.sh — the same pair invoked
 # by run.sh — so the venv at test/tmp/.venv stays the single source of Python
-# dependencies. `build`, `pyinstaller`, `sphinx` and `linuxdoc` are installed
-# into that venv on demand if not already there. Flatpak and AppImage build in
-# their own container/sandbox; their build.sh scripts have their own toolchain
-# checks.
+# dependencies. `build` and `pyinstaller` are installed into that venv on
+# demand if not already there. Flatpak and AppImage build in their own
+# container/sandbox; their build.sh scripts have their own toolchain checks.
 
 set -e
 
-CLEAN=0
-SERIAL=0
-RAM=0
-for arg in "$@"; do
-    case "$arg" in
-        --clean|-c) CLEAN=1 ;;
-        --serial)   SERIAL=1 ;;
-        --ram)      RAM=1 ;;
-        *) echo "Unknown option: $arg" >&2; exit 1 ;;
-    esac
-done
-
 SCRIPT_DIR=$(realpath "$(dirname "$0")")
 VERSION=$(cat "$SCRIPT_DIR/src/VERSION")
 DIST_DIR="$SCRIPT_DIR/dist"
 mkdir -p "$DIST_DIR"
 
-if [ "$CLEAN" -eq 1 ]; then
-    echo "-- clean: removing existing dist artifacts for version $VERSION"
-    rm -f "$DIST_DIR/testium-manual-${VERSION}.pdf"
-    rm -f "$DIST_DIR"/testium-${VERSION}-*.whl
-    rm -f "$DIST_DIR/testium-${VERSION}"
-    rm -f "$DIST_DIR/testium-${VERSION}.flatpak"
-    rm -f "$DIST_DIR"/Testium-${VERSION}-*.AppImage
-fi
-
 # Release note: copy it to dist/ and warn (but don't fail) if it has no entry
 # for the current version.
 RELEASE_NOTE_SRC="$SCRIPT_DIR/release_note.txt"
@@ -82,30 +42,8 @@ export REQ_PATH="$SCRIPT_DIR/src/requirements.txt"
 bash "$SCRIPT_DIR/scripts/build_env.sh"
 source "$SCRIPT_DIR/scripts/set_env.sh"
 
-# ---------- RAM mode: put build scratch on tmpfs (--ram) ----------------------
-# On slow storage (USB stick, SD card) the per-channel build dirs and temp
-# churn dominate. --ram redirects the PyInstaller workpath, the AppImage AppDir
-# and TMPDIR/PIP_CACHE_DIR to /dev/shm, and skips UPX. Flatpak is intentionally
-# NOT moved: flatpak-builder mounts its state dir with rofiles-fuse, and FUSE
-# can't mount on /dev/shm (fusermount: Permission denied) — so it builds on
-# disk. The tmpfs scratch is freed on exit.
-if [ "$RAM" -eq 1 ]; then
-    RAMROOT="/dev/shm/testium-build-${VERSION}"
-    echo "-- RAM mode: build scratch under $RAMROOT (tmpfs), freed on exit"
-    echo "   (flatpak builds on disk — rofiles-fuse can't mount on /dev/shm)"
-    rm -rf "$RAMROOT"
-    mkdir -p "$RAMROOT"/{tmp,pip,pyi-work,appdir}
-    export TMPDIR="$RAMROOT/tmp"
-    export PIP_CACHE_DIR="$RAMROOT/pip"
-    export PYI_WORKPATH="$RAMROOT/pyi-work"            # pyinstaller --workpath
-    export APPIMAGE_APPDIR_TMPFS="$RAMROOT/appdir"     # AppDir bind-mount
-    export TESTIUM_NO_UPX=1                            # skip slow UPX in the spec
-    trap 'rm -rf "$RAMROOT"' EXIT
-    if [ "$SERIAL" -ne 1 ]; then
-        echo "   note: with --ram, prefer adding --serial so each step gets the"
-        echo "         full tmpfs and you don't risk OOM (flatpak+appimage are ~1 GB each)."
-    fi
-fi
+# Ensure wheel/PyInstaller toolchains are present in the venv.
+python -m pip install --quiet --upgrade build pyinstaller
 
 step() {
     echo
@@ -114,186 +52,56 @@ step() {
     echo "================================================================"
 }
 
-# Kill a process and its whole descendant tree (children first) — used by the
-# interrupt handler so SIGINT also stops grandchildren the parallel jobs spawned
-# (podman container, flatpak-builder, pyinstaller …), not just the subshells.
-_kill_tree() {
-    local pid=$1 c
-    for c in $(pgrep -P "$pid" 2>/dev/null); do
-        _kill_tree "$c"
-    done
-    kill -TERM "$pid" 2>/dev/null || true
-}
-
-# Set as INT/TERM handler around the parallel wait. Stops every running build
-# tree, then exits — the EXIT trap (set under --ram) frees the tmpfs scratch.
-_interrupt() {
-    echo >&2
-    echo "-- interrupted: stopping running builds…" >&2
-    local pid
-    for pid in "${!PID2NAME[@]}"; do
-        _kill_tree "$pid"
-    done
-    exit 130
-}
-
-# ---------- artifact paths ----------------------------------------------------
-
+# 1. Manual PDF
+step "1/5  Manual PDF (version $VERSION)"
+bash "$SCRIPT_DIR/doc/manual/sphinx/build_doc.sh"
+MANUAL_SRC="$SCRIPT_DIR/doc/manual/testium_manual.pdf"
 MANUAL="$DIST_DIR/testium-manual-${VERSION}.pdf"
+cp -f "$MANUAL_SRC" "$MANUAL"
+
+# 2. Wheel — PEP 427 name kept (already contains version)
+step "2/5  Wheel (version $VERSION)"
+(
+    cd "$SCRIPT_DIR/src"
+    rm -rf dist build *.egg-info
+    python -m build --wheel
+)
+WHEEL_SRC=$(ls -1t "$SCRIPT_DIR/src/dist"/*.whl | head -1)
+WHEEL="$DIST_DIR/$(basename "$WHEEL_SRC")"
+cp -f "$WHEEL_SRC" "$WHEEL"
+
+# 3. PyInstaller binary
+step "3/5  PyInstaller binary (version $VERSION)"
+bash "$SCRIPT_DIR/package/pyinstaller/build.sh"
+PYI_SRC="$SCRIPT_DIR/package/pyinstaller/dist/testium"
 PYI_BIN="$DIST_DIR/testium-${VERSION}"
+cp -f "$PYI_SRC" "$PYI_BIN"
+
+# 4. Flatpak bundle
+step "4/5  Flatpak bundle (version $VERSION)"
+(
+    cd "$SCRIPT_DIR/package/flatpak"
+    bash build.sh
+)
+FLATPAK_SRC="$SCRIPT_DIR/package/flatpak/testium.flatpak"
 FLATPAK_BUNDLE="$DIST_DIR/testium-${VERSION}.flatpak"
-wheel_in_dist() { ls -1t "$DIST_DIR"/testium-${VERSION}-*.whl 2>/dev/null | head -1; }
-appimage_in_dist() { ls -1t "$DIST_DIR"/Testium-${VERSION}-*.AppImage 2>/dev/null | head -1; }
+cp -f "$FLATPAK_SRC" "$FLATPAK_BUNDLE"
 
-# ---------- per-step build functions (assume tools are installed) -------------
-
-build_wheel() {
-    if [ -n "$(wheel_in_dist)" ]; then echo "wheel: already built — skipping"; return 0; fi
-    echo "wheel: building"
-    (
-        cd "$SCRIPT_DIR/src"
-        rm -rf dist build *.egg-info
-        python -m build --wheel
-    )
-    local src; src=$(ls -1t "$SCRIPT_DIR/src/dist"/*.whl | head -1)
-    cp -f "$src" "$DIST_DIR/$(basename "$src")"
-    echo "wheel: done"
-}
-
-build_manual() {
-    if [ -f "$MANUAL" ]; then echo "manual: already built — skipping"; return 0; fi
-    echo "manual: building"
-    bash "$SCRIPT_DIR/doc/manual/sphinx/build_doc.sh"
-    cp -f "$SCRIPT_DIR/doc/manual/testium_manual.pdf" "$MANUAL"
-    echo "manual: done"
-}
-
-build_pyinstaller() {
-    if [ -f "$PYI_BIN" ]; then echo "pyinstaller: already built — skipping"; return 0; fi
-    echo "pyinstaller: building"
-    bash "$SCRIPT_DIR/package/pyinstaller/build.sh"
-    cp -f "$SCRIPT_DIR/package/pyinstaller/dist/testium" "$PYI_BIN"
-    echo "pyinstaller: done"
-}
-
-build_flatpak() {
-    if [ -f "$FLATPAK_BUNDLE" ]; then echo "flatpak: already built — skipping"; return 0; fi
-    echo "flatpak: building"
-    (
-        cd "$SCRIPT_DIR/package/flatpak"
-        bash build.sh
-    )
-    cp -f "$SCRIPT_DIR/package/flatpak/testium.flatpak" "$FLATPAK_BUNDLE"
-    echo "flatpak: done"
-}
-
-build_appimage() {
-    if [ -n "$(appimage_in_dist)" ]; then echo "appimage: already built — skipping"; return 0; fi
-    echo "appimage: building"
-    (
-        cd "$SCRIPT_DIR/package/appimage"
-        bash build.sh
-    )
-    local src; src=$(ls -1t "$SCRIPT_DIR/package/appimage"/*.AppImage 2>/dev/null | head -1)
-    cp -f "$src" "$DIST_DIR/$(basename "$src")"
-    chmod +x "$DIST_DIR/$(basename "$src")"
-    echo "appimage: done"
-}
-
-# ---------- serial prep: tool installs (shared venv) + flatpak runtimes -------
-
-step "Prep: build tools + runtimes (serial — shared venv)"
-
-[ -f "$MANUAL" ]   || python -m pip install --quiet --upgrade sphinx linuxdoc
-[ -n "$(wheel_in_dist)" ] || python -m pip install --quiet --upgrade build
-[ -f "$PYI_BIN" ]  || python -m pip install --quiet --upgrade pyinstaller
-
-if [ ! -f "$FLATPAK_BUNDLE" ]; then
-    FLATPAK_DEPS=(
-        "org.kde.Platform//6.10"
-        "org.kde.Sdk//6.10"
-        "io.qt.PySide.BaseApp//6.10"
-    )
-    if ! flatpak remotes --user | grep -q "^flathub"; then
-        echo "  Adding Flathub remote"
-        flatpak remote-add --user --if-not-exists flathub https://dl.flathub.org/repo/flathub.flatpakrepo
-    fi
-    for dep in "${FLATPAK_DEPS[@]}"; do
-        if ! flatpak info --user "$dep" &>/dev/null && ! flatpak info --system "$dep" &>/dev/null; then
-            echo "  Installing Flatpak dependency: $dep"
-            flatpak install --user --noninteractive flathub "$dep"
-        fi
-    done
-fi
-
-# ---------- serial: wheel (the AppImage installs it) --------------------------
-
-step "1/5  Wheel (version $VERSION)"
-build_wheel
-
-# ---------- build the rest --------------------------------------------------
-
-REST=(manual pyinstaller flatpak appimage)
-
-if [ "$SERIAL" -eq 1 ]; then
-    n=2
-    for name in "${REST[@]}"; do
-        step "$n/5  $name (version $VERSION)"
-        "build_$name"
-        n=$((n + 1))
-    done
-else
-    step "2-5/5  manual + pyinstaller + flatpak + appimage (parallel)"
-    LOGDIR="$DIST_DIR/.build-logs"
-    mkdir -p "$LOGDIR"
-    declare -A PID2NAME
-    for name in "${REST[@]}"; do
-        log="$LOGDIR/$name.log"
-        echo "  -> launching $name (log: $log)"
-        ( "build_$name" ) >"$log" 2>&1 &
-        PID2NAME[$!]="$name"
-    done
-
-    # From here until all jobs are reaped, Ctrl+C stops every build tree.
-    trap _interrupt INT TERM
-
-    # Reap in completion order (wait -n) so each result prints the moment that
-    # build finishes, not when its slot comes up in the array.
-    FAILED=()
-    remaining=${#PID2NAME[@]}
-    while [ "$remaining" -gt 0 ]; do
-        if wait -n -p donepid; then rc=0; else rc=$?; fi
-        name="${PID2NAME[$donepid]:-}"
-        [ -z "$name" ] && continue
-        if [ "$rc" -eq 0 ]; then
-            echo "  -> $name: OK"
-        else
-            echo "  -> $name: FAILED (rc=$rc)"
-            FAILED+=("$name")
-        fi
-        remaining=$((remaining - 1))
-    done
-
-    trap - INT TERM
-
-    if [ "${#FAILED[@]}" -gt 0 ]; then
-        for name in "${FAILED[@]}"; do
-            echo
-            echo "===================== $name log ====================="
-            cat "$LOGDIR/$name.log"
-        done
-        echo >&2
-        echo "BUILD FAILED: ${FAILED[*]} (logs under $LOGDIR)" >&2
-        exit 1
-    fi
-fi
-
-# ---------- summary -----------------------------------------------------------
+# 5. AppImage
+step "5/5  AppImage (version $VERSION)"
+(
+    cd "$SCRIPT_DIR/package/appimage"
+    bash build.sh
+)
+APPIMAGE_SRC=$(ls -1t "$SCRIPT_DIR/package/appimage"/*.AppImage 2>/dev/null | head -1)
+APPIMAGE="$DIST_DIR/$(basename "$APPIMAGE_SRC")"
+cp -f "$APPIMAGE_SRC" "$APPIMAGE"
+chmod +x "$APPIMAGE"
 
 step "All packages built"
 printf "  manual       : %s\n" "$MANUAL"
-printf "  wheel        : %s\n" "$(wheel_in_dist)"
+printf "  wheel        : %s\n" "$WHEEL"
 printf "  pyinstaller  : %s\n" "$PYI_BIN"
 printf "  flatpak      : %s\n" "$FLATPAK_BUNDLE"
-printf "  appimage     : %s\n" "$(appimage_in_dist)"
+printf "  appimage     : %s\n" "$APPIMAGE"
 printf "  release_note : %s\n" "$RELEASE_NOTE"

doc/manual/sphinx/source/modes.rst

@@ -23,47 +23,3 @@ graphical interface.
     :caption: call a test in batch mode
 
     testium -b test/my_test/main.tum
-
-.. _sec_language_server:
-
-Language server (editor support)
---------------------------------
-
-*testium* ships a `Language Server Protocol
-<https://microsoft.github.io/language-server-protocol/>`_ server so that
-``.tum`` files get editor assistance — completion of test item types, hover
-documentation of their parameters, and an outline view — in any LSP-capable
-editor.
-
-The server speaks LSP over standard input/output and is started with:
-
-.. code-block:: text
-    :caption: start the language server
-
-    testium lsp
-
-It is not meant to be launched directly by the user: an editor's LSP client
-spawns it and drives the exchange. A VSCode / VSCodium client extension,
-*testium_assist*, is provided for that purpose; any other LSP client (Neovim,
-Emacs ``lsp-mode``, …) can be pointed at ``testium lsp`` as well.
-
-The information the server exposes is the test item schema, which can also be
-dumped as JSON for inspection or tooling:
-
-.. code-block:: text
-    :caption: dump the item / parameter schema
-
-    testium schema
-
-Because the schema is built from *testium* itself, every new item type or
-parameter becomes available in the editor on the next *testium* upgrade, with
-no change to the client.
-
-The language server is included in the pre-built binary, Flatpak and AppImage
-releases. For a source or wheel installation, pull the optional ``lsp``
-dependencies:
-
-.. code-block:: text
-    :caption: enable the language server for a wheel / source install
-
-    pip install 'testium[lsp]'

doc/manual/sphinx/source/tum_syntax.rst

@@ -232,15 +232,6 @@ list of the main test item (and eventually of the loop test item).
 TUM file ``main`` item is itself a variant of test items with a name and an
 step list attributes.
 
-.. note::
-
-   Each test item declares the parameters it accepts. When a ``.tum`` file
-   uses a key the item does not know, *testium* emits a warning listing the
-   accepted parameter names (catching typos such as ``param_filee`` for
-   ``param_file``); a missing **required** parameter aborts loading with an
-   error pointing at the source ``.tum`` file. Valid existing tests are
-   unaffected.
-
 
 
 .. toctree::

package/appimage/AppImageBuilder.yml

@@ -77,10 +77,7 @@ AppDir:
         python3.11 -m pip install --break-system-packages --upgrade --isolated --no-input --ignore-installed --prefix=$TARGET_APPDIR/usr -r ../../src/requirements.txt
 
         export PIP_CONFIG_FILE=$HOME/.pip/pip.conf
-        # Install the wheel with the [lsp] extra so `testium lsp` (pygls) works
-        # from the AppImage. The extra pulls pygls/lsprotocol/cattrs/attrs from
-        # the index (network is available at build time, see get-pip above).
-        python3.11 -m pip install --break-system-packages --upgrade --isolated --no-input --ignore-installed --prefix=$TARGET_APPDIR/usr "../../src/dist/testium-{{APP_VERSION}}-py3-none-any.whl[lsp]"
+        python3.11 -m pip install --break-system-packages --upgrade --isolated --no-input --ignore-installed --prefix=$TARGET_APPDIR/usr ../../src/dist/testium-{{APP_VERSION}}-py3-none-any.whl
 
 
 AppImage:

package/appimage/build.sh

@@ -17,20 +17,11 @@ else
 fi
 echo "Using $RUNTIME — building testium $APP_VERSION AppImage..."
 
-# APPIMAGE_APPDIR_TMPFS (set by build_all --ram) bind-mounts a host tmpfs dir at
-# the AppDir build path, keeping the ~1 GB AppDir churn off slow storage.
-APPDIR_MOUNT=""
-if [ -n "$APPIMAGE_APPDIR_TMPFS" ]; then
-    mkdir -p "$APPIMAGE_APPDIR_TMPFS"
-    APPDIR_MOUNT="-v $APPIMAGE_APPDIR_TMPFS:/work/package/appimage/AppDir"
-fi
-
 # APPIMAGE_EXTRACT_AND_RUN=1 lets appimagetool run without FUSE in the container.
 $RUNTIME run --rm \
     --privileged \
     -e APPIMAGE_EXTRACT_AND_RUN=1 \
     -v "$REPO_ROOT:/work" \
-    $APPDIR_MOUNT \
     -w /work/package/appimage \
     debian:bookworm bash -c "
         set -e

package/flatpak/build.sh

@@ -7,19 +7,11 @@
 
 set -e
 
-# Build + install local. FLATPAK_BUILDDIR / FLATPAK_STATEDIR / FLATPAK_REPODIR
-# (set by build_all --ram) redirect the build dir, the state dir
-# (.flatpak-builder) and the ostree repo to tmpfs. flatpak-builder hardlinks
-# between the state dir and the build dir, so they MUST be on the same
-# filesystem — hence the state dir moves to tmpfs too (its download cache then
-# doesn't persist across --ram runs).
-BUILDDIR="${FLATPAK_BUILDDIR:-build}"
-STATEDIR="${FLATPAK_STATEDIR:-.flatpak-builder}"
-REPODIR="${FLATPAK_REPODIR:-repo}"
-flatpak-builder --user --verbose --force-clean --install --state-dir="$STATEDIR" --repo="$REPODIR" "$BUILDDIR" org.testium.Testium.yaml
+# Build + install local
+flatpak-builder --user --verbose --force-clean --install --repo=repo build org.testium.Testium.yaml
 
 # Génère le bundle distribuable
-flatpak build-bundle "$REPODIR" testium.flatpak org.testium.Testium
+flatpak build-bundle repo testium.flatpak org.testium.Testium
 echo "Bundle généré : $(pwd)/testium.flatpak"
 
 # Crée ~/.local/bin/testium pour pouvoir taper "testium" en console

package/flatpak/org.testium.Testium.yaml

@@ -16,11 +16,6 @@ finish-args:
   - --filesystem=home
   - --filesystem=/tmp
   - --filesystem=host-os
-  # Allow flatpak-spawn --host to launch host binaries (Python, Lua, git…)
-  # outside the sandbox. Required because the sandbox glibc/ABI is
-  # incompatible with arbitrary host shared libraries — we route py_func and
-  # lua_func through the host instead.
-  - --talk-name=org.freedesktop.Flatpak
 
 build-options:
   build-args:
@@ -28,23 +23,6 @@ build-options:
 
 modules:
   - python3-requirements.json
-
-  # Language-server deps for `testium lsp` (pygls + lsprotocol + cattrs + attrs
-  # + typing_extensions). Installed from PyPI at build time — the build already
-  # runs with --share=network (see build-options). The core runtime deps stay
-  # offline-pinned in python3-requirements.json; these are pure-python wheels,
-  # hence --only-binary=:all: (no compilation, deterministic).
-  - name: python3-lsp
-    buildsystem: simple
-    build-options:
-      build-args:
-        - --share=network
-    build-commands:
-      # Whole command single-quoted: the ':all: ' colon-space would otherwise
-      # make YAML parse this list item as a mapping, silently dropping the
-      # command (flatpak-builder then runs an empty module — installs nothing).
-      - 'pip3 install --prefix=${FLATPAK_DEST} --only-binary=:all: "pygls>=1.3"'
-
   # 1. Dépendances Python tierces (HORS PySide6)
   # Utilisez flatpak-pip-generator pour vos autres libs (ex: pyserial, requests, etc.)
   # - name: python3-requirements

package/pyinstaller/build.sh

@@ -2,15 +2,11 @@
 
 SCRIPT_DIR=$(realpath $( dirname "$0"))
 
-rm -rf "${SCRIPT_DIR}/build" "${SCRIPT_DIR}/dist"
+rm -r "${SCRIPT_DIR}/build" "${SCRIPT_DIR}/dist"
 
 pwd=$(pwd)
 cd ${SCRIPT_DIR}
-# PYI_WORKPATH (set by build_all --ram) puts the big intermediate build tree on
-# tmpfs; dist/ stays local so build_all can collect the binary.
-WORKARG=""
-[ -n "$PYI_WORKPATH" ] && WORKARG="--workpath $PYI_WORKPATH"
-pyinstaller $WORKARG testium.spec
+pyinstaller testium.spec
 RESULT=$?
 if [ -n "$1" ] && [ "$1" = "install" ]; then
     if [ $RESULT -eq 0 ]; then

package/pyinstaller/testium.spec

@@ -1,21 +1,5 @@
 # -*- mode: python ; coding: utf-8 -*-
 import os
-from PyInstaller.utils.hooks import collect_submodules
-
-# Language-server dependencies for `testium lsp`. pygls/lsprotocol register
-# converters and features dynamically, so we collect their submodules wholesale
-# and force-import their pure-python deps (cattrs/attrs/typing_extensions).
-# The testium lsp modules are imported lazily by the CLI dispatch
-# (`from lsp.server import serve`), which PyInstaller's static analysis misses —
-# hence the explicit names. No source files need bundling: the schema export is
-# now fully declarative (PARAMS + ACTIONS class attributes), so it no longer
-# reads .py source via inspect.getsource (which fails in a frozen build).
-_LSP_HIDDEN = (
-    collect_submodules("pygls")
-    + collect_submodules("lsprotocol")
-    + ["cattrs", "attr", "attrs", "typing_extensions",
-       "lsp", "lsp.server", "lsp.schema"]
-)
 
 # junit_xml is imported by post_exec scripts running under the *host* Python,
 # not the frozen interpreter — so bundling it via hiddenimports alone is not
@@ -70,7 +54,7 @@ a = Analysis(
                    "colorama",
                    "matplotlib",
                    "junit_xml",
-                   "lxml"] + _LSP_HIDDEN,
+                   "lxml"],
     hookspath=[],
     hooksconfig={},
     runtime_hooks=[],
@@ -89,9 +73,7 @@ exe = EXE(
     debug=False,
     bootloader_ignore_signals=False,
     strip=False,
-    # UPX is CPU+IO heavy for a marginal size gain — build_all --ram sets
-    # TESTIUM_NO_UPX=1 to skip it (much faster on slow/flash storage).
-    upx=not os.environ.get("TESTIUM_NO_UPX"),
+    upx=True,
     upx_exclude=[],
     runtime_tmpdir=None,
     console=True,

release_note.txt

@@ -1,49 +1,3 @@
-version 0.2
-==============
-- Test items: each item type now declares its accepted parameters
-  (``PARAMS = ParamSet(...)``). Typos in a ``.tum`` are surfaced as a
-  WARN listing the accepted names instead of being silently ignored;
-  missing required parameters error out at load time with the source
-  ``.tum`` file as context. No change to valid existing tests.
-- Editor support: testium now ships a language server. ``testium lsp``
-  gives ``.tum`` files item-type completion, hover documentation and an
-  outline view in any LSP-capable editor (a VSCode / VSCodium client is
-  provided separately). ``testium schema`` dumps the item/parameter
-  schema as JSON. The server works from every channel — bundled in the
-  binary / Flatpak / AppImage, and pulled by ``pip install testium[lsp]``
-  for wheel installs.
-- build_all.sh: the four heavy channels now build in parallel (results
-  reported as each finishes; Ctrl+C stops them cleanly). New ``--ram``
-  option builds on a tmpfs (``/dev/shm``) and skips UPX for much faster
-  packaging on USB-stick / SD-card storage.
-
-version 0.1.3
-==============
-- Stop interrupts engaged blocking steps (console, py_func, lua_func,
-  json_rpc, sleep) within ~200 ms instead of waiting for the step
-  to finish.
-- GUI Start / Stop / Pause flow simplified.
-- lua_func: a function returning nil is no longer reported as a failure.
-- ``-d python_bin=...`` and the GUI ``python_bin`` preference now reach
-  the eval subprocess (used to be silently ignored). ``param.yaml`` can
-  also override ``python_bin`` for py_func / cycle / post_exec.
-- Validation suite: ``test/validation/run.sh`` (and ``run.bat``)
-  runs the suite inside a dedicated venv in the system temp dir.
-- build_all.sh: ``release_note.txt`` and the user manual copied into
-  ``dist/``; warning if the file has no entry for the version being built.
-- Flatpak: every GUI file/directory dialog (open test, save report, log
-  path, default report/log dirs, python/lua interpreter pickers) now
-  bypasses the XDG document portal — the v0.1.2 fix was only on the
-  "open test" dialog.
-- Flatpak: py_func / lua_func / run sub-instance now execute on the host
-  via flatpak-spawn, lifting the previous glibc/ABI incompatibility that
-  prevented user-configured host Python or Lua interpreters from being
-  reached from the sandbox.
-- Validation suite: single entry point with ``--mode source|wheel|
-  pyinstaller|flatpak|appimage`` to validate any packaging channel
-  against the same item set; reports are stamped per mode.
-- GUI: the "Run tum" test item now uses the testium logo.
-
 version 0.1.2
 ==============
 - Flatpak: opening a test from the GUI now correctly finds its companion

schema/test.tum

@@ -0,0 +1,73 @@
+config_file:  
+  - premier
+  - saluT
+
+main:
+  name: Main file
+  steps:
+    - group:
+        name: Test
+        doc: Une peitite documentation
+        no_fail: true
+        steps:
+        - let:
+            values: 
+                - my_var: <| ${salut} |>
+        - check:
+            values: 
+            - <| ${salut} |>
+
+        - dialog_message:
+            question: c'est quoi?
+
+        - lua_func:
+            file: c'est quoi?
+            func_name: c'est quoi?
+        
+        - console:
+            console_name: cons_1
+            steps: 
+                - open:
+                    protocol: telnet
+                    terminal_path: ijfeifj
+                - read_until : {expected: "tutu", timeout: -4.5, mute: true}
+                - write: something
+                - writeln: tutu
+                - close : 
+
+        - json_rpc:
+            name: JSONRPC console Query
+            doc: JSONRPC console Query not waiting (only send)
+            console:
+                name : jsonrpc_server
+                prompt: "@@>"
+            timeout: 1
+            version: "2.0"
+            steps:
+            - query:
+                method: echo
+                params:
+                    - Hello world
+                    - [0, 1, 2, 3]
+                id: 3095372
+                no_wait: true
+        - json_rpc:
+            name: JSONRPC console Reception
+            doc: JSONRPC console reception of the previous request
+            console: {name : jsonrpc_server}
+            timeout: 1
+            steps:
+            - receive:
+                id: 3095372
+                timeout: 0.5
+
+report:
+    enabled: true
+    log_stored: true
+    export:
+        junit:
+          path: $(validation_report_path)
+          file_name: $(validation_report_file).junit
+        sqlite:
+          file_name: $(validation_report_file).sqlite
+          path: $(validation_report_path)

schema/test_schema/check.tum

@@ -0,0 +1,96 @@
+config_file:
+- param.yaml
+- items/check/param.yaml
+main:
+   name: Testium validation suite
+   steps:
+   -  group:
+         name: Test preparation
+         steps:
+         -  let:
+               condition: <| "$(os)" == "Linux" |>
+               name: Set test variables for Linux
+               values:
+               -  terminal_prompt: $(linux_prompt)
+               -  psep: /
+         -  let:
+               condition: <| "$(os)" == "Windows" |>
+               name: Set test variables for Windows
+               values:
+               -  terminal_prompt: $(windows_prompt)
+               -  psep: \
+   -  group:
+         name: Group of tests
+         steps:
+         -  let:
+               name: check test constants
+               values:
+                  - test: check
+                  - test_path: items/$(test)
+         -  group:
+               name: check test
+               steps:
+               -  sequence:
+                     steps:
+                     -  py_func:
+                           file: $(test_path)$(psep)check.py
+                           func_name: echo
+                           key: $(test)_PASS
+                           name: Dummy_int
+                           param:
+                           - 2
+                     -  py_func:
+                           file: $(test_path)$(psep)check.py
+                           func_name: echo
+                           key: $(test)_PASS
+                           name: Dummy_str
+                           param:
+                           - my taylor is rich
+                     -  check:
+                           key: $(test)_PASS
+                           name: Check condition on existing variable (PASS)
+                           values:
+                           - <| $(pfn_Dummy_int) > 1 |>
+                     -  check:
+                           key: $(test)_FAIL
+                           name: Check condition on existing variable (FAIL)
+                           values:
+                           - <| "tailor" in "$(pfn_Dummy_str)" |>
+                     filename: /home/renish/workspace/testium/code/test/validation/items/check/test.tum
+               -  sequence:
+                     steps:
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.txt
+                           -  html:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.html
+                           -  junit:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.junit
+                           name: Expected PASS $(test) test
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.txt
+                           -  html:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.html
+                           -  junit:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.junit
+                           name: Expected FAIL $(test) test
+                     filename: /home/renish/workspace/testium/code/test/validation/items/report.tum
+report:
+   enabled: true
+   export:
+      junit:
+         file_name: $(validation_report_file).junit
+         path: $(validation_report_path)
+      sqlite:
+         file_name: $(validation_report_file).sqlite
+         path: $(validation_report_path)
+   log_stored: true

schema/test_schema/common.tum

@@ -0,0 +1,453 @@
+config_file:
+- param.yaml
+- items/common/param.yaml
+main:
+   name: Testium validation suite
+   steps:
+   -  group:
+         name: Test preparation
+         steps:
+         -  let:
+               condition: <| "$(os)" == "Linux" |>
+               name: Set test variables for Linux
+               values:
+               -  terminal_prompt: $(linux_prompt)
+               -  psep: /
+         -  let:
+               condition: <| "$(os)" == "Windows" |>
+               name: Set test variables for Windows
+               values:
+               -  terminal_prompt: $(windows_prompt)
+               -  psep: \
+   -  group:
+         name: Group of tests
+         steps:
+         -  let:
+               name: common test constants
+               values:
+                  test: common
+                  test_path: items/$(test)
+         -  group:
+               name: common test
+               steps:
+               -  sequence:
+                     steps:
+                     -  group:
+                           name: Results
+                           steps:
+                           -  sequence:
+                                 steps:
+                                 -  group:
+                                       name: Expected Result
+                                       steps:
+                                       -  py_func:
+                                             expected_result: true
+                                             file: $(test_path)$(psep)results$(psep)results.py
+                                             func_name: echo
+                                             key: $(test)_PASS
+                                             name: Return True expect True
+                                             param:
+                                             - true
+                                       -  py_func:
+                                             expected_result: false
+                                             file: $(test_path)$(psep)results$(psep)results.py
+                                             func_name: echo
+                                             key: $(test)_FAIL
+                                             name: Return True expect False (must
+                                                fail)
+                                             param:
+                                             - true
+                                       -  py_func:
+                                             expected_result: None
+                                             file: $(test_path)$(psep)results$(psep)results.py
+                                             func_name: return_none
+                                             key: $(test)_PASS
+                                             name: Return None expect None
+                                       -  py_func:
+                                             expected_result: PASS
+                                             file: $(test_path)$(psep)results$(psep)results.py
+                                             func_name: return_none
+                                             key: $(test)_PASS
+                                             name: Return None expect PASS
+                                       -  py_func:
+                                             expected_result: 14
+                                             file: $(test_path)$(psep)results$(psep)results.py
+                                             func_name: return_none
+                                             key: $(test)_FAIL
+                                             name: Return None expect 14 (must fail)
+                                 -  group:
+                                       name: Expected Result Last test result
+                                       steps:
+                                       -  py_func:
+                                             file: $(test_path)$(psep)results$(psep)results.py
+                                             func_name: echo
+                                             key: $(test)_PASS
+                                             name: result is 28
+                                             param:
+                                             - 28
+                                       -  py_func:
+                                             expected_result: $(last_step_result)
+                                             file: $(test_path)$(psep)results$(psep)results.py
+                                             func_name: echo
+                                             key: $(test)_PASS
+                                             name: check that the last test result
+                                                is 28
+                                             param:
+                                             - 28
+                                 -  group:
+                                       name: Expected result Failure raised issue
+                                       steps:
+                                       -  py_func:
+                                             file: $(test_path)$(psep)results$(psep)results.py
+                                             func_name: raise_issue
+                                             key: $(test)_FAIL
+                                             name: Raise an issue (must fail)
+                                             param:
+                                             - $(str_example)
+                                       -  py_func:
+                                             expected_result: FAIL
+                                             file: $(test_path)$(psep)results$(psep)results.py
+                                             func_name: raise_issue
+                                             key: $(test)_PASS
+                                             name: Raise an issue and expected the
+                                                test to be FAIL
+                                             param:
+                                             - $(str_example)
+                                       -  py_func:
+                                             expected_result: FAIL
+                                             file: $(test_path)$(psep)results$(psep)results.py
+                                             func_name: echo
+                                             key: $(test)_FAIL
+                                             name: Return a String expect a FAILURE
+                                                (must fail)
+                                             param:
+                                             - $(str_example)
+                                 -  group:
+                                       name: process result
+                                       steps:
+                                       -  py_func:
+                                             file: $(test_path)$(psep)results$(psep)results.py
+                                             func_name: echo
+                                             key: $(test)_PASS
+                                             name: Process result equal String
+                                             param:
+                                             - $(str_example)
+                                             process_result: '''$(str_example)'' ==
+                                                ''$(result)'''
+                                       -  py_func:
+                                             expected_result: true
+                                             file: $(test_path)$(psep)results$(psep)results.py
+                                             func_name: echo
+                                             key: $(test)_FAIL
+                                             name: Process result string in the result
+                                                (must fail)
+                                             param:
+                                             - $(str_example)
+                                             process_result: '''44'' in ''$(result)'''
+                                       -  py_func:
+                                             file: $(test_path)$(psep)results$(psep)results.py
+                                             func_name: echo
+                                             key: $(test)_PASS
+                                             name: Save the result in a global variable
+                                             param:
+                                             - 44
+                                             store_result: process_result_value
+                                       -  py_func:
+                                             expected_result: $(process_result_value)
+                                             file: $(test_path)$(psep)results$(psep)results.py
+                                             func_name: echo
+                                             key: $(test)_PASS
+                                             name: Check the saved global variable
+                                             param:
+                                             - 44
+                                       -  py_func:
+                                             file: $(test_path)$(psep)results$(psep)results.py
+                                             func_name: echo
+                                             key: $(test)_PASS
+                                             name: store_result with process_result
+                                             param:
+                                             - $(str_example)
+                                             process_result: '''$(result)''.upper()'
+                                             store_result: upper_str_example
+                                       -  py_func:
+                                             expected_result: $(upper_str_example)
+                                             file: $(test_path)$(psep)results$(psep)results.py
+                                             func_name: echo
+                                             key: $(test)_PASS
+                                             name: Check store_result with process_result
+                                             param:
+                                             - $(str_example)
+                                             process_result: '''$(result)''.upper()'
+                                       -  let:
+                                             key: $(test)_PASS
+                                             name: "store_result on let item (None\
+                                                \ value \u2192 stores PASS)"
+                                             store_result: let_store_result
+                                             values:
+                                             -  dummy: 0
+                                       -  py_func:
+                                             expected_result: $(let_store_result)
+                                             file: $(test_path)$(psep)results$(psep)results.py
+                                             func_name: echo
+                                             key: $(test)_PASS
+                                             name: Check store_result on let stores
+                                                PASS
+                                             param:
+                                             - PASS
+                                       -  py_func:
+                                             expected_result: FAIL
+                                             file: $(test_path)$(psep)results$(psep)results.py
+                                             func_name: return_none
+                                             key: $(test)_FAIL
+                                             name: "store_result on failing test (None\
+                                                \ value \u2192 stores FAIL)"
+                                             store_result: none_fail_store_result
+                                       -  py_func:
+                                             expected_result: $(none_fail_store_result)
+                                             file: $(test_path)$(psep)results$(psep)results.py
+                                             func_name: echo
+                                             key: $(test)_PASS
+                                             name: Check store_result on failing test
+                                                stores FAIL
+                                             param:
+                                             - FAIL
+                                       -  py_func:
+                                             expected_result: FAIL
+                                             file: $(test_path)$(psep)results$(psep)results.py
+                                             func_name: return_none
+                                             key: $(test)_PASS
+                                             name: "store_result with no_fail (None\
+                                                \ value \u2192 stores real FAIL, not\
+                                                \ forced PASS)"
+                                             no_fail: true
+                                             store_result: none_nofail_store_result
+                                       -  py_func:
+                                             expected_result: $(none_nofail_store_result)
+                                             file: $(test_path)$(psep)results$(psep)results.py
+                                             func_name: echo
+                                             key: $(test)_PASS
+                                             name: Check store_result with no_fail
+                                                stores real FAIL
+                                             param:
+                                             - FAIL
+                                       -  py_func:
+                                             file: $(test_path)$(psep)results$(psep)results.py
+                                             func_name: return_none
+                                             key: $(test)_FAIL
+                                             name: Process result when result is None
+                                                (must fail)
+                                             process_result: $(result) is None
+                                 -  group:
+                                       name: no_fail result
+                                       steps:
+                                       -  py_func:
+                                             expected_result: false
+                                             file: $(test_path)$(psep)results$(psep)results.py
+                                             func_name: echo
+                                             key: $(test)_PASS
+                                             name: Return True expect False but no_fail=True
+                                             no_fail: true
+                                             param:
+                                             - true
+                                       -  py_func:
+                                             expected_result: false
+                                             file: $(test_path)$(psep)results$(psep)results.py
+                                             func_name: echo
+                                             key: $(test)_FAIL
+                                             name: Return True expect False but no_fail=False
+                                                (must fail)
+                                             no_fail: false
+                                             param:
+                                             - true
+                                       -  py_func:
+                                             expected_result: false
+                                             file: $(test_path)$(psep)results$(psep)results.py
+                                             func_name: echo
+                                             key: $(test)_PASS
+                                             name: Return True expect False but no_fail
+                                                expansed
+                                             no_fail: <| bool(0) == False |>
+                                             param:
+                                             - true
+                                       -  py_func:
+                                             expected_result: false
+                                             file: $(test_path)$(psep)results$(psep)results.py
+                                             func_name: echo
+                                             key: $(test)_FAIL
+                                             name: Return True expect False but no_fail
+                                                expansed (must fail)
+                                             no_fail: <| bool(1) == False |>
+                                             param:
+                                             - true
+                                 filename: /home/renish/workspace/testium/code/test/validation/items/common/results/test.tum
+                     -  group:
+                           name: Conditional
+                           steps:
+                           -  sequence:
+                                 steps:
+                                 -  loop:
+                                       doc: This loop illustrate the way to exit on
+                                          a condition.
+                                       exit_condition:
+                                          value: <| $(pfn_Echo function) > 3 |>
+                                       key: $(test)_PASS
+                                       name: Infine loop with conditional exit
+                                       steps:
+                                       -  sleep:
+                                             name: small wait
+                                             timeout: 0.2
+                                       -  py_func:
+                                             file: $(test_path)$(psep)conditional$(psep)conditional.py
+                                             func_name: echo
+                                             key: $(test)_PASS
+                                             name: Echo function
+                                             param:
+                                             - $(loop_param)
+                                       stop_on_failure: false
+                                 -  let:
+                                       key: $(test)_PASS
+                                       name: let
+                                       values:
+                                       -  conditional_exec: <| random.randint(1, 2) |>
+                                 -  console:
+                                       condition: <| $(conditional_exec) == 1 |>
+                                       console_name: consname
+                                       doc: Opening the console
+                                       key: $(test)_PASS
+                                       name: Console creation
+                                       steps:
+                                       -  open:
+                                             protocol: terminal
+                                             terminal_path: $(test_directory)
+                                       -  writeln: echo "terminal loaded"
+                                 -  console:
+                                       condition: <| $(conditional_exec) == 1 |>
+                                       console_name: consname
+                                       key: $(test)_PASS
+                                       name: Console read_until with timeout
+                                       steps:
+                                       -  read_until:
+                                             expected: terminal loaded
+                                             timeout: 5
+                                 -  console:
+                                       condition: <| $(conditional_exec) == 1 |>
+                                       console_name: consname
+                                       key: $(test)_PASS
+                                       name: Console write
+                                       steps:
+                                       -  writeln: echo 0
+                                 -  sleep:
+                                       condition: <| $(conditional_exec) == 1 |>
+                                       name: sleep item
+                                       timeout: 1
+                                 -  console:
+                                       condition: <| $(conditional_exec) == 1 |>
+                                       console_name: consname
+                                       key: $(test)_PASS
+                                       name: Console read_until immediate
+                                       steps:
+                                       -  read_until:
+                                             expected: '0'
+                                             timeout: 0
+                                 -  console:
+                                       condition: <| $(conditional_exec) == 1 |>
+                                       console_name: consname
+                                       key: $(test)_PASS
+                                       name: Console read_until immediate (2)
+                                       steps:
+                                       -  read_until:
+                                             expected: $(terminal_prompt)
+                                             timeout: 0
+                                 -  console:
+                                       condition: <| $(conditional_exec) == 1 |>
+                                       console_name: consname
+                                       key: $(test)_PASS
+                                       name: Console closure
+                                       steps:
+                                       -  close: consname
+                                 -  sleep:
+                                       condition: <| $(conditional_exec) == 2 |>
+                                       name: sleep item
+                                       timeout: 1
+                                 filename: /home/renish/workspace/testium/code/test/validation/items/common/conditional/test.tum
+                     -  group:
+                           name: Various syntax robustness
+                           steps:
+                           -  sequence:
+                                 steps:
+                                 -  sleep:
+                                       key: $(test)_PASS
+                                       timeout: 0.2
+                                 -  sleep:
+                                       key: $(test)_PASS
+                                       name: null
+                                       timeout: 0.2
+                                 -  sleep:
+                                       doc: null
+                                       key: $(test)_PASS
+                                       name: Empty "doc:" declared (must PASS)
+                                       timeout: 0.2
+                                 filename: /home/renish/workspace/testium/code/test/validation/items/common/syntax_robustness/test.tum
+                     -  group:
+                           name: Helper lib functions
+                           steps:
+                           -  py_func:
+                                 file: $(test_path)$(psep)helper_lib.py
+                                 func_name: check_os
+                                 key: $(test)_PASS
+                                 name: OS
+                                 param:
+                                 - $(os)
+                           -  py_func:
+                                 file: $(test_path)$(psep)helper_lib.py
+                                 func_name: check_get_main_dir
+                                 key: $(test)_PASS
+                                 name: get_main_dir
+                           -  py_func:
+                                 file: $(test_path)$(psep)helper_lib.py
+                                 func_name: check_timestamp_as_sec_conversion
+                                 key: $(test)_PASS
+                                 name: timestamp_as_sec conversion
+                           -  py_func:
+                                 file: $(test_path)$(psep)helper_lib.py
+                                 func_name: check_timestamp
+                                 key: $(test)_PASS
+                                 name: timestamp and timestamp_as_sec
+                     filename: /home/renish/workspace/testium/code/test/validation/items/common/test.tum
+               -  sequence:
+                     steps:
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.txt
+                           -  html:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.html
+                           -  junit:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.junit
+                           name: Expected PASS $(test) test
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.txt
+                           -  html:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.html
+                           -  junit:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.junit
+                           name: Expected FAIL $(test) test
+                     filename: /home/renish/workspace/testium/code/test/validation/items/report.tum
+report:
+   enabled: true
+   export:
+      junit:
+         file_name: $(validation_report_file).junit
+         path: $(validation_report_path)
+      sqlite:
+         file_name: $(validation_report_file).sqlite
+         path: $(validation_report_path)
+   log_stored: true

schema/test_schema/console.tum

@@ -0,0 +1,164 @@
+config_file:
+- param.yaml
+- items/console/param.yaml
+main:
+   name: Testium validation suite
+   steps:
+   -  group:
+         name: Test preparation
+         steps:
+         -  let:
+               condition: <| "$(os)" == "Linux" |>
+               name: Set test variables for Linux
+               values:
+               -  terminal_prompt: $(linux_prompt)
+               -  psep: /
+         -  let:
+               condition: <| "$(os)" == "Windows" |>
+               name: Set test variables for Windows
+               values:
+               -  terminal_prompt: $(windows_prompt)
+               -  psep: \
+   -  group:
+         name: Group of tests
+         steps:
+         -  let:
+               name: console test constants
+               values:
+                  - test: console
+                  - test_path: items/$(test)
+         -  group:
+               name: console test
+               steps:
+               -  sequence:
+                     steps:
+                     -  console:
+                           console_name: term
+                           doc: Opening the console
+                           key: $(test)_PASS
+                           name: Console creation
+                           steps:
+                           -  open:
+                                 protocol: terminal
+                                 terminal_path: $(test_directory)
+                           -  writeln: echo "endOfOpen"
+                     -  console:
+                           console_name: term
+                           key: $(test)_PASS
+                           name: Console read_until with timeout
+                           steps:
+                           -  read_until:
+                                 expected: endOfOpen
+                                 timeout: 5
+                     -  console:
+                           console_name: term
+                           key: $(test)_PASS
+                           name: Console write
+                           steps:
+                           -  writeln: echo 0
+                     -  sleep:
+                           name: sleep item
+                           timeout: 1
+                     -  console:
+                           console_name: term
+                           key: $(test)_PASS
+                           name: Console read_until immediate
+                           steps:
+                           -  read_until:
+                                 expected: '0'
+                                 timeout: 0
+                     -  console:
+                           console_name: term
+                           key: $(test)_PASS
+                           name: Console write
+                           steps:
+                           -  writeln: echo "HelloConsole"
+                     -  console:
+                           console_name: term
+                           key: $(test)_FAIL
+                           name: Console read_until fail
+                           steps:
+                           -  read_until:
+                                 expected: Something never prints
+                                 timeout: 1
+                     -  console:
+                           console_name: term
+                           key: $(test)_PASS
+                           name: Console write
+                           steps:
+                           -  writeln: echo "HelloConsole"
+                     -  console:
+                           console_name: term
+                           key: $(test)_PASS
+                           name: Console read_until no_fail
+                           steps:
+                           -  read_until:
+                                 expected: Something never prints
+                                 no_fail: true
+                                 timeout: 1
+                     -  console:
+                           console_name: term
+                           key: $(test)_PASS
+                           name: Console read_until muted
+                           steps:
+                           -  writeln: echo "HelloConsole"
+                           -  read_until:
+                                 expected: HelloConsole
+                                 mute: true
+                                 timeout: 1
+                     -  console:
+                           console_name: term
+                           key: $(test)_PASS
+                           name: Console read_until muted
+                           steps:
+                           -  writeln: echo "HelloConsole is PASS" && echo "endOfCmd"
+                           -  read_until:
+                                 expected: endOfCmd
+                                 process_result: '''Hello'' in r''''''$(result)''''''
+                                    and ''PASS'' in r''''''$(result)'''''' '
+                                 timeout: 1
+                     -  console:
+                           console_name: term
+                           execute_on_stop: true
+                           key: $(test)_PASS
+                           name: Console closure
+                           steps:
+                           -  close: term
+                     filename: /home/renish/workspace/testium/code/test/validation/items/console/test.tum
+               -  sequence:
+                     steps:
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.txt
+                           -  html:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.html
+                           -  junit:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.junit
+                           name: Expected PASS $(test) test
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.txt
+                           -  html:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.html
+                           -  junit:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.junit
+                           name: Expected FAIL $(test) test
+                     filename: /home/renish/workspace/testium/code/test/validation/items/report.tum
+report:
+   enabled: true
+   export:
+      junit:
+         file_name: $(validation_report_file).junit
+         path: $(validation_report_path)
+      sqlite:
+         file_name: $(validation_report_file).sqlite
+         path: $(validation_report_path)
+   log_stored: true

schema/test_schema/cycle.tum

@@ -0,0 +1,106 @@
+config_file:
+- param.yaml
+- items/cycle/param.yaml
+main:
+   name: Testium validation suite
+   steps:
+   -  group:
+         name: Test preparation
+         steps:
+         -  let:
+               condition: <| "$(os)" == "Linux" |>
+               name: Set test variables for Linux
+               values:
+                  terminal_prompt: $(linux_prompt)
+                  psep: /
+         -  let:
+               condition: <| "$(os)" == "Windows" |>
+               name: Set test variables for Windows
+               values:
+               -  terminal_prompt: $(windows_prompt)
+               -  psep: \
+   -  group:
+         name: Group of tests
+         steps:
+         -  let:
+               name: cycle test constants
+               values:
+                  - test: cycle
+                  - test_path: items/$(test)
+         -  group:
+               name: cycle test
+               steps:
+               -  sequence:
+                     steps:
+                     -  loop:
+                           iterator: 10
+                           key: $(test)_PASS
+                           name: Cycle number of loops
+                           steps:
+                           -  py_func:
+                                 file: $(test_path)$(psep)cycle.py
+                                 func_name: donothing
+                                 name: do nothing
+                     -  loop:
+                           iterator:
+                           - 12
+                           - 20
+                           - 30
+                           key: $(test)_PASS
+                           name: Cycle iterating on list
+                           steps:
+                           -  py_func:
+                                 file: $(test_path)$(psep)cycle.py
+                                 func_name: checkloopparam
+                                 name: check loop param
+                                 param:
+                                 - $(loop_param)
+                     -  loop:
+                           exit_condition:
+                              file: $(test_path)$(psep)cycle.py
+                              func_name: exitcondition
+                           key: $(test)_PASS
+                           name: Infinite loop with exit condition
+                           steps:
+                           -  py_func:
+                                 file: $(test_path)$(psep)cycle.py
+                                 func_name: donothing
+                                 name: do nothing
+                     filename: /home/renish/workspace/testium/code/test/validation/items/cycle/test.tum
+               -  sequence:
+                     steps:
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.txt
+                           -  html:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.html
+                           -  junit:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.junit
+                           name: Expected PASS $(test) test
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.txt
+                           -  html:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.html
+                           -  junit:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.junit
+                           name: Expected FAIL $(test) test
+                     filename: /home/renish/workspace/testium/code/test/validation/items/report.tum
+report:
+   enabled: true
+   export:
+      junit:
+         file_name: $(validation_report_file).junit
+         path: $(validation_report_path)
+      sqlite:
+         file_name: $(validation_report_file).sqlite
+         path: $(validation_report_path)
+   log_stored: true

schema/test_schema/dialogs.tum

@@ -0,0 +1,135 @@
+config_file:
+- param.yaml
+- items/dialogs/param.yaml
+main:
+   name: Testium validation suite
+   steps:
+   -  group:
+         name: Test preparation
+         steps:
+         -  let:
+               condition: <| "$(os)" == "Linux" |>
+               name: Set test variables for Linux
+               values:
+               -  terminal_prompt: $(linux_prompt)
+               -  psep: /
+         -  let:
+               condition: <| "$(os)" == "Windows" |>
+               name: Set test variables for Windows
+               values:
+               -  terminal_prompt: $(windows_prompt)
+               -  psep: \
+   -  group:
+         name: Group of tests
+         steps:
+         -  let:
+               name: dialogs test constants
+               values:
+                  test: dialogs
+                  test_path: items/$(test)
+         -  group:
+               name: dialogs test
+               steps:
+               -  sequence:
+                     steps:
+                     -  dialog_image:
+                           auto_result: ok
+                           condition: $(validation_dialogs)
+                           filename: $(test_path)$(psep)IMG_20140213_171455.jpg
+                           key: $(test)_PASS
+                           name: dialog image PASS
+                           question: click ok if you see the image
+                     -  dialog_image:
+                           auto_result: cancel
+                           condition: $(validation_dialogs)
+                           filename: $(test_path)$(psep)IMG_20140213_171455.jpg
+                           key: $(test)_FAIL
+                           name: dialog image FAIL
+                           question: click cancel
+                     -  dialog_references:
+                           auto_result: ok
+                           condition: $(validation_dialogs)
+                           key: $(test)_PASS
+                           name: dialog_reference PASS
+                           question: click ok
+                     -  dialog_references:
+                           auto_result: cancel
+                           condition: $(validation_dialogs)
+                           key: $(test)_FAIL
+                           name: dialog_reference FAIL
+                           question: click cancel
+                     -  dialog_value:
+                           auto_result: ok
+                           auto_value: '123'
+                           condition: $(validation_dialogs)
+                           key: $(test)_PASS
+                           name: dialog_value PASS
+                           question: enter 123 and click ok
+                     -  dialog_value:
+                           auto_result: ok
+                           condition: $(validation_dialogs)
+                           key: $(test)_FAIL
+                           name: dialog_value empty FAIL
+                           question: enter nothing and click ok
+                     -  dialog_value:
+                           auto_result: cancel
+                           condition: $(validation_dialogs)
+                           key: $(test)_FAIL
+                           name: dialog_value canceled FAIL
+                           question: enter nothing and click cancel
+                     -  dialog_message:
+                           auto_result: ok
+                           condition: $(validation_dialogs)
+                           key: $(test)_PASS
+                           name: dialog_message PASS
+                           question: click ok
+                     -  dialog_question:
+                           auto_result: 'yes'
+                           condition: $(validation_dialogs)
+                           key: $(test)_PASS
+                           name: dialog_question PASS
+                           question: click yes
+                     -  dialog_question:
+                           auto_result: 'no'
+                           condition: $(validation_dialogs)
+                           key: $(test)_FAIL
+                           name: dialog_question FAIL
+                           question: click no
+                     filename: /home/renish/workspace/testium/code/test/validation/items/dialogs/test.tum
+               -  sequence:
+                     steps:
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.txt
+                           -  html:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.html
+                           -  junit:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.junit
+                           name: Expected PASS $(test) test
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.txt
+                           -  html:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.html
+                           -  junit:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.junit
+                           name: Expected FAIL $(test) test
+                     filename: /home/renish/workspace/testium/code/test/validation/items/report.tum
+report:
+   enabled: true
+   export:
+      junit:
+         file_name: $(validation_report_file).junit
+         path: $(validation_report_path)
+      sqlite:
+         file_name: $(validation_report_file).sqlite
+         path: $(validation_report_path)
+   log_stored: true

schema/test_schema/expanse.tum

@@ -0,0 +1,130 @@
+config_file:
+- param.yaml
+- items/expanse/param.yaml
+main:
+   name: Testium validation suite
+   steps:
+   -  group:
+         name: Test preparation
+         steps:
+         -  let:
+               condition: <| "$(os)" == "Linux" |>
+               name: Set test variables for Linux
+               values:
+               -  terminal_prompt: $(linux_prompt)
+               -  psep: /
+         -  let:
+               condition: <| "$(os)" == "Windows" |>
+               name: Set test variables for Windows
+               values:
+               -  terminal_prompt: $(windows_prompt)
+               -  psep: \
+   -  group:
+         name: Group of tests
+         steps:
+         -  let:
+               name: expanse test constants
+               values:
+                  test: expanse
+                  test_path: items/$(test)
+         -  group:
+               name: expanse test
+               steps:
+               -  sequence:
+                     steps:
+                     -  check:
+                           key: $(test)_PASS
+                           name: Check variables expansion is correct (PASS)
+                           values:
+                           - <| $(expanse_index) == 1 |>
+                           - <| $(expanse_table)[$(expanse_index)] == 9012 |>
+                           - <| $(expanse_eval) == True |>
+                     -  let:
+                           key: $(test)_PASS
+                           name: Dynamic variables expansion
+                           values:
+                           -  expanse_select: <|"$(expanse_select)".replace("o", "a")|>
+                           -  expanse_index: $(expanse_index_$(expanse_select))
+                           -  expanse_table: $(expanse_table_$(expanse_select))
+                           -  expanse_eval: <|$(expanse_index) == 1|>
+                     -  check:
+                           key: $(test)_PASS
+                           name: Check variables expansion is correct (PASS)
+                           values:
+                           - <| $(expanse_index) == 0 |>
+                           - <| $(expanse_table)[$(expanse_index)] == "abcd" |>
+                           - <| $(expanse_eval) == False |>
+                     -  let:
+                           key: $(test)_PASS
+                           name: Complex variables expansion
+                           values:
+                           -  var1: expanse
+                           -  var2: var
+                           -  var3: bla
+                           -  var4: blo
+                           -  expanse_var_bla: 3
+                           -  expanse_blo_var: 5
+                           -  expanse_complex: <|<|$(expanse_$(var2)_$(var3))*6|>
+                                 + <|4*$($(var1)_$(var4)_$(var2))|>|>
+                     -  check:
+                           key: $(test)_PASS
+                           name: Check complex variables expansion is correct (PASS)
+                           values:
+                           - <| $(expanse_complex) == 38 |>
+                     -  let:
+                           key: $(test)_PASS
+                           name: Variables expansion in object
+                           values:
+                           -  expanse_key: b
+                           -  expanse_var: 3
+                           -  expanse_var_2: 6
+                           -  expanse_object:
+                              -  $(expanse_key): <|2**3|>
+                                 a: $(expanse_var_2)
+                              -  <|"bla".replace("a", "o")|>:
+                                 - <|$(expanse_var)*$(expanse_var_2)|>
+                                 - 25
+                     -  check:
+                           key: $(test)_PASS
+                           name: Check complex variables expansion is correct (PASS)
+                           values:
+                           - '<| $(expanse_object) == [{"a": 6, "b": 8}, {"blo": [18,
+                              25]}] |>'
+                     filename: /home/renish/workspace/testium/code/test/validation/items/expanse/test.tum
+               -  sequence:
+                     steps:
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.txt
+                           -  html:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.html
+                           -  junit:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.junit
+                           name: Expected PASS $(test) test
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.txt
+                           -  html:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.html
+                           -  junit:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.junit
+                           name: Expected FAIL $(test) test
+                     filename: /home/renish/workspace/testium/code/test/validation/items/report.tum
+report:
+   enabled: true
+   export:
+      junit:
+         file_name: $(validation_report_file).junit
+         path: $(validation_report_path)
+      sqlite:
+         file_name: $(validation_report_file).sqlite
+         path: $(validation_report_path)
+   log_stored: true

schema/test_schema/git.tum

@@ -0,0 +1,82 @@
+config_file:
+- param.yaml
+- items/git/param.yaml
+main:
+   name: Testium validation suite
+   steps:
+   -  group:
+         name: Test preparation
+         steps:
+         -  let:
+               condition: <| "$(os)" == "Linux" |>
+               name: Set test variables for Linux
+               values:
+               -  terminal_prompt: $(linux_prompt)
+               -  psep: /
+         -  let:
+               condition: <| "$(os)" == "Windows" |>
+               name: Set test variables for Windows
+               values:
+               -  terminal_prompt: $(windows_prompt)
+               -  psep: \
+   -  group:
+         name: Group of tests
+         steps:
+         -  let:
+               name: git test constants
+               values:
+                  test: git
+                  test_path: items/$(test)
+         -  group:
+               name: git test
+               steps:
+               -  sequence:
+                     steps:
+                     -  git:
+                           key: $(test)_PASS
+                           name: Testium repo
+                           repo: $(test_directory)
+                     -  git:
+                           key: $(test)_PASS
+                           name: Testium repo
+                           repo:
+                           - $(test_directory)
+                           - $(test_directory)
+                     filename: /home/renish/workspace/testium/code/test/validation/items/git/test.tum
+               -  sequence:
+                     steps:
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.txt
+                           -  html:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.html
+                           -  junit:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.junit
+                           name: Expected PASS $(test) test
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.txt
+                           -  html:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.html
+                           -  junit:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.junit
+                           name: Expected FAIL $(test) test
+                     filename: /home/renish/workspace/testium/code/test/validation/items/report.tum
+report:
+   enabled: true
+   export:
+      junit:
+         file_name: $(validation_report_file).junit
+         path: $(validation_report_path)
+      sqlite:
+         file_name: $(validation_report_file).sqlite
+         path: $(validation_report_path)
+   log_stored: true

schema/test_schema/include.tum

@@ -0,0 +1,129 @@
+config_file:
+- param.yaml
+- items/include/param.yaml
+main:
+   name: Testium validation suite
+   steps:
+   -  group:
+         name: Test preparation
+         steps:
+         -  let:
+               condition: <| "$(os)" == "Linux" |>
+               name: Set test variables for Linux
+               values:
+               -  terminal_prompt: $(linux_prompt)
+               -  psep: /
+         -  let:
+               condition: <| "$(os)" == "Windows" |>
+               name: Set test variables for Windows
+               values:
+               -  terminal_prompt: $(windows_prompt)
+               -  psep: \
+   -  group:
+         name: Group of tests
+         steps:
+         -  let:
+               name: include test constants
+               values:
+                  test: include
+                  test_path: items/$(test)
+         -  group:
+               name: include test
+               steps:
+               -  sequence:
+                     steps:
+                     -  sequence:
+                           steps:
+                           -  py_func:
+                                 file: $(test_path)$(psep)include.py
+                                 func_name: ValidationTest
+                                 key: $(test)_PASS
+                                 name: My first include test
+                                 param:
+                                 - $(test parameter)
+                           filename: /home/renish/workspace/testium/code/test/validation/items/include/inc
+                              no template/my first include.tum
+                     -  sequence:
+                           steps:
+                           -  py_func:
+                                 file: $(test_path)$(psep)include.py
+                                 func_name: ValidationTest
+                                 key: $(test)_PASS
+                                 name: My first include test
+                                 param:
+                                 - $(test parameter)
+                           filename: /home/renish/workspace/testium/code/test/validation/items/include/inc
+                              no template/my first include.tum
+                     -  sequence:
+                           steps:
+                           -  py_func:
+                                 file: $(test_path)$(psep)include.py
+                                 func_name: ValidationTest
+                                 key: $(test)_PASS
+                                 name: My second include test
+                                 param:
+                                 - My second include test parameter
+                           filename: /home/renish/workspace/testium/code/test/validation/items/include/inc
+                              with template/my second include.tum
+                     -  sequence:
+                           steps:
+                           -  py_func:
+                                 file: $(test_path)$(psep)include.py
+                                 func_name: ValidationTest
+                                 key: $(test)_PASS
+                                 name: My second include test
+                                 param:
+                                 - My second include test parameter
+                           filename: /home/renish/workspace/testium/code/test/validation/items/include/inc
+                              with template/my second include.tum
+                     -  let:
+                           name: Declare param for inclusion
+                           values:
+                           -  inc: Dali
+                           -  Dali_inc: Dalida
+                     -  sequence:
+                           steps:
+                           -  let:
+                                 name: Test param inclusion 1
+                                 values:
+                                 -  inclusion: $($(inc)_inc)
+                           filename: /home/renish/workspace/testium/code/test/validation/items/include/inc
+                              with template/my_3d_include.tum
+                     filename: /home/renish/workspace/testium/code/test/validation/items/include/test.tum
+               -  sequence:
+                     steps:
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.txt
+                           -  html:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.html
+                           -  junit:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.junit
+                           name: Expected PASS $(test) test
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.txt
+                           -  html:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.html
+                           -  junit:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.junit
+                           name: Expected FAIL $(test) test
+                     filename: /home/renish/workspace/testium/code/test/validation/items/report.tum
+report:
+   enabled: true
+   export:
+      junit:
+         file_name: $(validation_report_file).junit
+         path: $(validation_report_path)
+      sqlite:
+         file_name: $(validation_report_file).sqlite
+         path: $(validation_report_path)
+   log_stored: true

schema/test_schema/isolation.tum

@@ -0,0 +1,80 @@
+config_file:
+- param.yaml
+- items/isolation/param.yaml
+main:
+   name: Testium validation suite
+   steps:
+   -  group:
+         name: Test preparation
+         steps:
+         -  let:
+               condition: <| "$(os)" == "Linux" |>
+               name: Set test variables for Linux
+               values:
+               -  terminal_prompt: $(linux_prompt)
+               -  psep: /
+         -  let:
+               condition: <| "$(os)" == "Windows" |>
+               name: Set test variables for Windows
+               values:
+               -  terminal_prompt: $(windows_prompt)
+               -  psep: \
+   -  group:
+         name: Group of tests
+         steps:
+         -  let:
+               name: isolation test constants
+               values:
+                  test: isolation
+                  test_path: items/$(test)
+         -  group:
+               name: isolation test
+               steps:
+               -  sequence:
+                     steps:
+                     -  py_func:
+                           expected_result: true
+                           file: $(test_path)$(psep)check_isolation.py
+                           func_name: check_isolation
+                           key: $(test)_PASS
+                           name: py_func/lua_func do not depend on testium internals
+                           param:
+                           - $(testium_path)
+                     filename: /home/renish/workspace/testium/code/test/validation/items/isolation/test.tum
+               -  sequence:
+                     steps:
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.txt
+                           -  html:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.html
+                           -  junit:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.junit
+                           name: Expected PASS $(test) test
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.txt
+                           -  html:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.html
+                           -  junit:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.junit
+                           name: Expected FAIL $(test) test
+                     filename: /home/renish/workspace/testium/code/test/validation/items/report.tum
+report:
+   enabled: true
+   export:
+      junit:
+         file_name: $(validation_report_file).junit
+         path: $(validation_report_path)
+      sqlite:
+         file_name: $(validation_report_file).sqlite
+         path: $(validation_report_path)
+   log_stored: true

schema/test_schema/jsonrpc.tum

@@ -0,0 +1,490 @@
+config_file:
+- param.yaml
+- items/jsonrpc/param.yaml
+main:
+   name: Testium validation suite
+   steps:
+   -  group:
+         name: Test preparation
+         steps:
+         -  let:
+               condition: <| "$(os)" == "Linux" |>
+               name: Set test variables for Linux
+               values:
+               -  terminal_prompt: $(linux_prompt)
+               -  psep: /
+         -  let:
+               condition: <| "$(os)" == "Windows" |>
+               name: Set test variables for Windows
+               values:
+               -  terminal_prompt: $(windows_prompt)
+               -  psep: \
+   -  group:
+         name: Group of tests
+         steps:
+         -  let:
+               name: jsonrpc test constants
+               values:
+                  test: jsonrpc
+                  test_path: items/$(test)
+         -  group:
+               name: jsonrpc test
+               steps:
+               -  sequence:
+                     steps:
+                     -  console:
+                           console_name: jrpces
+                           doc: check if jrpc_echo_server.py is available
+                           key: $(test)_PASS
+                           name: json rpc echo server
+                           steps:
+                           -  open:
+                                 protocol: terminal
+                           -  read_until:
+                                 expected: $(terminal_prompt)
+                                 no_fail: true
+                                 timeout: 1
+                           -  writeln: test -f /home/renish/workspace/testium/code/test/validation/items/jsonrpc/jrpc_echo_server.py
+                                 && echo JRPC_OK
+                           -  read_until:
+                                 expected: JRPC_OK
+                                 no_fail: true
+                                 timeout: 2
+                     -  group:
+                           condition: <| 'JRPC_OK' in r'''$(cn_json rpc echo server)''' |>
+                           name: jsonrpc tests
+                           steps:
+                           -  console:
+                                 console_name: jrpces
+                                 key: $(test)_PASS
+                                 name: Start the json rpc echo server
+                                 steps:
+                                 -  writeln: python3 /home/renish/workspace/testium/code/test/validation/items/jsonrpc/jrpc_echo_server.py
+                                       -c /home/renish/workspace/testium/code/test/validation/items/jsonrpc/jrpces.ini
+                                 -  read_until:
+                                       expected: ready
+                                       timeout: 5
+                           -  console:
+                                 console_name: jsonrpc_server
+                                 doc: Opening the RAW TCP console
+                                 key: $(test)_PASS
+                                 name: Open the raw tcp Console
+                                 skipped: $(skip_tcp)
+                                 steps:
+                                 -  open:
+                                       protocol: rawtcp
+                                       tcp_host: localhost
+                                       tcp_port: 4321
+                           -  json_rpc:
+                                 console:
+                                    name: jsonrpc_server
+                                 key: $(test)_PASS
+                                 name: JSONRPC console Query waiting for reception
+                                 skipped: $(skip_tcp)
+                                 steps:
+                                 -  query:
+                                       method: echo
+                                       params:
+                                       - Hello World
+                                       -  a: 1
+                                          b: hello
+                                 timeout: 1
+                           -  json_rpc:
+                                 console:
+                                    name: jsonrpc_server
+                                 key: $(test)_PASS
+                                 name: JSONRPC console Query not waiting (only send)
+                                 skipped: $(skip_tcp)
+                                 steps:
+                                 -  query:
+                                       id: 3095372
+                                       method: echo
+                                       no_wait: true
+                                       params:
+                                       -  a: -1
+                                          b: olleh
+                                       - World Hello
+                                 timeout: 1
+                           -  sleep:
+                                 name: Small delay for the test
+                                 skipped: $(skip_tcp)
+                                 timeout: 1
+                           -  json_rpc:
+                                 console:
+                                    name: jsonrpc_server
+                                 key: $(test)_PASS
+                                 name: JSONRPC console Reception
+                                 skipped: $(skip_tcp)
+                                 steps:
+                                 -  receive:
+                                       id: 3095372
+                                 timeout: 1
+                           -  console:
+                                 console_name: jsonrpc_server
+                                 doc: Opening the RAW TCP console
+                                 execute_on_stop: true
+                                 key: $(test)_PASS
+                                 name: Close the raw tcp console
+                                 skipped: $(skip_tcp)
+                                 steps:
+                                 -  close: null
+                           -  json_rpc:
+                                 key: $(test)_PASS
+                                 name: JSONRPC UDP query waiting for reception
+                                 steps:
+                                 -  open: null
+                                 -  query:
+                                       method: echo
+                                       name: echo
+                                       params:
+                                       - Hello World
+                                       -  a: 1
+                                          b: hello
+                                       timeout: 1
+                                 -  close: null
+                                 timeout: 1
+                                 udp:
+                                    rcv_port: 8765
+                                    server: localhost
+                                    snd_port: 4323
+                           -  json_rpc:
+                                 key: $(test)_FAIL
+                                 name: Failing JSONRPC UDP query waiting for reception
+                                    (returning an error)
+                                 steps:
+                                 -  open: null
+                                 -  query:
+                                       method: echo2
+                                       params:
+                                       - Hello World
+                                       -  a: 1
+                                          b: hello
+                                       timeout: 1
+                                 -  close: null
+                                 timeout: 1
+                                 udp:
+                                    rcv_port: 8765
+                                    server: localhost
+                                    snd_port: 4323
+                           -  json_rpc:
+                                 key: $(test)_PASS
+                                 name: JSONRPC UDP query waiting for reception of
+                                    an expected error
+                                 steps:
+                                 -  open: null
+                                 -  query:
+                                       expected_result:
+                                          code: -32000
+                                          message: function not found
+                                       method: echo2
+                                       params:
+                                       - Hello World
+                                       -  a: 1
+                                          b: hello
+                                 -  close: null
+                                 timeout: 1
+                                 udp:
+                                    rcv_port: 8765
+                                    server: localhost
+                                    snd_port: 4323
+                           -  json_rpc:
+                                 doc: 'Failing JSONRPC UDP query waiting for reception
+                                    and checking result
+
+                                    and timeout elapses (wrong udp port)
+
+                                    '
+                                 key: $(test)_FAIL
+                                 name: Failing UDP JSONRPC query timeout elapses (wrong
+                                    udp port)
+                                 steps:
+                                 -  open: null
+                                 -  query:
+                                       method: echo
+                                       params:
+                                       - Hello World
+                                       -  a: 1
+                                          b: hello
+                                       timeout: 0.5
+                                 -  close: null
+                                 timeout: 1
+                                 udp:
+                                    rcv_port: 48393
+                                    server: localhost
+                                    snd_port: 4326
+                           -  json_rpc:
+                                 key: $(test)_PASS
+                                 name: JSONRPC UDP query not waiting (only send)
+                                 steps:
+                                 -  open: null
+                                 -  query:
+                                       id: 3095372
+                                       method: echo
+                                       no_wait: true
+                                       params:
+                                       -  a: -1
+                                          b: olleh
+                                       - World Hello
+                                 timeout: 1
+                                 udp:
+                                    rcv_port: 8765
+                                    server: localhost
+                                    snd_port: 4323
+                           -  sleep:
+                                 name: Small delay for the test
+                                 timeout: 1
+                           -  json_rpc:
+                                 key: $(test)_PASS
+                                 name: JSONRPC UDP Reception
+                                 steps:
+                                 -  receive:
+                                       id: 3095372
+                                 -  close: null
+                                 timeout: 1
+                                 udp:
+                                    rcv_port: 8765
+                                    server: localhost
+                                    snd_port: 4323
+                           -  json_rpc:
+                                 key: $(test)_PASS
+                                 name: JSONRPC UDP query not waiting (only send)
+                                 steps:
+                                 -  open: null
+                                 -  query:
+                                       id: 3095372
+                                       method: echo2
+                                       no_wait: true
+                                       params:
+                                       -  a: -1
+                                          b: olleh
+                                       - World Hello
+                                 timeout: 1
+                                 udp:
+                                    rcv_port: 8765
+                                    server: localhost
+                                    snd_port: 4323
+                           -  sleep:
+                                 name: Small delay for the test
+                                 timeout: 1
+                           -  json_rpc:
+                                 key: $(test)_FAIL
+                                 name: Failing JSONRPC UDP Reception (returning an
+                                    error)
+                                 steps:
+                                 -  receive:
+                                       id: 3095372
+                                       timeout: 1
+                                 -  close: null
+                                 timeout: 1
+                                 udp:
+                                    rcv_port: 8765
+                                    server: localhost
+                                    snd_port: 4323
+                           -  json_rpc:
+                                 doc: JSONRPC UDP query waiting for reception and
+                                    checking result
+                                 key: $(test)_PASS
+                                 name: UDP JSONRPC query waiting and checking
+                                 steps:
+                                 -  open: null
+                                 -  query:
+                                       expected_result:
+                                       -  - Hello World
+                                          -  a: 1
+                                             b: hello
+                                       - {}
+                                       method: echo
+                                       params:
+                                       - Hello World
+                                       -  a: 1
+                                          b: hello
+                                       timeout: 1
+                                 -  close: null
+                                 timeout: 1
+                                 udp:
+                                    rcv_port: 48393
+                                    server: localhost
+                                    snd_port: 4323
+                           -  json_rpc:
+                                 doc: JSONRPC UDP query waiting for reception and
+                                    checking result
+                                 key: $(test)_FAIL
+                                 name: Failing UDP JSONRPC query waiting and checking
+                                 steps:
+                                 -  open: null
+                                 -  query:
+                                       expected_result:
+                                       - []
+                                       - {}
+                                       method: echo
+                                       params:
+                                       - Hello World
+                                       -  a: 1
+                                          b: hello
+                                       timeout: 1
+                                 -  close: null
+                                 timeout: 1
+                                 udp:
+                                    rcv_port: 48393
+                                    server: localhost
+                                    snd_port: 4323
+                           -  json_rpc:
+                                 doc: JSONRPC UDP query not waiting, with the purpose
+                                    to check the result at reception
+                                 key: $(test)_PASS
+                                 name: UDP JSONRPC query not waiting (for checking)
+                                 steps:
+                                 -  open: null
+                                 -  query:
+                                       id: 3095372
+                                       method: echo
+                                       no_wait: true
+                                       params:
+                                       -  a: -1
+                                          b: olleh
+                                       - World Hello
+                                 timeout: 1
+                                 udp:
+                                    rcv_port: 9876
+                                    server: localhost
+                                    snd_port: 4323
+                           -  sleep:
+                                 name: Small delay for the test
+                                 timeout: 1
+                           -  json_rpc:
+                                 doc: JSONRPC UDP Reception and checking result
+                                 key: $(test)_PASS
+                                 name: UDP JSONRPC reception checking
+                                 steps:
+                                 -  receive:
+                                       expected_result:
+                                       -  -  a: -1
+                                             b: olleh
+                                          - World Hello
+                                       - {}
+                                       id: 3095372
+                                       timeout: 1
+                                 -  close: null
+                                 timeout: 1
+                                 udp:
+                                    rcv_port: 9876
+                                    server: localhost
+                                    snd_port: 4323
+                           -  json_rpc:
+                                 doc: JSONRPC UDP Reception and checking result
+                                 key: $(test)_FAIL
+                                 name: Failing UDP JSONRPC reception checking
+                                 steps:
+                                 -  receive:
+                                       expected_result:
+                                       -  -  a: -1
+                                             b: ollhe
+                                          - World Hello
+                                       - {}
+                                       id: 3095372
+                                       timeout: 1
+                                 -  close: null
+                                 timeout: 1
+                                 udp:
+                                    rcv_port: 9876
+                                    server: localhost
+                                    snd_port: 4323
+                           -  json_rpc:
+                                 doc: 'JSONRPC UDP query waiting for reception and
+                                    checking result with
+
+                                    replacing $(result) and evaluating string.
+
+                                    '
+                                 key: $(test)_PASS
+                                 name: UDP JSONRPC query waiting and evaluating result
+                                 steps:
+                                 -  open: null
+                                 -  query:
+                                       expected_result: 1
+                                       method: echo
+                                       params:
+                                       - Hello World
+                                       -  a: 1
+                                          b: hello
+                                       process_result: $(result)[0][1]['a']
+                                       timeout: 1
+                                 -  close: null
+                                 timeout: 1
+                                 udp:
+                                    rcv_port: 48393
+                                    server: localhost
+                                    snd_port: 4323
+                           -  json_rpc:
+                                 doc: 'JSONRPC UDP query waiting for reception and
+                                    checking result with
+
+                                    replacing $(result) and evaluating string.
+
+                                    '
+                                 key: $(test)_FAIL
+                                 name: Failing UDP JSONRPC query waiting and evaluating
+                                    result
+                                 steps:
+                                 -  open: null
+                                 -  query:
+                                       expected_result: $(result)[0][1]['a'] == 0
+                                       method: echo
+                                       params:
+                                       - Hello World
+                                       -  a: 1
+                                          b: hello
+                                       timeout: 1
+                                 -  close: null
+                                 timeout: 1
+                                 udp:
+                                    rcv_port: 48393
+                                    server: localhost
+                                    snd_port: 4323
+                     -  console:
+                           console_name: jrpces
+                           doc: check if the jsonrpc echo server is installed
+                           execute_on_stop: true
+                           key: $(test)_PASS
+                           name: Stop json rpc echo server
+                           steps:
+                           -  close:
+                                 protocol: terminal
+                     filename: /home/renish/workspace/testium/code/test/validation/items/jsonrpc/test.tum
+               -  sequence:
+                     steps:
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.txt
+                           -  html:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.html
+                           -  junit:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.junit
+                           name: Expected PASS $(test) test
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.txt
+                           -  html:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.html
+                           -  junit:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.junit
+                           name: Expected FAIL $(test) test
+                     filename: /home/renish/workspace/testium/code/test/validation/items/report.tum
+report:
+   enabled: true
+   export:
+      junit:
+         file_name: $(validation_report_file).junit
+         path: $(validation_report_path)
+      sqlite:
+         file_name: $(validation_report_file).sqlite
+         path: $(validation_report_path)
+   log_stored: true

schema/test_schema/let.tum

@@ -0,0 +1,157 @@
+config_file:
+- param.yaml
+- items/let/param.yaml
+main:
+   name: Testium validation suite
+   steps:
+   -  group:
+         name: Test preparation
+         steps:
+         -  let:
+               condition: <| "$(os)" == "Linux" |>
+               name: Set test variables for Linux
+               values:
+               -  terminal_prompt: $(linux_prompt)
+               -  psep: /
+         -  let:
+               condition: <| "$(os)" == "Windows" |>
+               name: Set test variables for Windows
+               values:
+               -  terminal_prompt: $(windows_prompt)
+               -  psep: \
+   -  group:
+         name: Group of tests
+         steps:
+         -  let:
+               name: let test constants
+               values:
+                  - test: let
+                  - test_path: items/$(test)
+         -  group:
+               name: let test
+               steps:
+               -  sequence:
+                     steps:
+                     -  loop:
+                           iterator: 10
+                           key: $(test)_PASS
+                           name: Cycle number of loops
+                           steps:
+                           -  py_func:
+                                 file: $(test_path)$(psep)let.py
+                                 func_name: donothing
+                                 name: do nothing
+                           -  let:
+                                 name: Let it be
+                                 values:
+                                    - be: <| $(loop_param) == $(it) |>
+                                    - it: $(loop_param)
+                     -  loop:
+                           iterator:
+                           - 12
+                           - 20
+                           - 30
+                           key: $(test)_PASS
+                           name: Cycle iterating on list
+                           steps:
+                           -  py_func:
+                                 file: $(test_path)$(psep)let.py
+                                 func_name: checkloopparam
+                                 name: check loop param
+                                 param:
+                                 - $(loop_param)
+                           -  let:
+                                 name: Let it be
+                                 values:
+                                 -  it: $(loop_param)
+                                 -  be: <| $(loop_param) == $(it) |>
+                     -  let:
+                           key: $(test)_PASS
+                           name: Get time
+                           values:
+                           -  loop_t0: $(ts_start_Cycle iterating on list)
+                           -  loop_t1: $(ts_end_Cycle iterating on list)
+                           -  loop_duration: $(ts_duration_Cycle iterating on list)
+                     -  let:
+                           key: $(test)_PASS
+                           name: Get parameter file value
+                           values:
+                           -  test_overwrite_me: <| $(overwrite_me) == True |>
+                     -  py_func:
+                           file: $(test_path)$(psep)let.py
+                           func_name: checkGlobalDic
+                           name: Check global dic pass
+                           param:
+                           - test_overwrite_me
+                           - true
+                     -  let:
+                           key: $(test)_PASS
+                           name: Overwrite parameter file value
+                           values:
+                           -  overwrite_me: false
+                     -  py_func:
+                           expected_result: $(overwrite_me) == False
+                           file: $(test_path)$(psep)let.py
+                           func_name: checkGlobalDic
+                           key: $(test)_FAIL
+                           name: Check global dic fail
+                           param:
+                           - overwrite_me
+                           - true
+                     -  py_func:
+                           expected_result: fail
+                           file: $(test_path)$(psep)let.py
+                           func_name: checkGlobalDic
+                           key: $(test)_PASS
+                           name: Check global dic fail
+                           param:
+                           - overwrite_me
+                           - true
+                     -  let:
+                           key: $(test)_PASS
+                           name: Evaluate Overwriting parameter file value
+                           values:
+                           -  test_overwrite_me: <| "$(overwrite_me)" == True |>
+                     -  check:
+                           key: $(test)_PASS
+                           name: Check Overwriting parameter file value
+                           values:
+                           - <| $(test_overwrite_me) == False |>
+                     filename: /home/renish/workspace/testium/code/test/validation/items/let/test.tum
+               -  sequence:
+                     steps:
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.txt
+                           -  html:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.html
+                           -  junit:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.junit
+                           name: Expected PASS $(test) test
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.txt
+                           -  html:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.html
+                           -  junit:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.junit
+                           name: Expected FAIL $(test) test
+                     filename: /home/renish/workspace/testium/code/test/validation/items/report.tum
+report:
+   enabled: true
+   export:
+      junit:
+         file_name: $(validation_report_file).junit
+         path: $(validation_report_path)
+      sqlite:
+         file_name: $(validation_report_file).sqlite
+         path: $(validation_report_path)
+   log_stored: true

schema/test_schema/lua_func.tum

@@ -0,0 +1,297 @@
+config_file:
+- param.yaml
+- items/lua_func/param.yaml
+main:
+   name: Testium validation suite
+   steps:
+   -  group:
+         name: Test preparation
+         steps:
+         -  let:
+               condition: <| "$(os)" == "Linux" |>
+               name: Set test variables for Linux
+               values:
+               -  terminal_prompt: $(linux_prompt)
+               -  psep: /
+         -  let:
+               condition: <| "$(os)" == "Windows" |>
+               name: Set test variables for Windows
+               values:
+               -  terminal_prompt: $(windows_prompt)
+               -  psep: \
+   -  group:
+         name: Group of tests
+         steps:
+         -  let:
+               name: lua_func test constants
+               values:
+                  test: lua_func
+                  test_path: items/$(test)
+         -  group:
+               name: lua_func test
+               steps:
+               -  sequence:
+                     steps:
+                     -  let:
+                           name: lua_func test constants,
+                           values:
+                              lua_func test parameter: test parameter lua_func
+                     -  lua_func:
+                           file: $(test_path)$(psep)lua_func.lua
+                           func_name: assertparam
+                           key: $(test)_FAIL
+                           name: fail lua_func
+                           param:
+                           - false
+                     -  lua_func:
+                           expected_result: FAIL
+                           file: $(test_path)$(psep)lua_func.lua
+                           func_name: assertparam
+                           key: $(test)_PASS
+                           name: fail lua_func with expected result FAIL
+                           param:
+                           - false
+                     -  lua_func:
+                           expected_result: FAIL
+                           file: $(test_path)$(psep)lua_func.lua
+                           func_name: assertparam
+                           key: $(test)_FAIL
+                           name: pass lua_func with expected result FAIL
+                           param:
+                           - true
+                     -  lua_func:
+                           expected_result: -1
+                           file: $(test_path)$(psep)lua_func.lua
+                           func_name: echo
+                           key: $(test)_PASS
+                           name: expected -1
+                           param:
+                           - -1
+                     -  lua_func:
+                           expected_result: 354848436 - 354848437
+                           file: $(test_path)$(psep)lua_func.lua
+                           func_name: echo
+                           key: $(test)_PASS
+                           name: expected eval
+                           param:
+                           - -1
+                     -  lua_func:
+                           expected_result: '[-1, ''a'', {''toto'': ''tata''}]'
+                           file: $(test_path)$(psep)lua_func.lua
+                           func_name: echo
+                           key: $(test)_PASS
+                           name: expected table
+                           param:
+                           -  - -1
+                              - a
+                              -  toto: tata
+                     -  lua_func:
+                           expected_result: $(lua_func test parameter)
+                           file: $(test_path)$(psep)lua_func.lua
+                           func_name: checkglobal
+                           key: $(test)_PASS
+                           name: global param lua_func
+                           param:
+                           - lua_func test parameter
+                     -  lua_func:
+                           expected_result: ($(lua_data_to_be_returned))[0]
+                           file: $(test_path)$(psep)lua_func.lua
+                           func_name: checkglobal2
+                           key: $(test)_PASS
+                           name: global param lua_func 1
+                           param:
+                           - 1
+                     -  lua_func:
+                           expected_result: ($(lua_data_to_be_returned))[1]
+                           file: $(test_path)$(psep)lua_func.lua
+                           func_name: checkglobal2
+                           key: $(test)_PASS
+                           name: global param lua_func 2
+                           param:
+                           - 2
+                     -  lua_func:
+                           expected_result: ($(lua_data_to_be_returned))[2]
+                           file: $(test_path)$(psep)lua_func.lua
+                           func_name: checkglobal2
+                           key: $(test)_PASS
+                           name: global param lua_func 3
+                           param:
+                           - 3
+                     -  let:
+                           key: $(test)_PASS
+                           name: python2func
+                           values:
+                           -  py: $(test_path)$(psep)lua_func.lua
+                     -  lua_func:
+                           file: $(test_path)$(psep)lua_func.lua
+                           func_name: should_not_be_called
+                           name: skipped_checkglobal
+                           param:
+                           - $(test parameter)
+                     -  lua_func:
+                           file: $(test_path)$(psep)lua_func.lua
+                           func_name: checkglobal
+                           name: skipped true
+                           param:
+                           - $(test parameter)
+                           skipped: true
+                     -  lua_func:
+                           file: $(test_path)$(psep)lua_func.lua
+                           func_name: checkglobal
+                           name: skipped 1
+                           param:
+                           - $(test parameter)
+                           skipped: true
+                     -  group:
+                           name: Function results check
+                           steps:
+                           -  group:
+                                 name: Function result failure
+                                 steps:
+                                 -  lua_func:
+                                       file: $(test_path)$(psep)lua_func.lua
+                                       func_name: echo
+                                       key: $(test)_PASS
+                                       name: int failure
+                                       param:
+                                       - -1
+                                 -  lua_func:
+                                       file: $(test_path)$(psep)lua_func.lua
+                                       func_name: echo
+                                       key: $(test)_PASS
+                                       name: float failure
+                                       param:
+                                       - -1.3
+                                 -  lua_func:
+                                       file: $(test_path)$(psep)lua_func.lua
+                                       func_name: echo
+                                       key: $(test)_PASS
+                                       name: String failure
+                                       param:
+                                       - FAIL
+                                 -  lua_func:
+                                       file: $(test_path)$(psep)lua_func.lua
+                                       func_name: tuple_return
+                                       key: $(test)_PASS
+                                       name: Tuple int,str failure
+                                       param:
+                                       - -1
+                                       - Got a failure
+                           -  group:
+                                 name: Functions result success
+                                 steps:
+                                 -  lua_func:
+                                       file: $(test_path)$(psep)lua_func.lua
+                                       func_name: echo
+                                       key: $(test)_PASS
+                                       name: int success
+                                       param:
+                                       - 0
+                                 -  lua_func:
+                                       file: $(test_path)$(psep)lua_func.lua
+                                       func_name: echo
+                                       key: $(test)_PASS
+                                       name: float success
+                                       param:
+                                       - 0.3
+                                 -  lua_func:
+                                       file: $(test_path)$(psep)lua_func.lua
+                                       func_name: echo
+                                       key: $(test)_PASS
+                                       name: String success
+                                       param:
+                                       - Something that is not only strictly FAIL
+                                 -  lua_func:
+                                       file: $(test_path)$(psep)lua_func.lua
+                                       func_name: tuple_return
+                                       key: $(test)_PASS
+                                       name: Tuple int,str success
+                                       param:
+                                       - 0
+                                       - OK
+                     -  lua_func:
+                           file: $(test_path)$(psep)lua_func.lua
+                           func_name: test_delgd
+                           key: $(test)_PASS
+                           name: delgd test
+                     -  lua_func:
+                           file: $(test_path)$(psep)lua_func.lua
+                           func_name: return_nothing
+                           key: $(test)_PASS
+                           name: function returning nothing should succeed
+                     -  lua_func:
+                           file: $(test_path)$(psep)lua_func.lua
+                           func_name: return_explicit_nil
+                           key: $(test)_PASS
+                           name: function returning explicit nil should succeed
+                     -  group:
+                           name: context_id tests
+                           steps:
+                           -  lua_func:
+                                 context_id: lua_ctx_test
+                                 expected_result: hello lua
+                                 file: $(test_path)$(psep)lua_func.lua
+                                 func_name: set_context_value
+                                 key: $(test)_PASS
+                                 name: set context value
+                                 param:
+                                 - hello lua
+                           -  lua_func:
+                                 context_id: lua_ctx_test
+                                 expected_result: hello lua
+                                 file: $(test_path)$(psep)lua_func.lua
+                                 func_name: get_context_value
+                                 key: $(test)_PASS
+                                 name: get context value (same context_id)
+                           -  lua_func:
+                                 expected_result: hello lua
+                                 file: $(test_path)$(psep)lua_func.lua
+                                 func_name: get_context_value
+                                 key: $(test)_PASS
+                                 name: get context value (no context_id, from main
+                                    gd)
+                           -  lua_func:
+                                 context_id: lua_ctx_other
+                                 expected_result: hello lua
+                                 file: $(test_path)$(psep)lua_func.lua
+                                 func_name: get_context_value
+                                 key: $(test)_PASS
+                                 name: get context value (different context_id)
+                     filename: /home/renish/workspace/testium/code/test/validation/items/lua_func/test.tum
+               -  sequence:
+                     steps:
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.txt
+                           -  html:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.html
+                           -  junit:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.junit
+                           name: Expected PASS $(test) test
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.txt
+                           -  html:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.html
+                           -  junit:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.junit
+                           name: Expected FAIL $(test) test
+                     filename: /home/renish/workspace/testium/code/test/validation/items/report.tum
+report:
+   enabled: true
+   export:
+      junit:
+         file_name: $(validation_report_file).junit
+         path: $(validation_report_path)
+      sqlite:
+         file_name: $(validation_report_file).sqlite
+         path: $(validation_report_path)
+   log_stored: true

schema/test_schema/parallel.tum

@@ -0,0 +1,414 @@
+config_file:
+- param.yaml
+- items/parallel/param.yaml
+main:
+   name: Testium validation suite
+   steps:
+   -  group:
+         name: Test preparation
+         steps:
+         -  let:
+               condition: <| "$(os)" == "Linux" |>
+               name: Set test variables for Linux
+               values:
+               -  terminal_prompt: $(linux_prompt)
+               -  psep: /
+         -  let:
+               condition: <| "$(os)" == "Windows" |>
+               name: Set test variables for Windows
+               values:
+               -  terminal_prompt: $(windows_prompt)
+               -  psep: \
+   -  group:
+         name: Group of tests
+         steps:
+         -  let:
+               name: parallel test constants
+               values:
+                  test: parallel
+                  test_path: items/$(test)
+         -  group:
+               name: parallel test
+               steps:
+               -  sequence:
+                     steps:
+                     -  parallel:
+                           branches:
+                           -  name: Branch A
+                              steps:
+                              -  let:
+                                    name: Set A done
+                                    values:
+                                    -  branch_a_done: true
+                           -  name: Branch B
+                              steps:
+                              -  let:
+                                    name: Set B done
+                                    values:
+                                    -  branch_b_done: true
+                           key: $(test)_PASS
+                           name: Both branches pass
+                           sync: all
+                     -  check:
+                           key: $(test)_PASS
+                           name: Both branches ran
+                           values:
+                           - <| $(branch_a_done) == True |>
+                           - <| $(branch_b_done) == True |>
+                     -  parallel:
+                           branches:
+                           -  name: Pass branch
+                              steps:
+                              -  let:
+                                    name: Set pass flag
+                                    values:
+                                    -  pass_branch_ran: true
+                           -  name: Fail branch
+                              steps:
+                              -  py_func:
+                                    expected_result: fail
+                                    file: $(test_path)$(psep)parallel.py
+                                    func_name: sleep_func
+                                    name: Raise exception
+                                    param:
+                                    - 0
+                           key: $(test)_PASS
+                           name: One branch fails
+                           no_fail: true
+                           sync: all
+                     -  check:
+                           key: $(test)_PASS
+                           name: Pass branch still ran
+                           values:
+                           - <| $(pass_branch_ran) == True |>
+                     -  let:
+                           name: Reset slow flag
+                           values:
+                           -  slow_done: false
+                     -  parallel:
+                           branches:
+                           -  name: Fast branch
+                              steps:
+                              -  let:
+                                    name: Fast done
+                                    values:
+                                    -  fast_done: true
+                           -  name: Slow branch
+                              steps:
+                              -  py_func:
+                                    file: $(test_path)$(psep)parallel.py
+                                    func_name: sleep_func
+                                    name: Sleep 2s
+                                    param:
+                                    - 2
+                              -  let:
+                                    name: Slow done
+                                    values:
+                                    -  slow_done: true
+                           key: $(test)_PASS
+                           name: sync any - first wins
+                           sync: any
+                     -  check:
+                           key: $(test)_PASS
+                           name: Fast branch ran, slow branch was stopped
+                           values:
+                           - <| $(fast_done) == True |>
+                           - <| $(slow_done) == False |>
+                     -  let:
+                           name: Reset sync flag
+                           values:
+                           -  sync_flag: ''
+                           -  waiter_ran: false
+                     -  parallel:
+                           branches:
+                           -  name: Setter branch
+                              steps:
+                              -  py_func:
+                                    file: $(test_path)$(psep)parallel.py
+                                    func_name: sleep_func
+                                    name: Sleep 0.3s then set flag
+                                    param:
+                                    - 0.3
+                              -  let:
+                                    name: Set sync flag
+                                    values:
+                                    -  sync_flag: ready
+                           -  name: Waiter branch
+                              steps:
+                              -  let:
+                                    name: Got flag
+                                    values:
+                                    -  waiter_ran: true
+                              wait_for:
+                                 condition: <| "$(sync_flag)" == "ready" |>
+                                 timeout: 10
+                           key: $(test)_PASS
+                           name: wait_for synchronization
+                           sync: all
+                     -  check:
+                           key: $(test)_PASS
+                           name: Waiter branch ran after flag was set
+                           values:
+                           - <| $(waiter_ran) == True |>
+                     -  parallel:
+                           branches:
+                           -  name: Sleep A
+                              steps:
+                              -  sleep:
+                                    name: Sleep 1s A
+                                    timeout: 1
+                           -  name: Sleep B
+                              steps:
+                              -  sleep:
+                                    name: Sleep 1s B
+                                    timeout: 1
+                           key: $(test)_PASS
+                           name: Timing test
+                           sync: all
+                     -  let:
+                           name: Capture parallel duration
+                           values:
+                           -  parallel_duration: $(ts_duration_Timing test)
+                     -  check:
+                           key: $(test)_PASS
+                           name: Duration < 1.8s (would be 2s if sequential)
+                           values:
+                           - <| float("$(parallel_duration)") < 1.8 |>
+                     -  let:
+                           name: Reset N flags
+                           values:
+                           -  n_a: false
+                           -  n_b: false
+                           -  n_c: false
+                           -  n_d: false
+                     -  parallel:
+                           branches:
+                           -  name: NA
+                              steps:
+                              -  let:
+                                    name: set n_a
+                                    values:
+                                    -  n_a: true
+                           -  name: NB
+                              steps:
+                              -  let:
+                                    name: set n_b
+                                    values:
+                                    -  n_b: true
+                           -  name: NC
+                              steps:
+                              -  let:
+                                    name: set n_c
+                                    values:
+                                    -  n_c: true
+                           -  name: ND
+                              steps:
+                              -  let:
+                                    name: set n_d
+                                    values:
+                                    -  n_d: true
+                           key: $(test)_PASS
+                           name: Four branches
+                           sync: all
+                     -  check:
+                           key: $(test)_PASS
+                           name: Four branches all set their flag
+                           values:
+                           - <| $(n_a) == True |>
+                           - <| $(n_b) == True |>
+                           - <| $(n_c) == True |>
+                           - <| $(n_d) == True |>
+                     -  let:
+                           name: Reset nested flags
+                           values:
+                           -  outer_x: false
+                           -  inner_x_1: false
+                           -  inner_x_2: false
+                     -  parallel:
+                           branches:
+                           -  name: Outer X
+                              steps:
+                              -  let:
+                                    name: set outer_x
+                                    values:
+                                    -  outer_x: true
+                              -  parallel:
+                                    branches:
+                                    -  name: Inner X1
+                                       steps:
+                                       -  let:
+                                             name: set inner_x_1
+                                             values:
+                                             -  inner_x_1: true
+                                    -  name: Inner X2
+                                       steps:
+                                       -  let:
+                                             name: set inner_x_2
+                                             values:
+                                             -  inner_x_2: true
+                                    name: Inner parallel
+                                    sync: all
+                           -  name: Outer Y
+                              steps:
+                              -  sleep:
+                                    name: brief sleep
+                                    timeout: 0
+                           key: $(test)_PASS
+                           name: Outer parallel
+                           sync: all
+                     -  check:
+                           key: $(test)_PASS
+                           name: Nested parallel set all flags
+                           values:
+                           - <| $(outer_x) == True |>
+                           - <| $(inner_x_1) == True |>
+                           - <| $(inner_x_2) == True |>
+                     -  let:
+                           name: Reset waiter timeout flag
+                           values:
+                           -  waiter_timeout_ran: false
+                     -  parallel:
+                           branches:
+                           -  name: Quick branch
+                              steps:
+                              -  sleep:
+                                    name: brief sleep
+                                    timeout: 0
+                           -  name: Doomed waiter
+                              steps:
+                              -  let:
+                                    name: should not run
+                                    values:
+                                    -  waiter_timeout_ran: true
+                              wait_for:
+                                 condition: <| "never" == "ready" |>
+                                 timeout: 1
+                           key: $(test)_PASS
+                           name: wait_for timeout
+                           no_fail: true
+                           sync: all
+                     -  check:
+                           key: $(test)_PASS
+                           name: Doomed waiter never ran its steps
+                           values:
+                           - <| $(waiter_timeout_ran) == False |>
+                     -  parallel:
+                           branches:
+                           -  name: ok branch
+                              steps:
+                              -  let:
+                                    name: noop
+                                    values:
+                                    -  noop_var: 1
+                           -  name: broken branch
+                              steps:
+                              -  py_func:
+                                    expected_result: fail
+                                    file: $(test_path)$(psep)parallel.py
+                                    func_name: sleep_func
+                                    name: Forced fail
+                                    param:
+                                    - 0
+                           key: $(test)_FAIL
+                           name: One branch really fails
+                           sync: all
+                     -  let:
+                           name: Reset branch condition flag
+                           values:
+                           -  cond_branch_ran: false
+                           -  other_branch_ran: false
+                     -  parallel:
+                           branches:
+                           -  condition: <| "always" == "false" |>
+                              name: Skipped branch
+                              steps:
+                              -  let:
+                                    name: should not run
+                                    values:
+                                    -  cond_branch_ran: true
+                           -  name: Other branch
+                              steps:
+                              -  let:
+                                    name: ran
+                                    values:
+                                    -  other_branch_ran: true
+                           key: $(test)_PASS
+                           name: Condition-skipped branch
+                           sync: all
+                     -  check:
+                           key: $(test)_PASS
+                           name: Skipped condition branch did not run
+                           values:
+                           - <| $(cond_branch_ran) == False |>
+                           - <| $(other_branch_ran) == True |>
+                     -  let:
+                           name: Reset loop counters
+                           values:
+                           -  loop_count_a: 0
+                           -  loop_count_b: 0
+                     -  loop:
+                           iterator: 3
+                           name: Loop wrapping parallel
+                           steps:
+                           -  parallel:
+                                 branches:
+                                 -  name: LA
+                                    steps:
+                                    -  let:
+                                          name: bump A
+                                          values:
+                                          -  loop_count_a: <| int("$(loop_count_a)")
+                                                + 1 |>
+                                 -  name: LB
+                                    steps:
+                                    -  let:
+                                          name: bump B
+                                          values:
+                                          -  loop_count_b: <| int("$(loop_count_b)")
+                                                + 1 |>
+                                 name: Per-iteration parallel
+                                 sync: all
+                     -  check:
+                           key: $(test)_PASS
+                           name: Both branches ran 3 times
+                           values:
+                           - <| int("$(loop_count_a)") == 3 |>
+                           - <| int("$(loop_count_b)") == 3 |>
+                     filename: /home/renish/workspace/testium/code/test/validation/items/parallel/test.tum
+               -  sequence:
+                     steps:
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.txt
+                           -  html:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.html
+                           -  junit:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.junit
+                           name: Expected PASS $(test) test
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.txt
+                           -  html:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.html
+                           -  junit:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.junit
+                           name: Expected FAIL $(test) test
+                     filename: /home/renish/workspace/testium/code/test/validation/items/report.tum
+report:
+   enabled: true
+   export:
+      junit:
+         file_name: $(validation_report_file).junit
+         path: $(validation_report_path)
+      sqlite:
+         file_name: $(validation_report_file).sqlite
+         path: $(validation_report_path)
+   log_stored: true

schema/test_schema/plot.tum

@@ -0,0 +1,137 @@
+config_file:
+- param.yaml
+- items/plot/param.yaml
+main:
+   name: Testium validation suite
+   steps:
+   -  group:
+         name: Test preparation
+         steps:
+         -  let:
+               condition: <| "$(os)" == "Linux" |>
+               name: Set test variables for Linux
+               values:
+               -  terminal_prompt: $(linux_prompt)
+               -  psep: /
+         -  let:
+               condition: <| "$(os)" == "Windows" |>
+               name: Set test variables for Windows
+               values:
+               -  terminal_prompt: $(windows_prompt)
+               -  psep: \
+   -  group:
+         name: Group of tests
+         steps:
+         -  let:
+               name: plot test constants
+               values:
+                  test: plot
+                  test_path: items/$(test)
+         -  group:
+               name: plot test
+               steps:
+               -  sequence:
+                     steps:
+                     -  group:
+                           condition: <| $(validation_dialogs) and not tm.text_mode()
+                              |>
+                           name: Plot test
+                           steps:
+                           -  plot:
+                                 key: $(test)_PASS
+                                 name: Open the plot
+                                 plot_name: Mon Plot
+                                 steps:
+                                 -  open:
+                                       log_path: $(validation_report_path)
+                           -  plot:
+                                 key: $(test)_PASS
+                                 name: Add periodic to the plot
+                                 plot_name: Mon Plot
+                                 steps:
+                                 -  periodic:
+                                       eval: '{"periodic": $(result)}'
+                                       file: $(test_path)$(psep)plot.py
+                                       func_name: random_value
+                                       period: 1
+                           -  sleep:
+                                 dialog: true
+                                 name: sleep
+                                 timeout: 3
+                           -  loop:
+                                 iterator: 10
+                                 name: Add of other data in the plot
+                                 steps:
+                                 -  plot:
+                                       key: $(test)_PASS
+                                       name: Add to the plot
+                                       plot_name: Mon Plot
+                                       steps:
+                                       -  add:
+                                             value1: $(loop_index)
+                                             value2: $(loop_index)+2
+                                 -  sleep:
+                                       name: sleep between values
+                                       timeout: 1
+                                 -  py_func:
+                                       file: $(test_path)$(psep)plot.py
+                                       func_name: LastValues
+                                       key: $(test)_PASS
+                                       name: last plot values
+                                       param:
+                                       - Mon Plot
+                           -  plot:
+                                 execute_on_stop: true
+                                 key: $(test)_PASS
+                                 name: Export
+                                 plot_name: Mon Plot
+                                 steps:
+                                 -  export: $(validation_report_path)/plot_export.pdf
+                                 -  export: $(validation_report_path)/plot_export.csv
+                           -  plot:
+                                 execute_on_stop: true
+                                 key: $(test)_PASS
+                                 name: Close the plot
+                                 plot_name: Mon Plot
+                                 steps:
+                                 -  close:
+                                       timeout: 2
+                                       wait_dialog_exit: true
+                     filename: /home/renish/workspace/testium/code/test/validation/items/plot/test.tum
+               -  sequence:
+                     steps:
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.txt
+                           -  html:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.html
+                           -  junit:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.junit
+                           name: Expected PASS $(test) test
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.txt
+                           -  html:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.html
+                           -  junit:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.junit
+                           name: Expected FAIL $(test) test
+                     filename: /home/renish/workspace/testium/code/test/validation/items/report.tum
+report:
+   enabled: true
+   export:
+      junit:
+         file_name: $(validation_report_file).junit
+         path: $(validation_report_path)
+      sqlite:
+         file_name: $(validation_report_file).sqlite
+         path: $(validation_report_path)
+   log_stored: true

schema/test_schema/py_func.tum

@@ -0,0 +1,326 @@
+config_file:
+- param.yaml
+- items/py_func/param.yaml
+main:
+   name: Testium validation suite
+   steps:
+   -  group:
+         name: Test preparation
+         steps:
+         -  let:
+               condition: <| "$(os)" == "Linux" |>
+               name: Set test variables for Linux
+               values:
+               -  terminal_prompt: $(linux_prompt)
+               -  psep: /
+         -  let:
+               condition: <| "$(os)" == "Windows" |>
+               name: Set test variables for Windows
+               values:
+               -  terminal_prompt: $(windows_prompt)
+               -  psep: \
+   -  group:
+         name: Group of tests
+         steps:
+         -  let:
+               name: py_func test constants
+               values:
+                  test: py_func
+                  test_path: items/$(test)
+         -  group:
+               name: py_func test
+               steps:
+               -  sequence:
+                     steps:
+                     -  let:
+                           name: py_func test constants,
+                           values:
+                              py_func test parameter: test parameter
+                     -  py_func:
+                           file: $(test_path)$(psep)py_func.py
+                           func_name: assertparam
+                           key: $(test)_PASS
+                           name: pass py_func
+                           param:
+                           - true
+                     -  py_func:
+                           file: $(test_path)$(psep)py_func.py
+                           func_name: assertparam
+                           key: $(test)_FAIL
+                           name: fail py_func
+                           param:
+                           - false
+                     -  py_func:
+                           expected_result: FAIL
+                           file: $(test_path)$(psep)py_func.py
+                           func_name: assertparam
+                           key: $(test)_PASS
+                           name: fail py_func with expected result "FAIL"
+                           param:
+                           - false
+                     -  py_func:
+                           expected_result: FAIL
+                           file: $(test_path)$(psep)py_func.py
+                           func_name: assertparam
+                           key: $(test)_FAIL
+                           name: pass py_func with expected result FAIL
+                           param:
+                           - true
+                     -  py_func:
+                           expected_result: -1
+                           file: $(test_path)$(psep)py_func.py
+                           func_name: echo
+                           key: $(test)_PASS
+                           name: expected -1
+                           param:
+                           - -1
+                     -  py_func:
+                           expected_result: 354848436 - 354848437
+                           file: $(test_path)$(psep)py_func.py
+                           func_name: echo
+                           key: $(test)_PASS
+                           name: expected eval
+                           param:
+                           - -1
+                     -  py_func:
+                           expected_result: '[-1, ''a'', {''toto'': ''tata''}]'
+                           file: $(test_path)$(psep)py_func.py
+                           func_name: echo
+                           key: $(test)_PASS
+                           name: expected table
+                           param:
+                           -  - -1
+                              - a
+                              -  toto: tata
+                     -  py_func:
+                           file: $(test_path)$(psep)py_func.py
+                           func_name: checkglobal
+                           key: $(test)_PASS
+                           name: global param py_func
+                           param:
+                           - $(py_func test parameter)
+                     -  let:
+                           key: $(test)_PASS
+                           name: python2func
+                           values:
+                           -  py: $(test_path)$(psep)py_func.py
+                     -  py_func:
+                           expected_result: $(py_func test parameter)
+                           file: $(py)
+                           func_name: checkglobal2
+                           key: $(test)_PASS
+                           name: global param py_func 2
+                     -  py_func:
+                           file: $(py)
+                           func_name: checkglobal
+                           key: $(test)_PASS
+                           name: global param py_func
+                           param:
+                           - $(py_func test parameter)
+                     -  py_func:
+                           file: $(test_path)$(psep)py_func.py
+                           func_name: should_not_be_called
+                           name: skipped_checkglobal
+                           param:
+                           - $(py_func test parameter)
+                     -  py_func:
+                           file: $(test_path)$(psep)py_func.py
+                           func_name: checkglobal
+                           name: skipped true
+                           param:
+                           - $(py_func test parameter)
+                           skipped: true
+                     -  py_func:
+                           file: $(test_path)$(psep)py_func.py
+                           func_name: checkglobal
+                           name: skipped 1
+                           param:
+                           - $(py_func test parameter)
+                           skipped: 1
+                     -  py_func:
+                           file: $(test_path)$(psep)py_func.py
+                           func_name: ValidationTest
+                           name: FunctionItem test
+                           param:
+                           - $(py_func test parameter)
+                     -  group:
+                           name: Function results check
+                           steps:
+                           -  group:
+                                 name: Function result 1
+                                 steps:
+                                 -  py_func:
+                                       expected_result: -1
+                                       file: $(test_path)$(psep)py_func.py
+                                       func_name: echo
+                                       key: $(test)_PASS
+                                       name: int failure
+                                       param:
+                                       - -1
+                                 -  py_func:
+                                       expected_result: -1.3
+                                       file: $(test_path)$(psep)py_func.py
+                                       func_name: echo
+                                       key: $(test)_PASS
+                                       name: float failure
+                                       param:
+                                       - -1.3
+                                 -  py_func:
+                                       expected_result: FAIL
+                                       file: $(test_path)$(psep)py_func.py
+                                       func_name: echo
+                                       key: $(test)_PASS
+                                       name: String failure
+                                       param:
+                                       - FAIL
+                                 -  py_func:
+                                       expected_result:
+                                       - -1
+                                       - Got a failure
+                                       file: $(test_path)$(psep)py_func.py
+                                       func_name: tuple_return
+                                       key: $(test)_PASS
+                                       name: Tuple int,str failure
+                                       param:
+                                       - -1
+                                       - Got a failure
+                           -  group:
+                                 name: Functions result 2
+                                 steps:
+                                 -  py_func:
+                                       expected_result: 0
+                                       file: $(test_path)$(psep)py_func.py
+                                       func_name: echo
+                                       key: $(test)_PASS
+                                       name: int success
+                                       param:
+                                       - 0
+                                 -  py_func:
+                                       expected_result: 0.3
+                                       file: $(test_path)$(psep)py_func.py
+                                       func_name: echo
+                                       key: $(test)_PASS
+                                       name: float success
+                                       param:
+                                       - 0.3
+                                 -  py_func:
+                                       expected_result: Something that is not only
+                                          strictly FAIL
+                                       file: $(test_path)$(psep)py_func.py
+                                       func_name: echo
+                                       key: $(test)_PASS
+                                       name: String success
+                                       param:
+                                       - Something that is not only strictly FAIL
+                                 -  py_func:
+                                       expected_result:
+                                       - 0
+                                       - OK
+                                       file: $(test_path)$(psep)py_func.py
+                                       func_name: tuple_return
+                                       key: $(test)_PASS
+                                       name: Tuple int,str success
+                                       param:
+                                       - 0
+                                       - OK
+                     -  py_func:
+                           file: $(test_path)$(psep)py_func.py
+                           func_name: test_delgd
+                           key: $(test)_PASS
+                           name: delgd test
+                     -  py_func:
+                           file: $(test_path)$(psep)py_func.py
+                           func_name: return_nothing
+                           key: $(test)_PASS
+                           name: function returning nothing should succeed
+                     -  py_func:
+                           file: $(test_path)$(psep)py_func.py
+                           func_name: return_explicit_none
+                           key: $(test)_PASS
+                           name: function returning explicit None should succeed
+                     -  group:
+                           name: context_id tests
+                           steps:
+                           -  py_func:
+                                 expected_result: hello context
+                                 file: $(test_path)$(psep)py_func.py
+                                 func_name: set_context_value
+                                 key: $(test)_PASS
+                                 name: set serializable value
+                                 param:
+                                 - hello context
+                           -  py_func:
+                                 context_id: ctx_test
+                                 expected_result: hello context
+                                 file: $(test_path)$(psep)py_func.py
+                                 func_name: get_context_value
+                                 key: $(test)_PASS
+                                 name: get serializable value (same context_id)
+                           -  py_func:
+                                 expected_result: hello context
+                                 file: $(test_path)$(psep)py_func.py
+                                 func_name: get_context_value
+                                 key: $(test)_PASS
+                                 name: get serializable value (no context_id, from
+                                    main gd)
+                           -  py_func:
+                                 context_id: ctx_other
+                                 expected_result: hello context
+                                 file: $(test_path)$(psep)py_func.py
+                                 func_name: get_context_value
+                                 key: $(test)_PASS
+                                 name: get serializable value (different context_id)
+                           -  py_func:
+                                 context_id: ctx_ns_test
+                                 expected_result: hello ns
+                                 file: $(test_path)$(psep)py_func.py
+                                 func_name: set_ns_value
+                                 key: $(test)_PASS
+                                 name: set non-serializable value
+                                 param:
+                                 - hello ns
+                           -  py_func:
+                                 context_id: ctx_ns_test
+                                 expected_result: hello ns
+                                 file: $(test_path)$(psep)py_func.py
+                                 func_name: get_ns_value
+                                 key: $(test)_PASS
+                                 name: get non-serializable value (same context_id)
+                     filename: /home/renish/workspace/testium/code/test/validation/items/py_func/test.tum
+               -  sequence:
+                     steps:
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.txt
+                           -  html:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.html
+                           -  junit:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.junit
+                           name: Expected PASS $(test) test
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.txt
+                           -  html:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.html
+                           -  junit:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.junit
+                           name: Expected FAIL $(test) test
+                     filename: /home/renish/workspace/testium/code/test/validation/items/report.tum
+report:
+   enabled: true
+   export:
+      junit:
+         file_name: $(validation_report_file).junit
+         path: $(validation_report_path)
+      sqlite:
+         file_name: $(validation_report_file).sqlite
+         path: $(validation_report_path)
+   log_stored: true

schema/test_schema/sleep.tum

@@ -0,0 +1,87 @@
+config_file:
+- param.yaml
+- items/sleep/param.yaml
+main:
+   name: Testium validation suite
+   steps:
+   -  group:
+         name: Test preparation
+         steps:
+         -  let:
+               condition: <| "$(os)" == "Linux" |>
+               name: Set test variables for Linux
+               values:
+               -  terminal_prompt: $(linux_prompt)
+               -  psep: /
+         -  let:
+               condition: <| "$(os)" == "Windows" |>
+               name: Set test variables for Windows
+               values:
+               -  terminal_prompt: $(windows_prompt)
+               -  psep: \
+   -  group:
+         name: Group of tests
+         steps:
+         -  let:
+               name: sleep test constants
+               values:
+                  test: sleep
+                  test_path: items/$(test)
+         -  group:
+               name: sleep test
+               steps:
+               -  sequence:
+                     steps:
+                     -  sleep:
+                           condition: $(validation_dialogs)
+                           dialog: true
+                           key: $(test)_PASS
+                           name: Sleep timeout with dialogs
+                           timeout: 3
+                     -  sleep:
+                           key: $(test)_PASS
+                           name: Sleep timeout without dialog
+                           timeout: 3.0
+                     -  sleep:
+                           key: $(test)_PASS
+                           name: Sleep timeout in textual format
+                           skipped: true
+                           timeout: 1h 3m 2s
+                     filename: /home/renish/workspace/testium/code/test/validation/items/sleep/test.tum
+               -  sequence:
+                     steps:
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.txt
+                           -  html:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.html
+                           -  junit:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.junit
+                           name: Expected PASS $(test) test
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.txt
+                           -  html:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.html
+                           -  junit:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.junit
+                           name: Expected FAIL $(test) test
+                     filename: /home/renish/workspace/testium/code/test/validation/items/report.tum
+report:
+   enabled: true
+   export:
+      junit:
+         file_name: $(validation_report_file).junit
+         path: $(validation_report_path)
+      sqlite:
+         file_name: $(validation_report_file).sqlite
+         path: $(validation_report_path)
+   log_stored: true

schema/test_schema/unittest.tum

@@ -0,0 +1,84 @@
+config_file:
+- param.yaml
+- items/unittest/param.yaml
+main:
+   name: Testium validation suite
+   steps:
+   -  group:
+         name: Test preparation
+         steps:
+         -  let:
+               condition: <| "$(os)" == "Linux" |>
+               name: Set test variables for Linux
+               values:
+               -  terminal_prompt: $(linux_prompt)
+               -  psep: /
+         -  let:
+               condition: <| "$(os)" == "Windows" |>
+               name: Set test variables for Windows
+               values:
+               -  terminal_prompt: $(windows_prompt)
+               -  psep: \
+   -  group:
+         name: Group of tests
+         steps:
+         -  let:
+               name: unittest test constants
+               values:
+                  test: unittest
+                  test_path: items/$(test)
+         -  group:
+               name: unittest test
+               steps:
+               -  sequence:
+                     steps:
+                     -  unittest:
+                           key: $(test)_PASS
+                           name: Unittest item
+                           test_file: /home/renish/workspace/testium/code/test/validation/items/unittest/unittest.py
+                           test_method: test_01_pass
+                     -  unittest:
+                           key: $(test)_FAIL
+                           name: Unittest item
+                           test_file: /home/renish/workspace/testium/code/test/validation/items/unittest/unittest.py
+                           test_method:
+                           - test_04_disabled
+                           - test_03_fail
+                     filename: /home/renish/workspace/testium/code/test/validation/items/unittest/test.tum
+               -  sequence:
+                     steps:
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.txt
+                           -  html:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.html
+                           -  junit:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.junit
+                           name: Expected PASS $(test) test
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.txt
+                           -  html:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.html
+                           -  junit:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.junit
+                           name: Expected FAIL $(test) test
+                     filename: /home/renish/workspace/testium/code/test/validation/items/report.tum
+report:
+   enabled: true
+   export:
+      junit:
+         file_name: $(validation_report_file).junit
+         path: $(validation_report_path)
+      sqlite:
+         file_name: $(validation_report_file).sqlite
+         path: $(validation_report_path)
+   log_stored: true

schema/tum.json

@@ -0,0 +1,532 @@
+{
+    "$schema": "https://json-schema.org/draft/2020-12/schema",
+
+    "$defs": {
+        "config_file":{
+            "desciption": "The list of the configuration files",
+            "type": "array",
+            "items": {
+                "type": "string"
+            }
+        },
+
+        "items":{
+            "properties": {
+                "name": { "type": "string" },
+                "stop_on_failure":{ "type": "boolean" },
+                "execute_on_stop":{ "type": "boolean" },
+                "skipped":{ "type": "boolean" },
+                "no_fail":{ "type": "boolean" },
+                "doc":{ "type": "string" },
+                "key":{ "type": "string" },
+                "report":{ "$ref": "#/$defs/report_export" },
+                "condition":{ "type": "string" },
+                "process_result":{ "type": "string" },
+                "expected_result":{ },
+                "store_result":{ "type": "string" }
+            }
+        },
+
+        "steps" : {
+            "required": ["steps"],
+            "properties": {
+                "version": { "type": "string" },
+                "steps": {
+                    "type": "array",
+                    "items": {
+                        "type": "object",
+                        "additionalProperties": false,
+                        "properties":{
+                            "let": { "$ref": "#/$defs/let" },
+                            "check": { "$ref": "#/$defs/check" },
+                            "dialog_question": { "$ref": "#/$defs/dialog" },
+                            "dialog_message": { "$ref": "#/$defs/dialog" },
+                            "dialog_note": { "$ref": "#/$defs/dialog" },
+                            "dialog_value": { "$ref": "#/$defs/dialog" },
+                            "dialog_image": { "$ref": "#/$defs/dialog" },
+                            "lua_func": { "$ref": "#/$defs/func" },
+                            "py_func": { "$ref": "#/$defs/func" },
+                            "console": { "$ref": "#/$defs/console" },
+                            "sleep": { "$ref": "#/$defs/sleep" },
+                            "json_rpc": { "$ref": "#/$defs/json_rpc" },
+                            "group": { "$ref": "#/$defs/group" },
+                            "sequence": { "$ref": "#/$defs/sequence" },
+                            "report": { "$ref": "#/$defs/report" },
+                            "loop": { "$ref": "#/$defs/loop" },
+                            "git": { "$ref": "#/$defs/git" },
+                            "unittest": { "$ref": "#/$defs/unittest" }
+
+                        }
+                    }
+                }
+            }
+        },
+
+        "cons_open" : {
+            "type":"object",
+            "allOf": 
+            [
+                { "$ref": "#/$defs/items" },
+                { 
+                    "properties": {
+                        "protocol" : {
+                            "enum": ["telnet", "ssh", "serial", "rawtcp", "terminal"]
+                        },
+                        "telnet_host" : {"type": "string" },
+                        "telnet_port" : {"type": "number" },
+                        "ssh_host": {"type": "string" },
+                        "ssh_user": {"type": "string" },
+                        "ssh_pwd": {"type": "string" },
+                        "serial_port": {"type": "string" },
+                        "serial_baudrate": {"type": "integer" },
+                        "buffered": {"type": "boolean" },
+                        "tcp_host" : {"type": "string" },
+                        "tcp_port" : {"type": "number" },
+                        "terminal_path": {"type":"string" },
+                        "shell": {"type":"string" }
+                    },
+                    "required": ["protocol"]
+                }
+            ]   
+        },
+
+        "cons_read": {
+            "type":"object",
+            "unevaluatedProperties": false,
+            "allOf": [
+                { "$ref": "#/$defs/items" },
+                {
+                    "properties": {
+                        "protocol" : {
+                            "enum": ["telnet", "ssh", "serial", "rawtcp", "terminal"]
+                        },
+                        "expected" : {"type": "string" },
+                        "timeout": {"type": "number" },
+                        "mute": {"type": "boolean" }
+                    },
+                    "required":["expected"]
+                }
+            ]
+        },
+
+        "console" : {
+            "description": "The let items",
+            "type": "object",
+            "unevaluatedProperties": false,
+            "allOf": [
+                { "$ref": "#/$defs/items" },
+                {
+                    "required": ["steps", "console_name"],
+                    "properties":{
+                        "console_name": {"type":"string" },
+                        "steps": {
+                            "type": "array",
+                            "items": {
+                                "type": "object",
+                                "properties":{
+                                    "open": {
+                                        "$ref": "#/$defs/cons_open"
+                                    },
+                                    "write": { "type": "string" },
+                                    "writeln": { "type": "string" },
+                                    "read_until": {
+                                        "$ref": "#/$defs/cons_read"
+                                    },
+                                    "close": { 
+                                        "anyOf": [
+                                            {"type": "null"},
+                                            {"type": "string"}
+                                        ]
+                                    }
+                                },
+                                "additionalProperties": false
+                            }
+                        }
+                    }
+                }
+            ]
+        },
+
+        "json_rpc_console": {
+            "type":"object",
+            "properties": {
+                "name" : {"type": "string" },
+                "prompt" : {"type": "string" }
+            },
+            "required":["name"],
+            "additionalProperties": false
+        },
+
+        "json_rpc_udp": {
+            "type":"object",
+            "properties": {
+                "server" : {"type": "string" },
+                "udp_snd_port" : {"type": "integer" },
+                "udp_rcv_port" : {"type": "integer" },
+                "bufsize" : {"type": "integer" }
+            },
+            "required":["name"],
+            "additionalProperties": false
+        },
+        
+        "json_rpc_query": {
+            "type":"object",
+            "properties": {
+                "method" : { "type": "string" },
+                "params" : { "type": "array" },
+                "id" : {"type": "integer" },
+                "no_wait" : {"type": "boolean" }
+            },
+            "required":["method"],
+            "additionalProperties": false
+        },
+
+        "json_rpc_receive": {
+            "type":"object",
+            "properties": {
+                "id" : {"type": "integer" },
+                "timeout" : {"type": "number" }
+            },
+            "required":["id"],
+            "additionalProperties": false
+        },
+
+        "json_rpc" : {
+            "description": "The json_rpc items",
+            "type": "object",
+            "unevaluatedProperties": false,
+            "allOf": [
+                { "$ref": "#/$defs/items" },
+                {
+                    "required": ["steps"],
+                    "properties":{
+                    
+                        "udp" : { "$ref": "#/$defs/json_rpc_udp" },
+                        "console" : { "$ref": "#/$defs/json_rpc_console" },
+                        "timeout": {"type": "number" },
+                        "version": {"enum": ["1.0", "2.0"]},
+                        "steps": {
+                            "type": "array",
+                            "items": {
+                                "type": "object",
+                                "additionalProperties": false,
+                                "properties":{
+                                    "open": { "type": "null" },
+                                    "query": { "$ref": "#/$defs/json_rpc_query" },
+                                    "receive": { "$ref": "#/$defs/json_rpc_receive" },
+                                    "writeln": { "type": "string" },
+                                    "read_until": { "$ref": "#/$defs/cons_read"
+                                    },
+                                    "close": { "type": "null" }
+                                }
+                            }
+                        }
+                    }
+                }
+            ]
+        },
+
+        "group" : {
+            "description": "The group items",
+            "type": "object",
+            "unevaluatedProperties": false,
+            "allOf": [
+                { "$ref": "#/$defs/items" },
+                { "$ref": "#/$defs/steps" }
+            ]
+        },
+
+        "sequence" : {
+            "description": "The sequence items",
+            "type": "object",
+            "unevaluatedProperties": false,
+            "allOf": [
+                { "$ref": "#/$defs/items" },
+                { "$ref": "#/$defs/steps" },
+                {
+                    "properties": {
+                        "filename": { "type": "string" }
+                    },
+                    "required": ["filename"]
+                }
+            ]
+        },
+
+        "loop" : {
+            "description": "The loop items",
+            "type": "object",
+            "unevaluatedProperties": false,
+            "allOf": [
+                { "$ref": "#/$defs/items" },
+                { "$ref": "#/$defs/steps" },
+                {
+                    "properties": {
+                        "iterator": { },
+                        "exit_condition": {
+                            "description": "the posibility to stop the loop",
+                            "type": "object",
+                            "additionalProperties": false,
+                            "properties": {
+                                "time" : {"type":"number"},
+                                "value" : {"type":"number"},
+                                "file" : {"type":"string"},
+                                "func_name" : {"type":"string"},
+                                "eval" : {"type":"string"}
+                            },
+                            "dependentRequired": {
+                                "file" : ["func_name"]
+                            }
+
+                        }
+                    }
+                }
+            ]
+        },
+
+        "main" : {
+            "description": "The Main items",
+            "type": "object",
+            "unevaluatedProperties": false,
+            "allOf": [
+                { "$ref": "#/$defs/items" },
+                { "$ref": "#/$defs/steps" },
+                {
+                    "properties": {
+                        "version": { "type": "string" }
+                    }
+                }
+            ]
+        },
+
+        "sleep" : {
+            "description": "Sleep for X time [secondes",
+            "type": "object",
+            "unevaluatedProperties": false,
+            "allOf": [
+                { "$ref": "#/$defs/items" },
+                {
+                    "properties":{
+                        "dialog": { "type": "boolean" },
+                        "timeout": { "type": "number" }
+                    },
+                    "required": ["timeout"]
+                }
+            ]
+        },
+
+        "let" : {
+            "description": "The let items",
+            "type": "object",
+            "unevaluatedProperties": false,
+            "allOf": [
+                { "$ref": "#/$defs/items" },
+                {
+                    "properties":{
+                        "values": {
+                            "anyOf": [
+                                {
+                                    "type": "array",
+                                    "items": {"type": "object" }
+                                },
+                                { "type":"object" }
+                            ]
+                        }
+                    },
+                    "required": ["values"]
+                }
+            ]
+        },
+
+        "check" : {
+            "description": "The let items",
+            "type": "object",
+            "unevaluatedProperties": false,
+            "allOf": [
+                { "$ref": "#/$defs/items" },
+                {
+                    "properties":{
+                        "values": { 
+                            "type": "array",
+                            "items": {"type": "string" }
+                        }
+                    },
+                    "required": ["values"]
+                }
+            ]
+        },
+
+        "git" : {
+            "description": "The git items",
+            "type": "object",
+            "unevaluatedProperties": false,
+            "allOf": [
+                { "$ref": "#/$defs/items" },
+                {
+                    "properties":{
+                        "repo": { "type": "string" }
+                    },
+                    "required": ["repo"]
+                }
+            ]
+        },
+
+        "unittest" : {
+            "description": "The unittest items",
+            "type": "object",
+            "unevaluatedProperties": false,
+            "allOf": [
+                { "$ref": "#/$defs/items" },
+                {
+                    "properties":{
+                        "test_file": { "type": "string" },
+                        "test_method": {
+                            "anyOf": [
+                                {"type":"string"},
+                                {
+                                    "type":"array",
+                                    "items": {"type":"string"}
+                                }
+                            ]
+                        }
+                    },
+                    "required": ["test_file"]
+                }
+            ]
+        },
+
+        "dialog" : {
+            "description": "The let items",
+            "type": "object",
+            "unevaluatedProperties": false,
+            "allOf": [
+                { "$ref": "#/$defs/items" },
+                {
+                    "properties":{
+                        "question": { "type": "string" },
+                        "auto_result": { "type": "string" },
+                        "auto_value": { "type": ["number", "string"] },
+                        "filename" : {"type": "string"},
+                        "reference" : {
+                            "type":"array",
+                            "items": { "type":"string"}
+                        }
+                    },
+                    "required": ["question"]
+                }
+            ]
+        },
+
+        "dialog_choice" : {
+            "description": "The one choi items",
+            "type": "object",
+            "additionalProperties": false,
+            "properties":{
+                "name": { "type": "string" },
+                "description": { "type": "string" },
+                "icon": { "type": "string"},
+                "choices": {
+                    "type":"array",
+                    "items": { "$ref": "#/$defs/dialog_choice" }
+                }
+            },
+            "required": ["name", "description"]
+        },
+        "dialog_choices" : {
+            "description": "The dialog choices items",
+            "type": "object",
+            "unevaluatedProperties": false,
+            "allOf": [
+                { "$ref": "#/$defs/items" },
+                {
+                    "properties":{
+                        "name": { "type": "string" },
+                        "question": { "type": "string" },
+                        "icon": { "type": "string"},
+                        "choices": {
+                            "type":"array",
+                            "items": { "$ref": "#/$defs/dialog_choice" }
+                        }
+                    },
+                    "required": ["question"]
+                }
+            ]
+        },
+        "func" : {
+            "description": "The py_fun and lua_func items",
+            "type": "object",
+            "unevaluatedProperties": false,
+            "allOf": [
+                { "$ref": "#/$defs/items" },
+                {
+                    "properties":{
+                        "file": { "type": "string" },
+                        "func_name": { "type": "string" },
+                        "context_id": { "type": "string" },
+                        "param" : {
+                            "anyOf": [
+                                {"type": "array"},
+                                {"type": "object"}
+                            ]
+                        }
+                    },
+                    "required": ["file", "func_name"]
+                }
+            ]
+        },
+
+        "report_export": {
+            "type": "object",
+            "properties": {
+                "path": {"type":"string" },
+                "file_name": {"type":"string" },
+                "pattern": {"type":"string" },
+                "key": {"type":"string" }
+            }
+        },
+
+        "report":{
+            "desciption": "The list of the configuration files",
+            "type": "object",
+            "additional_properties":false,
+            "properties": {
+                "enabled": {"type":"boolean" },
+                "log_stored": {"type":"boolean" },
+                "export": {
+                    "anyOf": [
+                        {
+                            "type": "object",
+                            "properties":{
+                                "html": { "$ref": "#/$defs/report_export" },
+                                "sqlite": { "$ref": "#/$defs/report_export" },
+                                "junit": { "$ref": "#/$defs/report_export" }
+                            }
+                        },
+                        {
+                            "type": "array",
+                            "items": {
+                                "type":"object",
+                                "properties":{
+                                    "html": { "$ref": "#/$defs/report_export" },
+                                    "sqlite": { "$ref": "#/$defs/report_export" },
+                                    "junit": { "$ref": "#/$defs/report_export" }
+                                }
+                            }
+                        }
+                    ]
+                }
+            }
+        }
+
+    },
+    
+    "type": "object",
+    "properties": {
+        "config_file" : { "$ref": "#/$defs/config_file"  },
+        "main": {  "$ref": "#/$defs/main"  },
+        "report" : { "$ref": "#/$defs/report"  }
+
+    },
+    "required" : ["main"],
+    "additionalProperties": false
+
+}

scripts/build_env.sh

@@ -33,11 +33,6 @@ if [ ! -d "$PY_VENV_DIR" ]; then
     python3 -m venv "$PY_VENV_DIR"
     source "$PY_VENV_DIR/bin/activate"
     pip install --extra-index-url https://pypi.python.org/pypi -r $REQ_PATH
-    # Language-server deps (the pyproject [lsp] extra). Installed here so the
-    # source run AND the PyInstaller build — both of which use this venv — can
-    # start / collect the `testium lsp` server. pip-installed wheel users get
-    # them via `pip install testium[lsp]` instead.
-    pip install --extra-index-url https://pypi.python.org/pypi "pygls>=1.3"
     # Validation suite plugin used to verify the report-exporter
     # entry-points discovery end-to-end.
     FAKE_EXPORTER_DIR="$(dirname "$REQ_PATH")/../test/validation/fake_exporter"

src/VERSION

@@ -1 +1 @@
-0.2
+0.1.2

src/pyproject.toml

@@ -30,12 +30,6 @@ dependencies = [
 ]
 dynamic = ["version"]
 
-[project.optional-dependencies]
-# `pip install testium[lsp]` adds the language-server dependencies. The
-# stdio-only LSP server (`testium lsp`) reuses the schema export from the
-# core install; pygls is the only marginal cost.
-lsp = ["pygls>=1.3"]
-
 [project.scripts]
 testium = "testium:main"
 

src/testium/__init__.py

@@ -11,30 +11,6 @@ sys.path.append(os.path.abspath(ourpath.parent))
 import interpreter.utils.constants as cst
 
 def main():
-    # Subcommand dispatch (must run *before* argparse so neither 'schema' nor
-    # 'lsp' has to share the GUI/batch flag surface). The subcommands also
-    # skip the multiprocessing 'spawn' setup which is only meaningful for the
-    # main runtime — schema is a pure stdout dump and lsp speaks JSON-RPC
-    # over stdio without ever forking a test process.
-    if len(sys.argv) >= 2 and sys.argv[1] in ("schema", "lsp"):
-        sub = sys.argv[1]
-        if sub == "schema":
-            from lsp.schema import dump_all_schemas_json
-            print(dump_all_schemas_json())
-            return
-        # lsp
-        try:
-            from lsp.server import serve
-        except ImportError as e:
-            print(
-                f"testium lsp: language server dependencies missing ({e.name}). "
-                "Install with: pip install 'testium[lsp]'",
-                file=sys.stderr,
-            )
-            sys.exit(2)
-        serve()
-        return
-
     # This line sets the method for the "Process" function. It is required for Linux
     # support of the test dialogs.
     multiprocessing.set_start_method('spawn')

src/testium/interpreter/process.py

@@ -16,7 +16,6 @@ from interpreter.utils.test_init import (
     env_init,
     prepare_global,
     update_global,
-    apply_overrides,
     set_standard_gd_keys,
     test_run_init,
     test_run_header,
@@ -211,19 +210,6 @@ class TestProcess(Process):
 
                 env_init()
 
-                # Apply GUI defaults and CLI defines to the global dict
-                # *before* eval_proc starts: bins.python_bin() reads
-                # ``python_bin`` from gd on its very first call (during
-                # eval_process_init) and caches the result. Without this,
-                # ``-d python_bin=...`` and the GUI ``python_bin`` preference
-                # would only take effect for items spawned *after* the cache
-                # was already populated with the auto-discovered interpreter,
-                # i.e. they would silently be ignored for eval_proc itself.
-                # _load_initial_params re-applies the same overrides after
-                # ``prepare_global()`` clears gd, so the gd value stays in
-                # sync with the cached path.
-                apply_overrides(self.__defs, self.__gui_defaults)
-
                 # Creation of the python evaluation process for loading of the complete test
                 eval_proc = eval_process_init(api_request, 10, test_dir)
                 eval_proc.start()

src/testium/interpreter/test_items/item_actions/__init__.py

@@ -5,13 +5,6 @@ from interpreter.test_items.item_actions.action import TestItemAction
 
 
 class TestItemActions(TestItem):
-    # Declarative action registry: subclasses set ``ACTIONS = {yaml_key: class}``
-    # as a class attribute (mirroring ``PARAMS``). It is read here to populate
-    # the runtime registry, and read identically by the schema export — no
-    # instantiation or source inspection required. ``register_actions()`` stays
-    # available as an imperative escape hatch for dynamic/conditional cases.
-    ACTIONS = {}
-
     def __init__(
         self, item_type, dict_actions, parent=None, status_queue=None, filename=""
     ):
@@ -19,7 +12,7 @@ class TestItemActions(TestItem):
         super().__init__(dict_actions, parent, status_queue, filename=filename)
         self._type = item_type
         self.is_container = False
-        self.action_classes = dict(type(self).ACTIONS)
+        self.action_classes = {}
         self.actions_token = None
         self.actions = []
         try:
@@ -31,9 +24,6 @@ class TestItemActions(TestItem):
             )
 
     def register_actions(self, **args: TestItemAction):
-        # Imperative escape hatch. The declarative ``ACTIONS`` class attribute
-        # covers every current subclass; use this only to add actions that
-        # can't be known at class-definition time (e.g. platform-conditional).
         for action_name, action_class in args.items():
             self.action_classes.update({action_name: action_class})
 

src/testium/interpreter/test_items/test_item.py

@@ -5,9 +5,6 @@ from copy import deepcopy
 from interpreter.test_items.test_result import TestResult, TestValue
 import api.testium as tm
 from interpreter.utils.params import TestItemParams
-from interpreter.utils.param_decl import (
-    Param, ParamSet, LIST, BLOCK, unknown_keys, missing_required,
-)
 from interpreter.utils.constants import TestItemType as cst_type
 from interpreter.utils.eval import eval_to_boolean, evaluate, post_evaluate
 from runtime.tum_except import ETUMSyntaxError, item_load_context
@@ -16,32 +13,6 @@ LOG_TEST_STOP = '<----- step "{}" finished'
 LOG_TEST_START = '-----> step "{}" started'
 
 
-# Parameters accepted by every test item, regardless of its type. Subclasses
-# concatenate their own ``PARAMS`` to this set; the merged result drives
-# unknown-param warnings and (later) the LSP schema export.
-COMMON_PARAMS = ParamSet(
-    Param("name",            doc="Display name shown in the GUI tree and reports."),
-    Param("doc",             doc="Free-form documentation; surfaced in tooltips."),
-    Param("skipped",         doc="If truthy, the step is skipped (static expression, "
-                                 "evaluated at load time)."),
-    Param("key",             doc="Report key used to classify the result "
-                                 "(typically <test>_PASS or <test>_FAIL)."),
-    Param("stop_on_failure", doc="If true, abort the surrounding container on failure."),
-    Param("execute_on_stop", doc="If true, run this step even when its container "
-                                 "is being stopped (cleanup)."),
-    Param("process_result",  doc="Post-evaluation expression applied to the test result."),
-    Param("store_result",    doc="Global-dict key in which to store the test result."),
-    Param("expected_result", doc="Expected outcome; the step is failed if it doesn't match."),
-    Param("no_fail",         doc="If truthy, never report a FAILURE for this step."),
-    Param("report",          doc="Per-step reporting override."),
-    Param("condition",       doc="Optional gating expression evaluated before each "
-                                 "run; false ⇒ the step is skipped."),
-    Param("steps",  kind=LIST, doc="Children (for container items)."),
-    Param("seq_filename",    doc="(internal) source .tum file of this step; injected "
-                                 "by the loader."),
-)
-
-
 class TestItem:
     pass
 
@@ -126,11 +97,6 @@ def test_data(item: TestItem, child: dict) -> dict:
 
 
 class TestItem:
-    # Subclasses override with their own ParamSet to opt into the declarative
-    # validation. While ``PARAMS`` is empty / unset, the base class skips the
-    # unknown-param check for this item type — keeps the migration incremental.
-    PARAMS = None
-
     def __init__(
         self, dict_item: dict = None, parent: TestItem = None, status_queue=None, filename = ""
     ):
@@ -168,13 +134,6 @@ class TestItem:
             # creation of the params object
             self._prms = TestItemParams(dict_item, parent)
 
-            # Declarative-params validation. Only kicks in when the concrete
-            # subclass declares ``PARAMS`` — items not yet migrated stay
-            # silent. Warnings (not errors) during the migration window so
-            # existing .tum files don't break suddenly; will be flipped to
-            # errors once every item has migrated.
-            self._validate_declared_params(dict_item)
-
             # getting parameters for the test item
             try:
                 self._name = self._prms.getParam("name", default="", processed=True)
@@ -231,36 +190,6 @@ class TestItem:
 
         self.result = TestResult(self, TestValue.FAILURE, "Failure by default")
 
-    def _validate_declared_params(self, dict_item):
-        """Warn on unknown / missing-required params, if PARAMS is declared.
-
-        The check is opt-in per subclass: it only runs when the concrete
-        class sets a non-empty ``PARAMS`` attribute. Items not yet migrated
-        produce no diagnostics — preserving the historical "silently accept
-        anything" behavior until they get their declaration.
-        """
-        if not self.PARAMS:
-            return
-        # ``self._type`` is the parent root type at this point (subclasses set
-        # it after super().__init__), so use the class name as a stable label
-        # in diagnostics. ``self._name`` was preset to the type name by every
-        # subclass before super() ran, which gives a useful prefix.
-        label = f"{type(self).__name__} '{self._name}'"
-        declared = COMMON_PARAMS + self.PARAMS
-        unknown = unknown_keys(declared, dict_item)
-        if unknown:
-            accepted = ", ".join(sorted(declared.names()))
-            for k in unknown:
-                tm.print_warn(
-                    f"{label}: unknown parameter '{k}'. Accepted: {accepted}."
-                )
-        missing = missing_required(declared, dict_item)
-        for k in missing:
-            raise ETUMSyntaxError(
-                f"{label}: required parameter '{k}' is missing.",
-                self._seq_filename,
-            )
-
     def _filter_dict_item(self, dict_item):
         # Stores the content of the step to be displayed
         # in the GUI

src/testium/interpreter/test_items/test_item_check.py

@@ -5,22 +5,11 @@ from runtime.tum_except import ETUMSyntaxError, item_load_context
 import api.testium as tm
 from interpreter.utils.constants import TestItemType as cst
 from interpreter.utils.eval import evaluate
-from interpreter.utils.param_decl import Param, ParamSet, LIST
 
 class TestItemCheckValue(TestItem):
     """check item usage.
     check usage:{check: {name: check my func output, steps: ['$(pfn_echo) < 5']}}
     """
-
-    PARAMS = ParamSet(
-        Param("values", kind=LIST, required=True,
-              doc="List of expressions to evaluate. Each is expanded then "
-                  "evaluated; non-truthy results fail the check."),
-        # 'steps' is intentionally not redeclared here — it's the deprecated
-        # alias of 'values' and is already accepted by COMMON_PARAMS for
-        # container items. A runtime warning is emitted when 'steps' is used.
-    )
-
     def __init__(self, dict_item, parent = None, status_queue=None, filename=""):
         self._name = cst.TYPE_CHECK.item_name
         super().__init__(dict_item, parent, status_queue, filename=filename)

src/testium/interpreter/test_items/test_item_choices_dialog.py

@@ -2,25 +2,11 @@ from interpreter.test_items.test_item import test_run
 from interpreter.test_items.test_result import TestValue
 from interpreter.test_items.test_item_dialog_base import TestItemDialogBase, _is_text_mode, _is_interactive
 from interpreter.utils.constants import TestItemType as cst
-from interpreter.utils.param_decl import Param, ParamSet, BLOCK
 from runtime.tum_except import item_load_context
 import api.testium as tm
 
 
 class TestItemChoicesDialog(TestItemDialogBase):
-
-    PARAMS = ParamSet(
-        Param("question", required=True,
-              doc="Prompt shown above the list of choices."),
-        Param("choices", kind=BLOCK, required=True,
-              doc="Tree of choices: either a list of strings, or a nested "
-                  "mapping {label: subchoices, ...} to build a multi-level menu."),
-        Param("icon", default=None,
-              doc="Default icon name shown next to each choice."),
-        Param("auto_result", default=None,
-              doc="Batch-mode selection (path or label). None ⇒ FAILURE."),
-    )
-
     def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
         self._name = cst.TYPE_CHOICES_DLG.item_name
         super().__init__(dict_item, parent, status_queue, filename=filename)

src/testium/interpreter/test_items/test_item_console.py

@@ -10,7 +10,6 @@ from interpreter.test_items.test_item import test_run
 from interpreter.test_items.item_actions import TestItemActions
 from interpreter.test_items.item_actions.action import TestItemAction
 from interpreter.utils.constants import TestItemType as cst
-from interpreter.utils.param_decl import Param, ParamSet
 from interpreter.test_items.test_result import TestResult, TestValue
 
 
@@ -22,38 +21,6 @@ class TestItemConsoleAction(TestItemAction):
 
 
 class TestItemConsoleOpen(TestItemConsoleAction):
-
-    PARAMS = ParamSet(
-        Param("protocol", required=True,
-              doc="Transport: 'telnet', 'ssh', 'rawtcp', 'serial' or 'terminal'."),
-        Param("write_delay", default=0,
-              doc="Inter-character write delay in ms (slow devices)."),
-        Param("log", doc="Path to a log file capturing the console traffic."),
-        Param("overwrite_log", default=True,
-              doc="If true, truncate the log file at open; else append."),
-        # telnet
-        Param("telnet_host", doc="Hostname/IP for the telnet target."),
-        Param("telnet_port", default=69, doc="TCP port for telnet."),
-        # ssh
-        Param("ssh_host", doc="Hostname/IP for the SSH target."),
-        Param("ssh_user", doc="SSH login user."),
-        Param("ssh_pwd",  doc="SSH password (if key-based auth is not used)."),
-        # rawtcp
-        Param("tcp_host", doc="Hostname/IP for a raw-TCP connection."),
-        Param("tcp_port", doc="TCP port for a raw-TCP connection."),
-        # serial
-        Param("serial_port",     doc="Serial device path (e.g. /dev/ttyUSB0 or COM3)."),
-        Param("serial_baudrate", doc="Serial baudrate."),
-        Param("buffered", default=True,
-              doc="If true, the serial console buffers received bytes between reads."),
-        # terminal
-        Param("terminal_path",
-              doc="Working directory for the local terminal protocol."),
-        Param("shell",
-              doc="Shell command used for the local terminal protocol "
-                  "(default: 'cmd.exe' on Windows, '/usr/bin/env bash' elsewhere)."),
-    )
-
     def __init__(
         self, action_name, dict_item, parent=None, status_queue=None, filename=""
     ):
@@ -316,17 +283,6 @@ class TestItemConsoleWriteLn(TestItemConsoleAction):
 
 
 class TestItemConsoleReadUntil(TestItemConsoleAction):
-
-    PARAMS = ParamSet(
-        Param("expected", required=True,
-              doc="Regex matched against incoming console output until found "
-                  "or until timeout."),
-        Param("timeout", default=-1,
-              doc="Seconds before giving up. Negative means infinite."),
-        Param("mute", default=False,
-              doc="If true, don't echo received bytes to testium's stdout/log."),
-    )
-
     def __init__(
         self, action_name, dict_item, parent=None, status_queue=None, filename=""
     ):
@@ -380,27 +336,18 @@ class TestItemConsoleReadUntil(TestItemConsoleAction):
 
 
 class TestItemConsole(TestItemActions):
-
-    PARAMS = ParamSet(
-        Param("console_name", required=True,
-              doc="Identifier of the console — used by every nested action to "
-                  "reach back the same transport. Multiple consoles can coexist "
-                  "as long as their names differ."),
-    )
-
-    ACTIONS = {
-        "open": TestItemConsoleOpen,
-        "close": TestItemConsoleClose,
-        "write": TestItemConsoleWrite,
-        "writeln": TestItemConsoleWriteLn,
-        "read_until": TestItemConsoleReadUntil,
-    }
-
     def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
         super().__init__(
             cst.TYPE_CONSOLE, dict_item, parent, status_queue, filename=filename
         )
 
+        self.register_actions(
+            open=TestItemConsoleOpen,
+            close=TestItemConsoleClose,
+            write=TestItemConsoleWrite,
+            writeln=TestItemConsoleWriteLn,
+            read_until=TestItemConsoleReadUntil,
+        )
         self.actions_token = {}
 
         global console

src/testium/interpreter/test_items/test_item_cycle.py

@@ -8,36 +8,9 @@ from interpreter.test_items.test_result import TestResult, TestValue
 import api.testium as tm
 from interpreter.utils.params import TestItemParams
 from interpreter.utils.constants import TestItemType as cst
-from interpreter.utils.param_decl import Param, ParamSet, BLOCK
-
-
-# Sub-block validation: 'cycle' accepts an 'exit_condition:' mapping whose
-# own params are reported here so unknown keys inside it can be flagged
-# during a future Block-aware diagnostic pass. For now the parent only
-# declares that 'exit_condition' is an accepted top-level key.
-EXIT_CONDITION_PARAMS = ParamSet(
-    Param("time",      doc="HH:MM time of day after which the loop exits."),
-    Param("value",     doc="Expression; when truthy the loop exits."),
-    Param("file",      doc="Python file containing the exit-condition function."),
-    Param("func_name", doc="Function name in 'file' returning the exit value."),
-    Param("param",     doc="Arguments passed to the exit function."),
-    Param("eval", default="",
-          doc="Post-evaluation expression applied to the function's return."),
-)
 
 
 class TestItemCycle(TestItem):
-
-    PARAMS = ParamSet(
-        Param("iterator",
-              doc="Iterable (or string expanding to one) driving the loop. "
-                  "The current value is exposed as $(loop_param)."),
-        Param("exit_condition", kind=BLOCK,
-              doc="Optional block stopping the loop early: combine 'time', "
-                  "'value', or a 'file'+'func_name' pair (with optional "
-                  "'param' and 'eval')."),
-    )
-
     def __init__(self, dict_cycle, parent=None, status_queue=None, filename=""):
         self._name = cst.TYPE_CYCLE.item_name
         super().__init__(dict_cycle, parent, status_queue, filename=filename)

src/testium/interpreter/test_items/test_item_git.py

@@ -1,7 +1,6 @@
 from interpreter.test_items.test_item import (TestItem, test_run)
 from interpreter.test_items.test_result import (TestValue)
 from interpreter.utils.constants import TestItemType as cst
-from interpreter.utils.param_decl import Param, ParamSet, LIST
 from runtime.tum_except import ETUMParamError, ETUMSyntaxError
 import interpreter.utils.version as git
 
@@ -9,13 +8,6 @@ class TestItemGit(TestItem):
     """
     This item expect only one parameter which is a string or list of string being the path to the git folder
     """
-
-    PARAMS = ParamSet(
-        Param("repo", kind=LIST, required=True,
-              doc="Path to a git checkout, or list of such paths. Each is "
-                  "reported with its current version (tag + dirty state)."),
-    )
-
     def __init__(self, dict_item, parent = None, status_queue=None, filename=""):
         self._name = cst.TYPE_GIT.item_name
         super().__init__(dict_item, parent, status_queue, filename=filename)

src/testium/interpreter/test_items/test_item_group.py

@@ -1,17 +1,10 @@
 from interpreter.test_items.test_item import (TestItem, test_run)
 from interpreter.test_items.test_result import (TestResult, TestValue)
 from interpreter.utils.constants import TestItemType as cst
-from interpreter.utils.param_decl import ParamSet
 from runtime.tum_except import ETUMSyntaxError
 import api.testium as tm
 
 class TestItemGroup(TestItem):
-
-    # 'group' has no item-specific parameters; 'steps' is handled by COMMON_PARAMS.
-    # Declaring an empty ParamSet still opts in to unknown-param validation
-    # (e.g. typo 'stop_on_failures').
-    PARAMS = ParamSet()
-
     def __init__(self, dict_cycle, parent = None, status_queue=None, filename=""):
         self._name = cst.TYPE_GROUP.item_name
         super().__init__(dict_cycle, parent, status_queue, filename=filename)

src/testium/interpreter/test_items/test_item_image_dialog.py

@@ -4,7 +4,6 @@ from interpreter.test_items.test_item import test_run
 from interpreter.test_items.test_result import TestValue
 from interpreter.test_items.test_item_dialog_base import TestItemDialogBase, _is_text_mode, _is_interactive
 from interpreter.utils.constants import TestItemType as cst
-from interpreter.utils.param_decl import Param, ParamSet
 from runtime.tum_except import item_load_context
 import api.testium as tm
 
@@ -13,17 +12,6 @@ class TestItemImageDialog(TestItemDialogBase):
     """dialog_image item usage.
     dialog_image name: Nice image, question: could you press the red button, filename: img.jpg
     """
-
-    PARAMS = ParamSet(
-        Param("question", required=True,
-              doc="Prompt shown above the image."),
-        Param("filename", required=True,
-              doc="Path to the image file (relative to the test directory or absolute)."),
-        Param("auto_result", default=None,
-              doc="Outcome used in batch/non-interactive mode. Truthy ⇒ SUCCESS, "
-                  "None ⇒ FAILURE."),
-    )
-
     def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
         self._name = cst.TYPE_IMAGE_DLG.item_name
         super().__init__(dict_item, parent, status_queue, filename=filename)

src/testium/interpreter/test_items/test_item_json_rpc/__init__.py

@@ -11,7 +11,6 @@ from interpreter.test_items.item_actions.action import TestItemAction
 
 from interpreter.utils.constants import TestItemType as cst
 from interpreter.utils.eval import evaluate
-from interpreter.utils.param_decl import Param, ParamSet, BLOCK
 
 from interpreter.test_items.test_item_json_rpc.jsonrpc_adapters import (
     JrpcAdapter,
@@ -77,20 +76,6 @@ class TestItemJSRPCActionClose(TestItemAction):
 
 class TestItemJSRPCActionQuery(TestItemAction):
 
-    PARAMS = ParamSet(
-        Param("method", required=True,
-              doc="JSON-RPC method name to call."),
-        Param("params",
-              doc="Parameters payload (list, dict or scalar) sent to the method."),
-        Param("id", default="rand",
-              doc="JSON-RPC request id. 'rand' (default) ⇒ a random integer is used."),
-        Param("no_wait", default=False,
-              doc="If true, send the request without waiting for a response."),
-        Param("timeout", default=None,
-              doc="Seconds to wait for a response. None ⇒ inherits the transport "
-                  "default."),
-    )
-
     def __init__(
         self, action_name, dict_item, parent=None, status_queue=None, filename=""
     ):
@@ -144,13 +129,6 @@ class TestItemJSRPCActionQuery(TestItemAction):
 
 class TestItemJSRPCActionReceive(TestItemAction):
 
-    PARAMS = ParamSet(
-        Param("id", required=True,
-              doc="JSON-RPC request id whose response we expect."),
-        Param("timeout", default=None,
-              doc="Seconds to wait for the response. None ⇒ transport default."),
-    )
-
     def __init__(
         self, action_name, dict_item, parent=None, status_queue=None, filename=""
     ):
@@ -194,29 +172,6 @@ class TestItemJSON_RPC(TestItemActions):
     This item TBD
     """
 
-    PARAMS = ParamSet(
-        Param("console", kind=BLOCK,
-              doc="Console-transport block: {console_name, …}. Either 'console' "
-                  "or 'udp' must be set."),
-        Param("udp", kind=BLOCK,
-              doc="UDP-transport block: {host, port, …}. Either 'console' or "
-                  "'udp' must be set."),
-        Param("version", default="1.0",
-              doc="JSON-RPC protocol version ('1.0' or '2.0')."),
-        Param("timeout", required=True,
-              doc="Default seconds to wait for a JSON-RPC response across all "
-                  "child query/receive actions."),
-        Param("mute", default=False,
-              doc="If true, don't echo wire traffic to the log."),
-    )
-
-    ACTIONS = {
-        "open": TestItemJSRPCActionOpen,
-        "close": TestItemJSRPCActionClose,
-        "query": TestItemJSRPCActionQuery,
-        "receive": TestItemJSRPCActionReceive,
-    }
-
     def __init__(
         self, dict_item: dict, parent: TestItem = None, status_queue=None, filename=""
     ):
@@ -224,6 +179,13 @@ class TestItemJSON_RPC(TestItemActions):
             cst.TYPE_JSON_RPC, dict_item, parent, status_queue, filename=filename
         )
 
+        self.register_actions(
+            open=TestItemJSRPCActionOpen,
+            close=TestItemJSRPCActionClose,
+            query=TestItemJSRPCActionQuery,
+            receive=TestItemJSRPCActionReceive,
+        )
+
         # Console specific params
         self._console = self._prms.getParam("console", required=False)
         # UDP specific params

src/testium/interpreter/test_items/test_item_let.py

@@ -8,20 +8,12 @@ from interpreter.test_items.test_result import (TestResult, TestValue)
 from runtime.tum_except import ETUMSyntaxError, item_load_context
 import api.testium as tm
 from interpreter.utils.constants import TestItemType as cst
-from interpreter.utils.param_decl import Param, ParamSet, LIST
 
 class TestItemLet(TestItem):
     """let item usage.
     let values: {variable1: a, variable2: /dev/ttyUSB0, variable3: 115200}
     let eval: {conditional_exec: "random.randint(1, 4)"}
     """
-
-    PARAMS = ParamSet(
-        Param("values", kind=LIST, required=True,
-              doc="Mapping (or list of single-pair mappings) of global-dict "
-                  "key → value to set. Values are expanded at execution time."),
-    )
-
     def __init__(self, dict_item, parent = None, status_queue=None, filename=""):
         self._name = cst.TYPE_LET.item_name
         super().__init__(dict_item, parent, status_queue, filename=filename)

src/testium/interpreter/test_items/test_item_lua_func.py

@@ -11,7 +11,6 @@ import api.testium as tm
 from interpreter.utils.lua_func_exec import LuaFuncExecEngine
 from interpreter.utils.api_srv import api_request
 from interpreter.utils.constants import TestItemType as cst
-from interpreter.utils.param_decl import Param, ParamSet, LIST
 
 _LUA_FUNC_CONTEXTS_KEY = "_lua_func_contexts"
 
@@ -22,21 +21,6 @@ class TestItemLuaFunc(TestItem):
     Optional: context_id: <id>  — share a persistent process with other lua_func items using the same id.
     """
 
-    PARAMS = ParamSet(
-        Param("file", required=True,
-              doc="Path to the .lua file containing the function."),
-        Param("func_name", required=True,
-              doc="Name of the function to call in the file."),
-        Param("param", kind=LIST,
-              doc="Arguments passed to the function. Each entry is expanded "
-                  "before the call. Special tokens $(loop_param) / $(loop_index) "
-                  "resolve from the surrounding cycle."),
-        Param("context_id", default=None,
-              doc="If set, the lua_func subprocess is kept alive and reused by "
-                  "every other lua_func item with the same context_id — enables "
-                  "shared in-memory state between successive calls."),
-    )
-
     def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
         self._name = cst.TYPE_LUA_FUNCTION.item_name
         super().__init__(dict_item, parent, status_queue, filename=filename)

src/testium/interpreter/test_items/test_item_msg_dialog.py

@@ -5,7 +5,6 @@ from interpreter.test_items.test_item import test_run
 from interpreter.test_items.test_result import TestValue
 from interpreter.test_items.test_item_dialog_base import TestItemDialogBase, _is_text_mode, _is_interactive
 from interpreter.utils.constants import TestItemType as cst
-from interpreter.utils.param_decl import Param, ParamSet
 from runtime.tum_except import item_load_context
 
 
@@ -13,15 +12,6 @@ class TestItemMsgDialog(TestItemDialogBase):
     """dialog_message item usage.
     dialog_message name: Nice message, question: Open the door and press OK
     """
-
-    PARAMS = ParamSet(
-        Param("question", required=True,
-              doc="Message body shown to the user. Multi-line strings are supported."),
-        Param("auto_result", default=None,
-              doc="Outcome used in batch/non-interactive mode instead of waiting "
-                  "for the user. Truthy ⇒ SUCCESS, None ⇒ FAILURE."),
-    )
-
     def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
         self._name = cst.TYPE_MESSAGE_DLG.item_name
         super().__init__(dict_item, parent, status_queue, filename=filename)

src/testium/interpreter/test_items/test_item_note_dialog.py

@@ -2,23 +2,11 @@ from interpreter.test_items.test_item import test_run
 from interpreter.test_items.test_result import TestValue
 from interpreter.test_items.test_item_dialog_base import TestItemDialogBase, _is_text_mode, _is_interactive
 from interpreter.utils.constants import TestItemType as cst
-from interpreter.utils.param_decl import Param, ParamSet
 from runtime.tum_except import item_load_context
 import api.testium as tm
 
 
 class TestItemNoteDialog(TestItemDialogBase):
-
-    PARAMS = ParamSet(
-        Param("question", required=True,
-              doc="Prompt shown above the note input field."),
-        Param("auto_result", default=None,
-              doc="Batch-mode outcome: None ⇒ FAILURE, 'cancel' ⇒ cancelled, "
-                  "any other truthy ⇒ SUCCESS with auto_value."),
-        Param("auto_value", default=None,
-              doc="Note text used in batch mode when auto_result is set."),
-    )
-
     def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
         self._name = cst.TYPE_NOTE_DLG.item_name
         super().__init__(dict_item, parent, status_queue, filename=filename)

src/testium/interpreter/test_items/test_item_parallel.py

@@ -6,7 +6,6 @@ from interpreter.test_items.test_item import test_run
 from interpreter.test_items.test_result import TestResult, TestValue
 from interpreter.utils.constants import TestItemType as cst
 from interpreter.utils.eval import eval_to_boolean
-from interpreter.utils.param_decl import Param, ParamSet, LIST, BLOCK, Enum
 from runtime.tum_except import ETUMSyntaxError
 from runtime.string_queue import StringQueue
 from runtime.stdout_redirect import stdio_redir
@@ -16,12 +15,6 @@ class TestItemParallelBranch(TestItemContainer):
     """One branch of a parallel item. Runs its children sequentially,
     optionally waiting for a condition before starting."""
 
-    PARAMS = ParamSet(
-        Param("wait_for", kind=BLOCK,
-              doc="Optional block {condition, timeout} that defers the branch "
-                  "start until the condition is truthy (or the timeout elapses)."),
-    )
-
     def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
         super().__init__(cst.TYPE_PARALLEL_BRANCH, dict_item, parent, status_queue, filename=filename)
         self._wait_condition = None
@@ -94,15 +87,6 @@ class TestItemParallel(TestItemContainer):
                   - ...
     """
 
-    PARAMS = ParamSet(
-        Param("branches", kind=LIST, required=True,
-              doc="List of branch blocks (each branch holds its own 'steps' "
-                  "and optional 'wait_for')."),
-        Param("sync", kind=Enum("all", "any"), default="all",
-              doc="'all' (default) waits for every branch; 'any' returns as "
-                  "soon as the first branch completes."),
-    )
-
     def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
         branches = dict_item.get("branches", [])
         if not branches:

src/testium/interpreter/test_items/test_item_py_func.py

@@ -11,7 +11,6 @@ import api.testium as tm
 from interpreter.utils.py_func_exec import PyFuncExecEngine
 from interpreter.utils.api_srv import api_request
 from interpreter.utils.constants import TestItemType as cst
-from interpreter.utils.param_decl import Param, ParamSet, LIST
 
 _PY_FUNC_CONTEXTS_KEY = "_py_func_contexts"
 
@@ -22,21 +21,6 @@ class TestItemPyFunc(TestItem):
     Optional: context_id: <id>  — share a persistent process with other py_func items using the same id.
     """
 
-    PARAMS = ParamSet(
-        Param("file", required=True,
-              doc="Path to the .py file containing the function."),
-        Param("func_name", required=True,
-              doc="Name of the function to call in the file."),
-        Param("param", kind=LIST,
-              doc="Arguments passed to the function. Each entry is expanded "
-                  "before the call. Special tokens $(loop_param) / $(loop_index) "
-                  "resolve from the surrounding cycle."),
-        Param("context_id", default=None,
-              doc="If set, the py_func subprocess is kept alive and reused by "
-                  "every other py_func item with the same context_id — enables "
-                  "shared in-memory state between successive calls."),
-    )
-
     def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
         self._name = cst.TYPE_PY_FUNCTION.item_name
         super().__init__(dict_item, parent, status_queue, filename=filename)

src/testium/interpreter/test_items/test_item_question_dialog.py

@@ -2,7 +2,6 @@ from interpreter.test_items.test_item import test_run
 from interpreter.test_items.test_result import TestValue
 from interpreter.test_items.test_item_dialog_base import TestItemDialogBase, _is_text_mode, _is_interactive
 from interpreter.utils.constants import TestItemType as cst
-from interpreter.utils.param_decl import Param, ParamSet
 from runtime.tum_except import item_load_context
 
 
@@ -10,14 +9,6 @@ class TestItemQuestionDialog(TestItemDialogBase):
     """dialog_question item usage.
     dialog_question name: Nice question, question: "If OK, press OK, If not, press cancel"
     """
-
-    PARAMS = ParamSet(
-        Param("question", required=True,
-              doc="Yes/No prompt presented to the user."),
-        Param("auto_result", default=None,
-              doc="Batch-mode answer ('yes'/'no' or truthy/falsy). None ⇒ FAILURE."),
-    )
-
     def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
         self._name = cst.TYPE_QUESTION_DLG.item_name
         super().__init__(dict_item, parent, status_queue, filename=filename)

src/testium/interpreter/test_items/test_item_report.py

@@ -3,17 +3,9 @@ from interpreter.test_items.test_item import (TestItem, test_run)
 from interpreter.test_items.test_result import (TestValue)
 from runtime.tum_except import ETUMSyntaxError
 from interpreter.utils.constants import TestItemType as cst
-from interpreter.utils.param_decl import Param, ParamSet, LIST
 from interpreter.test_report.test_report import Export
 
 class TestItemReport(TestItem):
-
-    PARAMS = ParamSet(
-        Param("export", kind=LIST, required=True,
-              doc="List of exporters to run (junit, sqlite, …). Each entry is a "
-                  "mapping describing the exporter type and its parameters."),
-    )
-
     def __init__(self, dict_item, parent = None, status_queue=None, filename=""):
         self._name = cst.TYPE_REPORT.item_name
         super().__init__(dict_item, parent, status_queue, filename=filename)

src/testium/interpreter/test_items/test_item_run.py

@@ -10,7 +10,6 @@ from interpreter.test_items.test_item import (TestItem, test_run)
 from interpreter.test_items.test_result import (TestValue)
 import api.testium as tm
 from interpreter.utils.constants import TestItemType as cst
-from interpreter.utils.param_decl import Param, ParamSet
 from runtime.tum_except import ETUMSyntaxError, ETUMRuntimeError, item_load_context
 
 
@@ -25,15 +24,9 @@ def _testium_launch_cmd():
     appimage = os.environ.get("APPIMAGE")
     if appimage:
         return [appimage]
-    # Flatpak: re-launch via the Flatpak app id, but on the host side —
-    # the `flatpak` CLI cannot run inside our sandbox (no D-Bus access to the
-    # host Flatpak service, and the host binary would need host libs that are
-    # ABI-incompatible with the sandbox runtime). flatpak-spawn proxies the
-    # call to the host via org.freedesktop.Flatpak (allowed by --talk-name in
-    # the manifest).
+    # Flatpak: re-launch via the Flatpak app id.
     if os.path.isfile("/.flatpak-info"):
-        return ["flatpak-spawn", "--host",
-                "flatpak", "run", "org.testium.Testium"]
+        return ["flatpak", "run", "org.testium.Testium"]
     # PyInstaller frozen exe: sys.executable is the binary itself.
     if getattr(sys, "frozen", False):
         return [sys.executable]
@@ -58,25 +51,6 @@ def nowInBetween(start, end):
 
 
 class TestItemRun(TestItem):
-
-    PARAMS = ParamSet(
-        Param("tum", required=True,
-              doc="Path to the .tum file launched in a fresh testium instance."),
-        Param("param_file", default="",
-              doc="Optional path to a param.yaml passed to the sub-instance."),
-        Param("log_file", default="",
-              doc="Path where the sub-instance writes its log."),
-        Param("report_file", default="",
-              doc="Path where the sub-instance writes its report."),
-        Param("start_time",
-              doc="HH:MM time of day after which the sub-instance may run."),
-        Param("end_time",
-              doc="HH:MM time of day after which the sub-instance no longer runs."),
-        Param("wait_for_exec",
-              doc="If true, block until the time window opens. Requires both "
-                  "start_time and end_time."),
-    )
-
     def __init__(self, dict_item, parent = None, status_queue=None, filename=""):
         self._name = cst.TYPE_RUN.item_name
         super().__init__(dict_item, parent, status_queue, filename=filename)

src/testium/interpreter/test_items/test_item_runtime_plot.py

@@ -11,7 +11,6 @@ from interpreter.test_items.item_actions import TestItemActions
 from interpreter.test_items.item_actions.action import TestItemAction
 from interpreter.utils.constants import TestItemType as cst
 from interpreter.utils.eval import evaluate
-from interpreter.utils.param_decl import Param, ParamSet, LIST
 
 
 class TestItemPlotAction(TestItemAction):
@@ -22,12 +21,6 @@ class TestItemPlotAction(TestItemAction):
 
 
 class TestItemPlotActionOpen(TestItemPlotAction):
-
-    PARAMS = ParamSet(
-        Param("log_path", default=None,
-              doc="Optional file to which the plot data are appended."),
-    )
-
     def __init__(
         self, action_name, dict_item, parent=None, status_queue=None, filename=""
     ):
@@ -64,15 +57,6 @@ class TestItemPlotActionOpen(TestItemPlotAction):
 
 
 class TestItemPlotActionClose(TestItemPlotAction):
-
-    PARAMS = ParamSet(
-        Param("wait_dialog_exit", default=False,
-              doc="If true, the close action blocks until the user closes the "
-                  "plot window (or timeout)."),
-        Param("timeout", default=-1,
-              doc="Seconds to wait when wait_dialog_exit is true. Negative ⇒ infinite."),
-    )
-
     def __init__(
         self, action_name, dict_item, parent=None, status_queue=None, filename=""
     ):
@@ -112,20 +96,6 @@ class TestItemPlotActionClose(TestItemPlotAction):
 
 
 class TestItemPlotActionPeriodic(TestItemPlotAction):
-
-    PARAMS = ParamSet(
-        Param("period", required=True,
-              doc="Seconds between two calls of the periodic function."),
-        Param("file", required=True,
-              doc="Path to the .py file holding the periodic function."),
-        Param("func_name", required=True,
-              doc="Name of the periodic function."),
-        Param("param", kind=LIST,
-              doc="Arguments passed to the periodic function on each call."),
-        Param("eval", default="",
-              doc="Post-evaluation applied to the function's return value."),
-    )
-
     def __init__(
         self, action_name, dict_item, parent=None, status_queue=None, filename=""
     ):
@@ -199,13 +169,6 @@ class TestItemPlotActionAdd(TestItemPlotAction):
 
 
 class TestItemPlotActionLastValues(TestItemPlotAction):
-
-    PARAMS = ParamSet(
-        Param("name", kind=LIST,
-              doc="List of plot variable names whose last sample is returned. "
-                  "Result is stored in $(plv_<plot_name>) as a dict."),
-    )
-
     def __init__(self, action_name, dict_item, parent=None, status_queue=None, filename=""):
         super().__init__(
             action_name, cst.TYPE_GRAPH_ACTION, dict_item, parent, status_queue, filename=filename
@@ -256,25 +219,18 @@ class TestItemPlotActionExport(TestItemPlotAction):
 
 
 class TestItemPlot(TestItemActions):
-
-    PARAMS = ParamSet(
-        Param("plot_name", required=True,
-              doc="Identifier of the plot window — referenced by every nested "
-                  "action and by $(plv_<plot_name>) for last-values output."),
-    )
-
-    ACTIONS = {
-        "open": TestItemPlotActionOpen,
-        "close": TestItemPlotActionClose,
-        "periodic": TestItemPlotActionPeriodic,
-        "add": TestItemPlotActionAdd,
-        "last_value": TestItemPlotActionLastValues,
-        "export": TestItemPlotActionExport,
-    }
-
     def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
         super().__init__(
             cst.TYPE_GRAPH, dict_item, parent, status_queue, filename=filename
         )
 
+        self.register_actions(
+            open=TestItemPlotActionOpen,
+            close=TestItemPlotActionClose,
+            periodic=TestItemPlotActionPeriodic,
+            add=TestItemPlotActionAdd,
+            last_value=TestItemPlotActionLastValues,
+            export=TestItemPlotActionExport,
+        )
+
         self.actions_token = self._prms.getParam("plot_name", required=True)

src/testium/interpreter/test_items/test_item_sleep.py

@@ -7,7 +7,6 @@ import api.testium as tm
 from interpreter.test_items.test_item import (TestItem, test_run)
 from interpreter.test_items.test_result import (TestValue)
 from interpreter.utils.constants import TestItemType as cst
-from interpreter.utils.param_decl import Param, ParamSet
 from runtime.tum_except import ETUMSyntaxError, ETUMRuntimeError, item_load_context
 
 class TestItemSleep(TestItem):
@@ -15,15 +14,6 @@ class TestItemSleep(TestItem):
     sleep timeout: 10
     """
 
-    PARAMS = ParamSet(
-        Param("timeout", required=True,
-              doc="Duration to sleep. Number of seconds, or a string "
-                  "like '1d 2h 30m 15s'."),
-        Param("dialog", default=False,
-              doc="If true, show a cancel dialog (GUI mode) or an interactive "
-                  "Ctrl+C-able countdown (text mode)."),
-    )
-
     def __init__(self, dict_item, parent = None, status_queue=None, filename=""):
         self._name = cst.TYPE_SLEEP.item_name
         super().__init__(dict_item, parent, status_queue, filename=filename)

src/testium/interpreter/test_items/test_item_tested_references.py

@@ -2,23 +2,11 @@ from interpreter.test_items.test_item import test_run
 from interpreter.test_items.test_result import TestValue
 from interpreter.test_items.test_item_dialog_base import TestItemDialogBase, _is_text_mode, _is_interactive
 from interpreter.utils.constants import TestItemType as cst
-from interpreter.utils.param_decl import Param, ParamSet, LIST
 from runtime.tum_except import item_load_context
 import api.testium as tm
 
 
 class TestItemTestedRefsDialog(TestItemDialogBase):
-
-    PARAMS = ParamSet(
-        Param("question", required=True,
-              doc="Prompt asking the operator to enter the tested references."),
-        Param("reference", kind=LIST,
-              doc="Pre-filled list of references shown in the dialog."),
-        Param("auto_result", default=None,
-              doc="Batch-mode outcome: None ⇒ FAILURE, truthy ⇒ SUCCESS with "
-                  "the pre-filled references."),
-    )
-
     def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
         self._name = cst.TYPE_REFERENCE_DLG.item_name
         super().__init__(dict_item, parent, status_queue, filename=filename)

src/testium/interpreter/test_items/test_item_unittest.py

@@ -11,7 +11,6 @@ from interpreter.test_items.test_item import (TestItem, test_run, LOG_TEST_STOP,
 from interpreter.test_items.test_result import (TestResult, TestValue)
 from interpreter.test_items.test_item import test_data
 from interpreter.utils.constants import TestItemType as cst
-from interpreter.utils.param_decl import Param, ParamSet, LIST
 from runtime.stdout_redirect import stdio_redir
 
 class UnittestResult(TextTestResult):
@@ -96,15 +95,6 @@ class TestItemUnittestElement(TestItem):
 
 
 class TestItemUnittestFile(TestItem):
-
-    PARAMS = ParamSet(
-        Param("test_file", required=True,
-              doc="Path to the Python unittest file (TestCase subclass)."),
-        Param("test_method", kind=LIST,
-              doc="Optional list of method names to restrict the run to. "
-                  "When empty, every test_* method in the file is run."),
-    )
-
     def __init__(self, dict_item, parent = None, status_queue=None, filename=""):
         self._name = cst.TYPE_UNITTEST.item_name
         super().__init__(dict_item, parent, status_queue, filename=filename)

src/testium/interpreter/test_items/test_item_value_dialog.py

@@ -2,7 +2,6 @@ from interpreter.test_items.test_item import test_run
 from interpreter.test_items.test_result import TestValue
 from interpreter.test_items.test_item_dialog_base import TestItemDialogBase, _is_text_mode, _is_interactive
 from interpreter.utils.constants import TestItemType as cst
-from interpreter.utils.param_decl import Param, ParamSet
 from runtime.tum_except import item_load_context
 import api.testium as tm
 
@@ -11,19 +10,6 @@ class TestItemValueDialog(TestItemDialogBase):
     """dialog_value item usage.
     dialog_value name: Enter value, question: "Which value did you measure?"
     """
-
-    PARAMS = ParamSet(
-        Param("question", required=True,
-              doc="Prompt shown above the value input field."),
-        Param("default", default="",
-              doc="Pre-filled value of the input field."),
-        Param("auto_result", default=None,
-              doc="Batch-mode outcome: None ⇒ FAILURE, 'cancel' ⇒ cancelled, "
-                  "any other truthy ⇒ SUCCESS with auto_value."),
-        Param("auto_value", default=None,
-              doc="Value used in batch mode when auto_result is set."),
-    )
-
     def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
         self._name = cst.TYPE_VALUE_DLG.item_name
         super().__init__(dict_item, parent, status_queue, filename=filename)

src/testium/interpreter/utils/bins.py

@@ -17,11 +17,8 @@ Public API
 ``reset()``             : clear the cache (mostly useful for tests)
 """
 
-import atexit
 import os
-import shutil
 import subprocess
-import tempfile
 
 import api.testium as tm
 from interpreter.utils.paths import sys_app_path_lin, sys_app_path_win
@@ -33,6 +30,20 @@ from runtime.tum_except import ETUMRuntimeError
 _PYTHON_CANDIDATES = ["python3", "python"]
 _LUA_CANDIDATES = ["lua", "lua5.5", "lua5.4", "lua5.3", "lua5.2", "lua5.1"]
 
+# When running inside a Flatpak, --filesystem=host-os mounts the host at
+# /run/host (read-only). Binaries and libraries from the host are not on the
+# sandbox PATH/LD_LIBRARY_PATH, so we probe and inject them explicitly.
+_FLATPAK_HOST_DIRS = [
+    "/run/host/usr/local/bin",
+    "/run/host/usr/bin",
+    "/run/host/bin",
+]
+_FLATPAK_HOST_LIB_DIRS = [
+    "/run/host/usr/lib",
+    "/run/host/usr/lib64",
+    "/run/host/usr/local/lib",
+]
+
 # Inside an AppImage, AppRun prepends $APPDIR/usr/bin to PATH and exports a
 # bundle-local PYTHONHOME / PYTHONPATH / LD_LIBRARY_PATH. We want py_func and
 # lua_func to run under the *host* interpreter (not the bundled one), so we
@@ -53,162 +64,78 @@ def _in_appimage():
     return "APPIMAGE" in os.environ
 
 
-def apply_host_libs(env):
-    """Strip bundle-local entries from *env* so a host binary can run cleanly.
+def apply_host_lua_paths(env):
+    """Prepend host Lua module dirs to LUA_PATH / LUA_CPATH (Flatpak only).
 
-    Only meaningful for AppImage: removes $APPDIR-prefixed entries from
-    LD_LIBRARY_PATH / PYTHONPATH / PATH and drops PYTHONHOME, so the host
-    interpreter doesn't try to load the bundled (incompatible) Python
-    lib/site-packages. Flatpak is handled via flatpak-spawn --host instead
-    (see flatpak_host_spawn), so the sandbox env is irrelevant there.
+    Must be called after user-defined lua_env overrides are applied, so host
+    paths are always first regardless of user config. User-defined paths remain
+    in the variable but after the host ones.
     """
-    if not _in_appimage():
+    if not _in_flatpak():
         return
-    appdir = os.environ.get("APPDIR", "")
-    if appdir:
-        for var, sep in (("LD_LIBRARY_PATH", ":"),
-                         ("PYTHONPATH", os.pathsep),
-                         ("PATH", os.pathsep)):
-            cur = env.get(var, "")
-            if not cur:
-                continue
-            cleaned = sep.join(
-                p for p in cur.split(sep)
-                if p and not p.startswith(appdir)
-            )
-            if cleaned:
-                env[var] = cleaned
-            else:
-                env.pop(var, None)
-    env.pop("PYTHONHOME", None)
+    _LUA_VERSIONS = ["5.5", "5.4", "5.3", "5.2", "5.1"]
+    _HOST = "/run/host/usr"
+    cpath_dirs, lpath_dirs = [], []
+    for v in _LUA_VERSIONS:
+        for base in [f"{_HOST}/lib/lua/{v}",
+                     f"{_HOST}/lib64/lua/{v}",
+                     f"{_HOST}/lib/x86_64-linux-gnu/lua/{v}"]:
+            cpath_dirs.append(f"{base}/?.so")
+        lpath_dirs.append(f"{_HOST}/share/lua/{v}/?.lua")
+        lpath_dirs.append(f"{_HOST}/share/lua/{v}/?/init.lua")
+    sep = ";"
+    host_cpath = sep.join(cpath_dirs)
+    host_lpath = sep.join(lpath_dirs)
+    # ;; keeps Lua's compiled-in defaults at the end as last resort
+    env["LUA_CPATH"] = host_cpath + sep + env.get("LUA_CPATH", ";;")
+    env["LUA_PATH"]  = host_lpath + sep + env.get("LUA_PATH",  ";;")
 
 
-# ---------- Flatpak: spawn on host outside the sandbox -----------------------
-#
-# Inside a Flatpak the sandbox glibc is incompatible with host shared libraries,
-# so we can't run host Python/Lua under the sandbox runtime — `LD_LIBRARY_PATH`
-# tricks hit a `_dl_call_libc_early_init` assertion. The supported way out is
-# `flatpak-spawn --host`, which talks to the session-bus Flatpak D-Bus service
-# (org.freedesktop.Flatpak.Development) and asks it to spawn a process in the
-# host execution environment instead of inside our sandbox. The manifest must
-# grant `--talk-name=org.freedesktop.Flatpak` for the D-Bus call to be allowed.
-#
-# The host process can't see our /app/ contents (sandbox-only), so when we
-# spawn host Python/Lua to run `py_func` / `lua_func`, the cwd must be a
-# directory both sides can reach. /tmp is shared (--filesystem=/tmp), so we
-# stage the testium package there once per process and reuse it for every
-# spawn. In source mode (testium under $HOME) the host already sees the
-# original path, so we skip the copy.
+def apply_host_libs(env):
+    """Prepare *env* for launching a host binary from inside our bundle.
 
-_staged_testium_path = None
-
-
-def _get_host_testium_path():
-    """Return a path to the testium package that the host can read.
-
-    - Source / wheel / PyInstaller install under $HOME → return testium_path()
-      as-is (host sees the same path via --filesystem=home).
-    - Flatpak bundle (testium under /app/) → stage a copy under /tmp on first
-      call and reuse it for the rest of the process.
+    - Flatpak: prepend host library dirs to LD_LIBRARY_PATH so the dynamic
+      linker can find host .so files mounted under /run/host.
+    - AppImage: strip $APPDIR-prefixed entries from LD_LIBRARY_PATH and
+      PYTHONPATH and drop PYTHONHOME, so the host interpreter doesn't try
+      to load the bundled (incompatible) Python lib/site-packages.
+    - Otherwise: no-op.
     """
-    global _staged_testium_path
-    if _staged_testium_path is not None:
-        return _staged_testium_path
-
-    # Imported lazily to avoid a circular import (paths.py -> api.testium).
-    from interpreter.utils.paths import testium_path
-    tp = testium_path()
-
-    if not tp.startswith("/app/"):
-        _staged_testium_path = tp
-        return tp
-
-    staged = tempfile.mkdtemp(prefix="testium_host_", dir="/tmp")
-    # copytree refuses to write into an existing dir unless dirs_exist_ok=True.
-    # mkdtemp creates the dir, so we copy *into* it.
-    for entry in os.listdir(tp):
-        src = os.path.join(tp, entry)
-        dst = os.path.join(staged, entry)
-        if os.path.isdir(src):
-            shutil.copytree(src, dst, symlinks=True)
-        else:
-            shutil.copy2(src, dst, follow_symlinks=False)
-    _staged_testium_path = staged
-    atexit.register(shutil.rmtree, staged, ignore_errors=True)
-    return staged
-
-
-_FORWARDED_ENV_KEYS = (
-    "HOME", "USER", "LOGNAME", "TMPDIR",
-    "XDG_RUNTIME_DIR", "XDG_DATA_HOME", "XDG_CONFIG_HOME", "XDG_CACHE_HOME",
-    "DBUS_SESSION_BUS_ADDRESS", "DISPLAY", "WAYLAND_DISPLAY",
-    "LANG", "LC_ALL",
-)
-
-
-def flatpak_host_spawn(interp_bin, cmd_args, host_cwd, extra_env=None):
-    """Build a flatpak-spawn --host command vector.
-
-    Args:
-        interp_bin: absolute path to the host interpreter (e.g. /usr/bin/python3).
-        cmd_args: list of arguments passed to the interpreter.
-        host_cwd: working directory on the host (must be reachable from host).
-        extra_env: optional {name: value} of env vars to set on the host side
-                   in addition to the default forwarded set. Values of ""
-                   unset the variable on the host.
-
-    Returns a list suitable for subprocess.Popen.
-    """
-    spawn = ["flatpak-spawn", "--host", f"--directory={host_cwd}"]
-    forwarded = {}
-    for key in _FORWARDED_ENV_KEYS:
-        val = os.environ.get(key)
-        if val:
-            forwarded[key] = val
-    if extra_env:
-        forwarded.update(extra_env)
-    for k, v in forwarded.items():
-        if v == "":
-            spawn.append(f"--unset-env={k}")
-        else:
-            spawn.append(f"--env={k}={v}")
-    spawn.append(interp_bin)
-    spawn.extend(cmd_args)
-    return spawn
-
-
-def _which_host_flatpak(name):
-    """Resolve a binary name (or absolute path) on the host via flatpak-spawn.
-
-    We can't probe /run/host/... because (a) only host-os is mounted there,
-    not arbitrary paths like /scratch, and (b) returning a /run/host path
-    would be useless — the host-side spawn sees a different filesystem and
-    needs the host-native path anyway.
-    """
-    if os.path.isabs(name):
-        cmd = flatpak_host_spawn("/bin/sh", ["-c", f'test -x "{name}"'],
-                                 host_cwd="/tmp")
-        try:
-            r = subprocess.run(cmd, capture_output=True, timeout=10)
-        except (FileNotFoundError, PermissionError, subprocess.TimeoutExpired):
-            return ""
-        return name if r.returncode == 0 else ""
-    cmd = flatpak_host_spawn("/bin/sh", ["-c", f'command -v "{name}"'],
-                             host_cwd="/tmp")
-    try:
-        r = subprocess.run(cmd, capture_output=True, text=True, timeout=10)
-    except (FileNotFoundError, PermissionError, subprocess.TimeoutExpired):
-        return ""
-    if r.returncode != 0:
-        return ""
-    return r.stdout.strip()
+    if _in_flatpak():
+        dirs = ":".join(d for d in _FLATPAK_HOST_LIB_DIRS if os.path.isdir(d))
+        if dirs:
+            existing = env.get("LD_LIBRARY_PATH", "")
+            env["LD_LIBRARY_PATH"] = dirs + (":" + existing if existing else "")
+        return
+    if _in_appimage():
+        appdir = os.environ.get("APPDIR", "")
+        if appdir:
+            for var, sep in (("LD_LIBRARY_PATH", ":"),
+                             ("PYTHONPATH", os.pathsep),
+                             ("PATH", os.pathsep)):
+                cur = env.get(var, "")
+                if not cur:
+                    continue
+                cleaned = sep.join(
+                    p for p in cur.split(sep)
+                    if p and not p.startswith(appdir)
+                )
+                if cleaned:
+                    env[var] = cleaned
+                else:
+                    env.pop(var, None)
+        env.pop("PYTHONHOME", None)
 
 
 def _which(name):
     if tm.OS() == "Windows":
         return sys_app_path_win(name)
     if _in_flatpak():
-        return _which_host_flatpak(name)
+        for d in _FLATPAK_HOST_DIRS:
+            p = os.path.join(d, name)
+            if os.path.isfile(p) and os.access(p, os.X_OK):
+                return p
+        return ""
     if _in_appimage():
         for d in _APPIMAGE_HOST_DIRS:
             p = os.path.join(d, name)
@@ -219,33 +146,14 @@ def _which(name):
 
 
 def _probe_env():
-    """Subprocess env for probing host binaries.
-
-    In AppImage we still need to scrub APPDIR-prefixed entries; in Flatpak we
-    delegate execution to the host via flatpak-spawn so the sandbox env doesn't
-    matter, but apply_host_libs is a no-op cost.
-    """
+    """Subprocess env for probing host binaries (adds host libs in Flatpak)."""
     env = os.environ.copy()
     apply_host_libs(env)
     return env
 
 
-def _run_probe(cmd):
-    """Run a probe command, dispatching through flatpak-spawn --host in Flatpak.
-
-    Returns (stdout, stderr) as str, or None on failure.
-    """
-    if _in_flatpak():
-        spawn = flatpak_host_spawn(cmd[0], cmd[1:], host_cwd="/tmp")
-        try:
-            r = subprocess.run(
-                spawn, capture_output=True, text=True, timeout=10,
-            )
-        except (FileNotFoundError, PermissionError, subprocess.TimeoutExpired):
-            return None
-        if r.returncode != 0:
-            return None
-        return r.stdout, r.stderr
+def _python_version(path):
+    cmd = [path, "-c", "import sys; print(sys.version_info[:3])"]
     try:
         r = subprocess.run(
             cmd, capture_output=True, text=True,
@@ -253,15 +161,8 @@ def _run_probe(cmd):
         )
     except (FileNotFoundError, PermissionError, subprocess.TimeoutExpired):
         return None
-    return r.stdout, r.stderr
-
-
-def _python_version(path):
-    out = _run_probe([path, "-c", "import sys; print(sys.version_info[:3])"])
-    if out is None:
-        return None
     try:
-        return eval(out[0])
+        return eval(r.stdout)
     except Exception:
         return None
 
@@ -272,11 +173,15 @@ def _is_python3(path):
 
 
 def _lua_version(path):
-    out = _run_probe([path, "-v"])
-    if out is None:
+    try:
+        r = subprocess.run(
+            [path, "-v"], capture_output=True, text=True, timeout=10,
+            env=_probe_env(),
+        )
+    except (FileNotFoundError, PermissionError, subprocess.TimeoutExpired):
         return None
     # On Windows the version banner goes to stderr.
-    line = out[0] or out[1]
+    line = r.stdout or r.stderr
     try:
         major, minor, _patch = line.split(" ")[1].split(".")
         return (int(major), int(minor))
@@ -297,33 +202,22 @@ _SPECS = {
     "lua":    ("Lua 5.1+",  "lua_bin",    _LUA_CANDIDATES,    _is_lua51),
 }
 
-# Cached per (name, override) so that runtime changes to gd[gd_key] —
-# e.g. ``python_bin`` set from a YAML config file loaded *after*
-# eval_proc has already resolved its own interpreter — are picked up by
-# the next lookup instead of returning the stale, auto-discovered path.
-# Long-lived subprocesses (eval_proc) keep whatever they captured at
-# construction time, but every new PyProcessBase / FuncExecEngine spawned
-# afterwards sees the current override.
 _resolved = {}
 
 
 def _resolve(name):
+    if name in _resolved:
+        return _resolved[name]
+
     display, gd_key, candidates, validator = _SPECS[name]
     override = tm.gd(gd_key, "") or ""
 
-    cached = _resolved.get(name)
-    if cached is not None and cached[0] == override:
-        return cached[1]
-
     path = ""
     if override:
         # Absolute path: accept as-is (user knows exactly what they want).
         # Bare name: resolve via _which() so the override stays host-only in
         # Flatpak/AppImage instead of silently picking the bundled interpreter.
-        # In Flatpak we always defer to _which() so even absolute paths are
-        # checked from the host's perspective (the sandbox can't see e.g.
-        # /scratch/... paths that the user may have configured).
-        if os.path.isabs(override) and not _in_flatpak():
+        if os.path.isabs(override):
             resolved = override if (os.path.isfile(override)
                                     and os.access(override, os.X_OK)) else ""
         else:
@@ -345,7 +239,7 @@ def _resolve(name):
                 path = p
                 break
 
-    _resolved[name] = (override, path)
+    _resolved[name] = path
     return path
 
 

src/testium/interpreter/utils/lua_process.py

@@ -47,16 +47,9 @@ class LuaProcessBase:
         if self._process is not None:
             raise ETUMRuntimeError("The function subprocess has already been started.")
 
-        # In Flatpak the host can't see /app/lib/testium/lua_func, so use a
-        # staged copy under /tmp (shared between sandbox and host).
-        if bins._in_flatpak():
-            func_proc_path = os.path.join(
-                bins._get_host_testium_path(), "lua_func"
-            )
-        else:
-            func_proc_path = os.path.realpath(
-                os.path.join(subproc_path(), "lua_func")
-            )
+        func_proc_path = os.path.realpath(
+            os.path.join(subproc_path(), "lua_func")
+        )
 
         # POpen config
         CUST_ENV = {
@@ -78,6 +71,7 @@ class LuaProcessBase:
                     env[k] = e
                 else:
                     env[k] = e + ";" + env.get(k, "")
+        bins.apply_host_lua_paths(env)
 
         sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
         sock.bind(("localhost", 0))
@@ -85,7 +79,8 @@ class LuaProcessBase:
         sock.close()
 
         # POpen params
-        cmd_args = [
+        params = [
+            self._lbin,
             "main.lua",
             "--timeout",
             f"{self._timeout}",
@@ -96,31 +91,14 @@ class LuaProcessBase:
         ]
 
         if tm.debug_enabled() and tm.gd("debug_rpc", False):
-            cmd_args.append("--verbose")
-
-        if bins._in_flatpak():
-            # Run on the host outside the sandbox: avoids glibc ABI mismatches
-            # between the Flatpak runtime and host shared libraries.
-            host_env = {
-                k: env[k] for k in ("LUA_PATH", "LUA_CPATH", "PATH")
-                if k in env and env[k]
-            }
-            params = bins.flatpak_host_spawn(
-                self._lbin, cmd_args, host_cwd=func_proc_path,
-                extra_env=host_env,
-            )
-            popen_kwargs = {}
-        else:
-            params = [self._lbin, *cmd_args]
-            popen_kwargs = {"env": env, "cwd": func_proc_path}
+            params.append("--verbose")
 
         self._process = subprocess.Popen(
-            params,
+            params, env=env, cwd=func_proc_path,
             stdin=subprocess.DEVNULL,
             stdout=subprocess.PIPE,
             stderr=subprocess.PIPE,
             restore_signals=False,
-            **popen_kwargs,
         )
         # Route subprocess stdout/stderr (lua require failures, syntax
         # errors, anything written to fd 1/2 before the in-script

src/testium/interpreter/utils/param_decl.py

@@ -1,175 +0,0 @@
-"""Declarative description of a test item's accepted parameters.
-
-Each ``TestItem`` subclass declares its parameter surface as a class
-attribute::
-
-    class TestItemFoo(TestItem):
-        PARAMS = ParamSet(
-            Param("bar",      required=True,  doc="The bar value."),
-            Param("baz",      default=0,      doc="Optional baz."),
-            Param("modes",    kind=LIST,      doc="Iterable of modes."),
-            Param("strategy", kind=ENUM("a", "b"), doc="..."),
-            Param("opts",     kind=BLOCK,     doc="Sub-block."),
-        )
-
-The base ``TestItem.__init__`` consumes both ``COMMON_PARAMS`` (defined
-in ``test_item.py``) and the subclass ``PARAMS`` to:
-
-* warn on any key in the user's YAML that isn't declared anywhere
-  (catches typos like ``param_filee``);
-* expose a machine-readable schema for documentation generation and,
-  eventually, an LSP server.
-
-The descriptor is **purely about shape and naming**. Type coercion and
-runtime checking of expanded values remain the responsibility of each
-item's ``execute()`` method — most parameters are expressions
-(``$(...)`` / ``<| ... |>``) whose effective type is only known after
-expansion, so a static type would be misleading.
-
-Validation of *values* (e.g. ``start_time`` must match HH:MM) can be
-attached per-param via ``validate=lambda v: ...`` and is applied at
-execution time on the expanded value, not at load time.
-"""
-
-from dataclasses import dataclass, field
-from typing import Any, Callable, Optional, Union
-
-
-# ---------- Parameter "kinds" -------------------------------------------------
-#
-# These describe the YAML *shape* expected for a parameter, not its
-# semantic type. They drive the LSP completion (do we suggest a single
-# value, a list, a sub-block, an enum picker?) and the unknown-param
-# diagnostic; nothing more.
-
-SCALAR = "scalar"  # single value (string, number, bool, expression, ...)
-LIST = "list"      # YAML list — the historical ``getParamAll`` case
-BLOCK = "block"    # nested dict — e.g. ``cycle.exit:``
-
-
-@dataclass(frozen=True)
-class Enum:
-    """Closed enumeration of acceptable scalar values."""
-    values: tuple
-
-    def __init__(self, *values):
-        # frozen=True forbids assignment; bypass via object.__setattr__.
-        object.__setattr__(self, "values", tuple(values))
-
-    def __repr__(self):
-        return f"Enum({', '.join(repr(v) for v in self.values)})"
-
-
-Kind = Union[str, Enum]
-
-
-# ---------- The descriptor ----------------------------------------------------
-
-_MISSING = object()
-
-
-@dataclass(frozen=True)
-class Param:
-    """Declarative description of one accepted parameter.
-
-    Attributes
-    ----------
-    name : str
-        The YAML key.
-    kind : ``SCALAR`` (default) | ``LIST`` | ``BLOCK`` | ``Enum(...)``
-        The YAML shape expected.
-    required : bool
-        If True, missing the parameter is a load-time error.
-    default : Any
-        Default value when the parameter is absent. ``_MISSING`` when no
-        default was set (used to distinguish "absent" from "None").
-    doc : str
-        Free-form description used for hover / generated documentation.
-    validate : Optional[Callable[[Any], bool]]
-        Optional post-expansion validator, evaluated at ``execute()``
-        time on the effective (expanded) value. Returning ``False``
-        raises a clear error pointing at the param.
-    """
-    name: str
-    kind: Kind = SCALAR
-    required: bool = False
-    default: Any = _MISSING
-    doc: str = ""
-    validate: Optional[Callable[[Any], bool]] = None
-
-    def has_default(self):
-        return self.default is not _MISSING
-
-    def to_schema(self):
-        """Return a dict suitable for JSON Schema generation."""
-        s = {"name": self.name, "required": self.required, "doc": self.doc}
-        if isinstance(self.kind, Enum):
-            s["kind"] = "enum"
-            s["enum"] = list(self.kind.values)
-        else:
-            s["kind"] = self.kind
-        if self.has_default():
-            s["default"] = self.default
-        return s
-
-
-class ParamSet:
-    """Ordered, name-indexed collection of ``Param`` descriptors.
-
-    Supports concatenation (``COMMON_PARAMS + SUBCLASS_PARAMS``) to
-    merge the common surface with each item's own params. Later
-    declarations override earlier ones (so a subclass can tighten a
-    common param's docstring without redeclaring everything).
-    """
-
-    def __init__(self, *params):
-        self._params = {}
-        for p in params:
-            self.add(p)
-
-    def add(self, param):
-        if not isinstance(param, Param):
-            raise TypeError(f"ParamSet only accepts Param instances, got {type(param).__name__}")
-        self._params[param.name] = param
-
-    def __iter__(self):
-        return iter(self._params.values())
-
-    def __contains__(self, name):
-        return name in self._params
-
-    def __getitem__(self, name):
-        return self._params[name]
-
-    def names(self):
-        return tuple(self._params.keys())
-
-    def __add__(self, other):
-        if not isinstance(other, ParamSet):
-            return NotImplemented
-        merged = ParamSet()
-        merged._params = {**self._params, **other._params}
-        return merged
-
-    def to_schema(self):
-        return [p.to_schema() for p in self._params.values()]
-
-
-# ---------- Validation primitives --------------------------------------------
-
-def unknown_keys(declared, user_dict):
-    """Return the user-provided keys that are not declared in *declared*.
-
-    *declared* is a ``ParamSet``; *user_dict* is the raw YAML mapping
-    for the item. Unknown keys catch typos and obsolete parameters.
-    """
-    if not isinstance(user_dict, dict):
-        return ()
-    return tuple(k for k in user_dict.keys() if k not in declared)
-
-
-def missing_required(declared, user_dict):
-    """Return the names of declared required params absent from *user_dict*."""
-    if not isinstance(user_dict, dict):
-        return tuple(p.name for p in declared if p.required)
-    return tuple(p.name for p in declared if p.required and p.name not in user_dict)

src/testium/interpreter/utils/py_process.py

@@ -61,18 +61,14 @@ class PyProcessBase:
         if sock is not None:
             sock.close()
 
-        # In Flatpak the host can't see /app/lib/testium, so use a staged copy
-        # under /tmp (shared between sandbox and host) for both cwd and as the
-        # root in PYTHONPATH. Outside Flatpak the original paths are used.
-        if bins._in_flatpak():
-            tstium_path = bins._get_host_testium_path()
-            func_proc_path = tstium_path
-        else:
-            tstium_path = os.path.realpath(testium_path())
-            func_proc_path = os.path.realpath(subproc_path())
+        # Add the path of the subprocess (root sources of testium)
+        tstium_path = os.path.realpath(testium_path())
+        func_proc_path = os.path.realpath(subproc_path())
         env["PYTHONPATH"] = tstium_path + os.pathsep + self._ppath + os.pathsep + env.get("PYTHONPATH", "")
 
-        cmd_args = [
+        params = [
+            self._pbin,
+            # "-m",
             "py_func",
             "-p",
             f"{self._port}",
@@ -81,31 +77,14 @@ class PyProcessBase:
         ]
 
         if tm.debug_enabled() and tm.gd("debug_rpc", False):
-            cmd_args.append("-v")
-
-        if bins._in_flatpak():
-            # Run on the host outside the sandbox: avoids glibc ABI mismatches
-            # between the Flatpak runtime and host shared libraries.
-            host_env = {
-                k: env[k] for k in ("PYTHONPATH", "PATH")
-                if k in env and env[k]
-            }
-            params = bins.flatpak_host_spawn(
-                self._pbin, cmd_args, host_cwd=func_proc_path,
-                extra_env=host_env,
-            )
-            popen_kwargs = {}
-        else:
-            params = [self._pbin, *cmd_args]
-            popen_kwargs = {"env": env, "cwd": func_proc_path}
+            params.append("-v")
 
         self._process = subprocess.Popen(
-            params,
+            params, env=env, cwd=func_proc_path,
             stdin=subprocess.DEVNULL,
             stdout=subprocess.PIPE,
             stderr=subprocess.PIPE,
             restore_signals=False,
-            **popen_kwargs,
         )
         # Route subprocess stdout/stderr (early-startup errors,
         # unhandled exceptions, anything written to fd 1/2 before the

src/testium/interpreter/utils/test_init.py

@@ -165,14 +165,11 @@ def env_init():
     _constants_init()
 
 
-def apply_overrides(defines, gui_defaults):
-    """Push GUI defaults then CLI defines into the global dict.
-
-    Extracted from update_global so it can be called *before* eval_proc
-    starts: interpreter overrides (python_bin, lua_bin) must be visible
-    to bins.python_bin() on its first lookup, which happens during
-    eval_process_init.
+def update_global(config_files, defines, gui_defaults, silent=False):
+    """Global dict updated with the content of the config file and a dict provided.
+    this function returns the resulting dict.
     """
+    # GUI preferences applied first
     for k, v in gui_defaults.items():
         try:
             val = ast.literal_eval(v)
@@ -180,6 +177,7 @@ def apply_overrides(defines, gui_defaults):
             val = v
         tm.setgd(k, val)
 
+    # Then command line defines
     for k, v in defines.items():
         try:
             val = ast.literal_eval(v)
@@ -187,14 +185,6 @@ def apply_overrides(defines, gui_defaults):
             val = v
         tm.setgd(k, val)
 
-
-def update_global(config_files, defines, gui_defaults, silent=False):
-    """Global dict updated with the content of the config file and a dict provided.
-    this function returns the resulting dict.
-    """
-    # GUI preferences applied first, then command line defines
-    apply_overrides(defines, gui_defaults)
-
     # Then the configuration files
     # load global dic before test item
     _feed_gd_with_params(config_files, silent)

src/testium/interpreter/utils/version.py

@@ -1,54 +1,10 @@
-import atexit
 import os
-import stat
 import sys
-import tempfile
 from importlib import import_module
 
 import interpreter.utils.settings as prefs
 import api.testium as tm
 
-
-# When running inside a Flatpak, the host /usr/bin/git is reachable at
-# /run/host/usr/bin/git but linked against host glibc/zlib, which the
-# sandbox can't load (``libz-ng.so.2`` not found). gitpython resolves git
-# eagerly on import and would crash the whole test run. We install a
-# tiny shell wrapper under /tmp that forwards to ``flatpak-spawn --host
-# git``, and point gitpython at it via ``GIT_PYTHON_GIT_EXECUTABLE``.
-_HOST_GIT_WRAPPER = None
-
-
-def _setup_flatpak_git():
-    global _HOST_GIT_WRAPPER
-    if not os.path.isfile("/.flatpak-info"):
-        return
-    if _HOST_GIT_WRAPPER is not None:
-        return
-    fd, path = tempfile.mkstemp(prefix="testium-git-host-", suffix=".sh", dir="/tmp")
-    with os.fdopen(fd, "w") as f:
-        f.write('#!/bin/sh\nexec flatpak-spawn --host git "$@"\n')
-    os.chmod(path, stat.S_IRWXU)
-    _HOST_GIT_WRAPPER = path
-    atexit.register(_cleanup_flatpak_git)
-    os.environ["GIT_PYTHON_GIT_EXECUTABLE"] = path
-    # Silence gitpython's warning if its refresh probe ever still fails;
-    # the wrapper itself should make the probe succeed.
-    os.environ.setdefault("GIT_PYTHON_REFRESH", "quiet")
-
-
-def _cleanup_flatpak_git():
-    global _HOST_GIT_WRAPPER
-    if _HOST_GIT_WRAPPER and os.path.isfile(_HOST_GIT_WRAPPER):
-        try:
-            os.unlink(_HOST_GIT_WRAPPER)
-        except OSError:
-            pass
-        _HOST_GIT_WRAPPER = None
-
-
-_setup_flatpak_git()
-
-
 _cached_versions = {}
 
 def repo_rev(path):

src/testium/lsp/__init__.py

@@ -1,16 +0,0 @@
-"""testium language tooling.
-
-Hosts the JSON-Schema-style schema export of every test item type, and a
-``pygls`` language server that consumes the same schema to provide
-completion / hover / diagnostics for ``.tum`` files in any LSP-capable
-editor (VSCode, neovim, Helix, Emacs, …).
-
-Entry points (both surfaced through the ``testium`` CLI):
-
-- ``testium schema`` — dump the schema of every item type as JSON on stdout.
-  Zero runtime dependencies; can be used by editors that already speak the
-  YAML JSON Schema extension to get static completion immediately.
-
-- ``testium lsp`` — start the language server over stdio. Requires the
-  ``pygls`` optional dependency (``pip install testium[lsp]``).
-"""

src/testium/lsp/__main__.py

@@ -1,6 +0,0 @@
-"""Entry point for ``python -m testium.lsp`` (alternative to ``testium lsp``)."""
-
-from lsp.server import serve
-
-if __name__ == "__main__":
-    serve()

src/testium/lsp/schema.py

@@ -1,122 +0,0 @@
-"""Schema export of the test item registry.
-
-Walks every ``TestItemType`` entry (``interpreter/utils/constants.py``),
-combines its declared ``PARAMS`` with the common ones, and returns a
-serialisable structure keyed by ``item_cmd`` — the YAML key the user
-writes (e.g. ``sleep``, ``py_func``, ``dialog_message``).
-
-Items intentionally without ``PARAMS`` (the unstructured-body classes
-like console ``write``/``writeln`` or plot ``add``/``export``) are
-emitted as ``"params_declared": false`` so consumers know to suggest
-nothing for them rather than reporting a closed empty set.
-
-Action items (children of ``parallel``, ``console``, ``json_rpc``,
-``plot``) are registered separately under each parent's ``actions``
-entry — they're not top-level YAML keys, they live nested inside a
-parent's ``steps:``.
-"""
-
-import json
-
-from interpreter.utils.constants import TestItemType
-from interpreter.utils.test_init import _constants_init
-
-
-# Action class -> parent cmd (the action's parent in the YAML). Action classes
-# aren't first-class TestItemType entries (TYPE_*_ACTION is one generic bucket),
-# so we resolve their YAML key from the parent's declarative ``ACTIONS`` map.
-def _collect_action_classes(parent_class):
-    """Return {action_yaml_key: action_class} for a TestItemActions parent.
-
-    Each parent declares its actions as a class-level ``ACTIONS = {key: class}``
-    attribute (see ``item_actions/TestItemActions``). We read it directly — no
-    instantiation, no source inspection — so this works identically whether the
-    package runs from source, a wheel, or a frozen (PyInstaller) build where the
-    ``.py`` source isn't on disk.
-    """
-    return dict(getattr(parent_class, "ACTIONS", None) or {})
-
-
-def _params_to_schema(item_class, common_params):
-    """Return the params-portion of an item's schema entry.
-
-    Common params are flagged so consumers can render them differently
-    (an editor might show "common" parameters in a separate group).
-    """
-    own = getattr(item_class, "PARAMS", None)
-    if own is None:
-        return {"params_declared": False}
-    common_names = set(common_params.names())
-    params = []
-    for p in common_params:
-        d = p.to_schema()
-        d["common"] = True
-        params.append(d)
-    for p in own:
-        if p.name in common_names:
-            # Subclass overrode a common param (e.g. tightened doc).
-            for d in params:
-                if d["name"] == p.name:
-                    d.update(p.to_schema())
-                    break
-            continue
-        d = p.to_schema()
-        d["common"] = False
-        params.append(d)
-    return {"params_declared": True, "params": params}
-
-
-def dump_all_schemas():
-    """Return the full schema as a Python dict ready for json.dumps.
-
-    Shape:
-        {
-          "items": {
-            "sleep": {
-              "display_name": "Sleep",
-              "params_declared": true,
-              "params": [{name, kind, required, default?, doc, common}, ...],
-            },
-            "console": {
-              ...,
-              "actions": {"open": {...}, "close": {...}, ...},
-            },
-            ...
-          }
-        }
-    """
-    _constants_init()
-    # Imported lazily — pulls test_item.py which references constants.
-    from interpreter.test_items.test_item import COMMON_PARAMS
-
-    out = {"items": {}}
-    for tp in TestItemType:
-        cls = getattr(tp, "item_class", None)
-        if cls is None:
-            continue
-        # Action types (CONSOLE_ACTION, GRAPH_ACTION, JSON_RPC_ACTION) have no
-        # standalone YAML representation — skip them here, they show up under
-        # their parent's "actions" key.
-        cmd = tp.item_cmd
-        if cmd.endswith("_action"):
-            continue
-        entry = {"display_name": tp.item_name}
-        entry.update(_params_to_schema(cls, COMMON_PARAMS))
-
-        actions = _collect_action_classes(cls)
-        if actions:
-            entry["actions"] = {
-                name: _params_to_schema(acls, COMMON_PARAMS)
-                for name, acls in actions.items()
-            }
-            for name in entry["actions"]:
-                entry["actions"][name]["display_name"] = name
-
-        out["items"][cmd] = entry
-    return out
-
-
-def dump_all_schemas_json(indent=2):
-    """Same as ``dump_all_schemas`` but serialised to a JSON string."""
-    return json.dumps(dump_all_schemas(), indent=indent, sort_keys=False,
-                      default=str)

src/testium/lsp/server.py

@@ -1,313 +0,0 @@
-"""LSP server for ``.tum`` files.
-
-Features available so far:
-
-- **Completion** — when the user starts a new YAML step (``- <cursor>``),
-  the server proposes the full list of known item types. The completion
-  item carries a short hover-style description listing required and
-  optional parameters.
-- **Hover** — over a known item-type word (``sleep``, ``py_func``, …)
-  the server renders the same description in a popup.
-- **Document symbols (outline)** — every ``- <type>:`` line becomes an
-  entry in the editor's outline view. Nesting follows YAML indentation,
-  so containers (``group``, ``loop``, ``parallel``, ``console`` …)
-  display their children as a subtree.
-
-The server speaks LSP over stdio. Start it with::
-
-    testium lsp
-
-Editors invoke it through their LSP client; the connection layer
-(``vscode-languageclient``, ``nvim-lspconfig``, ``lsp-mode``, …) takes
-care of the JSON-RPC framing.
-
-Architecture notes
-------------------
-
-The schema is built once at server start (``dump_all_schemas()``) and
-kept in memory; an editor restart picks up upstream changes. The schema
-is the **only** source of truth — when testium adds a new item type or
-parameter, the LSP automatically exposes it without any change here.
-
-The current handlers stay deliberately heuristic on the parser side:
-completion uses a line-prefix regex, outline a per-line ``- <known>:``
-sweep with indentation tracking. A proper YAML+Jinja parsing pass is
-still pending and is the prerequisite for *parameter*-level completion
-and diagnostics.
-"""
-
-import re
-
-try:
-    # pygls 2.x moved LanguageServer under pygls.lsp.server. We pin >=1.3 in
-    # the optional dependency to stay open to either family, but the import
-    # path differs — try the new one first, then the legacy one.
-    try:
-        from pygls.lsp.server import LanguageServer
-    except ImportError:
-        from pygls.server import LanguageServer  # pygls < 2
-    from lsprotocol.types import (
-        TEXT_DOCUMENT_COMPLETION,
-        TEXT_DOCUMENT_DOCUMENT_SYMBOL,
-        TEXT_DOCUMENT_HOVER,
-        CompletionItem,
-        CompletionItemKind,
-        CompletionList,
-        CompletionOptions,
-        CompletionParams,
-        DocumentSymbol,
-        DocumentSymbolParams,
-        Hover,
-        HoverParams,
-        InsertTextFormat,
-        MarkupContent,
-        MarkupKind,
-        Position,
-        Range,
-        SymbolKind,
-    )
-except ImportError as exc:
-    # Surfaced by the CLI dispatcher with a friendly install hint.
-    raise
-
-
-from lsp.schema import dump_all_schemas
-
-
-_LINE_START_STEP = re.compile(r"^\s*-\s*([A-Za-z_][A-Za-z0-9_]*)?\s*:?\s*$")
-
-# Matches "- <identifier>:" for outline / hover purposes. Captures the start
-# column of the identifier and the identifier itself. Trailing tokens after
-# the colon (inline-form params, comments) are tolerated.
-_STEP_LINE = re.compile(r"^(?P<lead>\s*-\s*)(?P<ident>[A-Za-z_][A-Za-z0-9_]*)\s*:")
-
-# Matches a ``name: <value>`` line under an item — used by the outline pass
-# to surface the user's display name next to the item type.
-_NAME_FIELD = re.compile(r"^\s*name\s*:\s*(?P<value>.+?)\s*$")
-
-# Word boundary used by hover to extract the identifier under the cursor.
-_IDENT_AT = re.compile(r"[A-Za-z_][A-Za-z0-9_]*")
-
-
-def _render_item_markdown(cmd, entry):
-    """Render an item-type's schema entry as a Markdown hover string.
-
-    Reused by both the completion-item documentation and the hover
-    handler so the editor presents identical information regardless of
-    how the user reached it.
-    """
-    detail = entry.get("display_name", cmd)
-    lines = [f"**{cmd}** — {detail}", ""]
-    if entry.get("params_declared"):
-        non_common = [p for p in entry["params"] if not p["common"]]
-        required = [p for p in non_common if p["required"]]
-        optional = [p for p in non_common if not p["required"]]
-        if required:
-            lines.append("Required parameters:")
-            for p in required:
-                lines.append(f"- `{p['name']}` — {p['doc']}")
-            lines.append("")
-        if optional:
-            lines.append("Optional parameters:")
-            for p in optional:
-                lines.append(f"- `{p['name']}` — {p['doc']}")
-    else:
-        lines.append("(Parameter list is not described — this item's body is the "
-                     "raw user value.)")
-    return "\n".join(lines)
-
-
-def _build_item_completions(schema):
-    """Return a list of CompletionItem covering every top-level item type.
-
-    Each completion inserts ``<name>:`` with the cursor positioned after
-    the colon so the user can immediately start typing parameters.
-    """
-    items = []
-    for cmd, entry in schema["items"].items():
-        if cmd == "default":
-            # Root sentinel; never appears as a YAML key.
-            continue
-        items.append(
-            CompletionItem(
-                label=cmd,
-                kind=CompletionItemKind.Class,
-                detail=entry.get("display_name", cmd),
-                documentation=MarkupContent(
-                    kind=MarkupKind.Markdown,
-                    value=_render_item_markdown(cmd, entry),
-                ),
-                insert_text=f"{cmd}:",
-                insert_text_format=InsertTextFormat.PlainText,
-            )
-        )
-    items.sort(key=lambda it: it.label)
-    return items
-
-
-def _word_at(line, character):
-    """Return ``(start, end, text)`` of the identifier under ``character``.
-
-    Returns ``None`` when the cursor isn't on a word. Used by hover.
-    """
-    for m in _IDENT_AT.finditer(line):
-        if m.start() <= character <= m.end():
-            return m.start(), m.end(), m.group(0)
-    return None
-
-
-def _build_document_symbols(lines, item_cmds):
-    """Walk ``lines`` and produce a nested ``DocumentSymbol`` tree.
-
-    Heuristics (no YAML parsing yet):
-    - Each ``- <known_cmd>:`` line becomes a symbol.
-    - Nesting follows the indentation of the leading ``-``: a deeper-
-      indented step is treated as a child of the most recent shallower
-      step.
-    - The symbol's ``detail`` is the ``name: <value>`` field if found
-      within a small window after the step header (no YAML parsing —
-      we just look at indented lines that aren't another ``- …`` step).
-
-    The result is suitable for the LSP outline panel even when the
-    surrounding YAML is mid-edit and structurally invalid.
-    """
-    root_children = []
-    # Each stack entry: (indent_col, children_list_to_append_to,
-    #                    pending_parent_symbol or None).
-    stack = [(-1, root_children, None)]
-
-    def _attach_name(parent_symbol, start_line):
-        """Look for the nearest ``name:`` field in the children of ``parent``."""
-        if parent_symbol is None or start_line + 1 >= len(lines):
-            return
-        base_indent = len(lines[start_line]) - len(lines[start_line].lstrip(" "))
-        for j in range(start_line + 1, min(start_line + 10, len(lines))):
-            l = lines[j]
-            stripped = l.lstrip(" ")
-            indent = len(l) - len(stripped)
-            if indent <= base_indent and stripped.strip() != "":
-                break
-            m = _NAME_FIELD.match(l)
-            if m:
-                value = m.group("value").strip("\"' ")
-                parent_symbol.detail = value
-                return
-
-    for i, raw_line in enumerate(lines):
-        m = _STEP_LINE.match(raw_line)
-        if not m:
-            continue
-        cmd = m.group("ident")
-        if cmd not in item_cmds:
-            continue
-        indent = len(m.group("lead")) - len(m.group("lead").lstrip(" "))
-        # Pop the stack until we find a parent with strictly smaller indent.
-        while stack and stack[-1][0] >= indent:
-            stack.pop()
-        if not stack:
-            stack.append((-1, root_children, None))
-        parent_children = stack[-1][1]
-
-        ident_start = m.start("ident")
-        ident_end = m.end("ident")
-        symbol = DocumentSymbol(
-            name=cmd,
-            detail=None,
-            kind=SymbolKind.Function,
-            range=Range(
-                start=Position(line=i, character=0),
-                end=Position(line=i, character=len(raw_line.rstrip("\n"))),
-            ),
-            selection_range=Range(
-                start=Position(line=i, character=ident_start),
-                end=Position(line=i, character=ident_end),
-            ),
-            children=[],
-        )
-        parent_children.append(symbol)
-        stack.append((indent, symbol.children, symbol))
-        _attach_name(symbol, i)
-    return root_children
-
-
-def _make_server():
-    server = LanguageServer("testium-lsp", "0.1.0")
-    schema = dump_all_schemas()
-    item_completions = _build_item_completions(schema)
-    # Set of cmd names accepted by the outline / hover passes. We include
-    # action names (console open/close/…, plot open/close/…, …) too so they
-    # appear in the outline tree and respond to hover.
-    item_cmds = set()
-    for cmd, entry in schema["items"].items():
-        if cmd == "default":
-            continue
-        item_cmds.add(cmd)
-        item_cmds.update(entry.get("actions", {}).keys())
-
-    @server.feature(
-        TEXT_DOCUMENT_COMPLETION,
-        CompletionOptions(trigger_characters=["-", " "]),
-    )
-    def completion(params: CompletionParams):
-        doc = server.workspace.get_text_document(params.text_document.uri)
-        line_idx = params.position.line
-        if line_idx >= len(doc.lines):
-            return CompletionList(is_incomplete=False, items=[])
-        line = doc.lines[line_idx]
-        # Only look at what's left of the cursor.
-        prefix = line[: params.position.character]
-        if not _LINE_START_STEP.match(prefix):
-            return CompletionList(is_incomplete=False, items=[])
-        return CompletionList(is_incomplete=False, items=item_completions)
-
-    @server.feature(TEXT_DOCUMENT_HOVER)
-    def hover(params: HoverParams):
-        doc = server.workspace.get_text_document(params.text_document.uri)
-        line_idx = params.position.line
-        if line_idx >= len(doc.lines):
-            return None
-        line = doc.lines[line_idx]
-        # Only respond when the cursor is on the type part of a step line
-        # ("- sleep:") — never for arbitrary words in a string.
-        step_match = _STEP_LINE.match(line)
-        if not step_match:
-            return None
-        word = _word_at(line, params.position.character)
-        if word is None:
-            return None
-        start, end, text = word
-        if text != step_match.group("ident") or text not in item_cmds:
-            return None
-        # Resolve the entry: top-level item, or action of any parent.
-        entry = schema["items"].get(text)
-        if entry is None:
-            for parent_entry in schema["items"].values():
-                actions = parent_entry.get("actions") or {}
-                if text in actions:
-                    entry = actions[text]
-                    break
-        if entry is None:
-            return None
-        return Hover(
-            contents=MarkupContent(
-                kind=MarkupKind.Markdown,
-                value=_render_item_markdown(text, entry),
-            ),
-            range=Range(
-                start=Position(line=line_idx, character=start),
-                end=Position(line=line_idx, character=end),
-            ),
-        )
-
-    @server.feature(TEXT_DOCUMENT_DOCUMENT_SYMBOL)
-    def document_symbols(params: DocumentSymbolParams):
-        doc = server.workspace.get_text_document(params.text_document.uri)
-        return _build_document_symbols(doc.lines, item_cmds)
-
-    return server
-
-
-def serve():
-    """Start the LSP server on stdio. Blocks until the client disconnects."""
-    server = _make_server()
-    server.start_io()

src/testium/main_win/file_dialog.py

@@ -1,21 +0,0 @@
-"""Helpers for Qt file/directory dialogs.
-
-In Flatpak the native QFileDialog goes through the XDG document portal,
-which returns ``/run/user/UID/doc/.../<file>`` and only exposes the
-selected file — sibling files (param.yaml, scripts, recent paths in
-preferences, ...) are unreachable. Forcing Qt's own non-native dialog
-makes it walk the real filesystem mounted via ``--filesystem=home``
-and return a regular path.
-"""
-
-import os
-
-from PySide6.QtWidgets import QFileDialog
-
-
-def options():
-    """Default ``QFileDialog`` options for the current runtime."""
-    opts = QFileDialog.Options()
-    if os.path.isfile("/.flatpak-info"):
-        opts |= QFileDialog.Option.DontUseNativeDialog
-    return opts

src/testium/main_win/preference_win/preference_win.py

@@ -3,7 +3,6 @@ from PySide6.QtWidgets import QDialog, QFileDialog
 from PySide6.QtGui import QFont
 
 from main_win.preference_win.preference_core_win import Ui_preferenceWindow
-from main_win import file_dialog
 
 import interpreter.utils.settings as prefs
 
@@ -194,7 +193,6 @@ class PrefWindow(QDialog):
             self,
             caption="Select the default report directory",
             dir=self.ui.editDefaultReportPath.text(),
-            options=file_dialog.options(),
         )
         if path:
             self.ui.editDefaultReportPath.setText(path)
@@ -205,7 +203,6 @@ class PrefWindow(QDialog):
             self,
             caption="Select the default log directory",
             dir=self.ui.editDefaultLogPath.text(),
-            options=file_dialog.options(),
         )
         if path:
             self.ui.editDefaultLogPath.setText(path)
@@ -216,7 +213,6 @@ class PrefWindow(QDialog):
             self,
             caption="Select the python interpreter",
             dir=self.ui.editPythonPath.text(),
-            options=file_dialog.options(),
         )
         if path:
             self.ui.editPythonPath.setText(path)
@@ -224,10 +220,7 @@ class PrefWindow(QDialog):
     @Slot()
     def on_butLuaPath_pressed(self):
         path, _ = QFileDialog.getOpenFileName(
-            self,
-            caption="Select the lua interpreter",
-            dir=self.ui.editLuaPath.text(),
-            options=file_dialog.options(),
+            self, caption="Select the lua interpreter", dir=self.ui.editLuaPath.text()
         )
         if path:
             self.ui.editLuaPath.setText(path)

src/testium/main_win/test_file_manager.py

@@ -9,7 +9,6 @@ from PySide6.QtWidgets import QApplication, QFileDialog, QProgressDialog
 from interpreter.process import TestProcess
 from interpreter.utils.test_ctrl import TestSetController
 from main_win.test_controller_service import TestControllerService
-from main_win import file_dialog
 import interpreter.utils.settings as prefs
 from runtime.tum_except import ETUMFileError, ETUMRuntimeError
 
@@ -213,9 +212,17 @@ class TestFileManager:
         d = ""
         if w.testFile is not None:
             d = os.path.dirname(w.testFile)
+        # In Flatpak the native dialog goes through the XDG document portal,
+        # which returns /run/user/UID/doc/.../test.tum and only exposes the
+        # selected file — sibling files (param.yaml, .py, etc.) are unreachable.
+        # Force Qt's own dialog, which walks the real filesystem mounted via
+        # --filesystem=home and returns a regular path with sibling access.
+        options = QFileDialog.Options()
+        if os.path.isfile("/.flatpak-info"):
+            options |= QFileDialog.Option.DontUseNativeDialog
         file_name, _ = QFileDialog.getOpenFileName(
             w, "Open the test file", d,
-            "testium file (*.tum);;All Files (*)", options=file_dialog.options()
+            "testium file (*.tum);;All Files (*)", options=options
         )
         if file_name:
             self.reload(file_name)

src/testium/main_win/testium_win.py

@@ -37,7 +37,6 @@ from interpreter.utils.icons import icon_prefix
 
 from main_win.test_run.outlog import OutLog
 from main_win.test_run.test_run import ThreadTestStatus
-from main_win import file_dialog
 import interpreter.utils.settings as prefs
 from runtime.stdout_redirect import stdio_redir
 import api.testium as tm
@@ -485,8 +484,7 @@ class MainWindow(QMainWindow, Ui_MainWindow):
         else:
             initialPath = None
         fileName, _ = QFileDialog.getSaveFileName(
-            self, "Path to Log file", initialPath, "Log Files (*.log);;All Files (*)",
-            options=file_dialog.options(),
+            self, "Path to Log file", initialPath, "Log Files (*.log);;All Files (*)"
         )
         if fileName:
             shutil.copy(self.logFileName, fileName)
@@ -527,8 +525,7 @@ class MainWindow(QMainWindow, Ui_MainWindow):
         else:
             initialPath = None
         fileName, _ = QFileDialog.getSaveFileName(
-            self, "Path to log file", initialPath, "Log Files (*.log);;All Files (*)",
-            options=file_dialog.options(),
+            self, "Path to log file", initialPath, "Log Files (*.log);;All Files (*)"
         )
         if fileName:
             self.editLogFilePath.setText(fileName)

test/validation/README.md

@@ -1,76 +1,10 @@
 # Validation
 
-This directory contains the testium validation suite. A single set of
-items (`items/`), fixtures and post-processing (`post_execution.py`) is
-re-used across every packaging channel.
+This directory contains the necessary material to run the testium validation.
 
-## Running the suite
+Here is the documentation on how to configure the validation, run it and check that the
+results are correct.
 
-```sh
-./test/validation/run.sh                       # default mode = source
-./test/validation/run.sh --mode wheel
-./test/validation/run.sh --mode pyinstaller
-./test/validation/run.sh --mode flatpak
-./test/validation/run.sh --mode appimage
-```
+# Tests
 
-On Windows (only `source`, `wheel`, `pyinstaller` are supported):
-
-```bat
-test\validation\run.bat --mode pyinstaller
-```
-
-Pass `clean` as the **first** argument to recreate the validation venv
-from scratch (useful after a system Python upgrade):
-
-```sh
-./test/validation/run.sh clean --mode flatpak
-```
-
-Any extra arguments after the mode flag are forwarded to testium.
-
-## Modes
-
-| Mode          | What it launches                                            | Prerequisite                                                     |
-|---------------|-------------------------------------------------------------|------------------------------------------------------------------|
-| `source`      | `python3 src/testium` via the project's `run.sh`            | none — works straight out of the repo                            |
-| `wheel`       | `python -m testium` inside a dedicated wheel venv           | `./build_all.sh` produced `dist/testium-<v>-py3-none-any.whl`    |
-| `pyinstaller` | `dist/testium-<v>` (frozen binary)                          | `./build_all.sh` produced the PyInstaller binary                 |
-| `flatpak`     | `flatpak run --command=testium org.testium.Testium`         | the Flatpak bundle is installed (`flatpak install --user dist/testium-<v>.flatpak`) |
-| `appimage`    | `dist/Testium-<v>-x86_64.AppImage`                          | `./build_all.sh` produced the AppImage                           |
-
-Each mode writes its results to a distinct report file
-(`validation-<mode>.sqlite` / `validation-<mode>-<item>.xml`), so you
-can run several modes in a row without clobbering previous reports.
-
-## How `python_bin` is pinned
-
-Every test-execution subprocess (inline `<| ... |>` evaluation,
-`py_func`, `cycle`, `post_execution`, …) is routed through a dedicated
-venv at `${TMPDIR:-/tmp}/testium-validation-venv`. The venv is created
-with `--system-site-packages` so existing system packages stay visible,
-then `junit-xml` is pip-installed for `post_execution.py`.
-
-This is a **host** venv. In every mode (including Flatpak) the
-test-execution subprocesses end up running on the host — directly for
-source/wheel/pyinstaller/appimage, and via `flatpak-spawn --host` for
-Flatpak — so the same venv works across modes. The wheel mode
-additionally creates a separate `testium-wheel-venv-<v>` to hold the
-installed wheel; that one is only used to launch testium itself.
-
-## What is checked
-
-The `venv` item under `items/venv/` asserts that the validation venv is
-actually being used:
-
-* `python_bin` is set in the global dict.
-* The eval subprocess (used for `<| ... |>` expressions) has
-  `sys.executable == python_bin`, `sys.prefix == dirname(dirname(python_bin))`,
-  and `sys.prefix != sys.base_prefix` (i.e. is actually inside a venv).
-* A `py_func` subprocess passes the same three checks.
-
-These checks use `abspath`/`normpath` rather than `realpath` on
-purpose: the venv's `bin/python3` is a symlink to the host interpreter,
-so `realpath` would map both venv and non-venv interpreters to the same
-target. `sys.prefix != sys.base_prefix` is the venv-specific marker
-that distinguishes the two cases.
+TBD

test/validation/items/venv/param.yaml

@@ -1 +0,0 @@
-no_param: Null

test/validation/items/venv/test.tum

@@ -1,53 +0,0 @@
-# venv test: assert that the dedicated validation venv is the python
-# being used for every test-execution subprocess (eval_proc / py_func /
-# cycle / ...). The venv path is pinned by ``-d python_bin=...`` in
-# test/validation/run.sh (or run.bat).
-#
-# We use ``abspath``/``normpath`` rather than ``realpath`` on purpose:
-# the venv's ``bin/python3`` is a symlink to the host python, so
-# realpath would map every venv interpreter to the same system path and
-# the comparison would silently pass even *without* a venv.
-# ``sys.prefix != sys.base_prefix`` is the venv-specific marker that
-# catches that case.
-
-- check:
-    name: python_bin is set in the global dict
-    key: $(test)_PASS
-    values:
-        - <| bool(r"$(python_bin)") |>
-
-- check:
-    name: eval_proc subprocess runs in the validation venv
-    key: $(test)_PASS
-    values:
-        - <| os.path.normpath(os.path.abspath(sys.executable)) == os.path.normpath(os.path.abspath(r"$(python_bin)")) |>
-
-- check:
-    name: eval_proc sys.prefix matches python_bin venv root
-    key: $(test)_PASS
-    values:
-        - <| os.path.normpath(os.path.abspath(sys.prefix)) == os.path.dirname(os.path.dirname(os.path.normpath(os.path.abspath(r"$(python_bin)")))) |>
-
-- check:
-    name: eval_proc is actually inside a venv (sys.prefix != sys.base_prefix)
-    key: $(test)_PASS
-    values:
-        - <| os.path.normpath(os.path.abspath(sys.prefix)) != os.path.normpath(os.path.abspath(sys.base_prefix)) |>
-
-- py_func:
-    name: py_func subprocess runs in the validation venv
-    key: $(test)_PASS
-    file: $(test_path)$(psep)verify_venv.py
-    func_name: check_sys_executable
-
-- py_func:
-    name: py_func sys.prefix matches python_bin venv root
-    key: $(test)_PASS
-    file: $(test_path)$(psep)verify_venv.py
-    func_name: check_sys_prefix_in_venv
-
-- py_func:
-    name: py_func is actually inside a venv
-    key: $(test)_PASS
-    file: $(test_path)$(psep)verify_venv.py
-    func_name: check_is_venv

test/validation/items/venv/verify_venv.py

@@ -1,62 +0,0 @@
-import os
-import sys
-
-import py_func.tm as tm
-
-
-def _norm(p):
-    # normpath + normcase, without resolving symlinks. realpath() would
-    # follow the venv's ``python3`` symlink to ``/usr/bin/python3.X`` and
-    # defeat the comparison.
-    return os.path.normcase(os.path.normpath(os.path.abspath(p)))
-
-
-def _venv_dir():
-    # python_bin is at ``<venv>/(bin|Scripts)/python*`` so the venv root
-    # is two levels above the executable.
-    exe = tm.gd("python_bin", "")
-    if not exe:
-        return ""
-    return os.path.dirname(os.path.dirname(_norm(exe)))
-
-
-def check_sys_executable():
-    """py_func subprocess: sys.executable must match the configured python_bin."""
-    expected = _norm(tm.gd("python_bin", ""))
-    actual = _norm(sys.executable)
-    if expected and actual == expected:
-        return True
-    return (
-        -1,
-        f"sys.executable={actual!r} differs from python_bin={expected!r}",
-    )
-
-
-def check_sys_prefix_in_venv():
-    """py_func subprocess: sys.prefix must match the venv root derived
-    from python_bin (two levels up from the executable)."""
-    venv = _venv_dir()
-    if not venv:
-        return (-1, "python_bin is not set in the global dict")
-    actual = _norm(sys.prefix)
-    if actual == venv:
-        return True
-    return (
-        -1,
-        f"sys.prefix={actual!r} is not the validation venv {venv!r}",
-    )
-
-
-def check_is_venv():
-    """py_func subprocess: confirm we are inside a venv, i.e. sys.prefix
-    differs from sys.base_prefix. This catches the case where python_bin
-    happens to be a system interpreter and the path-based check would
-    pass trivially."""
-    actual = _norm(sys.prefix)
-    base = _norm(sys.base_prefix)
-    if actual != base:
-        return True
-    return (
-        -1,
-        f"sys.prefix == sys.base_prefix == {actual!r}: not running in a venv",
-    )

test/validation/lsp_check.py

@@ -1,102 +0,0 @@
-#!/usr/bin/env python3
-"""Per-channel check of the testium language server.
-
-Given the channel's testium invocation as argv (e.g. ``flatpak run
---command=testium org.testium.Testium``, a PyInstaller binary path, or
-``python -m testium``), verify two things end-to-end against that exact build:
-
-  1. ``<cmd> schema`` produces valid JSON whose item registry still includes the
-     nested action sets (console/plot/json_rpc). This catches a frozen build
-     that lost the actions — the failure mode the declarative ``ACTIONS``
-     refactor fixed (no more ``inspect.getsource`` at runtime).
-  2. ``<cmd> lsp`` starts a real language server: it must answer an LSP
-     ``initialize`` request with a capabilities result and must NOT report the
-     pygls dependency as missing. This catches a channel that forgot to bundle
-     the ``[lsp]`` extra.
-
-Exits non-zero (with a diagnostic) on the first failure so the validation run
-fails loudly. Used by ``run.sh`` before launching the main suite.
-"""
-import json
-import subprocess
-import sys
-
-EXPECTED_ACTION_PARENTS = ("console", "plot", "json_rpc")
-
-
-def fail(msg):
-    print(f"LSP CHECK: FAIL — {msg}", file=sys.stderr)
-    sys.exit(1)
-
-
-def _extract_json(raw):
-    """Parse JSON from ``raw`` bytes, tolerating leading non-JSON noise.
-
-    The source-mode launcher (run.sh) may print env-setup lines before the
-    schema JSON, so we fall back to parsing from the first ``{``.
-    """
-    try:
-        return json.loads(raw)
-    except json.JSONDecodeError:
-        start = raw.find(b"{")
-        if start < 0:
-            raise
-        return json.loads(raw[start:])
-
-
-def check_schema(cmd):
-    try:
-        out = subprocess.run(cmd + ["schema"], capture_output=True, timeout=120)
-    except Exception as e:  # noqa: BLE001
-        fail(f"`{' '.join(cmd)} schema` could not run: {e}")
-    if out.returncode != 0:
-        fail(f"`schema` exited {out.returncode}: {out.stderr.decode()[:300]}")
-    try:
-        data = _extract_json(out.stdout)
-    except json.JSONDecodeError as e:
-        fail(f"`schema` output is not valid JSON: {e}")
-    items = data.get("items", {})
-    for parent in EXPECTED_ACTION_PARENTS:
-        actions = (items.get(parent) or {}).get("actions") or {}
-        if not actions:
-            fail(f"schema item '{parent}' has no actions — a frozen build lost "
-                 f"the declarative ACTIONS (item keys: {sorted(items)[:8]}…)")
-    print(f"LSP CHECK: schema OK ({len(items)} items; actions present for "
-          f"{', '.join(EXPECTED_ACTION_PARENTS)})")
-
-
-def check_lsp(cmd):
-    body = json.dumps({
-        "jsonrpc": "2.0", "id": 1, "method": "initialize",
-        "params": {"processId": None, "rootUri": None, "capabilities": {}},
-    }).encode()
-    msg = b"Content-Length: %d\r\n\r\n%s" % (len(body), body)
-    try:
-        out = subprocess.run(cmd + ["lsp"], input=msg,
-                             capture_output=True, timeout=30)
-        stdout, stderr = out.stdout, out.stderr
-    except subprocess.TimeoutExpired as e:
-        # A server that stays alive past initialize is fine — it just never saw
-        # a shutdown. Use whatever it wrote so far as the response.
-        stdout, stderr = e.stdout or b"", e.stderr or b""
-    blob = stdout + stderr
-    if b"dependencies missing" in blob:
-        fail("`lsp` reports the pygls dependency missing — this channel did "
-             "not bundle the [lsp] extra.")
-    if b'"capabilities"' not in stdout:
-        fail("`lsp` did not return an initialize result. "
-             f"stdout[:200]={stdout[:200]!r} stderr[:200]={stderr[:200]!r}")
-    print("LSP CHECK: lsp initialize OK (server answered with capabilities)")
-
-
-def main():
-    cmd = sys.argv[1:]
-    if not cmd:
-        fail("usage: lsp_check.py <testium-invocation...>")
-    check_schema(cmd)
-    check_lsp(cmd)
-    print("LSP CHECK: PASS")
-
-
-if __name__ == "__main__":
-    main()

test/validation/run.bat

@@ -1,131 +0,0 @@
-@echo off
-SETLOCAL EnableExtensions EnableDelayedExpansion
-
-REM Runs the testium validation suite against any installable channel of
-REM testium on Windows (source, wheel, pyinstaller).
-REM
-REM Usage:
-REM   test\validation\run.bat [clean] [--mode MODE] [extra testium args]
-REM
-REM   clean       remove the validation venv before recreating it
-REM               (must be the first argument; useful after a Python upgrade)
-REM
-REM   --mode MODE which testium build to validate. One of:
-REM                   source       (default) project's run.bat (src\testium)
-REM                   wheel        dist\testium-<v>-py3-none-any.whl
-REM                   pyinstaller  dist\testium-<v>.exe (or dist\testium-<v>)
-REM
-REM Every test-execution subprocess runs in a dedicated host venv under
-REM %TEMP%\testium-validation-venv (created with --system-site-packages,
-REM then junit-xml is pip-installed for post_execution.py).
-REM
-REM The report file is suffixed with the mode so consecutive runs in
-REM different modes don't overwrite each other.
-
-SET "SCRIPT_DIR=%~dp0"
-IF "%SCRIPT_DIR:~-1%"=="\" SET "SCRIPT_DIR=%SCRIPT_DIR:~0,-1%"
-SET "PROJECT_DIR=%SCRIPT_DIR%\..\.."
-SET /P VERSION=<"%PROJECT_DIR%\src\VERSION"
-
-REM ---------- arg parsing ----------------------------------------------------
-
-SET "MODE=source"
-SET "CLEAN=0"
-IF /I "%~1"=="clean" (
-    SET "CLEAN=1"
-    SHIFT
-)
-
-SET "EXTRA="
-:PARSE_ARGS
-IF "%~1"=="" GOTO ARGS_DONE
-IF /I "%~1"=="--mode" (
-    SET "MODE=%~2"
-    SHIFT
-    SHIFT
-    GOTO PARSE_ARGS
-)
-SET "EXTRA=!EXTRA! "%~1""
-SHIFT
-GOTO PARSE_ARGS
-:ARGS_DONE
-
-REM ---------- locate host python ---------------------------------------------
-
-SET "PYTHON_EXE="
-py --version >nul 2>&1
-IF %ERRORLEVEL% EQU 0 (
-    SET "PYTHON_EXE=py"
-    GOTO PYTHON_FOUND
-)
-python --version >nul 2>&1
-IF %ERRORLEVEL% EQU 0 (
-    SET "PYTHON_EXE=python"
-    GOTO PYTHON_FOUND
-)
-echo ERROR: Python could not be found on this system.
-exit /b 1
-:PYTHON_FOUND
-
-REM ---------- validation venv -------------------------------------------------
-
-SET "VENV_DIR=%TEMP%\testium-validation-venv"
-IF "%CLEAN%"=="1" IF EXIST "%VENV_DIR%" rmdir /s /q "%VENV_DIR%"
-
-IF NOT EXIST "%VENV_DIR%" (
-    echo Creating validation venv at %VENV_DIR%
-    %PYTHON_EXE% -m venv --system-site-packages "%VENV_DIR%"
-    IF !ERRORLEVEL! NEQ 0 (
-        echo ERROR while creating the validation venv.
-        exit /b 1
-    )
-    call "%VENV_DIR%\Scripts\pip" install --quiet --upgrade pip
-    call "%VENV_DIR%\Scripts\pip" install --quiet junit-xml
-)
-SET "VENV_PYTHON=%VENV_DIR%\Scripts\python.exe"
-
-REM ---------- shared "tail" forwarded to every launcher -----------------------
-REM Reports are stamped with the mode so successive runs don't clobber each other.
-
-SET "TAIL=-b -d "python_bin=%VENV_PYTHON%" -d "validation_report_file=validation-%MODE%" -- "%SCRIPT_DIR%\main.tum"%EXTRA%"
-
-REM ---------- per-mode launcher ----------------------------------------------
-
-echo -- validation mode: %MODE%
-
-IF /I "%MODE%"=="source"      GOTO MODE_SOURCE
-IF /I "%MODE%"=="wheel"       GOTO MODE_WHEEL
-IF /I "%MODE%"=="pyinstaller" GOTO MODE_PYI
-echo ERROR: unknown --mode '%MODE%'. Expected: source ^| wheel ^| pyinstaller.
-exit /b 1
-
-:MODE_SOURCE
-call "%PROJECT_DIR%\run.bat" %TAIL%
-exit /b %ERRORLEVEL%
-
-:MODE_WHEEL
-SET "WHEEL=%PROJECT_DIR%\dist\testium-%VERSION%-py3-none-any.whl"
-IF NOT EXIST "%WHEEL%" (
-    echo ERROR: wheel not found at %WHEEL% -- run build_all.sh first.
-    exit /b 1
-)
-SET "WHEEL_VENV=%TEMP%\testium-wheel-venv-%VERSION%"
-IF "%CLEAN%"=="1" IF EXIST "%WHEEL_VENV%" rmdir /s /q "%WHEEL_VENV%"
-IF NOT EXIST "%WHEEL_VENV%" (
-    echo Creating wheel venv at %WHEEL_VENV%
-    %PYTHON_EXE% -m venv --system-site-packages "%WHEEL_VENV%"
-    call "%WHEEL_VENV%\Scripts\pip" install --quiet --upgrade pip
-    call "%WHEEL_VENV%\Scripts\pip" install --quiet "%WHEEL%"
-)
-"%WHEEL_VENV%\Scripts\python.exe" -m testium %TAIL%
-exit /b %ERRORLEVEL%
-
-:MODE_PYI
-SET "PYI_BIN=%PROJECT_DIR%\dist\testium-%VERSION%.exe"
-IF NOT EXIST "%PYI_BIN%" SET "PYI_BIN=%PROJECT_DIR%\dist\testium-%VERSION%"
-IF NOT EXIST "%PYI_BIN%" (
-    echo ERROR: PyInstaller binary not found in %PROJECT_DIR%\dist -- run build_all.sh first.
-    exit /b 1
-)
-"%PYI_BIN%" %TAIL%
-exit /b %ERRORLEVEL%

test/validation/run.sh

@@ -1,153 +0,0 @@
-#!/bin/bash
-# Runs the testium validation suite against any installable channel of
-# testium (source, wheel, pyinstaller, flatpak, appimage).
-#
-# Usage:
-#   ./test/validation/run.sh [clean] [--mode MODE] [extra testium args]
-#
-#   clean           remove the validation venv before recreating it
-#                   (must be the first argument; useful after a Python upgrade)
-#
-#   --mode MODE     which testium build to validate. One of:
-#                       source       (default) src/testium via project run.sh
-#                       wheel        dist/testium-<v>-py3-none-any.whl
-#                       pyinstaller  dist/testium-<v>
-#                       flatpak      installed org.testium.Testium
-#                       appimage     dist/Testium-<v>-*.AppImage
-#
-# Every test-execution subprocess (inline <| ... |>, py_func, cycle,
-# post_execution, ...) runs in a dedicated host venv under
-# /tmp/testium-validation-venv. That venv is shared across modes —
-# even Flatpak reaches it via flatpak-spawn --host. The validation venv
-# is created with --system-site-packages so existing system packages
-# (PySide6, lxml, ...) stay visible, then junit-xml is pip-installed
-# for post_execution.py.
-#
-# The report file is suffixed with the mode (e.g. validation-flatpak.sqlite)
-# so consecutive runs in different modes don't overwrite each other.
-
-set -e
-
-SCRIPT_PATH="$(readlink -f "$0")"
-SCRIPT_DIR="$(realpath "$(dirname "$SCRIPT_PATH")")"
-PROJECT_DIR="$(realpath "$SCRIPT_DIR/../..")"
-VERSION="$(cat "$PROJECT_DIR/src/VERSION")"
-
-# ---------- arg parsing -------------------------------------------------------
-
-MODE=source
-
-if [ "${1:-}" = "clean" ]; then
-    CLEAN=1
-    shift
-else
-    CLEAN=0
-fi
-
-EXTRA=()
-while [ $# -gt 0 ]; do
-    case "$1" in
-        --mode)
-            MODE="$2"
-            shift 2
-            ;;
-        --mode=*)
-            MODE="${1#--mode=}"
-            shift
-            ;;
-        *)
-            EXTRA+=("$1")
-            shift
-            ;;
-    esac
-done
-
-# ---------- validation venv ---------------------------------------------------
-
-VENV_DIR="${TMPDIR:-/tmp}/testium-validation-venv"
-if [ "$CLEAN" -eq 1 ]; then
-    rm -rf "$VENV_DIR"
-fi
-
-if [ ! -d "$VENV_DIR" ]; then
-    echo "Creating validation venv at $VENV_DIR"
-    python3 -m venv --system-site-packages "$VENV_DIR"
-    "$VENV_DIR/bin/pip" install --quiet --upgrade pip
-    "$VENV_DIR/bin/pip" install --quiet junit-xml
-fi
-VENV_PYTHON="$VENV_DIR/bin/python3"
-
-# ---------- per-mode launcher -------------------------------------------------
-
-case "$MODE" in
-    source)
-        CMD=("$PROJECT_DIR/run.sh")
-        ;;
-    wheel)
-        WHEEL="$PROJECT_DIR/dist/testium-${VERSION}-py3-none-any.whl"
-        if [ ! -f "$WHEEL" ]; then
-            echo "ERROR: wheel not found at $WHEEL — run ./build_all.sh first." >&2
-            exit 1
-        fi
-        WHEEL_VENV="${TMPDIR:-/tmp}/testium-wheel-venv-${VERSION}"
-        if [ "$CLEAN" -eq 1 ]; then
-            rm -rf "$WHEEL_VENV"
-        fi
-        if [ ! -d "$WHEEL_VENV" ]; then
-            echo "Creating wheel venv at $WHEEL_VENV"
-            python3 -m venv --system-site-packages "$WHEEL_VENV"
-            "$WHEEL_VENV/bin/pip" install --quiet --upgrade pip
-            # Install with the [lsp] extra so the wheel channel is validated in
-            # its language-server-capable form (pulls pygls), matching how a
-            # user enables `testium lsp` from a wheel: pip install testium[lsp].
-            "$WHEEL_VENV/bin/pip" install --quiet "${WHEEL}[lsp]"
-        fi
-        CMD=("$WHEEL_VENV/bin/python" -m testium)
-        ;;
-    pyinstaller)
-        PYI_BIN="$PROJECT_DIR/dist/testium-${VERSION}"
-        if [ ! -x "$PYI_BIN" ]; then
-            echo "ERROR: PyInstaller binary not found at $PYI_BIN — run ./build_all.sh first." >&2
-            exit 1
-        fi
-        CMD=("$PYI_BIN")
-        ;;
-    flatpak)
-        if ! flatpak info --user org.testium.Testium &>/dev/null \
-           && ! flatpak info --system org.testium.Testium &>/dev/null; then
-            echo "ERROR: org.testium.Testium is not installed." >&2
-            echo "       flatpak install --user $PROJECT_DIR/dist/testium-${VERSION}.flatpak" >&2
-            exit 1
-        fi
-        CMD=(flatpak run --command=testium org.testium.Testium)
-        ;;
-    appimage)
-        APPIMAGE=$(ls -1t "$PROJECT_DIR/dist"/Testium-"${VERSION}"-*.AppImage 2>/dev/null | head -1)
-        if [ -z "$APPIMAGE" ] || [ ! -x "$APPIMAGE" ]; then
-            echo "ERROR: no AppImage for version $VERSION under $PROJECT_DIR/dist — run ./build_all.sh first." >&2
-            exit 1
-        fi
-        CMD=("$APPIMAGE")
-        ;;
-    *)
-        echo "ERROR: unknown --mode '$MODE'. Expected: source|wheel|pyinstaller|flatpak|appimage." >&2
-        exit 1
-        ;;
-esac
-
-# ---------- launch ------------------------------------------------------------
-
-echo "-- validation mode: $MODE"
-echo "-- launch: ${CMD[*]}"
-
-# ---------- LSP check (this exact channel) ------------------------------------
-# Verify `testium lsp` / `testium schema` work in the build under test before
-# running the suite: schema must keep its nested actions (declarative ACTIONS,
-# survives frozen builds) and the language server must start (pygls bundled).
-echo "-- LSP check ($MODE)"
-"$VENV_PYTHON" "$SCRIPT_DIR/lsp_check.py" "${CMD[@]}"
-
-exec "${CMD[@]}" -b \
-    -d "python_bin=$VENV_PYTHON" \
-    -d "validation_report_file=validation-$MODE" \
-    -- "$SCRIPT_DIR/main.tum" "${EXTRA[@]}"