Adding a some TUM template used to validate the JSON schema

Two new steps per language: function returning nothing and function
returning explicit nil/None. Both tagged $(test)_PASS — they would
have failed before the lua nil fix (Lua side reported nil result as
error). Python side already worked but is covered for parity.

.gitignore

@@ -8,6 +8,8 @@ dist
 /.vscode
 .venv/
 .flatpak-builder/
+package/flatpak/repo/
+package/flatpak/*.flatpak
 crash.tx*
 report_test.tx*
 *.autosave
@@ -24,6 +26,7 @@ package/appimage/*.AppImage
 package/appimage/src
 package/appimage/*.py
 AppDir
+*.squashfs
 doc/manual/doxygen
 doc/manual/sphinx/build/*
 doc/manual/sphinx/source/_build/*

CONTRIBUTING.md

@@ -0,0 +1,165 @@
+# Contributing to testium
+
+Thank you for your interest in contributing to testium.
+
+## License of contributions
+
+testium is licensed under the **European Union Public Licence v. 1.2 (EUPL-1.2)** —
+see the [LICENSE](LICENSE) file at the repository root.
+
+By submitting a contribution to this project (pull request, patch, issue
+attachment, or any other form of code, documentation or media), you agree
+that your contribution is licensed to the project and to the public under the
+**same EUPL-1.2** terms (or any later version of the EUPL approved by the
+European Commission), and you certify that:
+
+- you are the author of the contribution, or you have the right to submit it
+  under the EUPL-1.2;
+- to the best of your knowledge, the contribution does not infringe any
+  third-party intellectual-property rights;
+- the contribution may be redistributed by the project under the EUPL-1.2 and
+  any compatible licence listed in the EUPL-1.2 Appendix.
+
+This is the **inbound = outbound** rule: contributions come in under the same
+licence the project ships under.
+
+You retain copyright on your contribution. The project does **not** ask you
+to sign a CLA or assign your copyright.
+
+## SPDX header in new source files
+
+When creating a new source file, please include the following header at the
+top of the file (adjust the comment marker to the file's language):
+
+```python
+# SPDX-License-Identifier: EUPL-1.2
+# Copyright (c) <year> <your name>
+```
+
+For existing files, keep the header that is already there.
+
+## How to contribute
+
+1. Open an issue describing the change you want to make (bug, feature, doc).
+2. Fork the repository, create a topic branch.
+3. Commit with a clear message (one logical change per commit).
+4. Make sure the validation suite still passes:
+   ```
+   ./run.sh -b -- test/validation/main.tum
+   ```
+5. Open a pull request against `main`.
+
+## Coding conventions
+
+- Python ≥ 3.11
+- Follow existing style in the file you are modifying
+- Add or update tests in `test/validation/` for new test items or behaviours
+- Update `CLAUDE.md` and the Sphinx manual for user-visible changes
+
+## Development
+
+### Debugging in VSCode
+
+The recommended workflow:
+
+1. Add a debug configuration to `.vscode/launch.json`:
+   ```json
+   {
+       "configurations": [
+           {
+               "name": "Python : testium",
+               "type": "python",
+               "request": "launch",
+               "program": "${workspaceFolder}/src/testium",
+               "console": "integratedTerminal",
+               "args": ["-g"],
+               "justMyCode": true
+           }
+       ]
+   }
+   ```
+2. Install `debugpy` in the venv: `python -m pip install debugpy`.
+3. Open the *Run and Debug* tab and press play. testium starts; load and
+   run a `.tum` file. Set breakpoints where you want to investigate.
+
+### Qt GUI modification
+
+UI files (`*.ui`) are edited in **Qt Creator**. After editing, regenerate
+the corresponding Python and resource files:
+
+```sh
+scripts/qt_generate.sh
+```
+
+Icons come from <https://github.com/free-icons/free-icons>.
+
+### Sphinx documentation
+
+```sh
+pip install sphinx linuxdoc
+doc/manual/sphinx/build_doc.sh
+```
+
+PDF generation requires `texlive`:
+
+```sh
+sudo apt install texlive-full
+```
+
+### Validation suite
+
+Batch mode (CI-friendly, headless):
+
+```sh
+./run.sh -b -- test/validation/main.tum
+```
+
+GUI mode (loads the suite, click *Run* to execute and inspect the tree):
+
+```sh
+./run.sh test/validation/main.tum
+```
+
+GUI run-and-close (executes the suite, then closes):
+
+```sh
+./run.sh -r -- test/validation/main.tum
+```
+
+Subset run via the `items` define (works in any mode):
+
+```sh
+./run.sh -b -d "items=['parallel','common']" -- test/validation/main.tum
+```
+
+### Cross-distribution check
+
+`package/deb/test_distro.sh` spins up a Docker/Podman container of the
+target image, installs the expected system Python deps via apt (with
+pip fallback for what is missing), installs the testium wheel and runs
+the validation suite end-to-end. Currently green on `debian:bookworm`,
+`debian:trixie`, `ubuntu:24.04`.
+
+```sh
+./package/deb/test_distro.sh debian:trixie
+```
+
+## Release procedure
+
+1. Update `release_note.txt`.
+2. Bump the version in `src/VERSION`.
+3. Make sure the documentation is up to date — rebuild with
+   `doc/manual/sphinx/build_doc.sh` if needed.
+4. Push and tag the commit with the new version.
+5. Build the binary release: `package/pyinstaller/build.sh`.
+6. Run the validation suite against each generated binary.
+7. Confirm all validation results are green before publishing.
+
+## Reporting security issues
+
+Please do **not** report security vulnerabilities through public GitHub
+issues. Instead, send an email to the project maintainer directly.
+
+## Questions
+
+Open a GitHub Discussion or an issue tagged `question`.

DESIGN.md

@@ -0,0 +1,273 @@
+# Testium — Design Context
+
+## What is testium
+
+Testium is a test sequencer/runner written in Python. It executes YAML-based test scripts ("`.tum`" files) and supports two execution modes:
+
+- **GUI mode** (default, no flag): PySide6 Qt application (`src/testium/main_win/`)
+- **Batch mode** (`-b` / `--batch-execution`): headless, non-interactive, runs tests and exits
+
+Run from repo root: `./run.sh` (Linux) or `run.bat` / `run.ps1` (Windows).  
+Direct invocation: `python3 -m src/testium [-b] <test_file.tum>`
+
+## Architecture
+
+### Entry point
+`src/testium/__init__.py` — parses CLI args, dispatches to the two modes.  
+`multiprocessing.set_start_method('spawn')` is called early (required for Linux dialog subprocesses).
+
+### Core execution
+- `src/testium/interpreter/process.py` — `TestProcess(multiprocessing.Process)`: runs the test in a child process. Stdout is redirected via a `StringQueue` → pipe → parent thread (`capture_stdout`) that writes to real stdout.
+- `src/testium/interpreter/batch.py` — `Batch`: parent-side orchestrator for `-b` mode. Creates the `msg_queue`, starts `TestProcess`, waits for the "finished" signal.
+- `src/testium/interpreter/test_set.py` — `TestSet`: builds and executes the tree of test items.
+- `src/testium/interpreter/test_items/test_item*.py` — one file per test item type (check, cycle, group, let, unittest, py_func, lua_func, console, git, dialogs, report, parallel, …).
+
+### Communication channels (parent ↔ child process)
+- `msg_queue` (`multiprocessing.Queue`): carries status messages from child to parent.
+  - Item status: `{"id": <non-None>, "name": ..., "status": "started"|"finished", ...}`
+  - Global dict updates: `{"type": "gd_update"|"gd_delete", "key": ..., "value": ...}` — **no "id" key**
+  - Process finished: `{"id": None, "name": "test_process", "status": "finished"}` — id key present but `None`
+- `tst_ctrl` (`TestSetController`): sends control commands (execute, stop, pause, close, …) from parent to child.
+- stdout pipe (`multiprocessing.Pipe`): streams test output from child back to parent's `capture_stdout` thread.
+
+### Stdout pipeline (batch mode)
+```
+test item print()
+  → sys.stdout (StringQueue, in child)
+  → send_stdout thread (child) → pipe → capture_stdout thread (parent)
+  → print() → sys.stdout (TermLog wrapping real stdout, in parent)
+  → terminal
+```
+
+### Global dictionary
+`src/testium/interpreter/utils/globdict.py` — shared state accessible from test scripts via `tm.gd()` / `tm.setgd()`. When `set_update_queue()` is active (during test execution), every `setgd`/`delgd` on a non-`_`-prefixed key pushes a message to `msg_queue`.
+
+### Coloring (`-o` disables it)
+`src/testium/interpreter/utils/termlog.py` — `TermLog` wraps stdout with colorama-based line coloring (PASS=green, FAIL=red, WARN=yellow, …). Applied in parent process for batch mode. Auto-detects light/dark terminal background via (in order): `COLORFGBG` env var, OSC 11 query, default dark.
+
+### Dialog items in batch mode
+All dialog items (`dialog_image`, `dialog_question`, `dialog_references`, `dialog_value`, `dialog_message`, `dialog_choices`, `dialog_note`) follow this rule in non-interactive text mode (`-b`):
+- `auto_result` defined in the `.tum` → result controlled by it (`ok`/`yes` → SUCCESS, `cancel`/`no` → FAIL)
+- `auto_result` absent → FAIL with `"Dialog not supported in batch mode"`
+- `sleep dialog: true` → exception: just sleeps normally, no GUI, no failure
+
+`auto_result` (and `auto_value` for value/note dialogs) is intended for the validation test suite (`test/validation/`) only.
+
+### `parallel` item
+`src/testium/interpreter/test_items/test_item_parallel.py` — runs multiple branches concurrently.
+
+```yaml
+- parallel:
+    name: My parallel block
+    sync: all        # all: wait for all; any: stop as soon as one finishes
+    no_fail: true    # (optional) don't propagate branch failures to parent
+    branches:
+      - name: Branch A
+        wait_for:              # (optional) poll condition before starting
+          condition: <| expr |>
+          timeout: 10
+        steps:
+          - ...
+      - name: Branch B
+        steps:
+          - ...
+```
+
+- `TestItemParallel(TestItemContainer)`: mutates `dict_item["steps"]` to inject synthetic `parallel_branch` items so `load_test_recursively` loads branches normally as children.
+- `TestItemParallelBranch(TestItemContainer)`: container for one branch. `wait_for` polls every 0.1s up to `timeout` seconds before running steps.
+- `sync: any` calls `_stop_branch_recursively()` on all other branches when one *actually runs* (SUCCESS/FAILURE). A `NORUN` branch (disabled, condition not met) never wins the race.
+- Each branch runs in a daemon thread; the parent waits with `.join()`.
+- Branches stopped late (e.g. user disabled them in the GUI, or another sync:any branch already won) go through the normal `branch.stop() + branch.execute()` path so they always produce a clean DB entry via `addTest()`.
+- Exceptions raised in a branch's `execute()` are caught by `run_branch`, logged to stdout, and converted to a `FAILURE` result so they never disappear silently.
+- `sync: all` ignores `NORUN` branches when computing success (matches Group/Cycle semantics): only an actual `FAILURE` fails the parallel.
+- `TestItemSleep` is interruptible (polls `self._is_stopped` in a loop) so `sync: any` can stop slow branches quickly. `py_func` and `console` items are not interruptible; their full duration is observed before the branch returns.
+
+### `TestItemContainer` base class
+`src/testium/interpreter/test_items/test_item_container.py` — shared base for Group, Cycle, Parallel, and ParallelBranch. Provides `_run_children_sequentially()` which handles stop-on-failure, `executedOnStop` items, and returns `(TestResult, stopped_bool)`.
+
+### Report threading
+`src/testium/interpreter/test_report/test_report.py` — SQLite report with thread-safe writes:
+- `sqlite3.connect(..., check_same_thread=False)`
+- `self._lock = threading.Lock()` guards the SQLite `INSERT` only.
+- Per-item log capture (`stdio_redir.read()`) is naturally race-free thanks to per-thread buffers (see `StdoutProxy`).
+
+### Thread-aware stdout (`StdoutProxy`)
+`src/testium/runtime/stdout_redirect.py` — when `log_stored: True`, `intercept()` installs a `StdoutProxy` as `sys.stdout`/`sys.stderr` instead of a single shared `StringQueue`. The proxy:
+- Holds one `StringQueue` per thread (registered via `register_thread(buffer=...)`). The main thread uses a default buffer; each parallel branch's thread registers its own at start and unregisters at end. `stdio_redir.read()` reads the calling thread's buffer → `addTest()` of an item running in branch X reads X's clean, non-interleaved output.
+- For the live stream (terminal in batch / GUI panel), prefixes every line emitted from a branch's thread with `[<branch_name>] ` so concurrent branches stay readable.
+- Exposes `write` / `writeln` / `flush` (Python 3.14's `unittest` calls `stream.writeln()` directly without `_WritelnDecorator`).
+
+### Subprocess API contract (py_func / lua_func)
+
+User test scripts running inside a `py_func` or `lua_func` subprocess **must** use the JSON-RPC bridge to interact with testium state:
+
+- Python: `import py_func.tm as tm` — auto-generates wrappers for every function in `runtime/api.py:SUPPORTED_API`. `tm.gd`/`tm.setgd`/`tm.delgd` go through JSON-RPC to the parent.
+- Lua: `local tm = require("tm")` — same idea on the Lua side.
+
+`api.testium` is the *main-process* implementation; it is **not** exposed to subprocesses by design (not bundled in PyInstaller, not on the subprocess `PYTHONPATH` in pip-installed mode either when isolation is preserved). An import attempt from a subprocess script is a code smell and is detected by `test/validation/items/isolation/`.
+
+To add a new API call usable from subprocesses:
+1. Add the function to `api/testium.py`
+2. Add its name to `SUPPORTED_API` in `runtime/api.py`
+3. It is auto-exposed via JSON-RPC by `interpreter/utils/api_srv.py` and auto-wrapped by `py_func/tm.py:_make_api`
+
+### External interpreter resolution (`bins.py`)
+`src/testium/interpreter/utils/bins.py` — single source of truth for the paths to the external Python and Lua interpreters used by subprocesses.
+
+- `python_bin()` / `lua_bin()` : resolve once, cache in memory. User can override via the `python_bin` / `lua_bin` global dict keys (typically populated from the YAML config). Falls back to discovery on PATH (candidates: `python3`/`python` and `lua`/`lua5.5`/`lua5.4`/`lua5.3`/`lua5.2`/`lua5.1`).
+- `ensure(*names)` : called by `TestSet._validate_runtime_deps()` at test load. Always requires `python` (the eval engine always runs); requires `lua` only if a `lua_func` item is in the tree. Fails fast with a clear error citing tried candidates and override key.
+
+Engines (`PyProcessBase`, `LuaProcessBase`, `EvalExecEngine`) call `bins.python_bin()`/`bins.lua_bin()` themselves — call sites never pass an explicit binary path.
+
+## Key files
+
+| Path | Role |
+|------|------|
+| `src/testium/__init__.py` | CLI entry, mode dispatch |
+| `src/testium/interpreter/batch.py` | `-b` mode orchestrator |
+| `src/testium/interpreter/process.py` | Child test process |
+| `src/testium/interpreter/test_set.py` | Test tree builder/executor |
+| `src/testium/interpreter/test_items/test_item_container.py` | Base class for container items |
+| `src/testium/interpreter/test_items/test_item_parallel.py` | `parallel` and `parallel_branch` items |
+| `src/testium/interpreter/utils/globdict.py` | Global variable dict |
+| `src/testium/interpreter/utils/termlog.py` | Terminal color output |
+| `src/testium/runtime/stdout_redirect.py` | `StdioRedirect` singleton (`stdio_redir`) |
+| `src/testium/runtime/string_queue.py` | Thread-safe string buffer used for stdout redirection |
+| `src/testium/api/testium.py` | Public API for test scripts (`tm.*`) |
+| `src/testium/py_func/` | Python subprocess for `py_func` items (sandboxed: imports only `runtime/` and `py_func/`) |
+| `src/testium/lua_func/` | Lua subprocess scripts for `lua_func` items |
+
+## Package layout
+
+The whole project is a single Python package under `src/testium/`:
+
+```
+src/testium/
+├── __init__.py / __main__.py
+├── runtime/      internal plumbing (jrpc, stdout_redirect, string_queue, tum_except, api)
+├── api/          public SDK exposed to test scripts (`import api.testium as tm`)
+├── interpreter/  test execution engine (NOT visible to py_func/lua_func)
+├── main_win/     GUI (NOT visible to py_func/lua_func)
+├── py_func/      subprocess code for python_func items
+└── lua_func/     subprocess scripts for lua_func items (data files)
+```
+
+`subproc_path()` and `testium_path()` both return the package directory. The py_func subprocess is launched with cwd=that directory and `python3 py_func`. The contract that `py_func/` and `lua_func/` only depend on `runtime/` (no `interpreter`, `main_win`, `api`, `testium`) is enforced by `test/validation/items/isolation/`.
+
+## GUI icons (main_win)
+
+Icons live in `src/testium/main_win/resources/` with three theme variants:
+
+| Folder | Theme index | Usage |
+|--------|-------------|-------|
+| `color/` | 0 (default) | Coloured icons |
+| `black/` | 1 | Black silhouette on transparent |
+| `white/` | 2 | White silhouette on transparent (LA mode) |
+
+Icons are **64×64 PNG**. Black variants: RGBA with RGB=`(0,0,0)`, alpha varies. White variants: LA with luminance=`255`, alpha varies.
+
+The mapping item-type → icon filename is in `_ITEM_CONFIG` (`src/testium/main_win/test_tree_items/test_tree_item.py`). At runtime, `icon_prefix()` returns `:/color`, `:/black`, or `:/white` (Qt resource prefix) based on the user preference.
+
+All icons must be declared in `src/testium/main_win/resources/testium_core_win.qrc` (one entry per theme section). After any QRC change, regenerate the compiled resource file:
+```
+cd src/testium/main_win/resources
+pyside6-rcc testium_core_win.qrc -o testium_core_win_rc.py
+```
+
+Icons are assigned once when the test file is loaded (not updated live on theme change — a file reload is required).
+
+### `run` item
+`src/testium/interpreter/test_items/test_item_run.py` — launches a `.tum` file in a new testium instance (`-b` in batch mode, `-r` in GUI mode). Result:
+- **PASS** if the sub-instance launched and ran to completion (exit code is ignored)
+- **FAIL** if the file is not found, `wait_for_exec` is set without `start_time`/`end_time`, the time window was not reached, or any other launch error
+
+The sub-test's own pass/fail result is intentionally not propagated.
+
+The interpreter and entry point used to spawn the sub-instance are picked automatically by `_testium_launch_cmd()` based on how the parent was started (AppImage → `$APPIMAGE`; Flatpak → `flatpak run`; PyInstaller → the frozen binary; source/wheel → `[sys.executable, abspath(sys.argv[0])]`). The user cannot override either via the YAML — selecting a different testium binary or Python from a sub-test was removed because it was either ill-defined (bundle modes have no separable Python) or could mismatch the parent's environment in surprising ways.
+
+### Report exporters & plugins
+`src/testium/interpreter/test_report/test_report.py` — `_EXPORTER_REGISTRY` dict maps a format name (cmd key in the YAML `report.export`) to a lazy loader. Built-ins: `text`, `json`, `junit` (needs `junit_xml`), `html` (needs `lxml`). `sqlite` is the storage layer, no-op as an export.
+
+Third-party plugins are discovered at module import via `importlib.metadata.entry_points(group="testium.exporters")` — installing a wheel that declares such an entry point is enough, no testium config change needed:
+```toml
+[project.entry-points."testium.exporters"]
+my_format = "my_pkg:MyExporter"
+```
+Exporter contract: `__init__(self, name, con, path, pats, keys, no_header=False)` — the class does its work in `__init__` and writes to `path`.
+
+Behaviour on errors:
+- Unknown format → info line `[report] Export skipped: format "X" not found. Available: ...`, run continues.
+- Optional dependency missing → same info line with a pip-install hint, run continues.
+
+A real-world test plugin lives at `test/validation/fake_exporter/` (CSV exporter, auto-installed by `scripts/build_env.sh` and exercised by `test/validation/items/report_plugin/`).
+
+## Packaging
+
+Four distribution channels coexist, all sharing the single `src/testium/` package and the single `src/requirements.txt` dependency list:
+
+| Channel | Where | Build | Notes |
+|---------|-------|-------|-------|
+| Wheel (`pip install`) | `src/pyproject.toml` | `python -m build` | Vanilla Python package; entry point `testium = "testium:main"`. |
+| PyInstaller binary | `package/pyinstaller/` | `build.sh` | Single ~130 MB binary. `py_func`, `runtime`, `lua_func` bundled at `_MEIPASS` root so the **host** Python can find them when launched as `python3 py_func`. `api`/`interpreter` are **not** exposed (subprocess isolation). |
+| Flatpak | `package/flatpak/` | `build.sh` (uses `flatpak-builder`) | KDE 6.10 runtime. The bundled Python runs only the main process; `py_func` / `lua_func` MUST run under the **host** interpreter (no Python/Lua bundled). Produces a distributable `.flatpak` bundle. |
+| AppImage | `package/appimage/` | `build.sh` (Debian Bookworm container via Podman/Docker) | Bundles Python 3.11 for the main process; `py_func` / `lua_func` MUST run under the **host** interpreter. Build runs in a container so it works on Arch / any non-Debian host. |
+
+The `.deb` work-in-progress lives in `package/deb/`:
+- `test_distro.sh debian:bookworm | debian:trixie | ubuntu:24.04` spins up a Docker/Podman container, reports system package availability, falls back to pip for what's missing (`pyside6` on bookworm/ubuntu, `telnetlib3`, `junit_xml`), runs the validation suite. Currently green on the three targets.
+
+### Host-only py_func / lua_func in sandboxed bundles (Flatpak, AppImage)
+
+The bundled Python (Flatpak's runtime python, AppImage's `python3.11`) is reserved for the **main process only**. Subprocesses (`py_func`, `lua_func`, `git`) must use the host's interpreters and tools so user-installed modules (pyserial, junit_xml, …) are visible. This is enforced by `interpreter/utils/bins.py`:
+
+- `_in_flatpak()` (checks `/.flatpak-info`) and `_in_appimage()` (checks `APPIMAGE` env var) detect the sandbox.
+- `_which(name)` probes only host bin dirs in those modes:
+  - Flatpak: `/run/host/usr/{local/,}bin`, `/run/host/bin` (host mounted via `--filesystem=host-os`).
+  - AppImage: `/usr/local/bin`, `/usr/bin`, `/bin` (we are directly on the host filesystem).
+  - If the host has no python3/lua, `ensure()` raises `ETUMRuntimeError` at test load with the candidate list — no silent fallback to a bundled interpreter.
+- User overrides (`python_bin`/`lua_bin` in globdict): bare names are resolved through `_which()` (host-only), absolute paths are accepted as-is.
+- `apply_host_libs(env)` is called by `py_process.py` / `lua_process.py` on the env passed to Popen:
+  - Flatpak: prepends host lib dirs to `LD_LIBRARY_PATH` so the dynamic linker finds host `.so`'s.
+  - AppImage: strips `$APPDIR`-prefixed entries from `LD_LIBRARY_PATH` / `PYTHONPATH` / `PATH` and drops `PYTHONHOME`, so the host Python doesn't try to load the bundled stdlib/site-packages.
+- `apply_host_lua_paths(env)` (Flatpak only) prepends `/run/host/usr/{lib,share}/lua/X.Y` to `LUA_PATH` / `LUA_CPATH` so `cjson`, `socket`, etc. resolve. Must be called **after** user `lua_env` overrides so host paths win. AppImage relies on host Lua's compiled-in defaults.
+- `py_process.py` additionally pops `PYTHONUSERBASE` (set to `/var/data/python` by the Flatpak runtime, which would hide `~/.local/lib/...`).
+
+### Version reporting (`interpreter/utils/version.py`)
+
+Both Flatpak and AppImage export `TESTIUM_VERSION` from a launcher (Flatpak: launcher script in `org.testium.Testium.yaml`; AppImage: `runtime.env` in `AppImageBuilder.yml`). `get_testium_version()` checks `/.flatpak-info` / `APPIMAGE` and reads `TESTIUM_VERSION` rather than relying on package metadata or repo introspection.
+
+## Recent fixes / notable changes
+- Restructure: single `src/testium/` Python package (was 4 sibling top-levels: `testium`, `lib`, `py_func`, `lua_func`). `lib/` → `runtime/`, `libs/` → `api/`. `pip install` now produces a clean `site-packages/testium/` with no top-level pollution; `.lua` files travel via `package_data`.
+- `bins.py`: centralised resolution + cache of external `python3` / `lua` binaries. Replaces the scattered `tm.gd("python_bin")`/`tm.gd("lua_bin")` dance and the duplicated discovery logic in `py_process.py`/`lua_process.py`. Validates at test load via `TestSet._validate_runtime_deps()` so missing interpreters fail fast.
+- Subprocess API contract: user scripts in `py_func`/`lua_func` use the JSON-RPC bridge (`py_func.tm` / Lua `tm`) — never `api.testium` / `interpreter.*` directly. `SUPPORTED_API` extended with `OS`, `get_main_dir`, `init_timestamp`, `timestamp`, `timestamp_as_sec` so subprocess scripts have the same surface as main-process code.
+- Report exporter plugin registry (`test_report.py`): `_EXPORTER_REGISTRY` + `entry_points("testium.exporters")` discovery. Missing format → info line, run continues.
+- About dialog rework: `QVBoxLayout` (resizable), version + dirty/branch info in a `QLabel` (auto-sized), copyright + clickable EUPL-1.2 link.
+- `test_ctrl.control()`: drain stale responses (left over from polled `loaded()` after `clear()` race) instead of failing on a wrong cmd key — fixes a "Unexpected return error in test set controller" seen in GUI mode after a fast reload.
+- `lua_process.py`: stderr no longer DEVNULL'd so actual Lua errors (missing `cjson`/`socket`) surface instead of "Connection refused".
+- `run_post_exec`: failure message uses `print_warn` (was `print_debug` — silent in non-debug runs).
+- Python 3.11 compat: replaced PEP 701 nested-quote f-strings (e.g. `f"... {d["k"]} ..."`) with single-quote inner strings or string concatenation.
+- `parallel` item: new item with `sync: all|any`, `wait_for`, daemon threads, `_stop_branch_recursively()`. Each branch thread registers a per-thread stdout buffer.
+- `parallel_branch` icon: distinct single-arrow icon (`parallel_branch.png`).
+- `parallel` F1 panel: `steps` stripped from each branch dict.
+- `test_item_container.py`: shared base class extracted from Group/Cycle.
+- `test_item_sleep.py`: interruptible loop so `sync: any` can stop slow branches quickly.
+- `stdout_redirect.py`: `StdoutProxy` (thread-aware buffers + branch-prefixed live output, `writeln()` for Python 3.14 unittest).
+- `test_report.py`: thread-safe SQLite INSERT for parallel branch concurrency.
+- `terminal.py`: deleted — `-m`/`--terminal` mode removed.
+- `batch.py`: premature finish bug on `gd_update` (no `"id"` key) — fix uses `"id" in m and m["id"] is None`.
+- `batch.py`: `control("loaded")` deadlock on TestProcess crash — fix uses daemon thread + `threading.Event` + `is_alive()` polling.
+- `termlog.py`: light/dark terminal auto-detection (`COLORFGBG`, OSC 11) + write residue bug.
+- Dialog items: `auto_result`/`auto_value` for non-interactive text mode; dialogs without `auto_result` FAIL immediately in batch.
+- `run` item: renamed `tum_fime` → `tum`; removed `stdout=PIPE` deadlock; PASS on any completed subprocess.
+- `unittest` item: renamed from `unittest_file`.
+- GUI test tree: check and fold state preserved across same-file reloads.
+- Licence: EUPL-1.2.
+
+## Validation tests
+Located in `test/validation/`. Run with `-b` flag:
+```
+./run.sh -b -- test/validation/main.tum
+```
+Parallel item tests: `test/validation/items/parallel/test.tum`
+
+## Dependencies
+See `src/requirements.txt`. Key ones: `pyside6`, `pyyaml`, `jinja2`, `colorama`, `gitpython`, `pexpect`, `matplotlib`.

LICENSE

@@ -0,0 +1,315 @@
+Copyright (c) 2025-2026 François Dausseur
+
+Licensed under the EUPL
+
+
+                       EUROPEAN UNION PUBLIC LICENCE v. 1.2
+                       EUPL © the European Union 2007, 2016
+
+This European Union Public Licence (the 'EUPL') applies to the Work (as
+defined below) which is provided under the terms of this Licence. Any use of
+the Work, other than as authorised under this Licence is prohibited (to the
+extent such use is covered by a right of the copyright holder of the Work).
+
+The Work is provided under the terms of this Licence when the Licensor (as
+defined below) has placed the following notice immediately following the
+copyright notice for the Work:
+
+        Licensed under the EUPL
+
+or has expressed by any other means his willingness to license under the EUPL.
+
+
+1. Definitions
+
+In this Licence, the following terms have the following meaning:
+
+- 'The Licence': this Licence.
+
+- 'The Original Work': the work or software distributed or communicated by the
+  Licensor under this Licence, available as Source Code and also as Executable
+  Code as the case may be.
+
+- 'Derivative Works': the works or software that could be created by the
+  Licensee, based upon the Original Work or modifications thereof. This Licence
+  does not define the extent of modification or dependence on the Original
+  Work required in order to classify a work as a Derivative Work; this extent
+  is determined by copyright law applicable in the country mentioned in
+  Article 15.
+
+- 'The Work': the Original Work or its Derivative Works.
+
+- 'The Source Code': the human-readable form of the Work which is the most
+  convenient for people to study and modify.
+
+- 'The Executable Code': any code which has generally been compiled and which
+  is meant to be interpreted by a computer as a program.
+
+- 'The Licensor': the natural or legal person that distributes or communicates
+  the Work under the Licence.
+
+- 'Contributor(s)': any natural or legal person who modifies the Work under
+  the Licence, or otherwise contributes to the creation of a Derivative Work.
+
+- 'The Licensee' or 'You': any natural or legal person who makes any usage of
+  the Work under the terms of the Licence.
+
+- 'Distribution' or 'Communication': any act of selling, giving, lending,
+  renting, distributing, communicating, transmitting, or otherwise making
+  available, online or offline, copies of the Work or providing access to its
+  essential functionalities at the disposal of any other natural or legal
+  person.
+
+
+2. Scope of the rights granted by the Licence
+
+The Licensor hereby grants You a worldwide, royalty-free, non-exclusive,
+sublicensable licence to do the following, for the duration of copyright
+vested in the Original Work:
+
+- use the Work in any circumstance and for all usage,
+- reproduce the Work,
+- modify the Work, and make Derivative Works based upon the Work,
+- communicate to the public, including the right to make available or display
+  the Work or copies thereof to the public and perform publicly, as the case
+  may be, the Work,
+- distribute the Work or copies thereof,
+- lend and rent the Work or copies thereof,
+- sublicense rights in the Work or copies thereof.
+
+Those rights can be exercised on any media, supports and formats, whether now
+known or later invented, as far as the applicable law permits so.
+
+In the countries where moral rights apply, the Licensor waives his right to
+exercise his moral right to the extent allowed by law in order to make
+effective the licence of the economic rights here above listed.
+
+The Licensor grants to the Licensee royalty-free, non-exclusive usage rights
+to any patents held by the Licensor, to the extent necessary to make use of
+the rights granted on the Work under this Licence.
+
+
+3. Communication of the Source Code
+
+The Licensor may provide the Work either in its Source Code form, or as
+Executable Code. If the Work is provided as Executable Code, the Licensor
+provides in addition a machine-readable copy of the Source Code of the Work
+along with each copy of the Work that the Licensor distributes or indicates,
+in a notice following the copyright notice attached to the Work, a repository
+where the Source Code is easily and freely accessible for as long as the
+Licensor continues to distribute or communicate the Work.
+
+
+4. Limitations on copyright
+
+Nothing in this Licence is intended to deprive the Licensee of the benefits
+from any exception or limitation to the exclusive rights of the rights owners
+in the Work, of the exhaustion of those rights or of other applicable
+limitations thereto.
+
+
+5. Obligations of the Licensee
+
+The grant of the rights mentioned above is subject to some restrictions and
+obligations imposed on the Licensee. Those obligations are the following:
+
+Attribution right: The Licensee shall keep intact all copyright, patent or
+trademarks notices and all notices that refer to the Licence and to the
+disclaimer of warranties. The Licensee must include a copy of such notices
+and a copy of the Licence with every copy of the Work he/she distributes or
+communicates. The Licensee must cause any Derivative Work to carry prominent
+notices stating that the Work has been modified and the date of modification.
+
+Copyleft clause: If the Licensee distributes or communicates copies of the
+Original Works or Derivative Works, this Distribution or Communication will
+be done under the terms of this Licence or of a later version of this Licence
+unless the Original Work is expressly distributed only under this version of
+the Licence — for example by communicating 'EUPL v. 1.2 only'. The Licensee
+(becoming Licensor) cannot offer or impose any additional terms or conditions
+on the Work or Derivative Work that alter or restrict the terms of the
+Licence.
+
+Compatibility clause: If the Licensee Distributes or Communicates Derivative
+Works or copies thereof based upon both the Work and another work licensed
+under a Compatible Licence, this Distribution or Communication can be done
+under the terms of this Compatible Licence. For the sake of this clause,
+'Compatible Licence' refers to the licences listed in the appendix attached
+to this Licence. Should the Licensee's obligations under the Compatible
+Licence conflict with his/her obligations under this Licence, the obligations
+of the Compatible Licence shall prevail.
+
+Provision of Source Code: When distributing or communicating copies of the
+Work, the Licensee will provide a machine-readable copy of the Source Code or
+indicate a repository where this Source will be easily and freely available
+for as long as the Licensee continues to distribute or communicate the Work.
+
+Legal Protection: This Licence does not grant permission to use the trade
+names, trademarks, service marks, or names of the Licensor, except as
+required for reasonable and customary use in describing the origin of the
+Work and reproducing the content of the copyright notice.
+
+
+6. Chain of Authorship
+
+The original Licensor warrants that the copyright in the Original Work
+granted hereunder is owned by him/her or licensed to him/her and that he/she
+has the power and authority to grant the Licence.
+
+Each Contributor warrants that the copyright in the modifications he/she
+brings to the Work are owned by him/her or licensed to him/her and that
+he/she has the power and authority to grant the Licence.
+
+Each time You accept the Licence, the original Licensor and subsequent
+Contributors grant You a licence to their contributions to the Work, under
+the terms of this Licence.
+
+
+7. Disclaimer of Warranty
+
+The Work is a work in progress, which is continuously improved by numerous
+Contributors. It is not a finished work and may therefore contain defects or
+'bugs' inherent to this type of development.
+
+For the above reason, the Work is provided under the Licence on an 'as is'
+basis and without warranties of any kind concerning the Work, including
+without limitation merchantability, fitness for a particular purpose, absence
+of defects or errors, accuracy, non-infringement of intellectual property
+rights other than copyright as stated in Article 6 of this Licence.
+
+This disclaimer of warranty is an essential part of the Licence and a
+condition for the grant of any rights to the Work.
+
+
+8. Disclaimer of Liability
+
+Except in the cases of wilful misconduct or damages directly caused to
+natural persons, the Licensor will in no event be liable for any direct or
+indirect, material or moral, damages of any kind, arising out of the Licence
+or of the use of the Work, including without limitation, damages for loss of
+goodwill, work stoppage, computer failure or malfunction, loss of data or any
+commercial damage, even if the Licensor has been advised of the possibility
+of such damage. However, the Licensor will be liable under statutory product
+liability laws as far such laws apply to the Work.
+
+
+9. Additional agreements
+
+While distributing the Work, You may choose to conclude an additional
+agreement, defining obligations or services consistent with this Licence.
+However, if accepting obligations, You may act only on your own behalf and on
+your sole responsibility, not on behalf of the original Licensor or any other
+Contributor, and only if You agree to indemnify, defend, and hold each
+Contributor harmless for any liability incurred by, or claims asserted
+against such Contributor by the fact You have accepted any warranty or
+additional liability.
+
+
+10. Acceptance of the Licence
+
+The provisions of this Licence can be accepted by clicking on an icon 'I
+agree' placed under the bottom of a window displaying the text of this
+Licence or by affirming consent in any other similar way, in accordance with
+the rules of applicable law. Clicking on that icon indicates your clear and
+irrevocable acceptance of this Licence and all of its terms and conditions.
+
+Similarly, you irrevocably accept this Licence and all of its terms and
+conditions by exercising any rights granted to You by Article 2 of this
+Licence, such as the use of the Work, the creation by You of a Derivative
+Work or the Distribution or Communication by You of the Work or copies
+thereof.
+
+
+11. Information to the public
+
+In case of any Distribution or Communication of the Work by means of
+electronic communication by You (for example, by offering to download the
+Work from a remote location) the distribution channel or media (for example,
+a website) must at least provide to the public the information requested by
+the applicable law regarding the Licensor, the Licence and the way it may be
+accessible, concluded, stored and reproduced by the Licensee.
+
+
+12. Termination of the Licence
+
+The Licence and the rights granted hereunder will terminate automatically
+upon any breach by the Licensee of the terms of the Licence.
+
+Such a termination will not terminate the licences of any person who has
+received the Work from the Licensee under the Licence, provided such persons
+remain in full compliance with the Licence.
+
+
+13. Miscellaneous
+
+Without prejudice of Article 9 above, the Licence represents the complete
+agreement between the Parties as to the Work.
+
+If any provision of the Licence is invalid or unenforceable under applicable
+law, this will not affect the validity or enforceability of the Licence as a
+whole. Such provision will be construed or reformed so as necessary to make
+it valid and enforceable.
+
+The European Commission may publish other linguistic versions or new versions
+of this Licence or updated versions of the Appendix, so far this is required
+and reasonable, without reducing the scope of the rights granted by the
+Licence. New versions of the Licence will be published with a unique version
+number.
+
+All linguistic versions of this Licence, approved by the European Commission,
+have identical value. Parties can take advantage of the linguistic version of
+their choice.
+
+
+14. Jurisdiction
+
+Without prejudice to specific agreement between parties,
+
+- any litigation resulting from the interpretation of this License, arising
+  between the European Union institutions, bodies, offices or agencies, as a
+  Licensor, and any Licensee, will be subject to the jurisdiction of the
+  Court of Justice of the European Union, as laid down in article 272 of the
+  Treaty on the Functioning of the European Union,
+
+- any litigation arising between other parties and resulting from the
+  interpretation of this License, will be subject to the exclusive
+  jurisdiction of the competent court where the Licensor resides or conducts
+  its primary business.
+
+
+15. Applicable Law
+
+Without prejudice to specific agreement between parties,
+
+- this Licence shall be governed by the law of the European Union Member
+  State where the Licensor has his seat, resides or has his registered
+  office,
+
+- this licence shall be governed by Belgian law if the Licensor has no seat,
+  residence or registered office inside a European Union Member State.
+
+
+                                  Appendix
+
+
+'Compatible Licences' according to Article 5 EUPL are:
+
+- GNU General Public License (GPL) v. 2, v. 3
+- GNU Affero General Public License (AGPL) v. 3
+- Open Software License (OSL) v. 2.1, v. 3.0
+- Eclipse Public License (EPL) v. 1.0
+- CeCILL v. 2.0, v. 2.1
+- Mozilla Public Licence (MPL) v. 2
+- GNU Lesser General Public Licence (LGPL) v. 2.1, v. 3
+- Creative Commons Attribution-ShareAlike v. 3.0 Unported (CC BY-SA 3.0) for
+  works other than software
+- European Union Public Licence (EUPL) v. 1.1, v. 1.2
+- Québec Free and Open-Source Licence — Reciprocity (LiLiQ-R) or Strong
+  Reciprocity (LiLiQ-R+).
+
+The European Commission may update this Appendix to later versions of the
+above licences without producing a new version of the EUPL, as long as they
+provide the rights granted in Article 2 of this Licence and protect the
+covered Source Code from exclusive appropriation.
+
+All other changes or additions to this Appendix require the production of a
+new EUPL version.

README.md

@@ -1,173 +1,122 @@
-# Documentation
+# testium
 
-[See here](doc/manual/testium_manual.pdf).
+testium is a YAML-driven test sequencer for hardware-in-the-loop and
+integration testing. A test campaign is described in a `.tum` file as a tree
+of items (checks, console interactions, Python/Lua functions, parallel blocks,
+dialogs, …); testium executes the tree, captures results, and produces
+reports in several formats.
 
-# run testium
+## Documentation
 
-From the root path, on windows `cmd`:
+* [Quick start](doc/quick_start.md) — install and run your first test in
+  five minutes.
+* [Tutorial](doc/tutorial.md) — guided walk-through of the most common
+  test items with a runnable example.
+* [User manual (PDF)](doc/manual/testium_manual.pdf) — full reference.
+* [`doc/examples/`](doc/examples/) — runnable `.tum` snippets.
 
-    run.bat
+## Pre-built releases
 
-On windows powershell:
+Pre-built artifacts are published at
+<https://git.beafrancois.fr/v-and-v/testium/releases>:
 
-    run.ps1
+* **Python wheel** (`testium-<version>-py3-none-any.whl`) — install with
+  `pip install testium-*.whl`. Lighter than the binary; pulls Python
+  dependencies from PyPI on install.
+* **Self-contained Linux binary** (`testium`, built with PyInstaller) —
+  runnable directly, no Python installation required on the host. Lua
+  support still needs a system `lua` interpreter and the `lua-socket` /
+  `lua-cjson` modules.
+* **Flatpak bundle** (`testium.flatpak`) — install with:
 
-On linux:
+  ```sh
+  # Add Flathub (once, to fetch the KDE/PySide runtimes)
+  flatpak remote-add --user --if-not-exists flathub https://flathub.org/repo/flathub.flatpakrepo
 
-    ./run.sh
+  # Install the bundle
+  flatpak install --user testium.flatpak
+  ```
 
-The virtual environment is created if needed and *testium* is started.
+  After installation testium appears in the desktop application menu and the
+  `testium` command is available in the terminal (requires `~/.local/bin` in
+  `PATH`, which most modern distributions provide by default).
 
-# Manual setup
+## Quick start
 
-A python virtual environment should be created:
+From a checkout of the repository:
 
-    python3 -m venv <testium_venv>
+| OS | Command |
+|----|---------|
+| Linux | `./run.sh` |
+| Windows (cmd) | `run.bat` |
+| Windows (PowerShell) | `run.ps1` |
 
-## Requirements
+The wrapper creates a Python virtual environment on first run and starts
+testium in GUI mode. Add `-b path/to/test.tum` to run a test in batch mode.
 
-In the virtual environment, the following modules must be installed:
+## Manual installation
 
-* pyside6
-* pyserial
-* pyyaml
-* pexpect
-* gitpython
-* jinja2
-* colorama
-* matplotlib
-* junit-xml
-* lxml
+If the wrapper script does not fit your environment, set up testium manually:
 
-A `requirements.txt` file is also available in the git repository in the path `testium/src/`.
-
-
-## run testium
-
-from the testium path, execute
-
-    python3 -m src/testium
-
-# Doc generation
-
-## Install sphinx
-
-    pip install sphinx
-
-## Generate the doc
-
-Execute
-
-    doc/manual/sphinx/./build_doc.sh
-
-This command works if texlive package has been installed on the system. It can be done by invoking the following command.
-
-    sudo apt install texlive-full
-
-# QT GUI
-
-## QT GUI modification
-
-Open the ".ui" file with `qtcreator` and modify the gui. Then regenerate the python code.
-
-On linux, a helper script has been created:
-    scripts/./qt_generate.sh
-
-# Debugging
-
-In order to debug testium or your python script executed within testium.
-
-## In VSCODE
-
-This is the prefered method :
-
-1. Create a debug configuration like the following:
-
-```
-    "configurations": [
-        {
-            "name": "Python : testium",
-            "type": "python",
-            "request": "launch",
-            "program": "${workspaceFolder}/src/testium",
-            "console": "integratedTerminal",
-            "args": ["-g"],
-            "justMyCode": true
-        },
-    ]
+```sh
+python3 -m venv .venv
+source .venv/bin/activate
+pip install -r src/requirements.txt
 ```
 
-2. Install debugpy module in python
+Required Python packages (see `src/requirements.txt`):
+`pyside6`, `pyserial`, `pyyaml`, `pexpect`, `gitpython`, `jinja2`, `colorama`,
+`matplotlib`, `junit-xml`, `lxml`.
 
-   python -m pip install debugpy
-3. Then get to the "RUN AND DEBUG" tab and press the play button.
-4. A testium window will pops up ; start execution of your tum.
-5. Do not forget to put breakpoints where you want to investigate.
+For tests using `lua_func` items, install Lua (>= 5.1) plus the `socket` and
+`cjson` modules. On Debian/Ubuntu:
 
-## Icons
+```sh
+sudo apt install lua5.4 lua-socket lua-cjson
+```
 
-Icons are coming from the following site: https://github.com/free-icons/free-icons.git
+Run testium:
 
-# testium Release
+```sh
+python3 src/testium               # GUI
+python3 src/testium -b mytest.tum # batch
+```
 
-## Pre-requisite
+## Troubleshooting
 
-A `python` virtual environment must have been set as described above.
+### `wl_proxy_marshal_flags` symbol error
 
-### Install pyinstaller
+```
+testium: symbol lookup error: ... undefined symbol: wl_proxy_marshal_flags
+```
 
-Install `pyinstaller` package using pip.
+Force the X11 Qt backend:
 
-## Generate the binary package
+```sh
+export QT_QPA_PLATFORM=xcb
+testium
+```
 
-The procedure for a binary release is as follows:
+### `xcb plugin missing`
 
-1. update the `release_note.txt` file
-2. modify the version in `src/VERSION` file
-3. be sure that the documentation is up to date, and if not execute `doc/manual/sphinx/build_doc.sh` script
-4. push modifications and create a tag with the new version on the git repository
-5. generate an executable file by calling `package/pyinstaller/./build.sh`
-6. run the complete validation test for each generated binary
-7. check that all the validation results are OK
+```
+qt.qpa.plugin: Could not load the Qt platform plugin "xcb"
+```
 
-# Troubleshooting
+Install the missing system libraries:
 
-## The testium exe crashes `wl_proxy_marshal_flags`
+```sh
+sudo apt install libxcb-cursor0 libicu-dev libxcb-cursor-dev
+```
 
-### Error message
+## License
 
-    /testium: symbol lookup error: /tmp/_MEIOhDCPF/libQt6WaylandClient.so.6: undefined symbol: wl_proxy_marshal_flags
+Copyright © 2025-2026 François Dausseur.
 
-### Solution
+testium is distributed under the **European Union Public Licence v. 1.2
+(EUPL-1.2)** — see [`LICENSE`](LICENSE) for the full text. SPDX:
+`EUPL-1.2`.
 
-Set the appropriate environment variable
-
-    export QT_QPA_PLATFORM=xcb
-    testium
-
-## xcb plugin missing
-
-### Error message
-
-    qt.qpa.plugin: Could not load the Qt platform plugin "xcb" in "" even though it was found.
-
-### Solution
-
-A package is missing
-
-    sudo apt install libxcb-cursor0
-    sudo apt-get install libicu-dev
-    sudo apt-get install libxcb-cursor-dev
-
-## The testium appimage crashes when opening a file
-
-This is usually because wayland is defined as the default X server.
-
-To change it :
-
-* Disable Wayland by uncommenting WaylandEnable=false in the `/etc/gdm3/daemon.conf`
-* Add `QT_QPA_PLATFORM=xcb` in `/etc/environment`
-* After a reboot, check that the environment variable value returns `x11`:
-
-  $ echo $XDG_SESSION_TYPE
-  x11
+Contributions are accepted under the same licence (inbound = outbound).
+See [`CONTRIBUTING.md`](CONTRIBUTING.md) for development setup, debugging
+workflow, and the release procedure.

build_all.sh

@@ -0,0 +1,107 @@
+#!/bin/bash
+# Build every distribution channel of testium, in order:
+#   1. Manual PDF         -> dist/testium-manual-<v>.pdf
+#   2. Wheel              -> dist/testium-<v>-py3-none-any.whl   (PEP 427 name)
+#   3. PyInstaller binary -> dist/testium-<v>
+#   4. Flatpak bundle     -> dist/testium-<v>.flatpak
+#   5. AppImage           -> dist/Testium-<v>-x86_64.AppImage    (original name)
+# release_note.txt is copied to dist/ up front (with a warning if it has no
+# entry for the current version).
+#
+# All artifacts are collected (copied) under <repo>/dist/. Original outputs in
+# src/dist/, package/*/dist/, doc/manual/ are left in place. Wheel and AppImage
+# keep their original names (which already contain the version); manual,
+# pyinstaller and flatpak are renamed to testium(-manual)-<version>(.suff).
+#
+# Re-uses scripts/build_env.sh and scripts/set_env.sh — the same pair invoked
+# by run.sh — so the venv at test/tmp/.venv stays the single source of Python
+# dependencies. `build` and `pyinstaller` are installed into that venv on
+# demand if not already there. Flatpak and AppImage build in their own
+# container/sandbox; their build.sh scripts have their own toolchain checks.
+
+set -e
+
+SCRIPT_DIR=$(realpath "$(dirname "$0")")
+VERSION=$(cat "$SCRIPT_DIR/src/VERSION")
+DIST_DIR="$SCRIPT_DIR/dist"
+mkdir -p "$DIST_DIR"
+
+# Release note: copy it to dist/ and warn (but don't fail) if it has no entry
+# for the current version.
+RELEASE_NOTE_SRC="$SCRIPT_DIR/release_note.txt"
+RELEASE_NOTE="$DIST_DIR/release_note.txt"
+cp -f "$RELEASE_NOTE_SRC" "$RELEASE_NOTE"
+if ! grep -qE "^version $VERSION([^.0-9]|$)" "$RELEASE_NOTE_SRC"; then
+    echo "WARNING: release_note.txt has no entry for version $VERSION." >&2
+fi
+
+export PY_VENV_NAME=".venv"
+export PY_VENV_DIR="$SCRIPT_DIR/test/tmp/$PY_VENV_NAME"
+export REQ_PATH="$SCRIPT_DIR/src/requirements.txt"
+
+bash "$SCRIPT_DIR/scripts/build_env.sh"
+source "$SCRIPT_DIR/scripts/set_env.sh"
+
+# Ensure wheel/PyInstaller toolchains are present in the venv.
+python -m pip install --quiet --upgrade build pyinstaller
+
+step() {
+    echo
+    echo "================================================================"
+    echo "  $1"
+    echo "================================================================"
+}
+
+# 1. Manual PDF
+step "1/5  Manual PDF (version $VERSION)"
+bash "$SCRIPT_DIR/doc/manual/sphinx/build_doc.sh"
+MANUAL_SRC="$SCRIPT_DIR/doc/manual/testium_manual.pdf"
+MANUAL="$DIST_DIR/testium-manual-${VERSION}.pdf"
+cp -f "$MANUAL_SRC" "$MANUAL"
+
+# 2. Wheel — PEP 427 name kept (already contains version)
+step "2/5  Wheel (version $VERSION)"
+(
+    cd "$SCRIPT_DIR/src"
+    rm -rf dist build *.egg-info
+    python -m build --wheel
+)
+WHEEL_SRC=$(ls -1t "$SCRIPT_DIR/src/dist"/*.whl | head -1)
+WHEEL="$DIST_DIR/$(basename "$WHEEL_SRC")"
+cp -f "$WHEEL_SRC" "$WHEEL"
+
+# 3. PyInstaller binary
+step "3/5  PyInstaller binary (version $VERSION)"
+bash "$SCRIPT_DIR/package/pyinstaller/build.sh"
+PYI_SRC="$SCRIPT_DIR/package/pyinstaller/dist/testium"
+PYI_BIN="$DIST_DIR/testium-${VERSION}"
+cp -f "$PYI_SRC" "$PYI_BIN"
+
+# 4. Flatpak bundle
+step "4/5  Flatpak bundle (version $VERSION)"
+(
+    cd "$SCRIPT_DIR/package/flatpak"
+    bash build.sh
+)
+FLATPAK_SRC="$SCRIPT_DIR/package/flatpak/testium.flatpak"
+FLATPAK_BUNDLE="$DIST_DIR/testium-${VERSION}.flatpak"
+cp -f "$FLATPAK_SRC" "$FLATPAK_BUNDLE"
+
+# 5. AppImage
+step "5/5  AppImage (version $VERSION)"
+(
+    cd "$SCRIPT_DIR/package/appimage"
+    bash build.sh
+)
+APPIMAGE_SRC=$(ls -1t "$SCRIPT_DIR/package/appimage"/*.AppImage 2>/dev/null | head -1)
+APPIMAGE="$DIST_DIR/$(basename "$APPIMAGE_SRC")"
+cp -f "$APPIMAGE_SRC" "$APPIMAGE"
+chmod +x "$APPIMAGE"
+
+step "All packages built"
+printf "  manual       : %s\n" "$MANUAL"
+printf "  wheel        : %s\n" "$WHEEL"
+printf "  pyinstaller  : %s\n" "$PYI_BIN"
+printf "  flatpak      : %s\n" "$FLATPAK_BUNDLE"
+printf "  appimage     : %s\n" "$APPIMAGE"
+printf "  release_note : %s\n" "$RELEASE_NOTE"

doc/examples/dummy/cpt/endurance.tum

@@ -1,4 +1,4 @@
 
-  - unittest_file:
+  - unittest:
         name: Test 5
         test_file: dummy.py

doc/examples/dummy/cpt/temperature.tum

@@ -3,7 +3,7 @@ sequence: &endurance_test
     !include endurance.tum
 
 sequence:
-    - unittest_file:
+    - unittest:
         name: Test 3
         test_file: dummy.py
         test_method: test_01_pass
@@ -11,6 +11,6 @@ sequence:
         iterator: 10
         steps:
             *endurance_test
-    - unittest_file:
+    - unittest:
         name: Test 4
         test_file: dummy.py

doc/examples/dummy/dummy.tum

@@ -15,7 +15,7 @@ main:
                 - $(reference_1)
                 - $(reference_2)
             report_show_success: true
-        - unittest_file:
+        - unittest:
               name: Test 1
               test_file: dummy.py
               doc: |
@@ -23,7 +23,7 @@ main:
                   Voilà...
         - sleep:
             {name: Sleep between one and two, timeout: 10, dialog: true}
-        - unittest_file:
+        - unittest:
             {name: Test 2, test_file: dummy.py,execute_on_stop: true}
         - loop:
             name: Cycle Temperature

doc/examples/dummy/post_execution.py

@@ -1,5 +1,5 @@
 
-import libs.testium as tm
+import py_func.tm as tm
 
 def post_exec():
     print('Success !!!!')

doc/examples/example_cycle.tum

@@ -14,7 +14,7 @@ main:
         key: report-key-2
         stop_on_failure: True
         steps:
-          - unittest_file:
+          - unittest:
               name: unittest item
               doc: |
                 The purpose of this unittest test item is to demonstrate
@@ -41,7 +41,7 @@ main:
               param:
                   - 123
 
-          - unittest_file:
+          - unittest:
               name: Unittest item
               test_file: dummy/dummy.py
               test_method:
@@ -98,7 +98,7 @@ main:
         name: Infine loop unittest step crashes
         stop_on_failure: True
         steps:
-          - unittest_file:
+          - unittest:
               name: Unittest item
               test_file: dummy/dummy.py
               test_method:
@@ -243,7 +243,7 @@ main:
         name: Infinite loop
         skipped: True
         steps:
-          - unittest_file:
+          - unittest:
               name: Unittest item
               test_file: dummy/dummy.py
               test_method: test_01_pass

doc/examples/example_items.tum

@@ -9,7 +9,7 @@ main:
     name: Test Sample number one
     version: 0.1
     steps:
-        - unittest_file:
+        - unittest:
             name: Unittest item
             test_file: dummy/unittest_str.py
             doc: Unittest test
@@ -88,7 +88,7 @@ main:
             name: cycle item
             iterator : 3
             steps:
-                - unittest_file:
+                - unittest:
                     name: Unittest item
                     test_file: dummy/dummy.py
                     test_method: test_01_pass
@@ -99,7 +99,7 @@ main:
                     name: cycle item
                     iterator : 3
                     steps:
-                        - unittest_file:
+                        - unittest:
                             name: Unittest item
                             test_file: dummy/dummy.py
                             test_method: test_01_pass

doc/examples/example_simple.tum

@@ -20,6 +20,22 @@ main:
           param:
               - 123
 
+      - py_func:
+          name: python long wait
+          doc: The purpose of this step is to try the tasks "stop" interruption
+          file: utils.py
+          func_name: long_wait
+          param:
+              - 10
+
+      - lua_func:
+          name: lua long wait
+          doc: The purpose of this step is to try the tasks "stop" interruption
+          file: lua_func.lua
+          func_name: long_wait
+          param:
+              - 10
+
       - sleep:
           name: sleep item
           dialog: true

doc/examples/lua_func.lua

@@ -1,4 +1,5 @@
 tm = require("tm")
+socket = require("socket")
 
 local module = {}
 
@@ -7,4 +8,8 @@ function module.func_to_be_executed(param)
     return param
 end
 
+function module.long_wait(sec)
+    socket.sleep(sec)
+end
+
 return module

doc/examples/param.yaml

@@ -17,18 +17,3 @@ plot_log_path: /tmp/testium_plot/$(testrun_date)/$(testrun_time)/
 python_path_Windows: C:\Users\François\Applications\Python313\python.exe
 python_path_Linux: $(home)/tmp/tum_venv/bin/python3
 
-# lua_bin_Windows: C:\Lua\5.1
-# lua_bin_Linux: /usr/bin/lua
-
-LUA_PATH_Linux: /usr/share/lua/5.4/?.lua;/usr/local/share/lua/5.4/?.lua;/usr/local/share/lua/5.4/?/init.lua;/usr/share/lua/5.4/?/init.lua;/usr/local/lib/lua/5.4/?.lua;/usr/local/lib/lua/5.4/?/init.lua;/usr/lib/lua/5.4/?.lua;/usr/lib/lua/5.4/?/init.lua;./?.lua;./?/init.lua;/home/francois/.luarocks/share/lua/5.4/?.lua;/home/francois/.luarocks/share/lua/5.4/?/init.lua
-LUA_CPATH_Linux: /usr/local/lib/lua/5.4/?.so;/usr/lib/lua/5.4/?.so;/usr/local/lib/lua/5.4/loadall.so;/usr/lib/lua/5.4/loadall.so;./?.so;/home/francois/.luarocks/lib/lua/5.4/?.so
-PATH_Linux:
-
-LUA_PATH_Windows: ;.\?.lua;C:\Lua\5.1\lua\?.lua;C:\Lua\5.1\lua\?\init.lua;C:\Lua\5.1\?.lua;C:\Lua\5.1\?\init.lua;C:\Lua\5.1\lua\?.luac
-LUA_CPATH_Windows: .\?.dll;C:\Lua\5.1\?.dll;C:\Lua\5.1\loadall.dll;C:\Lua\5.1\clibs\?.dll;C:\Lua\5.1\clibs\loadall.dll;.\?51.dll;C:\Lua\5.1\?51.dll;C:\Lua\5.1\clibs\?51.dll
-PATH_Windows: ""
-
-lua_env:
-  PATH: $(PATH_$(os))
-  LUA_PATH: $(LUA_PATH_$(os))
-  LUA_CPATH: $(LUA_CPATH_$(os))

doc/examples/utils.py

@@ -1,3 +1,5 @@
+from time import sleep
+
 def dummy_exit(useless1, useless2):
     return True
 
@@ -10,4 +12,7 @@ def funcToBeExecuted (bla):
 
 def funcToBeExecuted2 (bla):
     print(bla)
-    return blo
+    return blo
+
+def long_wait (sec):
+    sleep(sec)

doc/manual/sphinx/source/cdl_interface.rst

@@ -3,7 +3,7 @@ Command Line Interface
 
 .. code-block:: text
 
-    usage: testium.pyw [-h] [--version] [-b] [-m] [-c CONFIG_FILE [CONFIG_FILE ...]] [-r] [-l LOG_FILE]
+    usage: testium.pyw [-h] [--version] [-b] [-c CONFIG_FILE [CONFIG_FILE ...]] [-r] [-l LOG_FILE]
                         [-d DEFINE [DEFINE ...]] [-p REPORT_FILE] [-t {sqlite,json,junit,html,text}]
                         [-n REPORT_PATTERN [REPORT_PATTERN ...]] [-i INCLUDE_PATH [INCLUDE_PATH ...]] [-o] [-g]
                         [test_file]
@@ -16,9 +16,8 @@ Command Line Interface
     --version             Returns the version of testium
     -b, --batch-execution
                             Executes the test in batch mode
-    -m, --terminal        Starts terminal mode
     -c CONFIG_FILE [CONFIG_FILE ...], --config-file CONFIG_FILE [CONFIG_FILE ...]
-    -o, --no-color        Deactivates stdout colors in batch and terminal mode
+    -o, --no-color        Deactivates stdout colors in batch mode
                             Configuration file
     -r, --run-and-close   Runs the test then closes the application
     -l LOG_FILE, --log-file LOG_FILE
@@ -45,17 +44,10 @@ Returns what's in the previous section.
 
 Executes the test in text mode. No need to have QT installed in that case.
 
-``-m, --terminal``
-------------------
-
-Starts a testium interactive console. It allows to run commands and sub-tests manually
-in a console.
-
-
 ``-o, --no-color``
 ------------------
 
-Switch allowing to disable the colored output in terminal or batch modes.
+Switch allowing to disable the colored output in batch mode.
 
 ``-c, --config-file``
 ---------------------

doc/manual/sphinx/source/conf.py

@@ -14,6 +14,7 @@ import os
 
 import sys
 sys.path.insert(0, os.path.abspath('../../../../src/testium/'))
+sys.path.insert(0, os.path.abspath('../../../../src/'))
 
 
 # -- Project information -----------------------------------------------------

doc/manual/sphinx/source/helper_lib.rst

@@ -4,7 +4,12 @@ Python helper library
 ======================
 
 A python library including helper function for python modules called from
-testium.
+testium ``py_func`` items.
+
+User scripts run inside the ``py_func`` subprocess and interact with testium
+through a JSON-RPC bridge — the ``py_func.tm`` module. They must **not**
+import ``api.testium`` or ``interpreter.*`` directly: those are main-process
+modules and may not even be reachable in a packaged build (PyInstaller, .deb).
 
 To include the support of this library in a python script, the following
 line must be included in the script header:
@@ -18,58 +23,38 @@ line must be included in the script header:
 
 Global variables helper functions
 ----------------------------------
-To manage values in the global variables dataset, the following testium library API
-must be used:
+To manage values in the global variables dataset:
 
-.. automodule:: interpreter.utils.globdict
+.. automodule:: py_func.tm
    :members: gd, setgd, delgd
    :undoc-members:
    :no-index:
 
-Console helper functions
-------------------------
-
-Every opened console instance is added to a list with the
-key ``console_instances`` of the global variables.
-
-The instance is removed from the list on close step of the ``console`` test item.
-
-To manage consoles from within ``py_func`` python functions,
-the following testium library API can be used:
-
-.. automodule:: libs.testium
-   :members: add_console, remove_console, console
-   :undoc-members:
-   :no-index:
-
 Plot helper functions
 ------------------------
 
-Every opened plot window instance is added to a list with the
-key ``plot_instances`` of the global variables.
+Add values to a running plot or read the last value from it:
 
-The instance is removed from the list on close step of the ``plot`` test item.
-
-To manage plots from within ``py_func`` python functions,
-the following testium library API can be used:
-
-.. automodule:: libs.testium
-   :members: add_plot, remove_plot, plot, add_plot_values, last_plot_value
+.. automodule:: py_func.tm
+   :members: add_plot_values, last_plot_value
    :undoc-members:
    :no-index:
 
+Console and plot **lifecycle** management (``add_console``, ``remove_console``,
+``console``, ``add_plot``, ``remove_plot``, ``plot``) is performed by the
+``console`` and ``plot`` test items themselves — not from user ``py_func``
+scripts. Use those test items to open/close consoles and plots.
+
 Other helper functions
 ------------------------
 
-.. automodule:: libs.testium
-   :members: OS, get_main_dir, timestamp, timestamp_as_sec
+.. automodule:: py_func.tm
+   :members: OS, get_main_dir, init_timestamp, timestamp, timestamp_as_sec, text_mode
    :undoc-members:
    :no-index:
 
 Debug mode
 ------------------------
 
-.. automodule:: libs.testium
-   :members: debug_enabled, enable_debug, print_debug, print_info, print_warn
-   :undoc-members:
-   :no-index:
+The ``test_debug`` global variable controls debug-only output. Read or write
+it via ``tm.gd("test_debug")`` / ``tm.setgd("test_debug", True)``.

doc/manual/sphinx/source/modes.rst

@@ -23,23 +23,3 @@ graphical interface.
     :caption: call a test in batch mode
 
     testium -b test/my_test/main.tum
-
-Terminal mode
--------------
-
-The terminal mode starts *testium* in interactive mode. From this console, some tests and
-sequences of tests can be called interactively.
-
-.. code-block:: text
-    :caption: call a test in terminal mode
-
-    $ testium -m
-    Configuration file loaded: /my/execution/path/param.yaml
-    [...]
-    ================================================================================
-    ====== Test configuration
-    ================================================================================
-    Test executed with testium    : 2.4.0 (binary release)
-
-
-    (testium)~

doc/manual/sphinx/source/other_features.rst

@@ -75,7 +75,7 @@ a tooltip on the test row.
         name: Test example
         steps:
 
-            - unittest_file:
+            - unittest:
                 name: unittest item
                 doc: |
                     The purpose of this unittest test item is to demonstrate
@@ -93,4 +93,4 @@ See illustration in :numref:`Figure %s<doc-illustration>`.
 Unittest
 ^^^^^^^^^
 
-For ``unittest_file`` type test items, the python docstring of the test method is used as documentation.
+For ``unittest`` type test items, the python docstring of the test method is used as documentation.

doc/manual/sphinx/source/overview.rst

@@ -6,7 +6,7 @@ This software is developed in python and it implements the Qt6 graphical framewo
 
 It has been developed since 2013 with production and development testing in mind.
 
-It's function is to automate the execution of tests. It can be invoked either as command line terminal application or as a graphical interface application.
+It's function is to automate the execution of tests. It can be invoked either as command line application or as a graphical interface application.
 
 Tests reports generation and customization are also in this tool's scope.
 

doc/manual/sphinx/source/reports.rst

@@ -6,18 +6,25 @@ Reports
 If a report is required (in addition to the log), the ``report`` YAML element
 must be added at the root of the TUM main test file.
 
-The ``report`` YAML element has the following form:
+The ``report`` element accepts a single export or a list of them under the
+``export`` key. Each export entry uses the format name as its key:
 
 .. code-block:: yaml
-    :caption: reports global settings
+    :caption: reports global settings — multiple exports
 
     report:
         enabled: True
-        file_name: $(test_name).rep
-        path: $(home)/reports
-        pattern: "Console%"
-        export: junit
-        log_stored: False
+        log_stored: True
+        export:
+            - sqlite:
+                path: $(home)/reports
+                file_name: $(test_name).db
+            - junit:
+                path: $(home)/reports
+                file_name: $(test_name).xml
+            - html:
+                path: $(home)/reports
+                file_name: $(test_name).html
 
 .. table:: report attributes
     :widths: 20, 30, 50
@@ -27,21 +34,93 @@ The ``report`` YAML element has the following form:
     +-----------------+-----------------------+-------------------------------------------+
     | ``enabled``     | ``True``              | Report activated                          |
     +-----------------+-----------------------+-------------------------------------------+
-    | ``file_name``   | /                     | Report file name                          |
+    | ``log_stored``  | ``False``             | When ``True``, captures stdout per test   |
+    |                 |                       | item so exports (html, json) can include  |
+    |                 |                       | the log of each item.                     |
     +-----------------+-----------------------+-------------------------------------------+
-    | ``path``        | ``$(report_path)``    | Report storage path By default, it uses   |
-    |                 |                       | the default one set in the                |
-    |                 |                       | preferences.                              |
+    | ``export``      | /                     | One export entry or a list of them. Each  |
+    |                 |                       | entry's key is the format name (see       |
+    |                 |                       | below).                                   |
     +-----------------+-----------------------+-------------------------------------------+
-    | ``pattern``     | /                     | The pattern in SQL wildachars syntax      |
-    |                 |                       | to be applied on test names to            |
-    |                 |                       | selected reported tests.                  |
+
+Each export entry supports the following sub-attributes:
+
+.. table:: export attributes
+    :widths: 20, 30, 50
+
     +-----------------+-----------------------+-------------------------------------------+
-    | ``export``      | /                     | The type of export. For exemple junit.    |
-    |                 |                       | By default, the sqlite format is          |
-    |                 |                       | used to generate reports.                 |
+    | Attribute       | default value         | Description                               |
     +-----------------+-----------------------+-------------------------------------------+
-    | ``log_stored``  | /                     | Defines if the output log of each         |
-    |                 |                       | test is accessible to generate the        |
-    |                 |                       | report export.                            |
+    | ``path``        | ``$(report_path)``    | Output directory.                         |
     +-----------------+-----------------------+-------------------------------------------+
+    | ``file_name``   | /                     | Output file name. May include             |
+    |                 |                       | ``$(...)`` global-dict expansions.        |
+    +-----------------+-----------------------+-------------------------------------------+
+    | ``pattern``     | /                     | One or more SQL ``LIKE`` patterns         |
+    |                 |                       | applied on the test ``name``.             |
+    +-----------------+-----------------------+-------------------------------------------+
+    | ``key``         | /                     | One or more SQL ``LIKE`` patterns         |
+    |                 |                       | applied on the test ``key``               |
+    |                 |                       | (the per-item ``key`` attribute).         |
+    +-----------------+-----------------------+-------------------------------------------+
+
+Built-in formats
+^^^^^^^^^^^^^^^^
+
+* ``sqlite`` — raw SQLite database (storage layer; selecting it persists the run).
+* ``text``   — simple indented text dump of the test tree.
+* ``json``   — full report as JSON: ``{"header": {...}, "tests": [...]}``.
+* ``junit``  — JUnit XML (requires the ``junit_xml`` Python package).
+* ``html``   — single HTML page with header, results table and per-item logs (requires ``lxml``).
+
+If a format is unknown or its optional dependency is missing, the export is
+skipped with an ``[report] Export skipped: ...`` info line on stdout — the
+test run is **not** interrupted.
+
+.. _sec_reports_plugins:
+
+Custom export formats (plugins)
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+A third-party Python package can register additional export formats via the
+``testium.exporters`` setuptools entry point group. Once installed in the same
+Python environment as testium, the format is auto-detected at startup and can
+be referenced from the YAML by its declared name.
+
+Plugin contract — a class with this constructor signature:
+
+.. code-block:: python
+    :caption: minimal exporter contract
+
+    class MyExporter:
+        def __init__(self, name, con, path, pats, keys, no_header=False):
+            # name      : str  — report name
+            # con       : sqlite3.Connection (read) — tables: header, tests
+            # path      : str  — output file path (already expansed)
+            # pats      : list[str] — LIKE filters on test_name (may be empty)
+            # keys      : list[str] — LIKE filters on report_key (may be empty)
+            # no_header : bool — skip header section (set by the inline
+            #             `report` test item)
+            ...  # do the work in __init__ and write to `path`
+
+Tables and columns of the SQLite report:
+
+* ``header(key TEXT, value TEXT)`` — keys: ``report_version``, ``test_file``,
+  ``test_name``, ``test_result``, ``test_revision``, ``testium_version``,
+  ``testrun_date``, ``testrun_time``, ``test_duration``.
+* ``tests`` — 12 columns: ``timestamp_start``, ``test_id``, ``parent_id``,
+  ``level``, ``test_name``, ``test_type``, ``report_key``, ``result``
+  (``PASS``/``FAIL``/``SKIP``), ``message``, ``duration`` (ms),
+  ``log`` (captured stdout when ``log_stored: True``), ``data`` (JSON of
+  values reported via ``self.reportValue(...)``).
+
+Declaration in the plugin's ``pyproject.toml``:
+
+.. code-block:: toml
+    :caption: registering an exporter via entry-points
+
+    [project.entry-points."testium.exporters"]
+    my_format = "my_pkg:MyExporter"
+
+The plugin is then usable in any ``.tum`` report block as ``my_format:`` —
+no testium configuration change required.

doc/manual/sphinx/source/test_items/group_test_item.rst

@@ -10,7 +10,7 @@ This element is of the following form:
         name: Group Item
         condition: <| "$(OS)" == "Linux" |>
         steps:
-            - unittest_file:
+            - unittest:
                 test_file: test_prod_alpha_13.py
                 test_method:
                     ...

doc/manual/sphinx/source/test_items/items_common_attributes.rst

@@ -33,7 +33,7 @@ if not provided is given in the table as well.
     |                       |                   | It depends on the test item to take it                |
     |                       |                   | into account or not.                                  |
     |                       |                   | For example it makes sense to use it                  |
-    |                       |                   | for ``unittest_file`` test type                       |
+    |                       |                   | for ``unittest`` test type                            |
     |                       |                   | because it  can contain many sub-tests,               |
     |                       |                   | but not for sleep test type.                          |
     |                       |                   | In cycles, it means that the child                    |
@@ -87,6 +87,10 @@ if not provided is given in the table as well.
     |                       |                   | see :ref:`Expected result<sec_expected_result>`       |
     |                       |                   | for details.                                          |
     +-----------------------+-------------------+-------------------------------------------------------+
+    | ``store_result``      | /                 | Store the test result in a global variable.           |
+    |                       |                   | see :ref:`Store result<sec_store_result>`             |
+    |                       |                   | for details.                                          |
+    +-----------------------+-------------------+-------------------------------------------------------+
 
 
 last test result
@@ -183,6 +187,61 @@ If the result and the expected_result is equal, the test will be *PASSED* if ``T
 The special ``$(result)`` variable is replaced in the ``expected_result`` attribute content with the test result value.
 
 
+.. _sec_store_result:
+
+Store result
+-----------------------------------------------
+
+The ``store_result`` attribute stores the test result into a named global variable,
+making it available to subsequent test items via ``$(variable_name)``.
+
+If the test item returns a value (e.g. ``py_func``, ``json_rpc``), that value is stored.
+If ``process_result`` is also specified, the stored value is the post-processed result.
+
+If the test item produces no value (result is ``None``), the stored value is the
+test status string: ``"PASS"`` or ``"FAIL"``, evaluated after ``expected_result``
+but **before** ``no_fail``. This ensures the real outcome is captured even when
+``no_fail: True`` would otherwise mask a failure.
+
+.. code-block:: yaml
+    :caption: Store a function return value
+
+    - py_func:
+        name: Read sensor
+        func_name: read_temperature
+        store_result: temperature
+
+    - py_func:
+        name: Check temperature in range
+        func_name: check_range
+        param: [$(temperature), 20, 30]
+
+.. code-block:: yaml
+    :caption: Store a post-processed value
+
+    - py_func:
+        name: Get firmware version string
+        func_name: get_version
+        process_result: "'$(result)'.split('.')[0]"
+        store_result: fw_major
+
+.. code-block:: yaml
+    :caption: Store the pass/fail status of a test with no return value
+
+    - console:
+        name: Send command
+        console_name: device
+        steps:
+            - writeln: reboot
+            - read_until: {expected: "ready", timeout: 10}
+        store_result: reboot_status
+
+    - py_func:
+        name: Use reboot status
+        func_name: log_status
+        param: [$(reboot_status)]
+
+
 Export attribute
 -----------------------------------------------
 

doc/manual/sphinx/source/test_items/loop_test_item.rst

@@ -12,7 +12,7 @@ This element is of the following form:
         name: Cycle Temperature
         iterator: 10
         steps:
-            - unittest_file:
+            - unittest:
                 test_file: test_prod_rio6_8093.py
             - py_func:
                 name: function test item

doc/manual/sphinx/source/test_items/lua_func_test_item.rst

@@ -39,11 +39,14 @@ The ``lua_func`` test item is of the form:
 Beside common test items attributes, lua_func item has specific attribute, some of which being mandatory.
 
 * ``file``: the script file name that contains the function to be executed.
-  Only python script format is supported.
+  Only Lua script format is supported.
 * ``func_name``: The function name to be executed.
 * ``param``: This is a list of parameters that are passed to the function
   in the order they are presented in the script. These parameters are not
   mandatory and are highly dependent of the function prototype.
+* ``context_id``: Optional. When set, all ``lua_func`` items sharing the same
+  ``context_id`` value run inside the same persistent Lua subprocess for the
+  duration of the test. See :ref:`lua_func context<sec_lua_func_context>` for details.
 
 .. code-block:: yaml
     :caption: ``lua_func`` test item example of usage
@@ -56,16 +59,71 @@ Beside common test items attributes, lua_func item has specific attribute, some
             - $(my_param)
 
 The result of the function (after eventual post treatment) is stored in the global
-variable named ``pfn_<func_name>``
+variable named ``lfn_<item_name>``
 (See :ref:`global variables<sec_global_variables>` for more detail
 on how to access to global variables from test items and scripts).
 
 In the example above, the global variable ``$(lfn_activity)``
 would be created at the end of the item execution. It would contain the resulting
-value of the funcToBeExecuted python function.
+value of the methodName Lua function.
 
 The ``lua_func`` will always result ``PASS``, except if the called function raises
-and exception or if the ``expected_result`` attribute is used.
+an exception or if the ``expected_result`` attribute is used.
+
+.. _sec_lua_func_context:
+
+Sharing state between ``lua_func`` calls
+------------------------------------------
+
+Each ``lua_func`` item without a ``context_id`` runs in a dedicated subprocess that
+is started and stopped around the call. Module-level variables are not preserved
+between two such items.
+
+Inside a ``lua_func`` script, the ``tm`` module exposes ``tm.setgd`` and ``tm.gd``
+to read and write the testium global dictionary of the test process. Values stored
+this way are accessible from any subsequent test item without requiring a shared
+subprocess.
+
+.. code-block:: lua
+    :caption: sharing a value via the global dictionary
+
+    local tm = require("tm")
+    local module = {}
+
+    function module.produce(val)
+        tm.setgd("my_shared_value", val)
+        return val
+    end
+
+    function module.consume()
+        return tm.gd("my_shared_value")
+    end
+
+    return module
+
+When ``context_id`` is set, all ``lua_func`` items that share the same identifier
+reuse the same persistent subprocess. This allows Lua-side state (upvalues, module
+cache) to be retained across calls beyond what ``tm.setgd`` persists.
+
+.. code-block:: yaml
+    :caption: ``lua_func`` items sharing a persistent subprocess
+
+    - lua_func:
+        name: produce value
+        file: my_script.lua
+        func_name: produce
+        context_id: my_context
+        param:
+            - hello
+
+    - lua_func:
+        name: consume value
+        file: my_script.lua
+        func_name: consume
+        context_id: my_context
+        expected_result: hello
+
+The shared subprocess is automatically stopped at the end of the test run.
 
 **Lua Interpreter environment setup**
 

doc/manual/sphinx/source/test_items/parallel_test_item.rst

@@ -0,0 +1,97 @@
+.. _sec_parallel_item:
+
+**parallel** test item
+============================================================
+
+This element is of the following form:
+
+.. code-block:: yaml
+    :caption: ``parallel`` test item usage example
+
+    - parallel:
+        name: My parallel block
+        sync: all
+        branches:
+          - name: Branch A
+            steps:
+                - py_func:
+                    name: Long operation
+                    file: long_op.py
+                    func_name: do_work
+          - name: Branch B
+            wait_for:
+                condition: <| "$(ready_flag)" == "True" |>
+                timeout: 30
+            steps:
+                - let:
+                    name: Mark done
+                    values:
+                        - branch_b_done: true
+
+The ``parallel`` element runs several sequences of items concurrently. Each
+inner sequence is called a *branch* and runs in its own thread. The parent
+test item waits for branches to finish according to the ``sync`` policy.
+
+Attributes
+--------------------
+
+* ``branches``: required. A list of branches to execute concurrently. Each
+  branch has a ``name`` and a ``steps`` list (same structure as a ``group``
+  item). It can also declare a ``wait_for`` precondition (see below).
+* ``sync``: optional, defaults to ``all``.
+
+  * ``all``: the parallel item completes when *every* branch has finished.
+    The result is ``PASS`` if no branch returned ``FAIL`` (skipped or
+    disabled branches are ignored, like in ``group``); otherwise ``FAIL``.
+  * ``any``: the parallel item completes as soon as the *first* branch
+    finishes. The remaining branches are stopped (their next test items
+    are not executed). The result is ``PASS`` if at least one branch
+    succeeded.
+
+* ``no_fail``: optional. When ``true``, a ``FAIL`` result is forced to
+  ``PASS`` for the parallel item itself (same semantics as for any test
+  item). Branches keep their own result.
+
+Branch attributes
+--------------------
+
+Each entry of ``branches`` is a dict with the following attributes:
+
+* ``name``: required. The branch name. Used in reports and as a prefix
+  in the live log output (each line printed by the branch is prefixed
+  with the branch name in square brackets, e.g. ``[Branch A]``, so
+  concurrent branches stay readable).
+* ``steps``: required. The list of test items executed sequentially
+  inside the branch.
+* ``wait_for``: optional. Forces the branch to wait until a condition is
+  met before running its steps. If the timeout elapses, the branch
+  returns ``FAIL`` (the steps are not run). Sub-attributes:
+
+  * ``condition``: a testium expression evaluated repeatedly (every
+    100 ms) until it returns ``True``.
+  * ``timeout``: maximum wait, in seconds. Defaults to 30.
+
+Reporting
+--------------------
+
+Each branch produces its own row in the SQLite report (with type
+``Parallel branch``), in addition to the parent ``Parallel`` row. The
+``log`` column of each row contains only the output emitted from that
+branch's thread, so logs are never mixed between concurrent branches.
+
+In the live (terminal / GUI) output, lines emitted from a branch are
+prefixed with the branch name in square brackets (e.g. ``[Branch A]``).
+The prefix is not stored in the SQLite log column.
+
+Notes
+--------------------
+
+* A ``sleep`` item inside a branch is interruptible: if another
+  ``sync: any`` branch wins the race, slow ``sleep`` items are aborted
+  within ~50 ms.
+* A ``py_func`` or ``console`` item inside a branch is **not**
+  interruptible: a ``sync: any`` stop will only take effect after the
+  current item returns. The branch will then skip its remaining steps.
+* When a user disables a branch in the GUI tree, the branch returns
+  ``SKIP`` instantly without affecting the others (it does *not* win a
+  ``sync: any`` race).

doc/manual/sphinx/source/test_items/py_func_test_item.rst

@@ -13,7 +13,7 @@ class ``py_func`` item
 
 This is the normal way of calling some custom python code.
 
-A class must be defined and derived from ``FunctionItem`` from the ``libs.testium`` module.
+A class must be defined and derived from ``FunctionItem`` from the ``py_func.tm`` module.
 
 From this class it is possible to define some custom reported values with the following API
 
@@ -89,6 +89,9 @@ some of which being mandatory.
 * ``param``: This is a list of parameters that are passed to the function
   in the order they are presented in the script. These parameters are not
   mandatory and are highly dependent of the function prototype.
+* ``context_id``: Optional. When set, all ``py_func`` items sharing the same
+  ``context_id`` value run inside the same persistent Python subprocess for the
+  duration of the test. See :ref:`py_func context<sec_py_func_context>` for details.
 
 .. code-block:: yaml
     :caption: ``py_func`` test item example of usage
@@ -111,6 +114,67 @@ value of the funcToBeExecuted python function.
 The ``py_func`` will always result ``PASS``, except if the called function raises
 and exception or if the ``expected_result`` attribute is used.
 
+.. _sec_py_func_context:
+
+Sharing state between ``py_func`` calls
+------------------------------------------
+
+Each ``py_func`` item without a ``context_id`` runs in a dedicated subprocess that
+is started and stopped around the call. State cannot be shared between two such
+items using module-level variables.
+
+Inside a ``py_func`` script, ``tm.setgd`` and ``tm.gd`` read and write the testium
+global dictionary. Values stored this way are accessible from any subsequent test
+item, including other ``py_func`` items, without requiring a shared subprocess.
+
+.. code-block:: python
+    :caption: sharing a value via the global dictionary
+
+    import py_func.tm as tm
+
+    def produce(val):
+        tm.setgd("my_shared_value", val)
+        return val
+
+    def consume():
+        return tm.gd("my_shared_value", None)
+
+When ``context_id`` is set, all ``py_func`` items that share the same identifier
+reuse the same persistent subprocess. This allows sharing any Python object across
+calls — including objects that cannot be transmitted to other processes.
+
+.. code-block:: python
+    :caption: sharing an object via ``context_id``
+
+    import py_func.tm as tm
+
+    def open_connection():
+        tm.setgd("conn", MyConnection())
+        return "ok"
+
+    def use_connection():
+        conn = tm.gd("conn")
+        return conn.status()
+
+.. code-block:: yaml
+    :caption: ``py_func`` items sharing a persistent subprocess
+
+    - py_func:
+        name: open connection
+        file: my_script.py
+        func_name: open_connection
+        context_id: my_context
+        expected_result: ok
+
+    - py_func:
+        name: use connection
+        file: my_script.py
+        func_name: use_connection
+        context_id: my_context
+        expected_result: open
+
+The shared subprocess is automatically stopped at the end of the test run.
+
 **Python Interpreter environment setup**
 
 Some global variables have an impact on the ``py_func`` test item behavior:

doc/manual/sphinx/source/test_items/run_test_item.rst

@@ -1,16 +1,23 @@
 **run** test item
 ============================================================
 
-This test item executes a new instance of testium.
+This test item executes a new instance of testium with the specified ``.tum`` file.
+
+* In **batch mode** (``-b``): the sub-instance is started with ``-b``.
+* In **GUI mode**: the sub-instance is started with ``-r`` (run and close).
+
+The item result is **PASS** if the sub-instance launched and ran to completion,
+regardless of whether the sub-tests passed or failed.
+It is **FAIL** if the file could not be found, the sub-instance could not be
+launched, or the time window was not reached (see ``start_time`` / ``end_time``).
 
 .. code-block:: yaml
     :caption: ``run`` test item usage example
 
     - run:
         name: Execute TUM
-        tum_fime: example_cycle.tum
+        tum: example_cycle.tum
         python_bin: python3
-        testium_path: /home/francois/projets/testium-new-report/testium.pyw
         log_file: $(home)/reports/test.log
         report_file: $(home)/reports/test.rep
 
@@ -19,12 +26,12 @@ Attributes
 
 run test item has the following specific attributes:
 
-* ``tum_fime``: mandatory the path of the file to execute, it can be relative to current execution folder,
-* ``param_file`` (optional) the path of the parameter file to use, otherwise default parameter file is used.
-* ``python_bin`` (optional) the path of a specific python to run your scripts,
-* ``testium_path`` (optional) the path of a specific testium to run your scripts,
-* ``log_file`` (optional) the path of log file to register, if not provided a file is created with timestamp at the location of TUM file.
-* ``report_file`` (optional), the path of report file to create
-* ``start_time`` (optional), start time for the script execution, in HH:MM format.
-* ``end_time`` (optional), end time for an execution within a time frame, in HH:MM format.
-* ``wait_for_exec`` (optional). True or False, wait to be in the execution window defined by start_time and end_time to run the script.
+* ``tum``: mandatory, the path of the file to execute. Can be relative to the current execution folder.
+* ``param_file`` (optional): the path of the parameter file to use; otherwise the default parameter file is used.
+* ``python_bin`` (optional): the path of a specific Python interpreter to use.
+* ``testium_path`` (optional): the path of a specific testium executable to use.
+* ``log_file`` (optional): the path of the log file. In GUI mode, if not provided, a file is created with a timestamp next to the ``.tum`` file. Not used in batch mode.
+* ``report_file`` (optional): the path of the report file to create.
+* ``start_time`` (optional): earliest time to execute the sub-instance, in ``HH:MM`` format.
+* ``end_time`` (optional): latest time for execution within a time frame, in ``HH:MM`` format.
+* ``wait_for_exec`` (optional): ``true`` to wait until the time window defined by ``start_time`` and ``end_time`` is reached before running. Requires both ``start_time`` and ``end_time``.

doc/manual/sphinx/source/test_items/unittest_test_item.rst

@@ -1,15 +1,15 @@
-**unittest_file** test item
+**unittest** test item
 ============================================================
 
-unittest_file test item allows the execution of unittest test script which
+unittest test item allows the execution of unittest test script which
 is part of python standard libraries.
 
 The tum file prototype is as followed:
 
 .. code-block:: yaml
-    :caption: ``unittest_file`` test item usage example
+    :caption: ``unittest`` test item usage example
 
-    - unittest_file:
+    - unittest:
         name: unitTest test item
         test_file: unitTestScript.py
         test_method:
@@ -23,7 +23,7 @@ Beside common test items attributes, unittest test item has specific attribute,
 
 * ``test_file``: it is the name (and eventually path) of the unittest file
   to be processed.
-* ``test_method``: it is an optional unittest_file test sub-item. If one or more
+* ``test_method``: it is an optional unittest test sub-item. If one or more
   elements are present, the unittest python script file is parsed and only
   the corresponding methods are included in the test tree. Otherwise, all
   the test methods are included in the test tree.
@@ -78,4 +78,4 @@ Here is an example how to use the console module from python ``unittest``.
 
         @classmethod
         def tearDownClass(cls):
-            cls.consA0.close()
+            cls.consA0.close()

doc/manual/sphinx/source/tum_syntax.rst

@@ -255,11 +255,12 @@ step list attributes.
    test_items/let_test_item.rst
    test_items/loop_test_item.rst
    test_items/lua_func_test_item.rst
+   test_items/parallel_test_item.rst
    test_items/plot_test_item.rst
    test_items/report_test_item.rst
    test_items/run_test_item.rst
    test_items/sleep_test_item.rst
-   test_items/unittest_file_test_item.rst
+   test_items/unittest_test_item.rst
 
 
 

doc/quick_start.md

@@ -0,0 +1,66 @@
+# Quick start
+
+Five minutes from zero to a passing test.
+
+## Install
+
+From a checkout of the repository:
+
+```sh
+./run.sh --version    # Linux
+run.bat               # Windows cmd
+```
+
+The wrapper creates a Python virtual environment on first run and verifies
+testium starts. If you prefer a manual install, see the README.
+
+## Your first test
+
+Create `hello.tum`:
+
+```yaml
+main:
+    name: hello world
+    steps:
+        - check:
+            name: 1 + 1 makes 2
+            values:
+                - <| 1 + 1 == 2 |>
+```
+
+Run it in batch mode:
+
+```sh
+./run.sh -b -- hello.tum
+```
+
+You should see something like:
+
+```
+-----> step "1 + 1 makes 2" started
+Check passed
+<----- step "1 + 1 makes 2" finished: PASS
+Test run success.
+```
+
+Replace `==` with `!=` and re-run — the step now ends with **FAIL** and
+the process exits with code 1.
+
+## Open it in the GUI
+
+```sh
+./run.sh hello.tum
+```
+
+The test tree appears in the left panel; click *Run test* in the toolbar.
+Each item turns green or red live as it executes. Use `F1` on a selected
+item to open its detail panel.
+
+## Where to go next
+
+* [`doc/tutorial.md`](tutorial.md) — a guided walk-through of the most
+  common test items (`py_func`, `let`, `group`, `condition`, `report`).
+* [`doc/examples/`](examples/) — runnable `.tum` snippets covering one
+  feature each.
+* [`doc/manual/testium_manual.pdf`](manual/testium_manual.pdf) —
+  full reference manual.

doc/tutorial.md

@@ -0,0 +1,223 @@
+# Tutorial — testing a small Python utility
+
+This walk-through builds, step by step, a testium campaign that exercises
+a small Python module. Each section adds one feature; you can follow
+along by editing a single `.tum` file and re-running it.
+
+If you have not yet run testium, start with [`quick_start.md`](quick_start.md).
+
+## The code under test
+
+Create `calc.py` next to your `.tum` file:
+
+```python
+def add(a, b):
+    return a + b
+
+def divide(a, b):
+    return a / b
+```
+
+## Step 1 — a static check
+
+The simplest item is `check`: it evaluates an expression and the test
+passes iff the expression is truthy. Create `tutorial.tum`:
+
+```yaml
+main:
+    name: calc.py campaign
+    steps:
+        - check:
+            name: addition is correct
+            values:
+                - <| 2 + 3 == 5 |>
+```
+
+The `<| ... |>` markers turn the body into a Python expression evaluated
+at run time. Run it:
+
+```sh
+./run.sh -b -- tutorial.tum
+```
+
+## Step 2 — call your code with `py_func`
+
+`check` only sees Python literals; to exercise `calc.py` we need a
+`py_func` item. Replace the step:
+
+```yaml
+        - py_func:
+            name: add 2 and 3
+            file: calc.py
+            func_name: add
+            param: [2, 3]
+            expected_result: 5
+```
+
+`expected_result` makes the item PASS only when the function returns
+exactly that value.
+
+The result is also stored in the global dict under `pfn_<name>`
+(here `pfn_add 2 and 3`).
+
+Anywhere in a `.tum`, `$(key)` is replaced at runtime by the value
+stored in the global dict under `key`. A subsequent step can read the
+result back with `$(pfn_<name>)`:
+
+```yaml
+        - check:
+            name: result was 5
+            values:
+                - <| $(pfn_add 2 and 3) == 5 |>
+```
+
+## Step 3 — group several checks
+
+Wrap the steps in a `group` to keep them visually together and let
+testium report a per-group status:
+
+```yaml
+main:
+    name: calc.py campaign
+    steps:
+        - group:
+            name: add
+            steps:
+                - py_func:
+                    name: 2 + 3
+                    file: calc.py
+                    func_name: add
+                    param: [2, 3]
+                    expected_result: 5
+                - py_func:
+                    name: -1 + 1
+                    file: calc.py
+                    func_name: add
+                    param: [-1, 1]
+                    expected_result: 0
+        - group:
+            name: divide
+            steps:
+                - py_func:
+                    name: 6 / 2
+                    file: calc.py
+                    func_name: divide
+                    param: [6, 2]
+                    expected_result: 3.0
+```
+
+A group fails as soon as one of its steps fails (set
+`stop_on_failure: false` to keep going).
+
+## Step 4 — define a variable with `let`
+
+Avoid hard-coding the same number twice with a variable:
+
+```yaml
+        - let:
+            name: define numerator
+            values:
+                - num: 6
+        - py_func:
+            name: divide num by 2
+            file: calc.py
+            func_name: divide
+            param:
+                - $(num)
+                - 2
+            expected_result: 3.0
+```
+
+`$(num)` expands to the global dict entry — when the stored value is a
+number it is substituted as a number, no need to wrap it in `<| ... |>`.
+
+## Step 5 — conditional execution
+
+Skip a step when a condition is false:
+
+```yaml
+        - py_func:
+            name: divide by zero only on linux
+            condition: <| "$(os)" == "Linux" |>
+            file: calc.py
+            func_name: divide
+            param: [1, 0]
+```
+
+Items skipped this way report `SKIP` and do not affect the overall
+result.
+
+## Step 6 — generate a report
+
+Add a `report` block at the root of the file:
+
+```yaml
+main:
+    name: calc.py campaign
+    steps:
+        # ... your steps here ...
+
+report:
+    enabled: true
+    log_stored: true
+    export:
+        - junit:
+            path: ./reports
+            file_name: calc.xml
+        - html:
+            path: ./reports
+            file_name: calc.html
+```
+
+The `path` directory must exist before the test runs — testium does not
+create it. Create it once:
+
+```sh
+mkdir -p reports
+```
+
+Re-run the test — `./reports/calc.xml` (CI-friendly) and
+`./reports/calc.html` (human-friendly) are produced. Set
+`log_stored: true` to include each item's captured stdout.
+
+## Step 7 — share state between calls
+
+By default each `py_func` runs in its own short-lived subprocess.
+To keep state across calls, use `context_id`:
+
+```yaml
+        - py_func:
+            name: open
+            file: calc.py
+            func_name: open_resource
+            context_id: my_ctx
+        - py_func:
+            name: use
+            file: calc.py
+            func_name: use_resource
+            context_id: my_ctx
+```
+
+Both steps share the same persistent Python interpreter, so `calc.py`
+can store any object in module-level globals or in `tm.setgd()`.
+
+To share data without `context_id`, write it to the testium global dict
+via the JSON-RPC bridge:
+
+```python
+import py_func.tm as tm
+
+def producer():
+    tm.setgd("computed", 42)
+
+def consumer():
+    return tm.gd("computed")
+```
+
+## Where to go next
+
+* [`doc/examples/`](examples/) — one runnable `.tum` per feature
+  (cycles, dialogs, console, plots, parallel, run-of-tum, …).
+* [`doc/manual/testium_manual.pdf`](manual/testium_manual.pdf) — full
+  reference manual covering every test item, every attribute and the
+  YAML syntax extensions.

package/appimage/AppImageBuilder.yml

@@ -24,9 +24,8 @@ AppDir:
 
     runtime:
         env:
-            SEQUENCER_REV: '{{APP_VERSION}}'
+            TESTIUM_VERSION: '{{APP_VERSION}}'
             PYTHONPATH: $APPDIR/usr/lib/python3.11/site-packages:$APPDIR/usr/lib/python3.11
-            QT_QPA_PLATFORM: xcb
 
         path_mappings:
             - /usr/share/matplotlib/mpl-data/matplotlibrc:$APPDIR/etc/matplotlibrc
@@ -69,12 +68,13 @@ AppDir:
 
         # Set python 3.11 as default
         ln -fs python3.11 $TARGET_APPDIR/usr/bin/python3
-        # Install pip
-        if [ ! -f "get-pip.py" ]; then curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py; fi
+
+        # Bootstrap pip into the AppDir Python
+        if [ ! -f "get-pip.py" ]; then curl -sS https://bootstrap.pypa.io/get-pip.py -o get-pip.py; fi
         python3.11 get-pip.py --break-system-packages
 
         # Install application dependencies in AppDir
-        python3.11 -m pip install --break-system-packages --upgrade --isolated --no-input --ignore-installed --prefix=$TARGET_APPDIR/usr -r requirements.txt
+        python3.11 -m pip install --break-system-packages --upgrade --isolated --no-input --ignore-installed --prefix=$TARGET_APPDIR/usr -r ../../src/requirements.txt
 
         export PIP_CONFIG_FILE=$HOME/.pip/pip.conf
         python3.11 -m pip install --break-system-packages --upgrade --isolated --no-input --ignore-installed --prefix=$TARGET_APPDIR/usr ../../src/dist/testium-{{APP_VERSION}}-py3-none-any.whl

package/appimage/build.sh

@@ -1,12 +1,54 @@
-#!/usr/bin/bash
+#!/bin/bash
+# Build the testium AppImage inside a Debian container (Podman or Docker).
+# The resulting .AppImage file is written to this directory.
 
-export APP_VERSION=$(<../../src/VERSION)
+set -e
 
-appimage-builder --recipe AppImageBuilder.yml
+REPO_ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
+APP_VERSION="$(<"$REPO_ROOT/src/VERSION")"
 
-RESULT=$?
-if [ -n "$1" ] && [ "$1" = "install" ]; then
-    if [ $RESULT -eq 0 ]; then
-        install -v "testium-${APP_VERSION}-x86_64.AppImage" "${HOME}/.local/bin/testium"
-    fi
+if command -v podman &>/dev/null; then
+    RUNTIME=podman
+elif command -v docker &>/dev/null; then
+    RUNTIME=docker
+else
+    echo "Error: neither podman nor docker found." >&2
+    exit 1
+fi
+echo "Using $RUNTIME — building testium $APP_VERSION AppImage..."
+
+# APPIMAGE_EXTRACT_AND_RUN=1 lets appimagetool run without FUSE in the container.
+$RUNTIME run --rm \
+    --privileged \
+    -e APPIMAGE_EXTRACT_AND_RUN=1 \
+    -v "$REPO_ROOT:/work" \
+    -w /work/package/appimage \
+    debian:bookworm bash -c "
+        set -e
+        export DEBIAN_FRONTEND=noninteractive
+
+        apt-get update -qq
+        apt-get install -y -qq \
+            python3 python3-pip python3-venv python3-build \
+            dpkg-dev fakeroot squashfs-tools wget curl file binutils \
+            libglib2.0-0 patchelf zsync > /dev/null
+
+        # Build the wheel
+        cd /work/src
+        python3 -m build --wheel --outdir dist/ > /dev/null
+        cd /work/package/appimage
+
+        # Install appimage-builder
+        pip3 install appimage-builder --quiet --break-system-packages
+
+        # Run the build
+        export APP_VERSION=$APP_VERSION
+        appimage-builder --recipe AppImageBuilder.yml --skip-test
+    "
+
+APPIMAGE_FILE=$(ls -1t Testium-*-x86_64.AppImage 2>/dev/null | head -1)
+echo "Done: ${APPIMAGE_FILE}"
+
+if [ "${1}" = "install" ] && [ -n "${APPIMAGE_FILE}" ]; then
+    install -v "${APPIMAGE_FILE}" "${HOME}/.local/bin/testium"
 fi

package/appimage/requirements.txt

@@ -1,3 +0,0 @@
-tables
-pandas
-scapy

package/deb/test_distro.sh

@@ -0,0 +1,141 @@
+#!/usr/bin/env bash
+# test_distro.sh — verify testium runs on a target Debian/Ubuntu distrib.
+#
+# Spins up a Docker container of the requested image, checks which expected
+# system Python packages are available (apt), installs them, installs the
+# testium wheel, and runs a smoke test that exercises batch mode + py_func
+# subprocess.
+#
+# Usage:
+#   ./test_distro.sh debian:bookworm
+#   ./test_distro.sh debian:trixie
+#   ./test_distro.sh ubuntu:24.04
+
+set -euo pipefail
+
+IMAGE="${1:?Usage: $0 <image>  e.g. debian:bookworm | debian:trixie | ubuntu:24.04}"
+ROOT=$(realpath "$(dirname "$0")/../..")
+
+# Container runtime: prefer docker if available, fall back to podman
+if command -v docker >/dev/null 2>&1; then
+    CTR=docker
+elif command -v podman >/dev/null 2>&1; then
+    CTR=podman
+else
+    echo "ERROR: neither docker nor podman is installed" >&2
+    exit 1
+fi
+echo "[host] Using $CTR"
+
+# --- Build the wheel on the host if it does not already exist
+WHEEL_DIR="$ROOT/src/dist"
+PYTHON_HOST="$ROOT/test/tmp/.venv/bin/python3"
+[ -x "$PYTHON_HOST" ] || PYTHON_HOST=python3
+if ! ls "$WHEEL_DIR"/testium-*.whl >/dev/null 2>&1; then
+    echo "[host] Building wheel..."
+    (cd "$ROOT/src" && "$PYTHON_HOST" -m build --wheel >/dev/null)
+fi
+WHEEL=$(ls "$WHEEL_DIR"/testium-*.whl | head -1)
+WHEEL_NAME=$(basename "$WHEEL")
+echo "[host] Using $WHEEL_NAME"
+
+# Expected system Python packages on the target distrib
+APT_PACKAGES=(
+    python3
+    python3-pip
+    python3-setuptools
+    python3-pyside6.qtwidgets
+    python3-yaml
+    python3-jinja2
+    python3-colorama
+    python3-git
+    python3-pexpect
+    python3-matplotlib
+    python3-lxml
+    python3-serial
+    python3-telnetlib3
+    lua5.4
+    lua-cjson
+    lua-socket
+    git
+)
+
+echo "=== Testing on $IMAGE ==="
+
+$CTR run --rm \
+    -v "$ROOT:/testium:ro" \
+    -e WHEEL_NAME="$WHEEL_NAME" \
+    -e PACKAGES="${APT_PACKAGES[*]}" \
+    "$IMAGE" \
+    bash -c '
+        set -e
+        export DEBIAN_FRONTEND=noninteractive
+        apt-get update -qq
+
+        # 1. Availability check
+        echo
+        echo "--- System package availability ---"
+        AVAILABLE=()
+        MISSING=()
+        for pkg in $PACKAGES; do
+            if apt-cache show "$pkg" >/dev/null 2>&1; then
+                AVAILABLE+=("$pkg")
+                echo "  OK     $pkg"
+            else
+                MISSING+=("$pkg")
+                echo "  MISSING $pkg"
+            fi
+        done
+        echo
+
+        # 2. Install available packages
+        echo "--- Installing system packages ---"
+        apt-get install -qq -y --no-install-recommends "${AVAILABLE[@]}" ca-certificates >/dev/null
+
+        # 3. Map missing apt packages to their PyPI equivalents and pip-install
+        #    them as a fallback (kept minimal so the run is still a "system"
+        #    install for the most part)
+        declare -A PIP_FALLBACK=(
+            [python3-pyside6.qtwidgets]=pyside6
+            [python3-telnetlib3]=telnetlib3
+        )
+        # junit_xml has no Debian package — install it via pip so the
+        # validation post_execution.py can import it.
+        EXTRA_PIP=(junit-xml)
+        PIP_PKGS=()
+        for m in "${MISSING[@]}"; do
+            fallback="${PIP_FALLBACK[$m]:-}"
+            if [ -n "$fallback" ]; then
+                PIP_PKGS+=("$fallback")
+            fi
+        done
+        PIP_PKGS+=("${EXTRA_PIP[@]}")
+        if [ ${#PIP_PKGS[@]} -gt 0 ]; then
+            echo "--- Installing missing deps via pip: ${PIP_PKGS[*]} ---"
+            pip install --break-system-packages "${PIP_PKGS[@]}" >/dev/null
+        fi
+
+        # 4. Install testium wheel
+        echo "--- Installing testium wheel ---"
+        pip install --break-system-packages --no-deps "/testium/src/dist/$WHEEL_NAME" >/dev/null
+
+        # 5. Install the fake_exporter plugin (needed by the report_plugin
+        #    validation test which exercises entry-points discovery).
+        #    Copy it first because /testium is mounted read-only and the
+        #    setuptools backend touches its build dir.
+        echo "--- Installing testium-fake-exporter (test plugin) ---"
+        cp -r /testium/test/validation/fake_exporter /tmp/fake_exporter
+        pip install --break-system-packages /tmp/fake_exporter >/dev/null
+
+        # 6. Run the full validation suite. Outputs are streamed live so
+        #    progress is visible — the suite takes a couple of minutes.
+        #    Reports go to /tmp/testium-validation since /testium is RO.
+        echo "--- Running validation suite ---"
+        mkdir -p /tmp/testium-validation
+        cd /testium
+        testium -b -o \
+            -d "validation_report_path=/tmp/testium-validation/" \
+            -- test/validation/main.tum
+    '
+
+echo "=== $IMAGE: PASS ==="

package/flatpak/build.sh

@@ -5,4 +5,22 @@
 # flatpak install flathub org.kde.Sdk//6.10
 # flatpak install flathub io.qt.PySide.BaseApp//6.10
 
-flatpak-builder --user --verbose --force-clean --install build org.testium.Testium.yaml
+set -e
+
+# Build + install local
+flatpak-builder --user --verbose --force-clean --install --repo=repo build org.testium.Testium.yaml
+
+# Génère le bundle distribuable
+flatpak build-bundle repo testium.flatpak org.testium.Testium
+echo "Bundle généré : $(pwd)/testium.flatpak"
+
+# Crée ~/.local/bin/testium pour pouvoir taper "testium" en console
+WRAPPER="$HOME/.local/bin/testium"
+mkdir -p "$HOME/.local/bin"
+cat > "$WRAPPER" <<'EOF'
+#!/bin/sh
+exec flatpak run org.testium.Testium "$@"
+EOF
+chmod +x "$WRAPPER"
+echo "Wrapper installé : $WRAPPER"
+echo "Assurez-vous que ~/.local/bin est dans votre PATH."

package/flatpak/org.testium.Testium-mime.xml

@@ -0,0 +1,7 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<mime-info xmlns="http://www.freedesktop.org/standards/shared-mime-info">
+  <mime-type type="application/x-testium">
+    <comment>Testium test script</comment>
+    <glob pattern="*.tum"/>
+  </mime-type>
+</mime-info>

package/flatpak/org.testium.Testium.desktop

@@ -0,0 +1,10 @@
+[Desktop Entry]
+Name=Testium
+GenericName=Test Sequencer
+Comment=YAML-based test sequencer and runner
+Exec=testium %f
+Icon=org.testium.Testium
+Type=Application
+Categories=Development;
+MimeType=application/x-testium;
+StartupNotify=true

package/flatpak/org.testium.Testium.yaml

@@ -13,7 +13,9 @@ finish-args:
   - --socket=wayland
   - --device=dri
   - --share=network
-  - --filesystem=home # Optionnel : si votre testium doit lire des fichiers utilisateurs
+  - --filesystem=home
+  - --filesystem=/tmp
+  - --filesystem=host-os
 
 build-options:
   build-args:
@@ -41,18 +43,41 @@ modules:
     sources:
       - type: dir
         path: ../../src
+      - type: file
+        path: org.testium.Testium.desktop
+      - type: file
+        path: org.testium.Testium-mime.xml
+      - type: file
+        path: ../../package/testium.png
     build-commands:
-      # On installe le code source dans /app/lib/testium
+      # Code source
       - mkdir -p /app/lib
-      - cp -r . /app/lib/
+      - cp -r testium /app/lib/
+      - cp VERSION /app/lib/testium/VERSION
 
-      # Création du launcher exécutable
+      # Launcher exécutable
       - mkdir -p /app/bin
       - |
         cat <<EOF > /app/bin/testium
         #!/bin/sh
-        # On ajoute le code source et l'extension PySide6 au PYTHONPATH
+        export TESTIUM_VERSION="\$(cat /app/lib/testium/VERSION 2>/dev/null || echo unknown)"
         export PYTHONPATH="/app/lib/testium:/usr/lib/sdk/pyside6/lib/python3.13/site-packages:\$PYTHONPATH"
-        exec python3 /app/lib/testium "\$@"
+        # Expose host binaries (git, python3, lua, …) for subprocess lookups.
+        # PATH is appended (not prepended) so the main process keeps the sandbox python3.
+        export PATH="\$PATH:/run/host/usr/local/bin:/run/host/usr/bin:/run/host/bin"
+        export GIT_PYTHON_GIT_EXECUTABLE="/run/host/usr/bin/git"
+        exec /usr/bin/python3 /app/lib/testium "\$@"
         EOF
       - chmod +x /app/bin/testium
+
+      # Icône
+      - mkdir -p /app/share/icons/hicolor/256x256/apps
+      - cp testium.png /app/share/icons/hicolor/256x256/apps/org.testium.Testium.png
+
+      # Entrée menu
+      - mkdir -p /app/share/applications
+      - cp org.testium.Testium.desktop /app/share/applications/
+
+      # Type MIME pour .tum
+      - mkdir -p /app/share/mime/packages
+      - cp org.testium.Testium-mime.xml /app/share/mime/packages/

package/pyinstaller/testium.spec

@@ -1,23 +1,50 @@
 # -*- mode: python ; coding: utf-8 -*-
+import os
+
+# junit_xml is imported by post_exec scripts running under the *host* Python,
+# not the frozen interpreter — so bundling it via hiddenimports alone is not
+# enough. We also drop its source files at the _MEIPASS root so the host
+# python3 finds them via the PYTHONPATH that py_process.py sets to
+# tstium_path (= _MEIPASS when frozen).
+import junit_xml as _junit_xml
+JUNIT_XML_DIR = os.path.dirname(_junit_xml.__file__)
 
 a = Analysis(
     ['../../src/testium/__main__.py'],
     pathex=['../../src/testium',
             '../../src/testium/main_win/resources'],
     binaries=[],
-    datas=[ ('../../src/VERSION', '.'),
-            ('../../src/lua_func', 'lua_func'),
-            ('../../src/py_func', 'py_func'),
-            ('../../src/lib', 'lib')],
+    # py_func/ and runtime/ are bundled at the _MEIPASS root because the
+    # py_func subprocess is launched with the *host* Python (not the
+    # frozen interpreter): it needs the source files on disk to find them
+    # via cwd=subproc_path() and `python3 py_func` + `from runtime.*`.
+    # py_func/, lua_func/ and runtime/ are bundled at the _MEIPASS root
+    # because the py_func subprocess is launched with the *host* Python
+    # (not the frozen interpreter): it needs the source files on disk to
+    # find them via cwd=subproc_path() and `python3 py_func` +
+    # `from runtime.*`. api/ and interpreter/ are intentionally NOT
+    # exposed: user py_func scripts must go through py_func.tm
+    # (JSON-RPC bridge) for any testium API call.
+    datas=[('../../src/VERSION', '.'),
+           ('../../src/testium/lua_func', 'lua_func'),
+           ('../../src/testium/py_func', 'py_func'),
+           ('../../src/testium/runtime', 'runtime'),
+           (JUNIT_XML_DIR, 'junit_xml')],
     hiddenimports=["git",
                    "interpreter",
                    "main_win",
-                   "libs",
-                   "libs.console",
-                   "libs.termconsole",
-                   "libs.console_ssh",
-                   "libs.raw_tcp_console",
-                   "libs.runtime_plot",
+                   "runtime",
+                   "py_func",
+                   "py_func.tm",
+                   "py_func.handle",
+                   "py_func.func_call",
+                   "api",
+                   "api.console",
+                   "api.termconsole",
+                   "api.console_ssh",
+                   "api.raw_tcp_console",
+                   "api.runtime_plot",
+                   "api.testium",
                    "matplotlib.backends.backend_pdf",
                    "telnetlib3",
                    "serial",

release_note.txt

@@ -1,3 +1,20 @@
+version 0.1.2
+==============
+- Flatpak: opening a test from the GUI now correctly finds its companion
+  files (param.yaml, .py scripts, ...).
+
+version 0.1.1
+==============
+- New install channels: Flatpak bundle and AppImage. The AppImage runs
+  on any distribution (built inside a Debian container).
+- About dialog: version is now correct in Flatpak and AppImage builds
+  (used to display "unknown").
+- GUI dialogs no longer hang on pure-Wayland sessions.
+- Plot "last values" API: more tolerant timeout on loaded machines.
+- run item: `testium_path` and `python_bin` parameters removed —
+  sub-instances are launched in the same packaging mode as the parent.
+- License: EUPL-1.2.
+
 version 0.1
 ==============
 - Start of the project

schema/test.tum

@@ -0,0 +1,73 @@
+config_file:  
+  - premier
+  - saluT
+
+main:
+  name: Main file
+  steps:
+    - group:
+        name: Test
+        doc: Une peitite documentation
+        no_fail: true
+        steps:
+        - let:
+            values: 
+                - my_var: <| ${salut} |>
+        - check:
+            values: 
+            - <| ${salut} |>
+
+        - dialog_message:
+            question: c'est quoi?
+
+        - lua_func:
+            file: c'est quoi?
+            func_name: c'est quoi?
+        
+        - console:
+            console_name: cons_1
+            steps: 
+                - open:
+                    protocol: telnet
+                    terminal_path: ijfeifj
+                - read_until : {expected: "tutu", timeout: -4.5, mute: true}
+                - write: something
+                - writeln: tutu
+                - close : 
+
+        - json_rpc:
+            name: JSONRPC console Query
+            doc: JSONRPC console Query not waiting (only send)
+            console:
+                name : jsonrpc_server
+                prompt: "@@>"
+            timeout: 1
+            version: "2.0"
+            steps:
+            - query:
+                method: echo
+                params:
+                    - Hello world
+                    - [0, 1, 2, 3]
+                id: 3095372
+                no_wait: true
+        - json_rpc:
+            name: JSONRPC console Reception
+            doc: JSONRPC console reception of the previous request
+            console: {name : jsonrpc_server}
+            timeout: 1
+            steps:
+            - receive:
+                id: 3095372
+                timeout: 0.5
+
+report:
+    enabled: true
+    log_stored: true
+    export:
+        junit:
+          path: $(validation_report_path)
+          file_name: $(validation_report_file).junit
+        sqlite:
+          file_name: $(validation_report_file).sqlite
+          path: $(validation_report_path)

schema/test_schema/check.tum

@@ -0,0 +1,96 @@
+config_file:
+- param.yaml
+- items/check/param.yaml
+main:
+   name: Testium validation suite
+   steps:
+   -  group:
+         name: Test preparation
+         steps:
+         -  let:
+               condition: <| "$(os)" == "Linux" |>
+               name: Set test variables for Linux
+               values:
+               -  terminal_prompt: $(linux_prompt)
+               -  psep: /
+         -  let:
+               condition: <| "$(os)" == "Windows" |>
+               name: Set test variables for Windows
+               values:
+               -  terminal_prompt: $(windows_prompt)
+               -  psep: \
+   -  group:
+         name: Group of tests
+         steps:
+         -  let:
+               name: check test constants
+               values:
+                  - test: check
+                  - test_path: items/$(test)
+         -  group:
+               name: check test
+               steps:
+               -  sequence:
+                     steps:
+                     -  py_func:
+                           file: $(test_path)$(psep)check.py
+                           func_name: echo
+                           key: $(test)_PASS
+                           name: Dummy_int
+                           param:
+                           - 2
+                     -  py_func:
+                           file: $(test_path)$(psep)check.py
+                           func_name: echo
+                           key: $(test)_PASS
+                           name: Dummy_str
+                           param:
+                           - my taylor is rich
+                     -  check:
+                           key: $(test)_PASS
+                           name: Check condition on existing variable (PASS)
+                           values:
+                           - <| $(pfn_Dummy_int) > 1 |>
+                     -  check:
+                           key: $(test)_FAIL
+                           name: Check condition on existing variable (FAIL)
+                           values:
+                           - <| "tailor" in "$(pfn_Dummy_str)" |>
+                     filename: /home/renish/workspace/testium/code/test/validation/items/check/test.tum
+               -  sequence:
+                     steps:
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.txt
+                           -  html:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.html
+                           -  junit:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.junit
+                           name: Expected PASS $(test) test
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.txt
+                           -  html:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.html
+                           -  junit:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.junit
+                           name: Expected FAIL $(test) test
+                     filename: /home/renish/workspace/testium/code/test/validation/items/report.tum
+report:
+   enabled: true
+   export:
+      junit:
+         file_name: $(validation_report_file).junit
+         path: $(validation_report_path)
+      sqlite:
+         file_name: $(validation_report_file).sqlite
+         path: $(validation_report_path)
+   log_stored: true

schema/test_schema/common.tum

@@ -0,0 +1,453 @@
+config_file:
+- param.yaml
+- items/common/param.yaml
+main:
+   name: Testium validation suite
+   steps:
+   -  group:
+         name: Test preparation
+         steps:
+         -  let:
+               condition: <| "$(os)" == "Linux" |>
+               name: Set test variables for Linux
+               values:
+               -  terminal_prompt: $(linux_prompt)
+               -  psep: /
+         -  let:
+               condition: <| "$(os)" == "Windows" |>
+               name: Set test variables for Windows
+               values:
+               -  terminal_prompt: $(windows_prompt)
+               -  psep: \
+   -  group:
+         name: Group of tests
+         steps:
+         -  let:
+               name: common test constants
+               values:
+                  test: common
+                  test_path: items/$(test)
+         -  group:
+               name: common test
+               steps:
+               -  sequence:
+                     steps:
+                     -  group:
+                           name: Results
+                           steps:
+                           -  sequence:
+                                 steps:
+                                 -  group:
+                                       name: Expected Result
+                                       steps:
+                                       -  py_func:
+                                             expected_result: true
+                                             file: $(test_path)$(psep)results$(psep)results.py
+                                             func_name: echo
+                                             key: $(test)_PASS
+                                             name: Return True expect True
+                                             param:
+                                             - true
+                                       -  py_func:
+                                             expected_result: false
+                                             file: $(test_path)$(psep)results$(psep)results.py
+                                             func_name: echo
+                                             key: $(test)_FAIL
+                                             name: Return True expect False (must
+                                                fail)
+                                             param:
+                                             - true
+                                       -  py_func:
+                                             expected_result: None
+                                             file: $(test_path)$(psep)results$(psep)results.py
+                                             func_name: return_none
+                                             key: $(test)_PASS
+                                             name: Return None expect None
+                                       -  py_func:
+                                             expected_result: PASS
+                                             file: $(test_path)$(psep)results$(psep)results.py
+                                             func_name: return_none
+                                             key: $(test)_PASS
+                                             name: Return None expect PASS
+                                       -  py_func:
+                                             expected_result: 14
+                                             file: $(test_path)$(psep)results$(psep)results.py
+                                             func_name: return_none
+                                             key: $(test)_FAIL
+                                             name: Return None expect 14 (must fail)
+                                 -  group:
+                                       name: Expected Result Last test result
+                                       steps:
+                                       -  py_func:
+                                             file: $(test_path)$(psep)results$(psep)results.py
+                                             func_name: echo
+                                             key: $(test)_PASS
+                                             name: result is 28
+                                             param:
+                                             - 28
+                                       -  py_func:
+                                             expected_result: $(last_step_result)
+                                             file: $(test_path)$(psep)results$(psep)results.py
+                                             func_name: echo
+                                             key: $(test)_PASS
+                                             name: check that the last test result
+                                                is 28
+                                             param:
+                                             - 28
+                                 -  group:
+                                       name: Expected result Failure raised issue
+                                       steps:
+                                       -  py_func:
+                                             file: $(test_path)$(psep)results$(psep)results.py
+                                             func_name: raise_issue
+                                             key: $(test)_FAIL
+                                             name: Raise an issue (must fail)
+                                             param:
+                                             - $(str_example)
+                                       -  py_func:
+                                             expected_result: FAIL
+                                             file: $(test_path)$(psep)results$(psep)results.py
+                                             func_name: raise_issue
+                                             key: $(test)_PASS
+                                             name: Raise an issue and expected the
+                                                test to be FAIL
+                                             param:
+                                             - $(str_example)
+                                       -  py_func:
+                                             expected_result: FAIL
+                                             file: $(test_path)$(psep)results$(psep)results.py
+                                             func_name: echo
+                                             key: $(test)_FAIL
+                                             name: Return a String expect a FAILURE
+                                                (must fail)
+                                             param:
+                                             - $(str_example)
+                                 -  group:
+                                       name: process result
+                                       steps:
+                                       -  py_func:
+                                             file: $(test_path)$(psep)results$(psep)results.py
+                                             func_name: echo
+                                             key: $(test)_PASS
+                                             name: Process result equal String
+                                             param:
+                                             - $(str_example)
+                                             process_result: '''$(str_example)'' ==
+                                                ''$(result)'''
+                                       -  py_func:
+                                             expected_result: true
+                                             file: $(test_path)$(psep)results$(psep)results.py
+                                             func_name: echo
+                                             key: $(test)_FAIL
+                                             name: Process result string in the result
+                                                (must fail)
+                                             param:
+                                             - $(str_example)
+                                             process_result: '''44'' in ''$(result)'''
+                                       -  py_func:
+                                             file: $(test_path)$(psep)results$(psep)results.py
+                                             func_name: echo
+                                             key: $(test)_PASS
+                                             name: Save the result in a global variable
+                                             param:
+                                             - 44
+                                             store_result: process_result_value
+                                       -  py_func:
+                                             expected_result: $(process_result_value)
+                                             file: $(test_path)$(psep)results$(psep)results.py
+                                             func_name: echo
+                                             key: $(test)_PASS
+                                             name: Check the saved global variable
+                                             param:
+                                             - 44
+                                       -  py_func:
+                                             file: $(test_path)$(psep)results$(psep)results.py
+                                             func_name: echo
+                                             key: $(test)_PASS
+                                             name: store_result with process_result
+                                             param:
+                                             - $(str_example)
+                                             process_result: '''$(result)''.upper()'
+                                             store_result: upper_str_example
+                                       -  py_func:
+                                             expected_result: $(upper_str_example)
+                                             file: $(test_path)$(psep)results$(psep)results.py
+                                             func_name: echo
+                                             key: $(test)_PASS
+                                             name: Check store_result with process_result
+                                             param:
+                                             - $(str_example)
+                                             process_result: '''$(result)''.upper()'
+                                       -  let:
+                                             key: $(test)_PASS
+                                             name: "store_result on let item (None\
+                                                \ value \u2192 stores PASS)"
+                                             store_result: let_store_result
+                                             values:
+                                             -  dummy: 0
+                                       -  py_func:
+                                             expected_result: $(let_store_result)
+                                             file: $(test_path)$(psep)results$(psep)results.py
+                                             func_name: echo
+                                             key: $(test)_PASS
+                                             name: Check store_result on let stores
+                                                PASS
+                                             param:
+                                             - PASS
+                                       -  py_func:
+                                             expected_result: FAIL
+                                             file: $(test_path)$(psep)results$(psep)results.py
+                                             func_name: return_none
+                                             key: $(test)_FAIL
+                                             name: "store_result on failing test (None\
+                                                \ value \u2192 stores FAIL)"
+                                             store_result: none_fail_store_result
+                                       -  py_func:
+                                             expected_result: $(none_fail_store_result)
+                                             file: $(test_path)$(psep)results$(psep)results.py
+                                             func_name: echo
+                                             key: $(test)_PASS
+                                             name: Check store_result on failing test
+                                                stores FAIL
+                                             param:
+                                             - FAIL
+                                       -  py_func:
+                                             expected_result: FAIL
+                                             file: $(test_path)$(psep)results$(psep)results.py
+                                             func_name: return_none
+                                             key: $(test)_PASS
+                                             name: "store_result with no_fail (None\
+                                                \ value \u2192 stores real FAIL, not\
+                                                \ forced PASS)"
+                                             no_fail: true
+                                             store_result: none_nofail_store_result
+                                       -  py_func:
+                                             expected_result: $(none_nofail_store_result)
+                                             file: $(test_path)$(psep)results$(psep)results.py
+                                             func_name: echo
+                                             key: $(test)_PASS
+                                             name: Check store_result with no_fail
+                                                stores real FAIL
+                                             param:
+                                             - FAIL
+                                       -  py_func:
+                                             file: $(test_path)$(psep)results$(psep)results.py
+                                             func_name: return_none
+                                             key: $(test)_FAIL
+                                             name: Process result when result is None
+                                                (must fail)
+                                             process_result: $(result) is None
+                                 -  group:
+                                       name: no_fail result
+                                       steps:
+                                       -  py_func:
+                                             expected_result: false
+                                             file: $(test_path)$(psep)results$(psep)results.py
+                                             func_name: echo
+                                             key: $(test)_PASS
+                                             name: Return True expect False but no_fail=True
+                                             no_fail: true
+                                             param:
+                                             - true
+                                       -  py_func:
+                                             expected_result: false
+                                             file: $(test_path)$(psep)results$(psep)results.py
+                                             func_name: echo
+                                             key: $(test)_FAIL
+                                             name: Return True expect False but no_fail=False
+                                                (must fail)
+                                             no_fail: false
+                                             param:
+                                             - true
+                                       -  py_func:
+                                             expected_result: false
+                                             file: $(test_path)$(psep)results$(psep)results.py
+                                             func_name: echo
+                                             key: $(test)_PASS
+                                             name: Return True expect False but no_fail
+                                                expansed
+                                             no_fail: <| bool(0) == False |>
+                                             param:
+                                             - true
+                                       -  py_func:
+                                             expected_result: false
+                                             file: $(test_path)$(psep)results$(psep)results.py
+                                             func_name: echo
+                                             key: $(test)_FAIL
+                                             name: Return True expect False but no_fail
+                                                expansed (must fail)
+                                             no_fail: <| bool(1) == False |>
+                                             param:
+                                             - true
+                                 filename: /home/renish/workspace/testium/code/test/validation/items/common/results/test.tum
+                     -  group:
+                           name: Conditional
+                           steps:
+                           -  sequence:
+                                 steps:
+                                 -  loop:
+                                       doc: This loop illustrate the way to exit on
+                                          a condition.
+                                       exit_condition:
+                                          value: <| $(pfn_Echo function) > 3 |>
+                                       key: $(test)_PASS
+                                       name: Infine loop with conditional exit
+                                       steps:
+                                       -  sleep:
+                                             name: small wait
+                                             timeout: 0.2
+                                       -  py_func:
+                                             file: $(test_path)$(psep)conditional$(psep)conditional.py
+                                             func_name: echo
+                                             key: $(test)_PASS
+                                             name: Echo function
+                                             param:
+                                             - $(loop_param)
+                                       stop_on_failure: false
+                                 -  let:
+                                       key: $(test)_PASS
+                                       name: let
+                                       values:
+                                       -  conditional_exec: <| random.randint(1, 2) |>
+                                 -  console:
+                                       condition: <| $(conditional_exec) == 1 |>
+                                       console_name: consname
+                                       doc: Opening the console
+                                       key: $(test)_PASS
+                                       name: Console creation
+                                       steps:
+                                       -  open:
+                                             protocol: terminal
+                                             terminal_path: $(test_directory)
+                                       -  writeln: echo "terminal loaded"
+                                 -  console:
+                                       condition: <| $(conditional_exec) == 1 |>
+                                       console_name: consname
+                                       key: $(test)_PASS
+                                       name: Console read_until with timeout
+                                       steps:
+                                       -  read_until:
+                                             expected: terminal loaded
+                                             timeout: 5
+                                 -  console:
+                                       condition: <| $(conditional_exec) == 1 |>
+                                       console_name: consname
+                                       key: $(test)_PASS
+                                       name: Console write
+                                       steps:
+                                       -  writeln: echo 0
+                                 -  sleep:
+                                       condition: <| $(conditional_exec) == 1 |>
+                                       name: sleep item
+                                       timeout: 1
+                                 -  console:
+                                       condition: <| $(conditional_exec) == 1 |>
+                                       console_name: consname
+                                       key: $(test)_PASS
+                                       name: Console read_until immediate
+                                       steps:
+                                       -  read_until:
+                                             expected: '0'
+                                             timeout: 0
+                                 -  console:
+                                       condition: <| $(conditional_exec) == 1 |>
+                                       console_name: consname
+                                       key: $(test)_PASS
+                                       name: Console read_until immediate (2)
+                                       steps:
+                                       -  read_until:
+                                             expected: $(terminal_prompt)
+                                             timeout: 0
+                                 -  console:
+                                       condition: <| $(conditional_exec) == 1 |>
+                                       console_name: consname
+                                       key: $(test)_PASS
+                                       name: Console closure
+                                       steps:
+                                       -  close: consname
+                                 -  sleep:
+                                       condition: <| $(conditional_exec) == 2 |>
+                                       name: sleep item
+                                       timeout: 1
+                                 filename: /home/renish/workspace/testium/code/test/validation/items/common/conditional/test.tum
+                     -  group:
+                           name: Various syntax robustness
+                           steps:
+                           -  sequence:
+                                 steps:
+                                 -  sleep:
+                                       key: $(test)_PASS
+                                       timeout: 0.2
+                                 -  sleep:
+                                       key: $(test)_PASS
+                                       name: null
+                                       timeout: 0.2
+                                 -  sleep:
+                                       doc: null
+                                       key: $(test)_PASS
+                                       name: Empty "doc:" declared (must PASS)
+                                       timeout: 0.2
+                                 filename: /home/renish/workspace/testium/code/test/validation/items/common/syntax_robustness/test.tum
+                     -  group:
+                           name: Helper lib functions
+                           steps:
+                           -  py_func:
+                                 file: $(test_path)$(psep)helper_lib.py
+                                 func_name: check_os
+                                 key: $(test)_PASS
+                                 name: OS
+                                 param:
+                                 - $(os)
+                           -  py_func:
+                                 file: $(test_path)$(psep)helper_lib.py
+                                 func_name: check_get_main_dir
+                                 key: $(test)_PASS
+                                 name: get_main_dir
+                           -  py_func:
+                                 file: $(test_path)$(psep)helper_lib.py
+                                 func_name: check_timestamp_as_sec_conversion
+                                 key: $(test)_PASS
+                                 name: timestamp_as_sec conversion
+                           -  py_func:
+                                 file: $(test_path)$(psep)helper_lib.py
+                                 func_name: check_timestamp
+                                 key: $(test)_PASS
+                                 name: timestamp and timestamp_as_sec
+                     filename: /home/renish/workspace/testium/code/test/validation/items/common/test.tum
+               -  sequence:
+                     steps:
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.txt
+                           -  html:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.html
+                           -  junit:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.junit
+                           name: Expected PASS $(test) test
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.txt
+                           -  html:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.html
+                           -  junit:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.junit
+                           name: Expected FAIL $(test) test
+                     filename: /home/renish/workspace/testium/code/test/validation/items/report.tum
+report:
+   enabled: true
+   export:
+      junit:
+         file_name: $(validation_report_file).junit
+         path: $(validation_report_path)
+      sqlite:
+         file_name: $(validation_report_file).sqlite
+         path: $(validation_report_path)
+   log_stored: true

schema/test_schema/console.tum

@@ -0,0 +1,164 @@
+config_file:
+- param.yaml
+- items/console/param.yaml
+main:
+   name: Testium validation suite
+   steps:
+   -  group:
+         name: Test preparation
+         steps:
+         -  let:
+               condition: <| "$(os)" == "Linux" |>
+               name: Set test variables for Linux
+               values:
+               -  terminal_prompt: $(linux_prompt)
+               -  psep: /
+         -  let:
+               condition: <| "$(os)" == "Windows" |>
+               name: Set test variables for Windows
+               values:
+               -  terminal_prompt: $(windows_prompt)
+               -  psep: \
+   -  group:
+         name: Group of tests
+         steps:
+         -  let:
+               name: console test constants
+               values:
+                  - test: console
+                  - test_path: items/$(test)
+         -  group:
+               name: console test
+               steps:
+               -  sequence:
+                     steps:
+                     -  console:
+                           console_name: term
+                           doc: Opening the console
+                           key: $(test)_PASS
+                           name: Console creation
+                           steps:
+                           -  open:
+                                 protocol: terminal
+                                 terminal_path: $(test_directory)
+                           -  writeln: echo "endOfOpen"
+                     -  console:
+                           console_name: term
+                           key: $(test)_PASS
+                           name: Console read_until with timeout
+                           steps:
+                           -  read_until:
+                                 expected: endOfOpen
+                                 timeout: 5
+                     -  console:
+                           console_name: term
+                           key: $(test)_PASS
+                           name: Console write
+                           steps:
+                           -  writeln: echo 0
+                     -  sleep:
+                           name: sleep item
+                           timeout: 1
+                     -  console:
+                           console_name: term
+                           key: $(test)_PASS
+                           name: Console read_until immediate
+                           steps:
+                           -  read_until:
+                                 expected: '0'
+                                 timeout: 0
+                     -  console:
+                           console_name: term
+                           key: $(test)_PASS
+                           name: Console write
+                           steps:
+                           -  writeln: echo "HelloConsole"
+                     -  console:
+                           console_name: term
+                           key: $(test)_FAIL
+                           name: Console read_until fail
+                           steps:
+                           -  read_until:
+                                 expected: Something never prints
+                                 timeout: 1
+                     -  console:
+                           console_name: term
+                           key: $(test)_PASS
+                           name: Console write
+                           steps:
+                           -  writeln: echo "HelloConsole"
+                     -  console:
+                           console_name: term
+                           key: $(test)_PASS
+                           name: Console read_until no_fail
+                           steps:
+                           -  read_until:
+                                 expected: Something never prints
+                                 no_fail: true
+                                 timeout: 1
+                     -  console:
+                           console_name: term
+                           key: $(test)_PASS
+                           name: Console read_until muted
+                           steps:
+                           -  writeln: echo "HelloConsole"
+                           -  read_until:
+                                 expected: HelloConsole
+                                 mute: true
+                                 timeout: 1
+                     -  console:
+                           console_name: term
+                           key: $(test)_PASS
+                           name: Console read_until muted
+                           steps:
+                           -  writeln: echo "HelloConsole is PASS" && echo "endOfCmd"
+                           -  read_until:
+                                 expected: endOfCmd
+                                 process_result: '''Hello'' in r''''''$(result)''''''
+                                    and ''PASS'' in r''''''$(result)'''''' '
+                                 timeout: 1
+                     -  console:
+                           console_name: term
+                           execute_on_stop: true
+                           key: $(test)_PASS
+                           name: Console closure
+                           steps:
+                           -  close: term
+                     filename: /home/renish/workspace/testium/code/test/validation/items/console/test.tum
+               -  sequence:
+                     steps:
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.txt
+                           -  html:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.html
+                           -  junit:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.junit
+                           name: Expected PASS $(test) test
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.txt
+                           -  html:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.html
+                           -  junit:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.junit
+                           name: Expected FAIL $(test) test
+                     filename: /home/renish/workspace/testium/code/test/validation/items/report.tum
+report:
+   enabled: true
+   export:
+      junit:
+         file_name: $(validation_report_file).junit
+         path: $(validation_report_path)
+      sqlite:
+         file_name: $(validation_report_file).sqlite
+         path: $(validation_report_path)
+   log_stored: true

schema/test_schema/cycle.tum

@@ -0,0 +1,106 @@
+config_file:
+- param.yaml
+- items/cycle/param.yaml
+main:
+   name: Testium validation suite
+   steps:
+   -  group:
+         name: Test preparation
+         steps:
+         -  let:
+               condition: <| "$(os)" == "Linux" |>
+               name: Set test variables for Linux
+               values:
+                  terminal_prompt: $(linux_prompt)
+                  psep: /
+         -  let:
+               condition: <| "$(os)" == "Windows" |>
+               name: Set test variables for Windows
+               values:
+               -  terminal_prompt: $(windows_prompt)
+               -  psep: \
+   -  group:
+         name: Group of tests
+         steps:
+         -  let:
+               name: cycle test constants
+               values:
+                  - test: cycle
+                  - test_path: items/$(test)
+         -  group:
+               name: cycle test
+               steps:
+               -  sequence:
+                     steps:
+                     -  loop:
+                           iterator: 10
+                           key: $(test)_PASS
+                           name: Cycle number of loops
+                           steps:
+                           -  py_func:
+                                 file: $(test_path)$(psep)cycle.py
+                                 func_name: donothing
+                                 name: do nothing
+                     -  loop:
+                           iterator:
+                           - 12
+                           - 20
+                           - 30
+                           key: $(test)_PASS
+                           name: Cycle iterating on list
+                           steps:
+                           -  py_func:
+                                 file: $(test_path)$(psep)cycle.py
+                                 func_name: checkloopparam
+                                 name: check loop param
+                                 param:
+                                 - $(loop_param)
+                     -  loop:
+                           exit_condition:
+                              file: $(test_path)$(psep)cycle.py
+                              func_name: exitcondition
+                           key: $(test)_PASS
+                           name: Infinite loop with exit condition
+                           steps:
+                           -  py_func:
+                                 file: $(test_path)$(psep)cycle.py
+                                 func_name: donothing
+                                 name: do nothing
+                     filename: /home/renish/workspace/testium/code/test/validation/items/cycle/test.tum
+               -  sequence:
+                     steps:
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.txt
+                           -  html:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.html
+                           -  junit:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.junit
+                           name: Expected PASS $(test) test
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.txt
+                           -  html:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.html
+                           -  junit:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.junit
+                           name: Expected FAIL $(test) test
+                     filename: /home/renish/workspace/testium/code/test/validation/items/report.tum
+report:
+   enabled: true
+   export:
+      junit:
+         file_name: $(validation_report_file).junit
+         path: $(validation_report_path)
+      sqlite:
+         file_name: $(validation_report_file).sqlite
+         path: $(validation_report_path)
+   log_stored: true

schema/test_schema/dialogs.tum

@@ -0,0 +1,135 @@
+config_file:
+- param.yaml
+- items/dialogs/param.yaml
+main:
+   name: Testium validation suite
+   steps:
+   -  group:
+         name: Test preparation
+         steps:
+         -  let:
+               condition: <| "$(os)" == "Linux" |>
+               name: Set test variables for Linux
+               values:
+               -  terminal_prompt: $(linux_prompt)
+               -  psep: /
+         -  let:
+               condition: <| "$(os)" == "Windows" |>
+               name: Set test variables for Windows
+               values:
+               -  terminal_prompt: $(windows_prompt)
+               -  psep: \
+   -  group:
+         name: Group of tests
+         steps:
+         -  let:
+               name: dialogs test constants
+               values:
+                  test: dialogs
+                  test_path: items/$(test)
+         -  group:
+               name: dialogs test
+               steps:
+               -  sequence:
+                     steps:
+                     -  dialog_image:
+                           auto_result: ok
+                           condition: $(validation_dialogs)
+                           filename: $(test_path)$(psep)IMG_20140213_171455.jpg
+                           key: $(test)_PASS
+                           name: dialog image PASS
+                           question: click ok if you see the image
+                     -  dialog_image:
+                           auto_result: cancel
+                           condition: $(validation_dialogs)
+                           filename: $(test_path)$(psep)IMG_20140213_171455.jpg
+                           key: $(test)_FAIL
+                           name: dialog image FAIL
+                           question: click cancel
+                     -  dialog_references:
+                           auto_result: ok
+                           condition: $(validation_dialogs)
+                           key: $(test)_PASS
+                           name: dialog_reference PASS
+                           question: click ok
+                     -  dialog_references:
+                           auto_result: cancel
+                           condition: $(validation_dialogs)
+                           key: $(test)_FAIL
+                           name: dialog_reference FAIL
+                           question: click cancel
+                     -  dialog_value:
+                           auto_result: ok
+                           auto_value: '123'
+                           condition: $(validation_dialogs)
+                           key: $(test)_PASS
+                           name: dialog_value PASS
+                           question: enter 123 and click ok
+                     -  dialog_value:
+                           auto_result: ok
+                           condition: $(validation_dialogs)
+                           key: $(test)_FAIL
+                           name: dialog_value empty FAIL
+                           question: enter nothing and click ok
+                     -  dialog_value:
+                           auto_result: cancel
+                           condition: $(validation_dialogs)
+                           key: $(test)_FAIL
+                           name: dialog_value canceled FAIL
+                           question: enter nothing and click cancel
+                     -  dialog_message:
+                           auto_result: ok
+                           condition: $(validation_dialogs)
+                           key: $(test)_PASS
+                           name: dialog_message PASS
+                           question: click ok
+                     -  dialog_question:
+                           auto_result: 'yes'
+                           condition: $(validation_dialogs)
+                           key: $(test)_PASS
+                           name: dialog_question PASS
+                           question: click yes
+                     -  dialog_question:
+                           auto_result: 'no'
+                           condition: $(validation_dialogs)
+                           key: $(test)_FAIL
+                           name: dialog_question FAIL
+                           question: click no
+                     filename: /home/renish/workspace/testium/code/test/validation/items/dialogs/test.tum
+               -  sequence:
+                     steps:
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.txt
+                           -  html:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.html
+                           -  junit:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.junit
+                           name: Expected PASS $(test) test
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.txt
+                           -  html:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.html
+                           -  junit:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.junit
+                           name: Expected FAIL $(test) test
+                     filename: /home/renish/workspace/testium/code/test/validation/items/report.tum
+report:
+   enabled: true
+   export:
+      junit:
+         file_name: $(validation_report_file).junit
+         path: $(validation_report_path)
+      sqlite:
+         file_name: $(validation_report_file).sqlite
+         path: $(validation_report_path)
+   log_stored: true

schema/test_schema/expanse.tum

@@ -0,0 +1,130 @@
+config_file:
+- param.yaml
+- items/expanse/param.yaml
+main:
+   name: Testium validation suite
+   steps:
+   -  group:
+         name: Test preparation
+         steps:
+         -  let:
+               condition: <| "$(os)" == "Linux" |>
+               name: Set test variables for Linux
+               values:
+               -  terminal_prompt: $(linux_prompt)
+               -  psep: /
+         -  let:
+               condition: <| "$(os)" == "Windows" |>
+               name: Set test variables for Windows
+               values:
+               -  terminal_prompt: $(windows_prompt)
+               -  psep: \
+   -  group:
+         name: Group of tests
+         steps:
+         -  let:
+               name: expanse test constants
+               values:
+                  test: expanse
+                  test_path: items/$(test)
+         -  group:
+               name: expanse test
+               steps:
+               -  sequence:
+                     steps:
+                     -  check:
+                           key: $(test)_PASS
+                           name: Check variables expansion is correct (PASS)
+                           values:
+                           - <| $(expanse_index) == 1 |>
+                           - <| $(expanse_table)[$(expanse_index)] == 9012 |>
+                           - <| $(expanse_eval) == True |>
+                     -  let:
+                           key: $(test)_PASS
+                           name: Dynamic variables expansion
+                           values:
+                           -  expanse_select: <|"$(expanse_select)".replace("o", "a")|>
+                           -  expanse_index: $(expanse_index_$(expanse_select))
+                           -  expanse_table: $(expanse_table_$(expanse_select))
+                           -  expanse_eval: <|$(expanse_index) == 1|>
+                     -  check:
+                           key: $(test)_PASS
+                           name: Check variables expansion is correct (PASS)
+                           values:
+                           - <| $(expanse_index) == 0 |>
+                           - <| $(expanse_table)[$(expanse_index)] == "abcd" |>
+                           - <| $(expanse_eval) == False |>
+                     -  let:
+                           key: $(test)_PASS
+                           name: Complex variables expansion
+                           values:
+                           -  var1: expanse
+                           -  var2: var
+                           -  var3: bla
+                           -  var4: blo
+                           -  expanse_var_bla: 3
+                           -  expanse_blo_var: 5
+                           -  expanse_complex: <|<|$(expanse_$(var2)_$(var3))*6|>
+                                 + <|4*$($(var1)_$(var4)_$(var2))|>|>
+                     -  check:
+                           key: $(test)_PASS
+                           name: Check complex variables expansion is correct (PASS)
+                           values:
+                           - <| $(expanse_complex) == 38 |>
+                     -  let:
+                           key: $(test)_PASS
+                           name: Variables expansion in object
+                           values:
+                           -  expanse_key: b
+                           -  expanse_var: 3
+                           -  expanse_var_2: 6
+                           -  expanse_object:
+                              -  $(expanse_key): <|2**3|>
+                                 a: $(expanse_var_2)
+                              -  <|"bla".replace("a", "o")|>:
+                                 - <|$(expanse_var)*$(expanse_var_2)|>
+                                 - 25
+                     -  check:
+                           key: $(test)_PASS
+                           name: Check complex variables expansion is correct (PASS)
+                           values:
+                           - '<| $(expanse_object) == [{"a": 6, "b": 8}, {"blo": [18,
+                              25]}] |>'
+                     filename: /home/renish/workspace/testium/code/test/validation/items/expanse/test.tum
+               -  sequence:
+                     steps:
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.txt
+                           -  html:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.html
+                           -  junit:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.junit
+                           name: Expected PASS $(test) test
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.txt
+                           -  html:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.html
+                           -  junit:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.junit
+                           name: Expected FAIL $(test) test
+                     filename: /home/renish/workspace/testium/code/test/validation/items/report.tum
+report:
+   enabled: true
+   export:
+      junit:
+         file_name: $(validation_report_file).junit
+         path: $(validation_report_path)
+      sqlite:
+         file_name: $(validation_report_file).sqlite
+         path: $(validation_report_path)
+   log_stored: true

schema/test_schema/git.tum

@@ -0,0 +1,82 @@
+config_file:
+- param.yaml
+- items/git/param.yaml
+main:
+   name: Testium validation suite
+   steps:
+   -  group:
+         name: Test preparation
+         steps:
+         -  let:
+               condition: <| "$(os)" == "Linux" |>
+               name: Set test variables for Linux
+               values:
+               -  terminal_prompt: $(linux_prompt)
+               -  psep: /
+         -  let:
+               condition: <| "$(os)" == "Windows" |>
+               name: Set test variables for Windows
+               values:
+               -  terminal_prompt: $(windows_prompt)
+               -  psep: \
+   -  group:
+         name: Group of tests
+         steps:
+         -  let:
+               name: git test constants
+               values:
+                  test: git
+                  test_path: items/$(test)
+         -  group:
+               name: git test
+               steps:
+               -  sequence:
+                     steps:
+                     -  git:
+                           key: $(test)_PASS
+                           name: Testium repo
+                           repo: $(test_directory)
+                     -  git:
+                           key: $(test)_PASS
+                           name: Testium repo
+                           repo:
+                           - $(test_directory)
+                           - $(test_directory)
+                     filename: /home/renish/workspace/testium/code/test/validation/items/git/test.tum
+               -  sequence:
+                     steps:
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.txt
+                           -  html:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.html
+                           -  junit:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.junit
+                           name: Expected PASS $(test) test
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.txt
+                           -  html:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.html
+                           -  junit:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.junit
+                           name: Expected FAIL $(test) test
+                     filename: /home/renish/workspace/testium/code/test/validation/items/report.tum
+report:
+   enabled: true
+   export:
+      junit:
+         file_name: $(validation_report_file).junit
+         path: $(validation_report_path)
+      sqlite:
+         file_name: $(validation_report_file).sqlite
+         path: $(validation_report_path)
+   log_stored: true

schema/test_schema/include.tum

@@ -0,0 +1,129 @@
+config_file:
+- param.yaml
+- items/include/param.yaml
+main:
+   name: Testium validation suite
+   steps:
+   -  group:
+         name: Test preparation
+         steps:
+         -  let:
+               condition: <| "$(os)" == "Linux" |>
+               name: Set test variables for Linux
+               values:
+               -  terminal_prompt: $(linux_prompt)
+               -  psep: /
+         -  let:
+               condition: <| "$(os)" == "Windows" |>
+               name: Set test variables for Windows
+               values:
+               -  terminal_prompt: $(windows_prompt)
+               -  psep: \
+   -  group:
+         name: Group of tests
+         steps:
+         -  let:
+               name: include test constants
+               values:
+                  test: include
+                  test_path: items/$(test)
+         -  group:
+               name: include test
+               steps:
+               -  sequence:
+                     steps:
+                     -  sequence:
+                           steps:
+                           -  py_func:
+                                 file: $(test_path)$(psep)include.py
+                                 func_name: ValidationTest
+                                 key: $(test)_PASS
+                                 name: My first include test
+                                 param:
+                                 - $(test parameter)
+                           filename: /home/renish/workspace/testium/code/test/validation/items/include/inc
+                              no template/my first include.tum
+                     -  sequence:
+                           steps:
+                           -  py_func:
+                                 file: $(test_path)$(psep)include.py
+                                 func_name: ValidationTest
+                                 key: $(test)_PASS
+                                 name: My first include test
+                                 param:
+                                 - $(test parameter)
+                           filename: /home/renish/workspace/testium/code/test/validation/items/include/inc
+                              no template/my first include.tum
+                     -  sequence:
+                           steps:
+                           -  py_func:
+                                 file: $(test_path)$(psep)include.py
+                                 func_name: ValidationTest
+                                 key: $(test)_PASS
+                                 name: My second include test
+                                 param:
+                                 - My second include test parameter
+                           filename: /home/renish/workspace/testium/code/test/validation/items/include/inc
+                              with template/my second include.tum
+                     -  sequence:
+                           steps:
+                           -  py_func:
+                                 file: $(test_path)$(psep)include.py
+                                 func_name: ValidationTest
+                                 key: $(test)_PASS
+                                 name: My second include test
+                                 param:
+                                 - My second include test parameter
+                           filename: /home/renish/workspace/testium/code/test/validation/items/include/inc
+                              with template/my second include.tum
+                     -  let:
+                           name: Declare param for inclusion
+                           values:
+                           -  inc: Dali
+                           -  Dali_inc: Dalida
+                     -  sequence:
+                           steps:
+                           -  let:
+                                 name: Test param inclusion 1
+                                 values:
+                                 -  inclusion: $($(inc)_inc)
+                           filename: /home/renish/workspace/testium/code/test/validation/items/include/inc
+                              with template/my_3d_include.tum
+                     filename: /home/renish/workspace/testium/code/test/validation/items/include/test.tum
+               -  sequence:
+                     steps:
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.txt
+                           -  html:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.html
+                           -  junit:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.junit
+                           name: Expected PASS $(test) test
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.txt
+                           -  html:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.html
+                           -  junit:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.junit
+                           name: Expected FAIL $(test) test
+                     filename: /home/renish/workspace/testium/code/test/validation/items/report.tum
+report:
+   enabled: true
+   export:
+      junit:
+         file_name: $(validation_report_file).junit
+         path: $(validation_report_path)
+      sqlite:
+         file_name: $(validation_report_file).sqlite
+         path: $(validation_report_path)
+   log_stored: true

schema/test_schema/isolation.tum

@@ -0,0 +1,80 @@
+config_file:
+- param.yaml
+- items/isolation/param.yaml
+main:
+   name: Testium validation suite
+   steps:
+   -  group:
+         name: Test preparation
+         steps:
+         -  let:
+               condition: <| "$(os)" == "Linux" |>
+               name: Set test variables for Linux
+               values:
+               -  terminal_prompt: $(linux_prompt)
+               -  psep: /
+         -  let:
+               condition: <| "$(os)" == "Windows" |>
+               name: Set test variables for Windows
+               values:
+               -  terminal_prompt: $(windows_prompt)
+               -  psep: \
+   -  group:
+         name: Group of tests
+         steps:
+         -  let:
+               name: isolation test constants
+               values:
+                  test: isolation
+                  test_path: items/$(test)
+         -  group:
+               name: isolation test
+               steps:
+               -  sequence:
+                     steps:
+                     -  py_func:
+                           expected_result: true
+                           file: $(test_path)$(psep)check_isolation.py
+                           func_name: check_isolation
+                           key: $(test)_PASS
+                           name: py_func/lua_func do not depend on testium internals
+                           param:
+                           - $(testium_path)
+                     filename: /home/renish/workspace/testium/code/test/validation/items/isolation/test.tum
+               -  sequence:
+                     steps:
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.txt
+                           -  html:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.html
+                           -  junit:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.junit
+                           name: Expected PASS $(test) test
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.txt
+                           -  html:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.html
+                           -  junit:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.junit
+                           name: Expected FAIL $(test) test
+                     filename: /home/renish/workspace/testium/code/test/validation/items/report.tum
+report:
+   enabled: true
+   export:
+      junit:
+         file_name: $(validation_report_file).junit
+         path: $(validation_report_path)
+      sqlite:
+         file_name: $(validation_report_file).sqlite
+         path: $(validation_report_path)
+   log_stored: true

schema/test_schema/jsonrpc.tum

@@ -0,0 +1,490 @@
+config_file:
+- param.yaml
+- items/jsonrpc/param.yaml
+main:
+   name: Testium validation suite
+   steps:
+   -  group:
+         name: Test preparation
+         steps:
+         -  let:
+               condition: <| "$(os)" == "Linux" |>
+               name: Set test variables for Linux
+               values:
+               -  terminal_prompt: $(linux_prompt)
+               -  psep: /
+         -  let:
+               condition: <| "$(os)" == "Windows" |>
+               name: Set test variables for Windows
+               values:
+               -  terminal_prompt: $(windows_prompt)
+               -  psep: \
+   -  group:
+         name: Group of tests
+         steps:
+         -  let:
+               name: jsonrpc test constants
+               values:
+                  test: jsonrpc
+                  test_path: items/$(test)
+         -  group:
+               name: jsonrpc test
+               steps:
+               -  sequence:
+                     steps:
+                     -  console:
+                           console_name: jrpces
+                           doc: check if jrpc_echo_server.py is available
+                           key: $(test)_PASS
+                           name: json rpc echo server
+                           steps:
+                           -  open:
+                                 protocol: terminal
+                           -  read_until:
+                                 expected: $(terminal_prompt)
+                                 no_fail: true
+                                 timeout: 1
+                           -  writeln: test -f /home/renish/workspace/testium/code/test/validation/items/jsonrpc/jrpc_echo_server.py
+                                 && echo JRPC_OK
+                           -  read_until:
+                                 expected: JRPC_OK
+                                 no_fail: true
+                                 timeout: 2
+                     -  group:
+                           condition: <| 'JRPC_OK' in r'''$(cn_json rpc echo server)''' |>
+                           name: jsonrpc tests
+                           steps:
+                           -  console:
+                                 console_name: jrpces
+                                 key: $(test)_PASS
+                                 name: Start the json rpc echo server
+                                 steps:
+                                 -  writeln: python3 /home/renish/workspace/testium/code/test/validation/items/jsonrpc/jrpc_echo_server.py
+                                       -c /home/renish/workspace/testium/code/test/validation/items/jsonrpc/jrpces.ini
+                                 -  read_until:
+                                       expected: ready
+                                       timeout: 5
+                           -  console:
+                                 console_name: jsonrpc_server
+                                 doc: Opening the RAW TCP console
+                                 key: $(test)_PASS
+                                 name: Open the raw tcp Console
+                                 skipped: $(skip_tcp)
+                                 steps:
+                                 -  open:
+                                       protocol: rawtcp
+                                       tcp_host: localhost
+                                       tcp_port: 4321
+                           -  json_rpc:
+                                 console:
+                                    name: jsonrpc_server
+                                 key: $(test)_PASS
+                                 name: JSONRPC console Query waiting for reception
+                                 skipped: $(skip_tcp)
+                                 steps:
+                                 -  query:
+                                       method: echo
+                                       params:
+                                       - Hello World
+                                       -  a: 1
+                                          b: hello
+                                 timeout: 1
+                           -  json_rpc:
+                                 console:
+                                    name: jsonrpc_server
+                                 key: $(test)_PASS
+                                 name: JSONRPC console Query not waiting (only send)
+                                 skipped: $(skip_tcp)
+                                 steps:
+                                 -  query:
+                                       id: 3095372
+                                       method: echo
+                                       no_wait: true
+                                       params:
+                                       -  a: -1
+                                          b: olleh
+                                       - World Hello
+                                 timeout: 1
+                           -  sleep:
+                                 name: Small delay for the test
+                                 skipped: $(skip_tcp)
+                                 timeout: 1
+                           -  json_rpc:
+                                 console:
+                                    name: jsonrpc_server
+                                 key: $(test)_PASS
+                                 name: JSONRPC console Reception
+                                 skipped: $(skip_tcp)
+                                 steps:
+                                 -  receive:
+                                       id: 3095372
+                                 timeout: 1
+                           -  console:
+                                 console_name: jsonrpc_server
+                                 doc: Opening the RAW TCP console
+                                 execute_on_stop: true
+                                 key: $(test)_PASS
+                                 name: Close the raw tcp console
+                                 skipped: $(skip_tcp)
+                                 steps:
+                                 -  close: null
+                           -  json_rpc:
+                                 key: $(test)_PASS
+                                 name: JSONRPC UDP query waiting for reception
+                                 steps:
+                                 -  open: null
+                                 -  query:
+                                       method: echo
+                                       name: echo
+                                       params:
+                                       - Hello World
+                                       -  a: 1
+                                          b: hello
+                                       timeout: 1
+                                 -  close: null
+                                 timeout: 1
+                                 udp:
+                                    rcv_port: 8765
+                                    server: localhost
+                                    snd_port: 4323
+                           -  json_rpc:
+                                 key: $(test)_FAIL
+                                 name: Failing JSONRPC UDP query waiting for reception
+                                    (returning an error)
+                                 steps:
+                                 -  open: null
+                                 -  query:
+                                       method: echo2
+                                       params:
+                                       - Hello World
+                                       -  a: 1
+                                          b: hello
+                                       timeout: 1
+                                 -  close: null
+                                 timeout: 1
+                                 udp:
+                                    rcv_port: 8765
+                                    server: localhost
+                                    snd_port: 4323
+                           -  json_rpc:
+                                 key: $(test)_PASS
+                                 name: JSONRPC UDP query waiting for reception of
+                                    an expected error
+                                 steps:
+                                 -  open: null
+                                 -  query:
+                                       expected_result:
+                                          code: -32000
+                                          message: function not found
+                                       method: echo2
+                                       params:
+                                       - Hello World
+                                       -  a: 1
+                                          b: hello
+                                 -  close: null
+                                 timeout: 1
+                                 udp:
+                                    rcv_port: 8765
+                                    server: localhost
+                                    snd_port: 4323
+                           -  json_rpc:
+                                 doc: 'Failing JSONRPC UDP query waiting for reception
+                                    and checking result
+
+                                    and timeout elapses (wrong udp port)
+
+                                    '
+                                 key: $(test)_FAIL
+                                 name: Failing UDP JSONRPC query timeout elapses (wrong
+                                    udp port)
+                                 steps:
+                                 -  open: null
+                                 -  query:
+                                       method: echo
+                                       params:
+                                       - Hello World
+                                       -  a: 1
+                                          b: hello
+                                       timeout: 0.5
+                                 -  close: null
+                                 timeout: 1
+                                 udp:
+                                    rcv_port: 48393
+                                    server: localhost
+                                    snd_port: 4326
+                           -  json_rpc:
+                                 key: $(test)_PASS
+                                 name: JSONRPC UDP query not waiting (only send)
+                                 steps:
+                                 -  open: null
+                                 -  query:
+                                       id: 3095372
+                                       method: echo
+                                       no_wait: true
+                                       params:
+                                       -  a: -1
+                                          b: olleh
+                                       - World Hello
+                                 timeout: 1
+                                 udp:
+                                    rcv_port: 8765
+                                    server: localhost
+                                    snd_port: 4323
+                           -  sleep:
+                                 name: Small delay for the test
+                                 timeout: 1
+                           -  json_rpc:
+                                 key: $(test)_PASS
+                                 name: JSONRPC UDP Reception
+                                 steps:
+                                 -  receive:
+                                       id: 3095372
+                                 -  close: null
+                                 timeout: 1
+                                 udp:
+                                    rcv_port: 8765
+                                    server: localhost
+                                    snd_port: 4323
+                           -  json_rpc:
+                                 key: $(test)_PASS
+                                 name: JSONRPC UDP query not waiting (only send)
+                                 steps:
+                                 -  open: null
+                                 -  query:
+                                       id: 3095372
+                                       method: echo2
+                                       no_wait: true
+                                       params:
+                                       -  a: -1
+                                          b: olleh
+                                       - World Hello
+                                 timeout: 1
+                                 udp:
+                                    rcv_port: 8765
+                                    server: localhost
+                                    snd_port: 4323
+                           -  sleep:
+                                 name: Small delay for the test
+                                 timeout: 1
+                           -  json_rpc:
+                                 key: $(test)_FAIL
+                                 name: Failing JSONRPC UDP Reception (returning an
+                                    error)
+                                 steps:
+                                 -  receive:
+                                       id: 3095372
+                                       timeout: 1
+                                 -  close: null
+                                 timeout: 1
+                                 udp:
+                                    rcv_port: 8765
+                                    server: localhost
+                                    snd_port: 4323
+                           -  json_rpc:
+                                 doc: JSONRPC UDP query waiting for reception and
+                                    checking result
+                                 key: $(test)_PASS
+                                 name: UDP JSONRPC query waiting and checking
+                                 steps:
+                                 -  open: null
+                                 -  query:
+                                       expected_result:
+                                       -  - Hello World
+                                          -  a: 1
+                                             b: hello
+                                       - {}
+                                       method: echo
+                                       params:
+                                       - Hello World
+                                       -  a: 1
+                                          b: hello
+                                       timeout: 1
+                                 -  close: null
+                                 timeout: 1
+                                 udp:
+                                    rcv_port: 48393
+                                    server: localhost
+                                    snd_port: 4323
+                           -  json_rpc:
+                                 doc: JSONRPC UDP query waiting for reception and
+                                    checking result
+                                 key: $(test)_FAIL
+                                 name: Failing UDP JSONRPC query waiting and checking
+                                 steps:
+                                 -  open: null
+                                 -  query:
+                                       expected_result:
+                                       - []
+                                       - {}
+                                       method: echo
+                                       params:
+                                       - Hello World
+                                       -  a: 1
+                                          b: hello
+                                       timeout: 1
+                                 -  close: null
+                                 timeout: 1
+                                 udp:
+                                    rcv_port: 48393
+                                    server: localhost
+                                    snd_port: 4323
+                           -  json_rpc:
+                                 doc: JSONRPC UDP query not waiting, with the purpose
+                                    to check the result at reception
+                                 key: $(test)_PASS
+                                 name: UDP JSONRPC query not waiting (for checking)
+                                 steps:
+                                 -  open: null
+                                 -  query:
+                                       id: 3095372
+                                       method: echo
+                                       no_wait: true
+                                       params:
+                                       -  a: -1
+                                          b: olleh
+                                       - World Hello
+                                 timeout: 1
+                                 udp:
+                                    rcv_port: 9876
+                                    server: localhost
+                                    snd_port: 4323
+                           -  sleep:
+                                 name: Small delay for the test
+                                 timeout: 1
+                           -  json_rpc:
+                                 doc: JSONRPC UDP Reception and checking result
+                                 key: $(test)_PASS
+                                 name: UDP JSONRPC reception checking
+                                 steps:
+                                 -  receive:
+                                       expected_result:
+                                       -  -  a: -1
+                                             b: olleh
+                                          - World Hello
+                                       - {}
+                                       id: 3095372
+                                       timeout: 1
+                                 -  close: null
+                                 timeout: 1
+                                 udp:
+                                    rcv_port: 9876
+                                    server: localhost
+                                    snd_port: 4323
+                           -  json_rpc:
+                                 doc: JSONRPC UDP Reception and checking result
+                                 key: $(test)_FAIL
+                                 name: Failing UDP JSONRPC reception checking
+                                 steps:
+                                 -  receive:
+                                       expected_result:
+                                       -  -  a: -1
+                                             b: ollhe
+                                          - World Hello
+                                       - {}
+                                       id: 3095372
+                                       timeout: 1
+                                 -  close: null
+                                 timeout: 1
+                                 udp:
+                                    rcv_port: 9876
+                                    server: localhost
+                                    snd_port: 4323
+                           -  json_rpc:
+                                 doc: 'JSONRPC UDP query waiting for reception and
+                                    checking result with
+
+                                    replacing $(result) and evaluating string.
+
+                                    '
+                                 key: $(test)_PASS
+                                 name: UDP JSONRPC query waiting and evaluating result
+                                 steps:
+                                 -  open: null
+                                 -  query:
+                                       expected_result: 1
+                                       method: echo
+                                       params:
+                                       - Hello World
+                                       -  a: 1
+                                          b: hello
+                                       process_result: $(result)[0][1]['a']
+                                       timeout: 1
+                                 -  close: null
+                                 timeout: 1
+                                 udp:
+                                    rcv_port: 48393
+                                    server: localhost
+                                    snd_port: 4323
+                           -  json_rpc:
+                                 doc: 'JSONRPC UDP query waiting for reception and
+                                    checking result with
+
+                                    replacing $(result) and evaluating string.
+
+                                    '
+                                 key: $(test)_FAIL
+                                 name: Failing UDP JSONRPC query waiting and evaluating
+                                    result
+                                 steps:
+                                 -  open: null
+                                 -  query:
+                                       expected_result: $(result)[0][1]['a'] == 0
+                                       method: echo
+                                       params:
+                                       - Hello World
+                                       -  a: 1
+                                          b: hello
+                                       timeout: 1
+                                 -  close: null
+                                 timeout: 1
+                                 udp:
+                                    rcv_port: 48393
+                                    server: localhost
+                                    snd_port: 4323
+                     -  console:
+                           console_name: jrpces
+                           doc: check if the jsonrpc echo server is installed
+                           execute_on_stop: true
+                           key: $(test)_PASS
+                           name: Stop json rpc echo server
+                           steps:
+                           -  close:
+                                 protocol: terminal
+                     filename: /home/renish/workspace/testium/code/test/validation/items/jsonrpc/test.tum
+               -  sequence:
+                     steps:
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.txt
+                           -  html:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.html
+                           -  junit:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.junit
+                           name: Expected PASS $(test) test
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.txt
+                           -  html:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.html
+                           -  junit:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.junit
+                           name: Expected FAIL $(test) test
+                     filename: /home/renish/workspace/testium/code/test/validation/items/report.tum
+report:
+   enabled: true
+   export:
+      junit:
+         file_name: $(validation_report_file).junit
+         path: $(validation_report_path)
+      sqlite:
+         file_name: $(validation_report_file).sqlite
+         path: $(validation_report_path)
+   log_stored: true

schema/test_schema/let.tum

@@ -0,0 +1,157 @@
+config_file:
+- param.yaml
+- items/let/param.yaml
+main:
+   name: Testium validation suite
+   steps:
+   -  group:
+         name: Test preparation
+         steps:
+         -  let:
+               condition: <| "$(os)" == "Linux" |>
+               name: Set test variables for Linux
+               values:
+               -  terminal_prompt: $(linux_prompt)
+               -  psep: /
+         -  let:
+               condition: <| "$(os)" == "Windows" |>
+               name: Set test variables for Windows
+               values:
+               -  terminal_prompt: $(windows_prompt)
+               -  psep: \
+   -  group:
+         name: Group of tests
+         steps:
+         -  let:
+               name: let test constants
+               values:
+                  - test: let
+                  - test_path: items/$(test)
+         -  group:
+               name: let test
+               steps:
+               -  sequence:
+                     steps:
+                     -  loop:
+                           iterator: 10
+                           key: $(test)_PASS
+                           name: Cycle number of loops
+                           steps:
+                           -  py_func:
+                                 file: $(test_path)$(psep)let.py
+                                 func_name: donothing
+                                 name: do nothing
+                           -  let:
+                                 name: Let it be
+                                 values:
+                                    - be: <| $(loop_param) == $(it) |>
+                                    - it: $(loop_param)
+                     -  loop:
+                           iterator:
+                           - 12
+                           - 20
+                           - 30
+                           key: $(test)_PASS
+                           name: Cycle iterating on list
+                           steps:
+                           -  py_func:
+                                 file: $(test_path)$(psep)let.py
+                                 func_name: checkloopparam
+                                 name: check loop param
+                                 param:
+                                 - $(loop_param)
+                           -  let:
+                                 name: Let it be
+                                 values:
+                                 -  it: $(loop_param)
+                                 -  be: <| $(loop_param) == $(it) |>
+                     -  let:
+                           key: $(test)_PASS
+                           name: Get time
+                           values:
+                           -  loop_t0: $(ts_start_Cycle iterating on list)
+                           -  loop_t1: $(ts_end_Cycle iterating on list)
+                           -  loop_duration: $(ts_duration_Cycle iterating on list)
+                     -  let:
+                           key: $(test)_PASS
+                           name: Get parameter file value
+                           values:
+                           -  test_overwrite_me: <| $(overwrite_me) == True |>
+                     -  py_func:
+                           file: $(test_path)$(psep)let.py
+                           func_name: checkGlobalDic
+                           name: Check global dic pass
+                           param:
+                           - test_overwrite_me
+                           - true
+                     -  let:
+                           key: $(test)_PASS
+                           name: Overwrite parameter file value
+                           values:
+                           -  overwrite_me: false
+                     -  py_func:
+                           expected_result: $(overwrite_me) == False
+                           file: $(test_path)$(psep)let.py
+                           func_name: checkGlobalDic
+                           key: $(test)_FAIL
+                           name: Check global dic fail
+                           param:
+                           - overwrite_me
+                           - true
+                     -  py_func:
+                           expected_result: fail
+                           file: $(test_path)$(psep)let.py
+                           func_name: checkGlobalDic
+                           key: $(test)_PASS
+                           name: Check global dic fail
+                           param:
+                           - overwrite_me
+                           - true
+                     -  let:
+                           key: $(test)_PASS
+                           name: Evaluate Overwriting parameter file value
+                           values:
+                           -  test_overwrite_me: <| "$(overwrite_me)" == True |>
+                     -  check:
+                           key: $(test)_PASS
+                           name: Check Overwriting parameter file value
+                           values:
+                           - <| $(test_overwrite_me) == False |>
+                     filename: /home/renish/workspace/testium/code/test/validation/items/let/test.tum
+               -  sequence:
+                     steps:
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.txt
+                           -  html:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.html
+                           -  junit:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.junit
+                           name: Expected PASS $(test) test
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.txt
+                           -  html:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.html
+                           -  junit:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.junit
+                           name: Expected FAIL $(test) test
+                     filename: /home/renish/workspace/testium/code/test/validation/items/report.tum
+report:
+   enabled: true
+   export:
+      junit:
+         file_name: $(validation_report_file).junit
+         path: $(validation_report_path)
+      sqlite:
+         file_name: $(validation_report_file).sqlite
+         path: $(validation_report_path)
+   log_stored: true

schema/test_schema/lua_func.tum

@@ -0,0 +1,297 @@
+config_file:
+- param.yaml
+- items/lua_func/param.yaml
+main:
+   name: Testium validation suite
+   steps:
+   -  group:
+         name: Test preparation
+         steps:
+         -  let:
+               condition: <| "$(os)" == "Linux" |>
+               name: Set test variables for Linux
+               values:
+               -  terminal_prompt: $(linux_prompt)
+               -  psep: /
+         -  let:
+               condition: <| "$(os)" == "Windows" |>
+               name: Set test variables for Windows
+               values:
+               -  terminal_prompt: $(windows_prompt)
+               -  psep: \
+   -  group:
+         name: Group of tests
+         steps:
+         -  let:
+               name: lua_func test constants
+               values:
+                  test: lua_func
+                  test_path: items/$(test)
+         -  group:
+               name: lua_func test
+               steps:
+               -  sequence:
+                     steps:
+                     -  let:
+                           name: lua_func test constants,
+                           values:
+                              lua_func test parameter: test parameter lua_func
+                     -  lua_func:
+                           file: $(test_path)$(psep)lua_func.lua
+                           func_name: assertparam
+                           key: $(test)_FAIL
+                           name: fail lua_func
+                           param:
+                           - false
+                     -  lua_func:
+                           expected_result: FAIL
+                           file: $(test_path)$(psep)lua_func.lua
+                           func_name: assertparam
+                           key: $(test)_PASS
+                           name: fail lua_func with expected result FAIL
+                           param:
+                           - false
+                     -  lua_func:
+                           expected_result: FAIL
+                           file: $(test_path)$(psep)lua_func.lua
+                           func_name: assertparam
+                           key: $(test)_FAIL
+                           name: pass lua_func with expected result FAIL
+                           param:
+                           - true
+                     -  lua_func:
+                           expected_result: -1
+                           file: $(test_path)$(psep)lua_func.lua
+                           func_name: echo
+                           key: $(test)_PASS
+                           name: expected -1
+                           param:
+                           - -1
+                     -  lua_func:
+                           expected_result: 354848436 - 354848437
+                           file: $(test_path)$(psep)lua_func.lua
+                           func_name: echo
+                           key: $(test)_PASS
+                           name: expected eval
+                           param:
+                           - -1
+                     -  lua_func:
+                           expected_result: '[-1, ''a'', {''toto'': ''tata''}]'
+                           file: $(test_path)$(psep)lua_func.lua
+                           func_name: echo
+                           key: $(test)_PASS
+                           name: expected table
+                           param:
+                           -  - -1
+                              - a
+                              -  toto: tata
+                     -  lua_func:
+                           expected_result: $(lua_func test parameter)
+                           file: $(test_path)$(psep)lua_func.lua
+                           func_name: checkglobal
+                           key: $(test)_PASS
+                           name: global param lua_func
+                           param:
+                           - lua_func test parameter
+                     -  lua_func:
+                           expected_result: ($(lua_data_to_be_returned))[0]
+                           file: $(test_path)$(psep)lua_func.lua
+                           func_name: checkglobal2
+                           key: $(test)_PASS
+                           name: global param lua_func 1
+                           param:
+                           - 1
+                     -  lua_func:
+                           expected_result: ($(lua_data_to_be_returned))[1]
+                           file: $(test_path)$(psep)lua_func.lua
+                           func_name: checkglobal2
+                           key: $(test)_PASS
+                           name: global param lua_func 2
+                           param:
+                           - 2
+                     -  lua_func:
+                           expected_result: ($(lua_data_to_be_returned))[2]
+                           file: $(test_path)$(psep)lua_func.lua
+                           func_name: checkglobal2
+                           key: $(test)_PASS
+                           name: global param lua_func 3
+                           param:
+                           - 3
+                     -  let:
+                           key: $(test)_PASS
+                           name: python2func
+                           values:
+                           -  py: $(test_path)$(psep)lua_func.lua
+                     -  lua_func:
+                           file: $(test_path)$(psep)lua_func.lua
+                           func_name: should_not_be_called
+                           name: skipped_checkglobal
+                           param:
+                           - $(test parameter)
+                     -  lua_func:
+                           file: $(test_path)$(psep)lua_func.lua
+                           func_name: checkglobal
+                           name: skipped true
+                           param:
+                           - $(test parameter)
+                           skipped: true
+                     -  lua_func:
+                           file: $(test_path)$(psep)lua_func.lua
+                           func_name: checkglobal
+                           name: skipped 1
+                           param:
+                           - $(test parameter)
+                           skipped: true
+                     -  group:
+                           name: Function results check
+                           steps:
+                           -  group:
+                                 name: Function result failure
+                                 steps:
+                                 -  lua_func:
+                                       file: $(test_path)$(psep)lua_func.lua
+                                       func_name: echo
+                                       key: $(test)_PASS
+                                       name: int failure
+                                       param:
+                                       - -1
+                                 -  lua_func:
+                                       file: $(test_path)$(psep)lua_func.lua
+                                       func_name: echo
+                                       key: $(test)_PASS
+                                       name: float failure
+                                       param:
+                                       - -1.3
+                                 -  lua_func:
+                                       file: $(test_path)$(psep)lua_func.lua
+                                       func_name: echo
+                                       key: $(test)_PASS
+                                       name: String failure
+                                       param:
+                                       - FAIL
+                                 -  lua_func:
+                                       file: $(test_path)$(psep)lua_func.lua
+                                       func_name: tuple_return
+                                       key: $(test)_PASS
+                                       name: Tuple int,str failure
+                                       param:
+                                       - -1
+                                       - Got a failure
+                           -  group:
+                                 name: Functions result success
+                                 steps:
+                                 -  lua_func:
+                                       file: $(test_path)$(psep)lua_func.lua
+                                       func_name: echo
+                                       key: $(test)_PASS
+                                       name: int success
+                                       param:
+                                       - 0
+                                 -  lua_func:
+                                       file: $(test_path)$(psep)lua_func.lua
+                                       func_name: echo
+                                       key: $(test)_PASS
+                                       name: float success
+                                       param:
+                                       - 0.3
+                                 -  lua_func:
+                                       file: $(test_path)$(psep)lua_func.lua
+                                       func_name: echo
+                                       key: $(test)_PASS
+                                       name: String success
+                                       param:
+                                       - Something that is not only strictly FAIL
+                                 -  lua_func:
+                                       file: $(test_path)$(psep)lua_func.lua
+                                       func_name: tuple_return
+                                       key: $(test)_PASS
+                                       name: Tuple int,str success
+                                       param:
+                                       - 0
+                                       - OK
+                     -  lua_func:
+                           file: $(test_path)$(psep)lua_func.lua
+                           func_name: test_delgd
+                           key: $(test)_PASS
+                           name: delgd test
+                     -  lua_func:
+                           file: $(test_path)$(psep)lua_func.lua
+                           func_name: return_nothing
+                           key: $(test)_PASS
+                           name: function returning nothing should succeed
+                     -  lua_func:
+                           file: $(test_path)$(psep)lua_func.lua
+                           func_name: return_explicit_nil
+                           key: $(test)_PASS
+                           name: function returning explicit nil should succeed
+                     -  group:
+                           name: context_id tests
+                           steps:
+                           -  lua_func:
+                                 context_id: lua_ctx_test
+                                 expected_result: hello lua
+                                 file: $(test_path)$(psep)lua_func.lua
+                                 func_name: set_context_value
+                                 key: $(test)_PASS
+                                 name: set context value
+                                 param:
+                                 - hello lua
+                           -  lua_func:
+                                 context_id: lua_ctx_test
+                                 expected_result: hello lua
+                                 file: $(test_path)$(psep)lua_func.lua
+                                 func_name: get_context_value
+                                 key: $(test)_PASS
+                                 name: get context value (same context_id)
+                           -  lua_func:
+                                 expected_result: hello lua
+                                 file: $(test_path)$(psep)lua_func.lua
+                                 func_name: get_context_value
+                                 key: $(test)_PASS
+                                 name: get context value (no context_id, from main
+                                    gd)
+                           -  lua_func:
+                                 context_id: lua_ctx_other
+                                 expected_result: hello lua
+                                 file: $(test_path)$(psep)lua_func.lua
+                                 func_name: get_context_value
+                                 key: $(test)_PASS
+                                 name: get context value (different context_id)
+                     filename: /home/renish/workspace/testium/code/test/validation/items/lua_func/test.tum
+               -  sequence:
+                     steps:
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.txt
+                           -  html:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.html
+                           -  junit:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.junit
+                           name: Expected PASS $(test) test
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.txt
+                           -  html:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.html
+                           -  junit:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.junit
+                           name: Expected FAIL $(test) test
+                     filename: /home/renish/workspace/testium/code/test/validation/items/report.tum
+report:
+   enabled: true
+   export:
+      junit:
+         file_name: $(validation_report_file).junit
+         path: $(validation_report_path)
+      sqlite:
+         file_name: $(validation_report_file).sqlite
+         path: $(validation_report_path)
+   log_stored: true

schema/test_schema/parallel.tum

@@ -0,0 +1,414 @@
+config_file:
+- param.yaml
+- items/parallel/param.yaml
+main:
+   name: Testium validation suite
+   steps:
+   -  group:
+         name: Test preparation
+         steps:
+         -  let:
+               condition: <| "$(os)" == "Linux" |>
+               name: Set test variables for Linux
+               values:
+               -  terminal_prompt: $(linux_prompt)
+               -  psep: /
+         -  let:
+               condition: <| "$(os)" == "Windows" |>
+               name: Set test variables for Windows
+               values:
+               -  terminal_prompt: $(windows_prompt)
+               -  psep: \
+   -  group:
+         name: Group of tests
+         steps:
+         -  let:
+               name: parallel test constants
+               values:
+                  test: parallel
+                  test_path: items/$(test)
+         -  group:
+               name: parallel test
+               steps:
+               -  sequence:
+                     steps:
+                     -  parallel:
+                           branches:
+                           -  name: Branch A
+                              steps:
+                              -  let:
+                                    name: Set A done
+                                    values:
+                                    -  branch_a_done: true
+                           -  name: Branch B
+                              steps:
+                              -  let:
+                                    name: Set B done
+                                    values:
+                                    -  branch_b_done: true
+                           key: $(test)_PASS
+                           name: Both branches pass
+                           sync: all
+                     -  check:
+                           key: $(test)_PASS
+                           name: Both branches ran
+                           values:
+                           - <| $(branch_a_done) == True |>
+                           - <| $(branch_b_done) == True |>
+                     -  parallel:
+                           branches:
+                           -  name: Pass branch
+                              steps:
+                              -  let:
+                                    name: Set pass flag
+                                    values:
+                                    -  pass_branch_ran: true
+                           -  name: Fail branch
+                              steps:
+                              -  py_func:
+                                    expected_result: fail
+                                    file: $(test_path)$(psep)parallel.py
+                                    func_name: sleep_func
+                                    name: Raise exception
+                                    param:
+                                    - 0
+                           key: $(test)_PASS
+                           name: One branch fails
+                           no_fail: true
+                           sync: all
+                     -  check:
+                           key: $(test)_PASS
+                           name: Pass branch still ran
+                           values:
+                           - <| $(pass_branch_ran) == True |>
+                     -  let:
+                           name: Reset slow flag
+                           values:
+                           -  slow_done: false
+                     -  parallel:
+                           branches:
+                           -  name: Fast branch
+                              steps:
+                              -  let:
+                                    name: Fast done
+                                    values:
+                                    -  fast_done: true
+                           -  name: Slow branch
+                              steps:
+                              -  py_func:
+                                    file: $(test_path)$(psep)parallel.py
+                                    func_name: sleep_func
+                                    name: Sleep 2s
+                                    param:
+                                    - 2
+                              -  let:
+                                    name: Slow done
+                                    values:
+                                    -  slow_done: true
+                           key: $(test)_PASS
+                           name: sync any - first wins
+                           sync: any
+                     -  check:
+                           key: $(test)_PASS
+                           name: Fast branch ran, slow branch was stopped
+                           values:
+                           - <| $(fast_done) == True |>
+                           - <| $(slow_done) == False |>
+                     -  let:
+                           name: Reset sync flag
+                           values:
+                           -  sync_flag: ''
+                           -  waiter_ran: false
+                     -  parallel:
+                           branches:
+                           -  name: Setter branch
+                              steps:
+                              -  py_func:
+                                    file: $(test_path)$(psep)parallel.py
+                                    func_name: sleep_func
+                                    name: Sleep 0.3s then set flag
+                                    param:
+                                    - 0.3
+                              -  let:
+                                    name: Set sync flag
+                                    values:
+                                    -  sync_flag: ready
+                           -  name: Waiter branch
+                              steps:
+                              -  let:
+                                    name: Got flag
+                                    values:
+                                    -  waiter_ran: true
+                              wait_for:
+                                 condition: <| "$(sync_flag)" == "ready" |>
+                                 timeout: 10
+                           key: $(test)_PASS
+                           name: wait_for synchronization
+                           sync: all
+                     -  check:
+                           key: $(test)_PASS
+                           name: Waiter branch ran after flag was set
+                           values:
+                           - <| $(waiter_ran) == True |>
+                     -  parallel:
+                           branches:
+                           -  name: Sleep A
+                              steps:
+                              -  sleep:
+                                    name: Sleep 1s A
+                                    timeout: 1
+                           -  name: Sleep B
+                              steps:
+                              -  sleep:
+                                    name: Sleep 1s B
+                                    timeout: 1
+                           key: $(test)_PASS
+                           name: Timing test
+                           sync: all
+                     -  let:
+                           name: Capture parallel duration
+                           values:
+                           -  parallel_duration: $(ts_duration_Timing test)
+                     -  check:
+                           key: $(test)_PASS
+                           name: Duration < 1.8s (would be 2s if sequential)
+                           values:
+                           - <| float("$(parallel_duration)") < 1.8 |>
+                     -  let:
+                           name: Reset N flags
+                           values:
+                           -  n_a: false
+                           -  n_b: false
+                           -  n_c: false
+                           -  n_d: false
+                     -  parallel:
+                           branches:
+                           -  name: NA
+                              steps:
+                              -  let:
+                                    name: set n_a
+                                    values:
+                                    -  n_a: true
+                           -  name: NB
+                              steps:
+                              -  let:
+                                    name: set n_b
+                                    values:
+                                    -  n_b: true
+                           -  name: NC
+                              steps:
+                              -  let:
+                                    name: set n_c
+                                    values:
+                                    -  n_c: true
+                           -  name: ND
+                              steps:
+                              -  let:
+                                    name: set n_d
+                                    values:
+                                    -  n_d: true
+                           key: $(test)_PASS
+                           name: Four branches
+                           sync: all
+                     -  check:
+                           key: $(test)_PASS
+                           name: Four branches all set their flag
+                           values:
+                           - <| $(n_a) == True |>
+                           - <| $(n_b) == True |>
+                           - <| $(n_c) == True |>
+                           - <| $(n_d) == True |>
+                     -  let:
+                           name: Reset nested flags
+                           values:
+                           -  outer_x: false
+                           -  inner_x_1: false
+                           -  inner_x_2: false
+                     -  parallel:
+                           branches:
+                           -  name: Outer X
+                              steps:
+                              -  let:
+                                    name: set outer_x
+                                    values:
+                                    -  outer_x: true
+                              -  parallel:
+                                    branches:
+                                    -  name: Inner X1
+                                       steps:
+                                       -  let:
+                                             name: set inner_x_1
+                                             values:
+                                             -  inner_x_1: true
+                                    -  name: Inner X2
+                                       steps:
+                                       -  let:
+                                             name: set inner_x_2
+                                             values:
+                                             -  inner_x_2: true
+                                    name: Inner parallel
+                                    sync: all
+                           -  name: Outer Y
+                              steps:
+                              -  sleep:
+                                    name: brief sleep
+                                    timeout: 0
+                           key: $(test)_PASS
+                           name: Outer parallel
+                           sync: all
+                     -  check:
+                           key: $(test)_PASS
+                           name: Nested parallel set all flags
+                           values:
+                           - <| $(outer_x) == True |>
+                           - <| $(inner_x_1) == True |>
+                           - <| $(inner_x_2) == True |>
+                     -  let:
+                           name: Reset waiter timeout flag
+                           values:
+                           -  waiter_timeout_ran: false
+                     -  parallel:
+                           branches:
+                           -  name: Quick branch
+                              steps:
+                              -  sleep:
+                                    name: brief sleep
+                                    timeout: 0
+                           -  name: Doomed waiter
+                              steps:
+                              -  let:
+                                    name: should not run
+                                    values:
+                                    -  waiter_timeout_ran: true
+                              wait_for:
+                                 condition: <| "never" == "ready" |>
+                                 timeout: 1
+                           key: $(test)_PASS
+                           name: wait_for timeout
+                           no_fail: true
+                           sync: all
+                     -  check:
+                           key: $(test)_PASS
+                           name: Doomed waiter never ran its steps
+                           values:
+                           - <| $(waiter_timeout_ran) == False |>
+                     -  parallel:
+                           branches:
+                           -  name: ok branch
+                              steps:
+                              -  let:
+                                    name: noop
+                                    values:
+                                    -  noop_var: 1
+                           -  name: broken branch
+                              steps:
+                              -  py_func:
+                                    expected_result: fail
+                                    file: $(test_path)$(psep)parallel.py
+                                    func_name: sleep_func
+                                    name: Forced fail
+                                    param:
+                                    - 0
+                           key: $(test)_FAIL
+                           name: One branch really fails
+                           sync: all
+                     -  let:
+                           name: Reset branch condition flag
+                           values:
+                           -  cond_branch_ran: false
+                           -  other_branch_ran: false
+                     -  parallel:
+                           branches:
+                           -  condition: <| "always" == "false" |>
+                              name: Skipped branch
+                              steps:
+                              -  let:
+                                    name: should not run
+                                    values:
+                                    -  cond_branch_ran: true
+                           -  name: Other branch
+                              steps:
+                              -  let:
+                                    name: ran
+                                    values:
+                                    -  other_branch_ran: true
+                           key: $(test)_PASS
+                           name: Condition-skipped branch
+                           sync: all
+                     -  check:
+                           key: $(test)_PASS
+                           name: Skipped condition branch did not run
+                           values:
+                           - <| $(cond_branch_ran) == False |>
+                           - <| $(other_branch_ran) == True |>
+                     -  let:
+                           name: Reset loop counters
+                           values:
+                           -  loop_count_a: 0
+                           -  loop_count_b: 0
+                     -  loop:
+                           iterator: 3
+                           name: Loop wrapping parallel
+                           steps:
+                           -  parallel:
+                                 branches:
+                                 -  name: LA
+                                    steps:
+                                    -  let:
+                                          name: bump A
+                                          values:
+                                          -  loop_count_a: <| int("$(loop_count_a)")
+                                                + 1 |>
+                                 -  name: LB
+                                    steps:
+                                    -  let:
+                                          name: bump B
+                                          values:
+                                          -  loop_count_b: <| int("$(loop_count_b)")
+                                                + 1 |>
+                                 name: Per-iteration parallel
+                                 sync: all
+                     -  check:
+                           key: $(test)_PASS
+                           name: Both branches ran 3 times
+                           values:
+                           - <| int("$(loop_count_a)") == 3 |>
+                           - <| int("$(loop_count_b)") == 3 |>
+                     filename: /home/renish/workspace/testium/code/test/validation/items/parallel/test.tum
+               -  sequence:
+                     steps:
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.txt
+                           -  html:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.html
+                           -  junit:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.junit
+                           name: Expected PASS $(test) test
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.txt
+                           -  html:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.html
+                           -  junit:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.junit
+                           name: Expected FAIL $(test) test
+                     filename: /home/renish/workspace/testium/code/test/validation/items/report.tum
+report:
+   enabled: true
+   export:
+      junit:
+         file_name: $(validation_report_file).junit
+         path: $(validation_report_path)
+      sqlite:
+         file_name: $(validation_report_file).sqlite
+         path: $(validation_report_path)
+   log_stored: true

schema/test_schema/plot.tum

@@ -0,0 +1,137 @@
+config_file:
+- param.yaml
+- items/plot/param.yaml
+main:
+   name: Testium validation suite
+   steps:
+   -  group:
+         name: Test preparation
+         steps:
+         -  let:
+               condition: <| "$(os)" == "Linux" |>
+               name: Set test variables for Linux
+               values:
+               -  terminal_prompt: $(linux_prompt)
+               -  psep: /
+         -  let:
+               condition: <| "$(os)" == "Windows" |>
+               name: Set test variables for Windows
+               values:
+               -  terminal_prompt: $(windows_prompt)
+               -  psep: \
+   -  group:
+         name: Group of tests
+         steps:
+         -  let:
+               name: plot test constants
+               values:
+                  test: plot
+                  test_path: items/$(test)
+         -  group:
+               name: plot test
+               steps:
+               -  sequence:
+                     steps:
+                     -  group:
+                           condition: <| $(validation_dialogs) and not tm.text_mode()
+                              |>
+                           name: Plot test
+                           steps:
+                           -  plot:
+                                 key: $(test)_PASS
+                                 name: Open the plot
+                                 plot_name: Mon Plot
+                                 steps:
+                                 -  open:
+                                       log_path: $(validation_report_path)
+                           -  plot:
+                                 key: $(test)_PASS
+                                 name: Add periodic to the plot
+                                 plot_name: Mon Plot
+                                 steps:
+                                 -  periodic:
+                                       eval: '{"periodic": $(result)}'
+                                       file: $(test_path)$(psep)plot.py
+                                       func_name: random_value
+                                       period: 1
+                           -  sleep:
+                                 dialog: true
+                                 name: sleep
+                                 timeout: 3
+                           -  loop:
+                                 iterator: 10
+                                 name: Add of other data in the plot
+                                 steps:
+                                 -  plot:
+                                       key: $(test)_PASS
+                                       name: Add to the plot
+                                       plot_name: Mon Plot
+                                       steps:
+                                       -  add:
+                                             value1: $(loop_index)
+                                             value2: $(loop_index)+2
+                                 -  sleep:
+                                       name: sleep between values
+                                       timeout: 1
+                                 -  py_func:
+                                       file: $(test_path)$(psep)plot.py
+                                       func_name: LastValues
+                                       key: $(test)_PASS
+                                       name: last plot values
+                                       param:
+                                       - Mon Plot
+                           -  plot:
+                                 execute_on_stop: true
+                                 key: $(test)_PASS
+                                 name: Export
+                                 plot_name: Mon Plot
+                                 steps:
+                                 -  export: $(validation_report_path)/plot_export.pdf
+                                 -  export: $(validation_report_path)/plot_export.csv
+                           -  plot:
+                                 execute_on_stop: true
+                                 key: $(test)_PASS
+                                 name: Close the plot
+                                 plot_name: Mon Plot
+                                 steps:
+                                 -  close:
+                                       timeout: 2
+                                       wait_dialog_exit: true
+                     filename: /home/renish/workspace/testium/code/test/validation/items/plot/test.tum
+               -  sequence:
+                     steps:
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.txt
+                           -  html:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.html
+                           -  junit:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.junit
+                           name: Expected PASS $(test) test
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.txt
+                           -  html:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.html
+                           -  junit:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.junit
+                           name: Expected FAIL $(test) test
+                     filename: /home/renish/workspace/testium/code/test/validation/items/report.tum
+report:
+   enabled: true
+   export:
+      junit:
+         file_name: $(validation_report_file).junit
+         path: $(validation_report_path)
+      sqlite:
+         file_name: $(validation_report_file).sqlite
+         path: $(validation_report_path)
+   log_stored: true

schema/test_schema/py_func.tum

@@ -0,0 +1,326 @@
+config_file:
+- param.yaml
+- items/py_func/param.yaml
+main:
+   name: Testium validation suite
+   steps:
+   -  group:
+         name: Test preparation
+         steps:
+         -  let:
+               condition: <| "$(os)" == "Linux" |>
+               name: Set test variables for Linux
+               values:
+               -  terminal_prompt: $(linux_prompt)
+               -  psep: /
+         -  let:
+               condition: <| "$(os)" == "Windows" |>
+               name: Set test variables for Windows
+               values:
+               -  terminal_prompt: $(windows_prompt)
+               -  psep: \
+   -  group:
+         name: Group of tests
+         steps:
+         -  let:
+               name: py_func test constants
+               values:
+                  test: py_func
+                  test_path: items/$(test)
+         -  group:
+               name: py_func test
+               steps:
+               -  sequence:
+                     steps:
+                     -  let:
+                           name: py_func test constants,
+                           values:
+                              py_func test parameter: test parameter
+                     -  py_func:
+                           file: $(test_path)$(psep)py_func.py
+                           func_name: assertparam
+                           key: $(test)_PASS
+                           name: pass py_func
+                           param:
+                           - true
+                     -  py_func:
+                           file: $(test_path)$(psep)py_func.py
+                           func_name: assertparam
+                           key: $(test)_FAIL
+                           name: fail py_func
+                           param:
+                           - false
+                     -  py_func:
+                           expected_result: FAIL
+                           file: $(test_path)$(psep)py_func.py
+                           func_name: assertparam
+                           key: $(test)_PASS
+                           name: fail py_func with expected result "FAIL"
+                           param:
+                           - false
+                     -  py_func:
+                           expected_result: FAIL
+                           file: $(test_path)$(psep)py_func.py
+                           func_name: assertparam
+                           key: $(test)_FAIL
+                           name: pass py_func with expected result FAIL
+                           param:
+                           - true
+                     -  py_func:
+                           expected_result: -1
+                           file: $(test_path)$(psep)py_func.py
+                           func_name: echo
+                           key: $(test)_PASS
+                           name: expected -1
+                           param:
+                           - -1
+                     -  py_func:
+                           expected_result: 354848436 - 354848437
+                           file: $(test_path)$(psep)py_func.py
+                           func_name: echo
+                           key: $(test)_PASS
+                           name: expected eval
+                           param:
+                           - -1
+                     -  py_func:
+                           expected_result: '[-1, ''a'', {''toto'': ''tata''}]'
+                           file: $(test_path)$(psep)py_func.py
+                           func_name: echo
+                           key: $(test)_PASS
+                           name: expected table
+                           param:
+                           -  - -1
+                              - a
+                              -  toto: tata
+                     -  py_func:
+                           file: $(test_path)$(psep)py_func.py
+                           func_name: checkglobal
+                           key: $(test)_PASS
+                           name: global param py_func
+                           param:
+                           - $(py_func test parameter)
+                     -  let:
+                           key: $(test)_PASS
+                           name: python2func
+                           values:
+                           -  py: $(test_path)$(psep)py_func.py
+                     -  py_func:
+                           expected_result: $(py_func test parameter)
+                           file: $(py)
+                           func_name: checkglobal2
+                           key: $(test)_PASS
+                           name: global param py_func 2
+                     -  py_func:
+                           file: $(py)
+                           func_name: checkglobal
+                           key: $(test)_PASS
+                           name: global param py_func
+                           param:
+                           - $(py_func test parameter)
+                     -  py_func:
+                           file: $(test_path)$(psep)py_func.py
+                           func_name: should_not_be_called
+                           name: skipped_checkglobal
+                           param:
+                           - $(py_func test parameter)
+                     -  py_func:
+                           file: $(test_path)$(psep)py_func.py
+                           func_name: checkglobal
+                           name: skipped true
+                           param:
+                           - $(py_func test parameter)
+                           skipped: true
+                     -  py_func:
+                           file: $(test_path)$(psep)py_func.py
+                           func_name: checkglobal
+                           name: skipped 1
+                           param:
+                           - $(py_func test parameter)
+                           skipped: 1
+                     -  py_func:
+                           file: $(test_path)$(psep)py_func.py
+                           func_name: ValidationTest
+                           name: FunctionItem test
+                           param:
+                           - $(py_func test parameter)
+                     -  group:
+                           name: Function results check
+                           steps:
+                           -  group:
+                                 name: Function result 1
+                                 steps:
+                                 -  py_func:
+                                       expected_result: -1
+                                       file: $(test_path)$(psep)py_func.py
+                                       func_name: echo
+                                       key: $(test)_PASS
+                                       name: int failure
+                                       param:
+                                       - -1
+                                 -  py_func:
+                                       expected_result: -1.3
+                                       file: $(test_path)$(psep)py_func.py
+                                       func_name: echo
+                                       key: $(test)_PASS
+                                       name: float failure
+                                       param:
+                                       - -1.3
+                                 -  py_func:
+                                       expected_result: FAIL
+                                       file: $(test_path)$(psep)py_func.py
+                                       func_name: echo
+                                       key: $(test)_PASS
+                                       name: String failure
+                                       param:
+                                       - FAIL
+                                 -  py_func:
+                                       expected_result:
+                                       - -1
+                                       - Got a failure
+                                       file: $(test_path)$(psep)py_func.py
+                                       func_name: tuple_return
+                                       key: $(test)_PASS
+                                       name: Tuple int,str failure
+                                       param:
+                                       - -1
+                                       - Got a failure
+                           -  group:
+                                 name: Functions result 2
+                                 steps:
+                                 -  py_func:
+                                       expected_result: 0
+                                       file: $(test_path)$(psep)py_func.py
+                                       func_name: echo
+                                       key: $(test)_PASS
+                                       name: int success
+                                       param:
+                                       - 0
+                                 -  py_func:
+                                       expected_result: 0.3
+                                       file: $(test_path)$(psep)py_func.py
+                                       func_name: echo
+                                       key: $(test)_PASS
+                                       name: float success
+                                       param:
+                                       - 0.3
+                                 -  py_func:
+                                       expected_result: Something that is not only
+                                          strictly FAIL
+                                       file: $(test_path)$(psep)py_func.py
+                                       func_name: echo
+                                       key: $(test)_PASS
+                                       name: String success
+                                       param:
+                                       - Something that is not only strictly FAIL
+                                 -  py_func:
+                                       expected_result:
+                                       - 0
+                                       - OK
+                                       file: $(test_path)$(psep)py_func.py
+                                       func_name: tuple_return
+                                       key: $(test)_PASS
+                                       name: Tuple int,str success
+                                       param:
+                                       - 0
+                                       - OK
+                     -  py_func:
+                           file: $(test_path)$(psep)py_func.py
+                           func_name: test_delgd
+                           key: $(test)_PASS
+                           name: delgd test
+                     -  py_func:
+                           file: $(test_path)$(psep)py_func.py
+                           func_name: return_nothing
+                           key: $(test)_PASS
+                           name: function returning nothing should succeed
+                     -  py_func:
+                           file: $(test_path)$(psep)py_func.py
+                           func_name: return_explicit_none
+                           key: $(test)_PASS
+                           name: function returning explicit None should succeed
+                     -  group:
+                           name: context_id tests
+                           steps:
+                           -  py_func:
+                                 expected_result: hello context
+                                 file: $(test_path)$(psep)py_func.py
+                                 func_name: set_context_value
+                                 key: $(test)_PASS
+                                 name: set serializable value
+                                 param:
+                                 - hello context
+                           -  py_func:
+                                 context_id: ctx_test
+                                 expected_result: hello context
+                                 file: $(test_path)$(psep)py_func.py
+                                 func_name: get_context_value
+                                 key: $(test)_PASS
+                                 name: get serializable value (same context_id)
+                           -  py_func:
+                                 expected_result: hello context
+                                 file: $(test_path)$(psep)py_func.py
+                                 func_name: get_context_value
+                                 key: $(test)_PASS
+                                 name: get serializable value (no context_id, from
+                                    main gd)
+                           -  py_func:
+                                 context_id: ctx_other
+                                 expected_result: hello context
+                                 file: $(test_path)$(psep)py_func.py
+                                 func_name: get_context_value
+                                 key: $(test)_PASS
+                                 name: get serializable value (different context_id)
+                           -  py_func:
+                                 context_id: ctx_ns_test
+                                 expected_result: hello ns
+                                 file: $(test_path)$(psep)py_func.py
+                                 func_name: set_ns_value
+                                 key: $(test)_PASS
+                                 name: set non-serializable value
+                                 param:
+                                 - hello ns
+                           -  py_func:
+                                 context_id: ctx_ns_test
+                                 expected_result: hello ns
+                                 file: $(test_path)$(psep)py_func.py
+                                 func_name: get_ns_value
+                                 key: $(test)_PASS
+                                 name: get non-serializable value (same context_id)
+                     filename: /home/renish/workspace/testium/code/test/validation/items/py_func/test.tum
+               -  sequence:
+                     steps:
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.txt
+                           -  html:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.html
+                           -  junit:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.junit
+                           name: Expected PASS $(test) test
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.txt
+                           -  html:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.html
+                           -  junit:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.junit
+                           name: Expected FAIL $(test) test
+                     filename: /home/renish/workspace/testium/code/test/validation/items/report.tum
+report:
+   enabled: true
+   export:
+      junit:
+         file_name: $(validation_report_file).junit
+         path: $(validation_report_path)
+      sqlite:
+         file_name: $(validation_report_file).sqlite
+         path: $(validation_report_path)
+   log_stored: true

schema/test_schema/sleep.tum

@@ -0,0 +1,87 @@
+config_file:
+- param.yaml
+- items/sleep/param.yaml
+main:
+   name: Testium validation suite
+   steps:
+   -  group:
+         name: Test preparation
+         steps:
+         -  let:
+               condition: <| "$(os)" == "Linux" |>
+               name: Set test variables for Linux
+               values:
+               -  terminal_prompt: $(linux_prompt)
+               -  psep: /
+         -  let:
+               condition: <| "$(os)" == "Windows" |>
+               name: Set test variables for Windows
+               values:
+               -  terminal_prompt: $(windows_prompt)
+               -  psep: \
+   -  group:
+         name: Group of tests
+         steps:
+         -  let:
+               name: sleep test constants
+               values:
+                  test: sleep
+                  test_path: items/$(test)
+         -  group:
+               name: sleep test
+               steps:
+               -  sequence:
+                     steps:
+                     -  sleep:
+                           condition: $(validation_dialogs)
+                           dialog: true
+                           key: $(test)_PASS
+                           name: Sleep timeout with dialogs
+                           timeout: 3
+                     -  sleep:
+                           key: $(test)_PASS
+                           name: Sleep timeout without dialog
+                           timeout: 3.0
+                     -  sleep:
+                           key: $(test)_PASS
+                           name: Sleep timeout in textual format
+                           skipped: true
+                           timeout: 1h 3m 2s
+                     filename: /home/renish/workspace/testium/code/test/validation/items/sleep/test.tum
+               -  sequence:
+                     steps:
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.txt
+                           -  html:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.html
+                           -  junit:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.junit
+                           name: Expected PASS $(test) test
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.txt
+                           -  html:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.html
+                           -  junit:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.junit
+                           name: Expected FAIL $(test) test
+                     filename: /home/renish/workspace/testium/code/test/validation/items/report.tum
+report:
+   enabled: true
+   export:
+      junit:
+         file_name: $(validation_report_file).junit
+         path: $(validation_report_path)
+      sqlite:
+         file_name: $(validation_report_file).sqlite
+         path: $(validation_report_path)
+   log_stored: true

schema/test_schema/unittest.tum

@@ -0,0 +1,84 @@
+config_file:
+- param.yaml
+- items/unittest/param.yaml
+main:
+   name: Testium validation suite
+   steps:
+   -  group:
+         name: Test preparation
+         steps:
+         -  let:
+               condition: <| "$(os)" == "Linux" |>
+               name: Set test variables for Linux
+               values:
+               -  terminal_prompt: $(linux_prompt)
+               -  psep: /
+         -  let:
+               condition: <| "$(os)" == "Windows" |>
+               name: Set test variables for Windows
+               values:
+               -  terminal_prompt: $(windows_prompt)
+               -  psep: \
+   -  group:
+         name: Group of tests
+         steps:
+         -  let:
+               name: unittest test constants
+               values:
+                  test: unittest
+                  test_path: items/$(test)
+         -  group:
+               name: unittest test
+               steps:
+               -  sequence:
+                     steps:
+                     -  unittest:
+                           key: $(test)_PASS
+                           name: Unittest item
+                           test_file: /home/renish/workspace/testium/code/test/validation/items/unittest/unittest.py
+                           test_method: test_01_pass
+                     -  unittest:
+                           key: $(test)_FAIL
+                           name: Unittest item
+                           test_file: /home/renish/workspace/testium/code/test/validation/items/unittest/unittest.py
+                           test_method:
+                           - test_04_disabled
+                           - test_03_fail
+                     filename: /home/renish/workspace/testium/code/test/validation/items/unittest/test.tum
+               -  sequence:
+                     steps:
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.txt
+                           -  html:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.html
+                           -  junit:
+                                 key: $(test)_PASS
+                                 path: $(validation_report_path)$(psep)$(test)_PASS.junit
+                           name: Expected PASS $(test) test
+                     -  report:
+                           export:
+                           -  text:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.txt
+                           -  html:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.html
+                           -  junit:
+                                 key: $(test)_FAIL
+                                 path: $(validation_report_path)$(psep)$(test)_FAIL.junit
+                           name: Expected FAIL $(test) test
+                     filename: /home/renish/workspace/testium/code/test/validation/items/report.tum
+report:
+   enabled: true
+   export:
+      junit:
+         file_name: $(validation_report_file).junit
+         path: $(validation_report_path)
+      sqlite:
+         file_name: $(validation_report_file).sqlite
+         path: $(validation_report_path)
+   log_stored: true

schema/tum.json

@@ -0,0 +1,532 @@
+{
+    "$schema": "https://json-schema.org/draft/2020-12/schema",
+
+    "$defs": {
+        "config_file":{
+            "desciption": "The list of the configuration files",
+            "type": "array",
+            "items": {
+                "type": "string"
+            }
+        },
+
+        "items":{
+            "properties": {
+                "name": { "type": "string" },
+                "stop_on_failure":{ "type": "boolean" },
+                "execute_on_stop":{ "type": "boolean" },
+                "skipped":{ "type": "boolean" },
+                "no_fail":{ "type": "boolean" },
+                "doc":{ "type": "string" },
+                "key":{ "type": "string" },
+                "report":{ "$ref": "#/$defs/report_export" },
+                "condition":{ "type": "string" },
+                "process_result":{ "type": "string" },
+                "expected_result":{ },
+                "store_result":{ "type": "string" }
+            }
+        },
+
+        "steps" : {
+            "required": ["steps"],
+            "properties": {
+                "version": { "type": "string" },
+                "steps": {
+                    "type": "array",
+                    "items": {
+                        "type": "object",
+                        "additionalProperties": false,
+                        "properties":{
+                            "let": { "$ref": "#/$defs/let" },
+                            "check": { "$ref": "#/$defs/check" },
+                            "dialog_question": { "$ref": "#/$defs/dialog" },
+                            "dialog_message": { "$ref": "#/$defs/dialog" },
+                            "dialog_note": { "$ref": "#/$defs/dialog" },
+                            "dialog_value": { "$ref": "#/$defs/dialog" },
+                            "dialog_image": { "$ref": "#/$defs/dialog" },
+                            "lua_func": { "$ref": "#/$defs/func" },
+                            "py_func": { "$ref": "#/$defs/func" },
+                            "console": { "$ref": "#/$defs/console" },
+                            "sleep": { "$ref": "#/$defs/sleep" },
+                            "json_rpc": { "$ref": "#/$defs/json_rpc" },
+                            "group": { "$ref": "#/$defs/group" },
+                            "sequence": { "$ref": "#/$defs/sequence" },
+                            "report": { "$ref": "#/$defs/report" },
+                            "loop": { "$ref": "#/$defs/loop" },
+                            "git": { "$ref": "#/$defs/git" },
+                            "unittest": { "$ref": "#/$defs/unittest" }
+
+                        }
+                    }
+                }
+            }
+        },
+
+        "cons_open" : {
+            "type":"object",
+            "allOf": 
+            [
+                { "$ref": "#/$defs/items" },
+                { 
+                    "properties": {
+                        "protocol" : {
+                            "enum": ["telnet", "ssh", "serial", "rawtcp", "terminal"]
+                        },
+                        "telnet_host" : {"type": "string" },
+                        "telnet_port" : {"type": "number" },
+                        "ssh_host": {"type": "string" },
+                        "ssh_user": {"type": "string" },
+                        "ssh_pwd": {"type": "string" },
+                        "serial_port": {"type": "string" },
+                        "serial_baudrate": {"type": "integer" },
+                        "buffered": {"type": "boolean" },
+                        "tcp_host" : {"type": "string" },
+                        "tcp_port" : {"type": "number" },
+                        "terminal_path": {"type":"string" },
+                        "shell": {"type":"string" }
+                    },
+                    "required": ["protocol"]
+                }
+            ]   
+        },
+
+        "cons_read": {
+            "type":"object",
+            "unevaluatedProperties": false,
+            "allOf": [
+                { "$ref": "#/$defs/items" },
+                {
+                    "properties": {
+                        "protocol" : {
+                            "enum": ["telnet", "ssh", "serial", "rawtcp", "terminal"]
+                        },
+                        "expected" : {"type": "string" },
+                        "timeout": {"type": "number" },
+                        "mute": {"type": "boolean" }
+                    },
+                    "required":["expected"]
+                }
+            ]
+        },
+
+        "console" : {
+            "description": "The let items",
+            "type": "object",
+            "unevaluatedProperties": false,
+            "allOf": [
+                { "$ref": "#/$defs/items" },
+                {
+                    "required": ["steps", "console_name"],
+                    "properties":{
+                        "console_name": {"type":"string" },
+                        "steps": {
+                            "type": "array",
+                            "items": {
+                                "type": "object",
+                                "properties":{
+                                    "open": {
+                                        "$ref": "#/$defs/cons_open"
+                                    },
+                                    "write": { "type": "string" },
+                                    "writeln": { "type": "string" },
+                                    "read_until": {
+                                        "$ref": "#/$defs/cons_read"
+                                    },
+                                    "close": { 
+                                        "anyOf": [
+                                            {"type": "null"},
+                                            {"type": "string"}
+                                        ]
+                                    }
+                                },
+                                "additionalProperties": false
+                            }
+                        }
+                    }
+                }
+            ]
+        },
+
+        "json_rpc_console": {
+            "type":"object",
+            "properties": {
+                "name" : {"type": "string" },
+                "prompt" : {"type": "string" }
+            },
+            "required":["name"],
+            "additionalProperties": false
+        },
+
+        "json_rpc_udp": {
+            "type":"object",
+            "properties": {
+                "server" : {"type": "string" },
+                "udp_snd_port" : {"type": "integer" },
+                "udp_rcv_port" : {"type": "integer" },
+                "bufsize" : {"type": "integer" }
+            },
+            "required":["name"],
+            "additionalProperties": false
+        },
+        
+        "json_rpc_query": {
+            "type":"object",
+            "properties": {
+                "method" : { "type": "string" },
+                "params" : { "type": "array" },
+                "id" : {"type": "integer" },
+                "no_wait" : {"type": "boolean" }
+            },
+            "required":["method"],
+            "additionalProperties": false
+        },
+
+        "json_rpc_receive": {
+            "type":"object",
+            "properties": {
+                "id" : {"type": "integer" },
+                "timeout" : {"type": "number" }
+            },
+            "required":["id"],
+            "additionalProperties": false
+        },
+
+        "json_rpc" : {
+            "description": "The json_rpc items",
+            "type": "object",
+            "unevaluatedProperties": false,
+            "allOf": [
+                { "$ref": "#/$defs/items" },
+                {
+                    "required": ["steps"],
+                    "properties":{
+                    
+                        "udp" : { "$ref": "#/$defs/json_rpc_udp" },
+                        "console" : { "$ref": "#/$defs/json_rpc_console" },
+                        "timeout": {"type": "number" },
+                        "version": {"enum": ["1.0", "2.0"]},
+                        "steps": {
+                            "type": "array",
+                            "items": {
+                                "type": "object",
+                                "additionalProperties": false,
+                                "properties":{
+                                    "open": { "type": "null" },
+                                    "query": { "$ref": "#/$defs/json_rpc_query" },
+                                    "receive": { "$ref": "#/$defs/json_rpc_receive" },
+                                    "writeln": { "type": "string" },
+                                    "read_until": { "$ref": "#/$defs/cons_read"
+                                    },
+                                    "close": { "type": "null" }
+                                }
+                            }
+                        }
+                    }
+                }
+            ]
+        },
+
+        "group" : {
+            "description": "The group items",
+            "type": "object",
+            "unevaluatedProperties": false,
+            "allOf": [
+                { "$ref": "#/$defs/items" },
+                { "$ref": "#/$defs/steps" }
+            ]
+        },
+
+        "sequence" : {
+            "description": "The sequence items",
+            "type": "object",
+            "unevaluatedProperties": false,
+            "allOf": [
+                { "$ref": "#/$defs/items" },
+                { "$ref": "#/$defs/steps" },
+                {
+                    "properties": {
+                        "filename": { "type": "string" }
+                    },
+                    "required": ["filename"]
+                }
+            ]
+        },
+
+        "loop" : {
+            "description": "The loop items",
+            "type": "object",
+            "unevaluatedProperties": false,
+            "allOf": [
+                { "$ref": "#/$defs/items" },
+                { "$ref": "#/$defs/steps" },
+                {
+                    "properties": {
+                        "iterator": { },
+                        "exit_condition": {
+                            "description": "the posibility to stop the loop",
+                            "type": "object",
+                            "additionalProperties": false,
+                            "properties": {
+                                "time" : {"type":"number"},
+                                "value" : {"type":"number"},
+                                "file" : {"type":"string"},
+                                "func_name" : {"type":"string"},
+                                "eval" : {"type":"string"}
+                            },
+                            "dependentRequired": {
+                                "file" : ["func_name"]
+                            }
+
+                        }
+                    }
+                }
+            ]
+        },
+
+        "main" : {
+            "description": "The Main items",
+            "type": "object",
+            "unevaluatedProperties": false,
+            "allOf": [
+                { "$ref": "#/$defs/items" },
+                { "$ref": "#/$defs/steps" },
+                {
+                    "properties": {
+                        "version": { "type": "string" }
+                    }
+                }
+            ]
+        },
+
+        "sleep" : {
+            "description": "Sleep for X time [secondes",
+            "type": "object",
+            "unevaluatedProperties": false,
+            "allOf": [
+                { "$ref": "#/$defs/items" },
+                {
+                    "properties":{
+                        "dialog": { "type": "boolean" },
+                        "timeout": { "type": "number" }
+                    },
+                    "required": ["timeout"]
+                }
+            ]
+        },
+
+        "let" : {
+            "description": "The let items",
+            "type": "object",
+            "unevaluatedProperties": false,
+            "allOf": [
+                { "$ref": "#/$defs/items" },
+                {
+                    "properties":{
+                        "values": {
+                            "anyOf": [
+                                {
+                                    "type": "array",
+                                    "items": {"type": "object" }
+                                },
+                                { "type":"object" }
+                            ]
+                        }
+                    },
+                    "required": ["values"]
+                }
+            ]
+        },
+
+        "check" : {
+            "description": "The let items",
+            "type": "object",
+            "unevaluatedProperties": false,
+            "allOf": [
+                { "$ref": "#/$defs/items" },
+                {
+                    "properties":{
+                        "values": { 
+                            "type": "array",
+                            "items": {"type": "string" }
+                        }
+                    },
+                    "required": ["values"]
+                }
+            ]
+        },
+
+        "git" : {
+            "description": "The git items",
+            "type": "object",
+            "unevaluatedProperties": false,
+            "allOf": [
+                { "$ref": "#/$defs/items" },
+                {
+                    "properties":{
+                        "repo": { "type": "string" }
+                    },
+                    "required": ["repo"]
+                }
+            ]
+        },
+
+        "unittest" : {
+            "description": "The unittest items",
+            "type": "object",
+            "unevaluatedProperties": false,
+            "allOf": [
+                { "$ref": "#/$defs/items" },
+                {
+                    "properties":{
+                        "test_file": { "type": "string" },
+                        "test_method": {
+                            "anyOf": [
+                                {"type":"string"},
+                                {
+                                    "type":"array",
+                                    "items": {"type":"string"}
+                                }
+                            ]
+                        }
+                    },
+                    "required": ["test_file"]
+                }
+            ]
+        },
+
+        "dialog" : {
+            "description": "The let items",
+            "type": "object",
+            "unevaluatedProperties": false,
+            "allOf": [
+                { "$ref": "#/$defs/items" },
+                {
+                    "properties":{
+                        "question": { "type": "string" },
+                        "auto_result": { "type": "string" },
+                        "auto_value": { "type": ["number", "string"] },
+                        "filename" : {"type": "string"},
+                        "reference" : {
+                            "type":"array",
+                            "items": { "type":"string"}
+                        }
+                    },
+                    "required": ["question"]
+                }
+            ]
+        },
+
+        "dialog_choice" : {
+            "description": "The one choi items",
+            "type": "object",
+            "additionalProperties": false,
+            "properties":{
+                "name": { "type": "string" },
+                "description": { "type": "string" },
+                "icon": { "type": "string"},
+                "choices": {
+                    "type":"array",
+                    "items": { "$ref": "#/$defs/dialog_choice" }
+                }
+            },
+            "required": ["name", "description"]
+        },
+        "dialog_choices" : {
+            "description": "The dialog choices items",
+            "type": "object",
+            "unevaluatedProperties": false,
+            "allOf": [
+                { "$ref": "#/$defs/items" },
+                {
+                    "properties":{
+                        "name": { "type": "string" },
+                        "question": { "type": "string" },
+                        "icon": { "type": "string"},
+                        "choices": {
+                            "type":"array",
+                            "items": { "$ref": "#/$defs/dialog_choice" }
+                        }
+                    },
+                    "required": ["question"]
+                }
+            ]
+        },
+        "func" : {
+            "description": "The py_fun and lua_func items",
+            "type": "object",
+            "unevaluatedProperties": false,
+            "allOf": [
+                { "$ref": "#/$defs/items" },
+                {
+                    "properties":{
+                        "file": { "type": "string" },
+                        "func_name": { "type": "string" },
+                        "context_id": { "type": "string" },
+                        "param" : {
+                            "anyOf": [
+                                {"type": "array"},
+                                {"type": "object"}
+                            ]
+                        }
+                    },
+                    "required": ["file", "func_name"]
+                }
+            ]
+        },
+
+        "report_export": {
+            "type": "object",
+            "properties": {
+                "path": {"type":"string" },
+                "file_name": {"type":"string" },
+                "pattern": {"type":"string" },
+                "key": {"type":"string" }
+            }
+        },
+
+        "report":{
+            "desciption": "The list of the configuration files",
+            "type": "object",
+            "additional_properties":false,
+            "properties": {
+                "enabled": {"type":"boolean" },
+                "log_stored": {"type":"boolean" },
+                "export": {
+                    "anyOf": [
+                        {
+                            "type": "object",
+                            "properties":{
+                                "html": { "$ref": "#/$defs/report_export" },
+                                "sqlite": { "$ref": "#/$defs/report_export" },
+                                "junit": { "$ref": "#/$defs/report_export" }
+                            }
+                        },
+                        {
+                            "type": "array",
+                            "items": {
+                                "type":"object",
+                                "properties":{
+                                    "html": { "$ref": "#/$defs/report_export" },
+                                    "sqlite": { "$ref": "#/$defs/report_export" },
+                                    "junit": { "$ref": "#/$defs/report_export" }
+                                }
+                            }
+                        }
+                    ]
+                }
+            }
+        }
+
+    },
+    
+    "type": "object",
+    "properties": {
+        "config_file" : { "$ref": "#/$defs/config_file"  },
+        "main": {  "$ref": "#/$defs/main"  },
+        "report" : { "$ref": "#/$defs/report"  }
+
+    },
+    "required" : ["main"],
+    "additionalProperties": false
+
+}

scripts/build_env.sh

@@ -20,6 +20,12 @@ if [ "$?" -ne 0 ]; then
     echo "venv must be installed on the host distribution."
     exit -1
 fi
+# Check if venv is installed
+python3 -c "import ensurepip"
+if [ "$?" -ne 0 ]; then
+    echo "ensurepip must be installed on the host distribution."
+    exit -1
+fi
 
 # Install the virtual environment if needed
 if [ ! -d "$PY_VENV_DIR" ]; then
@@ -27,4 +33,10 @@ if [ ! -d "$PY_VENV_DIR" ]; then
     python3 -m venv "$PY_VENV_DIR"
     source "$PY_VENV_DIR/bin/activate"
     pip install --extra-index-url https://pypi.python.org/pypi -r $REQ_PATH
+    # Validation suite plugin used to verify the report-exporter
+    # entry-points discovery end-to-end.
+    FAKE_EXPORTER_DIR="$(dirname "$REQ_PATH")/../test/validation/fake_exporter"
+    if [ -d "$FAKE_EXPORTER_DIR" ]; then
+        pip install -e "$FAKE_EXPORTER_DIR"
+    fi
 fi

src/LICENSE

@@ -0,0 +1,315 @@
+Copyright (c) 2025-2026 François Dausseur
+
+Licensed under the EUPL
+
+
+                       EUROPEAN UNION PUBLIC LICENCE v. 1.2
+                       EUPL © the European Union 2007, 2016
+
+This European Union Public Licence (the 'EUPL') applies to the Work (as
+defined below) which is provided under the terms of this Licence. Any use of
+the Work, other than as authorised under this Licence is prohibited (to the
+extent such use is covered by a right of the copyright holder of the Work).
+
+The Work is provided under the terms of this Licence when the Licensor (as
+defined below) has placed the following notice immediately following the
+copyright notice for the Work:
+
+        Licensed under the EUPL
+
+or has expressed by any other means his willingness to license under the EUPL.
+
+
+1. Definitions
+
+In this Licence, the following terms have the following meaning:
+
+- 'The Licence': this Licence.
+
+- 'The Original Work': the work or software distributed or communicated by the
+  Licensor under this Licence, available as Source Code and also as Executable
+  Code as the case may be.
+
+- 'Derivative Works': the works or software that could be created by the
+  Licensee, based upon the Original Work or modifications thereof. This Licence
+  does not define the extent of modification or dependence on the Original
+  Work required in order to classify a work as a Derivative Work; this extent
+  is determined by copyright law applicable in the country mentioned in
+  Article 15.
+
+- 'The Work': the Original Work or its Derivative Works.
+
+- 'The Source Code': the human-readable form of the Work which is the most
+  convenient for people to study and modify.
+
+- 'The Executable Code': any code which has generally been compiled and which
+  is meant to be interpreted by a computer as a program.
+
+- 'The Licensor': the natural or legal person that distributes or communicates
+  the Work under the Licence.
+
+- 'Contributor(s)': any natural or legal person who modifies the Work under
+  the Licence, or otherwise contributes to the creation of a Derivative Work.
+
+- 'The Licensee' or 'You': any natural or legal person who makes any usage of
+  the Work under the terms of the Licence.
+
+- 'Distribution' or 'Communication': any act of selling, giving, lending,
+  renting, distributing, communicating, transmitting, or otherwise making
+  available, online or offline, copies of the Work or providing access to its
+  essential functionalities at the disposal of any other natural or legal
+  person.
+
+
+2. Scope of the rights granted by the Licence
+
+The Licensor hereby grants You a worldwide, royalty-free, non-exclusive,
+sublicensable licence to do the following, for the duration of copyright
+vested in the Original Work:
+
+- use the Work in any circumstance and for all usage,
+- reproduce the Work,
+- modify the Work, and make Derivative Works based upon the Work,
+- communicate to the public, including the right to make available or display
+  the Work or copies thereof to the public and perform publicly, as the case
+  may be, the Work,
+- distribute the Work or copies thereof,
+- lend and rent the Work or copies thereof,
+- sublicense rights in the Work or copies thereof.
+
+Those rights can be exercised on any media, supports and formats, whether now
+known or later invented, as far as the applicable law permits so.
+
+In the countries where moral rights apply, the Licensor waives his right to
+exercise his moral right to the extent allowed by law in order to make
+effective the licence of the economic rights here above listed.
+
+The Licensor grants to the Licensee royalty-free, non-exclusive usage rights
+to any patents held by the Licensor, to the extent necessary to make use of
+the rights granted on the Work under this Licence.
+
+
+3. Communication of the Source Code
+
+The Licensor may provide the Work either in its Source Code form, or as
+Executable Code. If the Work is provided as Executable Code, the Licensor
+provides in addition a machine-readable copy of the Source Code of the Work
+along with each copy of the Work that the Licensor distributes or indicates,
+in a notice following the copyright notice attached to the Work, a repository
+where the Source Code is easily and freely accessible for as long as the
+Licensor continues to distribute or communicate the Work.
+
+
+4. Limitations on copyright
+
+Nothing in this Licence is intended to deprive the Licensee of the benefits
+from any exception or limitation to the exclusive rights of the rights owners
+in the Work, of the exhaustion of those rights or of other applicable
+limitations thereto.
+
+
+5. Obligations of the Licensee
+
+The grant of the rights mentioned above is subject to some restrictions and
+obligations imposed on the Licensee. Those obligations are the following:
+
+Attribution right: The Licensee shall keep intact all copyright, patent or
+trademarks notices and all notices that refer to the Licence and to the
+disclaimer of warranties. The Licensee must include a copy of such notices
+and a copy of the Licence with every copy of the Work he/she distributes or
+communicates. The Licensee must cause any Derivative Work to carry prominent
+notices stating that the Work has been modified and the date of modification.
+
+Copyleft clause: If the Licensee distributes or communicates copies of the
+Original Works or Derivative Works, this Distribution or Communication will
+be done under the terms of this Licence or of a later version of this Licence
+unless the Original Work is expressly distributed only under this version of
+the Licence — for example by communicating 'EUPL v. 1.2 only'. The Licensee
+(becoming Licensor) cannot offer or impose any additional terms or conditions
+on the Work or Derivative Work that alter or restrict the terms of the
+Licence.
+
+Compatibility clause: If the Licensee Distributes or Communicates Derivative
+Works or copies thereof based upon both the Work and another work licensed
+under a Compatible Licence, this Distribution or Communication can be done
+under the terms of this Compatible Licence. For the sake of this clause,
+'Compatible Licence' refers to the licences listed in the appendix attached
+to this Licence. Should the Licensee's obligations under the Compatible
+Licence conflict with his/her obligations under this Licence, the obligations
+of the Compatible Licence shall prevail.
+
+Provision of Source Code: When distributing or communicating copies of the
+Work, the Licensee will provide a machine-readable copy of the Source Code or
+indicate a repository where this Source will be easily and freely available
+for as long as the Licensee continues to distribute or communicate the Work.
+
+Legal Protection: This Licence does not grant permission to use the trade
+names, trademarks, service marks, or names of the Licensor, except as
+required for reasonable and customary use in describing the origin of the
+Work and reproducing the content of the copyright notice.
+
+
+6. Chain of Authorship
+
+The original Licensor warrants that the copyright in the Original Work
+granted hereunder is owned by him/her or licensed to him/her and that he/she
+has the power and authority to grant the Licence.
+
+Each Contributor warrants that the copyright in the modifications he/she
+brings to the Work are owned by him/her or licensed to him/her and that
+he/she has the power and authority to grant the Licence.
+
+Each time You accept the Licence, the original Licensor and subsequent
+Contributors grant You a licence to their contributions to the Work, under
+the terms of this Licence.
+
+
+7. Disclaimer of Warranty
+
+The Work is a work in progress, which is continuously improved by numerous
+Contributors. It is not a finished work and may therefore contain defects or
+'bugs' inherent to this type of development.
+
+For the above reason, the Work is provided under the Licence on an 'as is'
+basis and without warranties of any kind concerning the Work, including
+without limitation merchantability, fitness for a particular purpose, absence
+of defects or errors, accuracy, non-infringement of intellectual property
+rights other than copyright as stated in Article 6 of this Licence.
+
+This disclaimer of warranty is an essential part of the Licence and a
+condition for the grant of any rights to the Work.
+
+
+8. Disclaimer of Liability
+
+Except in the cases of wilful misconduct or damages directly caused to
+natural persons, the Licensor will in no event be liable for any direct or
+indirect, material or moral, damages of any kind, arising out of the Licence
+or of the use of the Work, including without limitation, damages for loss of
+goodwill, work stoppage, computer failure or malfunction, loss of data or any
+commercial damage, even if the Licensor has been advised of the possibility
+of such damage. However, the Licensor will be liable under statutory product
+liability laws as far such laws apply to the Work.
+
+
+9. Additional agreements
+
+While distributing the Work, You may choose to conclude an additional
+agreement, defining obligations or services consistent with this Licence.
+However, if accepting obligations, You may act only on your own behalf and on
+your sole responsibility, not on behalf of the original Licensor or any other
+Contributor, and only if You agree to indemnify, defend, and hold each
+Contributor harmless for any liability incurred by, or claims asserted
+against such Contributor by the fact You have accepted any warranty or
+additional liability.
+
+
+10. Acceptance of the Licence
+
+The provisions of this Licence can be accepted by clicking on an icon 'I
+agree' placed under the bottom of a window displaying the text of this
+Licence or by affirming consent in any other similar way, in accordance with
+the rules of applicable law. Clicking on that icon indicates your clear and
+irrevocable acceptance of this Licence and all of its terms and conditions.
+
+Similarly, you irrevocably accept this Licence and all of its terms and
+conditions by exercising any rights granted to You by Article 2 of this
+Licence, such as the use of the Work, the creation by You of a Derivative
+Work or the Distribution or Communication by You of the Work or copies
+thereof.
+
+
+11. Information to the public
+
+In case of any Distribution or Communication of the Work by means of
+electronic communication by You (for example, by offering to download the
+Work from a remote location) the distribution channel or media (for example,
+a website) must at least provide to the public the information requested by
+the applicable law regarding the Licensor, the Licence and the way it may be
+accessible, concluded, stored and reproduced by the Licensee.
+
+
+12. Termination of the Licence
+
+The Licence and the rights granted hereunder will terminate automatically
+upon any breach by the Licensee of the terms of the Licence.
+
+Such a termination will not terminate the licences of any person who has
+received the Work from the Licensee under the Licence, provided such persons
+remain in full compliance with the Licence.
+
+
+13. Miscellaneous
+
+Without prejudice of Article 9 above, the Licence represents the complete
+agreement between the Parties as to the Work.
+
+If any provision of the Licence is invalid or unenforceable under applicable
+law, this will not affect the validity or enforceability of the Licence as a
+whole. Such provision will be construed or reformed so as necessary to make
+it valid and enforceable.
+
+The European Commission may publish other linguistic versions or new versions
+of this Licence or updated versions of the Appendix, so far this is required
+and reasonable, without reducing the scope of the rights granted by the
+Licence. New versions of the Licence will be published with a unique version
+number.
+
+All linguistic versions of this Licence, approved by the European Commission,
+have identical value. Parties can take advantage of the linguistic version of
+their choice.
+
+
+14. Jurisdiction
+
+Without prejudice to specific agreement between parties,
+
+- any litigation resulting from the interpretation of this License, arising
+  between the European Union institutions, bodies, offices or agencies, as a
+  Licensor, and any Licensee, will be subject to the jurisdiction of the
+  Court of Justice of the European Union, as laid down in article 272 of the
+  Treaty on the Functioning of the European Union,
+
+- any litigation arising between other parties and resulting from the
+  interpretation of this License, will be subject to the exclusive
+  jurisdiction of the competent court where the Licensor resides or conducts
+  its primary business.
+
+
+15. Applicable Law
+
+Without prejudice to specific agreement between parties,
+
+- this Licence shall be governed by the law of the European Union Member
+  State where the Licensor has his seat, resides or has his registered
+  office,
+
+- this licence shall be governed by Belgian law if the Licensor has no seat,
+  residence or registered office inside a European Union Member State.
+
+
+                                  Appendix
+
+
+'Compatible Licences' according to Article 5 EUPL are:
+
+- GNU General Public License (GPL) v. 2, v. 3
+- GNU Affero General Public License (AGPL) v. 3
+- Open Software License (OSL) v. 2.1, v. 3.0
+- Eclipse Public License (EPL) v. 1.0
+- CeCILL v. 2.0, v. 2.1
+- Mozilla Public Licence (MPL) v. 2
+- GNU Lesser General Public Licence (LGPL) v. 2.1, v. 3
+- Creative Commons Attribution-ShareAlike v. 3.0 Unported (CC BY-SA 3.0) for
+  works other than software
+- European Union Public Licence (EUPL) v. 1.1, v. 1.2
+- Québec Free and Open-Source Licence — Reciprocity (LiLiQ-R) or Strong
+  Reciprocity (LiLiQ-R+).
+
+The European Commission may update this Appendix to later versions of the
+above licences without producing a new version of the EUPL, as long as they
+provide the rights granted in Article 2 of this Licence and protect the
+covered Source Code from exclusive appropriation.
+
+All other changes or additions to this Appendix require the production of a
+new EUPL version.

src/VERSION

@@ -1 +1 @@
-0.1
+0.1.2

src/lib/api.py

@@ -1,9 +0,0 @@
-
-SUPPORTED_API = [
-    "gd",
-    "setgd",
-    "delgd",
-    "add_plot_values",
-    "last_plot_value"
-]
-

src/lib/stdout_redirect.py

@@ -1,75 +0,0 @@
-import sys
-from threading import (Thread, Event)
-from lib.string_queue import StringQueue
-from time import (sleep)
-
-class StdioRedirect:
-
-    def __init__(self):
-        self.redirect_enabled = False
-        self.spy_enabled = False
-        self.ini_stdout = sys.stdout
-        self.ini_stderr = sys.stderr
-        self.stream = self.ini_stdout
-
-    def redirect(self, stream):
-        if not self.spy_enabled:
-            self.out_stream = stream
-            self.stream = self.out_stream
-            sys.stdout = self.out_stream
-            sys.stderr = self.out_stream
-            self.redirect_enabled = True
-
-    def restore(self):
-        if not self.spy_enabled and self.redirect_enabled:
-            sys.stdout = self.ini_stdout
-            sys.stderr = self.ini_stderr
-            self.redirect_enabled = False
-
-    def intercept(self):
-        if not self.spy_enabled:
-            self.thr_started = Event()
-            self.log_buf = StringQueue()
-            self.in_stream = StringQueue()
-            self.stop_output = Event()
-            self.thrd_out = Thread(target=self.interceptStdOut)
-            self.thrd_out.daemon = True
-            sys.stdout = self.in_stream
-            sys.stderr = self.in_stream
-            self.stream = self.in_stream
-            self.thrd_out.start()
-            self.thr_started.wait()
-            self.spy_enabled = True
-
-
-    def stop(self):
-        if self.spy_enabled:
-            sys.stdout = self.out_stream
-            sys.stderr = self.out_stream
-            self.stream = self.out_stream
-            self.stop_output.set()
-            self.thrd_out.join()
-            del self.log_buf
-            del self.in_stream
-            del self.stop_output
-            del self.thrd_out
-            del self.thr_started
-
-        self.spy_enabled = False
-
-    def interceptStdOut(self):
-        self.thr_started.set()
-        while not self.stop_output.is_set():
-            data = self.in_stream.read()
-            self.log_buf.write(data)
-            self.out_stream.write(data)
-            if data == '':
-                sleep(0.1)
-
-    def read(self):
-        ret = ''
-        if self.spy_enabled:
-            ret = self.log_buf.read()
-        return ret
-
-stdio_redir = StdioRedirect()

src/py_func/__main__.py

@@ -1,23 +0,0 @@
-from pathlib import Path
-import sys
-import traceback
-
-def exception_handler(typ_exc, value, trbk):
-    """Testium Exception handling"""
-    print("An unmanaged exception occured")
-    print(f"Critical failure : '{value}'.")
-    tb = traceback.format_exception(typ_exc, value, trbk)
-    print("".join(tb))
-
-sys.excepthook = exception_handler
-
-p = Path(__file__)
-p = p.parent / ".."
-p = p.resolve()
-
-sys.path.append(p)
-
-from py_func import main
-
-if __name__ == '__main__':
-    main()

src/py_func/tm.py

@@ -1,85 +0,0 @@
-
-import sys
-from py_func.handle import FuncHandler
-from lib.tum_except import ETUMRuntimeError
-from lib.api import SUPPORTED_API
-
-thismodule = sys.modules[__name__]
-# Shared FuncHandler instance used to forward API calls. Remains None
-# until `_init_api` is invoked.
-_func_call_thread = None
-
-
-###############################################################################
-# Dynamically create module-level functions for each supported API name.
-# Each generated function shares the implementation of `api_call` but
-# has a distinct name used as the remote action identifier.
-def _make_api(name):
-    def _wrapper(*params):
-        if _func_call_thread is not None:
-            res = _func_call_thread.call(name, params)
-            if "result" in res:
-                ret_val = res["result"]
-            elif "error" in res:
-                raise ETUMRuntimeError(f"api call to 'tm.{name}' failed with error '{res["error"]}'")
-            else:
-                raise ETUMRuntimeError("api call failure in jrpc client to be reported to testium support team.")
-            return ret_val
-        else:
-            raise ETUMRuntimeError("api not initialized")
-    _wrapper.__name__ = name
-    return _wrapper
-
-for k in SUPPORTED_API:
-    setattr(thismodule, k, _make_api(k))
-
-def _init_api(host, port, timeout):
-    """Start and initialize the remote function handler.
-
-    Starts a ``FuncHandler`` bound to ``port``, runs it and blocks until
-    it signals readiness.
-
-    Args:
-        port: port number or identifier passed to ``FuncHandler``.
-
-    Returns:
-        The initialized ``FuncHandler`` instance assigned to
-        ``_func_call_thread``.
-    """
-    global _func_call_thread
-    _func_call_thread = FuncHandler(host, port, timeout=timeout)
-    return _func_call_thread
-
-
-###############################################################################
-def _remote_print(*values):
-    """Forward print-like output to the remote handler.
-
-    If a ``_func_call_thread`` is available, this function calls the
-    handler with action name ``"print"`` and the provided values. Errors
-    during forwarding are ignored because printing is best-effort.
-    """
-    if _func_call_thread is not None:
-        try:
-            _func_call_thread.call("print", values)
-        except:
-            # Best-effort: ignore forwarding failures
-            pass
-
-
-###############################################################################
-class FunctionItem():
-    """Class allowing extended capabilities of function."""
-    module_count = 0
-
-    def __init__(self):
-        self._reported_value = {}
-
-    def reportValue(self, key, value):
-        self._reported_value[key] = value
-
-    def reportedValues(self):
-        return self._reported_value
-
-    def exec(self):
-        pass

src/pyproject.toml

@@ -8,9 +8,11 @@ requires-python = ">=3.11"
 authors = [
   {name = "François Dausseur", email = "francois@beafrancois.fr"},
 ]
+license = "EUPL-1.2"
+license-files = ["../LICENSE"]
 classifiers = [
   "Development Status :: 5 - Production/Stable",
-  "Programming Language :: Python"
+  "Programming Language :: Python",
 ]
 dependencies = [
     "setuptools",
@@ -33,7 +35,9 @@ testium = "testium:main"
 
 [tool.setuptools.packages.find]
 where=["."]
-exclude=["lua_func", "py_func"]
+
+[tool.setuptools.package-data]
+"testium.lua_func" = ["*.lua"]
 
 [tool.setuptools.dynamic]
 version = {file = ["VERSION"]}

src/testium/__init__.py

@@ -21,10 +21,8 @@ def main():
                         help="Returns the version of testium", action='store_true')
     parser.add_argument("-b", "--batch-execution",
                         help="Executes the test in batch mode", action='store_true')
-    parser.add_argument("-m", "--terminal",
-                        help="Starts terminal mode", action='store_true')
     parser.add_argument("-o", "--no-color",
-                        help="Deactivates stdout colors in batch and terminal mode", action='store_true')
+                        help="Deactivates stdout colors in batch mode", action='store_true')
     parser.add_argument("-c", "--config-file", help="Configuration file",
                         nargs='+',
                         default=[])
@@ -95,36 +93,13 @@ def main():
         from interpreter.utils.version import get_testium_version
         print(get_testium_version())
 
-    elif args.terminal:
-        import select
-        from interpreter.terminal import Terminal
-
-        if (lf != '') or (rf != '') or (tf != '') or (pn != []):
-            print('"-l", "-p", "-t", "-n" options are not supported in this mode.')
-
-        t = Terminal(os.getcwd(), cf, defines, args.no_color)
-
-        loop = 1
-        while loop:
-            try:
-                loop = 0
-                t.cmdloop()
-            except KeyboardInterrupt:
-                print("\n<ctrl-c>")
-                loop = 1
-            except Exception as exc:
-                if str(exc) == 'quit':
-                    break
-                print(exc)
-                loop = 1
-
-
     elif args.batch_execution:
         if (lf != ''):
             print('"-l" option is not supported in this mode.')
 
         from interpreter.batch import Batch
-        b = Batch(tf, cf, defines, rf, args.report_type, pn, args.no_color)
+        b = Batch(tf, cf, defines, rf, args.report_type, pn, args.no_color, text_mode=True)
+        sys.exit(0 if b.success else 1)
 
     else:
         from main_win.testium_win import MainWin

src/testium/api/console.py

@@ -11,6 +11,7 @@ import threading
 from telnetlib3 import Telnet, DO, WILL, WONT, TTYPE, IAC, SB, SE, theNULL
 
 TIMEOUT_NULL = 0.000001
+STOP_POLL_INTERVAL = 0.2
 
 
 class BytesStore(object):
@@ -123,12 +124,14 @@ A {classname}.close() is missing somewhere in your code !'.format(classname=type
         #    c = ''
         return c
 
-    def read_until(self, match, timeout=None, return_data=False, mute=False):
+    def read_until(self, match, timeout=None, return_data=False, mute=False, should_stop=None):
         """
         read until the string 'match is found
             If timeout is not set (None), this function runs indefinitely
             If timeout is set to zero, this function returns immediately
             If mute is set to True the characters read from the console will not be displayed
+            If should_stop is a callable, it is polled between reads (every STOP_POLL_INTERVAL
+            at most) and the loop exits early — like a timeout — when it returns True.
 
             If function fails (because of a timeout) it will return a 'status' integer set to -1
             otherwise it will return 0.
@@ -139,13 +142,6 @@ A {classname}.close() is missing somewhere in your code !'.format(classname=type
         status = -1
         if not match:
             raise ValueError('match parameter can not be empty')
-        # replace all '\r' by '\n' as any '\r' read will undergo the same replacement
-        # match = match.replace('\r\n', '\n')
-        # match = match.replace('\r', '')
-
-        # update the console timeout in conformity with what is required.
-
-        self.set_read_timeout(timeout)
 
         if timeout is None:
             timeout = 1000000
@@ -159,6 +155,7 @@ A {classname}.close() is missing somewhere in your code !'.format(classname=type
         # buffer is empty
         # Otherwise we are waiting for the timeout to rise
         if timeout < TIMEOUT_NULL:
+            self.set_read_timeout(0)
             data = self.readchar(0)
 
             while (status < 0) and ((data is not None) and (data != b'')):
@@ -191,39 +188,45 @@ A {classname}.close() is missing somewhere in your code !'.format(classname=type
 
         # Timeout different than zero
         else:
+            # Poll in short chunks so a stop request is honored within
+            # STOP_POLL_INTERVAL, regardless of the per-protocol blocking
+            # behavior of readchar().
+            self.set_read_timeout(STOP_POLL_INTERVAL)
 
             time_is_out = threading.Event()
             timer = threading.Timer(timeout, lambda: time_is_out.set())
             timer.start()
 
-            # We are waiting for the timeout to rise
+            try:
+                while (status < 0) and (not time_is_out.is_set()):
+                    if should_stop is not None and should_stop():
+                        break
 
-            while (status < 0) and (not time_is_out.isSet()):
-
-                data = self.readchar(timeout)
-                if data is not None:
-                    data = self._compute_char(data)
-                    if data != '':
-                        if not mute:
-                            self.string_buffer += data
-                        read_data += data
-
-                        search_deque.append(data)
-                        if search_deque == match_deque:
-                            timer.cancel()
-                            status = 0
-                            if (not mute) and (data != '\n'):
-                                self.string_buffer += '\n'
-
-                        if data == '\n' or (status >= 0):
-                            # the datas are written line by line for display optimisation in GUI mode
+                    data = self.readchar(STOP_POLL_INTERVAL)
+                    if data is not None:
+                        data = self._compute_char(data)
+                        if data != '':
                             if not mute:
-                                self.string_buffer = self.string_buffer.replace('\r\n', '\n')
-                                self.string_buffer = self.string_buffer.replace('\r', '')
-                                self.stream.write(self.string_buffer)
+                                self.string_buffer += data
+                            read_data += data
 
-                            date_str = str(datetime.now()).split('.')[0].split(' ')[1]
-                            self.string_buffer = '[{} {}]'.format(date_str, self.name)
+                            search_deque.append(data)
+                            if search_deque == match_deque:
+                                status = 0
+                                if (not mute) and (data != '\n'):
+                                    self.string_buffer += '\n'
+
+                            if data == '\n' or (status >= 0):
+                                # the datas are written line by line for display optimisation in GUI mode
+                                if not mute:
+                                    self.string_buffer = self.string_buffer.replace('\r\n', '\n')
+                                    self.string_buffer = self.string_buffer.replace('\r', '')
+                                    self.stream.write(self.string_buffer)
+
+                                date_str = str(datetime.now()).split('.')[0].split(' ')[1]
+                                self.string_buffer = '[{} {}]'.format(date_str, self.name)
+            finally:
+                timer.cancel()
 
         if return_data:
             return status, read_data
@@ -245,7 +248,7 @@ A {classname}.close() is missing somewhere in your code !'.format(classname=type
 if not sys.platform.startswith('win'):
     # import SshConsole if pexpect is installed
     try:
-        from libs.console_ssh import SshConsole
+        from api.console_ssh import SshConsole
 
     except ImportError:
         pass

src/testium/api/console_ssh.py

@@ -8,7 +8,7 @@ import os
 import pexpect
 from pexpect import ExceptionPexpect, TIMEOUT, EOF, spawn
 
-from libs.console import Console
+from api.console import Console
 
 # Exception classes used by this module.
 

src/testium/api/raw_tcp_console.py

@@ -3,7 +3,7 @@ import sys
 import socket
 import traceback
 
-from libs.console import *
+from api.console import *
 
 class RawTCPConsole(Console):
     TYPE = 'rawtcp'

src/testium/api/runtime_plot.py

@@ -1,5 +1,6 @@
 import sys
 import os
+import queue
 import multiprocessing as mp
 from threading import Timer
 from time import sleep, monotonic
@@ -16,9 +17,9 @@ import numpy as np
 import matplotlib.dates as mdates
 from datetime import datetime, timedelta, timezone
 
-import libs.testium as tm
+import api.testium as tm
 from interpreter.test_items.test_result import TestValue
-from lib.tum_except import ETUMRuntimeError
+from runtime.tum_except import ETUMRuntimeError
 from interpreter.utils.py_func_exec import PyFuncExecEngine
 from interpreter.utils.api_srv import api_request
 from interpreter.utils.eval import post_evaluate
@@ -270,7 +271,7 @@ class RuntimePlotPeriodic(PeriodicTimer):
         self.func_name = func_name
         self.args = args
         self.post_eval = post_eval
-        self.proc = PyFuncExecEngine(tm.gd("python_bin", ""), api_request, 10)
+        self.proc = PyFuncExecEngine(api_request, 10)
         self.proc.start()
         if not self.proc.wait_ready(10):
             raise ETUMRuntimeError(
@@ -367,7 +368,7 @@ class RuntimePlot:
             self.msg_queue_in.get()
         self.msg_queue_out.put({"command": "last_values"})
         try:
-            res = self.msg_queue_in.get(timeout=1)
-        except:
+            res = self.msg_queue_in.get(timeout=5)
+        except queue.Empty:
             raise ETUMRuntimeError(f"Impossible to retrieve the last values of the \"{self.name}\" plot")
         return res

src/testium/api/termconsole.py

@@ -10,7 +10,7 @@ import os
 
 ourPath = os.path.dirname(__file__)
 sys.path.append(ourPath)
-from libs.console import (Console, BytesStore, TIMEOUT_NULL)
+from api.console import (Console, BytesStore, TIMEOUT_NULL)
 
 class TermConsole(Console):
     TYPE = 'term'

src/testium/api/testium.py

@@ -4,7 +4,7 @@ import sys
 import textwrap
 from time import monotonic
 import interpreter.utils.globdict as globdict
-from lib.tum_except import (ETUMSyntaxError)
+from runtime.tum_except import (ETUMSyntaxError)
 
 ###############################################################################
 # Console helper functions
@@ -14,7 +14,7 @@ def add_console(console):
     ''' Function which adds a ``Console`` class instance to *testium*
 
     :param console: The ``Console`` instance.
-    :type console: ``libs.console.Console`` or child class instance
+    :type console: ``api.console.Console`` or child class instance
     :return: No returned value
 
     '''
@@ -48,7 +48,7 @@ def console(name):
     :param name: The name of the ``Console`` instance.
     :type name: str
     :return: The ``Console`` or child class object
-    :rtype: ``libs.console.Console`` or child class instance
+    :rtype: ``api.console.Console`` or child class instance
     """
     cons = None
     for c in globdict.gd('console_instances', []):
@@ -65,7 +65,7 @@ def add_plot(plot: object) -> None:
     ''' Function which adds a ``RuntimePlot`` class instance to *testium*
 
     :param plot: The ``RuntimePlot`` instance.
-    :type plot: ``libs.runtime_plot.RuntimePlot`` or child class instance
+    :type plot: ``api.runtime_plot.RuntimePlot`` or child class instance
     :return: No returned value
 
     '''
@@ -99,7 +99,7 @@ def plot(name: str) -> object:
     :param name: The name of the ``RuntimePlot`` instance.
     :type name: str
     :return: The ``RuntimePlot`` or child class object
-    :rtype: ``libs.runtime_plot.RuntimePlot`` or child class instance
+    :rtype: ``api.runtime_plot.RuntimePlot`` or child class instance
     """
     plot = None
     for g in globdict.gd('plot_instances', []):
@@ -209,6 +209,15 @@ def OS():
     return platform.system()
 
 
+def text_mode():
+    """Whether testium is running in text mode (batch ``-b`` or terminal ``-m``).
+
+    :return: ``True`` if running in text mode, ``False`` otherwise.
+    :rtype: bool
+    """
+    return bool(globdict.gd("_text_mode", False))
+
+
 def sys_encoding():
     if OS() == "Windows":
         enc = "oem"

src/testium/interpreter/batch.py

@@ -1,6 +1,7 @@
 import os
 import sys
 import platform
+import threading
 from time import sleep
 from signal import signal, SIGINT
 from queue import Empty
@@ -8,8 +9,8 @@ from multiprocessing import Queue
 
 from interpreter.process import TestProcess
 from interpreter.utils.test_ctrl import TestSetController
-from lib.tum_except import ETUMFileError
-from lib.stdout_redirect import stdio_redir
+from runtime.tum_except import ETUMFileError, ETUMRuntimeError
+from runtime.stdout_redirect import stdio_redir
 
 
 class Batch:
@@ -22,6 +23,7 @@ class Batch:
         report_type,
         report_pattern,
         no_color,
+        text_mode=False,
     ):
         try:
             try:
@@ -51,6 +53,7 @@ class Batch:
 
                 signal(SIGINT, self.sigint_handler)
 
+                self._success = False
                 msg_queue = Queue()
                 self.tst_ctrl = TestSetController()
                 tst_proc = TestProcess(
@@ -59,11 +62,21 @@ class Batch:
                     self.tst_ctrl,
                     config_files,
                     defines,
+                    text_mode=text_mode,
                 )
                 tst_proc.start()
 
-                while not self.tst_ctrl.control("loaded"):
-                    sleep(0.1)
+                # Wait for TestProcess to finish loading.
+                # Run the blocking control("loaded") in a daemon thread so we
+                # can watch for unexpected process death in the main thread.
+                _loaded_event = threading.Event()
+                def _wait_loaded():
+                    self.tst_ctrl.control("loaded")
+                    _loaded_event.set()
+                threading.Thread(target=_wait_loaded, daemon=True).start()
+                while not _loaded_event.wait(timeout=0.1):
+                    if not tst_proc.is_alive():
+                        raise ETUMRuntimeError("TestProcess terminated unexpectedly during load")
 
                 self.tst_ctrl.control(
                     "report",
@@ -78,14 +91,18 @@ class Batch:
                 while True:
                     try:
                         m = msg_queue.get(timeout=0.2)
-                        if m.get("id", None) is None:
-                            # No id -> finished
+                        if "id" in m and m["id"] is None:
+                            # id key present and None -> finished
+                            self._success = m.get("success", False)
                             break
                     except Empty:
+                        if not tst_proc.is_alive():
+                            break
                         continue
 
                 # Close the process and wait for termination
-                self.tst_ctrl.control("close")
+                if tst_proc.is_alive():
+                    self.tst_ctrl.control("close")
                 tst_proc.join()
 
             except Exception as e:
@@ -94,5 +111,12 @@ class Batch:
         finally:
             stdio_redir.restore()
 
+    @property
+    def success(self):
+        return self._success
+
     def sigint_handler(self, signal_received, frame):
-        self.tst_ctrl.control("stop")
+        try:
+            self.tst_ctrl.control("stop", timeout=5)
+        except Exception:
+            pass

src/testium/interpreter/process.py

@@ -1,13 +1,15 @@
 import os
+import signal
 from multiprocessing import Process, Queue, Pipe
 from queue import Empty
 from threading import Thread
 from time import sleep
 import copy
 
-from lib.string_queue import StringQueue
-from lib.tum_except import print_exception, ETUMRuntimeError, ETUMSyntaxError
-import libs.testium as tm
+from runtime.string_queue import StringQueue
+from runtime.tum_except import print_exception, ETUMRuntimeError, ETUMSyntaxError
+import api.testium as tm
+import interpreter.utils.globdict as globdict
 from interpreter.utils.params import expanse
 from interpreter.utils.test_ctrl import TestSetController
 from interpreter.utils.test_init import (
@@ -24,7 +26,7 @@ from interpreter.utils.test_init import (
 from interpreter.utils.constants import TestItemType as cst_type
 from interpreter.test_set import TestSet
 from interpreter.utils.include import TUMLoader, TUMLoaderNoIncludes, TUMLoaderRawIncludes
-from lib.stdout_redirect import stdio_redir
+from runtime.stdout_redirect import stdio_redir
 from interpreter.utils.template import template_to_test
 from interpreter.utils.yaml_load import yaml_load
 from interpreter.utils.py_eval import eval_process_init
@@ -40,6 +42,7 @@ class TestProcess(Process):
         config_files,
         defines,
         gui_defaults={},
+        text_mode=False,
     ) -> None:
         super().__init__()
         self.__fname = file_name
@@ -48,6 +51,7 @@ class TestProcess(Process):
         self.__cfgf = config_files
         self.__defs = defines
         self.__gui_defaults = gui_defaults  # default values coming from GUI prefs
+        self.__text_mode = text_mode
         self.__exec = False
         self.__loaded = False
         self.__closed = False
@@ -193,6 +197,7 @@ class TestProcess(Process):
 
 
     def run(self):
+        signal.signal(signal.SIGINT, signal.SIG_IGN)
         try:
             try:
                 # Thread for stdout redirection
@@ -206,7 +211,7 @@ class TestProcess(Process):
                 env_init()
 
                 # Creation of the python evaluation process for loading of the complete test
-                eval_proc = eval_process_init("", api_request, 10, test_dir)
+                eval_proc = eval_process_init(api_request, 10, test_dir)
                 eval_proc.start()
                 tm.print_debug(f"python bin is: '{eval_proc.python_bin}'.")
                 if not eval_proc.wait_ready(10):
@@ -223,6 +228,10 @@ Is the python exec path correct ?"""
                     # Load the test file
                     test_dict, param_files = self._load_test(init_param_files, glob_variables)
 
+                    if self.__text_mode:
+                        tm.setgd("_text_mode", True)
+                        tm.setgd("_interactive", False)
+
                     # Backup the global dict in case of restart of the test
                     gdict = backup_gd()
 
@@ -255,6 +264,7 @@ Is the python exec path correct ?"""
                                 try:
                                     test_run_init()
                                     print(test_run_header())
+                                    globdict.set_update_queue(self.__squeue)
                                     test_set.execute()
                                 finally:
                                     if test_set.success():
@@ -265,8 +275,16 @@ Is the python exec path correct ?"""
                                 test_set.run_post_exec()
                             finally:
                                 self.__exec = False
+                                # Stop shared context engines before restore_gd wipes them
+                                for engine in tm.gd("_py_func_contexts", {}).values():
+                                    engine.stop()
+                                    engine.join()
+                                for engine in tm.gd("_lua_func_contexts", {}).values():
+                                    engine.stop()
+                                    engine.join()
                                 # Sends signal to the GUI
-                                self.send_finished()
+                                self.send_finished(success=test_set.success())
+                                globdict.set_update_queue(None)
                                 restore_gd(gdict)
                         except Exception as e:
                             print_exception(e)
@@ -275,6 +293,13 @@ Is the python exec path correct ?"""
                     # Stop python eval execution process
                     eval_proc.stop()
                     eval_proc.join()
+                    # Stop shared func context engines (keep_context_id)
+                    for engine in tm.gd("_py_func_contexts", {}).values():
+                        engine.stop()
+                        engine.join()
+                    for engine in tm.gd("_lua_func_contexts", {}).values():
+                        engine.stop()
+                        engine.join()
 
             except Exception as e:
                 print_exception(e)
@@ -297,6 +322,9 @@ Is the python exec path correct ?"""
             "enabled_state": test_set.getEnabledState,
             "process_param": self.process_param,
             "set_test_outputs": self.set_test_outputs,
+            "get_gd_vars": self.get_gd_vars,
+            "set_gd_var": self.set_gd_var,
+            "del_gd_var": self.del_gd_var,
             "set_enabled_state": test_set.setEnabledState,
             "check_uncheck_all": test_set.checkUncheckAll,
             "get_folded": test_set.getFolded,
@@ -311,8 +339,10 @@ Is the python exec path correct ?"""
         stdio_redir.restore()
         stdio_redir.stop()
 
-    def send_finished(self):
+    def send_finished(self, success=None):
         status = {"id": None, "name": "test_process", "status": "finished"}
+        if success is not None:
+            status["success"] = success
         self.__squeue.put(status)
 
     def execute(self):
@@ -330,6 +360,25 @@ Is the python exec path correct ?"""
     def set_test_outputs(self, outputs: list):
         tm.setgd("test_outputs", outputs)
 
+    def get_gd_vars(self):
+        import json
+        result = {}
+        for k, v in globdict.global_dict.items():
+            if k.startswith("_"):
+                continue
+            try:
+                json.dumps(v)
+                result[k] = v
+            except (TypeError, ValueError):
+                pass
+        return result
+
+    def set_gd_var(self, name: str, value):
+        tm.setgd(name, value)
+
+    def del_gd_var(self, name: str):
+        tm.delgd(name)
+
     def process_control_commands(self, tctrl):
         term = False
         while (not term) and (not self.__closed):
@@ -382,7 +431,7 @@ Is the python exec path correct ?"""
             try:
                 # read the pipe data
                 data = cconn.recv()
-                print(data, end="")
+                print(data, end="", flush=True)
             except EOFError:
                 # exit the loop is the pipe is closed
                 break

src/testium/interpreter/terminal.py

@@ -1,244 +0,0 @@
-try:
-    import readline
-except:
-    pass
-from cmd import Cmd
-import os
-import sys
-from yaml import load, Loader
-import functools
-import platform
-import types
-import inspect
-
-# test modules
-from interpreter.utils.test_init import (
-    env_init, prepare_global, set_standard_gd_keys,
-    update_global, test_run_init, test_run_header, load_test)
-from interpreter.utils.globdict import (global_dict)
-import libs.testium as tm
-from interpreter.utils.constants import TestItemType as cst
-from interpreter.test_report.test_report import TestReport
-
-
-class FakeQueue:
-    def put(self, arg):
-        pass
-
-
-def func(self, args):
-    if not args.startswith("{"):
-        args = "{"+args+"}"
-    y = load(args, Loader)
-    obj = self.current_item(y, status_queue=FakeQueue())
-    obj.report = self.report
-    res = obj.execute()
-    if not (res.value is None):
-        print('result : {}'.format(res.value))
-    print(res.test_result)
-
-
-class Terminal(Cmd):
-    SUPPORTED_TESTS = [
-        cst.TYPE_SLEEP,
-        cst.TYPE_LET,
-        cst.TYPE_PY_FUNCTION,
-        cst.TYPE_LUA_FUNCTION,
-        cst.TYPE_CONSOLE,
-        cst.TYPE_IMAGE_DLG,
-        cst.TYPE_MESSAGE_DLG,
-        cst.TYPE_QUESTION_DLG,
-        cst.TYPE_VALUE_DLG,
-    ]
-
-    SUPPORTED_GROUPS = [
-        cst.TYPE_GROUP,
-        cst.TYPE_CYCLE
-    ]
-
-    def __init__(self, working_dir, config_files, defines, no_color):
-        super().__init__()
-        self.working_dir = working_dir
-        self.config_files = config_files
-        self.current_item = None
-        report = TestReport(None)
-        self.report = report
-
-        env_init()
-        prepare_global()
-        # Define the builtin variables
-        set_standard_gd_keys("Unnamed", self.working_dir, '', config_files)
-        update_global([], defines)
-
-        # creation of the functions
-        for tst in self.SUPPORTED_TESTS:
-            meth_name = "do_" + tst.item_cmd
-            # copy of the function
-            f = types.FunctionType(func.__code__, func.__globals__, name=meth_name,
-                                   argdefs=func.__defaults__,
-                                   closure=func.__closure__)
-            f = functools.update_wrapper(f, func)
-            f.__kwdefaults__ = func.__kwdefaults__
-            f.__doc__ = tst.item_class.__doc__
-            setattr(self, meth_name, types.MethodType(f, self))
-
-        test_run_init()
-        self.prompt = "(testium)~ "
-
-        # display header
-        print(test_run_header())
-        # redirect output
-
-        if 'Linux' in platform.system() and not no_color:
-            from lib.stdout_redirect import stdio_redir
-            try:
-                from interpreter.utils.termlog import TermLog
-                stdio_redir.redirect(TermLog(sys.stdout))
-            except ModuleNotFoundError:
-                tm.print_info('Colored console not supported by the system.' +
-                      ' If you want it, please install colorama module')
-
-    def precmd(self, line: str) -> str:
-        c = line.split(" ", 1)[0].strip()
-        self.current_item = None
-        for tst in self.SUPPORTED_TESTS:
-            if c == tst.item_cmd:
-                self.current_item = tst.item_class
-                break
-        return line
-
-    def load_test_recursively(self, tree_parent, parent_seq, status_queue):
-        try:
-            parent_seq_name = parent_seq['name']
-        except KeyError:
-            parent_seq['name'] = "sequence"
-        except TypeError:
-            raise Exception("Syntax error in an item of type {} which is a child of {}".format(
-                tree_parent.type(), tree_parent.parent().name()))
-        try:
-            parent_seq_actions = parent_seq['steps']
-        except KeyError:
-            raise Exception(' No action list found for "%s" sequence'
-                            % (parent_seq_name))
-        # if action is a dictionary , we assume it is a single action
-        # that has not been nested in a list, so do it
-        if isinstance(parent_seq_actions, (dict)):
-            parent_seq_actions = [parent_seq_actions]
-        if not isinstance(parent_seq_actions, (list, tuple)):
-            raise Exception('Actions list not valid.')
-        # first we merged to the same level 'sequence dict entries and list within the list
-        counter = 0
-        test_dir = tm.gd('test_directory')
-        while (counter < len(parent_seq_actions)):
-            action = parent_seq_actions[counter]
-            # if action is a list raise up to the the same level,
-            # ie insert action element into the parent_seq_actions
-            if isinstance(action, (list, tuple)):
-                parent_seq_actions[counter:counter+1] = action
-                continue
-            # if action is a NoneType skip and continue
-            # (when pointing to an unused alias for instance)
-            if action is None:
-                counter += 1
-                continue
-            # if action is a sequence we insert its entry into the action list
-            if 'sequence' in action:
-                parent_seq_actions[counter:counter+1] = action['sequence']
-                continue
-            else:
-                executed = False
-                for it in [*self.SUPPORTED_TESTS, *self.SUPPORTED_GROUPS]:
-                    if it.item_cmd in action:
-                        executed = True
-                        item = (it.item_class)(action[it.item_cmd],
-                                               tree_parent,
-                                               status_queue)
-                        # check for sequence type:
-                        if it.item_cmd == cst.TYPE_UNITTEST_FILE.item_cmd:
-                            item.setTestDir(test_dir)
-                            item.load()
-                        elif ((it.item_cmd == cst.TYPE_CYCLE.item_cmd) or
-                              (it.item_cmd == cst.TYPE_GROUP.item_cmd)):
-                            self.load_test_recursively(
-                                item, action[it.item_cmd], status_queue)
-
-                if not executed:
-                    raise Exception('action type is not known "{}"'.format(
-                        list(action.keys())[0]))
-
-            counter += 1
-
-    def __setReportRecursively(self, parent):
-        for i in range(parent.childCount()):
-            parent.child(i).report = self.report
-            self.__setReportRecursively(parent.child(i))
-
-    def setReport(self, root_item):
-        root_item.report = self.report
-        self.__setReportRecursively(root_item)
-
-    def get_names(self):
-        memb = inspect.getmembers(self)
-        return [n[0] for n in memb if (inspect.ismethod(n[1]) and n[0].startswith("do_"))]
-
-    def do_load(self, args):
-        """load function.
-
-This function loads and executes a testium sub-script.
-
-The loaded sequence can't be a main testium script ("testium -b" option is
-defined for such a usage).
-
-Accepted files are with extension "*.tum".
-
-usage:
-    load path/to/my/sequence.tum
-"""
-        file = args.strip()
-        suff = file[-4:]
-        if not suff in ['.tum']:
-            raise Exception('Wrong input file extension')
-
-        if not (os.path.exists(file) and os.path.isfile(file)):
-            raise Exception(
-                '"{}" does not exist or is not a file.'.format(file))
-
-        d, _ = load_test(file)
-        if not isinstance(d, list):
-            raise Exception(
-                "The file root object must be a list. A \"main\" tum can't be loaded from here (use batch mode instead).")
-
-        if (len(d) == 1) and isinstance(d[0], dict) and (not d[0].get('sequence', None) is None):
-            d = d[0]['sequence']
-
-        sq = FakeQueue()
-        root_item = (cst.TYPE_ROOT.item_class)(
-            dict_item={'steps': d}, status_queue=sq)
-        self.load_test_recursively(root_item, {'steps': d}, sq)
-        self.setReport(root_item)
-        res = root_item.execute()
-        if not (res.value is None):
-            print('"{}" execution overall result: {}'.format(file, res.value))
-        print(res.test_result)
-
-    def do_gd(self, args):
-        """Variables lists and values.
-
-usage:
-    gd
-    gd home
-"""
-        if args != '':
-            res = tm.gd(args, None)
-            if res is None:
-                raise Exception(
-                    'the variable: "{}" has not been found.'.format(args))
-            print(res)
-            return
-
-        for k in global_dict.keys():
-            print('{}: {}'.format(str(k), str(global_dict[k])))
-
-    def do_quit(self, args):
-        '''Quit the application.'''
-        raise Exception('quit')

src/testium/interpreter/test_items/dialog_choices_files/choices_dialog.py

@@ -5,7 +5,7 @@ from itertools import chain
 
 from PySide6.QtGui import QIcon, QPixmap
 from PySide6.QtWidgets import QApplication, QDialog, QDialogButtonBox
-from PySide6.QtCore import Qt, QSettings, QSize
+from PySide6.QtCore import Qt, QSettings, QTimer, QSize
 from PySide6.QtGui import QFont, QFontInfo
 from PySide6.QtWidgets import QTreeWidgetItem
 
@@ -185,7 +185,9 @@ def main(args, conn=None):
     SettingsApplication = "testium_choices_dlg_" + args[0]
     SettingsLastChoices = "last_choice"
     success = True
-    app = QApplication()
+    from interpreter.test_items import dialog_env
+    dialog_env.setup()
+    app = QApplication(['testium'])
     d = ChoicesDialog()
     d.setFixedSize(800, 600)
     d.setWindowFlags(Qt.WindowStaysOnTopHint)
@@ -205,6 +207,9 @@ def main(args, conn=None):
     d.connect_checked()
 
     d.choicesView.setFocus()
+    auto_result = args[4] if len(args) > 4 else None
+    if auto_result is not None:
+        QTimer.singleShot(2000, lambda: d.accept() if auto_result.lower() == 'ok' else d.reject())
     dres = d.exec()
 
     if dres == QDialog.Rejected:

src/testium/interpreter/test_items/dialog_choices_files/choices_dialog_win.py

@@ -3,7 +3,7 @@
 ################################################################################
 ## Form generated from reading UI file 'choices_dialog_win.ui'
 ##
-## Created by: Qt User Interface Compiler version 6.10.1
+## Created by: Qt User Interface Compiler version 6.11.0
 ##
 ## WARNING! All changes made in this file will be lost when recompiling UI file!
 ################################################################################

src/testium/interpreter/test_items/dialog_env.py

@@ -0,0 +1,17 @@
+"""Qt platform environment setup for dialog subprocesses.
+
+Call setup() at the start of every dialog subprocess main() function
+to ensure the correct Qt platform plugin is selected.
+"""
+import sys
+import os
+
+
+def setup():
+    """Configure the Qt environment for dialog subprocess usage."""
+    if sys.platform.startswith('linux'):
+        if os.environ.get('DISPLAY'):
+            # X11 available: force xcb to avoid crashes in multiprocessing subprocesses.
+            os.environ['QT_QPA_PLATFORM'] = 'xcb'
+        elif os.environ.get('WAYLAND_DISPLAY'):
+            os.environ['QT_QPA_PLATFORM'] = 'wayland'

src/testium/interpreter/test_items/dialog_image_files/dialog_image.py

@@ -1,7 +1,6 @@
 import sys
-import os
 
-from PySide6.QtCore import (Qt)
+from PySide6.QtCore import Qt, QTimer
 from PySide6.QtWidgets import (QApplication, QDialog)
 from PySide6 import (QtGui)
 
@@ -18,7 +17,9 @@ class TestDialogWindow(QDialog, dialog_image_win.Ui_Dialog):
 
 def main(args, conn):
     success = True
-    app = QApplication(args)
+    from interpreter.test_items import dialog_env
+    dialog_env.setup()
+    app = QApplication(['testium'])
     d = TestDialogWindow()
     d.setFixedSize(700,600)
     d.setWindowFlags(Qt.WindowStaysOnTopHint)
@@ -37,6 +38,10 @@ def main(args, conn):
 
         d.labelImage.setPixmap(QtGui.QPixmap.fromImage(image2))
 
+        auto_result = args[3] if len(args) > 3 else None
+        if auto_result is not None:
+            QTimer.singleShot(2000, lambda: d.accept() if auto_result.lower() == 'ok' else d.reject())
+
         dres = d.exec()
 
         if dres == QDialog.Rejected:

src/testium/interpreter/test_items/dialog_image_files/dialog_image_win.py

@@ -3,7 +3,7 @@
 ################################################################################
 ## Form generated from reading UI file 'dialog_image_win.ui'
 ##
-## Created by: Qt User Interface Compiler version 6.10.1
+## Created by: Qt User Interface Compiler version 6.11.0
 ##
 ## WARNING! All changes made in this file will be lost when recompiling UI file!
 ################################################################################

src/testium/interpreter/test_items/dialog_msg_files/msg_dialog.py

@@ -1,36 +1,39 @@
-import sys
-import os
-
-from PySide6.QtWidgets import (QApplication, QDialog)
-from PySide6.QtCore import (Qt)
-from PySide6.QtWidgets import QMessageBox
-from multiprocessing import  freeze_support
-
-def main(args):
-    app = QApplication(sys.argv)
-    reply = QMessageBox.information(None, args[0], args[1], QMessageBox.Ok)
-
-    if hasattr(sys, "frozen"):
-        #all standard streams are replaced by dummy one to avoid cx_freeze flushing bug.
-        class dummyStream:
-            ''' dummyStream behaves like a stream but does nothing. '''
-            def __init__(self): pass
-            def write(self,data): pass
-            def read(self,data): pass
-            def flush(self): pass
-            def close(self): pass
-
-        # and now redirect all default streams to this dummyStream:
-        sys.stdout = dummyStream()
-        sys.stderr = dummyStream()
-        sys.stdin = dummyStream()
-        sys.__stdout__ = dummyStream()
-        sys.__stderr__ = dummyStream()
-        sys.__stdin__ = dummyStream()
-
-
-if __name__ == '__main__':
-    main(sys.argv[1:])
-
-
-
+import sys
+from multiprocessing import freeze_support
+
+from PySide6.QtWidgets import (QApplication, QMessageBox)
+from PySide6.QtCore import Qt, QTimer
+
+
+def main(args):
+    from interpreter.test_items import dialog_env
+    dialog_env.setup()
+    app = QApplication(['testium'])
+    msg = QMessageBox()
+    msg.setWindowFlags(Qt.WindowStaysOnTopHint)
+    msg.setWindowTitle(args[0])
+    msg.setText(args[1])
+    msg.setIcon(QMessageBox.Information)
+    msg.setStandardButtons(QMessageBox.Ok)
+    if len(args) > 2:
+        QTimer.singleShot(2000, lambda: msg.button(QMessageBox.Ok).click())
+    msg.exec()
+
+    if hasattr(sys, "frozen"):
+        class dummyStream:
+            def __init__(self): pass
+            def write(self, data): pass
+            def read(self, data): pass
+            def flush(self): pass
+            def close(self): pass
+
+        sys.stdout = dummyStream()
+        sys.stderr = dummyStream()
+        sys.stdin = dummyStream()
+        sys.__stdout__ = dummyStream()
+        sys.__stderr__ = dummyStream()
+        sys.__stdin__ = dummyStream()
+
+
+if __name__ == '__main__':
+    main(sys.argv[1:])

src/testium/interpreter/test_items/dialog_note_files/dialog_note_win.py

@@ -3,7 +3,7 @@
 ################################################################################
 ## Form generated from reading UI file 'dialog_note_win.ui'
 ##
-## Created by: Qt User Interface Compiler version 6.10.1
+## Created by: Qt User Interface Compiler version 6.11.0
 ##
 ## WARNING! All changes made in this file will be lost when recompiling UI file!
 ################################################################################

src/testium/interpreter/test_items/dialog_note_files/test_dialog.py

@@ -2,7 +2,7 @@ import sys
 import os
 
 from PySide6.QtWidgets import (QApplication, QDialog)
-from PySide6.QtCore import (Qt)
+from PySide6.QtCore import Qt, QTimer
 from interpreter.test_items.dialog_note_files import dialog_note_win
 from multiprocessing import  freeze_support
 
@@ -14,13 +14,23 @@ class TestDialogWindow(QDialog, dialog_note_win.Ui_Dialog):
 
 def main(args, conn=None):
     success = True
-    app = QApplication(args)
+    from interpreter.test_items import dialog_env
+    dialog_env.setup()
+    app = QApplication(['testium'])
     d = TestDialogWindow()
     d.setFixedSize(387,224)
     d.setWindowFlags(Qt.WindowStaysOnTopHint)
     d.setWindowTitle(args[0])
     d.labelDialog.setText(args[1])
     d.textEdit.setFocus()
+    auto_result = args[2] if len(args) > 2 else None
+    if auto_result is not None:
+        auto_value = args[3] if len(args) > 3 else None
+        def _auto_close():
+            if auto_value is not None:
+                d.textEdit.setPlainText(auto_value)
+            d.accept() if auto_result.lower() == 'ok' else d.reject()
+        QTimer.singleShot(2000, _auto_close)
     dres = d.exec()
 
     if dres == QDialog.Rejected:

src/testium/interpreter/test_items/dialog_question_files/question_dialog.py

@@ -1,32 +1,43 @@
-import sys
-import os
-
-from PySide6.QtWidgets import (QApplication, QDialog)
-from PySide6.QtCore import (Qt)
-from PySide6.QtWidgets import QMessageBox
-from multiprocessing import  freeze_support
-
-def main(args, conn):
-    app = QApplication(sys.argv)
-    reply = QMessageBox.question(None, args[0], args[1], QMessageBox.Yes|QMessageBox.No)
-
-    conn.send(reply)
-    conn.close()
-
-    if hasattr(sys, "frozen"):
-        #all standard streams are replaced by dummy one to avoid cx_freeze flushing bug.
-        class dummyStream:
-            ''' dummyStream behaves like a stream but does nothing. '''
-            def __init__(self): pass
-            def write(self,data): pass
-            def read(self,data): pass
-            def flush(self): pass
-            def close(self): pass
-
-        # and now redirect all default streams to this dummyStream:
-        sys.stdout = dummyStream()
-        sys.stderr = dummyStream()
-        sys.stdin = dummyStream()
-        sys.__stdout__ = dummyStream()
-        sys.__stderr__ = dummyStream()
-        sys.__stdin__ = dummyStream()
+import sys
+from multiprocessing import freeze_support
+
+from PySide6.QtWidgets import (QApplication, QMessageBox)
+from PySide6.QtCore import Qt, QTimer
+
+
+def main(args, conn):
+    try:
+        from interpreter.test_items import dialog_env
+        dialog_env.setup()
+        app = QApplication(['testium'])
+        msg = QMessageBox()
+        msg.setWindowFlags(Qt.WindowStaysOnTopHint)
+        msg.setWindowTitle(args[0])
+        msg.setText(args[1])
+        msg.setIcon(QMessageBox.Question)
+        msg.setStandardButtons(QMessageBox.Yes | QMessageBox.No)
+        auto_result = args[2] if len(args) > 2 else None
+        if auto_result is not None:
+            btn = QMessageBox.Yes if auto_result.lower() == 'yes' else QMessageBox.No
+            QTimer.singleShot(2000, lambda: msg.button(btn).click())
+        reply = msg.exec()
+        conn.send(reply)
+    except Exception as e:
+        print(f"dialog_question error: {e}", file=sys.stderr)
+    finally:
+        conn.close()
+
+    if hasattr(sys, "frozen"):
+        class dummyStream:
+            def __init__(self): pass
+            def write(self, data): pass
+            def read(self, data): pass
+            def flush(self): pass
+            def close(self): pass
+
+        sys.stdout = dummyStream()
+        sys.stderr = dummyStream()
+        sys.stdin = dummyStream()
+        sys.__stdout__ = dummyStream()
+        sys.__stderr__ = dummyStream()
+        sys.__stdin__ = dummyStream()

src/testium/interpreter/test_items/dialog_sleep_files/dialog_sleep.py

@@ -39,7 +39,9 @@ class DialogSleepWindow(QDialog, dialog_sleep_win.Ui_SleepDialogWindow):
 
 def main(args, conn=None):
     success = True
-    app = QApplication(sys.argv)
+    from interpreter.test_items import dialog_env
+    dialog_env.setup()
+    app = QApplication(['testium'])
     d = DialogSleepWindow()
     d.setFixedSize(379,129)
     d.setWindowFlags(Qt.WindowStaysOnTopHint)

src/testium/interpreter/test_items/dialog_sleep_files/dialog_sleep_win.py

@@ -3,7 +3,7 @@
 ################################################################################
 ## Form generated from reading UI file 'dialog_sleep_win.ui'
 ##
-## Created by: Qt User Interface Compiler version 6.10.1
+## Created by: Qt User Interface Compiler version 6.11.0
 ##
 ## WARNING! All changes made in this file will be lost when recompiling UI file!
 ################################################################################

src/testium/interpreter/test_items/dialog_value_files/dialog_value_win.py

@@ -3,7 +3,7 @@
 ################################################################################
 ## Form generated from reading UI file 'dialog_value_win.ui'
 ##
-## Created by: Qt User Interface Compiler version 6.10.1
+## Created by: Qt User Interface Compiler version 6.11.0
 ##
 ## WARNING! All changes made in this file will be lost when recompiling UI file!
 ################################################################################

src/testium/interpreter/test_items/dialog_value_files/test_dialog.py

@@ -2,7 +2,7 @@ import sys
 import os
 
 from PySide6.QtWidgets import (QApplication, QDialog)
-from PySide6.QtCore import (Qt)
+from PySide6.QtCore import Qt, QTimer
 
 from interpreter.test_items.dialog_value_files import dialog_value_win
 from multiprocessing import  freeze_support
@@ -15,7 +15,9 @@ class TestDialogWindow(QDialog, dialog_value_win.Ui_Dialog):
 
 def main(args, conn=None):
     success = True
-    app = QApplication(args)
+    from interpreter.test_items import dialog_env
+    dialog_env.setup()
+    app = QApplication(['testium'])
     d = TestDialogWindow()
     d.setFixedSize(387,224)
     d.setWindowFlags(Qt.WindowStaysOnTopHint)
@@ -23,6 +25,14 @@ def main(args, conn=None):
     d.labelDialog.setText(args[1])
     d.lineEdit.setText(args[2])
     d.lineEdit.setFocus()
+    auto_result = args[3] if len(args) > 3 else None
+    if auto_result is not None:
+        auto_value = args[4] if len(args) > 4 else None
+        def _auto_close():
+            if auto_value is not None:
+                d.lineEdit.setText(auto_value)
+            d.accept() if auto_result.lower() == 'ok' else d.reject()
+        QTimer.singleShot(2000, _auto_close)
     dres = d.exec()
 
     if dres == QDialog.Rejected:

src/testium/interpreter/test_items/item_actions/__init__.py

@@ -1,4 +1,4 @@
-from lib.tum_except import ETUMSyntaxError
+from runtime.tum_except import ETUMSyntaxError
 from interpreter.test_items.test_item import TestItem, test_run, test_data
 from interpreter.test_items.test_result import TestResult, TestValue
 from interpreter.test_items.item_actions.action import TestItemAction

src/testium/interpreter/test_items/test_item.py

@@ -3,11 +3,11 @@ from time import sleep
 import yaml
 from copy import deepcopy
 from interpreter.test_items.test_result import TestResult, TestValue
-import libs.testium as tm
+import api.testium as tm
 from interpreter.utils.params import TestItemParams
 from interpreter.utils.constants import TestItemType as cst_type
 from interpreter.utils.eval import eval_to_boolean, evaluate, post_evaluate
-from lib.tum_except import ETUMSyntaxError
+from runtime.tum_except import ETUMSyntaxError, item_load_context
 
 LOG_TEST_STOP = '<----- step "{}" finished'
 LOG_TEST_START = '-----> step "{}" started'
@@ -20,52 +20,64 @@ class TestItem:
 def test_run(f):
     @wraps(f)
     def wrapper(self):
-        if not self.skipped:
-            if self.enabled:
-                self.run_test_init()
-                # Conditional execution
-                raw_condition = self._prms.getParam(
-                    "condition", default=None, processed=False
-                )
-                if raw_condition is None:
-                    condition = True
-                else:
-                    c = self._prms.expanse(raw_condition)
-                    if isinstance(c, bool):
-                        condition = c
-                    else:
-                        condition = False
-                        c = False
-
-                    if raw_condition == c:
-                        msg = f'"{c}"'
-                    else:
-                        msg = f'"{raw_condition}" --> "{c}"'
-
-                # Do we have to skip the test because of a true condition ?
-                if condition:
-                    if not raw_condition is None:
-                        msg = "condition met: " + msg
-                        self.result.reported = {"input_condition": msg}
-                        print(msg)
-                    # Test preparation
-                    self.run_before_test()
-                    # Test execution
-                    f(self)
-                else:
-                    msg = "condition not met: " + msg
-                    self.result.set(TestValue.NORUN, msg)
-                    self.result.reported = {"input_condition": msg}
-                self.run_test_end()
-            else:
-                self.result.set(TestValue.NORUN, "test disabled")
-                print("Test is disabled.")
-        else:
+        if self.skipped:
             self.result.set(TestValue.NORUN, "test skipped")
             print("Test is skipped.")
+            return self.result
+        
+        if not self.enabled:
+            self.result.set(TestValue.NORUN, "test disabled")
+            print("Test is disabled.")
+            return self.result
+
+        self.run_test_init()
+
+        while self._is_paused:
+            sleep(0.2)
+            if self.isStopped() :
+                self.result.set(TestValue.NORUN, "test stopped")
+                print("Test is Stopped.")
+                self._is_stopped = False    # Restore state for next run
+                return self.result
+
+        # Conditional execution
+        raw_condition = self._prms.getParam(
+            "condition", default=None, processed=False
+        )
+        if raw_condition is None:
+            condition = True
+        else:
+            c = self._prms.expanse(raw_condition)
+            if isinstance(c, bool):
+                condition = c
+            else:
+                condition = False
+                c = False
+
+            if raw_condition == c:
+                msg = f'"{c}"'
+            else:
+                msg = f'"{raw_condition}" --> "{c}"'
+
+        # Do we have to skip the test because of a true condition ?
+        if condition:
+            if not raw_condition is None:
+                msg = "condition met: " + msg
+                self.result.reported = {"input_condition": msg}
+                print(msg)
+            # Test preparation
+            self.run_before_test()
+            # Test execution
+            f(self)
+        else:
+            msg = "condition not met: " + msg
+            self.result.set(TestValue.NORUN, msg)
+            self.result.reported = {"input_condition": msg}
+        self.run_test_end()
 
         return self.result
 
+
     return wrapper
 
 
@@ -101,6 +113,7 @@ class TestItem:
         self.status_queue = status_queue
         self._execute_on_stop = False
         self._post_eval = None
+        self._store_result = None
         self._expected_result = None
         self._no_fail = None
         self._is_stopped = False
@@ -131,11 +144,11 @@ class TestItem:
                 if s:
                     try:
                         self.skipped = eval_to_boolean(s)
-                    except:
+                    except Exception as e:
                         raise ETUMSyntaxError(
                             f"'{self.cmd()}' test item named '{self.name()}':\nskipped expresion can only be a static expression as it is evaluated during loading of TUM : {s}",
                             self.seqFilename(),
-                        )
+                        ) from e
                 # This allow disabling test item directly by using its name inside param.yaml file
                 elif self._name in tm.gd("skipped_test_item", []):
                     self.skipped = True
@@ -155,6 +168,9 @@ class TestItem:
                 if "process_result" in dict_item:
                     self._post_eval = dict_item["process_result"]
 
+                if "store_result" in dict_item:
+                    self._store_result = dict_item["store_result"]
+
                 if "expected_result" in dict_item:
                     self._expected_result = dict_item["expected_result"]
 
@@ -164,11 +180,13 @@ class TestItem:
                 self.banner = LOG_TEST_START.format(self._name)
                 self.footer = LOG_TEST_STOP.format(self._name)
 
-            except:
+            except ETUMSyntaxError:
+                raise
+            except Exception as e:
                 raise ETUMSyntaxError(
-                    f"The '{self.cmd()}' test item named '{self.name()}' has a missing or wrong parameter",
+                    f"The '{self.cmd()}' test item named '{self.name()}' has an unexpected loading error: {e}",
                     self.seqFilename(),
-                )
+                ) from e
 
         self.result = TestResult(self, TestValue.FAILURE, "Failure by default")
 
@@ -249,8 +267,6 @@ class TestItem:
         self._sendStatusStarted()
         if self._is_breakpoint:
             self._is_paused = True
-            while self._is_paused:
-                sleep(0.2)
 
         if self.is_container:
             self.report.incLevel()
@@ -268,13 +284,13 @@ class TestItem:
         if self.is_container:
             self.report.decLevel()
 
-        while self._is_paused:
-            sleep(0.2)
-
         # Post evaluation of the test result
         self.process_result()
         # expected_result treatment
         self.result_expected()
+        # Store result in a global variable if requested (before no_fail so
+        # the real outcome is captured when result.value is None)
+        self.store_result()
         # Case of the no_fail true parameter
         self.process_no_fail()
 
@@ -301,6 +317,7 @@ class TestItem:
             self.process_report(self._reported)
         self.report.addTest(self, self.result, rk)
         self._sendStatusFinished()
+        
 
     def process_result(self):
         if self._post_eval is None:
@@ -317,6 +334,17 @@ class TestItem:
             print(e)
             self.result.set(TestValue.FAILURE, "Result processing failed")
 
+    def store_result(self):
+        if self._store_result is None:
+            return
+        var_name = self._prms.expanse(self._store_result)
+        if self.result.value is None:
+            value = str(self.result.test_result)
+        else:
+            value = self.result.value
+        tm.setgd(var_name, value)
+        print(f"Stored result in '$({var_name})': {value}")
+
     def process_report(self, report_eval):
         tm.print_debug(f"Export reported values:")
         rep_eval = self._prms.expanse(report_eval)

src/testium/interpreter/test_items/test_item_check.py

@@ -1,8 +1,8 @@
 
 from interpreter.test_items.test_item import (TestItem, test_run)
 from interpreter.test_items.test_result import TestValue
-from lib.tum_except import ETUMSyntaxError
-import libs.testium as tm
+from runtime.tum_except import ETUMSyntaxError, item_load_context
+import api.testium as tm
 from interpreter.utils.constants import TestItemType as cst
 from interpreter.utils.eval import evaluate
 
@@ -15,21 +15,16 @@ class TestItemCheckValue(TestItem):
         super().__init__(dict_item, parent, status_queue, filename=filename)
         self._type = cst.TYPE_CHECK
         self.is_container = False
-        try:
+        with item_load_context(self.cmd(), self.name(), self.seqFilename()):
             self._action_list = self._prms.getParamAll('steps', default=[], required=False)
             if len(self._action_list) > 0:
                 tm.print_warn("'steps' argument of check test item is deprecated and is replaced by 'values'")
             self._action_list += self._prms.getParamAll('values', default=[], required=False)
             if len(self._action_list) <= 0:
                 raise ETUMSyntaxError(
-                    f" The '{self.cmd()}' test item named '{self.name()}' must have a 'values' parameter",
+                    f"Missing required 'values' parameter",
                     self.seqFilename()
                 )
-        except:
-            raise ETUMSyntaxError(
-                f"The '{self.cmd()}' test item named '{self.name()}' (a child of: '{self.parent().name()}') has a missing or wrong parameter",
-                self.seqFilename(),
-            )
 
     @test_run
     def execute(self):

src/testium/interpreter/test_items/test_item_choices_dialog.py

@@ -1,50 +1,92 @@
-from multiprocessing import Process, Pipe
-
-from interpreter.test_items.test_item import TestItem, test_run
-from interpreter.test_items.test_result import TestResult, TestValue
-from interpreter.test_items.dialog_choices_files import choices_dialog
-import libs.testium as tm
-from lib.tum_except import ETUMSyntaxError
-from interpreter.utils.constants import TestItemType as cst
-
-
-class TestItemChoicesDialog(TestItem):
-    def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
-        self._name = cst.TYPE_CHOICES_DLG.item_name
-        super().__init__(dict_item, parent, status_queue, filename=filename)
-        self._type = cst.TYPE_CHOICES_DLG
-        self.is_container = False
-        try:
-            self._question = self._prms.getParam("question", required=True)
-            self._choices = self._prms.getParam("choices", required=True)
-            self._default_icon = self._prms.getParam(
-                "icon", required=False, default=None
-            )
-        except:
-            raise ETUMSyntaxError(
-                f"The '{self.cmd()}' test item named '{self.name()}' (a child of: '{self.parent().name()}') has a missing or wrong parameter",
-                self.seqFilename()
-            )
-
-    @test_run
-    def execute(self):
-        q = self._prms.expanse(self._question)
-        choices = self._prms.expanse(self._choices)
-        icon = self._prms.expanse(self._default_icon)
-        parent_conn, child_conn = Pipe()
-        p = Process(
-            target=choices_dialog.main, args=([self.name(), q, choices, icon], child_conn)
-        )
-        p.start()
-        val, succ = parent_conn.recv()
-        p.join()
-
-        self.result.value = val
-
-        if succ:
-            # The result of the test item is put into the global dict
-            tm.setgd("cs_" + self._name, val)
-            self.result.set(TestValue.SUCCESS, str(val))
-        else:
-            tm.delgd("cs_" + self._name)
-            self.result.set(TestValue.FAILURE, str(val))
+from interpreter.test_items.test_item import test_run
+from interpreter.test_items.test_result import TestValue
+from interpreter.test_items.test_item_dialog_base import TestItemDialogBase, _is_text_mode, _is_interactive
+from interpreter.utils.constants import TestItemType as cst
+from runtime.tum_except import item_load_context
+import api.testium as tm
+
+
+class TestItemChoicesDialog(TestItemDialogBase):
+    def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
+        self._name = cst.TYPE_CHOICES_DLG.item_name
+        super().__init__(dict_item, parent, status_queue, filename=filename)
+        self._type = cst.TYPE_CHOICES_DLG
+        self.is_container = False
+        with item_load_context(self.cmd(), self.name(), self.seqFilename()):
+            self._question = self._prms.getParam("question", required=True)
+            self._choices = self._prms.getParam("choices", required=True)
+            self._default_icon = self._prms.getParam("icon", required=False, default=None)
+            self._auto_result = self._prms.getParam("auto_result", required=False, default=None)
+
+    def _print_choices(self, choices, indent=0):
+        if not isinstance(choices, list):
+            return
+        for choice in choices:
+            name = choice.get("name", "")
+            desc = choice.get("description", "")
+            line = "  " * indent + f"- {name}"
+            if desc:
+                line += f": {desc}"
+            print(line)
+            sub = choice.get("choices", None)
+            if sub:
+                self._print_choices(sub, indent + 1)
+
+    def _all_checked(self, choices):
+        result = []
+        if not isinstance(choices, list):
+            return result
+        for choice in choices:
+            item = {"name": choice.get("name", ""), "checked": True}
+            sub = choice.get("choices", None)
+            if sub is not None:
+                item["choices"] = self._all_checked(sub)
+            result.append(item)
+        return result
+
+    @test_run
+    def execute(self):
+        q = self._prms.expanse(self._question)
+        choices = self._prms.expanse(self._choices)
+        icon = self._prms.expanse(self._default_icon)
+        if _is_text_mode():
+            print(f"Choices: {q}")
+            self._print_choices(choices)
+            if _is_interactive():
+                ans = input("Accept all? (y/n) [default: y]: ").strip().lower()
+                if ans in ('n', 'no'):
+                    tm.delgd("cs_" + self._name)
+                    self.result.set(TestValue.FAILURE, "Cancelled")
+                else:
+                    val = self._all_checked(choices)
+                    self.result.value = val
+                    tm.setgd("cs_" + self._name, val)
+                    self.result.set(TestValue.SUCCESS, str(val))
+            else:
+                ar = self._prms.expanse(self._auto_result) if self._auto_result is not None else None
+                if ar is None:
+                    self.result.set(TestValue.FAILURE, 'Dialog not supported in batch mode')
+                elif ar == 'cancel':
+                    tm.delgd("cs_" + self._name)
+                    self.result.set(TestValue.FAILURE, "Cancelled")
+                else:
+                    val = self._all_checked(choices)
+                    self.result.value = val
+                    tm.setgd("cs_" + self._name, val)
+                    self.result.set(TestValue.SUCCESS, str(val))
+            return
+        from interpreter.test_items.dialog_choices_files import choices_dialog
+        ar = self._prms.expanse(self._auto_result) if self._auto_result is not None else None
+        args = [self.name(), q, choices, icon] + ([ar] if ar is not None else [])
+        result = self._run_dialog_with_result(choices_dialog.main, args)
+        if result is None:
+            self.result.set(TestValue.FAILURE, "Dialog subprocess exited without returning a result")
+            return
+        val, succ = result
+        self.result.value = val
+        if succ:
+            tm.setgd("cs_" + self._name, val)
+            self.result.set(TestValue.SUCCESS, str(val))
+        else:
+            tm.delgd("cs_" + self._name)
+            self.result.set(TestValue.FAILURE, str(val))