33 Commits

Author SHA1 Message Date
354c5e12e8 bump to 0.2: declarative test item parameters
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-29 13:16:45 +02:00
b1a7dac0f3 item params: migrate every structured item to declarative PARAMS
Migrates the remaining test items to the ParamSet/Param declaration
introduced in d0721af:
  - dialogs: image, question, value, choices, tested_references
  - actions: check, run, report
  - console: parent + open/read_until actions
  - py_func / lua_func
  - containers: group, parallel + parallel_branch, unittest
  - complex: cycle (sub-block exit_condition documented in
    EXIT_CONDITION_PARAMS), git
  - runtime_plot: parent + open/close/periodic/last_value actions
  - json_rpc: parent + query/receive actions

Items intentionally without PARAMS (and therefore not validated) are
those whose body is the unstructured user value: console write/writeln,
plot add/export, and the json_rpc/console open & close actions. Same
for the internally-instantiated TestItemUnittestElement which passes
dict_item=None.

Behavior on valid .tum files is unchanged (validation suite source
mode: SUCCESS). Typos on declared params now surface as warnings
listing the accepted names; missing required params surface as load-
time errors with file context.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-29 11:14:42 +02:00
d0721af719 item params: declarative descriptor foundation + 4 pilot items
Adds utils/param_decl.py with Param/ParamSet/kind descriptors. TestItem
declares COMMON_PARAMS (name, doc, condition, key, skipped, …) and a
new _validate_declared_params() method that warns on unknown keys and
errors on missing required ones — opt-in per subclass (skipped while
PARAMS is None to keep the migration incremental).

Migrates sleep, let, msg_dialog, note_dialog as pilots. Behavior is
unchanged for any well-formed .tum; typos like 'timeoot' on a sleep
item now produce a clear WARN listing the accepted parameters.

The descriptor intentionally carries no Python type information —
parameter values that are $(…) / <|…|> expressions only acquire their
effective type after expansion, so a static type would be misleading.
Per-param post-expansion validators stay opt-in via validate=lambda.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-29 10:45:57 +02:00
d4889c2a2e flatpak: run host interpreters via flatpak-spawn; validation --mode flag
py_func, lua_func and the run item now reach host binaries through
`flatpak-spawn --host` instead of trying to load them under the
sandbox runtime (which fails with a glibc ABI mismatch). Adds
`--talk-name=org.freedesktop.Flatpak` to the manifest, stages the
/app/lib/testium tree under /tmp so the host can read it, and drops
the dead `_FLATPAK_HOST_DIRS` / lib-injection code paths that the
new approach makes obsolete.

Validation suite gains a `--mode source|wheel|pyinstaller|flatpak|
appimage` flag so the same item set can run against every packaging
channel; per-mode report file names avoid clobbering.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-19 18:42:32 +02:00
a260e2a56c build_all: incremental build + per-step toolchain install
Skip steps whose dist/ artifact already exists; add --clean/-c to
force a full rebuild. Install sphinx/linuxdoc, build, pyinstaller
in their respective steps instead of upfront. Auto-add flathub
remote and install missing Flatpak SDK/runtime deps before step 4.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-19 13:51:22 +02:00
dd584c9064 gui: bypass XDG portal for all file/dir dialogs in Flatpak
The v0.1.2 fix that forced Qt's non-native dialog for the "open test"
dialog only covered one call site. The same XDG-portal-vs-sibling-files
problem applies to every other QFileDialog in the GUI (save report,
log file path, default report/log dirs in preferences, python/lua
interpreter pickers).

Extracted a single ``file_dialog.options()`` helper in main_win/ and
threaded it through every getOpenFileName / getSaveFileName /
getExistingDirectory call in main_win/. Outside Flatpak the helper
returns an empty Options(), so the native dialog stays in use on
KDE / GNOME / Windows / macOS.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-19 08:32:14 +02:00
4d8cafb5a0 validation: dedicated venv + fix python_bin override timing
eval_proc was started before -d/GUI defines reached gd, so
``-d python_bin=...`` and the GUI ``python_bin`` preference were
silently ignored by the very subprocess that runs ``<| ... |>`` evals
(and only took effect for later items once the discovery cache had
already been seeded with the system interpreter). apply_overrides() is
now applied before eval_process_init(), and bins._resolve()'s cache is
keyed by (name, override) so a later param.yaml change re-resolves on
the next lookup.

The validation suite now ships a wrapper (run.sh / run.bat) that
creates a dedicated venv in the system temp dir and pins it via
``-d python_bin=...``. A new ``venv`` item asserts the override took
effect for both eval_proc and py_func paths, with a
``sys.prefix != sys.base_prefix`` marker to catch the case where the
override happens to be a system interpreter (path-equality alone would
miss it, the venv's ``bin/python3`` being a symlink to the host).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-19 08:19:57 +02:00
6f832cd67b validation: cover nil/None return from lua_func/py_func
Two new steps per language: function returning nothing and function
returning explicit nil/None. Both tagged $(test)_PASS — they would
have failed before the lua nil fix (Lua side reported nil result as
error). Python side already worked but is covered for parity.
2026-05-17 18:13:03 +02:00
ff46886865 lua_func: nil return is not an error
_handle_request was using the 1st pcall return as the error
discriminator, so any Lua function returning nothing (e.g. long_wait
in the example) was reported as failed. Discriminate on the 2nd
return (err) instead, and encode nil result as cjson.null so the
returned_value field stays present in the JSON-RPC response.
2026-05-17 18:04:51 +02:00
50d183d191 removed lua param, useless. 2026-05-17 10:43:25 +02:00
2177715641 examples: long_wait py_func/lua_func to exercise Stop
Two extra steps in example_simple.tum that sleep for 10s, used to
verify that pressing Stop interrupts engaged blocking steps.
2026-05-17 10:42:49 +02:00
a728f561be Make Stop interrupt blocking steps promptly
console.read_until polls a should_stop callback in 0.2s chunks across
all protocols. py_func/lua_func override stop() to tear down the worker
and wake the parent RPC wait. json_rpc adapters honor should_stop too.
Engaged leaf steps now report FAILURE on stop (sleep no-dialog was
silently SUCCESS).
2026-05-17 10:42:40 +02:00
116e528a7d Simplify the Start Stop Pause process (v-and-v/testium#20) 2026-05-16 13:36:18 +02:00
cc744e17a1 Adding ensurepip verification for the build environnement (required by venv) 2026-05-16 13:29:37 +02:00
ab39b49558 now the release note and the manual are copied into dist with build_all 2026-05-13 21:24:35 +02:00
95275c4418 Merge branch 'main' of ssh://git.beafrancois.fr:8328/v-and-v/testium 2026-05-13 14:09:41 +02:00
0d614c2921 release: 0.1.2 2026-05-13 14:05:47 +02:00
9466b091dd docs: rebuild manual PDF 2026-05-13 14:05:47 +02:00
511288bd03 build_all.sh: build wheel + pyinstaller + flatpak + appimage in one go
Collects all four artifacts under <repo>/dist/ (PyInstaller and Flatpak
renamed to testium-<version>(.suff); wheel and AppImage keep PEP 427 /
appimage-builder original names). Re-uses scripts/build_env.sh and
set_env.sh, same venv as run.sh. AppImage build.sh now picks the actual
output file dynamically instead of a hardcoded lowercase name.
2026-05-13 14:03:20 +02:00
51b144f60c Flatpak: bypass XDG portal for .tum open dialog
Native file dialog routes through the XDG document portal, which exposes
only the selected file at /run/user/UID/doc/... — siblings (param.yaml,
.py) are unreachable. Force Qt's non-native dialog in Flatpak so it walks
the real filesystem via --filesystem=home and returns a usable path.
2026-05-13 12:49:46 +02:00
dee8d4a682 generic design elements 2026-05-10 17:41:43 +02:00
e726d47547 generic design elements 2026-05-10 17:40:52 +02:00
5fd50e1c85 release_note: add 0.1.1 entry 2026-05-07 10:09:44 +02:00
51939a566a chore: add src/LICENSE, sync VERSION to 0.1.1 2026-05-07 10:06:17 +02:00
26fccda6bf docs(CLAUDE.md): rewrite Packaging section, document run launcher
Adds Flatpak/AppImage to the channels table, documents the host-only
contract and bins.py mechanics, explains the runtime-aware run launcher.
2026-05-07 10:06:06 +02:00
405fb82fca AppImage packaging: containerized build, host-only py_func/lua_func
build.sh runs appimage-builder in a Debian Bookworm container (Podman or
Docker) so it works on Arch / non-Debian hosts. Uses single src/requirements.txt;
TESTIUM_VERSION exported in runtime.env.
2026-05-07 10:05:58 +02:00
6064d96138 Flatpak packaging: desktop entry, MIME, distributable bundle
org.testium.Testium.yaml uses host Python/Lua only (no bundled interpreter).
build.sh exports a .flatpak bundle. README documents the install procedure.
2026-05-07 10:05:44 +02:00
0658540cc2 run item: runtime-aware launcher, drop testium_path/python_bin params
_testium_launch_cmd() returns the right entry point per mode (AppImage,
Flatpak, PyInstaller, source/wheel). Fixes PermissionError on read-only
__main__.py inside the AppImage squashfs mount.
2026-05-07 10:05:09 +02:00
7bf946dabe py_func/__main__: robust sys.path + diagnostic on import failure
Insert str() parent dir at sys.path[0] (was appending a Path object);
exception handler prints sys.executable and sys.path.
2026-05-07 10:05:00 +02:00
f52d7bbe53 runtime_plot: bump last_values timeout 1s -> 5s, narrow except
Dispatch chain (queue poll + Qt signal/slot + main thread) can exceed 1s on
loaded machines; bare except masked everything other than queue.Empty.
2026-05-07 10:04:44 +02:00
c83ebccb55 version: read TESTIUM_VERSION env in Flatpak/AppImage
Both bundles export it from their launcher; previous fragile __file__-relative
VERSION lookup was reporting 'unknown'.
2026-05-07 10:04:08 +02:00
f17ef8a3a1 dialog_env: pick wayland/xcb from $DISPLAY/$WAYLAND_DISPLAY
Was forcing xcb unconditionally, which hung dialogs on pure-Wayland sessions.
2026-05-07 10:03:54 +02:00
ddb18abc21 bins.py: host-only Python/Lua in sandboxed bundles
_which() probes host dirs only in Flatpak (/run/host/usr/bin) and AppImage
(/usr/bin); apply_host_libs prepares env for host subprocesses (prepend host
libs in Flatpak, strip $APPDIR pollution + PYTHONHOME in AppImage); user
override resolved via _which() for bare names.
2026-05-07 10:03:30 +02:00
72 changed files with 2544 additions and 249 deletions

3
.gitignore vendored
View File

@@ -8,6 +8,8 @@ dist
/.vscode
.venv/
.flatpak-builder/
package/flatpak/repo/
package/flatpak/*.flatpak
crash.tx*
report_test.tx*
*.autosave
@@ -24,6 +26,7 @@ package/appimage/*.AppImage
package/appimage/src
package/appimage/*.py
AppDir
*.squashfs
doc/manual/doxygen
doc/manual/sphinx/build/*
doc/manual/sphinx/source/_build/*

View File

@@ -1,4 +1,4 @@
# Testium — Claude Context
# Testium — Design Context
## What is testium
@@ -114,11 +114,20 @@ To add a new API call usable from subprocesses:
### External interpreter resolution (`bins.py`)
`src/testium/interpreter/utils/bins.py` — single source of truth for the paths to the external Python and Lua interpreters used by subprocesses.
- `python_bin()` / `lua_bin()` : resolve once, cache in memory. User can override via the `python_bin` / `lua_bin` global dict keys (typically populated from the YAML config). Falls back to discovery on PATH (candidates: `python3`/`python` and `lua`/`lua5.5`/`lua5.4`/`lua5.3`/`lua5.2`/`lua5.1`).
- `python_bin()` / `lua_bin()` : resolve and cache. The cache is keyed by `(name, override)` so that a later change to `gd[python_bin]` (typically when a `param.yaml` sets the key) triggers a re-resolution on the next lookup instead of returning the stale auto-discovered path. Falls back to discovery on PATH (candidates: `python3`/`python` and `lua`/`lua5.5`/`lua5.4`/`lua5.3`/`lua5.2`/`lua5.1`).
- `ensure(*names)` : called by `TestSet._validate_runtime_deps()` at test load. Always requires `python` (the eval engine always runs); requires `lua` only if a `lua_func` item is in the tree. Fails fast with a clear error citing tried candidates and override key.
Engines (`PyProcessBase`, `LuaProcessBase`, `EvalExecEngine`) call `bins.python_bin()`/`bins.lua_bin()` themselves — call sites never pass an explicit binary path.
#### Override-timing contract (`apply_overrides`)
`bins.python_bin()` is called for the **first** time inside `eval_process_init()` (the long-lived inline-`<| … |>` subprocess), which happens **before** the YAML param files are loaded. To make `-d python_bin=…` and the GUI `python_bin` preference take effect for `eval_proc` itself, `process.py:run()` applies them to gd **before** `eval_process_init()` via the `apply_overrides()` helper extracted from `update_global()`. The post-load `update_global()` call then re-applies the same overrides (after `prepare_global()` clears gd), keeping the gd value in sync with the cached resolution.
| Override source | `eval_proc` | `py_func` / `cycle` / `post_exec` |
|---|---|---|
| `-d python_bin=…` (CLI) | ✅ | ✅ |
| GUI `python_bin` preference | ✅ | ✅ |
| `python_bin: …` in `param.yaml` | ❌ (eval_proc already started) | ✅ (cache re-resolves on key change) |
## Key files
| Path | Role |
@@ -183,6 +192,8 @@ Icons are assigned once when the test file is loaded (not updated live on theme
The sub-test's own pass/fail result is intentionally not propagated.
The interpreter and entry point used to spawn the sub-instance are picked automatically by `_testium_launch_cmd()` based on how the parent was started (AppImage → `$APPIMAGE`; Flatpak → `flatpak run`; PyInstaller → the frozen binary; source/wheel → `[sys.executable, abspath(sys.argv[0])]`). The user cannot override either via the YAML — selecting a different testium binary or Python from a sub-test was removed because it was either ill-defined (bundle modes have no separable Python) or could mismatch the parent's environment in surprising ways.
### Report exporters & plugins
`src/testium/interpreter/test_report/test_report.py``_EXPORTER_REGISTRY` dict maps a format name (cmd key in the YAML `report.export`) to a lazy loader. Built-ins: `text`, `json`, `junit` (needs `junit_xml`), `html` (needs `lxml`). `sqlite` is the storage layer, no-op as an export.
@@ -201,17 +212,38 @@ A real-world test plugin lives at `test/validation/fake_exporter/` (CSV exporter
## Packaging
Three distribution channels coexist, sharing the single `src/testium/` package:
Four distribution channels coexist, all sharing the single `src/testium/` package and the single `src/requirements.txt` dependency list:
| Channel | Where | Notes |
|---------|-------|-------|
| Wheel (`pip install`) | `src/pyproject.toml` | Vanilla Python package; entry point `testium = "testium:main"` |
| PyInstaller binary | `package/pyinstaller/` | Single ~130 MB binary. `py_func`, `runtime`, `lua_func` bundled at `_MEIPASS` root so the **host** Python can find them when launched as `python3 py_func`. `api`/`interpreter` are **not** exposed (subprocess isolation). |
| Flatpak | `package/flatpak/` | (Existing recipe, not actively maintained in current refactor wave.) |
| Channel | Where | Build | Notes |
|---------|-------|-------|-------|
| Wheel (`pip install`) | `src/pyproject.toml` | `python -m build` | Vanilla Python package; entry point `testium = "testium:main"`. |
| PyInstaller binary | `package/pyinstaller/` | `build.sh` | Single ~130 MB binary. `py_func`, `runtime`, `lua_func` bundled at `_MEIPASS` root so the **host** Python can find them when launched as `python3 py_func`. `api`/`interpreter` are **not** exposed (subprocess isolation). |
| Flatpak | `package/flatpak/` | `build.sh` (uses `flatpak-builder`) | KDE 6.10 runtime. The bundled Python runs only the main process; `py_func` / `lua_func` MUST run under the **host** interpreter (no Python/Lua bundled). Produces a distributable `.flatpak` bundle. |
| AppImage | `package/appimage/` | `build.sh` (Debian Bookworm container via Podman/Docker) | Bundles Python 3.11 for the main process; `py_func` / `lua_func` MUST run under the **host** interpreter. Build runs in a container so it works on Arch / any non-Debian host. |
The `.deb` work-in-progress lives in `package/deb/`:
- `test_distro.sh debian:bookworm | debian:trixie | ubuntu:24.04` spins up a Docker/Podman container, reports system package availability, falls back to pip for what's missing (`pyside6` on bookworm/ubuntu, `telnetlib3`, `junit_xml`), runs the validation suite. Currently green on the three targets.
### Host-only py_func / lua_func in sandboxed bundles (Flatpak, AppImage)
The bundled Python (Flatpak's runtime python, AppImage's `python3.11`) is reserved for the **main process only**. Subprocesses (`py_func`, `lua_func`, `git`) must use the host's interpreters and tools so user-installed modules (pyserial, junit_xml, …) are visible. This is enforced by `interpreter/utils/bins.py`:
- `_in_flatpak()` (checks `/.flatpak-info`) and `_in_appimage()` (checks `APPIMAGE` env var) detect the sandbox.
- `_which(name)` probes only host bin dirs in those modes:
- Flatpak: `/run/host/usr/{local/,}bin`, `/run/host/bin` (host mounted via `--filesystem=host-os`).
- AppImage: `/usr/local/bin`, `/usr/bin`, `/bin` (we are directly on the host filesystem).
- If the host has no python3/lua, `ensure()` raises `ETUMRuntimeError` at test load with the candidate list — no silent fallback to a bundled interpreter.
- User overrides (`python_bin`/`lua_bin` in globdict): bare names are resolved through `_which()` (host-only), absolute paths are accepted as-is.
- `apply_host_libs(env)` is called by `py_process.py` / `lua_process.py` on the env passed to Popen:
- Flatpak: prepends host lib dirs to `LD_LIBRARY_PATH` so the dynamic linker finds host `.so`'s.
- AppImage: strips `$APPDIR`-prefixed entries from `LD_LIBRARY_PATH` / `PYTHONPATH` / `PATH` and drops `PYTHONHOME`, so the host Python doesn't try to load the bundled stdlib/site-packages.
- `apply_host_lua_paths(env)` (Flatpak only) prepends `/run/host/usr/{lib,share}/lua/X.Y` to `LUA_PATH` / `LUA_CPATH` so `cjson`, `socket`, etc. resolve. Must be called **after** user `lua_env` overrides so host paths win. AppImage relies on host Lua's compiled-in defaults.
- `py_process.py` additionally pops `PYTHONUSERBASE` (set to `/var/data/python` by the Flatpak runtime, which would hide `~/.local/lib/...`).
### Version reporting (`interpreter/utils/version.py`)
Both Flatpak and AppImage export `TESTIUM_VERSION` from a launcher (Flatpak: launcher script in `org.testium.Testium.yaml`; AppImage: `runtime.env` in `AppImageBuilder.yml`). `get_testium_version()` checks `/.flatpak-info` / `APPIMAGE` and reads `TESTIUM_VERSION` rather than relying on package metadata or repo introspection.
## Recent fixes / notable changes
- Restructure: single `src/testium/` Python package (was 4 sibling top-levels: `testium`, `lib`, `py_func`, `lua_func`). `lib/``runtime/`, `libs/``api/`. `pip install` now produces a clean `site-packages/testium/` with no top-level pollution; `.lua` files travel via `package_data`.
- `bins.py`: centralised resolution + cache of external `python3` / `lua` binaries. Replaces the scattered `tm.gd("python_bin")`/`tm.gd("lua_bin")` dance and the duplicated discovery logic in `py_process.py`/`lua_process.py`. Validates at test load via `TestSet._validate_runtime_deps()` so missing interpreters fail fast.
@@ -238,12 +270,18 @@ The `.deb` work-in-progress lives in `package/deb/`:
- `unittest` item: renamed from `unittest_file`.
- GUI test tree: check and fold state preserved across same-file reloads.
- Licence: EUPL-1.2.
- Interpreter override timing: `apply_overrides()` extracted from `update_global()` and called by `process.py:run()` before `eval_process_init()`, so `-d python_bin=…` / GUI prefs reach `bins.python_bin()` on its first lookup. `bins._resolve()` cache is now keyed by `(name, override)` so later `param.yaml` changes are picked up by subsequently constructed engines.
## Validation tests
Located in `test/validation/`. Run with `-b` flag:
Located in `test/validation/`. Two entry points:
```
./run.sh -b -- test/validation/main.tum
./test/validation/run.sh # wrapper — uses a dedicated venv (see below)
./run.sh -b -- test/validation/main.tum # direct — testium's own python is used for test execution
```
The `run.sh` / `run.bat` wrappers create a dedicated Python venv at `${TMPDIR:-/tmp}/testium-validation-venv` (Linux) or `%TEMP%\testium-validation-venv` (Windows), with `--system-site-packages` + `pip install junit-xml`, and run the suite with `-d python_bin=…` so every test-execution subprocess (eval_proc, py_func, cycle, post_exec) runs inside the venv. testium itself keeps running in the project's own environment. `clean` as the first argument recreates the venv.
The `venv` item (`test/validation/items/venv/`) asserts that the override actually took effect: `python_bin` is set, `sys.executable` matches it, `sys.prefix == dirname(dirname(python_bin))`, and `sys.prefix != sys.base_prefix` (the last marker catches the case where `python_bin` happens to be a system interpreter, which path-equality alone would miss because the venv's `bin/python3` is a symlink to the host). Both `eval_proc` (inline `<| … |>`) and `py_func` paths are exercised.
Parallel item tests: `test/validation/items/parallel/test.tum`
## Dependencies

View File

@@ -27,7 +27,19 @@ Pre-built artifacts are published at
runnable directly, no Python installation required on the host. Lua
support still needs a system `lua` interpreter and the `lua-socket` /
`lua-cjson` modules.
* **Flatpak** — *coming soon.*
* **Flatpak bundle** (`testium.flatpak`) — install with:
```sh
# Add Flathub (once, to fetch the KDE/PySide runtimes)
flatpak remote-add --user --if-not-exists flathub https://flathub.org/repo/flathub.flatpakrepo
# Install the bundle
flatpak install --user testium.flatpak
```
After installation testium appears in the desktop application menu and the
`testium` command is available in the terminal (requires `~/.local/bin` in
`PATH`, which most modern distributions provide by default).
## Quick start

164
build_all.sh Executable file
View File

@@ -0,0 +1,164 @@
#!/bin/bash
# Build every distribution channel of testium, in order:
# 1. Manual PDF -> dist/testium-manual-<v>.pdf
# 2. Wheel -> dist/testium-<v>-py3-none-any.whl (PEP 427 name)
# 3. PyInstaller binary -> dist/testium-<v>
# 4. Flatpak bundle -> dist/testium-<v>.flatpak
# 5. AppImage -> dist/Testium-<v>-x86_64.AppImage (original name)
# release_note.txt is copied to dist/ up front (with a warning if it has no
# entry for the current version).
#
# By default, a step is skipped if its artifact already exists in dist/.
# Pass --clean to remove existing dist/ artifacts and rebuild everything.
#
# All artifacts are collected (copied) under <repo>/dist/. Original outputs in
# src/dist/, package/*/dist/, doc/manual/ are left in place. Wheel and AppImage
# keep their original names (which already contain the version); manual,
# pyinstaller and flatpak are renamed to testium(-manual)-<version>(.suff).
#
# Re-uses scripts/build_env.sh and scripts/set_env.sh — the same pair invoked
# by run.sh — so the venv at test/tmp/.venv stays the single source of Python
# dependencies. `build`, `pyinstaller`, `sphinx` and `linuxdoc` are installed
# into that venv on demand if not already there. Flatpak and AppImage build in
# their own container/sandbox; their build.sh scripts have their own toolchain
# checks.
set -e
CLEAN=0
for arg in "$@"; do
case "$arg" in
--clean|-c) CLEAN=1 ;;
*) echo "Unknown option: $arg" >&2; exit 1 ;;
esac
done
SCRIPT_DIR=$(realpath "$(dirname "$0")")
VERSION=$(cat "$SCRIPT_DIR/src/VERSION")
DIST_DIR="$SCRIPT_DIR/dist"
mkdir -p "$DIST_DIR"
if [ "$CLEAN" -eq 1 ]; then
echo "-- clean: removing existing dist artifacts for version $VERSION"
rm -f "$DIST_DIR/testium-manual-${VERSION}.pdf"
rm -f "$DIST_DIR"/testium-${VERSION}-*.whl
rm -f "$DIST_DIR/testium-${VERSION}"
rm -f "$DIST_DIR/testium-${VERSION}.flatpak"
rm -f "$DIST_DIR"/Testium-${VERSION}-*.AppImage
fi
# Release note: copy it to dist/ and warn (but don't fail) if it has no entry
# for the current version.
RELEASE_NOTE_SRC="$SCRIPT_DIR/release_note.txt"
RELEASE_NOTE="$DIST_DIR/release_note.txt"
cp -f "$RELEASE_NOTE_SRC" "$RELEASE_NOTE"
if ! grep -qE "^version $VERSION([^.0-9]|$)" "$RELEASE_NOTE_SRC"; then
echo "WARNING: release_note.txt has no entry for version $VERSION." >&2
fi
export PY_VENV_NAME=".venv"
export PY_VENV_DIR="$SCRIPT_DIR/test/tmp/$PY_VENV_NAME"
export REQ_PATH="$SCRIPT_DIR/src/requirements.txt"
bash "$SCRIPT_DIR/scripts/build_env.sh"
source "$SCRIPT_DIR/scripts/set_env.sh"
step() {
echo
echo "================================================================"
echo " $1"
echo "================================================================"
}
skip() { echo " (already built — skipping)"; }
# 1. Manual PDF
MANUAL="$DIST_DIR/testium-manual-${VERSION}.pdf"
step "1/5 Manual PDF (version $VERSION)"
if [ ! -f "$MANUAL" ]; then
python -m pip install --quiet --upgrade sphinx linuxdoc
bash "$SCRIPT_DIR/doc/manual/sphinx/build_doc.sh"
cp -f "$SCRIPT_DIR/doc/manual/testium_manual.pdf" "$MANUAL"
else
skip
fi
# 2. Wheel — PEP 427 name kept (already contains version)
step "2/5 Wheel (version $VERSION)"
WHEEL=$(ls -1t "$DIST_DIR"/testium-${VERSION}-*.whl 2>/dev/null | head -1)
if [ -z "$WHEEL" ]; then
python -m pip install --quiet --upgrade build
(
cd "$SCRIPT_DIR/src"
rm -rf dist build *.egg-info
python -m build --wheel
)
WHEEL_SRC=$(ls -1t "$SCRIPT_DIR/src/dist"/*.whl | head -1)
WHEEL="$DIST_DIR/$(basename "$WHEEL_SRC")"
cp -f "$WHEEL_SRC" "$WHEEL"
else
skip
fi
# 3. PyInstaller binary
PYI_BIN="$DIST_DIR/testium-${VERSION}"
step "3/5 PyInstaller binary (version $VERSION)"
if [ ! -f "$PYI_BIN" ]; then
python -m pip install --quiet --upgrade pyinstaller
bash "$SCRIPT_DIR/package/pyinstaller/build.sh"
cp -f "$SCRIPT_DIR/package/pyinstaller/dist/testium" "$PYI_BIN"
else
skip
fi
# 4. Flatpak bundle
FLATPAK_BUNDLE="$DIST_DIR/testium-${VERSION}.flatpak"
step "4/5 Flatpak bundle (version $VERSION)"
if [ ! -f "$FLATPAK_BUNDLE" ]; then
FLATPAK_DEPS=(
"org.kde.Platform//6.10"
"org.kde.Sdk//6.10"
"io.qt.PySide.BaseApp//6.10"
)
if ! flatpak remotes --user | grep -q "^flathub"; then
echo " Adding Flathub remote"
flatpak remote-add --user --if-not-exists flathub https://dl.flathub.org/repo/flathub.flatpakrepo
fi
for dep in "${FLATPAK_DEPS[@]}"; do
if ! flatpak info --user "$dep" &>/dev/null && ! flatpak info --system "$dep" &>/dev/null; then
echo " Installing Flatpak dependency: $dep"
flatpak install --user --noninteractive flathub "$dep"
fi
done
(
cd "$SCRIPT_DIR/package/flatpak"
bash build.sh
)
cp -f "$SCRIPT_DIR/package/flatpak/testium.flatpak" "$FLATPAK_BUNDLE"
else
skip
fi
# 5. AppImage
step "5/5 AppImage (version $VERSION)"
APPIMAGE=$(ls -1t "$DIST_DIR"/Testium-${VERSION}-*.AppImage 2>/dev/null | head -1)
if [ -z "$APPIMAGE" ]; then
(
cd "$SCRIPT_DIR/package/appimage"
bash build.sh
)
APPIMAGE_SRC=$(ls -1t "$SCRIPT_DIR/package/appimage"/*.AppImage 2>/dev/null | head -1)
APPIMAGE="$DIST_DIR/$(basename "$APPIMAGE_SRC")"
cp -f "$APPIMAGE_SRC" "$APPIMAGE"
chmod +x "$APPIMAGE"
else
skip
fi
step "All packages built"
printf " manual : %s\n" "$MANUAL"
printf " wheel : %s\n" "$WHEEL"
printf " pyinstaller : %s\n" "$PYI_BIN"
printf " flatpak : %s\n" "$FLATPAK_BUNDLE"
printf " appimage : %s\n" "$APPIMAGE"
printf " release_note : %s\n" "$RELEASE_NOTE"

View File

@@ -20,6 +20,22 @@ main:
param:
- 123
- py_func:
name: python long wait
doc: The purpose of this step is to try the tasks "stop" interruption
file: utils.py
func_name: long_wait
param:
- 10
- lua_func:
name: lua long wait
doc: The purpose of this step is to try the tasks "stop" interruption
file: lua_func.lua
func_name: long_wait
param:
- 10
- sleep:
name: sleep item
dialog: true

View File

@@ -1,4 +1,5 @@
tm = require("tm")
socket = require("socket")
local module = {}
@@ -7,4 +8,8 @@ function module.func_to_be_executed(param)
return param
end
function module.long_wait(sec)
socket.sleep(sec)
end
return module

View File

@@ -17,18 +17,3 @@ plot_log_path: /tmp/testium_plot/$(testrun_date)/$(testrun_time)/
python_path_Windows: C:\Users\François\Applications\Python313\python.exe
python_path_Linux: $(home)/tmp/tum_venv/bin/python3
# lua_bin_Windows: C:\Lua\5.1
# lua_bin_Linux: /usr/bin/lua
LUA_PATH_Linux: /usr/share/lua/5.4/?.lua;/usr/local/share/lua/5.4/?.lua;/usr/local/share/lua/5.4/?/init.lua;/usr/share/lua/5.4/?/init.lua;/usr/local/lib/lua/5.4/?.lua;/usr/local/lib/lua/5.4/?/init.lua;/usr/lib/lua/5.4/?.lua;/usr/lib/lua/5.4/?/init.lua;./?.lua;./?/init.lua;/home/francois/.luarocks/share/lua/5.4/?.lua;/home/francois/.luarocks/share/lua/5.4/?/init.lua
LUA_CPATH_Linux: /usr/local/lib/lua/5.4/?.so;/usr/lib/lua/5.4/?.so;/usr/local/lib/lua/5.4/loadall.so;/usr/lib/lua/5.4/loadall.so;./?.so;/home/francois/.luarocks/lib/lua/5.4/?.so
PATH_Linux:
LUA_PATH_Windows: ;.\?.lua;C:\Lua\5.1\lua\?.lua;C:\Lua\5.1\lua\?\init.lua;C:\Lua\5.1\?.lua;C:\Lua\5.1\?\init.lua;C:\Lua\5.1\lua\?.luac
LUA_CPATH_Windows: .\?.dll;C:\Lua\5.1\?.dll;C:\Lua\5.1\loadall.dll;C:\Lua\5.1\clibs\?.dll;C:\Lua\5.1\clibs\loadall.dll;.\?51.dll;C:\Lua\5.1\?51.dll;C:\Lua\5.1\clibs\?51.dll
PATH_Windows: ""
lua_env:
PATH: $(PATH_$(os))
LUA_PATH: $(LUA_PATH_$(os))
LUA_CPATH: $(LUA_CPATH_$(os))

View File

@@ -1,3 +1,5 @@
from time import sleep
def dummy_exit(useless1, useless2):
return True
@@ -11,3 +13,6 @@ def funcToBeExecuted (bla):
def funcToBeExecuted2 (bla):
print(bla)
return blo
def long_wait (sec):
sleep(sec)

Binary file not shown.

View File

@@ -24,9 +24,8 @@ AppDir:
runtime:
env:
SEQUENCER_REV: '{{APP_VERSION}}'
TESTIUM_VERSION: '{{APP_VERSION}}'
PYTHONPATH: $APPDIR/usr/lib/python3.11/site-packages:$APPDIR/usr/lib/python3.11
QT_QPA_PLATFORM: xcb
path_mappings:
- /usr/share/matplotlib/mpl-data/matplotlibrc:$APPDIR/etc/matplotlibrc
@@ -69,12 +68,13 @@ AppDir:
# Set python 3.11 as default
ln -fs python3.11 $TARGET_APPDIR/usr/bin/python3
# Install pip
if [ ! -f "get-pip.py" ]; then curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py; fi
# Bootstrap pip into the AppDir Python
if [ ! -f "get-pip.py" ]; then curl -sS https://bootstrap.pypa.io/get-pip.py -o get-pip.py; fi
python3.11 get-pip.py --break-system-packages
# Install application dependencies in AppDir
python3.11 -m pip install --break-system-packages --upgrade --isolated --no-input --ignore-installed --prefix=$TARGET_APPDIR/usr -r requirements.txt
python3.11 -m pip install --break-system-packages --upgrade --isolated --no-input --ignore-installed --prefix=$TARGET_APPDIR/usr -r ../../src/requirements.txt
export PIP_CONFIG_FILE=$HOME/.pip/pip.conf
python3.11 -m pip install --break-system-packages --upgrade --isolated --no-input --ignore-installed --prefix=$TARGET_APPDIR/usr ../../src/dist/testium-{{APP_VERSION}}-py3-none-any.whl

View File

@@ -1,12 +1,54 @@
#!/usr/bin/bash
#!/bin/bash
# Build the testium AppImage inside a Debian container (Podman or Docker).
# The resulting .AppImage file is written to this directory.
export APP_VERSION=$(<../../src/VERSION)
set -e
appimage-builder --recipe AppImageBuilder.yml
REPO_ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
APP_VERSION="$(<"$REPO_ROOT/src/VERSION")"
RESULT=$?
if [ -n "$1" ] && [ "$1" = "install" ]; then
if [ $RESULT -eq 0 ]; then
install -v "testium-${APP_VERSION}-x86_64.AppImage" "${HOME}/.local/bin/testium"
fi
if command -v podman &>/dev/null; then
RUNTIME=podman
elif command -v docker &>/dev/null; then
RUNTIME=docker
else
echo "Error: neither podman nor docker found." >&2
exit 1
fi
echo "Using $RUNTIME — building testium $APP_VERSION AppImage..."
# APPIMAGE_EXTRACT_AND_RUN=1 lets appimagetool run without FUSE in the container.
$RUNTIME run --rm \
--privileged \
-e APPIMAGE_EXTRACT_AND_RUN=1 \
-v "$REPO_ROOT:/work" \
-w /work/package/appimage \
debian:bookworm bash -c "
set -e
export DEBIAN_FRONTEND=noninteractive
apt-get update -qq
apt-get install -y -qq \
python3 python3-pip python3-venv python3-build \
dpkg-dev fakeroot squashfs-tools wget curl file binutils \
libglib2.0-0 patchelf zsync > /dev/null
# Build the wheel
cd /work/src
python3 -m build --wheel --outdir dist/ > /dev/null
cd /work/package/appimage
# Install appimage-builder
pip3 install appimage-builder --quiet --break-system-packages
# Run the build
export APP_VERSION=$APP_VERSION
appimage-builder --recipe AppImageBuilder.yml --skip-test
"
APPIMAGE_FILE=$(ls -1t Testium-*-x86_64.AppImage 2>/dev/null | head -1)
echo "Done: ${APPIMAGE_FILE}"
if [ "${1}" = "install" ] && [ -n "${APPIMAGE_FILE}" ]; then
install -v "${APPIMAGE_FILE}" "${HOME}/.local/bin/testium"
fi

View File

@@ -1,3 +0,0 @@
tables
pandas
scapy

View File

@@ -5,4 +5,22 @@
# flatpak install flathub org.kde.Sdk//6.10
# flatpak install flathub io.qt.PySide.BaseApp//6.10
flatpak-builder --user --verbose --force-clean --install build org.testium.Testium.yaml
set -e
# Build + install local
flatpak-builder --user --verbose --force-clean --install --repo=repo build org.testium.Testium.yaml
# Génère le bundle distribuable
flatpak build-bundle repo testium.flatpak org.testium.Testium
echo "Bundle généré : $(pwd)/testium.flatpak"
# Crée ~/.local/bin/testium pour pouvoir taper "testium" en console
WRAPPER="$HOME/.local/bin/testium"
mkdir -p "$HOME/.local/bin"
cat > "$WRAPPER" <<'EOF'
#!/bin/sh
exec flatpak run org.testium.Testium "$@"
EOF
chmod +x "$WRAPPER"
echo "Wrapper installé : $WRAPPER"
echo "Assurez-vous que ~/.local/bin est dans votre PATH."

View File

@@ -0,0 +1,7 @@
<?xml version="1.0" encoding="UTF-8"?>
<mime-info xmlns="http://www.freedesktop.org/standards/shared-mime-info">
<mime-type type="application/x-testium">
<comment>Testium test script</comment>
<glob pattern="*.tum"/>
</mime-type>
</mime-info>

View File

@@ -0,0 +1,10 @@
[Desktop Entry]
Name=Testium
GenericName=Test Sequencer
Comment=YAML-based test sequencer and runner
Exec=testium %f
Icon=org.testium.Testium
Type=Application
Categories=Development;
MimeType=application/x-testium;
StartupNotify=true

View File

@@ -13,7 +13,14 @@ finish-args:
- --socket=wayland
- --device=dri
- --share=network
- --filesystem=home # Optionnel : si votre testium doit lire des fichiers utilisateurs
- --filesystem=home
- --filesystem=/tmp
- --filesystem=host-os
# Allow flatpak-spawn --host to launch host binaries (Python, Lua, git…)
# outside the sandbox. Required because the sandbox glibc/ABI is
# incompatible with arbitrary host shared libraries — we route py_func and
# lua_func through the host instead.
- --talk-name=org.freedesktop.Flatpak
build-options:
build-args:
@@ -41,18 +48,41 @@ modules:
sources:
- type: dir
path: ../../src
- type: file
path: org.testium.Testium.desktop
- type: file
path: org.testium.Testium-mime.xml
- type: file
path: ../../package/testium.png
build-commands:
# On installe le code source dans /app/lib/testium
# Code source
- mkdir -p /app/lib
- cp -r . /app/lib/
- cp -r testium /app/lib/
- cp VERSION /app/lib/testium/VERSION
# Création du launcher exécutable
# Launcher exécutable
- mkdir -p /app/bin
- |
cat <<EOF > /app/bin/testium
#!/bin/sh
# On ajoute le code source et l'extension PySide6 au PYTHONPATH
export TESTIUM_VERSION="\$(cat /app/lib/testium/VERSION 2>/dev/null || echo unknown)"
export PYTHONPATH="/app/lib/testium:/usr/lib/sdk/pyside6/lib/python3.13/site-packages:\$PYTHONPATH"
exec python3 /app/lib/testium "\$@"
# Expose host binaries (git, python3, lua, …) for subprocess lookups.
# PATH is appended (not prepended) so the main process keeps the sandbox python3.
export PATH="\$PATH:/run/host/usr/local/bin:/run/host/usr/bin:/run/host/bin"
export GIT_PYTHON_GIT_EXECUTABLE="/run/host/usr/bin/git"
exec /usr/bin/python3 /app/lib/testium "\$@"
EOF
- chmod +x /app/bin/testium
# Icône
- mkdir -p /app/share/icons/hicolor/256x256/apps
- cp testium.png /app/share/icons/hicolor/256x256/apps/org.testium.Testium.png
# Entrée menu
- mkdir -p /app/share/applications
- cp org.testium.Testium.desktop /app/share/applications/
# Type MIME pour .tum
- mkdir -p /app/share/mime/packages
- cp org.testium.Testium-mime.xml /app/share/mime/packages/

View File

@@ -1,3 +1,55 @@
version 0.2
==============
- Test items: each item type now declares its accepted parameters
(``PARAMS = ParamSet(...)``). Typos in a ``.tum`` are surfaced as a
WARN listing the accepted names instead of being silently ignored;
missing required parameters error out at load time with the source
``.tum`` file as context. No change to valid existing tests.
version 0.1.3
==============
- Stop interrupts engaged blocking steps (console, py_func, lua_func,
json_rpc, sleep) within ~200 ms instead of waiting for the step
to finish.
- GUI Start / Stop / Pause flow simplified.
- lua_func: a function returning nil is no longer reported as a failure.
- ``-d python_bin=...`` and the GUI ``python_bin`` preference now reach
the eval subprocess (used to be silently ignored). ``param.yaml`` can
also override ``python_bin`` for py_func / cycle / post_exec.
- Validation suite: ``test/validation/run.sh`` (and ``run.bat``)
runs the suite inside a dedicated venv in the system temp dir.
- build_all.sh: ``release_note.txt`` and the user manual copied into
``dist/``; warning if the file has no entry for the version being built.
- Flatpak: every GUI file/directory dialog (open test, save report, log
path, default report/log dirs, python/lua interpreter pickers) now
bypasses the XDG document portal — the v0.1.2 fix was only on the
"open test" dialog.
- Flatpak: py_func / lua_func / run sub-instance now execute on the host
via flatpak-spawn, lifting the previous glibc/ABI incompatibility that
prevented user-configured host Python or Lua interpreters from being
reached from the sandbox.
- Validation suite: single entry point with ``--mode source|wheel|
pyinstaller|flatpak|appimage`` to validate any packaging channel
against the same item set; reports are stamped per mode.
- GUI: the "Run tum" test item now uses the testium logo.
version 0.1.2
==============
- Flatpak: opening a test from the GUI now correctly finds its companion
files (param.yaml, .py scripts, ...).
version 0.1.1
==============
- New install channels: Flatpak bundle and AppImage. The AppImage runs
on any distribution (built inside a Debian container).
- About dialog: version is now correct in Flatpak and AppImage builds
(used to display "unknown").
- GUI dialogs no longer hang on pure-Wayland sessions.
- Plot "last values" API: more tolerant timeout on loaded machines.
- run item: `testium_path` and `python_bin` parameters removed —
sub-instances are launched in the same packaging mode as the parent.
- License: EUPL-1.2.
version 0.1
==============
- Start of the project

View File

@@ -20,6 +20,12 @@ if [ "$?" -ne 0 ]; then
echo "venv must be installed on the host distribution."
exit -1
fi
# Check if venv is installed
python3 -c "import ensurepip"
if [ "$?" -ne 0 ]; then
echo "ensurepip must be installed on the host distribution."
exit -1
fi
# Install the virtual environment if needed
if [ ! -d "$PY_VENV_DIR" ]; then

315
src/LICENSE Normal file
View File

@@ -0,0 +1,315 @@
Copyright (c) 2025-2026 François Dausseur
Licensed under the EUPL
EUROPEAN UNION PUBLIC LICENCE v. 1.2
EUPL © the European Union 2007, 2016
This European Union Public Licence (the 'EUPL') applies to the Work (as
defined below) which is provided under the terms of this Licence. Any use of
the Work, other than as authorised under this Licence is prohibited (to the
extent such use is covered by a right of the copyright holder of the Work).
The Work is provided under the terms of this Licence when the Licensor (as
defined below) has placed the following notice immediately following the
copyright notice for the Work:
Licensed under the EUPL
or has expressed by any other means his willingness to license under the EUPL.
1. Definitions
In this Licence, the following terms have the following meaning:
- 'The Licence': this Licence.
- 'The Original Work': the work or software distributed or communicated by the
Licensor under this Licence, available as Source Code and also as Executable
Code as the case may be.
- 'Derivative Works': the works or software that could be created by the
Licensee, based upon the Original Work or modifications thereof. This Licence
does not define the extent of modification or dependence on the Original
Work required in order to classify a work as a Derivative Work; this extent
is determined by copyright law applicable in the country mentioned in
Article 15.
- 'The Work': the Original Work or its Derivative Works.
- 'The Source Code': the human-readable form of the Work which is the most
convenient for people to study and modify.
- 'The Executable Code': any code which has generally been compiled and which
is meant to be interpreted by a computer as a program.
- 'The Licensor': the natural or legal person that distributes or communicates
the Work under the Licence.
- 'Contributor(s)': any natural or legal person who modifies the Work under
the Licence, or otherwise contributes to the creation of a Derivative Work.
- 'The Licensee' or 'You': any natural or legal person who makes any usage of
the Work under the terms of the Licence.
- 'Distribution' or 'Communication': any act of selling, giving, lending,
renting, distributing, communicating, transmitting, or otherwise making
available, online or offline, copies of the Work or providing access to its
essential functionalities at the disposal of any other natural or legal
person.
2. Scope of the rights granted by the Licence
The Licensor hereby grants You a worldwide, royalty-free, non-exclusive,
sublicensable licence to do the following, for the duration of copyright
vested in the Original Work:
- use the Work in any circumstance and for all usage,
- reproduce the Work,
- modify the Work, and make Derivative Works based upon the Work,
- communicate to the public, including the right to make available or display
the Work or copies thereof to the public and perform publicly, as the case
may be, the Work,
- distribute the Work or copies thereof,
- lend and rent the Work or copies thereof,
- sublicense rights in the Work or copies thereof.
Those rights can be exercised on any media, supports and formats, whether now
known or later invented, as far as the applicable law permits so.
In the countries where moral rights apply, the Licensor waives his right to
exercise his moral right to the extent allowed by law in order to make
effective the licence of the economic rights here above listed.
The Licensor grants to the Licensee royalty-free, non-exclusive usage rights
to any patents held by the Licensor, to the extent necessary to make use of
the rights granted on the Work under this Licence.
3. Communication of the Source Code
The Licensor may provide the Work either in its Source Code form, or as
Executable Code. If the Work is provided as Executable Code, the Licensor
provides in addition a machine-readable copy of the Source Code of the Work
along with each copy of the Work that the Licensor distributes or indicates,
in a notice following the copyright notice attached to the Work, a repository
where the Source Code is easily and freely accessible for as long as the
Licensor continues to distribute or communicate the Work.
4. Limitations on copyright
Nothing in this Licence is intended to deprive the Licensee of the benefits
from any exception or limitation to the exclusive rights of the rights owners
in the Work, of the exhaustion of those rights or of other applicable
limitations thereto.
5. Obligations of the Licensee
The grant of the rights mentioned above is subject to some restrictions and
obligations imposed on the Licensee. Those obligations are the following:
Attribution right: The Licensee shall keep intact all copyright, patent or
trademarks notices and all notices that refer to the Licence and to the
disclaimer of warranties. The Licensee must include a copy of such notices
and a copy of the Licence with every copy of the Work he/she distributes or
communicates. The Licensee must cause any Derivative Work to carry prominent
notices stating that the Work has been modified and the date of modification.
Copyleft clause: If the Licensee distributes or communicates copies of the
Original Works or Derivative Works, this Distribution or Communication will
be done under the terms of this Licence or of a later version of this Licence
unless the Original Work is expressly distributed only under this version of
the Licence — for example by communicating 'EUPL v. 1.2 only'. The Licensee
(becoming Licensor) cannot offer or impose any additional terms or conditions
on the Work or Derivative Work that alter or restrict the terms of the
Licence.
Compatibility clause: If the Licensee Distributes or Communicates Derivative
Works or copies thereof based upon both the Work and another work licensed
under a Compatible Licence, this Distribution or Communication can be done
under the terms of this Compatible Licence. For the sake of this clause,
'Compatible Licence' refers to the licences listed in the appendix attached
to this Licence. Should the Licensee's obligations under the Compatible
Licence conflict with his/her obligations under this Licence, the obligations
of the Compatible Licence shall prevail.
Provision of Source Code: When distributing or communicating copies of the
Work, the Licensee will provide a machine-readable copy of the Source Code or
indicate a repository where this Source will be easily and freely available
for as long as the Licensee continues to distribute or communicate the Work.
Legal Protection: This Licence does not grant permission to use the trade
names, trademarks, service marks, or names of the Licensor, except as
required for reasonable and customary use in describing the origin of the
Work and reproducing the content of the copyright notice.
6. Chain of Authorship
The original Licensor warrants that the copyright in the Original Work
granted hereunder is owned by him/her or licensed to him/her and that he/she
has the power and authority to grant the Licence.
Each Contributor warrants that the copyright in the modifications he/she
brings to the Work are owned by him/her or licensed to him/her and that
he/she has the power and authority to grant the Licence.
Each time You accept the Licence, the original Licensor and subsequent
Contributors grant You a licence to their contributions to the Work, under
the terms of this Licence.
7. Disclaimer of Warranty
The Work is a work in progress, which is continuously improved by numerous
Contributors. It is not a finished work and may therefore contain defects or
'bugs' inherent to this type of development.
For the above reason, the Work is provided under the Licence on an 'as is'
basis and without warranties of any kind concerning the Work, including
without limitation merchantability, fitness for a particular purpose, absence
of defects or errors, accuracy, non-infringement of intellectual property
rights other than copyright as stated in Article 6 of this Licence.
This disclaimer of warranty is an essential part of the Licence and a
condition for the grant of any rights to the Work.
8. Disclaimer of Liability
Except in the cases of wilful misconduct or damages directly caused to
natural persons, the Licensor will in no event be liable for any direct or
indirect, material or moral, damages of any kind, arising out of the Licence
or of the use of the Work, including without limitation, damages for loss of
goodwill, work stoppage, computer failure or malfunction, loss of data or any
commercial damage, even if the Licensor has been advised of the possibility
of such damage. However, the Licensor will be liable under statutory product
liability laws as far such laws apply to the Work.
9. Additional agreements
While distributing the Work, You may choose to conclude an additional
agreement, defining obligations or services consistent with this Licence.
However, if accepting obligations, You may act only on your own behalf and on
your sole responsibility, not on behalf of the original Licensor or any other
Contributor, and only if You agree to indemnify, defend, and hold each
Contributor harmless for any liability incurred by, or claims asserted
against such Contributor by the fact You have accepted any warranty or
additional liability.
10. Acceptance of the Licence
The provisions of this Licence can be accepted by clicking on an icon 'I
agree' placed under the bottom of a window displaying the text of this
Licence or by affirming consent in any other similar way, in accordance with
the rules of applicable law. Clicking on that icon indicates your clear and
irrevocable acceptance of this Licence and all of its terms and conditions.
Similarly, you irrevocably accept this Licence and all of its terms and
conditions by exercising any rights granted to You by Article 2 of this
Licence, such as the use of the Work, the creation by You of a Derivative
Work or the Distribution or Communication by You of the Work or copies
thereof.
11. Information to the public
In case of any Distribution or Communication of the Work by means of
electronic communication by You (for example, by offering to download the
Work from a remote location) the distribution channel or media (for example,
a website) must at least provide to the public the information requested by
the applicable law regarding the Licensor, the Licence and the way it may be
accessible, concluded, stored and reproduced by the Licensee.
12. Termination of the Licence
The Licence and the rights granted hereunder will terminate automatically
upon any breach by the Licensee of the terms of the Licence.
Such a termination will not terminate the licences of any person who has
received the Work from the Licensee under the Licence, provided such persons
remain in full compliance with the Licence.
13. Miscellaneous
Without prejudice of Article 9 above, the Licence represents the complete
agreement between the Parties as to the Work.
If any provision of the Licence is invalid or unenforceable under applicable
law, this will not affect the validity or enforceability of the Licence as a
whole. Such provision will be construed or reformed so as necessary to make
it valid and enforceable.
The European Commission may publish other linguistic versions or new versions
of this Licence or updated versions of the Appendix, so far this is required
and reasonable, without reducing the scope of the rights granted by the
Licence. New versions of the Licence will be published with a unique version
number.
All linguistic versions of this Licence, approved by the European Commission,
have identical value. Parties can take advantage of the linguistic version of
their choice.
14. Jurisdiction
Without prejudice to specific agreement between parties,
- any litigation resulting from the interpretation of this License, arising
between the European Union institutions, bodies, offices or agencies, as a
Licensor, and any Licensee, will be subject to the jurisdiction of the
Court of Justice of the European Union, as laid down in article 272 of the
Treaty on the Functioning of the European Union,
- any litigation arising between other parties and resulting from the
interpretation of this License, will be subject to the exclusive
jurisdiction of the competent court where the Licensor resides or conducts
its primary business.
15. Applicable Law
Without prejudice to specific agreement between parties,
- this Licence shall be governed by the law of the European Union Member
State where the Licensor has his seat, resides or has his registered
office,
- this licence shall be governed by Belgian law if the Licensor has no seat,
residence or registered office inside a European Union Member State.
Appendix
'Compatible Licences' according to Article 5 EUPL are:
- GNU General Public License (GPL) v. 2, v. 3
- GNU Affero General Public License (AGPL) v. 3
- Open Software License (OSL) v. 2.1, v. 3.0
- Eclipse Public License (EPL) v. 1.0
- CeCILL v. 2.0, v. 2.1
- Mozilla Public Licence (MPL) v. 2
- GNU Lesser General Public Licence (LGPL) v. 2.1, v. 3
- Creative Commons Attribution-ShareAlike v. 3.0 Unported (CC BY-SA 3.0) for
works other than software
- European Union Public Licence (EUPL) v. 1.1, v. 1.2
- Québec Free and Open-Source Licence — Reciprocity (LiLiQ-R) or Strong
Reciprocity (LiLiQ-R+).
The European Commission may update this Appendix to later versions of the
above licences without producing a new version of the EUPL, as long as they
provide the rights granted in Article 2 of this Licence and protect the
covered Source Code from exclusive appropriation.
All other changes or additions to this Appendix require the production of a
new EUPL version.

View File

@@ -11,6 +11,7 @@ import threading
from telnetlib3 import Telnet, DO, WILL, WONT, TTYPE, IAC, SB, SE, theNULL
TIMEOUT_NULL = 0.000001
STOP_POLL_INTERVAL = 0.2
class BytesStore(object):
@@ -123,12 +124,14 @@ A {classname}.close() is missing somewhere in your code !'.format(classname=type
# c = ''
return c
def read_until(self, match, timeout=None, return_data=False, mute=False):
def read_until(self, match, timeout=None, return_data=False, mute=False, should_stop=None):
"""
read until the string 'match is found
If timeout is not set (None), this function runs indefinitely
If timeout is set to zero, this function returns immediately
If mute is set to True the characters read from the console will not be displayed
If should_stop is a callable, it is polled between reads (every STOP_POLL_INTERVAL
at most) and the loop exits early — like a timeout — when it returns True.
If function fails (because of a timeout) it will return a 'status' integer set to -1
otherwise it will return 0.
@@ -139,13 +142,6 @@ A {classname}.close() is missing somewhere in your code !'.format(classname=type
status = -1
if not match:
raise ValueError('match parameter can not be empty')
# replace all '\r' by '\n' as any '\r' read will undergo the same replacement
# match = match.replace('\r\n', '\n')
# match = match.replace('\r', '')
# update the console timeout in conformity with what is required.
self.set_read_timeout(timeout)
if timeout is None:
timeout = 1000000
@@ -159,6 +155,7 @@ A {classname}.close() is missing somewhere in your code !'.format(classname=type
# buffer is empty
# Otherwise we are waiting for the timeout to rise
if timeout < TIMEOUT_NULL:
self.set_read_timeout(0)
data = self.readchar(0)
while (status < 0) and ((data is not None) and (data != b'')):
@@ -191,16 +188,21 @@ A {classname}.close() is missing somewhere in your code !'.format(classname=type
# Timeout different than zero
else:
# Poll in short chunks so a stop request is honored within
# STOP_POLL_INTERVAL, regardless of the per-protocol blocking
# behavior of readchar().
self.set_read_timeout(STOP_POLL_INTERVAL)
time_is_out = threading.Event()
timer = threading.Timer(timeout, lambda: time_is_out.set())
timer.start()
# We are waiting for the timeout to rise
try:
while (status < 0) and (not time_is_out.is_set()):
if should_stop is not None and should_stop():
break
while (status < 0) and (not time_is_out.isSet()):
data = self.readchar(timeout)
data = self.readchar(STOP_POLL_INTERVAL)
if data is not None:
data = self._compute_char(data)
if data != '':
@@ -210,7 +212,6 @@ A {classname}.close() is missing somewhere in your code !'.format(classname=type
search_deque.append(data)
if search_deque == match_deque:
timer.cancel()
status = 0
if (not mute) and (data != '\n'):
self.string_buffer += '\n'
@@ -224,6 +225,8 @@ A {classname}.close() is missing somewhere in your code !'.format(classname=type
date_str = str(datetime.now()).split('.')[0].split(' ')[1]
self.string_buffer = '[{} {}]'.format(date_str, self.name)
finally:
timer.cancel()
if return_data:
return status, read_data

View File

@@ -1,5 +1,6 @@
import sys
import os
import queue
import multiprocessing as mp
from threading import Timer
from time import sleep, monotonic
@@ -367,7 +368,7 @@ class RuntimePlot:
self.msg_queue_in.get()
self.msg_queue_out.put({"command": "last_values"})
try:
res = self.msg_queue_in.get(timeout=1)
except:
res = self.msg_queue_in.get(timeout=5)
except queue.Empty:
raise ETUMRuntimeError(f"Impossible to retrieve the last values of the \"{self.name}\" plot")
return res

View File

@@ -16,6 +16,7 @@ from interpreter.utils.test_init import (
env_init,
prepare_global,
update_global,
apply_overrides,
set_standard_gd_keys,
test_run_init,
test_run_header,
@@ -210,6 +211,19 @@ class TestProcess(Process):
env_init()
# Apply GUI defaults and CLI defines to the global dict
# *before* eval_proc starts: bins.python_bin() reads
# ``python_bin`` from gd on its very first call (during
# eval_process_init) and caches the result. Without this,
# ``-d python_bin=...`` and the GUI ``python_bin`` preference
# would only take effect for items spawned *after* the cache
# was already populated with the auto-discovered interpreter,
# i.e. they would silently be ignored for eval_proc itself.
# _load_initial_params re-applies the same overrides after
# ``prepare_global()`` clears gd, so the gd value stays in
# sync with the cached path.
apply_overrides(self.__defs, self.__gui_defaults)
# Creation of the python evaluation process for loading of the complete test
eval_proc = eval_process_init(api_request, 10, test_dir)
eval_proc.start()

View File

@@ -10,6 +10,8 @@ import os
def setup():
"""Configure the Qt environment for dialog subprocess usage."""
if sys.platform.startswith('linux'):
# On Linux/Wayland, force X11 (via XWayland) to avoid crashes
# when Qt is initialized inside a multiprocessing subprocess.
if os.environ.get('DISPLAY'):
# X11 available: force xcb to avoid crashes in multiprocessing subprocesses.
os.environ['QT_QPA_PLATFORM'] = 'xcb'
elif os.environ.get('WAYLAND_DISPLAY'):
os.environ['QT_QPA_PLATFORM'] = 'wayland'

View File

@@ -5,6 +5,9 @@ from copy import deepcopy
from interpreter.test_items.test_result import TestResult, TestValue
import api.testium as tm
from interpreter.utils.params import TestItemParams
from interpreter.utils.param_decl import (
Param, ParamSet, LIST, BLOCK, unknown_keys, missing_required,
)
from interpreter.utils.constants import TestItemType as cst_type
from interpreter.utils.eval import eval_to_boolean, evaluate, post_evaluate
from runtime.tum_except import ETUMSyntaxError, item_load_context
@@ -13,6 +16,32 @@ LOG_TEST_STOP = '<----- step "{}" finished'
LOG_TEST_START = '-----> step "{}" started'
# Parameters accepted by every test item, regardless of its type. Subclasses
# concatenate their own ``PARAMS`` to this set; the merged result drives
# unknown-param warnings and (later) the LSP schema export.
COMMON_PARAMS = ParamSet(
Param("name", doc="Display name shown in the GUI tree and reports."),
Param("doc", doc="Free-form documentation; surfaced in tooltips."),
Param("skipped", doc="If truthy, the step is skipped (static expression, "
"evaluated at load time)."),
Param("key", doc="Report key used to classify the result "
"(typically <test>_PASS or <test>_FAIL)."),
Param("stop_on_failure", doc="If true, abort the surrounding container on failure."),
Param("execute_on_stop", doc="If true, run this step even when its container "
"is being stopped (cleanup)."),
Param("process_result", doc="Post-evaluation expression applied to the test result."),
Param("store_result", doc="Global-dict key in which to store the test result."),
Param("expected_result", doc="Expected outcome; the step is failed if it doesn't match."),
Param("no_fail", doc="If truthy, never report a FAILURE for this step."),
Param("report", doc="Per-step reporting override."),
Param("condition", doc="Optional gating expression evaluated before each "
"run; false ⇒ the step is skipped."),
Param("steps", kind=LIST, doc="Children (for container items)."),
Param("seq_filename", doc="(internal) source .tum file of this step; injected "
"by the loader."),
)
class TestItem:
pass
@@ -20,9 +49,26 @@ class TestItem:
def test_run(f):
@wraps(f)
def wrapper(self):
if not self.skipped:
if self.enabled:
if self.skipped:
self.result.set(TestValue.NORUN, "test skipped")
print("Test is skipped.")
return self.result
if not self.enabled:
self.result.set(TestValue.NORUN, "test disabled")
print("Test is disabled.")
return self.result
self.run_test_init()
while self._is_paused:
sleep(0.2)
if self.isStopped() :
self.result.set(TestValue.NORUN, "test stopped")
print("Test is Stopped.")
self._is_stopped = False # Restore state for next run
return self.result
# Conditional execution
raw_condition = self._prms.getParam(
"condition", default=None, processed=False
@@ -57,15 +103,10 @@ def test_run(f):
self.result.set(TestValue.NORUN, msg)
self.result.reported = {"input_condition": msg}
self.run_test_end()
else:
self.result.set(TestValue.NORUN, "test disabled")
print("Test is disabled.")
else:
self.result.set(TestValue.NORUN, "test skipped")
print("Test is skipped.")
return self.result
return wrapper
@@ -85,6 +126,11 @@ def test_data(item: TestItem, child: dict) -> dict:
class TestItem:
# Subclasses override with their own ParamSet to opt into the declarative
# validation. While ``PARAMS`` is empty / unset, the base class skips the
# unknown-param check for this item type — keeps the migration incremental.
PARAMS = None
def __init__(
self, dict_item: dict = None, parent: TestItem = None, status_queue=None, filename = ""
):
@@ -122,6 +168,13 @@ class TestItem:
# creation of the params object
self._prms = TestItemParams(dict_item, parent)
# Declarative-params validation. Only kicks in when the concrete
# subclass declares ``PARAMS`` — items not yet migrated stay
# silent. Warnings (not errors) during the migration window so
# existing .tum files don't break suddenly; will be flipped to
# errors once every item has migrated.
self._validate_declared_params(dict_item)
# getting parameters for the test item
try:
self._name = self._prms.getParam("name", default="", processed=True)
@@ -178,6 +231,36 @@ class TestItem:
self.result = TestResult(self, TestValue.FAILURE, "Failure by default")
def _validate_declared_params(self, dict_item):
"""Warn on unknown / missing-required params, if PARAMS is declared.
The check is opt-in per subclass: it only runs when the concrete
class sets a non-empty ``PARAMS`` attribute. Items not yet migrated
produce no diagnostics — preserving the historical "silently accept
anything" behavior until they get their declaration.
"""
if not self.PARAMS:
return
# ``self._type`` is the parent root type at this point (subclasses set
# it after super().__init__), so use the class name as a stable label
# in diagnostics. ``self._name`` was preset to the type name by every
# subclass before super() ran, which gives a useful prefix.
label = f"{type(self).__name__} '{self._name}'"
declared = COMMON_PARAMS + self.PARAMS
unknown = unknown_keys(declared, dict_item)
if unknown:
accepted = ", ".join(sorted(declared.names()))
for k in unknown:
tm.print_warn(
f"{label}: unknown parameter '{k}'. Accepted: {accepted}."
)
missing = missing_required(declared, dict_item)
for k in missing:
raise ETUMSyntaxError(
f"{label}: required parameter '{k}' is missing.",
self._seq_filename,
)
def _filter_dict_item(self, dict_item):
# Stores the content of the step to be displayed
# in the GUI
@@ -255,8 +338,6 @@ class TestItem:
self._sendStatusStarted()
if self._is_breakpoint:
self._is_paused = True
while self._is_paused:
sleep(0.2)
if self.is_container:
self.report.incLevel()
@@ -274,9 +355,6 @@ class TestItem:
if self.is_container:
self.report.decLevel()
while self._is_paused:
sleep(0.2)
# Post evaluation of the test result
self.process_result()
# expected_result treatment
@@ -311,6 +389,7 @@ class TestItem:
self.report.addTest(self, self.result, rk)
self._sendStatusFinished()
def process_result(self):
if self._post_eval is None:
return

View File

@@ -5,11 +5,22 @@ from runtime.tum_except import ETUMSyntaxError, item_load_context
import api.testium as tm
from interpreter.utils.constants import TestItemType as cst
from interpreter.utils.eval import evaluate
from interpreter.utils.param_decl import Param, ParamSet, LIST
class TestItemCheckValue(TestItem):
"""check item usage.
check usage:{check: {name: check my func output, steps: ['$(pfn_echo) < 5']}}
"""
PARAMS = ParamSet(
Param("values", kind=LIST, required=True,
doc="List of expressions to evaluate. Each is expanded then "
"evaluated; non-truthy results fail the check."),
# 'steps' is intentionally not redeclared here — it's the deprecated
# alias of 'values' and is already accepted by COMMON_PARAMS for
# container items. A runtime warning is emitted when 'steps' is used.
)
def __init__(self, dict_item, parent = None, status_queue=None, filename=""):
self._name = cst.TYPE_CHECK.item_name
super().__init__(dict_item, parent, status_queue, filename=filename)

View File

@@ -2,11 +2,25 @@ from interpreter.test_items.test_item import test_run
from interpreter.test_items.test_result import TestValue
from interpreter.test_items.test_item_dialog_base import TestItemDialogBase, _is_text_mode, _is_interactive
from interpreter.utils.constants import TestItemType as cst
from interpreter.utils.param_decl import Param, ParamSet, BLOCK
from runtime.tum_except import item_load_context
import api.testium as tm
class TestItemChoicesDialog(TestItemDialogBase):
PARAMS = ParamSet(
Param("question", required=True,
doc="Prompt shown above the list of choices."),
Param("choices", kind=BLOCK, required=True,
doc="Tree of choices: either a list of strings, or a nested "
"mapping {label: subchoices, ...} to build a multi-level menu."),
Param("icon", default=None,
doc="Default icon name shown next to each choice."),
Param("auto_result", default=None,
doc="Batch-mode selection (path or label). None ⇒ FAILURE."),
)
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
self._name = cst.TYPE_CHOICES_DLG.item_name
super().__init__(dict_item, parent, status_queue, filename=filename)

View File

@@ -10,6 +10,7 @@ from interpreter.test_items.test_item import test_run
from interpreter.test_items.item_actions import TestItemActions
from interpreter.test_items.item_actions.action import TestItemAction
from interpreter.utils.constants import TestItemType as cst
from interpreter.utils.param_decl import Param, ParamSet
from interpreter.test_items.test_result import TestResult, TestValue
@@ -21,6 +22,38 @@ class TestItemConsoleAction(TestItemAction):
class TestItemConsoleOpen(TestItemConsoleAction):
PARAMS = ParamSet(
Param("protocol", required=True,
doc="Transport: 'telnet', 'ssh', 'rawtcp', 'serial' or 'terminal'."),
Param("write_delay", default=0,
doc="Inter-character write delay in ms (slow devices)."),
Param("log", doc="Path to a log file capturing the console traffic."),
Param("overwrite_log", default=True,
doc="If true, truncate the log file at open; else append."),
# telnet
Param("telnet_host", doc="Hostname/IP for the telnet target."),
Param("telnet_port", default=69, doc="TCP port for telnet."),
# ssh
Param("ssh_host", doc="Hostname/IP for the SSH target."),
Param("ssh_user", doc="SSH login user."),
Param("ssh_pwd", doc="SSH password (if key-based auth is not used)."),
# rawtcp
Param("tcp_host", doc="Hostname/IP for a raw-TCP connection."),
Param("tcp_port", doc="TCP port for a raw-TCP connection."),
# serial
Param("serial_port", doc="Serial device path (e.g. /dev/ttyUSB0 or COM3)."),
Param("serial_baudrate", doc="Serial baudrate."),
Param("buffered", default=True,
doc="If true, the serial console buffers received bytes between reads."),
# terminal
Param("terminal_path",
doc="Working directory for the local terminal protocol."),
Param("shell",
doc="Shell command used for the local terminal protocol "
"(default: 'cmd.exe' on Windows, '/usr/bin/env bash' elsewhere)."),
)
def __init__(
self, action_name, dict_item, parent=None, status_queue=None, filename=""
):
@@ -283,6 +316,17 @@ class TestItemConsoleWriteLn(TestItemConsoleAction):
class TestItemConsoleReadUntil(TestItemConsoleAction):
PARAMS = ParamSet(
Param("expected", required=True,
doc="Regex matched against incoming console output until found "
"or until timeout."),
Param("timeout", default=-1,
doc="Seconds before giving up. Negative means infinite."),
Param("mute", default=False,
doc="If true, don't echo received bytes to testium's stdout/log."),
)
def __init__(
self, action_name, dict_item, parent=None, status_queue=None, filename=""
):
@@ -307,11 +351,17 @@ class TestItemConsoleReadUntil(TestItemConsoleAction):
try:
status, data = cons.read_until(
ru, timeout=read_timeout, return_data=True, mute=mute
ru, timeout=read_timeout, return_data=True, mute=mute,
should_stop=self.isStopped,
)
if status == 0:
self.result.set(TestValue.SUCCESS)
self.result.value = data
elif self.isStopped():
self.result.set(
result=TestValue.FAILURE,
message="Console read aborted on stop request",
)
else:
self.result.set(result=TestValue.FAILURE, message="No matching text")
if mute:
@@ -330,6 +380,14 @@ class TestItemConsoleReadUntil(TestItemConsoleAction):
class TestItemConsole(TestItemActions):
PARAMS = ParamSet(
Param("console_name", required=True,
doc="Identifier of the console — used by every nested action to "
"reach back the same transport. Multiple consoles can coexist "
"as long as their names differ."),
)
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
super().__init__(
cst.TYPE_CONSOLE, dict_item, parent, status_queue, filename=filename

View File

@@ -8,9 +8,36 @@ from interpreter.test_items.test_result import TestResult, TestValue
import api.testium as tm
from interpreter.utils.params import TestItemParams
from interpreter.utils.constants import TestItemType as cst
from interpreter.utils.param_decl import Param, ParamSet, BLOCK
# Sub-block validation: 'cycle' accepts an 'exit_condition:' mapping whose
# own params are reported here so unknown keys inside it can be flagged
# during a future Block-aware diagnostic pass. For now the parent only
# declares that 'exit_condition' is an accepted top-level key.
EXIT_CONDITION_PARAMS = ParamSet(
Param("time", doc="HH:MM time of day after which the loop exits."),
Param("value", doc="Expression; when truthy the loop exits."),
Param("file", doc="Python file containing the exit-condition function."),
Param("func_name", doc="Function name in 'file' returning the exit value."),
Param("param", doc="Arguments passed to the exit function."),
Param("eval", default="",
doc="Post-evaluation expression applied to the function's return."),
)
class TestItemCycle(TestItem):
PARAMS = ParamSet(
Param("iterator",
doc="Iterable (or string expanding to one) driving the loop. "
"The current value is exposed as $(loop_param)."),
Param("exit_condition", kind=BLOCK,
doc="Optional block stopping the loop early: combine 'time', "
"'value', or a 'file'+'func_name' pair (with optional "
"'param' and 'eval')."),
)
def __init__(self, dict_cycle, parent=None, status_queue=None, filename=""):
self._name = cst.TYPE_CYCLE.item_name
super().__init__(dict_cycle, parent, status_queue, filename=filename)

View File

@@ -1,6 +1,7 @@
from interpreter.test_items.test_item import (TestItem, test_run)
from interpreter.test_items.test_result import (TestValue)
from interpreter.utils.constants import TestItemType as cst
from interpreter.utils.param_decl import Param, ParamSet, LIST
from runtime.tum_except import ETUMParamError, ETUMSyntaxError
import interpreter.utils.version as git
@@ -8,6 +9,13 @@ class TestItemGit(TestItem):
"""
This item expect only one parameter which is a string or list of string being the path to the git folder
"""
PARAMS = ParamSet(
Param("repo", kind=LIST, required=True,
doc="Path to a git checkout, or list of such paths. Each is "
"reported with its current version (tag + dirty state)."),
)
def __init__(self, dict_item, parent = None, status_queue=None, filename=""):
self._name = cst.TYPE_GIT.item_name
super().__init__(dict_item, parent, status_queue, filename=filename)

View File

@@ -1,10 +1,17 @@
from interpreter.test_items.test_item import (TestItem, test_run)
from interpreter.test_items.test_result import (TestResult, TestValue)
from interpreter.utils.constants import TestItemType as cst
from interpreter.utils.param_decl import ParamSet
from runtime.tum_except import ETUMSyntaxError
import api.testium as tm
class TestItemGroup(TestItem):
# 'group' has no item-specific parameters; 'steps' is handled by COMMON_PARAMS.
# Declaring an empty ParamSet still opts in to unknown-param validation
# (e.g. typo 'stop_on_failures').
PARAMS = ParamSet()
def __init__(self, dict_cycle, parent = None, status_queue=None, filename=""):
self._name = cst.TYPE_GROUP.item_name
super().__init__(dict_cycle, parent, status_queue, filename=filename)

View File

@@ -4,6 +4,7 @@ from interpreter.test_items.test_item import test_run
from interpreter.test_items.test_result import TestValue
from interpreter.test_items.test_item_dialog_base import TestItemDialogBase, _is_text_mode, _is_interactive
from interpreter.utils.constants import TestItemType as cst
from interpreter.utils.param_decl import Param, ParamSet
from runtime.tum_except import item_load_context
import api.testium as tm
@@ -12,6 +13,17 @@ class TestItemImageDialog(TestItemDialogBase):
"""dialog_image item usage.
dialog_image name: Nice image, question: could you press the red button, filename: img.jpg
"""
PARAMS = ParamSet(
Param("question", required=True,
doc="Prompt shown above the image."),
Param("filename", required=True,
doc="Path to the image file (relative to the test directory or absolute)."),
Param("auto_result", default=None,
doc="Outcome used in batch/non-interactive mode. Truthy ⇒ SUCCESS, "
"None ⇒ FAILURE."),
)
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
self._name = cst.TYPE_IMAGE_DLG.item_name
super().__init__(dict_item, parent, status_queue, filename=filename)

View File

@@ -11,6 +11,7 @@ from interpreter.test_items.item_actions.action import TestItemAction
from interpreter.utils.constants import TestItemType as cst
from interpreter.utils.eval import evaluate
from interpreter.utils.param_decl import Param, ParamSet, BLOCK
from interpreter.test_items.test_item_json_rpc.jsonrpc_adapters import (
JrpcAdapter,
@@ -76,6 +77,20 @@ class TestItemJSRPCActionClose(TestItemAction):
class TestItemJSRPCActionQuery(TestItemAction):
PARAMS = ParamSet(
Param("method", required=True,
doc="JSON-RPC method name to call."),
Param("params",
doc="Parameters payload (list, dict or scalar) sent to the method."),
Param("id", default="rand",
doc="JSON-RPC request id. 'rand' (default) ⇒ a random integer is used."),
Param("no_wait", default=False,
doc="If true, send the request without waiting for a response."),
Param("timeout", default=None,
doc="Seconds to wait for a response. None ⇒ inherits the transport "
"default."),
)
def __init__(
self, action_name, dict_item, parent=None, status_queue=None, filename=""
):
@@ -105,6 +120,7 @@ class TestItemJSRPCActionQuery(TestItemAction):
jrpc_id = randint(1, (2**32) - 1)
send_only = self._prms.expanse(self._send_only)
timeout = self._prms.expanse(self._timeout)
self.token.set_should_stop(self.isStopped)
try:
success, result = self.token.query(
meth, obj, jrpc_id, send_only, timeout=timeout
@@ -128,6 +144,13 @@ class TestItemJSRPCActionQuery(TestItemAction):
class TestItemJSRPCActionReceive(TestItemAction):
PARAMS = ParamSet(
Param("id", required=True,
doc="JSON-RPC request id whose response we expect."),
Param("timeout", default=None,
doc="Seconds to wait for the response. None ⇒ transport default."),
)
def __init__(
self, action_name, dict_item, parent=None, status_queue=None, filename=""
):
@@ -146,6 +169,7 @@ class TestItemJSRPCActionReceive(TestItemAction):
def execute(self):
timeout = self._prms.expanse(self._timeout)
jrpc_id = self._prms.expanse(self._jrpc_id)
self.token.set_should_stop(self.isStopped)
try:
success, result = self.token.receive(jrpc_id, timeout)
@@ -170,6 +194,22 @@ class TestItemJSON_RPC(TestItemActions):
This item TBD
"""
PARAMS = ParamSet(
Param("console", kind=BLOCK,
doc="Console-transport block: {console_name, …}. Either 'console' "
"or 'udp' must be set."),
Param("udp", kind=BLOCK,
doc="UDP-transport block: {host, port, …}. Either 'console' or "
"'udp' must be set."),
Param("version", default="1.0",
doc="JSON-RPC protocol version ('1.0' or '2.0')."),
Param("timeout", required=True,
doc="Default seconds to wait for a JSON-RPC response across all "
"child query/receive actions."),
Param("mute", default=False,
doc="If true, don't echo wire traffic to the log."),
)
def __init__(
self, dict_item: dict, parent: TestItem = None, status_queue=None, filename=""
):

View File

@@ -2,10 +2,11 @@ import json
import socket
import re
import struct
import time
from runtime.tum_except import ETUMRuntimeError
import api.testium as tm
from api.console import Console
from api.console import Console, STOP_POLL_INTERVAL
def is_ip_address(address):
@@ -45,9 +46,16 @@ class JrpcAdapter:
self._jrpc_version = version
self._mute = mute
self._timeout = timeout
# Optional callable polled by _receive() implementations to abort
# waits early when the test is being stopped. Set by the test item
# action before each query/receive call.
self._should_stop = None
if not (version == "1.0" or version == "2.0"):
raise ETUMRuntimeError("Invalid JSONRPC version passed.")
def set_should_stop(self, cb):
self._should_stop = cb
@property
def timeout(self):
return self._timeout
@@ -249,13 +257,23 @@ class JrpcUdpAdapter(JrpcAdapter):
print(f" | sent to @{self._server}:{self._snd_port}")
def _receive(self, timeout: float) -> str:
# configures the reception timeout
self.sock.settimeout(timeout)
# Receives the answer from the server
# Poll in short chunks so a stop request is honored within
# STOP_POLL_INTERVAL.
self.sock.settimeout(STOP_POLL_INTERVAL)
deadline = time.monotonic() + float(timeout)
data = None
addr = None
while True:
if self._should_stop is not None and self._should_stop():
raise ETUMRuntimeError("JSONRPC udp receive aborted on stop request.")
try:
data, addr = self.sock.recvfrom(self._bufsize)
break
except socket.timeout:
if time.monotonic() >= deadline:
raise ETUMRuntimeError(
"JSONRPC udp answer took too long. Try to increase the timeout."
)
# In case of buffer overload we chose to complain
if len(data) >= self._bufsize:
@@ -271,10 +289,6 @@ class JrpcUdpAdapter(JrpcAdapter):
print(f" | UDP answer: '{res}'")
print(f" | received from @{addr[0]}:{addr[1]}")
except socket.timeout:
raise ETUMRuntimeError(
"JSONRPC udp answer took too long. Try to increase the timeout."
)
return res
def _build_query(self, method: str, obj, jrpc_id: int):
@@ -339,11 +353,16 @@ class JrpcConsoleAdapter(JrpcAdapter):
def _receive(self, timeout: float) -> str:
status, data = self._cons.read_until(
self._endswith, timeout, return_data=True, mute=self._mute
self._endswith, timeout, return_data=True, mute=self._mute,
should_stop=self._should_stop,
)
# if we did not receive anything, we complain
if not status == 0:
if self._should_stop is not None and self._should_stop():
raise ETUMRuntimeError(
f"JSONRPC console receive aborted on stop request."
)
raise ETUMRuntimeError(
f"The '{self._cons.name}' console did not answer in the requested time."
)

View File

@@ -8,12 +8,20 @@ from interpreter.test_items.test_result import (TestResult, TestValue)
from runtime.tum_except import ETUMSyntaxError, item_load_context
import api.testium as tm
from interpreter.utils.constants import TestItemType as cst
from interpreter.utils.param_decl import Param, ParamSet, LIST
class TestItemLet(TestItem):
"""let item usage.
let values: {variable1: a, variable2: /dev/ttyUSB0, variable3: 115200}
let eval: {conditional_exec: "random.randint(1, 4)"}
"""
PARAMS = ParamSet(
Param("values", kind=LIST, required=True,
doc="Mapping (or list of single-pair mappings) of global-dict "
"key → value to set. Values are expanded at execution time."),
)
def __init__(self, dict_item, parent = None, status_queue=None, filename=""):
self._name = cst.TYPE_LET.item_name
super().__init__(dict_item, parent, status_queue, filename=filename)

View File

@@ -11,6 +11,7 @@ import api.testium as tm
from interpreter.utils.lua_func_exec import LuaFuncExecEngine
from interpreter.utils.api_srv import api_request
from interpreter.utils.constants import TestItemType as cst
from interpreter.utils.param_decl import Param, ParamSet, LIST
_LUA_FUNC_CONTEXTS_KEY = "_lua_func_contexts"
@@ -21,6 +22,21 @@ class TestItemLuaFunc(TestItem):
Optional: context_id: <id> — share a persistent process with other lua_func items using the same id.
"""
PARAMS = ParamSet(
Param("file", required=True,
doc="Path to the .lua file containing the function."),
Param("func_name", required=True,
doc="Name of the function to call in the file."),
Param("param", kind=LIST,
doc="Arguments passed to the function. Each entry is expanded "
"before the call. Special tokens $(loop_param) / $(loop_index) "
"resolve from the surrounding cycle."),
Param("context_id", default=None,
doc="If set, the lua_func subprocess is kept alive and reused by "
"every other lua_func item with the same context_id — enables "
"shared in-memory state between successive calls."),
)
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
self._name = cst.TYPE_LUA_FUNCTION.item_name
super().__init__(dict_item, parent, status_queue, filename=filename)
@@ -45,6 +61,18 @@ class TestItemLuaFunc(TestItem):
tm.setgd(_LUA_FUNC_CONTEXTS_KEY, contexts)
return contexts[ctx_id], True
def stop(self):
super().stop()
# Tear down the worker so any in-flight func_call returns promptly.
# join() clears _rpc/_process so a subsequent item reusing the same
# context_id can restart the engine cleanly.
try:
engine, _ = self._get_engine()
engine.stop()
engine.join()
except Exception:
pass
@test_run
def execute(self):
self.result.set(
@@ -96,8 +124,14 @@ Is the lua environnment well defined in the "LUA_PATH" and "LUA_CPATH" variables
return
except ConnectionAbortedError:
self.result.set(TestValue.FAILURE, "lua_func aborted on stop request")
print("lua_func aborted on stop request.")
except:
traceback.print_exception(*sys.exc_info())
if self.isStopped():
self.result.set(TestValue.FAILURE, "lua_func aborted on stop request")
else:
self.result.set(
TestValue.FAILURE,
'Unrecoverable "lua_func" item error from {}'.format(self.func_name),

View File

@@ -5,6 +5,7 @@ from interpreter.test_items.test_item import test_run
from interpreter.test_items.test_result import TestValue
from interpreter.test_items.test_item_dialog_base import TestItemDialogBase, _is_text_mode, _is_interactive
from interpreter.utils.constants import TestItemType as cst
from interpreter.utils.param_decl import Param, ParamSet
from runtime.tum_except import item_load_context
@@ -12,6 +13,15 @@ class TestItemMsgDialog(TestItemDialogBase):
"""dialog_message item usage.
dialog_message name: Nice message, question: Open the door and press OK
"""
PARAMS = ParamSet(
Param("question", required=True,
doc="Message body shown to the user. Multi-line strings are supported."),
Param("auto_result", default=None,
doc="Outcome used in batch/non-interactive mode instead of waiting "
"for the user. Truthy ⇒ SUCCESS, None ⇒ FAILURE."),
)
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
self._name = cst.TYPE_MESSAGE_DLG.item_name
super().__init__(dict_item, parent, status_queue, filename=filename)

View File

@@ -2,11 +2,23 @@ from interpreter.test_items.test_item import test_run
from interpreter.test_items.test_result import TestValue
from interpreter.test_items.test_item_dialog_base import TestItemDialogBase, _is_text_mode, _is_interactive
from interpreter.utils.constants import TestItemType as cst
from interpreter.utils.param_decl import Param, ParamSet
from runtime.tum_except import item_load_context
import api.testium as tm
class TestItemNoteDialog(TestItemDialogBase):
PARAMS = ParamSet(
Param("question", required=True,
doc="Prompt shown above the note input field."),
Param("auto_result", default=None,
doc="Batch-mode outcome: None ⇒ FAILURE, 'cancel' ⇒ cancelled, "
"any other truthy ⇒ SUCCESS with auto_value."),
Param("auto_value", default=None,
doc="Note text used in batch mode when auto_result is set."),
)
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
self._name = cst.TYPE_NOTE_DLG.item_name
super().__init__(dict_item, parent, status_queue, filename=filename)

View File

@@ -6,6 +6,7 @@ from interpreter.test_items.test_item import test_run
from interpreter.test_items.test_result import TestResult, TestValue
from interpreter.utils.constants import TestItemType as cst
from interpreter.utils.eval import eval_to_boolean
from interpreter.utils.param_decl import Param, ParamSet, LIST, BLOCK, Enum
from runtime.tum_except import ETUMSyntaxError
from runtime.string_queue import StringQueue
from runtime.stdout_redirect import stdio_redir
@@ -15,6 +16,12 @@ class TestItemParallelBranch(TestItemContainer):
"""One branch of a parallel item. Runs its children sequentially,
optionally waiting for a condition before starting."""
PARAMS = ParamSet(
Param("wait_for", kind=BLOCK,
doc="Optional block {condition, timeout} that defers the branch "
"start until the condition is truthy (or the timeout elapses)."),
)
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
super().__init__(cst.TYPE_PARALLEL_BRANCH, dict_item, parent, status_queue, filename=filename)
self._wait_condition = None
@@ -87,6 +94,15 @@ class TestItemParallel(TestItemContainer):
- ...
"""
PARAMS = ParamSet(
Param("branches", kind=LIST, required=True,
doc="List of branch blocks (each branch holds its own 'steps' "
"and optional 'wait_for')."),
Param("sync", kind=Enum("all", "any"), default="all",
doc="'all' (default) waits for every branch; 'any' returns as "
"soon as the first branch completes."),
)
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
branches = dict_item.get("branches", [])
if not branches:

View File

@@ -11,6 +11,7 @@ import api.testium as tm
from interpreter.utils.py_func_exec import PyFuncExecEngine
from interpreter.utils.api_srv import api_request
from interpreter.utils.constants import TestItemType as cst
from interpreter.utils.param_decl import Param, ParamSet, LIST
_PY_FUNC_CONTEXTS_KEY = "_py_func_contexts"
@@ -21,6 +22,21 @@ class TestItemPyFunc(TestItem):
Optional: context_id: <id> — share a persistent process with other py_func items using the same id.
"""
PARAMS = ParamSet(
Param("file", required=True,
doc="Path to the .py file containing the function."),
Param("func_name", required=True,
doc="Name of the function to call in the file."),
Param("param", kind=LIST,
doc="Arguments passed to the function. Each entry is expanded "
"before the call. Special tokens $(loop_param) / $(loop_index) "
"resolve from the surrounding cycle."),
Param("context_id", default=None,
doc="If set, the py_func subprocess is kept alive and reused by "
"every other py_func item with the same context_id — enables "
"shared in-memory state between successive calls."),
)
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
self._name = cst.TYPE_PY_FUNCTION.item_name
super().__init__(dict_item, parent, status_queue, filename=filename)
@@ -45,6 +61,18 @@ class TestItemPyFunc(TestItem):
tm.setgd(_PY_FUNC_CONTEXTS_KEY, contexts)
return contexts[ctx_id], True
def stop(self):
super().stop()
# Tear down the worker so any in-flight func_call returns promptly.
# join() clears _rpc/_process so a subsequent item reusing the same
# context_id can restart the engine cleanly.
try:
engine, _ = self._get_engine()
engine.stop()
engine.join()
except Exception:
pass
@test_run
def execute(self):
self.result.set(
@@ -94,8 +122,14 @@ python_bin = {tm.gd("python_bin", "no python path defined")}"""
return
except ConnectionAbortedError:
self.result.set(TestValue.FAILURE, "py_func aborted on stop request")
print("py_func aborted on stop request.")
except:
traceback.print_exception(*sys.exc_info())
if self.isStopped():
self.result.set(TestValue.FAILURE, "py_func aborted on stop request")
else:
self.result.set(
TestValue.FAILURE,
'Unrecoverable "py_func" item error from {}'.format(self.func_name),

View File

@@ -2,6 +2,7 @@ from interpreter.test_items.test_item import test_run
from interpreter.test_items.test_result import TestValue
from interpreter.test_items.test_item_dialog_base import TestItemDialogBase, _is_text_mode, _is_interactive
from interpreter.utils.constants import TestItemType as cst
from interpreter.utils.param_decl import Param, ParamSet
from runtime.tum_except import item_load_context
@@ -9,6 +10,14 @@ class TestItemQuestionDialog(TestItemDialogBase):
"""dialog_question item usage.
dialog_question name: Nice question, question: "If OK, press OK, If not, press cancel"
"""
PARAMS = ParamSet(
Param("question", required=True,
doc="Yes/No prompt presented to the user."),
Param("auto_result", default=None,
doc="Batch-mode answer ('yes'/'no' or truthy/falsy). None ⇒ FAILURE."),
)
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
self._name = cst.TYPE_QUESTION_DLG.item_name
super().__init__(dict_item, parent, status_queue, filename=filename)

View File

@@ -3,9 +3,17 @@ from interpreter.test_items.test_item import (TestItem, test_run)
from interpreter.test_items.test_result import (TestValue)
from runtime.tum_except import ETUMSyntaxError
from interpreter.utils.constants import TestItemType as cst
from interpreter.utils.param_decl import Param, ParamSet, LIST
from interpreter.test_report.test_report import Export
class TestItemReport(TestItem):
PARAMS = ParamSet(
Param("export", kind=LIST, required=True,
doc="List of exporters to run (junit, sqlite, …). Each entry is a "
"mapping describing the exporter type and its parameters."),
)
def __init__(self, dict_item, parent = None, status_queue=None, filename=""):
self._name = cst.TYPE_REPORT.item_name
super().__init__(dict_item, parent, status_queue, filename=filename)

View File

@@ -10,9 +10,42 @@ from interpreter.test_items.test_item import (TestItem, test_run)
from interpreter.test_items.test_result import (TestValue)
import api.testium as tm
from interpreter.utils.constants import TestItemType as cst
from interpreter.utils.param_decl import Param, ParamSet
from runtime.tum_except import ETUMSyntaxError, ETUMRuntimeError, item_load_context
def _testium_launch_cmd():
"""Command prefix to launch a fresh testium instance, runtime-aware.
AppImage / Flatpak / PyInstaller / wheel / source all need a different
entry point than just the path to __main__.py (which may be a .py inside
a read-only bundle, or unreachable from the sub-instance's cwd).
"""
# AppImage: the env var holds the path to the .AppImage file itself.
appimage = os.environ.get("APPIMAGE")
if appimage:
return [appimage]
# Flatpak: re-launch via the Flatpak app id, but on the host side —
# the `flatpak` CLI cannot run inside our sandbox (no D-Bus access to the
# host Flatpak service, and the host binary would need host libs that are
# ABI-incompatible with the sandbox runtime). flatpak-spawn proxies the
# call to the host via org.freedesktop.Flatpak (allowed by --talk-name in
# the manifest).
if os.path.isfile("/.flatpak-info"):
return ["flatpak-spawn", "--host",
"flatpak", "run", "org.testium.Testium"]
# PyInstaller frozen exe: sys.executable is the binary itself.
if getattr(sys, "frozen", False):
return [sys.executable]
# Source / wheel: re-use the same Python with the same entry point that
# launched this instance, made absolute so cwd changes in the sub-instance
# don't break the lookup. argv[0] is either:
# - the package directory (source: `python3 src/testium ...`)
# - the console_scripts wrapper (wheel: `/usr/bin/testium`)
# Both are runnable as `python <argv0>`.
return [sys.executable, os.path.abspath(sys.argv[0])]
def nowInBetween(start, end):
"""
Check wether current time is within boundaries
@@ -25,6 +58,25 @@ def nowInBetween(start, end):
class TestItemRun(TestItem):
PARAMS = ParamSet(
Param("tum", required=True,
doc="Path to the .tum file launched in a fresh testium instance."),
Param("param_file", default="",
doc="Optional path to a param.yaml passed to the sub-instance."),
Param("log_file", default="",
doc="Path where the sub-instance writes its log."),
Param("report_file", default="",
doc="Path where the sub-instance writes its report."),
Param("start_time",
doc="HH:MM time of day after which the sub-instance may run."),
Param("end_time",
doc="HH:MM time of day after which the sub-instance no longer runs."),
Param("wait_for_exec",
doc="If true, block until the time window opens. Requires both "
"start_time and end_time."),
)
def __init__(self, dict_item, parent = None, status_queue=None, filename=""):
self._name = cst.TYPE_RUN.item_name
super().__init__(dict_item, parent, status_queue, filename=filename)
@@ -33,8 +85,6 @@ class TestItemRun(TestItem):
with item_load_context(self.cmd(), self.name(), self.seqFilename()):
self.tum_file = self._prms.getParam('tum', required=True)
self.param_file = self._prms.getParam('param_file', default='')
self.python_bin = self._prms.getParam('python_bin', default='')
self.testium_path = self._prms.getParam('testium_path', default='')
self.log_path = self._prms.getParam('log_file', default='')
self.report_path = self._prms.getParam('report_file', default='')
self.start_time = self._prms.getParam('start_time')
@@ -52,18 +102,9 @@ class TestItemRun(TestItem):
'"{}" file could not be found'.format(file_path))
self.tum_file = file_path
pf = self._prms.expanse(self.param_file)
pp = self._prms.expanse(self.python_bin)
sp = self._prms.expanse(self.testium_path)
lp = self._prms.expanse(self.log_path)
rp = self._prms.expanse(self.report_path)
cmd = []
if sp == '':
sp = sys.argv[0]
if pp != '':
cmd.append(pp)
elif not os.path.isfile(sp) or not os.access(sp, os.X_OK):
cmd.append(sys.executable)
cmd.append(sp)
cmd = _testium_launch_cmd()
if tm.text_mode():
cmd.append("-b")
else:

View File

@@ -11,6 +11,7 @@ from interpreter.test_items.item_actions import TestItemActions
from interpreter.test_items.item_actions.action import TestItemAction
from interpreter.utils.constants import TestItemType as cst
from interpreter.utils.eval import evaluate
from interpreter.utils.param_decl import Param, ParamSet, LIST
class TestItemPlotAction(TestItemAction):
@@ -21,6 +22,12 @@ class TestItemPlotAction(TestItemAction):
class TestItemPlotActionOpen(TestItemPlotAction):
PARAMS = ParamSet(
Param("log_path", default=None,
doc="Optional file to which the plot data are appended."),
)
def __init__(
self, action_name, dict_item, parent=None, status_queue=None, filename=""
):
@@ -57,6 +64,15 @@ class TestItemPlotActionOpen(TestItemPlotAction):
class TestItemPlotActionClose(TestItemPlotAction):
PARAMS = ParamSet(
Param("wait_dialog_exit", default=False,
doc="If true, the close action blocks until the user closes the "
"plot window (or timeout)."),
Param("timeout", default=-1,
doc="Seconds to wait when wait_dialog_exit is true. Negative ⇒ infinite."),
)
def __init__(
self, action_name, dict_item, parent=None, status_queue=None, filename=""
):
@@ -96,6 +112,20 @@ class TestItemPlotActionClose(TestItemPlotAction):
class TestItemPlotActionPeriodic(TestItemPlotAction):
PARAMS = ParamSet(
Param("period", required=True,
doc="Seconds between two calls of the periodic function."),
Param("file", required=True,
doc="Path to the .py file holding the periodic function."),
Param("func_name", required=True,
doc="Name of the periodic function."),
Param("param", kind=LIST,
doc="Arguments passed to the periodic function on each call."),
Param("eval", default="",
doc="Post-evaluation applied to the function's return value."),
)
def __init__(
self, action_name, dict_item, parent=None, status_queue=None, filename=""
):
@@ -169,6 +199,13 @@ class TestItemPlotActionAdd(TestItemPlotAction):
class TestItemPlotActionLastValues(TestItemPlotAction):
PARAMS = ParamSet(
Param("name", kind=LIST,
doc="List of plot variable names whose last sample is returned. "
"Result is stored in $(plv_<plot_name>) as a dict."),
)
def __init__(self, action_name, dict_item, parent=None, status_queue=None, filename=""):
super().__init__(
action_name, cst.TYPE_GRAPH_ACTION, dict_item, parent, status_queue, filename=filename
@@ -219,6 +256,13 @@ class TestItemPlotActionExport(TestItemPlotAction):
class TestItemPlot(TestItemActions):
PARAMS = ParamSet(
Param("plot_name", required=True,
doc="Identifier of the plot window — referenced by every nested "
"action and by $(plv_<plot_name>) for last-values output."),
)
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
super().__init__(
cst.TYPE_GRAPH, dict_item, parent, status_queue, filename=filename

View File

@@ -7,6 +7,7 @@ import api.testium as tm
from interpreter.test_items.test_item import (TestItem, test_run)
from interpreter.test_items.test_result import (TestValue)
from interpreter.utils.constants import TestItemType as cst
from interpreter.utils.param_decl import Param, ParamSet
from runtime.tum_except import ETUMSyntaxError, ETUMRuntimeError, item_load_context
class TestItemSleep(TestItem):
@@ -14,6 +15,15 @@ class TestItemSleep(TestItem):
sleep timeout: 10
"""
PARAMS = ParamSet(
Param("timeout", required=True,
doc="Duration to sleep. Number of seconds, or a string "
"like '1d 2h 30m 15s'."),
Param("dialog", default=False,
doc="If true, show a cancel dialog (GUI mode) or an interactive "
"Ctrl+C-able countdown (text mode)."),
)
def __init__(self, dict_item, parent = None, status_queue=None, filename=""):
self._name = cst.TYPE_SLEEP.item_name
super().__init__(dict_item, parent, status_queue, filename=filename)
@@ -80,4 +90,7 @@ class TestItemSleep(TestItem):
end_time = _time.time() + float(timeout)
while _time.time() < end_time and not self._is_stopped:
sleep(min(0.05, end_time - _time.time()))
if self._is_stopped:
self.result.set(TestValue.FAILURE, 'Sleep aborted on stop request')
else:
self.result.set(TestValue.SUCCESS, 'Sleep %s sec' % (str(timeout)))

View File

@@ -2,11 +2,23 @@ from interpreter.test_items.test_item import test_run
from interpreter.test_items.test_result import TestValue
from interpreter.test_items.test_item_dialog_base import TestItemDialogBase, _is_text_mode, _is_interactive
from interpreter.utils.constants import TestItemType as cst
from interpreter.utils.param_decl import Param, ParamSet, LIST
from runtime.tum_except import item_load_context
import api.testium as tm
class TestItemTestedRefsDialog(TestItemDialogBase):
PARAMS = ParamSet(
Param("question", required=True,
doc="Prompt asking the operator to enter the tested references."),
Param("reference", kind=LIST,
doc="Pre-filled list of references shown in the dialog."),
Param("auto_result", default=None,
doc="Batch-mode outcome: None ⇒ FAILURE, truthy ⇒ SUCCESS with "
"the pre-filled references."),
)
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
self._name = cst.TYPE_REFERENCE_DLG.item_name
super().__init__(dict_item, parent, status_queue, filename=filename)

View File

@@ -11,6 +11,7 @@ from interpreter.test_items.test_item import (TestItem, test_run, LOG_TEST_STOP,
from interpreter.test_items.test_result import (TestResult, TestValue)
from interpreter.test_items.test_item import test_data
from interpreter.utils.constants import TestItemType as cst
from interpreter.utils.param_decl import Param, ParamSet, LIST
from runtime.stdout_redirect import stdio_redir
class UnittestResult(TextTestResult):
@@ -95,6 +96,15 @@ class TestItemUnittestElement(TestItem):
class TestItemUnittestFile(TestItem):
PARAMS = ParamSet(
Param("test_file", required=True,
doc="Path to the Python unittest file (TestCase subclass)."),
Param("test_method", kind=LIST,
doc="Optional list of method names to restrict the run to. "
"When empty, every test_* method in the file is run."),
)
def __init__(self, dict_item, parent = None, status_queue=None, filename=""):
self._name = cst.TYPE_UNITTEST.item_name
super().__init__(dict_item, parent, status_queue, filename=filename)

View File

@@ -2,6 +2,7 @@ from interpreter.test_items.test_item import test_run
from interpreter.test_items.test_result import TestValue
from interpreter.test_items.test_item_dialog_base import TestItemDialogBase, _is_text_mode, _is_interactive
from interpreter.utils.constants import TestItemType as cst
from interpreter.utils.param_decl import Param, ParamSet
from runtime.tum_except import item_load_context
import api.testium as tm
@@ -10,6 +11,19 @@ class TestItemValueDialog(TestItemDialogBase):
"""dialog_value item usage.
dialog_value name: Enter value, question: "Which value did you measure?"
"""
PARAMS = ParamSet(
Param("question", required=True,
doc="Prompt shown above the value input field."),
Param("default", default="",
doc="Pre-filled value of the input field."),
Param("auto_result", default=None,
doc="Batch-mode outcome: None ⇒ FAILURE, 'cancel' ⇒ cancelled, "
"any other truthy ⇒ SUCCESS with auto_value."),
Param("auto_value", default=None,
doc="Value used in batch mode when auto_result is set."),
)
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
self._name = cst.TYPE_VALUE_DLG.item_name
super().__init__(dict_item, parent, status_queue, filename=filename)

View File

@@ -17,8 +17,11 @@ Public API
``reset()`` : clear the cache (mostly useful for tests)
"""
import atexit
import os
import shutil
import subprocess
import tempfile
import api.testium as tm
from interpreter.utils.paths import sys_app_path_lin, sys_app_path_win
@@ -30,23 +33,235 @@ from runtime.tum_except import ETUMRuntimeError
_PYTHON_CANDIDATES = ["python3", "python"]
_LUA_CANDIDATES = ["lua", "lua5.5", "lua5.4", "lua5.3", "lua5.2", "lua5.1"]
# Inside an AppImage, AppRun prepends $APPDIR/usr/bin to PATH and exports a
# bundle-local PYTHONHOME / PYTHONPATH / LD_LIBRARY_PATH. We want py_func and
# lua_func to run under the *host* interpreter (not the bundled one), so we
# probe standard host bin dirs directly and scrub APPDIR-prefixed entries from
# the env passed to host subprocesses.
_APPIMAGE_HOST_DIRS = [
"/usr/local/bin",
"/usr/bin",
"/bin",
]
def _in_flatpak():
return os.path.isfile("/.flatpak-info")
def _in_appimage():
return "APPIMAGE" in os.environ
def apply_host_libs(env):
"""Strip bundle-local entries from *env* so a host binary can run cleanly.
Only meaningful for AppImage: removes $APPDIR-prefixed entries from
LD_LIBRARY_PATH / PYTHONPATH / PATH and drops PYTHONHOME, so the host
interpreter doesn't try to load the bundled (incompatible) Python
lib/site-packages. Flatpak is handled via flatpak-spawn --host instead
(see flatpak_host_spawn), so the sandbox env is irrelevant there.
"""
if not _in_appimage():
return
appdir = os.environ.get("APPDIR", "")
if appdir:
for var, sep in (("LD_LIBRARY_PATH", ":"),
("PYTHONPATH", os.pathsep),
("PATH", os.pathsep)):
cur = env.get(var, "")
if not cur:
continue
cleaned = sep.join(
p for p in cur.split(sep)
if p and not p.startswith(appdir)
)
if cleaned:
env[var] = cleaned
else:
env.pop(var, None)
env.pop("PYTHONHOME", None)
# ---------- Flatpak: spawn on host outside the sandbox -----------------------
#
# Inside a Flatpak the sandbox glibc is incompatible with host shared libraries,
# so we can't run host Python/Lua under the sandbox runtime — `LD_LIBRARY_PATH`
# tricks hit a `_dl_call_libc_early_init` assertion. The supported way out is
# `flatpak-spawn --host`, which talks to the session-bus Flatpak D-Bus service
# (org.freedesktop.Flatpak.Development) and asks it to spawn a process in the
# host execution environment instead of inside our sandbox. The manifest must
# grant `--talk-name=org.freedesktop.Flatpak` for the D-Bus call to be allowed.
#
# The host process can't see our /app/ contents (sandbox-only), so when we
# spawn host Python/Lua to run `py_func` / `lua_func`, the cwd must be a
# directory both sides can reach. /tmp is shared (--filesystem=/tmp), so we
# stage the testium package there once per process and reuse it for every
# spawn. In source mode (testium under $HOME) the host already sees the
# original path, so we skip the copy.
_staged_testium_path = None
def _get_host_testium_path():
"""Return a path to the testium package that the host can read.
- Source / wheel / PyInstaller install under $HOME → return testium_path()
as-is (host sees the same path via --filesystem=home).
- Flatpak bundle (testium under /app/) → stage a copy under /tmp on first
call and reuse it for the rest of the process.
"""
global _staged_testium_path
if _staged_testium_path is not None:
return _staged_testium_path
# Imported lazily to avoid a circular import (paths.py -> api.testium).
from interpreter.utils.paths import testium_path
tp = testium_path()
if not tp.startswith("/app/"):
_staged_testium_path = tp
return tp
staged = tempfile.mkdtemp(prefix="testium_host_", dir="/tmp")
# copytree refuses to write into an existing dir unless dirs_exist_ok=True.
# mkdtemp creates the dir, so we copy *into* it.
for entry in os.listdir(tp):
src = os.path.join(tp, entry)
dst = os.path.join(staged, entry)
if os.path.isdir(src):
shutil.copytree(src, dst, symlinks=True)
else:
shutil.copy2(src, dst, follow_symlinks=False)
_staged_testium_path = staged
atexit.register(shutil.rmtree, staged, ignore_errors=True)
return staged
_FORWARDED_ENV_KEYS = (
"HOME", "USER", "LOGNAME", "TMPDIR",
"XDG_RUNTIME_DIR", "XDG_DATA_HOME", "XDG_CONFIG_HOME", "XDG_CACHE_HOME",
"DBUS_SESSION_BUS_ADDRESS", "DISPLAY", "WAYLAND_DISPLAY",
"LANG", "LC_ALL",
)
def flatpak_host_spawn(interp_bin, cmd_args, host_cwd, extra_env=None):
"""Build a flatpak-spawn --host command vector.
Args:
interp_bin: absolute path to the host interpreter (e.g. /usr/bin/python3).
cmd_args: list of arguments passed to the interpreter.
host_cwd: working directory on the host (must be reachable from host).
extra_env: optional {name: value} of env vars to set on the host side
in addition to the default forwarded set. Values of ""
unset the variable on the host.
Returns a list suitable for subprocess.Popen.
"""
spawn = ["flatpak-spawn", "--host", f"--directory={host_cwd}"]
forwarded = {}
for key in _FORWARDED_ENV_KEYS:
val = os.environ.get(key)
if val:
forwarded[key] = val
if extra_env:
forwarded.update(extra_env)
for k, v in forwarded.items():
if v == "":
spawn.append(f"--unset-env={k}")
else:
spawn.append(f"--env={k}={v}")
spawn.append(interp_bin)
spawn.extend(cmd_args)
return spawn
def _which_host_flatpak(name):
"""Resolve a binary name (or absolute path) on the host via flatpak-spawn.
We can't probe /run/host/... because (a) only host-os is mounted there,
not arbitrary paths like /scratch, and (b) returning a /run/host path
would be useless — the host-side spawn sees a different filesystem and
needs the host-native path anyway.
"""
if os.path.isabs(name):
cmd = flatpak_host_spawn("/bin/sh", ["-c", f'test -x "{name}"'],
host_cwd="/tmp")
try:
r = subprocess.run(cmd, capture_output=True, timeout=10)
except (FileNotFoundError, PermissionError, subprocess.TimeoutExpired):
return ""
return name if r.returncode == 0 else ""
cmd = flatpak_host_spawn("/bin/sh", ["-c", f'command -v "{name}"'],
host_cwd="/tmp")
try:
r = subprocess.run(cmd, capture_output=True, text=True, timeout=10)
except (FileNotFoundError, PermissionError, subprocess.TimeoutExpired):
return ""
if r.returncode != 0:
return ""
return r.stdout.strip()
def _which(name):
func = sys_app_path_win if tm.OS() == "Windows" else sys_app_path_lin
return func(name)
if tm.OS() == "Windows":
return sys_app_path_win(name)
if _in_flatpak():
return _which_host_flatpak(name)
if _in_appimage():
for d in _APPIMAGE_HOST_DIRS:
p = os.path.join(d, name)
if os.path.isfile(p) and os.access(p, os.X_OK):
return p
return ""
return sys_app_path_lin(name)
def _python_version(path):
cmd = [path, "-c", "import sys; print(sys.version_info[:3])"]
def _probe_env():
"""Subprocess env for probing host binaries.
In AppImage we still need to scrub APPDIR-prefixed entries; in Flatpak we
delegate execution to the host via flatpak-spawn so the sandbox env doesn't
matter, but apply_host_libs is a no-op cost.
"""
env = os.environ.copy()
apply_host_libs(env)
return env
def _run_probe(cmd):
"""Run a probe command, dispatching through flatpak-spawn --host in Flatpak.
Returns (stdout, stderr) as str, or None on failure.
"""
if _in_flatpak():
spawn = flatpak_host_spawn(cmd[0], cmd[1:], host_cwd="/tmp")
try:
r = subprocess.run(
cmd, capture_output=True, text=True,
encoding=tm.sys_encoding(), timeout=10,
spawn, capture_output=True, text=True, timeout=10,
)
except (FileNotFoundError, PermissionError, subprocess.TimeoutExpired):
return None
if r.returncode != 0:
return None
return r.stdout, r.stderr
try:
return eval(r.stdout)
r = subprocess.run(
cmd, capture_output=True, text=True,
encoding=tm.sys_encoding(), timeout=10, env=_probe_env(),
)
except (FileNotFoundError, PermissionError, subprocess.TimeoutExpired):
return None
return r.stdout, r.stderr
def _python_version(path):
out = _run_probe([path, "-c", "import sys; print(sys.version_info[:3])"])
if out is None:
return None
try:
return eval(out[0])
except Exception:
return None
@@ -57,14 +272,11 @@ def _is_python3(path):
def _lua_version(path):
try:
r = subprocess.run(
[path, "-v"], capture_output=True, text=True, timeout=10,
)
except (FileNotFoundError, PermissionError, subprocess.TimeoutExpired):
out = _run_probe([path, "-v"])
if out is None:
return None
# On Windows the version banner goes to stderr.
line = r.stdout or r.stderr
line = out[0] or out[1]
try:
major, minor, _patch = line.split(" ")[1].split(".")
return (int(major), int(minor))
@@ -85,20 +297,39 @@ _SPECS = {
"lua": ("Lua 5.1+", "lua_bin", _LUA_CANDIDATES, _is_lua51),
}
# Cached per (name, override) so that runtime changes to gd[gd_key] —
# e.g. ``python_bin`` set from a YAML config file loaded *after*
# eval_proc has already resolved its own interpreter — are picked up by
# the next lookup instead of returning the stale, auto-discovered path.
# Long-lived subprocesses (eval_proc) keep whatever they captured at
# construction time, but every new PyProcessBase / FuncExecEngine spawned
# afterwards sees the current override.
_resolved = {}
def _resolve(name):
if name in _resolved:
return _resolved[name]
display, gd_key, candidates, validator = _SPECS[name]
override = tm.gd(gd_key, "") or ""
cached = _resolved.get(name)
if cached is not None and cached[0] == override:
return cached[1]
path = ""
if override:
if shutil.which(override) and validator(override):
path = override
# Absolute path: accept as-is (user knows exactly what they want).
# Bare name: resolve via _which() so the override stays host-only in
# Flatpak/AppImage instead of silently picking the bundled interpreter.
# In Flatpak we always defer to _which() so even absolute paths are
# checked from the host's perspective (the sandbox can't see e.g.
# /scratch/... paths that the user may have configured).
if os.path.isabs(override) and not _in_flatpak():
resolved = override if (os.path.isfile(override)
and os.access(override, os.X_OK)) else ""
else:
resolved = _which(override)
if resolved and validator(resolved):
path = resolved
else:
tm.print_warn(
f"Configured {display} interpreter '{override}' is not usable; "
@@ -114,7 +345,7 @@ def _resolve(name):
path = p
break
_resolved[name] = path
_resolved[name] = (override, path)
return path

View File

@@ -47,6 +47,13 @@ class LuaProcessBase:
if self._process is not None:
raise ETUMRuntimeError("The function subprocess has already been started.")
# In Flatpak the host can't see /app/lib/testium/lua_func, so use a
# staged copy under /tmp (shared between sandbox and host).
if bins._in_flatpak():
func_proc_path = os.path.join(
bins._get_host_testium_path(), "lua_func"
)
else:
func_proc_path = os.path.realpath(
os.path.join(subproc_path(), "lua_func")
)
@@ -60,6 +67,7 @@ class LuaProcessBase:
lua_env = tm.gd("lua_env", {})
env = os.environ.copy()
bins.apply_host_libs(env)
if not isinstance(lua_env, dict):
raise ETUMRuntimeError(f"The 'lua_env' global value should be a dictionary. But it is '{lua_env}'.")
@@ -77,8 +85,7 @@ class LuaProcessBase:
sock.close()
# POpen params
params = [
self._lbin,
cmd_args = [
"main.lua",
"--timeout",
f"{self._timeout}",
@@ -89,14 +96,31 @@ class LuaProcessBase:
]
if tm.debug_enabled() and tm.gd("debug_rpc", False):
params.append("--verbose")
cmd_args.append("--verbose")
if bins._in_flatpak():
# Run on the host outside the sandbox: avoids glibc ABI mismatches
# between the Flatpak runtime and host shared libraries.
host_env = {
k: env[k] for k in ("LUA_PATH", "LUA_CPATH", "PATH")
if k in env and env[k]
}
params = bins.flatpak_host_spawn(
self._lbin, cmd_args, host_cwd=func_proc_path,
extra_env=host_env,
)
popen_kwargs = {}
else:
params = [self._lbin, *cmd_args]
popen_kwargs = {"env": env, "cwd": func_proc_path}
self._process = subprocess.Popen(
params, env=env, cwd=func_proc_path,
params,
stdin=subprocess.DEVNULL,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
restore_signals=False,
**popen_kwargs,
)
# Route subprocess stdout/stderr (lua require failures, syntax
# errors, anything written to fd 1/2 before the in-script
@@ -144,4 +168,12 @@ class LuaProcessBase:
"""
if self._rpc is not None:
self._rpc.stop()
# Force-kill the worker if it's still running. Needed when user code
# in the worker is stuck and won't notice the parent closing the RPC
# socket on its own.
if self._process is not None and self._process.poll() is None:
try:
self._process.terminate()
except Exception:
pass

View File

@@ -0,0 +1,175 @@
"""Declarative description of a test item's accepted parameters.
Each ``TestItem`` subclass declares its parameter surface as a class
attribute::
class TestItemFoo(TestItem):
PARAMS = ParamSet(
Param("bar", required=True, doc="The bar value."),
Param("baz", default=0, doc="Optional baz."),
Param("modes", kind=LIST, doc="Iterable of modes."),
Param("strategy", kind=ENUM("a", "b"), doc="..."),
Param("opts", kind=BLOCK, doc="Sub-block."),
)
The base ``TestItem.__init__`` consumes both ``COMMON_PARAMS`` (defined
in ``test_item.py``) and the subclass ``PARAMS`` to:
* warn on any key in the user's YAML that isn't declared anywhere
(catches typos like ``param_filee``);
* expose a machine-readable schema for documentation generation and,
eventually, an LSP server.
The descriptor is **purely about shape and naming**. Type coercion and
runtime checking of expanded values remain the responsibility of each
item's ``execute()`` method — most parameters are expressions
(``$(...)`` / ``<| ... |>``) whose effective type is only known after
expansion, so a static type would be misleading.
Validation of *values* (e.g. ``start_time`` must match HH:MM) can be
attached per-param via ``validate=lambda v: ...`` and is applied at
execution time on the expanded value, not at load time.
"""
from dataclasses import dataclass, field
from typing import Any, Callable, Optional, Union
# ---------- Parameter "kinds" -------------------------------------------------
#
# These describe the YAML *shape* expected for a parameter, not its
# semantic type. They drive the LSP completion (do we suggest a single
# value, a list, a sub-block, an enum picker?) and the unknown-param
# diagnostic; nothing more.
SCALAR = "scalar" # single value (string, number, bool, expression, ...)
LIST = "list" # YAML list — the historical ``getParamAll`` case
BLOCK = "block" # nested dict — e.g. ``cycle.exit:``
@dataclass(frozen=True)
class Enum:
"""Closed enumeration of acceptable scalar values."""
values: tuple
def __init__(self, *values):
# frozen=True forbids assignment; bypass via object.__setattr__.
object.__setattr__(self, "values", tuple(values))
def __repr__(self):
return f"Enum({', '.join(repr(v) for v in self.values)})"
Kind = Union[str, Enum]
# ---------- The descriptor ----------------------------------------------------
_MISSING = object()
@dataclass(frozen=True)
class Param:
"""Declarative description of one accepted parameter.
Attributes
----------
name : str
The YAML key.
kind : ``SCALAR`` (default) | ``LIST`` | ``BLOCK`` | ``Enum(...)``
The YAML shape expected.
required : bool
If True, missing the parameter is a load-time error.
default : Any
Default value when the parameter is absent. ``_MISSING`` when no
default was set (used to distinguish "absent" from "None").
doc : str
Free-form description used for hover / generated documentation.
validate : Optional[Callable[[Any], bool]]
Optional post-expansion validator, evaluated at ``execute()``
time on the effective (expanded) value. Returning ``False``
raises a clear error pointing at the param.
"""
name: str
kind: Kind = SCALAR
required: bool = False
default: Any = _MISSING
doc: str = ""
validate: Optional[Callable[[Any], bool]] = None
def has_default(self):
return self.default is not _MISSING
def to_schema(self):
"""Return a dict suitable for JSON Schema generation."""
s = {"name": self.name, "required": self.required, "doc": self.doc}
if isinstance(self.kind, Enum):
s["kind"] = "enum"
s["enum"] = list(self.kind.values)
else:
s["kind"] = self.kind
if self.has_default():
s["default"] = self.default
return s
class ParamSet:
"""Ordered, name-indexed collection of ``Param`` descriptors.
Supports concatenation (``COMMON_PARAMS + SUBCLASS_PARAMS``) to
merge the common surface with each item's own params. Later
declarations override earlier ones (so a subclass can tighten a
common param's docstring without redeclaring everything).
"""
def __init__(self, *params):
self._params = {}
for p in params:
self.add(p)
def add(self, param):
if not isinstance(param, Param):
raise TypeError(f"ParamSet only accepts Param instances, got {type(param).__name__}")
self._params[param.name] = param
def __iter__(self):
return iter(self._params.values())
def __contains__(self, name):
return name in self._params
def __getitem__(self, name):
return self._params[name]
def names(self):
return tuple(self._params.keys())
def __add__(self, other):
if not isinstance(other, ParamSet):
return NotImplemented
merged = ParamSet()
merged._params = {**self._params, **other._params}
return merged
def to_schema(self):
return [p.to_schema() for p in self._params.values()]
# ---------- Validation primitives --------------------------------------------
def unknown_keys(declared, user_dict):
"""Return the user-provided keys that are not declared in *declared*.
*declared* is a ``ParamSet``; *user_dict* is the raw YAML mapping
for the item. Unknown keys catch typos and obsolete parameters.
"""
if not isinstance(user_dict, dict):
return ()
return tuple(k for k in user_dict.keys() if k not in declared)
def missing_required(declared, user_dict):
"""Return the names of declared required params absent from *user_dict*."""
if not isinstance(user_dict, dict):
return tuple(p.name for p in declared if p.required)
return tuple(p.name for p in declared if p.required and p.name not in user_dict)

View File

@@ -42,6 +42,10 @@ class PyProcessBase:
raise ETUMRuntimeError(f"The 'py_env' global value should be a dictionary. But it is '{py_env}'.")
env = os.environ.copy()
bins.apply_host_libs(env)
# PYTHONUSERBASE is set by the Flatpak runtime to isolate sandbox
# user packages; remove it so the host Python finds ~/.local packages.
env.pop("PYTHONUSERBASE", None)
for k, v in self.CUST_ENV.items():
e = py_env.get(k, "")
if e != "":
@@ -57,14 +61,18 @@ class PyProcessBase:
if sock is not None:
sock.close()
# Add the path of the subprocess (root sources of testium)
# In Flatpak the host can't see /app/lib/testium, so use a staged copy
# under /tmp (shared between sandbox and host) for both cwd and as the
# root in PYTHONPATH. Outside Flatpak the original paths are used.
if bins._in_flatpak():
tstium_path = bins._get_host_testium_path()
func_proc_path = tstium_path
else:
tstium_path = os.path.realpath(testium_path())
func_proc_path = os.path.realpath(subproc_path())
env["PYTHONPATH"] = tstium_path + os.pathsep + self._ppath + os.pathsep + env.get("PYTHONPATH", "")
params = [
self._pbin,
# "-m",
cmd_args = [
"py_func",
"-p",
f"{self._port}",
@@ -73,14 +81,31 @@ class PyProcessBase:
]
if tm.debug_enabled() and tm.gd("debug_rpc", False):
params.append("-v")
cmd_args.append("-v")
if bins._in_flatpak():
# Run on the host outside the sandbox: avoids glibc ABI mismatches
# between the Flatpak runtime and host shared libraries.
host_env = {
k: env[k] for k in ("PYTHONPATH", "PATH")
if k in env and env[k]
}
params = bins.flatpak_host_spawn(
self._pbin, cmd_args, host_cwd=func_proc_path,
extra_env=host_env,
)
popen_kwargs = {}
else:
params = [self._pbin, *cmd_args]
popen_kwargs = {"env": env, "cwd": func_proc_path}
self._process = subprocess.Popen(
params, env=env, cwd=func_proc_path,
params,
stdin=subprocess.DEVNULL,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
restore_signals=False,
**popen_kwargs,
)
# Route subprocess stdout/stderr (early-startup errors,
# unhandled exceptions, anything written to fd 1/2 before the
@@ -119,3 +144,11 @@ class PyProcessBase:
def stop(self):
if self._rpc is not None:
self._rpc.stop()
# Force-kill the worker if it's still running. Needed when user code
# in the worker is stuck (e.g. sleep, blocking I/O) and won't notice
# the parent closing the RPC socket on its own.
if self._process is not None and self._process.poll() is None:
try:
self._process.terminate()
except Exception:
pass

View File

@@ -165,11 +165,14 @@ def env_init():
_constants_init()
def update_global(config_files, defines, gui_defaults, silent=False):
"""Global dict updated with the content of the config file and a dict provided.
this function returns the resulting dict.
def apply_overrides(defines, gui_defaults):
"""Push GUI defaults then CLI defines into the global dict.
Extracted from update_global so it can be called *before* eval_proc
starts: interpreter overrides (python_bin, lua_bin) must be visible
to bins.python_bin() on its first lookup, which happens during
eval_process_init.
"""
# GUI preferences applied first
for k, v in gui_defaults.items():
try:
val = ast.literal_eval(v)
@@ -177,7 +180,6 @@ def update_global(config_files, defines, gui_defaults, silent=False):
val = v
tm.setgd(k, val)
# Then command line defines
for k, v in defines.items():
try:
val = ast.literal_eval(v)
@@ -185,6 +187,14 @@ def update_global(config_files, defines, gui_defaults, silent=False):
val = v
tm.setgd(k, val)
def update_global(config_files, defines, gui_defaults, silent=False):
"""Global dict updated with the content of the config file and a dict provided.
this function returns the resulting dict.
"""
# GUI preferences applied first, then command line defines
apply_overrides(defines, gui_defaults)
# Then the configuration files
# load global dic before test item
_feed_gd_with_params(config_files, silent)

View File

@@ -31,10 +31,15 @@ def get_version(path :str)-> str:
return "Warning git not supported in your settings, version of {} unknown".format(path)
def get_testium_version():
# Flatpak bundle
if os.path.isfile('/.flatpak-info'):
ver = os.environ.get('TESTIUM_VERSION', '').strip()
return (ver if ver else 'unknown') + " (flatpak release)"
# AppImage
if 'APPIMAGE' in os.environ:
ver = os.getenv('SEQUENCER_REV', 'unknown')
return ver + " (binary release)"
ver = os.environ.get('TESTIUM_VERSION', '').strip()
return (ver if ver else 'unknown') + " (binary release)"
# PyInstaller frozen exe
if getattr(sys, 'frozen', False):

View File

@@ -41,8 +41,7 @@ end
--- INTERNAL: Handle requests from the client
function JSONRPC:_handle_request(req)
local method = self.methods[req.method]
local ok, ret
local res, err
local ok, ret, err
if not method then
if req.id then self:_send_error(req.id, string.format("Method '%s' not registered in lua server")) end
return
@@ -52,15 +51,18 @@ function JSONRPC:_handle_request(req)
-- Only send response if it's not a Notification (notifications have no ID)
if req.id then
if ok then
res = ret
if res == nil then
if not ok then
-- pcall trapped a runtime error in the method itself.
self:_send_error(req.id, tostring(ret))
elseif err ~= nil then
-- Method ran but signaled a logical error via its 2nd return.
self:_send_error(req.id, tostring(err))
else
self:_send({ jsonrpc = "2.0", result = { returned_value = res }, id = req.id })
end
else
self:_send_error(req.id, tostring(err))
-- Success. A user function returning nothing yields ret==nil;
-- encode it as JSON null so "returned_value" stays present.
local val = ret
if val == nil then val = json.null end
self:_send({ jsonrpc = "2.0", result = { returned_value = val }, id = req.id })
end
end
end

View File

@@ -0,0 +1,21 @@
"""Helpers for Qt file/directory dialogs.
In Flatpak the native QFileDialog goes through the XDG document portal,
which returns ``/run/user/UID/doc/.../<file>`` and only exposes the
selected file — sibling files (param.yaml, scripts, recent paths in
preferences, ...) are unreachable. Forcing Qt's own non-native dialog
makes it walk the real filesystem mounted via ``--filesystem=home``
and return a regular path.
"""
import os
from PySide6.QtWidgets import QFileDialog
def options():
"""Default ``QFileDialog`` options for the current runtime."""
opts = QFileDialog.Options()
if os.path.isfile("/.flatpak-info"):
opts |= QFileDialog.Option.DontUseNativeDialog
return opts

View File

@@ -3,6 +3,7 @@ from PySide6.QtWidgets import QDialog, QFileDialog
from PySide6.QtGui import QFont
from main_win.preference_win.preference_core_win import Ui_preferenceWindow
from main_win import file_dialog
import interpreter.utils.settings as prefs
@@ -193,6 +194,7 @@ class PrefWindow(QDialog):
self,
caption="Select the default report directory",
dir=self.ui.editDefaultReportPath.text(),
options=file_dialog.options(),
)
if path:
self.ui.editDefaultReportPath.setText(path)
@@ -203,6 +205,7 @@ class PrefWindow(QDialog):
self,
caption="Select the default log directory",
dir=self.ui.editDefaultLogPath.text(),
options=file_dialog.options(),
)
if path:
self.ui.editDefaultLogPath.setText(path)
@@ -213,6 +216,7 @@ class PrefWindow(QDialog):
self,
caption="Select the python interpreter",
dir=self.ui.editPythonPath.text(),
options=file_dialog.options(),
)
if path:
self.ui.editPythonPath.setText(path)
@@ -220,7 +224,10 @@ class PrefWindow(QDialog):
@Slot()
def on_butLuaPath_pressed(self):
path, _ = QFileDialog.getOpenFileName(
self, caption="Select the lua interpreter", dir=self.ui.editLuaPath.text()
self,
caption="Select the lua interpreter",
dir=self.ui.editLuaPath.text(),
options=file_dialog.options(),
)
if path:
self.ui.editLuaPath.setText(path)

View File

@@ -9,6 +9,7 @@ from PySide6.QtWidgets import QApplication, QFileDialog, QProgressDialog
from interpreter.process import TestProcess
from interpreter.utils.test_ctrl import TestSetController
from main_win.test_controller_service import TestControllerService
from main_win import file_dialog
import interpreter.utils.settings as prefs
from runtime.tum_except import ETUMFileError, ETUMRuntimeError
@@ -213,7 +214,8 @@ class TestFileManager:
if w.testFile is not None:
d = os.path.dirname(w.testFile)
file_name, _ = QFileDialog.getOpenFileName(
w, "Open the test file", d, "testium file (*.tum);;All Files (*)"
w, "Open the test file", d,
"testium file (*.tum);;All Files (*)", options=file_dialog.options()
)
if file_name:
self.reload(file_name)

View File

@@ -176,7 +176,7 @@ class TestRunner:
w.actionOpenTest.setDisabled(True)
w.actionExit.setDisabled(True)
icon = QtGui.QIcon()
icon.addPixmap(QtGui.QPixmap(icon_prefix() + "/pause.png"), QtGui.QIcon.Normal, QtGui.QIcon.Off)
icon.addPixmap(QtGui.QPixmap(icon_prefix() + "/pause2.png"), QtGui.QIcon.Normal, QtGui.QIcon.On)
w.actionStart_test.setIcon(icon)
w.actionStart_test.setText("Pause test")
w.actionPreferences.setDisabled(True)

View File

@@ -37,6 +37,7 @@ from interpreter.utils.icons import icon_prefix
from main_win.test_run.outlog import OutLog
from main_win.test_run.test_run import ThreadTestStatus
from main_win import file_dialog
import interpreter.utils.settings as prefs
from runtime.stdout_redirect import stdio_redir
import api.testium as tm
@@ -484,7 +485,8 @@ class MainWindow(QMainWindow, Ui_MainWindow):
else:
initialPath = None
fileName, _ = QFileDialog.getSaveFileName(
self, "Path to Log file", initialPath, "Log Files (*.log);;All Files (*)"
self, "Path to Log file", initialPath, "Log Files (*.log);;All Files (*)",
options=file_dialog.options(),
)
if fileName:
shutil.copy(self.logFileName, fileName)
@@ -525,7 +527,8 @@ class MainWindow(QMainWindow, Ui_MainWindow):
else:
initialPath = None
fileName, _ = QFileDialog.getSaveFileName(
self, "Path to log file", initialPath, "Log Files (*.log);;All Files (*)"
self, "Path to log file", initialPath, "Log Files (*.log);;All Files (*)",
options=file_dialog.options(),
)
if fileName:
self.editLogFilePath.setText(fileName)

View File

@@ -8,14 +8,18 @@ def exception_handler(typ_exc, value, trbk):
print(f"Critical failure : '{value}'.")
tb = traceback.format_exception(typ_exc, value, trbk)
print("".join(tb))
print(f" python : {sys.executable}")
print(f" sys.path : {sys.path}")
sys.excepthook = exception_handler
p = Path(__file__)
p = p.parent / ".."
p = p.resolve()
sys.path.append(p)
# Make the parent directory of py_func/ (= the testium package dir, which also
# contains runtime/, lua_func/, …) the first entry on sys.path so `from py_func
# import main` and `from runtime…` resolve regardless of cwd or how this script
# was invoked. str() because some importers don't play well with PathLike entries.
_pkg_parent = str((Path(__file__).resolve().parent / "..").resolve())
if _pkg_parent not in sys.path:
sys.path.insert(0, _pkg_parent)
from py_func import main

View File

@@ -200,6 +200,7 @@ class JsonRpcConnection:
Raises:
TimeoutError: If no response is received within `timeout`.
ConnectionAbortedError: If stop() was called while waiting.
"""
req_id = next(self.id_gen)
@@ -214,7 +215,12 @@ class JsonRpcConnection:
self.pending.pop(req_id, None)
raise TimeoutError("Timeout JSON-RPC")
return self.pending.pop(req_id)["response"]
entry = self.pending.pop(req_id)
if entry["response"] is None:
# Woken by stop() (or by a malformed dispatch) rather than by a
# real response — abort the call so callers don't block further.
raise ConnectionAbortedError("JSON-RPC client stopped")
return entry["response"]
def print_info(self, msg):
if self.dbg_out is not None:
@@ -223,6 +229,10 @@ class JsonRpcConnection:
def stop(self):
if self.running:
self.running = False
# Wake any in-flight call() so it doesn't sit on its (default 1h)
# timeout. The response stays None and call() raises ConnectionAbortedError.
for entry in list(self.pending.values()):
entry["event"].set()
def join(self):
self.recv_thread.join()

View File

@@ -1,10 +1,76 @@
# Validation
This directory contains the necessary material to run the testium validation.
This directory contains the testium validation suite. A single set of
items (`items/`), fixtures and post-processing (`post_execution.py`) is
re-used across every packaging channel.
Here is the documentation on how to configure the validation, run it and check that the
results are correct.
## Running the suite
# Tests
```sh
./test/validation/run.sh # default mode = source
./test/validation/run.sh --mode wheel
./test/validation/run.sh --mode pyinstaller
./test/validation/run.sh --mode flatpak
./test/validation/run.sh --mode appimage
```
TBD
On Windows (only `source`, `wheel`, `pyinstaller` are supported):
```bat
test\validation\run.bat --mode pyinstaller
```
Pass `clean` as the **first** argument to recreate the validation venv
from scratch (useful after a system Python upgrade):
```sh
./test/validation/run.sh clean --mode flatpak
```
Any extra arguments after the mode flag are forwarded to testium.
## Modes
| Mode | What it launches | Prerequisite |
|---------------|-------------------------------------------------------------|------------------------------------------------------------------|
| `source` | `python3 src/testium` via the project's `run.sh` | none — works straight out of the repo |
| `wheel` | `python -m testium` inside a dedicated wheel venv | `./build_all.sh` produced `dist/testium-<v>-py3-none-any.whl` |
| `pyinstaller` | `dist/testium-<v>` (frozen binary) | `./build_all.sh` produced the PyInstaller binary |
| `flatpak` | `flatpak run --command=testium org.testium.Testium` | the Flatpak bundle is installed (`flatpak install --user dist/testium-<v>.flatpak`) |
| `appimage` | `dist/Testium-<v>-x86_64.AppImage` | `./build_all.sh` produced the AppImage |
Each mode writes its results to a distinct report file
(`validation-<mode>.sqlite` / `validation-<mode>-<item>.xml`), so you
can run several modes in a row without clobbering previous reports.
## How `python_bin` is pinned
Every test-execution subprocess (inline `<| ... |>` evaluation,
`py_func`, `cycle`, `post_execution`, …) is routed through a dedicated
venv at `${TMPDIR:-/tmp}/testium-validation-venv`. The venv is created
with `--system-site-packages` so existing system packages stay visible,
then `junit-xml` is pip-installed for `post_execution.py`.
This is a **host** venv. In every mode (including Flatpak) the
test-execution subprocesses end up running on the host — directly for
source/wheel/pyinstaller/appimage, and via `flatpak-spawn --host` for
Flatpak — so the same venv works across modes. The wheel mode
additionally creates a separate `testium-wheel-venv-<v>` to hold the
installed wheel; that one is only used to launch testium itself.
## What is checked
The `venv` item under `items/venv/` asserts that the validation venv is
actually being used:
* `python_bin` is set in the global dict.
* The eval subprocess (used for `<| ... |>` expressions) has
`sys.executable == python_bin`, `sys.prefix == dirname(dirname(python_bin))`,
and `sys.prefix != sys.base_prefix` (i.e. is actually inside a venv).
* A `py_func` subprocess passes the same three checks.
These checks use `abspath`/`normpath` rather than `realpath` on
purpose: the venv's `bin/python3` is a symlink to the host interpreter,
so `realpath` would map both venv and non-venv interpreters to the same
target. `sys.prefix != sys.base_prefix` is the venv-specific marker
that distinguishes the two cases.

View File

@@ -49,4 +49,12 @@ function module.test_delgd()
return 0
end
function module.return_nothing()
-- Returns no value: ret is nil but no error.
end
function module.return_explicit_nil()
return nil
end
return module

View File

@@ -186,6 +186,18 @@
file: $(test_path)$(psep)lua_func.lua
func_name: test_delgd
- lua_func:
name: function returning nothing should succeed
key: $(test)_PASS
file: $(test_path)$(psep)lua_func.lua
func_name: return_nothing
- lua_func:
name: function returning explicit nil should succeed
key: $(test)_PASS
file: $(test_path)$(psep)lua_func.lua
func_name: return_explicit_nil
- group:
name: context_id tests
steps:

View File

@@ -54,3 +54,10 @@ def test_delgd():
tm.delgd("_py_delgd_test")
assert tm.gd("_py_delgd_test", None) is None
return 0
def return_nothing():
# Falls off the end: implicit None return, no error.
pass
def return_explicit_none():
return None

View File

@@ -196,6 +196,18 @@
file: $(test_path)$(psep)py_func.py
func_name: test_delgd
- py_func:
name: function returning nothing should succeed
key: $(test)_PASS
file: $(test_path)$(psep)py_func.py
func_name: return_nothing
- py_func:
name: function returning explicit None should succeed
key: $(test)_PASS
file: $(test_path)$(psep)py_func.py
func_name: return_explicit_none
- group:
name: context_id tests
steps:

View File

@@ -0,0 +1 @@
no_param: Null

View File

@@ -0,0 +1,53 @@
# venv test: assert that the dedicated validation venv is the python
# being used for every test-execution subprocess (eval_proc / py_func /
# cycle / ...). The venv path is pinned by ``-d python_bin=...`` in
# test/validation/run.sh (or run.bat).
#
# We use ``abspath``/``normpath`` rather than ``realpath`` on purpose:
# the venv's ``bin/python3`` is a symlink to the host python, so
# realpath would map every venv interpreter to the same system path and
# the comparison would silently pass even *without* a venv.
# ``sys.prefix != sys.base_prefix`` is the venv-specific marker that
# catches that case.
- check:
name: python_bin is set in the global dict
key: $(test)_PASS
values:
- <| bool(r"$(python_bin)") |>
- check:
name: eval_proc subprocess runs in the validation venv
key: $(test)_PASS
values:
- <| os.path.normpath(os.path.abspath(sys.executable)) == os.path.normpath(os.path.abspath(r"$(python_bin)")) |>
- check:
name: eval_proc sys.prefix matches python_bin venv root
key: $(test)_PASS
values:
- <| os.path.normpath(os.path.abspath(sys.prefix)) == os.path.dirname(os.path.dirname(os.path.normpath(os.path.abspath(r"$(python_bin)")))) |>
- check:
name: eval_proc is actually inside a venv (sys.prefix != sys.base_prefix)
key: $(test)_PASS
values:
- <| os.path.normpath(os.path.abspath(sys.prefix)) != os.path.normpath(os.path.abspath(sys.base_prefix)) |>
- py_func:
name: py_func subprocess runs in the validation venv
key: $(test)_PASS
file: $(test_path)$(psep)verify_venv.py
func_name: check_sys_executable
- py_func:
name: py_func sys.prefix matches python_bin venv root
key: $(test)_PASS
file: $(test_path)$(psep)verify_venv.py
func_name: check_sys_prefix_in_venv
- py_func:
name: py_func is actually inside a venv
key: $(test)_PASS
file: $(test_path)$(psep)verify_venv.py
func_name: check_is_venv

View File

@@ -0,0 +1,62 @@
import os
import sys
import py_func.tm as tm
def _norm(p):
# normpath + normcase, without resolving symlinks. realpath() would
# follow the venv's ``python3`` symlink to ``/usr/bin/python3.X`` and
# defeat the comparison.
return os.path.normcase(os.path.normpath(os.path.abspath(p)))
def _venv_dir():
# python_bin is at ``<venv>/(bin|Scripts)/python*`` so the venv root
# is two levels above the executable.
exe = tm.gd("python_bin", "")
if not exe:
return ""
return os.path.dirname(os.path.dirname(_norm(exe)))
def check_sys_executable():
"""py_func subprocess: sys.executable must match the configured python_bin."""
expected = _norm(tm.gd("python_bin", ""))
actual = _norm(sys.executable)
if expected and actual == expected:
return True
return (
-1,
f"sys.executable={actual!r} differs from python_bin={expected!r}",
)
def check_sys_prefix_in_venv():
"""py_func subprocess: sys.prefix must match the venv root derived
from python_bin (two levels up from the executable)."""
venv = _venv_dir()
if not venv:
return (-1, "python_bin is not set in the global dict")
actual = _norm(sys.prefix)
if actual == venv:
return True
return (
-1,
f"sys.prefix={actual!r} is not the validation venv {venv!r}",
)
def check_is_venv():
"""py_func subprocess: confirm we are inside a venv, i.e. sys.prefix
differs from sys.base_prefix. This catches the case where python_bin
happens to be a system interpreter and the path-based check would
pass trivially."""
actual = _norm(sys.prefix)
base = _norm(sys.base_prefix)
if actual != base:
return True
return (
-1,
f"sys.prefix == sys.base_prefix == {actual!r}: not running in a venv",
)

131
test/validation/run.bat Normal file
View File

@@ -0,0 +1,131 @@
@echo off
SETLOCAL EnableExtensions EnableDelayedExpansion
REM Runs the testium validation suite against any installable channel of
REM testium on Windows (source, wheel, pyinstaller).
REM
REM Usage:
REM test\validation\run.bat [clean] [--mode MODE] [extra testium args]
REM
REM clean remove the validation venv before recreating it
REM (must be the first argument; useful after a Python upgrade)
REM
REM --mode MODE which testium build to validate. One of:
REM source (default) project's run.bat (src\testium)
REM wheel dist\testium-<v>-py3-none-any.whl
REM pyinstaller dist\testium-<v>.exe (or dist\testium-<v>)
REM
REM Every test-execution subprocess runs in a dedicated host venv under
REM %TEMP%\testium-validation-venv (created with --system-site-packages,
REM then junit-xml is pip-installed for post_execution.py).
REM
REM The report file is suffixed with the mode so consecutive runs in
REM different modes don't overwrite each other.
SET "SCRIPT_DIR=%~dp0"
IF "%SCRIPT_DIR:~-1%"=="\" SET "SCRIPT_DIR=%SCRIPT_DIR:~0,-1%"
SET "PROJECT_DIR=%SCRIPT_DIR%\..\.."
SET /P VERSION=<"%PROJECT_DIR%\src\VERSION"
REM ---------- arg parsing ----------------------------------------------------
SET "MODE=source"
SET "CLEAN=0"
IF /I "%~1"=="clean" (
SET "CLEAN=1"
SHIFT
)
SET "EXTRA="
:PARSE_ARGS
IF "%~1"=="" GOTO ARGS_DONE
IF /I "%~1"=="--mode" (
SET "MODE=%~2"
SHIFT
SHIFT
GOTO PARSE_ARGS
)
SET "EXTRA=!EXTRA! "%~1""
SHIFT
GOTO PARSE_ARGS
:ARGS_DONE
REM ---------- locate host python ---------------------------------------------
SET "PYTHON_EXE="
py --version >nul 2>&1
IF %ERRORLEVEL% EQU 0 (
SET "PYTHON_EXE=py"
GOTO PYTHON_FOUND
)
python --version >nul 2>&1
IF %ERRORLEVEL% EQU 0 (
SET "PYTHON_EXE=python"
GOTO PYTHON_FOUND
)
echo ERROR: Python could not be found on this system.
exit /b 1
:PYTHON_FOUND
REM ---------- validation venv -------------------------------------------------
SET "VENV_DIR=%TEMP%\testium-validation-venv"
IF "%CLEAN%"=="1" IF EXIST "%VENV_DIR%" rmdir /s /q "%VENV_DIR%"
IF NOT EXIST "%VENV_DIR%" (
echo Creating validation venv at %VENV_DIR%
%PYTHON_EXE% -m venv --system-site-packages "%VENV_DIR%"
IF !ERRORLEVEL! NEQ 0 (
echo ERROR while creating the validation venv.
exit /b 1
)
call "%VENV_DIR%\Scripts\pip" install --quiet --upgrade pip
call "%VENV_DIR%\Scripts\pip" install --quiet junit-xml
)
SET "VENV_PYTHON=%VENV_DIR%\Scripts\python.exe"
REM ---------- shared "tail" forwarded to every launcher -----------------------
REM Reports are stamped with the mode so successive runs don't clobber each other.
SET "TAIL=-b -d "python_bin=%VENV_PYTHON%" -d "validation_report_file=validation-%MODE%" -- "%SCRIPT_DIR%\main.tum"%EXTRA%"
REM ---------- per-mode launcher ----------------------------------------------
echo -- validation mode: %MODE%
IF /I "%MODE%"=="source" GOTO MODE_SOURCE
IF /I "%MODE%"=="wheel" GOTO MODE_WHEEL
IF /I "%MODE%"=="pyinstaller" GOTO MODE_PYI
echo ERROR: unknown --mode '%MODE%'. Expected: source ^| wheel ^| pyinstaller.
exit /b 1
:MODE_SOURCE
call "%PROJECT_DIR%\run.bat" %TAIL%
exit /b %ERRORLEVEL%
:MODE_WHEEL
SET "WHEEL=%PROJECT_DIR%\dist\testium-%VERSION%-py3-none-any.whl"
IF NOT EXIST "%WHEEL%" (
echo ERROR: wheel not found at %WHEEL% -- run build_all.sh first.
exit /b 1
)
SET "WHEEL_VENV=%TEMP%\testium-wheel-venv-%VERSION%"
IF "%CLEAN%"=="1" IF EXIST "%WHEEL_VENV%" rmdir /s /q "%WHEEL_VENV%"
IF NOT EXIST "%WHEEL_VENV%" (
echo Creating wheel venv at %WHEEL_VENV%
%PYTHON_EXE% -m venv --system-site-packages "%WHEEL_VENV%"
call "%WHEEL_VENV%\Scripts\pip" install --quiet --upgrade pip
call "%WHEEL_VENV%\Scripts\pip" install --quiet "%WHEEL%"
)
"%WHEEL_VENV%\Scripts\python.exe" -m testium %TAIL%
exit /b %ERRORLEVEL%
:MODE_PYI
SET "PYI_BIN=%PROJECT_DIR%\dist\testium-%VERSION%.exe"
IF NOT EXIST "%PYI_BIN%" SET "PYI_BIN=%PROJECT_DIR%\dist\testium-%VERSION%"
IF NOT EXIST "%PYI_BIN%" (
echo ERROR: PyInstaller binary not found in %PROJECT_DIR%\dist -- run build_all.sh first.
exit /b 1
)
"%PYI_BIN%" %TAIL%
exit /b %ERRORLEVEL%

143
test/validation/run.sh Executable file
View File

@@ -0,0 +1,143 @@
#!/bin/bash
# Runs the testium validation suite against any installable channel of
# testium (source, wheel, pyinstaller, flatpak, appimage).
#
# Usage:
# ./test/validation/run.sh [clean] [--mode MODE] [extra testium args]
#
# clean remove the validation venv before recreating it
# (must be the first argument; useful after a Python upgrade)
#
# --mode MODE which testium build to validate. One of:
# source (default) src/testium via project run.sh
# wheel dist/testium-<v>-py3-none-any.whl
# pyinstaller dist/testium-<v>
# flatpak installed org.testium.Testium
# appimage dist/Testium-<v>-*.AppImage
#
# Every test-execution subprocess (inline <| ... |>, py_func, cycle,
# post_execution, ...) runs in a dedicated host venv under
# /tmp/testium-validation-venv. That venv is shared across modes —
# even Flatpak reaches it via flatpak-spawn --host. The validation venv
# is created with --system-site-packages so existing system packages
# (PySide6, lxml, ...) stay visible, then junit-xml is pip-installed
# for post_execution.py.
#
# The report file is suffixed with the mode (e.g. validation-flatpak.sqlite)
# so consecutive runs in different modes don't overwrite each other.
set -e
SCRIPT_PATH="$(readlink -f "$0")"
SCRIPT_DIR="$(realpath "$(dirname "$SCRIPT_PATH")")"
PROJECT_DIR="$(realpath "$SCRIPT_DIR/../..")"
VERSION="$(cat "$PROJECT_DIR/src/VERSION")"
# ---------- arg parsing -------------------------------------------------------
MODE=source
if [ "${1:-}" = "clean" ]; then
CLEAN=1
shift
else
CLEAN=0
fi
EXTRA=()
while [ $# -gt 0 ]; do
case "$1" in
--mode)
MODE="$2"
shift 2
;;
--mode=*)
MODE="${1#--mode=}"
shift
;;
*)
EXTRA+=("$1")
shift
;;
esac
done
# ---------- validation venv ---------------------------------------------------
VENV_DIR="${TMPDIR:-/tmp}/testium-validation-venv"
if [ "$CLEAN" -eq 1 ]; then
rm -rf "$VENV_DIR"
fi
if [ ! -d "$VENV_DIR" ]; then
echo "Creating validation venv at $VENV_DIR"
python3 -m venv --system-site-packages "$VENV_DIR"
"$VENV_DIR/bin/pip" install --quiet --upgrade pip
"$VENV_DIR/bin/pip" install --quiet junit-xml
fi
VENV_PYTHON="$VENV_DIR/bin/python3"
# ---------- per-mode launcher -------------------------------------------------
case "$MODE" in
source)
CMD=("$PROJECT_DIR/run.sh")
;;
wheel)
WHEEL="$PROJECT_DIR/dist/testium-${VERSION}-py3-none-any.whl"
if [ ! -f "$WHEEL" ]; then
echo "ERROR: wheel not found at $WHEEL — run ./build_all.sh first." >&2
exit 1
fi
WHEEL_VENV="${TMPDIR:-/tmp}/testium-wheel-venv-${VERSION}"
if [ "$CLEAN" -eq 1 ]; then
rm -rf "$WHEEL_VENV"
fi
if [ ! -d "$WHEEL_VENV" ]; then
echo "Creating wheel venv at $WHEEL_VENV"
python3 -m venv --system-site-packages "$WHEEL_VENV"
"$WHEEL_VENV/bin/pip" install --quiet --upgrade pip
"$WHEEL_VENV/bin/pip" install --quiet "$WHEEL"
fi
CMD=("$WHEEL_VENV/bin/python" -m testium)
;;
pyinstaller)
PYI_BIN="$PROJECT_DIR/dist/testium-${VERSION}"
if [ ! -x "$PYI_BIN" ]; then
echo "ERROR: PyInstaller binary not found at $PYI_BIN — run ./build_all.sh first." >&2
exit 1
fi
CMD=("$PYI_BIN")
;;
flatpak)
if ! flatpak info --user org.testium.Testium &>/dev/null \
&& ! flatpak info --system org.testium.Testium &>/dev/null; then
echo "ERROR: org.testium.Testium is not installed." >&2
echo " flatpak install --user $PROJECT_DIR/dist/testium-${VERSION}.flatpak" >&2
exit 1
fi
CMD=(flatpak run --command=testium org.testium.Testium)
;;
appimage)
APPIMAGE=$(ls -1t "$PROJECT_DIR/dist"/Testium-"${VERSION}"-*.AppImage 2>/dev/null | head -1)
if [ -z "$APPIMAGE" ] || [ ! -x "$APPIMAGE" ]; then
echo "ERROR: no AppImage for version $VERSION under $PROJECT_DIR/dist — run ./build_all.sh first." >&2
exit 1
fi
CMD=("$APPIMAGE")
;;
*)
echo "ERROR: unknown --mode '$MODE'. Expected: source|wheel|pyinstaller|flatpak|appimage." >&2
exit 1
;;
esac
# ---------- launch ------------------------------------------------------------
echo "-- validation mode: $MODE"
echo "-- launch: ${CMD[*]}"
exec "${CMD[@]}" -b \
-d "python_bin=$VENV_PYTHON" \
-d "validation_report_file=validation-$MODE" \
-- "$SCRIPT_DIR/main.tum" "${EXTRA[@]}"