diff --git a/CLAUDE.md b/CLAUDE.md index 67f212d..8cbeb46 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -92,11 +92,33 @@ All dialog items (`dialog_image`, `dialog_question`, `dialog_references`, `dialo - Per-item log capture (`stdio_redir.read()`) is naturally race-free thanks to per-thread buffers (see `StdoutProxy`). ### Thread-aware stdout (`StdoutProxy`) -`src/lib/stdout_redirect.py` — when `log_stored: True`, `intercept()` installs a `StdoutProxy` as `sys.stdout`/`sys.stderr` instead of a single shared `StringQueue`. The proxy: +`src/testium/runtime/stdout_redirect.py` — when `log_stored: True`, `intercept()` installs a `StdoutProxy` as `sys.stdout`/`sys.stderr` instead of a single shared `StringQueue`. The proxy: - Holds one `StringQueue` per thread (registered via `register_thread(buffer=...)`). The main thread uses a default buffer; each parallel branch's thread registers its own at start and unregisters at end. `stdio_redir.read()` reads the calling thread's buffer → `addTest()` of an item running in branch X reads X's clean, non-interleaved output. - For the live stream (terminal in batch / GUI panel), prefixes every line emitted from a branch's thread with `[] ` so concurrent branches stay readable. - Exposes `write` / `writeln` / `flush` (Python 3.14's `unittest` calls `stream.writeln()` directly without `_WritelnDecorator`). +### Subprocess API contract (py_func / lua_func) + +User test scripts running inside a `py_func` or `lua_func` subprocess **must** use the JSON-RPC bridge to interact with testium state: + +- Python: `import py_func.tm as tm` — auto-generates wrappers for every function in `runtime/api.py:SUPPORTED_API`. `tm.gd`/`tm.setgd`/`tm.delgd` go through JSON-RPC to the parent. +- Lua: `local tm = require("tm")` — same idea on the Lua side. + +`api.testium` is the *main-process* implementation; it is **not** exposed to subprocesses by design (not bundled in PyInstaller, not on the subprocess `PYTHONPATH` in pip-installed mode either when isolation is preserved). An import attempt from a subprocess script is a code smell and is detected by `test/validation/items/isolation/`. + +To add a new API call usable from subprocesses: +1. Add the function to `api/testium.py` +2. Add its name to `SUPPORTED_API` in `runtime/api.py` +3. It is auto-exposed via JSON-RPC by `interpreter/utils/api_srv.py` and auto-wrapped by `py_func/tm.py:_make_api` + +### External interpreter resolution (`bins.py`) +`src/testium/interpreter/utils/bins.py` — single source of truth for the paths to the external Python and Lua interpreters used by subprocesses. + +- `python_bin()` / `lua_bin()` : resolve once, cache in memory. User can override via the `python_bin` / `lua_bin` global dict keys (typically populated from the YAML config). Falls back to discovery on PATH (candidates: `python3`/`python` and `lua`/`lua5.5`/`lua5.4`/`lua5.3`/`lua5.2`/`lua5.1`). +- `ensure(*names)` : called by `TestSet._validate_runtime_deps()` at test load. Always requires `python` (the eval engine always runs); requires `lua` only if a `lua_func` item is in the tree. Fails fast with a clear error citing tried candidates and override key. + +Engines (`PyProcessBase`, `LuaProcessBase`, `EvalExecEngine`) call `bins.python_bin()`/`bins.lua_bin()` themselves — call sites never pass an explicit binary path. + ## Key files | Path | Role | @@ -161,23 +183,61 @@ Icons are assigned once when the test file is loaded (not updated live on theme The sub-test's own pass/fail result is intentionally not propagated. +### Report exporters & plugins +`src/testium/interpreter/test_report/test_report.py` — `_EXPORTER_REGISTRY` dict maps a format name (cmd key in the YAML `report.export`) to a lazy loader. Built-ins: `text`, `json`, `junit` (needs `junit_xml`), `html` (needs `lxml`). `sqlite` is the storage layer, no-op as an export. + +Third-party plugins are discovered at module import via `importlib.metadata.entry_points(group="testium.exporters")` — installing a wheel that declares such an entry point is enough, no testium config change needed: +```toml +[project.entry-points."testium.exporters"] +my_format = "my_pkg:MyExporter" +``` +Exporter contract: `__init__(self, name, con, path, pats, keys, no_header=False)` — the class does its work in `__init__` and writes to `path`. + +Behaviour on errors: +- Unknown format → info line `[report] Export skipped: format "X" not found. Available: ...`, run continues. +- Optional dependency missing → same info line with a pip-install hint, run continues. + +A real-world test plugin lives at `test/validation/fake_exporter/` (CSV exporter, auto-installed by `scripts/build_env.sh` and exercised by `test/validation/items/report_plugin/`). + +## Packaging + +Three distribution channels coexist, sharing the single `src/testium/` package: + +| Channel | Where | Notes | +|---------|-------|-------| +| Wheel (`pip install`) | `src/pyproject.toml` | Vanilla Python package; entry point `testium = "testium:main"` | +| PyInstaller binary | `package/pyinstaller/` | Single ~130 MB binary. `py_func`, `runtime`, `lua_func` bundled at `_MEIPASS` root so the **host** Python can find them when launched as `python3 py_func`. `api`/`interpreter` are **not** exposed (subprocess isolation). | +| Flatpak | `package/flatpak/` | (Existing recipe, not actively maintained in current refactor wave.) | + +The `.deb` work-in-progress lives in `package/deb/`: +- `test_distro.sh debian:bookworm | debian:trixie | ubuntu:24.04` spins up a Docker/Podman container, reports system package availability, falls back to pip for what's missing (`pyside6` on bookworm/ubuntu, `telnetlib3`, `junit_xml`), runs the validation suite. Currently green on the three targets. + ## Recent fixes / notable changes -- `parallel` item: new item with `sync: all|any`, `wait_for`, daemon threads, `_stop_branch_recursively()`. Each branch thread registers a per-thread stdout buffer with `stdio_redir.register_thread(...)` so its log capture and live-output prefix work in isolation. -- `parallel_branch` icon: distinct single-arrow icon (`parallel_branch.png`) separate from the parallel container's three-arrow icon. -- `parallel` F1 panel: `steps` stripped from each branch dict so the panel shows per-branch attributes without duplicating the tree. -- `test_item_container.py`: new `TestItemContainer` base class extracted from Group/Cycle patterns -- `test_item_sleep.py`: interruptible loop (checks `self._is_stopped`) instead of blocking `time.sleep()` so `sync: any` can stop slow branches quickly -- `stdout_redirect.py`: rewrote `intercept()` to install a `StdoutProxy` (thread-aware: per-thread capture buffers + branch-prefixed live output). Adds `writeln()` for Python 3.14 unittest compatibility. -- `test_report.py`: `check_same_thread=False` + lock around the SQLite `INSERT` for parallel branch concurrency. Log capture itself is race-free thanks to per-thread buffers. +- Restructure: single `src/testium/` Python package (was 4 sibling top-levels: `testium`, `lib`, `py_func`, `lua_func`). `lib/` → `runtime/`, `libs/` → `api/`. `pip install` now produces a clean `site-packages/testium/` with no top-level pollution; `.lua` files travel via `package_data`. +- `bins.py`: centralised resolution + cache of external `python3` / `lua` binaries. Replaces the scattered `tm.gd("python_bin")`/`tm.gd("lua_bin")` dance and the duplicated discovery logic in `py_process.py`/`lua_process.py`. Validates at test load via `TestSet._validate_runtime_deps()` so missing interpreters fail fast. +- Subprocess API contract: user scripts in `py_func`/`lua_func` use the JSON-RPC bridge (`py_func.tm` / Lua `tm`) — never `api.testium` / `interpreter.*` directly. `SUPPORTED_API` extended with `OS`, `get_main_dir`, `init_timestamp`, `timestamp`, `timestamp_as_sec` so subprocess scripts have the same surface as main-process code. +- Report exporter plugin registry (`test_report.py`): `_EXPORTER_REGISTRY` + `entry_points("testium.exporters")` discovery. Missing format → info line, run continues. +- About dialog rework: `QVBoxLayout` (resizable), version + dirty/branch info in a `QLabel` (auto-sized), copyright + clickable EUPL-1.2 link. +- `test_ctrl.control()`: drain stale responses (left over from polled `loaded()` after `clear()` race) instead of failing on a wrong cmd key — fixes a "Unexpected return error in test set controller" seen in GUI mode after a fast reload. +- `lua_process.py`: stderr no longer DEVNULL'd so actual Lua errors (missing `cjson`/`socket`) surface instead of "Connection refused". +- `run_post_exec`: failure message uses `print_warn` (was `print_debug` — silent in non-debug runs). +- Python 3.11 compat: replaced PEP 701 nested-quote f-strings (e.g. `f"... {d["k"]} ..."`) with single-quote inner strings or string concatenation. +- `parallel` item: new item with `sync: all|any`, `wait_for`, daemon threads, `_stop_branch_recursively()`. Each branch thread registers a per-thread stdout buffer. +- `parallel_branch` icon: distinct single-arrow icon (`parallel_branch.png`). +- `parallel` F1 panel: `steps` stripped from each branch dict. +- `test_item_container.py`: shared base class extracted from Group/Cycle. +- `test_item_sleep.py`: interruptible loop so `sync: any` can stop slow branches quickly. +- `stdout_redirect.py`: `StdoutProxy` (thread-aware buffers + branch-prefixed live output, `writeln()` for Python 3.14 unittest). +- `test_report.py`: thread-safe SQLite INSERT for parallel branch concurrency. - `terminal.py`: deleted — `-m`/`--terminal` mode removed. -- `batch.py`: premature loop exit when `gd_update` messages (no `"id"` key) were mistaken for the "finished" signal — fix: `"id" in m and m["id"] is None` -- `batch.py`: `control("loaded")` deadlock if `TestProcess` crashed before `cmd_th` started — fix: daemon thread + `threading.Event` + `is_alive()` polling -- `termlog.py`: `COLOR_DEFAULT = Fore.WHITE` invisible on light terminals; added auto-detection + light palette. Also fixed `write()` residue accumulation bug (`s[pos:]` → `s[pos+1:]`). -- Dialog items: `auto_result`/`auto_value` now used in non-interactive text mode; dialogs without `auto_result` FAIL immediately in batch mode. -- `run` item: renamed `tum_fime` → `tum`; removed `stdout=PIPE` (caused deadlock with `multiprocessing` spawn); result simplified to PASS on any completed subprocess. -- `unittest` item: renamed from `unittest_file` (cmd key, display name, Python constant `TYPE_UNITTEST_FILE` → `TYPE_UNITTEST`). -- GUI test tree: check and fold state preserved across same-file reloads (`test_file_manager.py`). -- Licence: EUPL-1.2 (`LICENSE`, `CONTRIBUTING.md`, `pyproject.toml`). +- `batch.py`: premature finish bug on `gd_update` (no `"id"` key) — fix uses `"id" in m and m["id"] is None`. +- `batch.py`: `control("loaded")` deadlock on TestProcess crash — fix uses daemon thread + `threading.Event` + `is_alive()` polling. +- `termlog.py`: light/dark terminal auto-detection (`COLORFGBG`, OSC 11) + write residue bug. +- Dialog items: `auto_result`/`auto_value` for non-interactive text mode; dialogs without `auto_result` FAIL immediately in batch. +- `run` item: renamed `tum_fime` → `tum`; removed `stdout=PIPE` deadlock; PASS on any completed subprocess. +- `unittest` item: renamed from `unittest_file`. +- GUI test tree: check and fold state preserved across same-file reloads. +- Licence: EUPL-1.2. ## Validation tests Located in `test/validation/`. Run with `-b` flag: diff --git a/doc/examples/dummy/post_execution.py b/doc/examples/dummy/post_execution.py index af4c56f..a74de05 100644 --- a/doc/examples/dummy/post_execution.py +++ b/doc/examples/dummy/post_execution.py @@ -1,5 +1,5 @@ -import api.testium as tm +import py_func.tm as tm def post_exec(): print('Success !!!!') diff --git a/doc/manual/sphinx/source/helper_lib.rst b/doc/manual/sphinx/source/helper_lib.rst index 900358a..a8fdeda 100644 --- a/doc/manual/sphinx/source/helper_lib.rst +++ b/doc/manual/sphinx/source/helper_lib.rst @@ -4,7 +4,12 @@ Python helper library ====================== A python library including helper function for python modules called from -testium. +testium ``py_func`` items. + +User scripts run inside the ``py_func`` subprocess and interact with testium +through a JSON-RPC bridge — the ``py_func.tm`` module. They must **not** +import ``api.testium`` or ``interpreter.*`` directly: those are main-process +modules and may not even be reachable in a packaged build (PyInstaller, .deb). To include the support of this library in a python script, the following line must be included in the script header: @@ -18,58 +23,38 @@ line must be included in the script header: Global variables helper functions ---------------------------------- -To manage values in the global variables dataset, the following testium library API -must be used: +To manage values in the global variables dataset: .. automodule:: py_func.tm :members: gd, setgd, delgd :undoc-members: :no-index: -Console helper functions ------------------------- - -Every opened console instance is added to a list with the -key ``console_instances`` of the global variables. - -The instance is removed from the list on close step of the ``console`` test item. - -To manage consoles from within ``py_func`` python functions, -the following testium library API can be used: - -.. automodule:: libs.testium - :members: add_console, remove_console, console - :undoc-members: - :no-index: - Plot helper functions ------------------------ -Every opened plot window instance is added to a list with the -key ``plot_instances`` of the global variables. +Add values to a running plot or read the last value from it: -The instance is removed from the list on close step of the ``plot`` test item. - -To manage plots from within ``py_func`` python functions, -the following testium library API can be used: - -.. automodule:: libs.testium - :members: add_plot, remove_plot, plot, add_plot_values, last_plot_value +.. automodule:: py_func.tm + :members: add_plot_values, last_plot_value :undoc-members: :no-index: +Console and plot **lifecycle** management (``add_console``, ``remove_console``, +``console``, ``add_plot``, ``remove_plot``, ``plot``) is performed by the +``console`` and ``plot`` test items themselves — not from user ``py_func`` +scripts. Use those test items to open/close consoles and plots. + Other helper functions ------------------------ -.. automodule:: libs.testium - :members: OS, get_main_dir, timestamp, timestamp_as_sec +.. automodule:: py_func.tm + :members: OS, get_main_dir, init_timestamp, timestamp, timestamp_as_sec, text_mode :undoc-members: :no-index: Debug mode ------------------------ -.. automodule:: libs.testium - :members: debug_enabled, enable_debug, print_debug, print_info, print_warn - :undoc-members: - :no-index: +The ``test_debug`` global variable controls debug-only output. Read or write +it via ``tm.gd("test_debug")`` / ``tm.setgd("test_debug", True)``. diff --git a/doc/manual/sphinx/source/reports.rst b/doc/manual/sphinx/source/reports.rst index e4c78c5..ccc93ff 100644 --- a/doc/manual/sphinx/source/reports.rst +++ b/doc/manual/sphinx/source/reports.rst @@ -6,18 +6,25 @@ Reports If a report is required (in addition to the log), the ``report`` YAML element must be added at the root of the TUM main test file. -The ``report`` YAML element has the following form: +The ``report`` element accepts a single export or a list of them under the +``export`` key. Each export entry uses the format name as its key: .. code-block:: yaml - :caption: reports global settings + :caption: reports global settings — multiple exports report: enabled: True - file_name: $(test_name).rep - path: $(home)/reports - pattern: "Console%" - export: junit - log_stored: False + log_stored: True + export: + - sqlite: + path: $(home)/reports + file_name: $(test_name).db + - junit: + path: $(home)/reports + file_name: $(test_name).xml + - html: + path: $(home)/reports + file_name: $(test_name).html .. table:: report attributes :widths: 20, 30, 50 @@ -27,21 +34,93 @@ The ``report`` YAML element has the following form: +-----------------+-----------------------+-------------------------------------------+ | ``enabled`` | ``True`` | Report activated | +-----------------+-----------------------+-------------------------------------------+ - | ``file_name`` | / | Report file name | + | ``log_stored`` | ``False`` | When ``True``, captures stdout per test | + | | | item so exports (html, json) can include | + | | | the log of each item. | +-----------------+-----------------------+-------------------------------------------+ - | ``path`` | ``$(report_path)`` | Report storage path By default, it uses | - | | | the default one set in the | - | | | preferences. | + | ``export`` | / | One export entry or a list of them. Each | + | | | entry's key is the format name (see | + | | | below). | +-----------------+-----------------------+-------------------------------------------+ - | ``pattern`` | / | The pattern in SQL wildachars syntax | - | | | to be applied on test names to | - | | | selected reported tests. | + +Each export entry supports the following sub-attributes: + +.. table:: export attributes + :widths: 20, 30, 50 + +-----------------+-----------------------+-------------------------------------------+ - | ``export`` | / | The type of export. For exemple junit. | - | | | By default, the sqlite format is | - | | | used to generate reports. | + | Attribute | default value | Description | +-----------------+-----------------------+-------------------------------------------+ - | ``log_stored`` | / | Defines if the output log of each | - | | | test is accessible to generate the | - | | | report export. | + | ``path`` | ``$(report_path)`` | Output directory. | +-----------------+-----------------------+-------------------------------------------+ + | ``file_name`` | / | Output file name. May include | + | | | ``$(...)`` global-dict expansions. | + +-----------------+-----------------------+-------------------------------------------+ + | ``pattern`` | / | One or more SQL ``LIKE`` patterns | + | | | applied on the test ``name``. | + +-----------------+-----------------------+-------------------------------------------+ + | ``key`` | / | One or more SQL ``LIKE`` patterns | + | | | applied on the test ``key`` | + | | | (the per-item ``key`` attribute). | + +-----------------+-----------------------+-------------------------------------------+ + +Built-in formats +^^^^^^^^^^^^^^^^ + +* ``sqlite`` — raw SQLite database (storage layer; selecting it persists the run). +* ``text`` — simple indented text dump of the test tree. +* ``json`` — full report as JSON: ``{"header": {...}, "tests": [...]}``. +* ``junit`` — JUnit XML (requires the ``junit_xml`` Python package). +* ``html`` — single HTML page with header, results table and per-item logs (requires ``lxml``). + +If a format is unknown or its optional dependency is missing, the export is +skipped with an ``[report] Export skipped: ...`` info line on stdout — the +test run is **not** interrupted. + +.. _sec_reports_plugins: + +Custom export formats (plugins) +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +A third-party Python package can register additional export formats via the +``testium.exporters`` setuptools entry point group. Once installed in the same +Python environment as testium, the format is auto-detected at startup and can +be referenced from the YAML by its declared name. + +Plugin contract — a class with this constructor signature: + +.. code-block:: python + :caption: minimal exporter contract + + class MyExporter: + def __init__(self, name, con, path, pats, keys, no_header=False): + # name : str — report name + # con : sqlite3.Connection (read) — tables: header, tests + # path : str — output file path (already expansed) + # pats : list[str] — LIKE filters on test_name (may be empty) + # keys : list[str] — LIKE filters on report_key (may be empty) + # no_header : bool — skip header section (set by the inline + # `report` test item) + ... # do the work in __init__ and write to `path` + +Tables and columns of the SQLite report: + +* ``header(key TEXT, value TEXT)`` — keys: ``report_version``, ``test_file``, + ``test_name``, ``test_result``, ``test_revision``, ``testium_version``, + ``testrun_date``, ``testrun_time``, ``test_duration``. +* ``tests`` — 12 columns: ``timestamp_start``, ``test_id``, ``parent_id``, + ``level``, ``test_name``, ``test_type``, ``report_key``, ``result`` + (``PASS``/``FAIL``/``SKIP``), ``message``, ``duration`` (ms), + ``log`` (captured stdout when ``log_stored: True``), ``data`` (JSON of + values reported via ``self.reportValue(...)``). + +Declaration in the plugin's ``pyproject.toml``: + +.. code-block:: toml + :caption: registering an exporter via entry-points + + [project.entry-points."testium.exporters"] + my_format = "my_pkg:MyExporter" + +The plugin is then usable in any ``.tum`` report block as ``my_format:`` — +no testium configuration change required. diff --git a/doc/manual/sphinx/source/test_items/py_func_test_item.rst b/doc/manual/sphinx/source/test_items/py_func_test_item.rst index b31be24..469f2a9 100644 --- a/doc/manual/sphinx/source/test_items/py_func_test_item.rst +++ b/doc/manual/sphinx/source/test_items/py_func_test_item.rst @@ -13,7 +13,7 @@ class ``py_func`` item This is the normal way of calling some custom python code. -A class must be defined and derived from ``FunctionItem`` from the ``libs.testium`` module. +A class must be defined and derived from ``FunctionItem`` from the ``py_func.tm`` module. From this class it is possible to define some custom reported values with the following API diff --git a/doc/manual/testium_manual.pdf b/doc/manual/testium_manual.pdf index d2c1c05..61fdb29 100644 Binary files a/doc/manual/testium_manual.pdf and b/doc/manual/testium_manual.pdf differ diff --git a/package/deb/test_distro.sh b/package/deb/test_distro.sh new file mode 100755 index 0000000..e0a3186 --- /dev/null +++ b/package/deb/test_distro.sh @@ -0,0 +1,141 @@ +#!/usr/bin/env bash +# test_distro.sh — verify testium runs on a target Debian/Ubuntu distrib. +# +# Spins up a Docker container of the requested image, checks which expected +# system Python packages are available (apt), installs them, installs the +# testium wheel, and runs a smoke test that exercises batch mode + py_func +# subprocess. +# +# Usage: +# ./test_distro.sh debian:bookworm +# ./test_distro.sh debian:trixie +# ./test_distro.sh ubuntu:24.04 + +set -euo pipefail + +IMAGE="${1:?Usage: $0 e.g. debian:bookworm | debian:trixie | ubuntu:24.04}" +ROOT=$(realpath "$(dirname "$0")/../..") + +# Container runtime: prefer docker if available, fall back to podman +if command -v docker >/dev/null 2>&1; then + CTR=docker +elif command -v podman >/dev/null 2>&1; then + CTR=podman +else + echo "ERROR: neither docker nor podman is installed" >&2 + exit 1 +fi +echo "[host] Using $CTR" + +# --- Build the wheel on the host if it does not already exist +WHEEL_DIR="$ROOT/src/dist" +PYTHON_HOST="$ROOT/test/tmp/.venv/bin/python3" +[ -x "$PYTHON_HOST" ] || PYTHON_HOST=python3 +if ! ls "$WHEEL_DIR"/testium-*.whl >/dev/null 2>&1; then + echo "[host] Building wheel..." + (cd "$ROOT/src" && "$PYTHON_HOST" -m build --wheel >/dev/null) +fi +WHEEL=$(ls "$WHEEL_DIR"/testium-*.whl | head -1) +WHEEL_NAME=$(basename "$WHEEL") +echo "[host] Using $WHEEL_NAME" + +# Expected system Python packages on the target distrib +APT_PACKAGES=( + python3 + python3-pip + python3-setuptools + python3-pyside6.qtwidgets + python3-yaml + python3-jinja2 + python3-colorama + python3-git + python3-pexpect + python3-matplotlib + python3-lxml + python3-serial + python3-telnetlib3 + lua5.4 + lua-cjson + lua-socket + git +) + +echo "=== Testing on $IMAGE ===" + +$CTR run --rm \ + -v "$ROOT:/testium:ro" \ + -e WHEEL_NAME="$WHEEL_NAME" \ + -e PACKAGES="${APT_PACKAGES[*]}" \ + "$IMAGE" \ + bash -c ' + set -e + export DEBIAN_FRONTEND=noninteractive + apt-get update -qq + + # 1. Availability check + echo + echo "--- System package availability ---" + AVAILABLE=() + MISSING=() + for pkg in $PACKAGES; do + if apt-cache show "$pkg" >/dev/null 2>&1; then + AVAILABLE+=("$pkg") + echo " OK $pkg" + else + MISSING+=("$pkg") + echo " MISSING $pkg" + fi + done + echo + + # 2. Install available packages + echo "--- Installing system packages ---" + apt-get install -qq -y --no-install-recommends "${AVAILABLE[@]}" ca-certificates >/dev/null + + # 3. Map missing apt packages to their PyPI equivalents and pip-install + # them as a fallback (kept minimal so the run is still a "system" + # install for the most part) + declare -A PIP_FALLBACK=( + [python3-pyside6.qtwidgets]=pyside6 + [python3-telnetlib3]=telnetlib3 + ) + # junit_xml has no Debian package — install it via pip so the + # validation post_execution.py can import it. + EXTRA_PIP=(junit-xml) + PIP_PKGS=() + for m in "${MISSING[@]}"; do + fallback="${PIP_FALLBACK[$m]:-}" + if [ -n "$fallback" ]; then + PIP_PKGS+=("$fallback") + fi + done + PIP_PKGS+=("${EXTRA_PIP[@]}") + if [ ${#PIP_PKGS[@]} -gt 0 ]; then + echo "--- Installing missing deps via pip: ${PIP_PKGS[*]} ---" + pip install --break-system-packages "${PIP_PKGS[@]}" >/dev/null + fi + + # 4. Install testium wheel + echo "--- Installing testium wheel ---" + pip install --break-system-packages --no-deps "/testium/src/dist/$WHEEL_NAME" >/dev/null + + # 5. Install the fake_exporter plugin (needed by the report_plugin + # validation test which exercises entry-points discovery). + # Copy it first because /testium is mounted read-only and the + # setuptools backend touches its build dir. + echo "--- Installing testium-fake-exporter (test plugin) ---" + cp -r /testium/test/validation/fake_exporter /tmp/fake_exporter + pip install --break-system-packages /tmp/fake_exporter >/dev/null + + # 6. Run the full validation suite. Outputs are streamed live so + # progress is visible — the suite takes a couple of minutes. + # Reports go to /tmp/testium-validation since /testium is RO. + echo "--- Running validation suite ---" + mkdir -p /tmp/testium-validation + cd /testium + testium -b -o \ + -d "validation_report_path=/tmp/testium-validation/" \ + -- test/validation/main.tum + ' + +echo "=== $IMAGE: PASS ===" diff --git a/package/pyinstaller/testium.spec b/package/pyinstaller/testium.spec index f33d90d..35e430b 100644 --- a/package/pyinstaller/testium.spec +++ b/package/pyinstaller/testium.spec @@ -1,4 +1,13 @@ # -*- mode: python ; coding: utf-8 -*- +import os + +# junit_xml is imported by post_exec scripts running under the *host* Python, +# not the frozen interpreter — so bundling it via hiddenimports alone is not +# enough. We also drop its source files at the _MEIPASS root so the host +# python3 finds them via the PYTHONPATH that py_process.py sets to +# tstium_path (= _MEIPASS when frozen). +import junit_xml as _junit_xml +JUNIT_XML_DIR = os.path.dirname(_junit_xml.__file__) a = Analysis( ['../../src/testium/__main__.py'], @@ -9,10 +18,18 @@ a = Analysis( # py_func subprocess is launched with the *host* Python (not the # frozen interpreter): it needs the source files on disk to find them # via cwd=subproc_path() and `python3 py_func` + `from runtime.*`. + # py_func/, lua_func/ and runtime/ are bundled at the _MEIPASS root + # because the py_func subprocess is launched with the *host* Python + # (not the frozen interpreter): it needs the source files on disk to + # find them via cwd=subproc_path() and `python3 py_func` + + # `from runtime.*`. api/ and interpreter/ are intentionally NOT + # exposed: user py_func scripts must go through py_func.tm + # (JSON-RPC bridge) for any testium API call. datas=[('../../src/VERSION', '.'), ('../../src/testium/lua_func', 'lua_func'), ('../../src/testium/py_func', 'py_func'), - ('../../src/testium/runtime', 'runtime')], + ('../../src/testium/runtime', 'runtime'), + (JUNIT_XML_DIR, 'junit_xml')], hiddenimports=["git", "interpreter", "main_win", diff --git a/src/testium/api/runtime_plot.py b/src/testium/api/runtime_plot.py index fdaef8c..0fc3149 100644 --- a/src/testium/api/runtime_plot.py +++ b/src/testium/api/runtime_plot.py @@ -270,7 +270,7 @@ class RuntimePlotPeriodic(PeriodicTimer): self.func_name = func_name self.args = args self.post_eval = post_eval - self.proc = PyFuncExecEngine(tm.gd("python_bin", ""), api_request, 10) + self.proc = PyFuncExecEngine(api_request, 10) self.proc.start() if not self.proc.wait_ready(10): raise ETUMRuntimeError( diff --git a/src/testium/interpreter/process.py b/src/testium/interpreter/process.py index 859f026..1f883e0 100644 --- a/src/testium/interpreter/process.py +++ b/src/testium/interpreter/process.py @@ -211,7 +211,7 @@ class TestProcess(Process): env_init() # Creation of the python evaluation process for loading of the complete test - eval_proc = eval_process_init("", api_request, 10, test_dir) + eval_proc = eval_process_init(api_request, 10, test_dir) eval_proc.start() tm.print_debug(f"python bin is: '{eval_proc.python_bin}'.") if not eval_proc.wait_ready(10): diff --git a/src/testium/interpreter/test_items/test_item_cycle.py b/src/testium/interpreter/test_items/test_item_cycle.py index b3d3369..c04fc8e 100644 --- a/src/testium/interpreter/test_items/test_item_cycle.py +++ b/src/testium/interpreter/test_items/test_item_cycle.py @@ -207,7 +207,7 @@ then considered as 'False'""") else: pl = [self._currentLoop] - proc = PyFuncExecEngine(tm.gd("python_bin", ""), api_request, 10) + proc = PyFuncExecEngine(api_request, 10) proc.start() if not proc.wait_ready(10): raise ETUMRuntimeError( diff --git a/src/testium/interpreter/test_items/test_item_lua_func.py b/src/testium/interpreter/test_items/test_item_lua_func.py index c4da3b0..1b9ba54 100644 --- a/src/testium/interpreter/test_items/test_item_lua_func.py +++ b/src/testium/interpreter/test_items/test_item_lua_func.py @@ -31,7 +31,7 @@ class TestItemLuaFunc(TestItem): self.func_name = self._prms.getParam("func_name", required=True) self.params = self._prms.getParamAll("param") self._context_id = self._prms.getParam("context_id", default=None, processed=False) - self._lua_func_proc = LuaFuncExecEngine(tm.gd("lua_bin", ""), api_request, 10) + self._lua_func_proc = LuaFuncExecEngine(api_request, 10) def _get_engine(self): """Return (engine, persistent). If context_id is set, use a shared persistent engine.""" @@ -41,7 +41,7 @@ class TestItemLuaFunc(TestItem): ctx_id = self._prms.expanse(self._context_id) contexts = tm.gd(_LUA_FUNC_CONTEXTS_KEY, {}) if ctx_id not in contexts: - contexts[ctx_id] = LuaFuncExecEngine(tm.gd("lua_bin", ""), api_request, 10) + contexts[ctx_id] = LuaFuncExecEngine(api_request, 10) tm.setgd(_LUA_FUNC_CONTEXTS_KEY, contexts) return contexts[ctx_id], True diff --git a/src/testium/interpreter/test_items/test_item_py_func.py b/src/testium/interpreter/test_items/test_item_py_func.py index dbd2bd6..59af448 100644 --- a/src/testium/interpreter/test_items/test_item_py_func.py +++ b/src/testium/interpreter/test_items/test_item_py_func.py @@ -31,7 +31,7 @@ class TestItemPyFunc(TestItem): self.func_name = self._prms.getParam("func_name", required=True) self.params = self._prms.getParamAll("param") self._context_id = self._prms.getParam("context_id", default=None, processed=False) - self._py_func_proc = PyFuncExecEngine(tm.gd("python_bin", ""), api_request, 10) + self._py_func_proc = PyFuncExecEngine(api_request, 10) def _get_engine(self): """Return (engine, persistent). If context_id is set, use a shared persistent engine.""" @@ -41,7 +41,7 @@ class TestItemPyFunc(TestItem): ctx_id = self._prms.expanse(self._context_id) contexts = tm.gd(_PY_FUNC_CONTEXTS_KEY, {}) if ctx_id not in contexts: - contexts[ctx_id] = PyFuncExecEngine(tm.gd("python_bin", ""), api_request, 10) + contexts[ctx_id] = PyFuncExecEngine(api_request, 10) tm.setgd(_PY_FUNC_CONTEXTS_KEY, contexts) return contexts[ctx_id], True diff --git a/src/testium/interpreter/test_items/test_item_runtime_plot.py b/src/testium/interpreter/test_items/test_item_runtime_plot.py index 11c921e..e6257a2 100644 --- a/src/testium/interpreter/test_items/test_item_runtime_plot.py +++ b/src/testium/interpreter/test_items/test_item_runtime_plot.py @@ -40,6 +40,7 @@ class TestItemPlotActionOpen(TestItemPlotAction): try: gname = self._prms.expanse(self.token) lpath = self._prms.expanse(self._log_path) + runtime_plot = importlib.import_module("api.runtime_plot") gr = runtime_plot.RuntimePlot(gname, lpath) tm.add_plot(gr) @@ -233,6 +234,3 @@ class TestItemPlot(TestItemActions): ) self.actions_token = self._prms.getParam("plot_name", required=True) - - global runtime_plot - runtime_plot = importlib.import_module("api.runtime_plot") diff --git a/src/testium/interpreter/test_report/test_report.py b/src/testium/interpreter/test_report/test_report.py index c5901e9..938bbd3 100644 --- a/src/testium/interpreter/test_report/test_report.py +++ b/src/testium/interpreter/test_report/test_report.py @@ -58,7 +58,6 @@ def _discover_plugins(): try: cls = ep.load() _EXPORTER_REGISTRY[ep.name] = lambda c=cls: c - print(f'[testium] Loaded report exporter plugin: "{ep.name}"') except Exception as e: print(f'[testium] Failed to load report exporter plugin "{ep.name}": {e}') except Exception: diff --git a/src/testium/interpreter/test_set.py b/src/testium/interpreter/test_set.py index f222271..2189be1 100644 --- a/src/testium/interpreter/test_set.py +++ b/src/testium/interpreter/test_set.py @@ -8,6 +8,7 @@ import interpreter.utils.settings as prefs from interpreter.test_report.test_report import TestReport from interpreter.utils.py_func_exec import PyFuncExecEngine from interpreter.utils.api_srv import api_request +from interpreter.utils import bins from runtime.tum_except import ETUMRuntimeError from interpreter.utils.constants import TestItemType as cst_type import interpreter.utils.constants as cst @@ -49,6 +50,28 @@ class TestSet: self._tree = self.__loadTestTree(tum_fime) self.dict_report = self._testdict.get("report", None) self.set_post_exec() + self._validate_runtime_deps() + + def _validate_runtime_deps(self): + """Resolve external interpreters needed by this test tree and fail + early with a clear message if any is missing. + + Python is always required (the eval engine always runs). Lua is + only required when at least one ``lua_func`` item is present. + """ + needed = ["python"] + if self.__has_item_type(self._rootItem, cst_type.TYPE_LUA_FUNCTION): + needed.append("lua") + bins.ensure(*needed) + + def __has_item_type(self, parent, item_type): + for i in range(parent.childCount()): + child = parent.child(i) + if child.type() == item_type.item_name: + return True + if self.__has_item_type(child, item_type): + return True + return False def execute(self): self._report = TestReport(self.dict_report) @@ -352,7 +375,7 @@ class TestSet: tm.print_debug(f' No file: "{post_exec_file}".') return - proc = PyFuncExecEngine(tm.gd("python_bin", ""), api_request, 10) + proc = PyFuncExecEngine(api_request, 10) # start the process for executing external python proc.start() try: @@ -367,13 +390,13 @@ class TestSet: # tests backup is done here succ, res = proc.func_call(post_exec_file, "post_exec", []) if not succ == TestValue.SUCCESS: - tm.print_debug( + tm.print_warn( f"Test success but the \"post_exec\" function failed: {res}" ) else: succ, res = proc.func_call(post_exec_file, "post_exec_fail", []) if not succ == TestValue.SUCCESS: - tm.print_debug( + tm.print_warn( f"Test failed but the \"post_exec_fail\" function failed: {res}" ) finally: diff --git a/src/testium/interpreter/utils/bins.py b/src/testium/interpreter/utils/bins.py new file mode 100644 index 0000000..710dd30 --- /dev/null +++ b/src/testium/interpreter/utils/bins.py @@ -0,0 +1,151 @@ +"""Centralised resolution of external interpreter paths (Python, Lua). + +The user can override the path through the global dict via the keys +``python_bin`` and ``lua_bin`` (typically populated from a YAML config). +When unset, the system PATH is searched for known candidates. + +Resolution is cached in memory: each interpreter is resolved at most +once per testium process. Subsequent calls return the cached value. + +Public API +---------- +``python_bin()`` : resolved python3 path (or "" if missing) +``lua_bin()`` : resolved lua >= 5.1 path (or "" if missing) +``ensure(*names)`` : resolve every name and raise a clear error if + any is missing — meant for early validation at + test load time +``reset()`` : clear the cache (mostly useful for tests) +""" + +import shutil +import subprocess + +import api.testium as tm +from interpreter.utils.paths import sys_app_path_lin, sys_app_path_win +from runtime.tum_except import ETUMRuntimeError + + +# ---------- Discovery primitives --------------------------------------------- + +_PYTHON_CANDIDATES = ["python3", "python"] +_LUA_CANDIDATES = ["lua", "lua5.5", "lua5.4", "lua5.3", "lua5.2", "lua5.1"] + + +def _which(name): + func = sys_app_path_win if tm.OS() == "Windows" else sys_app_path_lin + return func(name) + + +def _python_version(path): + cmd = [path, "-c", "import sys; print(sys.version_info[:3])"] + try: + r = subprocess.run( + cmd, capture_output=True, text=True, + encoding=tm.sys_encoding(), timeout=10, + ) + except (FileNotFoundError, PermissionError, subprocess.TimeoutExpired): + return None + try: + return eval(r.stdout) + except Exception: + return None + + +def _is_python3(path): + v = _python_version(path) + return v is not None and v[0] == 3 + + +def _lua_version(path): + try: + r = subprocess.run( + [path, "-v"], capture_output=True, text=True, timeout=10, + ) + except (FileNotFoundError, PermissionError, subprocess.TimeoutExpired): + return None + # On Windows the version banner goes to stderr. + line = r.stdout or r.stderr + try: + major, minor, _patch = line.split(" ")[1].split(".") + return (int(major), int(minor)) + except (IndexError, ValueError): + return None + + +def _is_lua51(path): + v = _lua_version(path) + return v is not None and v >= (5, 1) + + +# ---------- Resolver --------------------------------------------------------- + +# (display name, globdict override key, candidate list, validator) +_SPECS = { + "python": ("Python 3", "python_bin", _PYTHON_CANDIDATES, _is_python3), + "lua": ("Lua 5.1+", "lua_bin", _LUA_CANDIDATES, _is_lua51), +} + +_resolved = {} + + +def _resolve(name): + if name in _resolved: + return _resolved[name] + + display, gd_key, candidates, validator = _SPECS[name] + override = tm.gd(gd_key, "") or "" + + path = "" + if override: + if shutil.which(override) and validator(override): + path = override + else: + tm.print_warn( + f"Configured {display} interpreter '{override}' is not usable; " + f"falling back to discovery." + ) + + if not path: + for c in candidates: + p = _which(c) + if not p: + continue + if validator(p): + path = p + break + + _resolved[name] = path + return path + + +def python_bin(): + return _resolve("python") + + +def lua_bin(): + return _resolve("lua") + + +def ensure(*names): + """Resolve each of the given names; raise if any is missing. + + Meant to be called at test load with the set of interpreters the + test tree actually needs, so the user gets a clear error before + execution starts instead of deep inside an engine spawn. + """ + missing = [] + for n in names: + if not _resolve(n): + display, gd_key, candidates, _ = _SPECS[n] + missing.append( + f" - {display}: tried {candidates} on PATH, none usable. " + f"Set '{gd_key}' in the YAML config to override." + ) + if missing: + raise ETUMRuntimeError( + "Required external interpreter(s) not found:\n" + "\n".join(missing) + ) + + +def reset(): + _resolved.clear() diff --git a/src/testium/interpreter/utils/lua_func_exec.py b/src/testium/interpreter/utils/lua_func_exec.py index c6c58c6..f2d90d1 100644 --- a/src/testium/interpreter/utils/lua_func_exec.py +++ b/src/testium/interpreter/utils/lua_func_exec.py @@ -29,7 +29,7 @@ class LuaFuncExecEngine(LuaProcessBase): # In case an error was encountered in the called function elif "error" in answer: - msg = f"{answer["error"]}" + msg = f"{answer['error']}" return TestValue.FAILURE, msg else: diff --git a/src/testium/interpreter/utils/lua_process.py b/src/testium/interpreter/utils/lua_process.py index 55928bb..e24688a 100644 --- a/src/testium/interpreter/utils/lua_process.py +++ b/src/testium/interpreter/utils/lua_process.py @@ -1,6 +1,5 @@ import os import sys -import shutil import subprocess import socket @@ -8,85 +7,7 @@ import api.testium as tm from runtime.jrpc import JsonRpcClient from interpreter.utils.paths import subproc_path from runtime.tum_except import ETUMRuntimeError -from interpreter.utils.paths import sys_app_path_lin, sys_app_path_win - -def _lua_version(path: str): - cmd = f'"{path}" -v' - try: - result = subprocess.run( - cmd, - shell=True, - capture_output=True, - text=True, - encoding=tm.sys_encoding(), - timeout=10, - ) - # Under windows, the output is on stderr - data = result.stdout or result.stderr - except (FileNotFoundError, PermissionError, subprocess.TimeoutExpired) as e: - data = "" - try: - vers = ((data.split(" "))[1]).split(".") - if len(vers) != 3: - vers = (0, 0, 0) - except: - vers = (0, 0, 0) - return tuple(vers) - - -def _is_lua51(lua_bin): - res = False - v = _lua_version(lua_bin) - if (v[0] == "5") and (v[1] >= "1"): - res = True - return res - - -def _sys_lua_bin(): - sys_lua_bin = tm.gd("_sys_lua_bin", "") - if sys_lua_bin != "": - return sys_lua_bin - - cur_os = tm.OS() - if cur_os == "Windows": - func = sys_app_path_win - else: - func = sys_app_path_lin - - sys_lua_bin = func("lua") - if (sys_lua_bin != "") and not _is_lua51(sys_lua_bin): - tm.print_debug(f"'{sys_lua_bin}' not a lua 5.1 min.") - sys_lua_bin = "" - - tm.print_debug(f"lua bin is: '{sys_lua_bin}'.") - tm.setgd("_sys_lua_bin", sys_lua_bin) - return sys_lua_bin - - -def _is_lua_interpreter(path: str, timeout=2) -> bool: - """ - Checks if the given path points to a valid Lua interpreter. - - Args: - path (str): Path to the executable to check. - timeout (int, optional): Timeout for the subprocess in seconds. Defaults to 2. - - Returns: - bool: True if the path is a Lua interpreter, False otherwise. - """ - try: - result = subprocess.run( - [path, "-v"], - stdout=subprocess.PIPE, - stderr=subprocess.PIPE, - text=True, - timeout=timeout, - ) - return (result.returncode == 0) and ( - (result.stdout.startswith("Lua") or result.stderr.startswith("Lua")) - ) - except (FileNotFoundError, PermissionError, subprocess.TimeoutExpired): - return False +from interpreter.utils import bins class LuaProcessBase: @@ -96,35 +17,15 @@ class LuaProcessBase: "LUA_CPATH": {"replace": True}, } - def __init__(self, lua_bin="", request_handler=None, timeout=10): - """ - Initializes the Lua function execution engine. - - Args: - lua_bin (str, optional): Path to the Lua interpreter. Defaults to system path. - request_handler: Handler for JSON-RPC requests. - timeout (int, optional): Timeout for operations in seconds. Defaults to 10. + def __init__(self, request_handler=None, timeout=10): + """Initializes the Lua function execution engine. Raises: - ETUMRuntimeError: If the Lua path is invalid or no interpreter is found. + ETUMRuntimeError: If no Lua >= 5.1 interpreter is found. """ - if lua_bin != "": - if shutil.which(lua_bin) is None: - raise ETUMRuntimeError( - f"The passed lua path is not pointing to an executable: '{lua_bin}'" - ) - - if not _is_lua_interpreter(lua_bin): - raise ETUMRuntimeError( - f"The passed executable is not a lua interpreter: '{lua_bin}'" - ) - else: - lua_bin = _sys_lua_bin() - if lua_bin == "": - raise ETUMRuntimeError(f"No valid lua interpreter found") - tm.setgd("lua_bin", lua_bin) - - self._lbin = lua_bin + self._lbin = bins.lua_bin() + if not self._lbin: + raise ETUMRuntimeError("No valid Lua 5.1+ interpreter found") self._req_handler = request_handler self._process = None self._port = 0 diff --git a/src/testium/interpreter/utils/py_eval.py b/src/testium/interpreter/utils/py_eval.py index def3ebe..9d9275c 100644 --- a/src/testium/interpreter/utils/py_eval.py +++ b/src/testium/interpreter/utils/py_eval.py @@ -6,9 +6,9 @@ import api.testium as tm eval_process = None -def eval_process_init(python_bin, request_handler, timeout, python_path): +def eval_process_init(request_handler, timeout, python_path): global eval_process - eval_process = EvalExecEngine(python_bin, request_handler, timeout, python_path) + eval_process = EvalExecEngine(request_handler, timeout, python_path) return eval_process diff --git a/src/testium/interpreter/utils/py_func_exec.py b/src/testium/interpreter/utils/py_func_exec.py index 6f64b7b..efc8fe6 100644 --- a/src/testium/interpreter/utils/py_func_exec.py +++ b/src/testium/interpreter/utils/py_func_exec.py @@ -29,7 +29,7 @@ class PyFuncExecEngine(PyProcessBase): # In case an error was encountered in the called function elif "error" in answer: - msg = f"{answer["error"]}" + msg = f"{answer['error']}" return TestValue.FAILURE, msg else: diff --git a/src/testium/interpreter/utils/py_process.py b/src/testium/interpreter/utils/py_process.py index 1703368..9383945 100644 --- a/src/testium/interpreter/utils/py_process.py +++ b/src/testium/interpreter/utils/py_process.py @@ -1,77 +1,12 @@ import os -import shutil import sys import subprocess import socket from runtime.jrpc import JsonRpcClient import api.testium as tm -from interpreter.utils.paths import sys_app_path_lin, sys_app_path_win from runtime.tum_except import ETUMRuntimeError from interpreter.utils.paths import testium_path, subproc_path - - -def _python_version(path: str): - cmd = f'"{path}" -c "import sys; print(sys.version_info[:3])"' - try: - result = subprocess.run( - cmd, - shell=True, - capture_output=True, - text=True, - encoding=tm.sys_encoding(), - timeout=10, - ) - data = result.stdout - except (FileNotFoundError, PermissionError, subprocess.TimeoutExpired) as e: - tm.print_debug(str(e)) - data = "" - return eval(data) - - -def _is_python3(python_bin): - try: - v = _python_version(python_bin) - if v[0] == 3: - res = True - except: - res = False - - return res - - -def _is_python_interpreter(path: str, timeout=2) -> bool: - try: - result = subprocess.run( - [path, "-c", "import sys; print(sys.executable)"], - stdout=subprocess.PIPE, - stderr=subprocess.PIPE, - text=True, - timeout=timeout, - ) - return result.returncode == 0 - except (FileNotFoundError, PermissionError, subprocess.TimeoutExpired): - return False - - -def _sys_python_bin(): - sys_python_bin = "" - - cur_os = tm.OS() - if cur_os == "Windows": - func = sys_app_path_win - else: - func = sys_app_path_lin - - exe = ["python3", "python"] - for e in exe: - sys_python_bin = func(e) - if sys_python_bin == "": - continue - if _is_python3(sys_python_bin): - break - sys_python_bin = "" - - return sys_python_bin +from interpreter.utils import bins class PyProcessBase: @@ -80,29 +15,10 @@ class PyProcessBase: "PYTHONPATH": {"replace": True}, } - def __init__(self, python_bin="", request_handler=None, timeout=10, python_path=""): - self._pbin = python_bin - if (self._pbin is not None) and (self._pbin != ""): - - if shutil.which(self._pbin) is None: - raise ETUMRuntimeError( - f"The passed python path is not pointing to an executable: '{self._pbin}'" - ) - - if not _is_python_interpreter(self._pbin): - raise ETUMRuntimeError( - f"The passed executable is not a python interpreter: '{self._pbin}'" - ) - - else: - self._pbin = tm.gd("_cached_python_bin", "") - if self._pbin == "": - self._pbin = _sys_python_bin() - tm.setgd("_cached_python_bin", self._pbin) - - if self._pbin == "": - raise ETUMRuntimeError(f"No valid python interpreter found") - + def __init__(self, request_handler=None, timeout=10, python_path=""): + self._pbin = bins.python_bin() + if not self._pbin: + raise ETUMRuntimeError("No valid Python 3 interpreter found") self._ppath = python_path self._req_handler = request_handler self._process = None diff --git a/src/testium/interpreter/utils/test_ctrl.py b/src/testium/interpreter/utils/test_ctrl.py index 6896db8..1b5064d 100644 --- a/src/testium/interpreter/utils/test_ctrl.py +++ b/src/testium/interpreter/utils/test_ctrl.py @@ -25,12 +25,17 @@ class TestSetController: if "timeout" in args: timeout = args.pop("timeout") self._test_ctrl.put({cmd: args}) - res = self._test_resp.get(block, timeout) - if isinstance(res, tuple): - raise ETUMRuntimeError(f"Test set command '{cmd}' failed: '{res[1]}'") - if isinstance(res, dict) and not cmd in res.keys(): - raise ETUMRuntimeError(f"Unexpected return error in test set controller") - return res[cmd] + # Drain stale responses (left over from earlier polled commands that + # we had given up on waiting). They can land in the queue after our + # clear() because the TestProcess may have pulled their request + # before the clear, processed them, and pushed the response after. + while True: + res = self._test_resp.get(block, timeout) + if isinstance(res, tuple): + raise ETUMRuntimeError(f"Test set command '{cmd}' failed: '{res[1]}'") + if isinstance(res, dict) and cmd in res.keys(): + return res[cmd] + # Anything else is a stale response — discard and keep waiting. def clear(self): while True: diff --git a/src/testium/py_func/handle.py b/src/testium/py_func/handle.py index 9108223..c6f45f6 100644 --- a/src/testium/py_func/handle.py +++ b/src/testium/py_func/handle.py @@ -41,7 +41,7 @@ class FuncHandler(JsonRpcSrv): except Exception as e: tb = traceback.format_exc() return { - "error": f"bad jrpc req handler 'func_call' arguments ({"\n".join(tb.splitlines())}). To be reported to testium support team." + "error": "bad jrpc req handler 'func_call' arguments (" + "\n".join(tb.splitlines()) + "). To be reported to testium support team." } if method == "eval": try: @@ -57,7 +57,7 @@ class FuncHandler(JsonRpcSrv): except Exception as e: tb = traceback.format_exc() return { - "error": f"bad jrpc req handler 'eval' arguments ({"\n".join(tb.splitlines())}). To be reported to testium support team." + "error": "bad jrpc req handler 'eval' arguments (" + "\n".join(tb.splitlines()) + "). To be reported to testium support team." } else: return { diff --git a/src/testium/py_func/tm.py b/src/testium/py_func/tm.py index cdfea6d..cb86570 100644 --- a/src/testium/py_func/tm.py +++ b/src/testium/py_func/tm.py @@ -28,7 +28,7 @@ def _make_api(name): if "result" in res: ret_val = res["result"] elif "error" in res: - raise ETUMRuntimeError(f"api call to 'tm.{name}' failed with error '{res["error"]}'") + raise ETUMRuntimeError(f"api call to 'tm.{name}' failed with error '{res['error']}'") else: raise ETUMRuntimeError("api call failure in jrpc client to be reported to testium support team.") return ret_val diff --git a/src/testium/runtime/api.py b/src/testium/runtime/api.py index 92f9acd..7b630c9 100644 --- a/src/testium/runtime/api.py +++ b/src/testium/runtime/api.py @@ -6,5 +6,10 @@ SUPPORTED_API = [ "add_plot_values", "last_plot_value", "text_mode", + "OS", + "get_main_dir", + "init_timestamp", + "timestamp", + "timestamp_as_sec", ] diff --git a/src/testium/runtime/jrpc.py b/src/testium/runtime/jrpc.py index b3b45d8..1fd2580 100644 --- a/src/testium/runtime/jrpc.py +++ b/src/testium/runtime/jrpc.py @@ -145,7 +145,7 @@ class JsonRpcConnection: self.pending[msg["id"]]["response"] = msg self.pending[msg["id"]]["event"].set() else: - self.print_info(f"msg id '{msg["id"]}' inconsistency") + self.print_info(f"msg id '{msg['id']}' inconsistency") # ---------- Handler ---------- def _handle_request(self, meth, params, rid=None): diff --git a/test/validation/items/common/helper_lib.py b/test/validation/items/common/helper_lib.py index 197d97c..b9e7ff7 100644 --- a/test/validation/items/common/helper_lib.py +++ b/test/validation/items/common/helper_lib.py @@ -1,4 +1,4 @@ -import api.testium as libtm +import py_func.tm as libtm def check_os(expected_os): diff --git a/test/validation/items/parallel/parallel.py b/test/validation/items/parallel/parallel.py index f8a7527..8a3a57e 100644 --- a/test/validation/items/parallel/parallel.py +++ b/test/validation/items/parallel/parallel.py @@ -1,5 +1,5 @@ import time -import api.testium as tm +import py_func.tm as tm def sleep_func(duration): diff --git a/test/validation/param.yaml b/test/validation/param.yaml index f2cf096..9dd8abf 100644 --- a/test/validation/param.yaml +++ b/test/validation/param.yaml @@ -30,17 +30,10 @@ linux_prompt: "$ " inc_no_template: "inc no template" inc_with_template: "inc with template" -lua_rev: 5.5 - -LUA_PATH_Linux: /usr/share/lua/$(lua_rev)/?.lua;/usr/local/share/lua/$(lua_rev)/?.lua;/usr/local/share/lua/$(lua_rev)/?/init.lua;/usr/share/lua/$(lua_rev)/?/init.lua;/usr/local/lib/lua/$(lua_rev)/?.lua;/usr/local/lib/lua/$(lua_rev)/?/init.lua;/usr/lib/lua/$(lua_rev)/?.lua;/usr/lib/lua/$(lua_rev)/?/init.lua;./?.lua;./?/init.lua;/home/francois/.luarocks/share/lua/$(lua_rev)/?.lua;/home/francois/.luarocks/share/lua/$(lua_rev)/?/init.lua -LUA_CPATH_Linux: /usr/local/lib/lua/$(lua_rev)/?.so;/usr/lib/lua/$(lua_rev)/?.so;/usr/local/lib/lua/$(lua_rev)/loadall.so;/usr/lib/lua/$(lua_rev)/loadall.so;./?.so;/home/francois/.luarocks/lib/lua/$(lua_rev)/?.so -PATH_Linux: - -LUA_PATH_Windows: ;.\?.lua;C:\Lua\5.1\lua\?.lua;C:\Lua\5.1\lua\?\init.lua;C:\Lua\5.1\?.lua;C:\Lua\5.1\?\init.lua;C:\Lua\5.1\lua\?.luac -LUA_CPATH_Windows: .\?.dll;C:\Lua\5.1\?.dll;C:\Lua\5.1\loadall.dll;C:\Lua\5.1\clibs\?.dll;C:\Lua\5.1\clibs\loadall.dll;.\?51.dll;C:\Lua\5.1\?51.dll;C:\Lua\5.1\clibs\?51.dll -PATH_Windows: "" - -lua_env: - PATH: $(PATH_$(os)) - LUA_PATH: $(LUA_PATH_$(os)) - LUA_CPATH: $(LUA_CPATH_$(os)) \ No newline at end of file +# LUA_PATH / LUA_CPATH are intentionally NOT set: the lua interpreter's +# compiled-in defaults already point to the version-matching directories +# (/usr/share/lua/X.Y, /usr/lib/.../lua/X.Y) where the system packages +# lua-cjson and lua-socket install their files. Hard-coding a `lua_rev` +# here would break as soon as the host's lua differs from that value +# (which is exactly what happened on Debian Bookworm with lua5.5 +# vs. lua-cjson built for lua5.4). \ No newline at end of file