11 Commits

Author SHA1 Message Date
358ade8c98 Inc version 2026-05-05 09:21:43 +02:00
46bdb44cfb Route py_func/lua_func subprocess stdio into the parent log
stdout/stderr of the subprocesses were going to DEVNULL — early-startup
errors (lua require failures, exceptions before stdio_redir kicks in)
were lost.

New helper proc_drain.drain_to_log spawns a daemon thread per pipe that
print()s each line through stdio_redir, so it reaches the log + live
output. Used by py_process and lua_process with [py_func]/[lua_func]
prefixes.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-05 09:20:53 +02:00
41519c97cb Fix testium --version reporting "unknown" when installed from a wheel
get_testium_version() used pkg_resources (deprecated, slow to import)
and a narrow catch on git.InvalidGitRepositoryError; any other git
exception fell through to the outer except and returned "unknown".

- Use importlib.metadata.version("testium") to read the wheel
  version that setuptools bakes from src/VERSION at build time. Works
  out of any source checkout — pip-installed copies report
  "<x.y> (wheel release)" instead of "unknown".
- Source-checkout path: tried first when prefs.git_supported, broadly
  catches Exception so a missing repo / detached worktree / etc. no
  longer hides the wheel-metadata fallback.
- PyInstaller path: graceful "unknown (binary release)" if the bundled
  VERSION file is unreadable, instead of an unhandled exception.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-05 09:19:22 +02:00
b9475c6e9b docs: refocus README on users, add quick_start + tutorial, fill CONTRIBUTING
- README.md: pruned developer-oriented sections (Sphinx setup, Qt
  Creator workflow, VSCode debugging, release procedure, AppImage
  Wayland note) and replaced them with a user-facing layout: pre-built
  releases pointer, quick start, manual install, troubleshooting,
  licence.
- CONTRIBUTING.md: absorbed the developer content (debugging in VSCode,
  Qt GUI regen, Sphinx build, validation suite — batch + GUI variants,
  cross-distrib check, release procedure).
- doc/quick_start.md: 5-minute path from install to a passing test,
  in batch mode and in the GUI.
- doc/tutorial.md: guided walk-through against a small calc.py
  module — check, py_func, expected_result, $(...) expansion, group,
  let, condition, report (with the mkdir reminder), context_id.
- CLAUDE.md: subprocess API contract, bins.py, report-exporter
  plugin section, packaging matrix (wheel / PyInstaller / Flatpak /
  .deb work-in-progress), refreshed recent-fixes list. README/CLAUDE
  validation command no longer carries the spurious "-l" flag (which
  is GUI-only and a no-op in batch).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-05 09:18:59 +02:00
d3c5bd01e5 lua and python bin detection rationalized: bins.py module created.
Added some api accessible from python and lua sub_processes. Now the tests only access to py_func.tm instead of direct api.testium module access.

Corrected some f"xxx" to allow working with old python (bookworm).

Changed param.yaml of the test to allow lua to work in all situations.

Various other small fixes for frozen app, wheel.

Tested in all situations, and OK. Ready for tag !

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-03 10:16:56 +02:00
077e1a97c1 Update PyInstaller spec for the new package layout
- Drop the now-obsolete src/lib and src/py_func data entries (those
  paths no longer exist)
- Add src/testium/py_func and src/testium/runtime as bundle-root data
  dirs: the py_func subprocess is launched with the *host* Python
  (not the frozen interpreter), so it needs the source files on disk
  at cwd=subproc_path() to find py_func/__main__.py and import from
  runtime.*
- Hidden imports updated: libs.* → api.*, plus py_func.* explicitly
  declared so PyInstaller pulls them into the bundle even though
  they are loaded as data

Smoke-tested: built binary runs `testium -b`, py_func subprocess works.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-02 14:07:48 +02:00
35ca0a8b45 pyinstaller package updated 2026-05-02 09:58:46 +02:00
4529da7aee Restructure: consolidate everything inside testium/ package
Move src/lib/ → src/testium/runtime/ (internal plumbing)
Move src/testium/libs/ → src/testium/api/ (public SDK for test scripts)
Move src/py_func/ → src/testium/py_func/ (Python subprocess)
Move src/lua_func/ → src/testium/lua_func/ (Lua subprocess data)

The package now ships as a single coherent unit instead of four sibling
top-level packages (testium, lib, py_func, lua_func) — pip install
gives a clean site-packages/testium/ with no namespace pollution; .lua
files travel with the wheel via package_data; the wheel installs
cleanly and `testium -b` runs end-to-end including py_func subprocesses
and entry-point exporter plugins.

Naming:
- runtime/ (internal, no API guarantees) clearer than lib/
- api/ (public SDK consumed as `import api.testium as tm`) clearer than libs/

Imports updated en masse: from lib. → from runtime. and from libs. →
from api., plus the importlib.import_module("libs.*") strings in
test_item_console.py and test_item_runtime_plot.py. Test/example
scripts (helper_lib.py, parallel.py, post_execution.py) and the
fake_exporter test suite migrated too.

paths.py: subproc_path() now returns testium_path() — both point at
the testium package directory since the subprocesses live inside.

pyproject.toml: removed exclude=["lua_func", "py_func"] (no longer
needed), added package-data for testium.lua_func/*.lua, removed the
license classifier (PEP 639 conflict with license expression).

Subprocess isolation contract: py_func/ and lua_func/ may only import
runtime/ and their own modules — never interpreter/, main_win/, api/,
or testium/. Enforced by test/validation/items/isolation/ which runs a
py_func that statically scans subprocess source files for forbidden
imports. The contract holds today; the test prevents future drift.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-02 09:28:40 +02:00
8bd9b3e9d6 Add plugin registry for report exporters
Replace the hardcoded if/elif in Export.exec() with a dict registry.
Built-in formats (text, json, junit, html) are registered as lazy
loaders; missing optional deps (junit_xml, lxml) print a clear message
with a pip install hint instead of raising. Entry-points
(group "testium.exporters") are discovered at import time — installed
plugins are auto-detected with no extra config.

An unknown or unavailable format prints an info line and skips the
export; the test run is not interrupted.

Validation:
- New testium-fake-exporter package under test/validation/fake_exporter/
  installed automatically by scripts/build_env.sh on venv creation.
  It registers fake_format via entry-points and exports the tests
  table to CSV — a real, useful exporter that exercises the plugin
  contract end-to-end (entry-point discovery, dispatch, SQLite query).
- New dedicated items/report_plugin/ test exercises both the
  unknown-format skip path and the fake_format plugin path, with a
  py_func check (file_check.py) on the produced CSV. Runs once per
  validation suite.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-01 23:16:10 +02:00
a70b70db54 Rework About dialog: licence, copyright, proper version display
- about_win.ui: QVBoxLayout, version shown in a word-wrap QLabel
  (sized to content, no oversized text area), add labelCopyright
  (© 2025-2026 François Dausseur) and labelLicence (EUPL-1.2 link)
- about_win.py: regenerated from UI
- testium_win.py: set labelVersion from get_testium_version() (branch,
  dirty flag, commit or binary/Flatpak label)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-01 19:30:37 +02:00
d7f25718d0 CLAUDE.md: consolidate recent fixes, fix PASS/FAIL terminology
Merge the two "Recent fixes" sections into one (branches are gone),
add parallel_branch icon, F1 panel, test-tree state, unittest rename,
run item rename, licence. Fix SUCCESS/FAILURE → PASS/FAIL in run item.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-01 19:02:02 +02:00
110 changed files with 1714 additions and 840 deletions

123
CLAUDE.md
View File

@@ -92,11 +92,33 @@ All dialog items (`dialog_image`, `dialog_question`, `dialog_references`, `dialo
- Per-item log capture (`stdio_redir.read()`) is naturally race-free thanks to per-thread buffers (see `StdoutProxy`).
### Thread-aware stdout (`StdoutProxy`)
`src/lib/stdout_redirect.py` — when `log_stored: True`, `intercept()` installs a `StdoutProxy` as `sys.stdout`/`sys.stderr` instead of a single shared `StringQueue`. The proxy:
`src/testium/runtime/stdout_redirect.py` — when `log_stored: True`, `intercept()` installs a `StdoutProxy` as `sys.stdout`/`sys.stderr` instead of a single shared `StringQueue`. The proxy:
- Holds one `StringQueue` per thread (registered via `register_thread(buffer=...)`). The main thread uses a default buffer; each parallel branch's thread registers its own at start and unregisters at end. `stdio_redir.read()` reads the calling thread's buffer → `addTest()` of an item running in branch X reads X's clean, non-interleaved output.
- For the live stream (terminal in batch / GUI panel), prefixes every line emitted from a branch's thread with `[<branch_name>] ` so concurrent branches stay readable.
- Exposes `write` / `writeln` / `flush` (Python 3.14's `unittest` calls `stream.writeln()` directly without `_WritelnDecorator`).
### Subprocess API contract (py_func / lua_func)
User test scripts running inside a `py_func` or `lua_func` subprocess **must** use the JSON-RPC bridge to interact with testium state:
- Python: `import py_func.tm as tm` — auto-generates wrappers for every function in `runtime/api.py:SUPPORTED_API`. `tm.gd`/`tm.setgd`/`tm.delgd` go through JSON-RPC to the parent.
- Lua: `local tm = require("tm")` — same idea on the Lua side.
`api.testium` is the *main-process* implementation; it is **not** exposed to subprocesses by design (not bundled in PyInstaller, not on the subprocess `PYTHONPATH` in pip-installed mode either when isolation is preserved). An import attempt from a subprocess script is a code smell and is detected by `test/validation/items/isolation/`.
To add a new API call usable from subprocesses:
1. Add the function to `api/testium.py`
2. Add its name to `SUPPORTED_API` in `runtime/api.py`
3. It is auto-exposed via JSON-RPC by `interpreter/utils/api_srv.py` and auto-wrapped by `py_func/tm.py:_make_api`
### External interpreter resolution (`bins.py`)
`src/testium/interpreter/utils/bins.py` — single source of truth for the paths to the external Python and Lua interpreters used by subprocesses.
- `python_bin()` / `lua_bin()` : resolve once, cache in memory. User can override via the `python_bin` / `lua_bin` global dict keys (typically populated from the YAML config). Falls back to discovery on PATH (candidates: `python3`/`python` and `lua`/`lua5.5`/`lua5.4`/`lua5.3`/`lua5.2`/`lua5.1`).
- `ensure(*names)` : called by `TestSet._validate_runtime_deps()` at test load. Always requires `python` (the eval engine always runs); requires `lua` only if a `lua_func` item is in the tree. Fails fast with a clear error citing tried candidates and override key.
Engines (`PyProcessBase`, `LuaProcessBase`, `EvalExecEngine`) call `bins.python_bin()`/`bins.lua_bin()` themselves — call sites never pass an explicit binary path.
## Key files
| Path | Role |
@@ -109,9 +131,28 @@ All dialog items (`dialog_image`, `dialog_question`, `dialog_references`, `dialo
| `src/testium/interpreter/test_items/test_item_parallel.py` | `parallel` and `parallel_branch` items |
| `src/testium/interpreter/utils/globdict.py` | Global variable dict |
| `src/testium/interpreter/utils/termlog.py` | Terminal color output |
| `src/lib/stdout_redirect.py` | `StdioRedirect` singleton (`stdio_redir`) |
| `src/lib/string_queue.py` | Thread-safe string buffer used for stdout redirection |
| `src/testium/libs/testium.py` | Public API for test scripts (`tm.*`) |
| `src/testium/runtime/stdout_redirect.py` | `StdioRedirect` singleton (`stdio_redir`) |
| `src/testium/runtime/string_queue.py` | Thread-safe string buffer used for stdout redirection |
| `src/testium/api/testium.py` | Public API for test scripts (`tm.*`) |
| `src/testium/py_func/` | Python subprocess for `py_func` items (sandboxed: imports only `runtime/` and `py_func/`) |
| `src/testium/lua_func/` | Lua subprocess scripts for `lua_func` items |
## Package layout
The whole project is a single Python package under `src/testium/`:
```
src/testium/
├── __init__.py / __main__.py
├── runtime/ internal plumbing (jrpc, stdout_redirect, string_queue, tum_except, api)
├── api/ public SDK exposed to test scripts (`import api.testium as tm`)
├── interpreter/ test execution engine (NOT visible to py_func/lua_func)
├── main_win/ GUI (NOT visible to py_func/lua_func)
├── py_func/ subprocess code for python_func items
└── lua_func/ subprocess scripts for lua_func items (data files)
```
`subproc_path()` and `testium_path()` both return the package directory. The py_func subprocess is launched with cwd=that directory and `python3 py_func`. The contract that `py_func/` and `lua_func/` only depend on `runtime/` (no `interpreter`, `main_win`, `api`, `testium`) is enforced by `test/validation/items/isolation/`.
## GUI icons (main_win)
@@ -137,31 +178,71 @@ Icons are assigned once when the test file is loaded (not updated live on theme
### `run` item
`src/testium/interpreter/test_items/test_item_run.py` — launches a `.tum` file in a new testium instance (`-b` in batch mode, `-r` in GUI mode). Result:
- **SUCCESS** if the sub-instance launched and ran to completion (exit code is ignored)
- **FAILURE** if the file is not found, `wait_for_exec` is set without `start_time`/`end_time`, the time window was not reached, or any other launch error
- **PASS** if the sub-instance launched and ran to completion (exit code is ignored)
- **FAIL** if the file is not found, `wait_for_exec` is set without `start_time`/`end_time`, the time window was not reached, or any other launch error
The sub-test's own pass/fail result is intentionally not propagated.
## Recent fixes (branch `parallel_execution`)
- `test_item_parallel.py`: new `parallel` item with `sync: all|any`, `wait_for`, daemon threads, `_stop_branch_recursively()`. Each branch thread registers a per-thread stdout buffer with `stdio_redir.register_thread(...)` so its log capture and live-output prefix work in isolation.
- `test_item_container.py`: new `TestItemContainer` base class extracted from Group/Cycle patterns
- `test_item_sleep.py`: interruptible loop (checks `self._is_stopped`) instead of blocking `time.sleep()` so `sync: any` can stop slow branches quickly
- `stdout_redirect.py`: rewrote `intercept()` to install a `StdoutProxy` (thread-aware: per-thread capture buffers + branch-prefixed live output). Adds `writeln()` for Python 3.14 unittest compatibility.
- `test_report.py`: `check_same_thread=False` + lock around the SQLite `INSERT` for parallel branch concurrency. Log capture itself is race-free thanks to per-thread buffers.
- `__init__.py`: removed `-m`/`--terminal` mode
- `terminal.py`: deleted
### Report exporters & plugins
`src/testium/interpreter/test_report/test_report.py``_EXPORTER_REGISTRY` dict maps a format name (cmd key in the YAML `report.export`) to a lazy loader. Built-ins: `text`, `json`, `junit` (needs `junit_xml`), `html` (needs `lxml`). `sqlite` is the storage layer, no-op as an export.
## Recent fixes (branch `text_no_pyside`)
- `batch.py`: premature loop exit when `gd_update` messages (no `"id"` key) were mistaken for the "finished" signal — fix: `"id" in m and m["id"] is None`
- `batch.py`: `control("loaded")` deadlock if `TestProcess` crashed before `cmd_th` started — fix: daemon thread + `threading.Event` + `is_alive()` polling
- `termlog.py`: `COLOR_DEFAULT = Fore.WHITE` invisible on light terminals; added auto-detection + light palette. Also fixed `write()` residue accumulation bug (`s[pos:]``s[pos+1:]`).
- Dialog items: `auto_result`/`auto_value` now used in non-interactive text mode; dialogs without `auto_result` FAIL immediately in batch mode.
- `run` item: removed `stdout=PIPE` (caused deadlock with `multiprocessing` spawn); simplified result to SUCCESS on any completed subprocess.
Third-party plugins are discovered at module import via `importlib.metadata.entry_points(group="testium.exporters")` — installing a wheel that declares such an entry point is enough, no testium config change needed:
```toml
[project.entry-points."testium.exporters"]
my_format = "my_pkg:MyExporter"
```
Exporter contract: `__init__(self, name, con, path, pats, keys, no_header=False)` — the class does its work in `__init__` and writes to `path`.
Behaviour on errors:
- Unknown format → info line `[report] Export skipped: format "X" not found. Available: ...`, run continues.
- Optional dependency missing → same info line with a pip-install hint, run continues.
A real-world test plugin lives at `test/validation/fake_exporter/` (CSV exporter, auto-installed by `scripts/build_env.sh` and exercised by `test/validation/items/report_plugin/`).
## Packaging
Three distribution channels coexist, sharing the single `src/testium/` package:
| Channel | Where | Notes |
|---------|-------|-------|
| Wheel (`pip install`) | `src/pyproject.toml` | Vanilla Python package; entry point `testium = "testium:main"` |
| PyInstaller binary | `package/pyinstaller/` | Single ~130 MB binary. `py_func`, `runtime`, `lua_func` bundled at `_MEIPASS` root so the **host** Python can find them when launched as `python3 py_func`. `api`/`interpreter` are **not** exposed (subprocess isolation). |
| Flatpak | `package/flatpak/` | (Existing recipe, not actively maintained in current refactor wave.) |
The `.deb` work-in-progress lives in `package/deb/`:
- `test_distro.sh debian:bookworm | debian:trixie | ubuntu:24.04` spins up a Docker/Podman container, reports system package availability, falls back to pip for what's missing (`pyside6` on bookworm/ubuntu, `telnetlib3`, `junit_xml`), runs the validation suite. Currently green on the three targets.
## Recent fixes / notable changes
- Restructure: single `src/testium/` Python package (was 4 sibling top-levels: `testium`, `lib`, `py_func`, `lua_func`). `lib/``runtime/`, `libs/``api/`. `pip install` now produces a clean `site-packages/testium/` with no top-level pollution; `.lua` files travel via `package_data`.
- `bins.py`: centralised resolution + cache of external `python3` / `lua` binaries. Replaces the scattered `tm.gd("python_bin")`/`tm.gd("lua_bin")` dance and the duplicated discovery logic in `py_process.py`/`lua_process.py`. Validates at test load via `TestSet._validate_runtime_deps()` so missing interpreters fail fast.
- Subprocess API contract: user scripts in `py_func`/`lua_func` use the JSON-RPC bridge (`py_func.tm` / Lua `tm`) — never `api.testium` / `interpreter.*` directly. `SUPPORTED_API` extended with `OS`, `get_main_dir`, `init_timestamp`, `timestamp`, `timestamp_as_sec` so subprocess scripts have the same surface as main-process code.
- Report exporter plugin registry (`test_report.py`): `_EXPORTER_REGISTRY` + `entry_points("testium.exporters")` discovery. Missing format → info line, run continues.
- About dialog rework: `QVBoxLayout` (resizable), version + dirty/branch info in a `QLabel` (auto-sized), copyright + clickable EUPL-1.2 link.
- `test_ctrl.control()`: drain stale responses (left over from polled `loaded()` after `clear()` race) instead of failing on a wrong cmd key — fixes a "Unexpected return error in test set controller" seen in GUI mode after a fast reload.
- `lua_process.py`: stderr no longer DEVNULL'd so actual Lua errors (missing `cjson`/`socket`) surface instead of "Connection refused".
- `run_post_exec`: failure message uses `print_warn` (was `print_debug` — silent in non-debug runs).
- Python 3.11 compat: replaced PEP 701 nested-quote f-strings (e.g. `f"... {d["k"]} ..."`) with single-quote inner strings or string concatenation.
- `parallel` item: new item with `sync: all|any`, `wait_for`, daemon threads, `_stop_branch_recursively()`. Each branch thread registers a per-thread stdout buffer.
- `parallel_branch` icon: distinct single-arrow icon (`parallel_branch.png`).
- `parallel` F1 panel: `steps` stripped from each branch dict.
- `test_item_container.py`: shared base class extracted from Group/Cycle.
- `test_item_sleep.py`: interruptible loop so `sync: any` can stop slow branches quickly.
- `stdout_redirect.py`: `StdoutProxy` (thread-aware buffers + branch-prefixed live output, `writeln()` for Python 3.14 unittest).
- `test_report.py`: thread-safe SQLite INSERT for parallel branch concurrency.
- `terminal.py`: deleted — `-m`/`--terminal` mode removed.
- `batch.py`: premature finish bug on `gd_update` (no `"id"` key) — fix uses `"id" in m and m["id"] is None`.
- `batch.py`: `control("loaded")` deadlock on TestProcess crash — fix uses daemon thread + `threading.Event` + `is_alive()` polling.
- `termlog.py`: light/dark terminal auto-detection (`COLORFGBG`, OSC 11) + write residue bug.
- Dialog items: `auto_result`/`auto_value` for non-interactive text mode; dialogs without `auto_result` FAIL immediately in batch.
- `run` item: renamed `tum_fime``tum`; removed `stdout=PIPE` deadlock; PASS on any completed subprocess.
- `unittest` item: renamed from `unittest_file`.
- GUI test tree: check and fold state preserved across same-file reloads.
- Licence: EUPL-1.2.
## Validation tests
Located in `test/validation/`. Run with `-b` flag:
```
./run.sh -b -l mon_log.log -- test/validation/main.tum
./run.sh -b -- test/validation/main.tum
```
Parallel item tests: `test/validation/items/parallel/test.tum`

View File

@@ -45,7 +45,7 @@ For existing files, keep the header that is already there.
3. Commit with a clear message (one logical change per commit).
4. Make sure the validation suite still passes:
```
./run.sh -b -l mon_log.log -- test/validation/main.tum
./run.sh -b -- test/validation/main.tum
```
5. Open a pull request against `main`.
@@ -56,6 +56,105 @@ For existing files, keep the header that is already there.
- Add or update tests in `test/validation/` for new test items or behaviours
- Update `CLAUDE.md` and the Sphinx manual for user-visible changes
## Development
### Debugging in VSCode
The recommended workflow:
1. Add a debug configuration to `.vscode/launch.json`:
```json
{
"configurations": [
{
"name": "Python : testium",
"type": "python",
"request": "launch",
"program": "${workspaceFolder}/src/testium",
"console": "integratedTerminal",
"args": ["-g"],
"justMyCode": true
}
]
}
```
2. Install `debugpy` in the venv: `python -m pip install debugpy`.
3. Open the *Run and Debug* tab and press play. testium starts; load and
run a `.tum` file. Set breakpoints where you want to investigate.
### Qt GUI modification
UI files (`*.ui`) are edited in **Qt Creator**. After editing, regenerate
the corresponding Python and resource files:
```sh
scripts/qt_generate.sh
```
Icons come from <https://github.com/free-icons/free-icons>.
### Sphinx documentation
```sh
pip install sphinx linuxdoc
doc/manual/sphinx/build_doc.sh
```
PDF generation requires `texlive`:
```sh
sudo apt install texlive-full
```
### Validation suite
Batch mode (CI-friendly, headless):
```sh
./run.sh -b -- test/validation/main.tum
```
GUI mode (loads the suite, click *Run* to execute and inspect the tree):
```sh
./run.sh test/validation/main.tum
```
GUI run-and-close (executes the suite, then closes):
```sh
./run.sh -r -- test/validation/main.tum
```
Subset run via the `items` define (works in any mode):
```sh
./run.sh -b -d "items=['parallel','common']" -- test/validation/main.tum
```
### Cross-distribution check
`package/deb/test_distro.sh` spins up a Docker/Podman container of the
target image, installs the expected system Python deps via apt (with
pip fallback for what is missing), installs the testium wheel and runs
the validation suite end-to-end. Currently green on `debian:bookworm`,
`debian:trixie`, `ubuntu:24.04`.
```sh
./package/deb/test_distro.sh debian:trixie
```
## Release procedure
1. Update `release_note.txt`.
2. Bump the version in `src/VERSION`.
3. Make sure the documentation is up to date — rebuild with
`doc/manual/sphinx/build_doc.sh` if needed.
4. Push and tag the commit with the new version.
5. Build the binary release: `package/pyinstaller/build.sh`.
6. Run the validation suite against each generated binary.
7. Confirm all validation results are green before publishing.
## Reporting security issues
Please do **not** report security vulnerabilities through public GitHub

227
README.md
View File

@@ -1,185 +1,110 @@
# Documentation
# testium
[See here](doc/manual/testium_manual.pdf).
testium is a YAML-driven test sequencer for hardware-in-the-loop and
integration testing. A test campaign is described in a `.tum` file as a tree
of items (checks, console interactions, Python/Lua functions, parallel blocks,
dialogs, …); testium executes the tree, captures results, and produces
reports in several formats.
# License
## Documentation
Copyright (c) 2025-2026 François Dausseur.
* [Quick start](doc/quick_start.md) — install and run your first test in
five minutes.
* [Tutorial](doc/tutorial.md) — guided walk-through of the most common
test items with a runnable example.
* [User manual (PDF)](doc/manual/testium_manual.pdf) — full reference.
* [`doc/examples/`](doc/examples/) — runnable `.tum` snippets.
testium is distributed under the **European Union Public Licence v. 1.2
(EUPL-1.2)** — see the [LICENSE](LICENSE) file for the full text.
## Pre-built releases
SPDX identifier: `EUPL-1.2`
Pre-built artifacts are published at
<https://git.beafrancois.fr/v-and-v/testium/releases>:
Contributions are accepted under the same licence (inbound = outbound). See
[CONTRIBUTING.md](CONTRIBUTING.md) for details.
* **Python wheel** (`testium-<version>-py3-none-any.whl`) — install with
`pip install testium-*.whl`. Lighter than the binary; pulls Python
dependencies from PyPI on install.
* **Self-contained Linux binary** (`testium`, built with PyInstaller) —
runnable directly, no Python installation required on the host. Lua
support still needs a system `lua` interpreter and the `lua-socket` /
`lua-cjson` modules.
* **Flatpak** — *coming soon.*
# run testium
## Quick start
From the root path, on windows `cmd`:
From a checkout of the repository:
run.bat
| OS | Command |
|----|---------|
| Linux | `./run.sh` |
| Windows (cmd) | `run.bat` |
| Windows (PowerShell) | `run.ps1` |
On windows powershell:
The wrapper creates a Python virtual environment on first run and starts
testium in GUI mode. Add `-b path/to/test.tum` to run a test in batch mode.
run.ps1
## Manual installation
On linux:
If the wrapper script does not fit your environment, set up testium manually:
./run.sh
The virtual environment is created if needed and *testium* is started.
# Manual setup
A python virtual environment should be created:
python3 -m venv <testium_venv>
## Requirements
In the virtual environment, the following modules must be installed:
* pyside6
* pyserial
* pyyaml
* pexpect
* gitpython
* jinja2
* colorama
* matplotlib
* junit-xml
* lxml
A `requirements.txt` file is also available in the git repository in the path `testium/src/`.
## run testium
from the testium path, execute
python3 -m src/testium
# Doc generation
## Install sphinx
pip install sphinx linuxdoc
## Generate the doc
Execute
doc/manual/sphinx/./build_doc.sh
This command works if texlive package has been installed on the system. It can be done by invoking the following command.
sudo apt install texlive-full
# QT GUI
## QT GUI modification
Open the ".ui" file with `qtcreator` and modify the gui. Then regenerate the python code.
On linux, a helper script has been created:
scripts/./qt_generate.sh
# Debugging
In order to debug testium or your python script executed within testium.
## In VSCODE
This is the prefered method :
1. Create a debug configuration like the following:
```
"configurations": [
{
"name": "Python : testium",
"type": "python",
"request": "launch",
"program": "${workspaceFolder}/src/testium",
"console": "integratedTerminal",
"args": ["-g"],
"justMyCode": true
},
]
```sh
python3 -m venv .venv
source .venv/bin/activate
pip install -r src/requirements.txt
```
2. Install debugpy module in python
Required Python packages (see `src/requirements.txt`):
`pyside6`, `pyserial`, `pyyaml`, `pexpect`, `gitpython`, `jinja2`, `colorama`,
`matplotlib`, `junit-xml`, `lxml`.
python -m pip install debugpy
3. Then get to the "RUN AND DEBUG" tab and press the play button.
4. A testium window will pops up ; start execution of your tum.
5. Do not forget to put breakpoints where you want to investigate.
For tests using `lua_func` items, install Lua (>= 5.1) plus the `socket` and
`cjson` modules. On Debian/Ubuntu:
## Icons
```sh
sudo apt install lua5.4 lua-socket lua-cjson
```
Icons are coming from the following site: https://github.com/free-icons/free-icons.git
Run testium:
# testium Release
```sh
python3 src/testium # GUI
python3 src/testium -b mytest.tum # batch
```
## Pre-requisite
## Troubleshooting
A `python` virtual environment must have been set as described above.
### `wl_proxy_marshal_flags` symbol error
### Install pyinstaller
```
testium: symbol lookup error: ... undefined symbol: wl_proxy_marshal_flags
```
Install `pyinstaller` package using pip.
## Generate the binary package
The procedure for a binary release is as follows:
1. update the `release_note.txt` file
2. modify the version in `src/VERSION` file
3. be sure that the documentation is up to date, and if not execute `doc/manual/sphinx/build_doc.sh` script
4. push modifications and create a tag with the new version on the git repository
5. generate an executable file by calling `package/pyinstaller/./build.sh`
6. run the complete validation test for each generated binary
7. check that all the validation results are OK
# Troubleshooting
## The testium exe crashes `wl_proxy_marshal_flags`
### Error message
/testium: symbol lookup error: /tmp/_MEIOhDCPF/libQt6WaylandClient.so.6: undefined symbol: wl_proxy_marshal_flags
### Solution
Set the appropriate environment variable
Force the X11 Qt backend:
```sh
export QT_QPA_PLATFORM=xcb
testium
```
## xcb plugin missing
### `xcb plugin missing`
### Error message
```
qt.qpa.plugin: Could not load the Qt platform plugin "xcb"
```
qt.qpa.plugin: Could not load the Qt platform plugin "xcb" in "" even though it was found.
Install the missing system libraries:
### Solution
```sh
sudo apt install libxcb-cursor0 libicu-dev libxcb-cursor-dev
```
A package is missing
## License
sudo apt install libxcb-cursor0
sudo apt-get install libicu-dev
sudo apt-get install libxcb-cursor-dev
Copyright © 2025-2026 François Dausseur.
## The testium appimage crashes when opening a file
testium is distributed under the **European Union Public Licence v. 1.2
(EUPL-1.2)** — see [`LICENSE`](LICENSE) for the full text. SPDX:
`EUPL-1.2`.
This is usually because wayland is defined as the default X server.
To change it :
* Disable Wayland by uncommenting WaylandEnable=false in the `/etc/gdm3/daemon.conf`
* Add `QT_QPA_PLATFORM=xcb` in `/etc/environment`
* After a reboot, check that the environment variable value returns `x11`:
$ echo $XDG_SESSION_TYPE
x11
Contributions are accepted under the same licence (inbound = outbound).
See [`CONTRIBUTING.md`](CONTRIBUTING.md) for development setup, debugging
workflow, and the release procedure.

View File

@@ -1,5 +1,5 @@
import libs.testium as tm
import py_func.tm as tm
def post_exec():
print('Success !!!!')

View File

@@ -4,7 +4,12 @@ Python helper library
======================
A python library including helper function for python modules called from
testium.
testium ``py_func`` items.
User scripts run inside the ``py_func`` subprocess and interact with testium
through a JSON-RPC bridge — the ``py_func.tm`` module. They must **not**
import ``api.testium`` or ``interpreter.*`` directly: those are main-process
modules and may not even be reachable in a packaged build (PyInstaller, .deb).
To include the support of this library in a python script, the following
line must be included in the script header:
@@ -18,58 +23,38 @@ line must be included in the script header:
Global variables helper functions
----------------------------------
To manage values in the global variables dataset, the following testium library API
must be used:
To manage values in the global variables dataset:
.. automodule:: py_func.tm
:members: gd, setgd, delgd
:undoc-members:
:no-index:
Console helper functions
------------------------
Every opened console instance is added to a list with the
key ``console_instances`` of the global variables.
The instance is removed from the list on close step of the ``console`` test item.
To manage consoles from within ``py_func`` python functions,
the following testium library API can be used:
.. automodule:: libs.testium
:members: add_console, remove_console, console
:undoc-members:
:no-index:
Plot helper functions
------------------------
Every opened plot window instance is added to a list with the
key ``plot_instances`` of the global variables.
Add values to a running plot or read the last value from it:
The instance is removed from the list on close step of the ``plot`` test item.
To manage plots from within ``py_func`` python functions,
the following testium library API can be used:
.. automodule:: libs.testium
:members: add_plot, remove_plot, plot, add_plot_values, last_plot_value
.. automodule:: py_func.tm
:members: add_plot_values, last_plot_value
:undoc-members:
:no-index:
Console and plot **lifecycle** management (``add_console``, ``remove_console``,
``console``, ``add_plot``, ``remove_plot``, ``plot``) is performed by the
``console`` and ``plot`` test items themselves — not from user ``py_func``
scripts. Use those test items to open/close consoles and plots.
Other helper functions
------------------------
.. automodule:: libs.testium
:members: OS, get_main_dir, timestamp, timestamp_as_sec
.. automodule:: py_func.tm
:members: OS, get_main_dir, init_timestamp, timestamp, timestamp_as_sec, text_mode
:undoc-members:
:no-index:
Debug mode
------------------------
.. automodule:: libs.testium
:members: debug_enabled, enable_debug, print_debug, print_info, print_warn
:undoc-members:
:no-index:
The ``test_debug`` global variable controls debug-only output. Read or write
it via ``tm.gd("test_debug")`` / ``tm.setgd("test_debug", True)``.

View File

@@ -6,18 +6,25 @@ Reports
If a report is required (in addition to the log), the ``report`` YAML element
must be added at the root of the TUM main test file.
The ``report`` YAML element has the following form:
The ``report`` element accepts a single export or a list of them under the
``export`` key. Each export entry uses the format name as its key:
.. code-block:: yaml
:caption: reports global settings
:caption: reports global settings — multiple exports
report:
enabled: True
file_name: $(test_name).rep
log_stored: True
export:
- sqlite:
path: $(home)/reports
pattern: "Console%"
export: junit
log_stored: False
file_name: $(test_name).db
- junit:
path: $(home)/reports
file_name: $(test_name).xml
- html:
path: $(home)/reports
file_name: $(test_name).html
.. table:: report attributes
:widths: 20, 30, 50
@@ -27,21 +34,93 @@ The ``report`` YAML element has the following form:
+-----------------+-----------------------+-------------------------------------------+
| ``enabled`` | ``True`` | Report activated |
+-----------------+-----------------------+-------------------------------------------+
| ``file_name`` | / | Report file name |
| ``log_stored`` | ``False`` | When ``True``, captures stdout per test |
| | | item so exports (html, json) can include |
| | | the log of each item. |
+-----------------+-----------------------+-------------------------------------------+
| ``path`` | ``$(report_path)`` | Report storage path By default, it uses |
| | | the default one set in the |
| | | preferences. |
| ``export`` | / | One export entry or a list of them. Each |
| | | entry's key is the format name (see |
| | | below). |
+-----------------+-----------------------+-------------------------------------------+
| ``pattern`` | / | The pattern in SQL wildachars syntax |
| | | to be applied on test names to |
| | | selected reported tests. |
Each export entry supports the following sub-attributes:
.. table:: export attributes
:widths: 20, 30, 50
+-----------------+-----------------------+-------------------------------------------+
| ``export`` | / | The type of export. For exemple junit. |
| | | By default, the sqlite format is |
| | | used to generate reports. |
| Attribute | default value | Description |
+-----------------+-----------------------+-------------------------------------------+
| ``log_stored`` | / | Defines if the output log of each |
| | | test is accessible to generate the |
| | | report export. |
| ``path`` | ``$(report_path)`` | Output directory. |
+-----------------+-----------------------+-------------------------------------------+
| ``file_name`` | / | Output file name. May include |
| | | ``$(...)`` global-dict expansions. |
+-----------------+-----------------------+-------------------------------------------+
| ``pattern`` | / | One or more SQL ``LIKE`` patterns |
| | | applied on the test ``name``. |
+-----------------+-----------------------+-------------------------------------------+
| ``key`` | / | One or more SQL ``LIKE`` patterns |
| | | applied on the test ``key`` |
| | | (the per-item ``key`` attribute). |
+-----------------+-----------------------+-------------------------------------------+
Built-in formats
^^^^^^^^^^^^^^^^
* ``sqlite`` — raw SQLite database (storage layer; selecting it persists the run).
* ``text`` — simple indented text dump of the test tree.
* ``json`` — full report as JSON: ``{"header": {...}, "tests": [...]}``.
* ``junit`` — JUnit XML (requires the ``junit_xml`` Python package).
* ``html`` — single HTML page with header, results table and per-item logs (requires ``lxml``).
If a format is unknown or its optional dependency is missing, the export is
skipped with an ``[report] Export skipped: ...`` info line on stdout — the
test run is **not** interrupted.
.. _sec_reports_plugins:
Custom export formats (plugins)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
A third-party Python package can register additional export formats via the
``testium.exporters`` setuptools entry point group. Once installed in the same
Python environment as testium, the format is auto-detected at startup and can
be referenced from the YAML by its declared name.
Plugin contract — a class with this constructor signature:
.. code-block:: python
:caption: minimal exporter contract
class MyExporter:
def __init__(self, name, con, path, pats, keys, no_header=False):
# name : str — report name
# con : sqlite3.Connection (read) — tables: header, tests
# path : str — output file path (already expansed)
# pats : list[str] — LIKE filters on test_name (may be empty)
# keys : list[str] — LIKE filters on report_key (may be empty)
# no_header : bool — skip header section (set by the inline
# `report` test item)
... # do the work in __init__ and write to `path`
Tables and columns of the SQLite report:
* ``header(key TEXT, value TEXT)`` — keys: ``report_version``, ``test_file``,
``test_name``, ``test_result``, ``test_revision``, ``testium_version``,
``testrun_date``, ``testrun_time``, ``test_duration``.
* ``tests`` — 12 columns: ``timestamp_start``, ``test_id``, ``parent_id``,
``level``, ``test_name``, ``test_type``, ``report_key``, ``result``
(``PASS``/``FAIL``/``SKIP``), ``message``, ``duration`` (ms),
``log`` (captured stdout when ``log_stored: True``), ``data`` (JSON of
values reported via ``self.reportValue(...)``).
Declaration in the plugin's ``pyproject.toml``:
.. code-block:: toml
:caption: registering an exporter via entry-points
[project.entry-points."testium.exporters"]
my_format = "my_pkg:MyExporter"
The plugin is then usable in any ``.tum`` report block as ``my_format:``
no testium configuration change required.

View File

@@ -13,7 +13,7 @@ class ``py_func`` item
This is the normal way of calling some custom python code.
A class must be defined and derived from ``FunctionItem`` from the ``libs.testium`` module.
A class must be defined and derived from ``FunctionItem`` from the ``py_func.tm`` module.
From this class it is possible to define some custom reported values with the following API

Binary file not shown.

66
doc/quick_start.md Normal file
View File

@@ -0,0 +1,66 @@
# Quick start
Five minutes from zero to a passing test.
## Install
From a checkout of the repository:
```sh
./run.sh --version # Linux
run.bat # Windows cmd
```
The wrapper creates a Python virtual environment on first run and verifies
testium starts. If you prefer a manual install, see the README.
## Your first test
Create `hello.tum`:
```yaml
main:
name: hello world
steps:
- check:
name: 1 + 1 makes 2
values:
- <| 1 + 1 == 2 |>
```
Run it in batch mode:
```sh
./run.sh -b -- hello.tum
```
You should see something like:
```
-----> step "1 + 1 makes 2" started
Check passed
<----- step "1 + 1 makes 2" finished: PASS
Test run success.
```
Replace `==` with `!=` and re-run — the step now ends with **FAIL** and
the process exits with code 1.
## Open it in the GUI
```sh
./run.sh hello.tum
```
The test tree appears in the left panel; click *Run test* in the toolbar.
Each item turns green or red live as it executes. Use `F1` on a selected
item to open its detail panel.
## Where to go next
* [`doc/tutorial.md`](tutorial.md) — a guided walk-through of the most
common test items (`py_func`, `let`, `group`, `condition`, `report`).
* [`doc/examples/`](examples/) — runnable `.tum` snippets covering one
feature each.
* [`doc/manual/testium_manual.pdf`](manual/testium_manual.pdf) —
full reference manual.

223
doc/tutorial.md Normal file
View File

@@ -0,0 +1,223 @@
# Tutorial — testing a small Python utility
This walk-through builds, step by step, a testium campaign that exercises
a small Python module. Each section adds one feature; you can follow
along by editing a single `.tum` file and re-running it.
If you have not yet run testium, start with [`quick_start.md`](quick_start.md).
## The code under test
Create `calc.py` next to your `.tum` file:
```python
def add(a, b):
return a + b
def divide(a, b):
return a / b
```
## Step 1 — a static check
The simplest item is `check`: it evaluates an expression and the test
passes iff the expression is truthy. Create `tutorial.tum`:
```yaml
main:
name: calc.py campaign
steps:
- check:
name: addition is correct
values:
- <| 2 + 3 == 5 |>
```
The `<| ... |>` markers turn the body into a Python expression evaluated
at run time. Run it:
```sh
./run.sh -b -- tutorial.tum
```
## Step 2 — call your code with `py_func`
`check` only sees Python literals; to exercise `calc.py` we need a
`py_func` item. Replace the step:
```yaml
- py_func:
name: add 2 and 3
file: calc.py
func_name: add
param: [2, 3]
expected_result: 5
```
`expected_result` makes the item PASS only when the function returns
exactly that value.
The result is also stored in the global dict under `pfn_<name>`
(here `pfn_add 2 and 3`).
Anywhere in a `.tum`, `$(key)` is replaced at runtime by the value
stored in the global dict under `key`. A subsequent step can read the
result back with `$(pfn_<name>)`:
```yaml
- check:
name: result was 5
values:
- <| $(pfn_add 2 and 3) == 5 |>
```
## Step 3 — group several checks
Wrap the steps in a `group` to keep them visually together and let
testium report a per-group status:
```yaml
main:
name: calc.py campaign
steps:
- group:
name: add
steps:
- py_func:
name: 2 + 3
file: calc.py
func_name: add
param: [2, 3]
expected_result: 5
- py_func:
name: -1 + 1
file: calc.py
func_name: add
param: [-1, 1]
expected_result: 0
- group:
name: divide
steps:
- py_func:
name: 6 / 2
file: calc.py
func_name: divide
param: [6, 2]
expected_result: 3.0
```
A group fails as soon as one of its steps fails (set
`stop_on_failure: false` to keep going).
## Step 4 — define a variable with `let`
Avoid hard-coding the same number twice with a variable:
```yaml
- let:
name: define numerator
values:
- num: 6
- py_func:
name: divide num by 2
file: calc.py
func_name: divide
param:
- $(num)
- 2
expected_result: 3.0
```
`$(num)` expands to the global dict entry — when the stored value is a
number it is substituted as a number, no need to wrap it in `<| ... |>`.
## Step 5 — conditional execution
Skip a step when a condition is false:
```yaml
- py_func:
name: divide by zero only on linux
condition: <| "$(os)" == "Linux" |>
file: calc.py
func_name: divide
param: [1, 0]
```
Items skipped this way report `SKIP` and do not affect the overall
result.
## Step 6 — generate a report
Add a `report` block at the root of the file:
```yaml
main:
name: calc.py campaign
steps:
# ... your steps here ...
report:
enabled: true
log_stored: true
export:
- junit:
path: ./reports
file_name: calc.xml
- html:
path: ./reports
file_name: calc.html
```
The `path` directory must exist before the test runs — testium does not
create it. Create it once:
```sh
mkdir -p reports
```
Re-run the test — `./reports/calc.xml` (CI-friendly) and
`./reports/calc.html` (human-friendly) are produced. Set
`log_stored: true` to include each item's captured stdout.
## Step 7 — share state between calls
By default each `py_func` runs in its own short-lived subprocess.
To keep state across calls, use `context_id`:
```yaml
- py_func:
name: open
file: calc.py
func_name: open_resource
context_id: my_ctx
- py_func:
name: use
file: calc.py
func_name: use_resource
context_id: my_ctx
```
Both steps share the same persistent Python interpreter, so `calc.py`
can store any object in module-level globals or in `tm.setgd()`.
To share data without `context_id`, write it to the testium global dict
via the JSON-RPC bridge:
```python
import py_func.tm as tm
def producer():
tm.setgd("computed", 42)
def consumer():
return tm.gd("computed")
```
## Where to go next
* [`doc/examples/`](examples/) — one runnable `.tum` per feature
(cycles, dialogs, console, plots, parallel, run-of-tum, …).
* [`doc/manual/testium_manual.pdf`](manual/testium_manual.pdf) — full
reference manual covering every test item, every attribute and the
YAML syntax extensions.

141
package/deb/test_distro.sh Executable file
View File

@@ -0,0 +1,141 @@
#!/usr/bin/env bash
# test_distro.sh — verify testium runs on a target Debian/Ubuntu distrib.
#
# Spins up a Docker container of the requested image, checks which expected
# system Python packages are available (apt), installs them, installs the
# testium wheel, and runs a smoke test that exercises batch mode + py_func
# subprocess.
#
# Usage:
# ./test_distro.sh debian:bookworm
# ./test_distro.sh debian:trixie
# ./test_distro.sh ubuntu:24.04
set -euo pipefail
IMAGE="${1:?Usage: $0 <image> e.g. debian:bookworm | debian:trixie | ubuntu:24.04}"
ROOT=$(realpath "$(dirname "$0")/../..")
# Container runtime: prefer docker if available, fall back to podman
if command -v docker >/dev/null 2>&1; then
CTR=docker
elif command -v podman >/dev/null 2>&1; then
CTR=podman
else
echo "ERROR: neither docker nor podman is installed" >&2
exit 1
fi
echo "[host] Using $CTR"
# --- Build the wheel on the host if it does not already exist
WHEEL_DIR="$ROOT/src/dist"
PYTHON_HOST="$ROOT/test/tmp/.venv/bin/python3"
[ -x "$PYTHON_HOST" ] || PYTHON_HOST=python3
if ! ls "$WHEEL_DIR"/testium-*.whl >/dev/null 2>&1; then
echo "[host] Building wheel..."
(cd "$ROOT/src" && "$PYTHON_HOST" -m build --wheel >/dev/null)
fi
WHEEL=$(ls "$WHEEL_DIR"/testium-*.whl | head -1)
WHEEL_NAME=$(basename "$WHEEL")
echo "[host] Using $WHEEL_NAME"
# Expected system Python packages on the target distrib
APT_PACKAGES=(
python3
python3-pip
python3-setuptools
python3-pyside6.qtwidgets
python3-yaml
python3-jinja2
python3-colorama
python3-git
python3-pexpect
python3-matplotlib
python3-lxml
python3-serial
python3-telnetlib3
lua5.4
lua-cjson
lua-socket
git
)
echo "=== Testing on $IMAGE ==="
$CTR run --rm \
-v "$ROOT:/testium:ro" \
-e WHEEL_NAME="$WHEEL_NAME" \
-e PACKAGES="${APT_PACKAGES[*]}" \
"$IMAGE" \
bash -c '
set -e
export DEBIAN_FRONTEND=noninteractive
apt-get update -qq
# 1. Availability check
echo
echo "--- System package availability ---"
AVAILABLE=()
MISSING=()
for pkg in $PACKAGES; do
if apt-cache show "$pkg" >/dev/null 2>&1; then
AVAILABLE+=("$pkg")
echo " OK $pkg"
else
MISSING+=("$pkg")
echo " MISSING $pkg"
fi
done
echo
# 2. Install available packages
echo "--- Installing system packages ---"
apt-get install -qq -y --no-install-recommends "${AVAILABLE[@]}" ca-certificates >/dev/null
# 3. Map missing apt packages to their PyPI equivalents and pip-install
# them as a fallback (kept minimal so the run is still a "system"
# install for the most part)
declare -A PIP_FALLBACK=(
[python3-pyside6.qtwidgets]=pyside6
[python3-telnetlib3]=telnetlib3
)
# junit_xml has no Debian package — install it via pip so the
# validation post_execution.py can import it.
EXTRA_PIP=(junit-xml)
PIP_PKGS=()
for m in "${MISSING[@]}"; do
fallback="${PIP_FALLBACK[$m]:-}"
if [ -n "$fallback" ]; then
PIP_PKGS+=("$fallback")
fi
done
PIP_PKGS+=("${EXTRA_PIP[@]}")
if [ ${#PIP_PKGS[@]} -gt 0 ]; then
echo "--- Installing missing deps via pip: ${PIP_PKGS[*]} ---"
pip install --break-system-packages "${PIP_PKGS[@]}" >/dev/null
fi
# 4. Install testium wheel
echo "--- Installing testium wheel ---"
pip install --break-system-packages --no-deps "/testium/src/dist/$WHEEL_NAME" >/dev/null
# 5. Install the fake_exporter plugin (needed by the report_plugin
# validation test which exercises entry-points discovery).
# Copy it first because /testium is mounted read-only and the
# setuptools backend touches its build dir.
echo "--- Installing testium-fake-exporter (test plugin) ---"
cp -r /testium/test/validation/fake_exporter /tmp/fake_exporter
pip install --break-system-packages /tmp/fake_exporter >/dev/null
# 6. Run the full validation suite. Outputs are streamed live so
# progress is visible — the suite takes a couple of minutes.
# Reports go to /tmp/testium-validation since /testium is RO.
echo "--- Running validation suite ---"
mkdir -p /tmp/testium-validation
cd /testium
testium -b -o \
-d "validation_report_path=/tmp/testium-validation/" \
-- test/validation/main.tum
'
echo "=== $IMAGE: PASS ==="

View File

@@ -1,23 +1,50 @@
# -*- mode: python ; coding: utf-8 -*-
import os
# junit_xml is imported by post_exec scripts running under the *host* Python,
# not the frozen interpreter — so bundling it via hiddenimports alone is not
# enough. We also drop its source files at the _MEIPASS root so the host
# python3 finds them via the PYTHONPATH that py_process.py sets to
# tstium_path (= _MEIPASS when frozen).
import junit_xml as _junit_xml
JUNIT_XML_DIR = os.path.dirname(_junit_xml.__file__)
a = Analysis(
['../../src/testium/__main__.py'],
pathex=['../../src/testium',
'../../src/testium/main_win/resources'],
binaries=[],
# py_func/ and runtime/ are bundled at the _MEIPASS root because the
# py_func subprocess is launched with the *host* Python (not the
# frozen interpreter): it needs the source files on disk to find them
# via cwd=subproc_path() and `python3 py_func` + `from runtime.*`.
# py_func/, lua_func/ and runtime/ are bundled at the _MEIPASS root
# because the py_func subprocess is launched with the *host* Python
# (not the frozen interpreter): it needs the source files on disk to
# find them via cwd=subproc_path() and `python3 py_func` +
# `from runtime.*`. api/ and interpreter/ are intentionally NOT
# exposed: user py_func scripts must go through py_func.tm
# (JSON-RPC bridge) for any testium API call.
datas=[('../../src/VERSION', '.'),
('../../src/lua_func', 'lua_func'),
('../../src/py_func', 'py_func'),
('../../src/lib', 'lib')],
('../../src/testium/lua_func', 'lua_func'),
('../../src/testium/py_func', 'py_func'),
('../../src/testium/runtime', 'runtime'),
(JUNIT_XML_DIR, 'junit_xml')],
hiddenimports=["git",
"interpreter",
"main_win",
"libs",
"libs.console",
"libs.termconsole",
"libs.console_ssh",
"libs.raw_tcp_console",
"libs.runtime_plot",
"runtime",
"py_func",
"py_func.tm",
"py_func.handle",
"py_func.func_call",
"api",
"api.console",
"api.termconsole",
"api.console_ssh",
"api.raw_tcp_console",
"api.runtime_plot",
"api.testium",
"matplotlib.backends.backend_pdf",
"telnetlib3",
"serial",

View File

@@ -27,4 +27,10 @@ if [ ! -d "$PY_VENV_DIR" ]; then
python3 -m venv "$PY_VENV_DIR"
source "$PY_VENV_DIR/bin/activate"
pip install --extra-index-url https://pypi.python.org/pypi -r $REQ_PATH
# Validation suite plugin used to verify the report-exporter
# entry-points discovery end-to-end.
FAKE_EXPORTER_DIR="$(dirname "$REQ_PATH")/../test/validation/fake_exporter"
if [ -d "$FAKE_EXPORTER_DIR" ]; then
pip install -e "$FAKE_EXPORTER_DIR"
fi
fi

View File

@@ -1 +1 @@
0.1
0.2

View File

@@ -13,7 +13,6 @@ license-files = ["../LICENSE"]
classifiers = [
"Development Status :: 5 - Production/Stable",
"Programming Language :: Python",
"License :: OSI Approved :: European Union Public Licence 1.2 (EUPL 1.2)",
]
dependencies = [
"setuptools",
@@ -36,7 +35,9 @@ testium = "testium:main"
[tool.setuptools.packages.find]
where=["."]
exclude=["lua_func", "py_func"]
[tool.setuptools.package-data]
"testium.lua_func" = ["*.lua"]
[tool.setuptools.dynamic]
version = {file = ["VERSION"]}

View File

@@ -245,7 +245,7 @@ A {classname}.close() is missing somewhere in your code !'.format(classname=type
if not sys.platform.startswith('win'):
# import SshConsole if pexpect is installed
try:
from libs.console_ssh import SshConsole
from api.console_ssh import SshConsole
except ImportError:
pass

View File

@@ -8,7 +8,7 @@ import os
import pexpect
from pexpect import ExceptionPexpect, TIMEOUT, EOF, spawn
from libs.console import Console
from api.console import Console
# Exception classes used by this module.

View File

@@ -3,7 +3,7 @@ import sys
import socket
import traceback
from libs.console import *
from api.console import *
class RawTCPConsole(Console):
TYPE = 'rawtcp'

View File

@@ -16,9 +16,9 @@ import numpy as np
import matplotlib.dates as mdates
from datetime import datetime, timedelta, timezone
import libs.testium as tm
import api.testium as tm
from interpreter.test_items.test_result import TestValue
from lib.tum_except import ETUMRuntimeError
from runtime.tum_except import ETUMRuntimeError
from interpreter.utils.py_func_exec import PyFuncExecEngine
from interpreter.utils.api_srv import api_request
from interpreter.utils.eval import post_evaluate
@@ -270,7 +270,7 @@ class RuntimePlotPeriodic(PeriodicTimer):
self.func_name = func_name
self.args = args
self.post_eval = post_eval
self.proc = PyFuncExecEngine(tm.gd("python_bin", ""), api_request, 10)
self.proc = PyFuncExecEngine(api_request, 10)
self.proc.start()
if not self.proc.wait_ready(10):
raise ETUMRuntimeError(

View File

@@ -10,7 +10,7 @@ import os
ourPath = os.path.dirname(__file__)
sys.path.append(ourPath)
from libs.console import (Console, BytesStore, TIMEOUT_NULL)
from api.console import (Console, BytesStore, TIMEOUT_NULL)
class TermConsole(Console):
TYPE = 'term'

View File

@@ -4,7 +4,7 @@ import sys
import textwrap
from time import monotonic
import interpreter.utils.globdict as globdict
from lib.tum_except import (ETUMSyntaxError)
from runtime.tum_except import (ETUMSyntaxError)
###############################################################################
# Console helper functions
@@ -14,7 +14,7 @@ def add_console(console):
''' Function which adds a ``Console`` class instance to *testium*
:param console: The ``Console`` instance.
:type console: ``libs.console.Console`` or child class instance
:type console: ``api.console.Console`` or child class instance
:return: No returned value
'''
@@ -48,7 +48,7 @@ def console(name):
:param name: The name of the ``Console`` instance.
:type name: str
:return: The ``Console`` or child class object
:rtype: ``libs.console.Console`` or child class instance
:rtype: ``api.console.Console`` or child class instance
"""
cons = None
for c in globdict.gd('console_instances', []):
@@ -65,7 +65,7 @@ def add_plot(plot: object) -> None:
''' Function which adds a ``RuntimePlot`` class instance to *testium*
:param plot: The ``RuntimePlot`` instance.
:type plot: ``libs.runtime_plot.RuntimePlot`` or child class instance
:type plot: ``api.runtime_plot.RuntimePlot`` or child class instance
:return: No returned value
'''
@@ -99,7 +99,7 @@ def plot(name: str) -> object:
:param name: The name of the ``RuntimePlot`` instance.
:type name: str
:return: The ``RuntimePlot`` or child class object
:rtype: ``libs.runtime_plot.RuntimePlot`` or child class instance
:rtype: ``api.runtime_plot.RuntimePlot`` or child class instance
"""
plot = None
for g in globdict.gd('plot_instances', []):

View File

@@ -9,8 +9,8 @@ from multiprocessing import Queue
from interpreter.process import TestProcess
from interpreter.utils.test_ctrl import TestSetController
from lib.tum_except import ETUMFileError, ETUMRuntimeError
from lib.stdout_redirect import stdio_redir
from runtime.tum_except import ETUMFileError, ETUMRuntimeError
from runtime.stdout_redirect import stdio_redir
class Batch:

View File

@@ -6,9 +6,9 @@ from threading import Thread
from time import sleep
import copy
from lib.string_queue import StringQueue
from lib.tum_except import print_exception, ETUMRuntimeError, ETUMSyntaxError
import libs.testium as tm
from runtime.string_queue import StringQueue
from runtime.tum_except import print_exception, ETUMRuntimeError, ETUMSyntaxError
import api.testium as tm
import interpreter.utils.globdict as globdict
from interpreter.utils.params import expanse
from interpreter.utils.test_ctrl import TestSetController
@@ -26,7 +26,7 @@ from interpreter.utils.test_init import (
from interpreter.utils.constants import TestItemType as cst_type
from interpreter.test_set import TestSet
from interpreter.utils.include import TUMLoader, TUMLoaderNoIncludes, TUMLoaderRawIncludes
from lib.stdout_redirect import stdio_redir
from runtime.stdout_redirect import stdio_redir
from interpreter.utils.template import template_to_test
from interpreter.utils.yaml_load import yaml_load
from interpreter.utils.py_eval import eval_process_init
@@ -211,7 +211,7 @@ class TestProcess(Process):
env_init()
# Creation of the python evaluation process for loading of the complete test
eval_proc = eval_process_init("", api_request, 10, test_dir)
eval_proc = eval_process_init(api_request, 10, test_dir)
eval_proc.start()
tm.print_debug(f"python bin is: '{eval_proc.python_bin}'.")
if not eval_proc.wait_ready(10):

View File

@@ -1,4 +1,4 @@
from lib.tum_except import ETUMSyntaxError
from runtime.tum_except import ETUMSyntaxError
from interpreter.test_items.test_item import TestItem, test_run, test_data
from interpreter.test_items.test_result import TestResult, TestValue
from interpreter.test_items.item_actions.action import TestItemAction

View File

@@ -3,11 +3,11 @@ from time import sleep
import yaml
from copy import deepcopy
from interpreter.test_items.test_result import TestResult, TestValue
import libs.testium as tm
import api.testium as tm
from interpreter.utils.params import TestItemParams
from interpreter.utils.constants import TestItemType as cst_type
from interpreter.utils.eval import eval_to_boolean, evaluate, post_evaluate
from lib.tum_except import ETUMSyntaxError, item_load_context
from runtime.tum_except import ETUMSyntaxError, item_load_context
LOG_TEST_STOP = '<----- step "{}" finished'
LOG_TEST_START = '-----> step "{}" started'

View File

@@ -1,8 +1,8 @@
from interpreter.test_items.test_item import (TestItem, test_run)
from interpreter.test_items.test_result import TestValue
from lib.tum_except import ETUMSyntaxError, item_load_context
import libs.testium as tm
from runtime.tum_except import ETUMSyntaxError, item_load_context
import api.testium as tm
from interpreter.utils.constants import TestItemType as cst
from interpreter.utils.eval import evaluate

View File

@@ -2,8 +2,8 @@ from interpreter.test_items.test_item import test_run
from interpreter.test_items.test_result import TestValue
from interpreter.test_items.test_item_dialog_base import TestItemDialogBase, _is_text_mode, _is_interactive
from interpreter.utils.constants import TestItemType as cst
from lib.tum_except import item_load_context
import libs.testium as tm
from runtime.tum_except import item_load_context
import api.testium as tm
class TestItemChoicesDialog(TestItemDialogBase):

View File

@@ -3,9 +3,9 @@ import os
import importlib
import traceback
import libs.testium as tm
from lib.tum_except import ETUMSyntaxError
from lib.stdout_redirect import stdio_redir
import api.testium as tm
from runtime.tum_except import ETUMSyntaxError
from runtime.stdout_redirect import stdio_redir
from interpreter.test_items.test_item import test_run
from interpreter.test_items.item_actions import TestItemActions
from interpreter.test_items.item_actions.action import TestItemAction
@@ -345,17 +345,17 @@ class TestItemConsole(TestItemActions):
self.actions_token = {}
global console
console = importlib.import_module("libs.console")
console = importlib.import_module("api.console")
if not sys.platform.startswith("win"):
global console_ssh
console_ssh = importlib.import_module("libs.console_ssh")
console_ssh = importlib.import_module("api.console_ssh")
global termconsole
termconsole = importlib.import_module("libs.termconsole")
termconsole = importlib.import_module("api.termconsole")
global raw_tcp_console
raw_tcp_console = importlib.import_module("libs.raw_tcp_console")
raw_tcp_console = importlib.import_module("api.raw_tcp_console")
self.actions_token["console_name"] = self._prms.getParam(
"console_name", required=True

View File

@@ -1,11 +1,11 @@
import traceback
from lib.tum_except import ETUMSyntaxError, ETUMRuntimeError
from runtime.tum_except import ETUMSyntaxError, ETUMRuntimeError
from interpreter.utils.py_func_exec import PyFuncExecEngine
from interpreter.utils.api_srv import api_request
from interpreter.test_items.test_item import TestItem, test_run
from interpreter.test_items.test_result import TestResult, TestValue
import libs.testium as tm
import api.testium as tm
from interpreter.utils.params import TestItemParams
from interpreter.utils.constants import TestItemType as cst
@@ -207,7 +207,7 @@ then considered as 'False'""")
else:
pl = [self._currentLoop]
proc = PyFuncExecEngine(tm.gd("python_bin", ""), api_request, 10)
proc = PyFuncExecEngine(api_request, 10)
proc.start()
if not proc.wait_ready(10):
raise ETUMRuntimeError(

View File

@@ -1,6 +1,6 @@
import multiprocessing
import libs.testium as tm
import api.testium as tm
from interpreter.test_items.test_item import TestItem

View File

@@ -1,7 +1,7 @@
from interpreter.test_items.test_item import (TestItem, test_run)
from interpreter.test_items.test_result import (TestValue)
from interpreter.utils.constants import TestItemType as cst
from lib.tum_except import ETUMParamError, ETUMSyntaxError
from runtime.tum_except import ETUMParamError, ETUMSyntaxError
import interpreter.utils.version as git
class TestItemGit(TestItem):

View File

@@ -1,8 +1,8 @@
from interpreter.test_items.test_item import (TestItem, test_run)
from interpreter.test_items.test_result import (TestResult, TestValue)
from interpreter.utils.constants import TestItemType as cst
from lib.tum_except import ETUMSyntaxError
import libs.testium as tm
from runtime.tum_except import ETUMSyntaxError
import api.testium as tm
class TestItemGroup(TestItem):
def __init__(self, dict_cycle, parent = None, status_queue=None, filename=""):

View File

@@ -4,8 +4,8 @@ from interpreter.test_items.test_item import test_run
from interpreter.test_items.test_result import TestValue
from interpreter.test_items.test_item_dialog_base import TestItemDialogBase, _is_text_mode, _is_interactive
from interpreter.utils.constants import TestItemType as cst
from lib.tum_except import item_load_context
import libs.testium as tm
from runtime.tum_except import item_load_context
import api.testium as tm
class TestItemImageDialog(TestItemDialogBase):

View File

@@ -2,7 +2,7 @@ import sys
import traceback
from random import randint
from lib.tum_except import ETUMSyntaxError
from runtime.tum_except import ETUMSyntaxError
from interpreter.test_items.test_item import TestItem, test_run
from interpreter.test_items.test_result import TestResult, TestValue

View File

@@ -3,9 +3,9 @@ import socket
import re
import struct
from lib.tum_except import ETUMRuntimeError
import libs.testium as tm
from libs.console import Console
from runtime.tum_except import ETUMRuntimeError
import api.testium as tm
from api.console import Console
def is_ip_address(address):

View File

@@ -5,8 +5,8 @@ import time
from interpreter.test_items.test_item import (TestItem, test_run)
from interpreter.test_items.test_result import (TestResult, TestValue)
from lib.tum_except import ETUMSyntaxError, item_load_context
import libs.testium as tm
from runtime.tum_except import ETUMSyntaxError, item_load_context
import api.testium as tm
from interpreter.utils.constants import TestItemType as cst
class TestItemLet(TestItem):

View File

@@ -4,10 +4,10 @@ import traceback
import pprint
import textwrap
from lib.tum_except import ETUMSyntaxError, ETUMRuntimeError, item_load_context
from runtime.tum_except import ETUMSyntaxError, ETUMRuntimeError, item_load_context
from interpreter.test_items.test_item import TestItem, test_run
from interpreter.test_items.test_result import TestValue
import libs.testium as tm
import api.testium as tm
from interpreter.utils.lua_func_exec import LuaFuncExecEngine
from interpreter.utils.api_srv import api_request
from interpreter.utils.constants import TestItemType as cst
@@ -31,7 +31,7 @@ class TestItemLuaFunc(TestItem):
self.func_name = self._prms.getParam("func_name", required=True)
self.params = self._prms.getParamAll("param")
self._context_id = self._prms.getParam("context_id", default=None, processed=False)
self._lua_func_proc = LuaFuncExecEngine(tm.gd("lua_bin", ""), api_request, 10)
self._lua_func_proc = LuaFuncExecEngine(api_request, 10)
def _get_engine(self):
"""Return (engine, persistent). If context_id is set, use a shared persistent engine."""
@@ -41,7 +41,7 @@ class TestItemLuaFunc(TestItem):
ctx_id = self._prms.expanse(self._context_id)
contexts = tm.gd(_LUA_FUNC_CONTEXTS_KEY, {})
if ctx_id not in contexts:
contexts[ctx_id] = LuaFuncExecEngine(tm.gd("lua_bin", ""), api_request, 10)
contexts[ctx_id] = LuaFuncExecEngine(api_request, 10)
tm.setgd(_LUA_FUNC_CONTEXTS_KEY, contexts)
return contexts[ctx_id], True

View File

@@ -5,7 +5,7 @@ from interpreter.test_items.test_item import test_run
from interpreter.test_items.test_result import TestValue
from interpreter.test_items.test_item_dialog_base import TestItemDialogBase, _is_text_mode, _is_interactive
from interpreter.utils.constants import TestItemType as cst
from lib.tum_except import item_load_context
from runtime.tum_except import item_load_context
class TestItemMsgDialog(TestItemDialogBase):

View File

@@ -2,8 +2,8 @@ from interpreter.test_items.test_item import test_run
from interpreter.test_items.test_result import TestValue
from interpreter.test_items.test_item_dialog_base import TestItemDialogBase, _is_text_mode, _is_interactive
from interpreter.utils.constants import TestItemType as cst
from lib.tum_except import item_load_context
import libs.testium as tm
from runtime.tum_except import item_load_context
import api.testium as tm
class TestItemNoteDialog(TestItemDialogBase):

View File

@@ -6,9 +6,9 @@ from interpreter.test_items.test_item import test_run
from interpreter.test_items.test_result import TestResult, TestValue
from interpreter.utils.constants import TestItemType as cst
from interpreter.utils.eval import eval_to_boolean
from lib.tum_except import ETUMSyntaxError
from lib.string_queue import StringQueue
from lib.stdout_redirect import stdio_redir
from runtime.tum_except import ETUMSyntaxError
from runtime.string_queue import StringQueue
from runtime.stdout_redirect import stdio_redir
class TestItemParallelBranch(TestItemContainer):

View File

@@ -4,10 +4,10 @@ import time
import pprint
import textwrap
from lib.tum_except import ETUMSyntaxError, ETUMRuntimeError, item_load_context
from runtime.tum_except import ETUMSyntaxError, ETUMRuntimeError, item_load_context
from interpreter.test_items.test_item import TestItem, test_run
from interpreter.test_items.test_result import TestValue
import libs.testium as tm
import api.testium as tm
from interpreter.utils.py_func_exec import PyFuncExecEngine
from interpreter.utils.api_srv import api_request
from interpreter.utils.constants import TestItemType as cst
@@ -31,7 +31,7 @@ class TestItemPyFunc(TestItem):
self.func_name = self._prms.getParam("func_name", required=True)
self.params = self._prms.getParamAll("param")
self._context_id = self._prms.getParam("context_id", default=None, processed=False)
self._py_func_proc = PyFuncExecEngine(tm.gd("python_bin", ""), api_request, 10)
self._py_func_proc = PyFuncExecEngine(api_request, 10)
def _get_engine(self):
"""Return (engine, persistent). If context_id is set, use a shared persistent engine."""
@@ -41,7 +41,7 @@ class TestItemPyFunc(TestItem):
ctx_id = self._prms.expanse(self._context_id)
contexts = tm.gd(_PY_FUNC_CONTEXTS_KEY, {})
if ctx_id not in contexts:
contexts[ctx_id] = PyFuncExecEngine(tm.gd("python_bin", ""), api_request, 10)
contexts[ctx_id] = PyFuncExecEngine(api_request, 10)
tm.setgd(_PY_FUNC_CONTEXTS_KEY, contexts)
return contexts[ctx_id], True

View File

@@ -2,7 +2,7 @@ from interpreter.test_items.test_item import test_run
from interpreter.test_items.test_result import TestValue
from interpreter.test_items.test_item_dialog_base import TestItemDialogBase, _is_text_mode, _is_interactive
from interpreter.utils.constants import TestItemType as cst
from lib.tum_except import item_load_context
from runtime.tum_except import item_load_context
class TestItemQuestionDialog(TestItemDialogBase):

View File

@@ -1,7 +1,7 @@
from interpreter.test_items.test_item import (TestItem, test_run)
from interpreter.test_items.test_result import (TestValue)
from lib.tum_except import ETUMSyntaxError
from runtime.tum_except import ETUMSyntaxError
from interpreter.utils.constants import TestItemType as cst
from interpreter.test_report.test_report import Export

View File

@@ -8,9 +8,9 @@ import traceback
from interpreter.test_items.test_item import (TestItem, test_run)
from interpreter.test_items.test_result import (TestValue)
import libs.testium as tm
import api.testium as tm
from interpreter.utils.constants import TestItemType as cst
from lib.tum_except import ETUMSyntaxError, ETUMRuntimeError, item_load_context
from runtime.tum_except import ETUMSyntaxError, ETUMRuntimeError, item_load_context
def nowInBetween(start, end):

View File

@@ -3,8 +3,8 @@ import importlib
import traceback
from functools import wraps
import libs.testium as tm
from lib.tum_except import ETUMSyntaxError, item_load_context
import api.testium as tm
from runtime.tum_except import ETUMSyntaxError, item_load_context
from interpreter.test_items.test_item import TestItem, test_run
from interpreter.test_items.test_result import TestResult, TestValue
from interpreter.test_items.item_actions import TestItemActions
@@ -40,6 +40,7 @@ class TestItemPlotActionOpen(TestItemPlotAction):
try:
gname = self._prms.expanse(self.token)
lpath = self._prms.expanse(self._log_path)
runtime_plot = importlib.import_module("api.runtime_plot")
gr = runtime_plot.RuntimePlot(gname, lpath)
tm.add_plot(gr)
@@ -233,6 +234,3 @@ class TestItemPlot(TestItemActions):
)
self.actions_token = self._prms.getParam("plot_name", required=True)
global runtime_plot
runtime_plot = importlib.import_module("libs.runtime_plot")

View File

@@ -3,11 +3,11 @@ from time import sleep
from datetime import timedelta
from multiprocessing import Process, Pipe
import libs.testium as tm
import api.testium as tm
from interpreter.test_items.test_item import (TestItem, test_run)
from interpreter.test_items.test_result import (TestValue)
from interpreter.utils.constants import TestItemType as cst
from lib.tum_except import ETUMSyntaxError, ETUMRuntimeError, item_load_context
from runtime.tum_except import ETUMSyntaxError, ETUMRuntimeError, item_load_context
class TestItemSleep(TestItem):
"""sleep item usage.

View File

@@ -2,8 +2,8 @@ from interpreter.test_items.test_item import test_run
from interpreter.test_items.test_result import TestValue
from interpreter.test_items.test_item_dialog_base import TestItemDialogBase, _is_text_mode, _is_interactive
from interpreter.utils.constants import TestItemType as cst
from lib.tum_except import item_load_context
import libs.testium as tm
from runtime.tum_except import item_load_context
import api.testium as tm
class TestItemTestedRefsDialog(TestItemDialogBase):

View File

@@ -4,14 +4,14 @@ from unittest import (TestCase, TestSuite, TextTestRunner,
TextTestResult)
from unittest.loader import defaultTestLoader
import libs.testium as tm
from lib.tum_except import (ETUMFileError)
import api.testium as tm
from runtime.tum_except import (ETUMFileError)
from interpreter.utils.modules import load_source
from interpreter.test_items.test_item import (TestItem, test_run, LOG_TEST_STOP, LOG_TEST_START)
from interpreter.test_items.test_result import (TestResult, TestValue)
from interpreter.test_items.test_item import test_data
from interpreter.utils.constants import TestItemType as cst
from lib.stdout_redirect import stdio_redir
from runtime.stdout_redirect import stdio_redir
class UnittestResult(TextTestResult):
"""Test result adapted for unittest test"""

View File

@@ -2,8 +2,8 @@ from interpreter.test_items.test_item import test_run
from interpreter.test_items.test_result import TestValue
from interpreter.test_items.test_item_dialog_base import TestItemDialogBase, _is_text_mode, _is_interactive
from interpreter.utils.constants import TestItemType as cst
from lib.tum_except import item_load_context
import libs.testium as tm
from runtime.tum_except import item_load_context
import api.testium as tm
class TestItemValueDialog(TestItemDialogBase):

View File

@@ -1,4 +1,4 @@
from lib.tum_except import (ETUMRuntimeError)
from runtime.tum_except import (ETUMRuntimeError)
from datetime import datetime
from enum import Enum

View File

@@ -3,7 +3,7 @@ import os
import interpreter.test_report.test_report as tr
from interpreter.utils.paths import prepare_file_to_save
import interpreter.utils.constants as cst
import libs.testium as tm
import api.testium as tm
class ReportExport:

View File

@@ -1,5 +1,5 @@
from junit_xml import (TestSuite, TestCase)
import libs.testium as tm
import api.testium as tm
from interpreter.test_items.test_result import (TestValue)
import interpreter.test_report.report_export as rpe
import interpreter.test_report.test_report as tr

View File

@@ -4,8 +4,8 @@ from functools import wraps
import sqlite3
from time import (time, sleep)
import traceback
from lib.tum_except import (ETUMRuntimeError, ETUMSyntaxError)
from lib.stdout_redirect import stdio_redir
from runtime.tum_except import (ETUMRuntimeError, ETUMSyntaxError)
from runtime.stdout_redirect import stdio_redir
from interpreter.utils.params import (expanse)
from interpreter.utils.paths import prepare_file_to_save
import interpreter.utils.constants as cst
@@ -20,6 +20,52 @@ sqlite3.register_converter('JSON', convert_json)
TEST_REPORT_FILE_REV = '0.1'
def _load_text():
from interpreter.test_report.report_export_txt import ReportExportTxt
return ReportExportTxt
def _load_json():
from interpreter.test_report.report_export_json import ReportExportJSON
return ReportExportJSON
def _load_junit():
try:
from interpreter.test_report.report_export_junit import ReportExportJUnit
return ReportExportJUnit
except ModuleNotFoundError:
raise ETUMRuntimeError(
'Report format "junit" requires "junit_xml" — pip install junit-xml')
def _load_html():
try:
from interpreter.test_report.report_export_html import ReportExportHTML
return ReportExportHTML
except ModuleNotFoundError:
raise ETUMRuntimeError(
'Report format "html" requires "lxml" — pip install lxml')
_EXPORTER_REGISTRY: dict = {
cst.REP_TYPE_TEXT: _load_text,
cst.REP_TYPE_JSON: _load_json,
cst.REP_TYPE_JUNIT: _load_junit,
cst.REP_TYPE_HTML: _load_html,
}
def _discover_plugins():
try:
from importlib.metadata import entry_points
for ep in entry_points(group='testium.exporters'):
try:
cls = ep.load()
_EXPORTER_REGISTRY[ep.name] = lambda c=cls: c
except Exception as e:
print(f'[testium] Failed to load report exporter plugin "{ep.name}": {e}')
except Exception:
pass
_discover_plugins()
def tr_procedure(f):
@wraps(f)
def wrapper(self, *args, **kwds):
@@ -82,28 +128,19 @@ class Export:
else:
path = os.path.join(path, fname)
if et == cst.REP_TYPE_TEXT:
from interpreter.test_report.report_export_txt import ReportExportTxt
ReportExportTxt(name, con, path, pats, keys, no_header)
elif et == cst.REP_TYPE_JSON:
from interpreter.test_report.report_export_json import ReportExportJSON
ReportExportJSON(name, con, path, pats, keys, no_header)
elif et == cst.REP_TYPE_JUNIT:
try:
from interpreter.test_report.report_export_junit import ReportExportJUnit
ReportExportJUnit(name, con, path, pats, keys, no_header)
except ModuleNotFoundError:
raise ETUMRuntimeError('"junit_xml" module not available')
elif et == cst.REP_TYPE_HTML:
try:
from interpreter.test_report.report_export_html import ReportExportHTML
ReportExportHTML(name, con, path, pats, keys, no_header)
except ModuleNotFoundError:
raise ETUMRuntimeError('"lxml" module not available')
elif et == cst.REP_TYPE_SQLITE:
if et == cst.REP_TYPE_SQLITE:
pass
elif et in _EXPORTER_REGISTRY:
try:
cls = _EXPORTER_REGISTRY[et]()
cls(name, con, path, pats, keys, no_header)
except ETUMRuntimeError as e:
print(f'[report] Export skipped: {e}')
else:
raise ETUMSyntaxError('Report export not recognized')
available = ', '.join(
sorted(_EXPORTER_REGISTRY.keys()) + [cst.REP_TYPE_SQLITE])
print(f'[report] Export skipped: format "{et}" not found. '
f'Available: {available}')
class TestReport:
TEST_COLS = [[cst.DB_TEST_TIMESTAMP_START, 'INT'],

View File

@@ -2,13 +2,14 @@ import os
import datetime
from queue import Queue
from interpreter.utils.params import expanse
import libs.testium as tm
from lib.tum_except import ETUMSyntaxError
import api.testium as tm
from runtime.tum_except import ETUMSyntaxError
import interpreter.utils.settings as prefs
from interpreter.test_report.test_report import TestReport
from interpreter.utils.py_func_exec import PyFuncExecEngine
from interpreter.utils.api_srv import api_request
from lib.tum_except import ETUMRuntimeError
from interpreter.utils import bins
from runtime.tum_except import ETUMRuntimeError
from interpreter.utils.constants import TestItemType as cst_type
import interpreter.utils.constants as cst
from interpreter.utils.constants import TEST_TYPE_LIST
@@ -49,6 +50,28 @@ class TestSet:
self._tree = self.__loadTestTree(tum_fime)
self.dict_report = self._testdict.get("report", None)
self.set_post_exec()
self._validate_runtime_deps()
def _validate_runtime_deps(self):
"""Resolve external interpreters needed by this test tree and fail
early with a clear message if any is missing.
Python is always required (the eval engine always runs). Lua is
only required when at least one ``lua_func`` item is present.
"""
needed = ["python"]
if self.__has_item_type(self._rootItem, cst_type.TYPE_LUA_FUNCTION):
needed.append("lua")
bins.ensure(*needed)
def __has_item_type(self, parent, item_type):
for i in range(parent.childCount()):
child = parent.child(i)
if child.type() == item_type.item_name:
return True
if self.__has_item_type(child, item_type):
return True
return False
def execute(self):
self._report = TestReport(self.dict_report)
@@ -352,7 +375,7 @@ class TestSet:
tm.print_debug(f' No file: "{post_exec_file}".')
return
proc = PyFuncExecEngine(tm.gd("python_bin", ""), api_request, 10)
proc = PyFuncExecEngine(api_request, 10)
# start the process for executing external python
proc.start()
try:
@@ -367,13 +390,13 @@ class TestSet:
# tests backup is done here
succ, res = proc.func_call(post_exec_file, "post_exec", [])
if not succ == TestValue.SUCCESS:
tm.print_debug(
tm.print_warn(
f"Test success but the \"post_exec\" function failed: {res}"
)
else:
succ, res = proc.func_call(post_exec_file, "post_exec_fail", [])
if not succ == TestValue.SUCCESS:
tm.print_debug(
tm.print_warn(
f"Test failed but the \"post_exec_fail\" function failed: {res}"
)
finally:

View File

@@ -1,6 +1,6 @@
from lib.api import SUPPORTED_API
from runtime.api import SUPPORTED_API
import libs.testium as tm
import api.testium as tm
# Fill the api_dict with the function of tm
api_dict = {k: getattr(tm, k) for k in SUPPORTED_API if hasattr(tm, k)}

View File

@@ -0,0 +1,151 @@
"""Centralised resolution of external interpreter paths (Python, Lua).
The user can override the path through the global dict via the keys
``python_bin`` and ``lua_bin`` (typically populated from a YAML config).
When unset, the system PATH is searched for known candidates.
Resolution is cached in memory: each interpreter is resolved at most
once per testium process. Subsequent calls return the cached value.
Public API
----------
``python_bin()`` : resolved python3 path (or "" if missing)
``lua_bin()`` : resolved lua >= 5.1 path (or "" if missing)
``ensure(*names)`` : resolve every name and raise a clear error if
any is missing — meant for early validation at
test load time
``reset()`` : clear the cache (mostly useful for tests)
"""
import shutil
import subprocess
import api.testium as tm
from interpreter.utils.paths import sys_app_path_lin, sys_app_path_win
from runtime.tum_except import ETUMRuntimeError
# ---------- Discovery primitives ---------------------------------------------
_PYTHON_CANDIDATES = ["python3", "python"]
_LUA_CANDIDATES = ["lua", "lua5.5", "lua5.4", "lua5.3", "lua5.2", "lua5.1"]
def _which(name):
func = sys_app_path_win if tm.OS() == "Windows" else sys_app_path_lin
return func(name)
def _python_version(path):
cmd = [path, "-c", "import sys; print(sys.version_info[:3])"]
try:
r = subprocess.run(
cmd, capture_output=True, text=True,
encoding=tm.sys_encoding(), timeout=10,
)
except (FileNotFoundError, PermissionError, subprocess.TimeoutExpired):
return None
try:
return eval(r.stdout)
except Exception:
return None
def _is_python3(path):
v = _python_version(path)
return v is not None and v[0] == 3
def _lua_version(path):
try:
r = subprocess.run(
[path, "-v"], capture_output=True, text=True, timeout=10,
)
except (FileNotFoundError, PermissionError, subprocess.TimeoutExpired):
return None
# On Windows the version banner goes to stderr.
line = r.stdout or r.stderr
try:
major, minor, _patch = line.split(" ")[1].split(".")
return (int(major), int(minor))
except (IndexError, ValueError):
return None
def _is_lua51(path):
v = _lua_version(path)
return v is not None and v >= (5, 1)
# ---------- Resolver ---------------------------------------------------------
# (display name, globdict override key, candidate list, validator)
_SPECS = {
"python": ("Python 3", "python_bin", _PYTHON_CANDIDATES, _is_python3),
"lua": ("Lua 5.1+", "lua_bin", _LUA_CANDIDATES, _is_lua51),
}
_resolved = {}
def _resolve(name):
if name in _resolved:
return _resolved[name]
display, gd_key, candidates, validator = _SPECS[name]
override = tm.gd(gd_key, "") or ""
path = ""
if override:
if shutil.which(override) and validator(override):
path = override
else:
tm.print_warn(
f"Configured {display} interpreter '{override}' is not usable; "
f"falling back to discovery."
)
if not path:
for c in candidates:
p = _which(c)
if not p:
continue
if validator(p):
path = p
break
_resolved[name] = path
return path
def python_bin():
return _resolve("python")
def lua_bin():
return _resolve("lua")
def ensure(*names):
"""Resolve each of the given names; raise if any is missing.
Meant to be called at test load with the set of interpreters the
test tree actually needs, so the user gets a clear error before
execution starts instead of deep inside an engine spawn.
"""
missing = []
for n in names:
if not _resolve(n):
display, gd_key, candidates, _ = _SPECS[n]
missing.append(
f" - {display}: tried {candidates} on PATH, none usable. "
f"Set '{gd_key}' in the YAML config to override."
)
if missing:
raise ETUMRuntimeError(
"Required external interpreter(s) not found:\n" + "\n".join(missing)
)
def reset():
_resolved.clear()

View File

@@ -1,6 +1,6 @@
import libs.testium as tm
import api.testium as tm
from interpreter.utils.py_eval import eval_exec
from lib.tum_except import ETUMSyntaxError, ETUMRuntimeError
from runtime.tum_except import ETUMSyntaxError, ETUMRuntimeError
def evaluate(val, **replacement_dict):

View File

@@ -1,8 +1,8 @@
import yaml
import os.path
import libs.testium as tm
import api.testium as tm
from interpreter.utils.params import expanse
from lib.tum_except import ETUMFileError
from runtime.tum_except import ETUMFileError
from interpreter.utils.template import template_to_test
from copy import copy
from interpreter.utils.globdict import global_dict

View File

@@ -1,6 +1,6 @@
from interpreter.utils.lua_process import LuaProcessBase
from lib.tum_except import ETUMRuntimeError
from runtime.tum_except import ETUMRuntimeError
from interpreter.test_items.test_result import TestValue
@@ -29,7 +29,7 @@ class LuaFuncExecEngine(LuaProcessBase):
# In case an error was encountered in the called function
elif "error" in answer:
msg = f"{answer["error"]}"
msg = f"{answer['error']}"
return TestValue.FAILURE, msg
else:

View File

@@ -1,92 +1,14 @@
import os
import sys
import shutil
import subprocess
import socket
import libs.testium as tm
from lib.jrpc import JsonRpcClient
import api.testium as tm
from runtime.jrpc import JsonRpcClient
from interpreter.utils.paths import subproc_path
from lib.tum_except import ETUMRuntimeError
from interpreter.utils.paths import sys_app_path_lin, sys_app_path_win
def _lua_version(path: str):
cmd = f'"{path}" -v'
try:
result = subprocess.run(
cmd,
shell=True,
capture_output=True,
text=True,
encoding=tm.sys_encoding(),
timeout=10,
)
# Under windows, the output is on stderr
data = result.stdout or result.stderr
except (FileNotFoundError, PermissionError, subprocess.TimeoutExpired) as e:
data = ""
try:
vers = ((data.split(" "))[1]).split(".")
if len(vers) != 3:
vers = (0, 0, 0)
except:
vers = (0, 0, 0)
return tuple(vers)
def _is_lua51(lua_bin):
res = False
v = _lua_version(lua_bin)
if (v[0] == "5") and (v[1] >= "1"):
res = True
return res
def _sys_lua_bin():
sys_lua_bin = tm.gd("_sys_lua_bin", "")
if sys_lua_bin != "":
return sys_lua_bin
cur_os = tm.OS()
if cur_os == "Windows":
func = sys_app_path_win
else:
func = sys_app_path_lin
sys_lua_bin = func("lua")
if (sys_lua_bin != "") and not _is_lua51(sys_lua_bin):
tm.print_debug(f"'{sys_lua_bin}' not a lua 5.1 min.")
sys_lua_bin = ""
tm.print_debug(f"lua bin is: '{sys_lua_bin}'.")
tm.setgd("_sys_lua_bin", sys_lua_bin)
return sys_lua_bin
def _is_lua_interpreter(path: str, timeout=2) -> bool:
"""
Checks if the given path points to a valid Lua interpreter.
Args:
path (str): Path to the executable to check.
timeout (int, optional): Timeout for the subprocess in seconds. Defaults to 2.
Returns:
bool: True if the path is a Lua interpreter, False otherwise.
"""
try:
result = subprocess.run(
[path, "-v"],
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
timeout=timeout,
)
return (result.returncode == 0) and (
(result.stdout.startswith("Lua") or result.stderr.startswith("Lua"))
)
except (FileNotFoundError, PermissionError, subprocess.TimeoutExpired):
return False
from runtime.tum_except import ETUMRuntimeError
from interpreter.utils import bins
from interpreter.utils.proc_drain import drain_to_log
class LuaProcessBase:
@@ -96,35 +18,15 @@ class LuaProcessBase:
"LUA_CPATH": {"replace": True},
}
def __init__(self, lua_bin="", request_handler=None, timeout=10):
"""
Initializes the Lua function execution engine.
Args:
lua_bin (str, optional): Path to the Lua interpreter. Defaults to system path.
request_handler: Handler for JSON-RPC requests.
timeout (int, optional): Timeout for operations in seconds. Defaults to 10.
def __init__(self, request_handler=None, timeout=10):
"""Initializes the Lua function execution engine.
Raises:
ETUMRuntimeError: If the Lua path is invalid or no interpreter is found.
ETUMRuntimeError: If no Lua >= 5.1 interpreter is found.
"""
if lua_bin != "":
if shutil.which(lua_bin) is None:
raise ETUMRuntimeError(
f"The passed lua path is not pointing to an executable: '{lua_bin}'"
)
if not _is_lua_interpreter(lua_bin):
raise ETUMRuntimeError(
f"The passed executable is not a lua interpreter: '{lua_bin}'"
)
else:
lua_bin = _sys_lua_bin()
if lua_bin == "":
raise ETUMRuntimeError(f"No valid lua interpreter found")
tm.setgd("lua_bin", lua_bin)
self._lbin = lua_bin
self._lbin = bins.lua_bin()
if not self._lbin:
raise ETUMRuntimeError("No valid Lua 5.1+ interpreter found")
self._req_handler = request_handler
self._process = None
self._port = 0
@@ -192,10 +94,14 @@ class LuaProcessBase:
self._process = subprocess.Popen(
params, env=env, cwd=func_proc_path,
stdin=subprocess.DEVNULL,
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
restore_signals=False,
)
# Route subprocess stdout/stderr (lua require failures, syntax
# errors, anything written to fd 1/2 before the in-script
# remote_print is set up) into the parent's log.
drain_to_log(self._process, prefix="[lua_func] ")
self._rpc = JsonRpcClient(
"localhost", self._port, req_handler=self._req_handler

View File

@@ -1,5 +1,5 @@
import interpreter.utils.globdict as globdict
from lib.tum_except import ETUMSyntaxError, ETUMRuntimeError
from runtime.tum_except import ETUMSyntaxError, ETUMRuntimeError
glob_eval_func = None

View File

@@ -5,7 +5,7 @@ from pathlib import Path
import testium
from interpreter.utils.params import expanse
import subprocess
import libs.testium as tm
import api.testium as tm
def testium_path():
@@ -18,12 +18,9 @@ def testium_path():
return str(Path(tp).parent.resolve())
def subproc_path():
if getattr(sys, 'frozen', False):
# Exécuté depuis le .exe
return sys._MEIPASS
tp = inspect.getfile(inspect.getmodule(testium))
return str(Path(tp).parent.parent.resolve())
# py_func and lua_func now live inside the testium package; their cwd
# is the testium package root, same as testium_path().
return testium_path()
def prepare_file_to_save(file_name, file_ext=""):
iname = file_name

View File

@@ -0,0 +1,48 @@
"""Drain a subprocess stdout/stderr into testium's print pipeline.
Captured lines go through the parent's stdio_redir, so they reach the
test log AND the live output (terminal in batch mode, GUI text panel
in -r mode). This is essential for diagnosing early-startup errors
of py_func / lua_func subprocesses (missing modules, unhandled
exceptions before the in-process redirection kicks in, lua
``require`` failures, anything written to fd 1/2 directly).
"""
import threading
def _drain_pipe(pipe, prefix):
try:
for raw in iter(pipe.readline, b""):
line = raw.decode("utf-8", errors="replace").rstrip("\r\n")
if not line:
continue
if prefix:
print(f"{prefix}{line}")
else:
print(line)
finally:
try:
pipe.close()
except Exception:
pass
def drain_to_log(process, prefix=""):
"""Spawn daemon threads that read ``process.stdout`` and
``process.stderr`` line by line and print each line through the
parent's stdout (so it reaches the log + live output).
Each thread exits cleanly when the subprocess closes the
corresponding pipe (i.e. when it exits). Daemon flag ensures they
do not block testium exit.
"""
threads = []
for pipe in (process.stdout, process.stderr):
if pipe is None:
continue
t = threading.Thread(
target=_drain_pipe, args=(pipe, prefix), daemon=True,
)
t.start()
threads.append(t)
return threads

View File

@@ -1,14 +1,14 @@
from interpreter.utils.py_process import PyProcessBase
from lib.tum_except import ETUMRuntimeError
import libs.testium as tm
from runtime.tum_except import ETUMRuntimeError
import api.testium as tm
eval_process = None
def eval_process_init(python_bin, request_handler, timeout, python_path):
def eval_process_init(request_handler, timeout, python_path):
global eval_process
eval_process = EvalExecEngine(python_bin, request_handler, timeout, python_path)
eval_process = EvalExecEngine(request_handler, timeout, python_path)
return eval_process

View File

@@ -1,6 +1,6 @@
from interpreter.utils.py_process import PyProcessBase
from lib.tum_except import ETUMRuntimeError
from runtime.tum_except import ETUMRuntimeError
from interpreter.test_items.test_result import TestValue
@@ -29,7 +29,7 @@ class PyFuncExecEngine(PyProcessBase):
# In case an error was encountered in the called function
elif "error" in answer:
msg = f"{answer["error"]}"
msg = f"{answer['error']}"
return TestValue.FAILURE, msg
else:

View File

@@ -1,77 +1,13 @@
import os
import shutil
import sys
import subprocess
import socket
from lib.jrpc import JsonRpcClient
import libs.testium as tm
from interpreter.utils.paths import sys_app_path_lin, sys_app_path_win
from lib.tum_except import ETUMRuntimeError
from runtime.jrpc import JsonRpcClient
import api.testium as tm
from runtime.tum_except import ETUMRuntimeError
from interpreter.utils.paths import testium_path, subproc_path
def _python_version(path: str):
cmd = f'"{path}" -c "import sys; print(sys.version_info[:3])"'
try:
result = subprocess.run(
cmd,
shell=True,
capture_output=True,
text=True,
encoding=tm.sys_encoding(),
timeout=10,
)
data = result.stdout
except (FileNotFoundError, PermissionError, subprocess.TimeoutExpired) as e:
tm.print_debug(str(e))
data = ""
return eval(data)
def _is_python3(python_bin):
try:
v = _python_version(python_bin)
if v[0] == 3:
res = True
except:
res = False
return res
def _is_python_interpreter(path: str, timeout=2) -> bool:
try:
result = subprocess.run(
[path, "-c", "import sys; print(sys.executable)"],
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
timeout=timeout,
)
return result.returncode == 0
except (FileNotFoundError, PermissionError, subprocess.TimeoutExpired):
return False
def _sys_python_bin():
sys_python_bin = ""
cur_os = tm.OS()
if cur_os == "Windows":
func = sys_app_path_win
else:
func = sys_app_path_lin
exe = ["python3", "python"]
for e in exe:
sys_python_bin = func(e)
if sys_python_bin == "":
continue
if _is_python3(sys_python_bin):
break
sys_python_bin = ""
return sys_python_bin
from interpreter.utils import bins
from interpreter.utils.proc_drain import drain_to_log
class PyProcessBase:
@@ -80,29 +16,10 @@ class PyProcessBase:
"PYTHONPATH": {"replace": True},
}
def __init__(self, python_bin="", request_handler=None, timeout=10, python_path=""):
self._pbin = python_bin
if (self._pbin is not None) and (self._pbin != ""):
if shutil.which(self._pbin) is None:
raise ETUMRuntimeError(
f"The passed python path is not pointing to an executable: '{self._pbin}'"
)
if not _is_python_interpreter(self._pbin):
raise ETUMRuntimeError(
f"The passed executable is not a python interpreter: '{self._pbin}'"
)
else:
self._pbin = tm.gd("_cached_python_bin", "")
if self._pbin == "":
self._pbin = _sys_python_bin()
tm.setgd("_cached_python_bin", self._pbin)
if self._pbin == "":
raise ETUMRuntimeError(f"No valid python interpreter found")
def __init__(self, request_handler=None, timeout=10, python_path=""):
self._pbin = bins.python_bin()
if not self._pbin:
raise ETUMRuntimeError("No valid Python 3 interpreter found")
self._ppath = python_path
self._req_handler = request_handler
self._process = None
@@ -161,10 +78,15 @@ class PyProcessBase:
self._process = subprocess.Popen(
params, env=env, cwd=func_proc_path,
stdin=subprocess.DEVNULL,
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
restore_signals=False,
)
# Route subprocess stdout/stderr (early-startup errors,
# unhandled exceptions, anything written to fd 1/2 before the
# in-process JSON-RPC stdio_redir kicks in) into the parent's
# log.
drain_to_log(self._process, prefix="[py_func] ")
self._rpc = JsonRpcClient(
"localhost", self._port, req_handler=self._req_handler

View File

@@ -2,7 +2,7 @@ import os
import configparser
import json
import platform
from lib.tum_except import ETUMRuntimeError
from runtime.tum_except import ETUMRuntimeError
SettingsCompany = 'Testium'
SettingsApplication = 'testium'

View File

@@ -4,7 +4,7 @@ from jinja2 import Template
from jinja2.exceptions import TemplateSyntaxError, TemplateError, UndefinedError
from tempfile import TemporaryFile
from interpreter.utils.yaml_load import print_yaml
from lib.tum_except import ETUMSyntaxError
from runtime.tum_except import ETUMSyntaxError
def template_to_test(filename: str, params: list):

View File

@@ -1,6 +1,6 @@
from multiprocessing import Queue
from queue import Empty
from lib.tum_except import ETUMRuntimeError
from runtime.tum_except import ETUMRuntimeError
class TestSetController:
@@ -25,12 +25,17 @@ class TestSetController:
if "timeout" in args:
timeout = args.pop("timeout")
self._test_ctrl.put({cmd: args})
# Drain stale responses (left over from earlier polled commands that
# we had given up on waiting). They can land in the queue after our
# clear() because the TestProcess may have pulled their request
# before the clear, processed them, and pushed the response after.
while True:
res = self._test_resp.get(block, timeout)
if isinstance(res, tuple):
raise ETUMRuntimeError(f"Test set command '{cmd}' failed: '{res[1]}'")
if isinstance(res, dict) and not cmd in res.keys():
raise ETUMRuntimeError(f"Unexpected return error in test set controller")
if isinstance(res, dict) and cmd in res.keys():
return res[cmd]
# Anything else is a stale response — discard and keep waiting.
def clear(self):
while True:

View File

@@ -7,13 +7,13 @@ import yaml
import copy
from interpreter.utils.constants import TestItemType as cst
import libs.testium as tm
import api.testium as tm
import interpreter.utils.globdict as globdict
import interpreter.utils.settings as prefs
from interpreter.utils.paths import testium_path
from interpreter.utils.yaml_load import yaml_load
from interpreter.utils import clear_recursively
from lib.tum_except import ETUMSyntaxError
from runtime.tum_except import ETUMSyntaxError
from interpreter.utils.params import expanse, eval_func_init
from interpreter.utils.eval import evaluate
from interpreter.utils.version import (

View File

@@ -3,7 +3,7 @@ import sys
from importlib import import_module
import interpreter.utils.settings as prefs
import libs.testium as tm
import api.testium as tm
_cached_versions = {}
@@ -31,39 +31,42 @@ def get_version(path :str)-> str:
return "Warning git not supported in your settings, version of {} unknown".format(path)
def get_testium_version():
# case where we're executing from an Appimage
# AppImage
if 'APPIMAGE' in os.environ:
ver = 'unknown'
if 'SEQUENCER_REV' in os.environ:
ver = os.getenv('SEQUENCER_REV')
return (ver + " (binary release)")
ver = os.getenv('SEQUENCER_REV', 'unknown')
return ver + " (binary release)"
# case where we're executing from pyinstaller exe
# PyInstaller frozen exe
if getattr(sys, 'frozen', False):
file_path = os.path.join(sys._MEIPASS, "VERSION")
with open(file_path, 'r') as file:
ver = file.read()
return (ver + " (binary release)")
try:
with open(file_path, 'r') as f:
ver = f.read().strip()
return ver + " (binary release)"
except OSError:
return "unknown (binary release)"
# Executed from sources
try:
# Source checkout: prefer git revision when available
if prefs.settings.git_supported:
try:
git = import_module("git")
path = tm.get_main_dir()
return repo_rev(tm.get_main_dir())
except Exception:
# Not a git repo (typical pip install): fall through.
pass
# Pip-installed wheel: use the package metadata baked from VERSION
try:
return repo_rev(path)
except git.InvalidGitRepositoryError:
pkg_rec = import_module("pkg_resources")
from importlib.metadata import version as _pkg_version
from importlib.metadata import PackageNotFoundError
try:
ret = pkg_rec.get_distribution("testium").version
_cached_versions.update({path: ret})
return str(ret) + " (wheel release)"
except:
return "Warning : testium not versioned"
else:
return "Warning git not supported in your settings, version of testium is unknown."
except:
return ("Unknown")
return _pkg_version("testium") + " (wheel release)"
except PackageNotFoundError:
pass
except ImportError:
pass
return "unknown"
def get_modifications(path : str)-> str:

View File

@@ -1,8 +1,8 @@
from yaml.parser import ParserError
from yaml import load, Loader
from yaml.scanner import ScannerError
from libs.testium import print_debug
from lib.tum_except import ETUMSyntaxError
from api.testium import print_debug
from runtime.tum_except import ETUMSyntaxError
import io

View File

@@ -3,7 +3,7 @@
################################################################################
## Form generated from reading UI file 'about_win.ui'
##
## Created by: Qt User Interface Compiler version 6.11.0
## Created by: Qt User Interface Compiler version 6.10.2
##
## WARNING! All changes made in this file will be lost when recompiling UI file!
################################################################################
@@ -16,39 +16,50 @@ from PySide6.QtGui import (QBrush, QColor, QConicalGradient, QCursor,
QImage, QKeySequence, QLinearGradient, QPainter,
QPalette, QPixmap, QRadialGradient, QTransform)
from PySide6.QtWidgets import (QAbstractButton, QApplication, QDialog, QDialogButtonBox,
QFrame, QLabel, QPlainTextEdit, QSizePolicy,
QWidget)
QLabel, QSizePolicy, QVBoxLayout, QWidget)
import about_win_rc
class Ui_About(object):
def setupUi(self, About):
if not About.objectName():
About.setObjectName(u"About")
About.resize(400, 247)
self.buttonBox = QDialogButtonBox(About)
self.buttonBox.setObjectName(u"buttonBox")
self.buttonBox.setGeometry(QRect(30, 200, 341, 32))
self.buttonBox.setOrientation(Qt.Horizontal)
self.buttonBox.setStandardButtons(QDialogButtonBox.Ok)
About.resize(500, 220)
self.verticalLayout = QVBoxLayout(About)
self.verticalLayout.setSpacing(6)
self.verticalLayout.setObjectName(u"verticalLayout")
self.verticalLayout.setContentsMargins(20, 16, 20, 16)
self.label = QLabel(About)
self.label.setObjectName(u"label")
self.label.setGeometry(QRect(30, 20, 341, 31))
font = QFont()
font.setPointSize(14)
self.label.setFont(font)
self.label.setWordWrap(True)
self.verticalLayout.addWidget(self.label)
self.labelVersion = QLabel(About)
self.labelVersion.setObjectName(u"labelVersion")
self.labelVersion.setGeometry(QRect(30, 60, 341, 16))
self.plainTextEdit = QPlainTextEdit(About)
self.plainTextEdit.setObjectName(u"plainTextEdit")
self.plainTextEdit.setGeometry(QRect(30, 100, 341, 91))
self.plainTextEdit.setFrameShape(QFrame.NoFrame)
self.plainTextEdit.setFrameShadow(QFrame.Sunken)
self.plainTextEdit.setReadOnly(True)
self.labelCesUnitVersion = QLabel(About)
self.labelCesUnitVersion.setObjectName(u"labelCesUnitVersion")
self.labelCesUnitVersion.setGeometry(QRect(30, 70, 341, 16))
self.labelVersion.setWordWrap(True)
self.verticalLayout.addWidget(self.labelVersion)
self.labelCopyright = QLabel(About)
self.labelCopyright.setObjectName(u"labelCopyright")
self.verticalLayout.addWidget(self.labelCopyright)
self.labelLicence = QLabel(About)
self.labelLicence.setObjectName(u"labelLicence")
self.labelLicence.setOpenExternalLinks(True)
self.verticalLayout.addWidget(self.labelLicence)
self.buttonBox = QDialogButtonBox(About)
self.buttonBox.setObjectName(u"buttonBox")
self.buttonBox.setOrientation(Qt.Horizontal)
self.buttonBox.setStandardButtons(QDialogButtonBox.Ok)
self.verticalLayout.addWidget(self.buttonBox)
self.retranslateUi(About)
self.buttonBox.accepted.connect(About.accept)
@@ -57,10 +68,10 @@ class Ui_About(object):
# setupUi
def retranslateUi(self, About):
About.setWindowTitle(QCoreApplication.translate("About", u"A propos", None))
About.setWindowTitle(QCoreApplication.translate("About", u"\u00c0 propos", None))
self.label.setText(QCoreApplication.translate("About", u"Testium", None))
self.labelVersion.setText(QCoreApplication.translate("About", u"Version", None))
self.plainTextEdit.setPlainText(QCoreApplication.translate("About", u"This gui was developed with the help of Qt by Fran\u00e7ois Dausseur.", None))
self.labelCesUnitVersion.setText(QCoreApplication.translate("About", u"Version", None))
self.labelVersion.setText("")
self.labelCopyright.setText(QCoreApplication.translate("About", u"\u00a9 2025-2026 Fran\u00e7ois Dausseur", None))
self.labelLicence.setText(QCoreApplication.translate("About", u"Licensed under <a href=\"https://eupl.eu/1.2/en/\">EUPL-1.2</a>", None))
# retranslateUi

View File

@@ -6,38 +6,31 @@
<rect>
<x>0</x>
<y>0</y>
<width>400</width>
<height>247</height>
<width>500</width>
<height>220</height>
</rect>
</property>
<property name="windowTitle">
<string>A propos</string>
<string>À propos</string>
</property>
<widget class="QDialogButtonBox" name="buttonBox">
<property name="geometry">
<rect>
<x>30</x>
<y>200</y>
<width>341</width>
<height>32</height>
</rect>
<layout class="QVBoxLayout" name="verticalLayout">
<property name="leftMargin">
<number>20</number>
</property>
<property name="orientation">
<enum>Qt::Horizontal</enum>
<property name="topMargin">
<number>16</number>
</property>
<property name="standardButtons">
<set>QDialogButtonBox::Ok</set>
<property name="rightMargin">
<number>20</number>
</property>
</widget>
<property name="bottomMargin">
<number>16</number>
</property>
<property name="spacing">
<number>6</number>
</property>
<item>
<widget class="QLabel" name="label">
<property name="geometry">
<rect>
<x>30</x>
<y>20</y>
<width>341</width>
<height>31</height>
</rect>
</property>
<property name="font">
<font>
<pointsize>14</pointsize>
@@ -46,58 +39,46 @@
<property name="text">
<string>Testium</string>
</property>
</widget>
</item>
<item>
<widget class="QLabel" name="labelVersion">
<property name="text">
<string/>
</property>
<property name="wordWrap">
<bool>true</bool>
</property>
</widget>
<widget class="QLabel" name="labelVersion">
<property name="geometry">
<rect>
<x>30</x>
<y>60</y>
<width>341</width>
<height>16</height>
</rect>
</property>
</item>
<item>
<widget class="QLabel" name="labelCopyright">
<property name="text">
<string>Version</string>
<string>© 2025-2026 François Dausseur</string>
</property>
</widget>
<widget class="QPlainTextEdit" name="plainTextEdit">
<property name="geometry">
<rect>
<x>30</x>
<y>100</y>
<width>341</width>
<height>91</height>
</rect>
</item>
<item>
<widget class="QLabel" name="labelLicence">
<property name="text">
<string>Licensed under &lt;a href=&quot;https://eupl.eu/1.2/en/&quot;&gt;EUPL-1.2&lt;/a&gt;</string>
</property>
<property name="frameShape">
<enum>QFrame::NoFrame</enum>
</property>
<property name="frameShadow">
<enum>QFrame::Sunken</enum>
</property>
<property name="readOnly">
<property name="openExternalLinks">
<bool>true</bool>
</property>
<property name="plainText">
<string>This gui was developed with the help of Qt by François Dausseur.</string>
</property>
</widget>
<widget class="QLabel" name="labelCesUnitVersion">
<property name="geometry">
<rect>
<x>30</x>
<y>70</y>
<width>341</width>
<height>16</height>
</rect>
</property>
<property name="text">
<string>Version</string>
</widget>
</item>
<item>
<widget class="QDialogButtonBox" name="buttonBox">
<property name="orientation">
<enum>Qt::Horizontal</enum>
</property>
<property name="standardButtons">
<set>QDialogButtonBox::Ok</set>
</property>
</widget>
</item>
</layout>
</widget>
<resources>
<include location="../resources/about_win.qrc"/>

View File

@@ -10,7 +10,7 @@ from interpreter.process import TestProcess
from interpreter.utils.test_ctrl import TestSetController
from main_win.test_controller_service import TestControllerService
import interpreter.utils.settings as prefs
from lib.tum_except import ETUMFileError, ETUMRuntimeError
from runtime.tum_except import ETUMFileError, ETUMRuntimeError
class TestFileManager:

View File

@@ -10,12 +10,12 @@ from PySide6.QtGui import (QFont, QFontInfo)
from time import (time)
from main_win.test_tree_items.common import (TEST_COLS, TEST_COLS_WITH_TIME)
from lib.tum_except import (ETUMFileError, ETUMSyntaxError)
from runtime.tum_except import (ETUMFileError, ETUMSyntaxError)
from main_win.test_controller_service import TestControllerService
from main_win.test_tree_items.test_tree_item import make_tree_item
from interpreter.test_items.test_result import (TestValue)
import libs.testium as tm
import api.testium as tm
import interpreter.utils.settings as prefs
from interpreter.utils.constants import TestItemType as cst
from interpreter.utils.icons import icon_prefix

View File

@@ -5,7 +5,7 @@ from PySide6.QtGui import (QIcon, QPixmap, QBrush, QColor)
from PySide6.QtCore import Qt
from PySide6.QtWidgets import (QTreeWidgetItem)
from interpreter.utils.icons import icon_prefix
from libs.testium import print_warn
from api.testium import print_warn
# Maps item_name (from TestItemType.item_name) to visual config.
# Keys: icon (required), icon_on (optional 2nd state), expanded, unfoldable, no_breakpoint

View File

@@ -30,7 +30,7 @@ from main_win.f1_win.d_f1_win import DialogF1
from main_win.test_tree import QTestTree
from main_win.test_run.thread_output import ThreadTestOutput
from lib.string_queue import StringQueue
from runtime.string_queue import StringQueue
from interpreter.process import TestProcess
from interpreter.utils.test_ctrl import TestSetController
from interpreter.utils.icons import icon_prefix
@@ -38,14 +38,14 @@ from interpreter.utils.icons import icon_prefix
from main_win.test_run.outlog import OutLog
from main_win.test_run.test_run import ThreadTestStatus
import interpreter.utils.settings as prefs
from lib.stdout_redirect import stdio_redir
import libs.testium as tm
from interpreter.utils.version import get_testium_version
from runtime.stdout_redirect import stdio_redir
import api.testium as tm
from interpreter.utils.test_init import (
env_init,
locate_report_file,
)
from lib.tum_except import ETUMFileError, ETUMRuntimeError
from interpreter.utils.version import get_testium_version
from runtime.tum_except import ETUMFileError, ETUMRuntimeError
from main_win.test_controller_service import TestControllerService
from main_win.test_runner import TestRunner, TestState
from main_win.test_file_manager import TestFileManager
@@ -206,8 +206,7 @@ class MainWindow(QMainWindow, Ui_MainWindow):
self.d_about_win = QDialog()
self.about_win = Ui_About()
self.about_win.setupUi(self.d_about_win)
self.about_win.labelVersion.setText("testium - " + get_testium_version())
self.about_win.labelCesUnitVersion.setText("")
self.about_win.labelVersion.setText(get_testium_version())
self.d_about_win.setModal(True)
self.d_f1_win = DialogF1(self)

View File

@@ -6,7 +6,7 @@ from PySide6.QtGui import QCursor, QDesktopServices, QFont
from main_win.text_log_highlighter import TextLogHighlighter
import libs.testium as tm
import api.testium as tm
class QTextLog(QPlainTextEdit):
def __init__(self, parent):

View File

@@ -1,7 +1,7 @@
#!/usr/bin/env python
import multiprocessing
from py_func.tm import _init_api, _remote_print
from lib.stdout_redirect import stdio_redir
from runtime.stdout_redirect import stdio_redir
class TcpStdOut:

View File

@@ -5,7 +5,7 @@ from pathlib import Path
import importlib
import traceback
from lib.tum_except import ETUMRuntimeError, ETUMSyntaxError
from runtime.tum_except import ETUMRuntimeError, ETUMSyntaxError
from py_func import tm

View File

@@ -7,8 +7,8 @@ import math
import json
import traceback
from lib.jrpc import JsonRpcSrv
from lib.tum_except import ETUMRuntimeError, print_exception
from runtime.jrpc import JsonRpcSrv
from runtime.tum_except import ETUMRuntimeError, print_exception
import py_func.tm as tm
from py_func.func_call import func_exec
@@ -41,7 +41,7 @@ class FuncHandler(JsonRpcSrv):
except Exception as e:
tb = traceback.format_exc()
return {
"error": f"bad jrpc req handler 'func_call' arguments ({"\n".join(tb.splitlines())}). To be reported to testium support team."
"error": "bad jrpc req handler 'func_call' arguments (" + "\n".join(tb.splitlines()) + "). To be reported to testium support team."
}
if method == "eval":
try:
@@ -57,7 +57,7 @@ class FuncHandler(JsonRpcSrv):
except Exception as e:
tb = traceback.format_exc()
return {
"error": f"bad jrpc req handler 'eval' arguments ({"\n".join(tb.splitlines())}). To be reported to testium support team."
"error": "bad jrpc req handler 'eval' arguments (" + "\n".join(tb.splitlines()) + "). To be reported to testium support team."
}
else:
return {

View File

@@ -2,8 +2,8 @@
import json
import sys
from py_func.handle import FuncHandler
from lib.tum_except import ETUMRuntimeError
from lib.api import SUPPORTED_API
from runtime.tum_except import ETUMRuntimeError
from runtime.api import SUPPORTED_API
thismodule = sys.modules[__name__]
_func_call_thread = None
@@ -28,7 +28,7 @@ def _make_api(name):
if "result" in res:
ret_val = res["result"]
elif "error" in res:
raise ETUMRuntimeError(f"api call to 'tm.{name}' failed with error '{res["error"]}'")
raise ETUMRuntimeError(f"api call to 'tm.{name}' failed with error '{res['error']}'")
else:
raise ETUMRuntimeError("api call failure in jrpc client to be reported to testium support team.")
return ret_val

View File

View File

@@ -6,5 +6,10 @@ SUPPORTED_API = [
"add_plot_values",
"last_plot_value",
"text_mode",
"OS",
"get_main_dir",
"init_timestamp",
"timestamp",
"timestamp_as_sec",
]

View File

@@ -6,11 +6,11 @@ import itertools
from time import sleep
from typing import Callable, Any
try:
import libs.testium as tm
import api.testium as tm
except:
import py_func.tm as tm
from lib.tum_except import ETUMRuntimeError
from runtime.tum_except import ETUMRuntimeError
"""Lightweight JSON-RPC 2.0 helpers over TCP sockets.
@@ -145,7 +145,7 @@ class JsonRpcConnection:
self.pending[msg["id"]]["response"] = msg
self.pending[msg["id"]]["event"].set()
else:
self.print_info(f"msg id '{msg["id"]}' inconsistency")
self.print_info(f"msg id '{msg['id']}' inconsistency")
# ---------- Handler ----------
def _handle_request(self, meth, params, rid=None):

View File

@@ -1,7 +1,7 @@
import sys
import threading
from threading import (Thread, Event)
from lib.string_queue import StringQueue
from runtime.string_queue import StringQueue
from time import (sleep)

View File

@@ -1,4 +1,4 @@
import libs.testium as tm
import api.testium as tm
def RetreiveData(console_name):
print("--------------- retrieving data ---------------")

View File

@@ -1,4 +1,4 @@
import libs.testium as tm
import api.testium as tm
def RetreiveData(console_name):
print("--------------- retrieving data ---------------")

View File

@@ -0,0 +1,42 @@
"""CSV report exporter — used as a real plugin by the testium validation suite.
Demonstrates the contract: take the SQLite connection, output path, optional
name/key filters, and produce the output. Has no dependency on testium
internals.
"""
import csv
class FakeExporter:
COLUMNS = [
'timestamp_start',
'test_id',
'parent_id',
'level',
'test_name',
'test_type',
'report_key',
'result',
'message',
'duration',
]
def __init__(self, name, con, path, pats, keys, no_header=False):
clauses = []
for p in pats:
clauses.append(f'test_name LIKE "{p}"')
for k in keys:
clauses.append(f'report_key LIKE "{k}"')
where = ('WHERE ' + ' OR '.join(clauses) + ' ') if clauses else ''
cols = ', '.join(self.COLUMNS)
rows = con.execute(
f'SELECT {cols} FROM tests {where}ORDER BY timestamp_start'
).fetchall()
with open(path, 'w', newline='', encoding='utf-8') as f:
writer = csv.writer(f)
if not no_header:
writer.writerow(self.COLUMNS)
for row in rows:
writer.writerow(row)

Some files were not shown because too many files have changed in this diff Show More