Compare commits
37 Commits
v0.1
...
refactor/i
| Author | SHA1 | Date | |
|---|---|---|---|
| 354c5e12e8 | |||
| b1a7dac0f3 | |||
| d0721af719 | |||
| d4889c2a2e | |||
| a260e2a56c | |||
| dd584c9064 | |||
| 4d8cafb5a0 | |||
| 6f832cd67b | |||
| ff46886865 | |||
| 50d183d191 | |||
| 2177715641 | |||
| a728f561be | |||
| 116e528a7d | |||
| cc744e17a1 | |||
| ab39b49558 | |||
| 95275c4418 | |||
| 0d614c2921 | |||
| 9466b091dd | |||
| 511288bd03 | |||
| 51b144f60c | |||
| dee8d4a682 | |||
| e726d47547 | |||
| 5fd50e1c85 | |||
| 51939a566a | |||
| 26fccda6bf | |||
| 405fb82fca | |||
| 6064d96138 | |||
| 0658540cc2 | |||
| 7bf946dabe | |||
| f52d7bbe53 | |||
| c83ebccb55 | |||
| f17ef8a3a1 | |||
| ddb18abc21 | |||
| 358ade8c98 | |||
| 46bdb44cfb | |||
| 41519c97cb | |||
| b9475c6e9b |
3
.gitignore
vendored
3
.gitignore
vendored
@@ -8,6 +8,8 @@ dist
|
||||
/.vscode
|
||||
.venv/
|
||||
.flatpak-builder/
|
||||
package/flatpak/repo/
|
||||
package/flatpak/*.flatpak
|
||||
crash.tx*
|
||||
report_test.tx*
|
||||
*.autosave
|
||||
@@ -24,6 +26,7 @@ package/appimage/*.AppImage
|
||||
package/appimage/src
|
||||
package/appimage/*.py
|
||||
AppDir
|
||||
*.squashfs
|
||||
doc/manual/doxygen
|
||||
doc/manual/sphinx/build/*
|
||||
doc/manual/sphinx/source/_build/*
|
||||
|
||||
101
CONTRIBUTING.md
101
CONTRIBUTING.md
@@ -45,7 +45,7 @@ For existing files, keep the header that is already there.
|
||||
3. Commit with a clear message (one logical change per commit).
|
||||
4. Make sure the validation suite still passes:
|
||||
```
|
||||
./run.sh -b -l mon_log.log -- test/validation/main.tum
|
||||
./run.sh -b -- test/validation/main.tum
|
||||
```
|
||||
5. Open a pull request against `main`.
|
||||
|
||||
@@ -56,6 +56,105 @@ For existing files, keep the header that is already there.
|
||||
- Add or update tests in `test/validation/` for new test items or behaviours
|
||||
- Update `CLAUDE.md` and the Sphinx manual for user-visible changes
|
||||
|
||||
## Development
|
||||
|
||||
### Debugging in VSCode
|
||||
|
||||
The recommended workflow:
|
||||
|
||||
1. Add a debug configuration to `.vscode/launch.json`:
|
||||
```json
|
||||
{
|
||||
"configurations": [
|
||||
{
|
||||
"name": "Python : testium",
|
||||
"type": "python",
|
||||
"request": "launch",
|
||||
"program": "${workspaceFolder}/src/testium",
|
||||
"console": "integratedTerminal",
|
||||
"args": ["-g"],
|
||||
"justMyCode": true
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
2. Install `debugpy` in the venv: `python -m pip install debugpy`.
|
||||
3. Open the *Run and Debug* tab and press play. testium starts; load and
|
||||
run a `.tum` file. Set breakpoints where you want to investigate.
|
||||
|
||||
### Qt GUI modification
|
||||
|
||||
UI files (`*.ui`) are edited in **Qt Creator**. After editing, regenerate
|
||||
the corresponding Python and resource files:
|
||||
|
||||
```sh
|
||||
scripts/qt_generate.sh
|
||||
```
|
||||
|
||||
Icons come from <https://github.com/free-icons/free-icons>.
|
||||
|
||||
### Sphinx documentation
|
||||
|
||||
```sh
|
||||
pip install sphinx linuxdoc
|
||||
doc/manual/sphinx/build_doc.sh
|
||||
```
|
||||
|
||||
PDF generation requires `texlive`:
|
||||
|
||||
```sh
|
||||
sudo apt install texlive-full
|
||||
```
|
||||
|
||||
### Validation suite
|
||||
|
||||
Batch mode (CI-friendly, headless):
|
||||
|
||||
```sh
|
||||
./run.sh -b -- test/validation/main.tum
|
||||
```
|
||||
|
||||
GUI mode (loads the suite, click *Run* to execute and inspect the tree):
|
||||
|
||||
```sh
|
||||
./run.sh test/validation/main.tum
|
||||
```
|
||||
|
||||
GUI run-and-close (executes the suite, then closes):
|
||||
|
||||
```sh
|
||||
./run.sh -r -- test/validation/main.tum
|
||||
```
|
||||
|
||||
Subset run via the `items` define (works in any mode):
|
||||
|
||||
```sh
|
||||
./run.sh -b -d "items=['parallel','common']" -- test/validation/main.tum
|
||||
```
|
||||
|
||||
### Cross-distribution check
|
||||
|
||||
`package/deb/test_distro.sh` spins up a Docker/Podman container of the
|
||||
target image, installs the expected system Python deps via apt (with
|
||||
pip fallback for what is missing), installs the testium wheel and runs
|
||||
the validation suite end-to-end. Currently green on `debian:bookworm`,
|
||||
`debian:trixie`, `ubuntu:24.04`.
|
||||
|
||||
```sh
|
||||
./package/deb/test_distro.sh debian:trixie
|
||||
```
|
||||
|
||||
## Release procedure
|
||||
|
||||
1. Update `release_note.txt`.
|
||||
2. Bump the version in `src/VERSION`.
|
||||
3. Make sure the documentation is up to date — rebuild with
|
||||
`doc/manual/sphinx/build_doc.sh` if needed.
|
||||
4. Push and tag the commit with the new version.
|
||||
5. Build the binary release: `package/pyinstaller/build.sh`.
|
||||
6. Run the validation suite against each generated binary.
|
||||
7. Confirm all validation results are green before publishing.
|
||||
|
||||
## Reporting security issues
|
||||
|
||||
Please do **not** report security vulnerabilities through public GitHub
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# Testium — Claude Context
|
||||
# Testium — Design Context
|
||||
|
||||
## What is testium
|
||||
|
||||
@@ -114,11 +114,20 @@ To add a new API call usable from subprocesses:
|
||||
### External interpreter resolution (`bins.py`)
|
||||
`src/testium/interpreter/utils/bins.py` — single source of truth for the paths to the external Python and Lua interpreters used by subprocesses.
|
||||
|
||||
- `python_bin()` / `lua_bin()` : resolve once, cache in memory. User can override via the `python_bin` / `lua_bin` global dict keys (typically populated from the YAML config). Falls back to discovery on PATH (candidates: `python3`/`python` and `lua`/`lua5.5`/`lua5.4`/`lua5.3`/`lua5.2`/`lua5.1`).
|
||||
- `python_bin()` / `lua_bin()` : resolve and cache. The cache is keyed by `(name, override)` so that a later change to `gd[python_bin]` (typically when a `param.yaml` sets the key) triggers a re-resolution on the next lookup instead of returning the stale auto-discovered path. Falls back to discovery on PATH (candidates: `python3`/`python` and `lua`/`lua5.5`/`lua5.4`/`lua5.3`/`lua5.2`/`lua5.1`).
|
||||
- `ensure(*names)` : called by `TestSet._validate_runtime_deps()` at test load. Always requires `python` (the eval engine always runs); requires `lua` only if a `lua_func` item is in the tree. Fails fast with a clear error citing tried candidates and override key.
|
||||
|
||||
Engines (`PyProcessBase`, `LuaProcessBase`, `EvalExecEngine`) call `bins.python_bin()`/`bins.lua_bin()` themselves — call sites never pass an explicit binary path.
|
||||
|
||||
#### Override-timing contract (`apply_overrides`)
|
||||
`bins.python_bin()` is called for the **first** time inside `eval_process_init()` (the long-lived inline-`<| … |>` subprocess), which happens **before** the YAML param files are loaded. To make `-d python_bin=…` and the GUI `python_bin` preference take effect for `eval_proc` itself, `process.py:run()` applies them to gd **before** `eval_process_init()` via the `apply_overrides()` helper extracted from `update_global()`. The post-load `update_global()` call then re-applies the same overrides (after `prepare_global()` clears gd), keeping the gd value in sync with the cached resolution.
|
||||
|
||||
| Override source | `eval_proc` | `py_func` / `cycle` / `post_exec` |
|
||||
|---|---|---|
|
||||
| `-d python_bin=…` (CLI) | ✅ | ✅ |
|
||||
| GUI `python_bin` preference | ✅ | ✅ |
|
||||
| `python_bin: …` in `param.yaml` | ❌ (eval_proc already started) | ✅ (cache re-resolves on key change) |
|
||||
|
||||
## Key files
|
||||
|
||||
| Path | Role |
|
||||
@@ -183,6 +192,8 @@ Icons are assigned once when the test file is loaded (not updated live on theme
|
||||
|
||||
The sub-test's own pass/fail result is intentionally not propagated.
|
||||
|
||||
The interpreter and entry point used to spawn the sub-instance are picked automatically by `_testium_launch_cmd()` based on how the parent was started (AppImage → `$APPIMAGE`; Flatpak → `flatpak run`; PyInstaller → the frozen binary; source/wheel → `[sys.executable, abspath(sys.argv[0])]`). The user cannot override either via the YAML — selecting a different testium binary or Python from a sub-test was removed because it was either ill-defined (bundle modes have no separable Python) or could mismatch the parent's environment in surprising ways.
|
||||
|
||||
### Report exporters & plugins
|
||||
`src/testium/interpreter/test_report/test_report.py` — `_EXPORTER_REGISTRY` dict maps a format name (cmd key in the YAML `report.export`) to a lazy loader. Built-ins: `text`, `json`, `junit` (needs `junit_xml`), `html` (needs `lxml`). `sqlite` is the storage layer, no-op as an export.
|
||||
|
||||
@@ -201,17 +212,38 @@ A real-world test plugin lives at `test/validation/fake_exporter/` (CSV exporter
|
||||
|
||||
## Packaging
|
||||
|
||||
Three distribution channels coexist, sharing the single `src/testium/` package:
|
||||
Four distribution channels coexist, all sharing the single `src/testium/` package and the single `src/requirements.txt` dependency list:
|
||||
|
||||
| Channel | Where | Notes |
|
||||
|---------|-------|-------|
|
||||
| Wheel (`pip install`) | `src/pyproject.toml` | Vanilla Python package; entry point `testium = "testium:main"` |
|
||||
| PyInstaller binary | `package/pyinstaller/` | Single ~130 MB binary. `py_func`, `runtime`, `lua_func` bundled at `_MEIPASS` root so the **host** Python can find them when launched as `python3 py_func`. `api`/`interpreter` are **not** exposed (subprocess isolation). |
|
||||
| Flatpak | `package/flatpak/` | (Existing recipe, not actively maintained in current refactor wave.) |
|
||||
| Channel | Where | Build | Notes |
|
||||
|---------|-------|-------|-------|
|
||||
| Wheel (`pip install`) | `src/pyproject.toml` | `python -m build` | Vanilla Python package; entry point `testium = "testium:main"`. |
|
||||
| PyInstaller binary | `package/pyinstaller/` | `build.sh` | Single ~130 MB binary. `py_func`, `runtime`, `lua_func` bundled at `_MEIPASS` root so the **host** Python can find them when launched as `python3 py_func`. `api`/`interpreter` are **not** exposed (subprocess isolation). |
|
||||
| Flatpak | `package/flatpak/` | `build.sh` (uses `flatpak-builder`) | KDE 6.10 runtime. The bundled Python runs only the main process; `py_func` / `lua_func` MUST run under the **host** interpreter (no Python/Lua bundled). Produces a distributable `.flatpak` bundle. |
|
||||
| AppImage | `package/appimage/` | `build.sh` (Debian Bookworm container via Podman/Docker) | Bundles Python 3.11 for the main process; `py_func` / `lua_func` MUST run under the **host** interpreter. Build runs in a container so it works on Arch / any non-Debian host. |
|
||||
|
||||
The `.deb` work-in-progress lives in `package/deb/`:
|
||||
- `test_distro.sh debian:bookworm | debian:trixie | ubuntu:24.04` spins up a Docker/Podman container, reports system package availability, falls back to pip for what's missing (`pyside6` on bookworm/ubuntu, `telnetlib3`, `junit_xml`), runs the validation suite. Currently green on the three targets.
|
||||
|
||||
### Host-only py_func / lua_func in sandboxed bundles (Flatpak, AppImage)
|
||||
|
||||
The bundled Python (Flatpak's runtime python, AppImage's `python3.11`) is reserved for the **main process only**. Subprocesses (`py_func`, `lua_func`, `git`) must use the host's interpreters and tools so user-installed modules (pyserial, junit_xml, …) are visible. This is enforced by `interpreter/utils/bins.py`:
|
||||
|
||||
- `_in_flatpak()` (checks `/.flatpak-info`) and `_in_appimage()` (checks `APPIMAGE` env var) detect the sandbox.
|
||||
- `_which(name)` probes only host bin dirs in those modes:
|
||||
- Flatpak: `/run/host/usr/{local/,}bin`, `/run/host/bin` (host mounted via `--filesystem=host-os`).
|
||||
- AppImage: `/usr/local/bin`, `/usr/bin`, `/bin` (we are directly on the host filesystem).
|
||||
- If the host has no python3/lua, `ensure()` raises `ETUMRuntimeError` at test load with the candidate list — no silent fallback to a bundled interpreter.
|
||||
- User overrides (`python_bin`/`lua_bin` in globdict): bare names are resolved through `_which()` (host-only), absolute paths are accepted as-is.
|
||||
- `apply_host_libs(env)` is called by `py_process.py` / `lua_process.py` on the env passed to Popen:
|
||||
- Flatpak: prepends host lib dirs to `LD_LIBRARY_PATH` so the dynamic linker finds host `.so`'s.
|
||||
- AppImage: strips `$APPDIR`-prefixed entries from `LD_LIBRARY_PATH` / `PYTHONPATH` / `PATH` and drops `PYTHONHOME`, so the host Python doesn't try to load the bundled stdlib/site-packages.
|
||||
- `apply_host_lua_paths(env)` (Flatpak only) prepends `/run/host/usr/{lib,share}/lua/X.Y` to `LUA_PATH` / `LUA_CPATH` so `cjson`, `socket`, etc. resolve. Must be called **after** user `lua_env` overrides so host paths win. AppImage relies on host Lua's compiled-in defaults.
|
||||
- `py_process.py` additionally pops `PYTHONUSERBASE` (set to `/var/data/python` by the Flatpak runtime, which would hide `~/.local/lib/...`).
|
||||
|
||||
### Version reporting (`interpreter/utils/version.py`)
|
||||
|
||||
Both Flatpak and AppImage export `TESTIUM_VERSION` from a launcher (Flatpak: launcher script in `org.testium.Testium.yaml`; AppImage: `runtime.env` in `AppImageBuilder.yml`). `get_testium_version()` checks `/.flatpak-info` / `APPIMAGE` and reads `TESTIUM_VERSION` rather than relying on package metadata or repo introspection.
|
||||
|
||||
## Recent fixes / notable changes
|
||||
- Restructure: single `src/testium/` Python package (was 4 sibling top-levels: `testium`, `lib`, `py_func`, `lua_func`). `lib/` → `runtime/`, `libs/` → `api/`. `pip install` now produces a clean `site-packages/testium/` with no top-level pollution; `.lua` files travel via `package_data`.
|
||||
- `bins.py`: centralised resolution + cache of external `python3` / `lua` binaries. Replaces the scattered `tm.gd("python_bin")`/`tm.gd("lua_bin")` dance and the duplicated discovery logic in `py_process.py`/`lua_process.py`. Validates at test load via `TestSet._validate_runtime_deps()` so missing interpreters fail fast.
|
||||
@@ -238,12 +270,18 @@ The `.deb` work-in-progress lives in `package/deb/`:
|
||||
- `unittest` item: renamed from `unittest_file`.
|
||||
- GUI test tree: check and fold state preserved across same-file reloads.
|
||||
- Licence: EUPL-1.2.
|
||||
- Interpreter override timing: `apply_overrides()` extracted from `update_global()` and called by `process.py:run()` before `eval_process_init()`, so `-d python_bin=…` / GUI prefs reach `bins.python_bin()` on its first lookup. `bins._resolve()` cache is now keyed by `(name, override)` so later `param.yaml` changes are picked up by subsequently constructed engines.
|
||||
|
||||
## Validation tests
|
||||
Located in `test/validation/`. Run with `-b` flag:
|
||||
Located in `test/validation/`. Two entry points:
|
||||
```
|
||||
./run.sh -b -l mon_log.log -- test/validation/main.tum
|
||||
./test/validation/run.sh # wrapper — uses a dedicated venv (see below)
|
||||
./run.sh -b -- test/validation/main.tum # direct — testium's own python is used for test execution
|
||||
```
|
||||
The `run.sh` / `run.bat` wrappers create a dedicated Python venv at `${TMPDIR:-/tmp}/testium-validation-venv` (Linux) or `%TEMP%\testium-validation-venv` (Windows), with `--system-site-packages` + `pip install junit-xml`, and run the suite with `-d python_bin=…` so every test-execution subprocess (eval_proc, py_func, cycle, post_exec) runs inside the venv. testium itself keeps running in the project's own environment. `clean` as the first argument recreates the venv.
|
||||
|
||||
The `venv` item (`test/validation/items/venv/`) asserts that the override actually took effect: `python_bin` is set, `sys.executable` matches it, `sys.prefix == dirname(dirname(python_bin))`, and `sys.prefix != sys.base_prefix` (the last marker catches the case where `python_bin` happens to be a system interpreter, which path-equality alone would miss because the venv's `bin/python3` is a symlink to the host). Both `eval_proc` (inline `<| … |>`) and `py_func` paths are exercised.
|
||||
|
||||
Parallel item tests: `test/validation/items/parallel/test.tum`
|
||||
|
||||
## Dependencies
|
||||
295
README.md
295
README.md
@@ -1,185 +1,122 @@
|
||||
# Documentation
|
||||
# testium
|
||||
|
||||
[See here](doc/manual/testium_manual.pdf).
|
||||
testium is a YAML-driven test sequencer for hardware-in-the-loop and
|
||||
integration testing. A test campaign is described in a `.tum` file as a tree
|
||||
of items (checks, console interactions, Python/Lua functions, parallel blocks,
|
||||
dialogs, …); testium executes the tree, captures results, and produces
|
||||
reports in several formats.
|
||||
|
||||
# License
|
||||
## Documentation
|
||||
|
||||
Copyright (c) 2025-2026 François Dausseur.
|
||||
* [Quick start](doc/quick_start.md) — install and run your first test in
|
||||
five minutes.
|
||||
* [Tutorial](doc/tutorial.md) — guided walk-through of the most common
|
||||
test items with a runnable example.
|
||||
* [User manual (PDF)](doc/manual/testium_manual.pdf) — full reference.
|
||||
* [`doc/examples/`](doc/examples/) — runnable `.tum` snippets.
|
||||
|
||||
## Pre-built releases
|
||||
|
||||
Pre-built artifacts are published at
|
||||
<https://git.beafrancois.fr/v-and-v/testium/releases>:
|
||||
|
||||
* **Python wheel** (`testium-<version>-py3-none-any.whl`) — install with
|
||||
`pip install testium-*.whl`. Lighter than the binary; pulls Python
|
||||
dependencies from PyPI on install.
|
||||
* **Self-contained Linux binary** (`testium`, built with PyInstaller) —
|
||||
runnable directly, no Python installation required on the host. Lua
|
||||
support still needs a system `lua` interpreter and the `lua-socket` /
|
||||
`lua-cjson` modules.
|
||||
* **Flatpak bundle** (`testium.flatpak`) — install with:
|
||||
|
||||
```sh
|
||||
# Add Flathub (once, to fetch the KDE/PySide runtimes)
|
||||
flatpak remote-add --user --if-not-exists flathub https://flathub.org/repo/flathub.flatpakrepo
|
||||
|
||||
# Install the bundle
|
||||
flatpak install --user testium.flatpak
|
||||
```
|
||||
|
||||
After installation testium appears in the desktop application menu and the
|
||||
`testium` command is available in the terminal (requires `~/.local/bin` in
|
||||
`PATH`, which most modern distributions provide by default).
|
||||
|
||||
## Quick start
|
||||
|
||||
From a checkout of the repository:
|
||||
|
||||
| OS | Command |
|
||||
|----|---------|
|
||||
| Linux | `./run.sh` |
|
||||
| Windows (cmd) | `run.bat` |
|
||||
| Windows (PowerShell) | `run.ps1` |
|
||||
|
||||
The wrapper creates a Python virtual environment on first run and starts
|
||||
testium in GUI mode. Add `-b path/to/test.tum` to run a test in batch mode.
|
||||
|
||||
## Manual installation
|
||||
|
||||
If the wrapper script does not fit your environment, set up testium manually:
|
||||
|
||||
```sh
|
||||
python3 -m venv .venv
|
||||
source .venv/bin/activate
|
||||
pip install -r src/requirements.txt
|
||||
```
|
||||
|
||||
Required Python packages (see `src/requirements.txt`):
|
||||
`pyside6`, `pyserial`, `pyyaml`, `pexpect`, `gitpython`, `jinja2`, `colorama`,
|
||||
`matplotlib`, `junit-xml`, `lxml`.
|
||||
|
||||
For tests using `lua_func` items, install Lua (>= 5.1) plus the `socket` and
|
||||
`cjson` modules. On Debian/Ubuntu:
|
||||
|
||||
```sh
|
||||
sudo apt install lua5.4 lua-socket lua-cjson
|
||||
```
|
||||
|
||||
Run testium:
|
||||
|
||||
```sh
|
||||
python3 src/testium # GUI
|
||||
python3 src/testium -b mytest.tum # batch
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### `wl_proxy_marshal_flags` symbol error
|
||||
|
||||
```
|
||||
testium: symbol lookup error: ... undefined symbol: wl_proxy_marshal_flags
|
||||
```
|
||||
|
||||
Force the X11 Qt backend:
|
||||
|
||||
```sh
|
||||
export QT_QPA_PLATFORM=xcb
|
||||
testium
|
||||
```
|
||||
|
||||
### `xcb plugin missing`
|
||||
|
||||
```
|
||||
qt.qpa.plugin: Could not load the Qt platform plugin "xcb"
|
||||
```
|
||||
|
||||
Install the missing system libraries:
|
||||
|
||||
```sh
|
||||
sudo apt install libxcb-cursor0 libicu-dev libxcb-cursor-dev
|
||||
```
|
||||
|
||||
## License
|
||||
|
||||
Copyright © 2025-2026 François Dausseur.
|
||||
|
||||
testium is distributed under the **European Union Public Licence v. 1.2
|
||||
(EUPL-1.2)** — see the [LICENSE](LICENSE) file for the full text.
|
||||
(EUPL-1.2)** — see [`LICENSE`](LICENSE) for the full text. SPDX:
|
||||
`EUPL-1.2`.
|
||||
|
||||
SPDX identifier: `EUPL-1.2`
|
||||
|
||||
Contributions are accepted under the same licence (inbound = outbound). See
|
||||
[CONTRIBUTING.md](CONTRIBUTING.md) for details.
|
||||
|
||||
# run testium
|
||||
|
||||
From the root path, on windows `cmd`:
|
||||
|
||||
run.bat
|
||||
|
||||
On windows powershell:
|
||||
|
||||
run.ps1
|
||||
|
||||
On linux:
|
||||
|
||||
./run.sh
|
||||
|
||||
The virtual environment is created if needed and *testium* is started.
|
||||
|
||||
# Manual setup
|
||||
|
||||
A python virtual environment should be created:
|
||||
|
||||
python3 -m venv <testium_venv>
|
||||
|
||||
## Requirements
|
||||
|
||||
In the virtual environment, the following modules must be installed:
|
||||
|
||||
* pyside6
|
||||
* pyserial
|
||||
* pyyaml
|
||||
* pexpect
|
||||
* gitpython
|
||||
* jinja2
|
||||
* colorama
|
||||
* matplotlib
|
||||
* junit-xml
|
||||
* lxml
|
||||
|
||||
A `requirements.txt` file is also available in the git repository in the path `testium/src/`.
|
||||
|
||||
|
||||
## run testium
|
||||
|
||||
from the testium path, execute
|
||||
|
||||
python3 -m src/testium
|
||||
|
||||
# Doc generation
|
||||
|
||||
## Install sphinx
|
||||
|
||||
pip install sphinx linuxdoc
|
||||
|
||||
## Generate the doc
|
||||
|
||||
Execute
|
||||
|
||||
doc/manual/sphinx/./build_doc.sh
|
||||
|
||||
This command works if texlive package has been installed on the system. It can be done by invoking the following command.
|
||||
|
||||
sudo apt install texlive-full
|
||||
|
||||
# QT GUI
|
||||
|
||||
## QT GUI modification
|
||||
|
||||
Open the ".ui" file with `qtcreator` and modify the gui. Then regenerate the python code.
|
||||
|
||||
On linux, a helper script has been created:
|
||||
scripts/./qt_generate.sh
|
||||
|
||||
# Debugging
|
||||
|
||||
In order to debug testium or your python script executed within testium.
|
||||
|
||||
## In VSCODE
|
||||
|
||||
This is the prefered method :
|
||||
|
||||
1. Create a debug configuration like the following:
|
||||
|
||||
```
|
||||
"configurations": [
|
||||
{
|
||||
"name": "Python : testium",
|
||||
"type": "python",
|
||||
"request": "launch",
|
||||
"program": "${workspaceFolder}/src/testium",
|
||||
"console": "integratedTerminal",
|
||||
"args": ["-g"],
|
||||
"justMyCode": true
|
||||
},
|
||||
]
|
||||
```
|
||||
|
||||
2. Install debugpy module in python
|
||||
|
||||
python -m pip install debugpy
|
||||
3. Then get to the "RUN AND DEBUG" tab and press the play button.
|
||||
4. A testium window will pops up ; start execution of your tum.
|
||||
5. Do not forget to put breakpoints where you want to investigate.
|
||||
|
||||
## Icons
|
||||
|
||||
Icons are coming from the following site: https://github.com/free-icons/free-icons.git
|
||||
|
||||
# testium Release
|
||||
|
||||
## Pre-requisite
|
||||
|
||||
A `python` virtual environment must have been set as described above.
|
||||
|
||||
### Install pyinstaller
|
||||
|
||||
Install `pyinstaller` package using pip.
|
||||
|
||||
## Generate the binary package
|
||||
|
||||
The procedure for a binary release is as follows:
|
||||
|
||||
1. update the `release_note.txt` file
|
||||
2. modify the version in `src/VERSION` file
|
||||
3. be sure that the documentation is up to date, and if not execute `doc/manual/sphinx/build_doc.sh` script
|
||||
4. push modifications and create a tag with the new version on the git repository
|
||||
5. generate an executable file by calling `package/pyinstaller/./build.sh`
|
||||
6. run the complete validation test for each generated binary
|
||||
7. check that all the validation results are OK
|
||||
|
||||
# Troubleshooting
|
||||
|
||||
## The testium exe crashes `wl_proxy_marshal_flags`
|
||||
|
||||
### Error message
|
||||
|
||||
/testium: symbol lookup error: /tmp/_MEIOhDCPF/libQt6WaylandClient.so.6: undefined symbol: wl_proxy_marshal_flags
|
||||
|
||||
### Solution
|
||||
|
||||
Set the appropriate environment variable
|
||||
|
||||
export QT_QPA_PLATFORM=xcb
|
||||
testium
|
||||
|
||||
## xcb plugin missing
|
||||
|
||||
### Error message
|
||||
|
||||
qt.qpa.plugin: Could not load the Qt platform plugin "xcb" in "" even though it was found.
|
||||
|
||||
### Solution
|
||||
|
||||
A package is missing
|
||||
|
||||
sudo apt install libxcb-cursor0
|
||||
sudo apt-get install libicu-dev
|
||||
sudo apt-get install libxcb-cursor-dev
|
||||
|
||||
## The testium appimage crashes when opening a file
|
||||
|
||||
This is usually because wayland is defined as the default X server.
|
||||
|
||||
To change it :
|
||||
|
||||
* Disable Wayland by uncommenting WaylandEnable=false in the `/etc/gdm3/daemon.conf`
|
||||
* Add `QT_QPA_PLATFORM=xcb` in `/etc/environment`
|
||||
* After a reboot, check that the environment variable value returns `x11`:
|
||||
|
||||
$ echo $XDG_SESSION_TYPE
|
||||
x11
|
||||
Contributions are accepted under the same licence (inbound = outbound).
|
||||
See [`CONTRIBUTING.md`](CONTRIBUTING.md) for development setup, debugging
|
||||
workflow, and the release procedure.
|
||||
|
||||
164
build_all.sh
Executable file
164
build_all.sh
Executable file
@@ -0,0 +1,164 @@
|
||||
#!/bin/bash
|
||||
# Build every distribution channel of testium, in order:
|
||||
# 1. Manual PDF -> dist/testium-manual-<v>.pdf
|
||||
# 2. Wheel -> dist/testium-<v>-py3-none-any.whl (PEP 427 name)
|
||||
# 3. PyInstaller binary -> dist/testium-<v>
|
||||
# 4. Flatpak bundle -> dist/testium-<v>.flatpak
|
||||
# 5. AppImage -> dist/Testium-<v>-x86_64.AppImage (original name)
|
||||
# release_note.txt is copied to dist/ up front (with a warning if it has no
|
||||
# entry for the current version).
|
||||
#
|
||||
# By default, a step is skipped if its artifact already exists in dist/.
|
||||
# Pass --clean to remove existing dist/ artifacts and rebuild everything.
|
||||
#
|
||||
# All artifacts are collected (copied) under <repo>/dist/. Original outputs in
|
||||
# src/dist/, package/*/dist/, doc/manual/ are left in place. Wheel and AppImage
|
||||
# keep their original names (which already contain the version); manual,
|
||||
# pyinstaller and flatpak are renamed to testium(-manual)-<version>(.suff).
|
||||
#
|
||||
# Re-uses scripts/build_env.sh and scripts/set_env.sh — the same pair invoked
|
||||
# by run.sh — so the venv at test/tmp/.venv stays the single source of Python
|
||||
# dependencies. `build`, `pyinstaller`, `sphinx` and `linuxdoc` are installed
|
||||
# into that venv on demand if not already there. Flatpak and AppImage build in
|
||||
# their own container/sandbox; their build.sh scripts have their own toolchain
|
||||
# checks.
|
||||
|
||||
set -e
|
||||
|
||||
CLEAN=0
|
||||
for arg in "$@"; do
|
||||
case "$arg" in
|
||||
--clean|-c) CLEAN=1 ;;
|
||||
*) echo "Unknown option: $arg" >&2; exit 1 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
SCRIPT_DIR=$(realpath "$(dirname "$0")")
|
||||
VERSION=$(cat "$SCRIPT_DIR/src/VERSION")
|
||||
DIST_DIR="$SCRIPT_DIR/dist"
|
||||
mkdir -p "$DIST_DIR"
|
||||
|
||||
if [ "$CLEAN" -eq 1 ]; then
|
||||
echo "-- clean: removing existing dist artifacts for version $VERSION"
|
||||
rm -f "$DIST_DIR/testium-manual-${VERSION}.pdf"
|
||||
rm -f "$DIST_DIR"/testium-${VERSION}-*.whl
|
||||
rm -f "$DIST_DIR/testium-${VERSION}"
|
||||
rm -f "$DIST_DIR/testium-${VERSION}.flatpak"
|
||||
rm -f "$DIST_DIR"/Testium-${VERSION}-*.AppImage
|
||||
fi
|
||||
|
||||
# Release note: copy it to dist/ and warn (but don't fail) if it has no entry
|
||||
# for the current version.
|
||||
RELEASE_NOTE_SRC="$SCRIPT_DIR/release_note.txt"
|
||||
RELEASE_NOTE="$DIST_DIR/release_note.txt"
|
||||
cp -f "$RELEASE_NOTE_SRC" "$RELEASE_NOTE"
|
||||
if ! grep -qE "^version $VERSION([^.0-9]|$)" "$RELEASE_NOTE_SRC"; then
|
||||
echo "WARNING: release_note.txt has no entry for version $VERSION." >&2
|
||||
fi
|
||||
|
||||
export PY_VENV_NAME=".venv"
|
||||
export PY_VENV_DIR="$SCRIPT_DIR/test/tmp/$PY_VENV_NAME"
|
||||
export REQ_PATH="$SCRIPT_DIR/src/requirements.txt"
|
||||
|
||||
bash "$SCRIPT_DIR/scripts/build_env.sh"
|
||||
source "$SCRIPT_DIR/scripts/set_env.sh"
|
||||
|
||||
step() {
|
||||
echo
|
||||
echo "================================================================"
|
||||
echo " $1"
|
||||
echo "================================================================"
|
||||
}
|
||||
|
||||
skip() { echo " (already built — skipping)"; }
|
||||
|
||||
# 1. Manual PDF
|
||||
MANUAL="$DIST_DIR/testium-manual-${VERSION}.pdf"
|
||||
step "1/5 Manual PDF (version $VERSION)"
|
||||
if [ ! -f "$MANUAL" ]; then
|
||||
python -m pip install --quiet --upgrade sphinx linuxdoc
|
||||
bash "$SCRIPT_DIR/doc/manual/sphinx/build_doc.sh"
|
||||
cp -f "$SCRIPT_DIR/doc/manual/testium_manual.pdf" "$MANUAL"
|
||||
else
|
||||
skip
|
||||
fi
|
||||
|
||||
# 2. Wheel — PEP 427 name kept (already contains version)
|
||||
step "2/5 Wheel (version $VERSION)"
|
||||
WHEEL=$(ls -1t "$DIST_DIR"/testium-${VERSION}-*.whl 2>/dev/null | head -1)
|
||||
if [ -z "$WHEEL" ]; then
|
||||
python -m pip install --quiet --upgrade build
|
||||
(
|
||||
cd "$SCRIPT_DIR/src"
|
||||
rm -rf dist build *.egg-info
|
||||
python -m build --wheel
|
||||
)
|
||||
WHEEL_SRC=$(ls -1t "$SCRIPT_DIR/src/dist"/*.whl | head -1)
|
||||
WHEEL="$DIST_DIR/$(basename "$WHEEL_SRC")"
|
||||
cp -f "$WHEEL_SRC" "$WHEEL"
|
||||
else
|
||||
skip
|
||||
fi
|
||||
|
||||
# 3. PyInstaller binary
|
||||
PYI_BIN="$DIST_DIR/testium-${VERSION}"
|
||||
step "3/5 PyInstaller binary (version $VERSION)"
|
||||
if [ ! -f "$PYI_BIN" ]; then
|
||||
python -m pip install --quiet --upgrade pyinstaller
|
||||
bash "$SCRIPT_DIR/package/pyinstaller/build.sh"
|
||||
cp -f "$SCRIPT_DIR/package/pyinstaller/dist/testium" "$PYI_BIN"
|
||||
else
|
||||
skip
|
||||
fi
|
||||
|
||||
# 4. Flatpak bundle
|
||||
FLATPAK_BUNDLE="$DIST_DIR/testium-${VERSION}.flatpak"
|
||||
step "4/5 Flatpak bundle (version $VERSION)"
|
||||
if [ ! -f "$FLATPAK_BUNDLE" ]; then
|
||||
FLATPAK_DEPS=(
|
||||
"org.kde.Platform//6.10"
|
||||
"org.kde.Sdk//6.10"
|
||||
"io.qt.PySide.BaseApp//6.10"
|
||||
)
|
||||
if ! flatpak remotes --user | grep -q "^flathub"; then
|
||||
echo " Adding Flathub remote"
|
||||
flatpak remote-add --user --if-not-exists flathub https://dl.flathub.org/repo/flathub.flatpakrepo
|
||||
fi
|
||||
for dep in "${FLATPAK_DEPS[@]}"; do
|
||||
if ! flatpak info --user "$dep" &>/dev/null && ! flatpak info --system "$dep" &>/dev/null; then
|
||||
echo " Installing Flatpak dependency: $dep"
|
||||
flatpak install --user --noninteractive flathub "$dep"
|
||||
fi
|
||||
done
|
||||
(
|
||||
cd "$SCRIPT_DIR/package/flatpak"
|
||||
bash build.sh
|
||||
)
|
||||
cp -f "$SCRIPT_DIR/package/flatpak/testium.flatpak" "$FLATPAK_BUNDLE"
|
||||
else
|
||||
skip
|
||||
fi
|
||||
|
||||
# 5. AppImage
|
||||
step "5/5 AppImage (version $VERSION)"
|
||||
APPIMAGE=$(ls -1t "$DIST_DIR"/Testium-${VERSION}-*.AppImage 2>/dev/null | head -1)
|
||||
if [ -z "$APPIMAGE" ]; then
|
||||
(
|
||||
cd "$SCRIPT_DIR/package/appimage"
|
||||
bash build.sh
|
||||
)
|
||||
APPIMAGE_SRC=$(ls -1t "$SCRIPT_DIR/package/appimage"/*.AppImage 2>/dev/null | head -1)
|
||||
APPIMAGE="$DIST_DIR/$(basename "$APPIMAGE_SRC")"
|
||||
cp -f "$APPIMAGE_SRC" "$APPIMAGE"
|
||||
chmod +x "$APPIMAGE"
|
||||
else
|
||||
skip
|
||||
fi
|
||||
|
||||
step "All packages built"
|
||||
printf " manual : %s\n" "$MANUAL"
|
||||
printf " wheel : %s\n" "$WHEEL"
|
||||
printf " pyinstaller : %s\n" "$PYI_BIN"
|
||||
printf " flatpak : %s\n" "$FLATPAK_BUNDLE"
|
||||
printf " appimage : %s\n" "$APPIMAGE"
|
||||
printf " release_note : %s\n" "$RELEASE_NOTE"
|
||||
@@ -20,6 +20,22 @@ main:
|
||||
param:
|
||||
- 123
|
||||
|
||||
- py_func:
|
||||
name: python long wait
|
||||
doc: The purpose of this step is to try the tasks "stop" interruption
|
||||
file: utils.py
|
||||
func_name: long_wait
|
||||
param:
|
||||
- 10
|
||||
|
||||
- lua_func:
|
||||
name: lua long wait
|
||||
doc: The purpose of this step is to try the tasks "stop" interruption
|
||||
file: lua_func.lua
|
||||
func_name: long_wait
|
||||
param:
|
||||
- 10
|
||||
|
||||
- sleep:
|
||||
name: sleep item
|
||||
dialog: true
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
tm = require("tm")
|
||||
socket = require("socket")
|
||||
|
||||
local module = {}
|
||||
|
||||
@@ -7,4 +8,8 @@ function module.func_to_be_executed(param)
|
||||
return param
|
||||
end
|
||||
|
||||
function module.long_wait(sec)
|
||||
socket.sleep(sec)
|
||||
end
|
||||
|
||||
return module
|
||||
@@ -17,18 +17,3 @@ plot_log_path: /tmp/testium_plot/$(testrun_date)/$(testrun_time)/
|
||||
python_path_Windows: C:\Users\François\Applications\Python313\python.exe
|
||||
python_path_Linux: $(home)/tmp/tum_venv/bin/python3
|
||||
|
||||
# lua_bin_Windows: C:\Lua\5.1
|
||||
# lua_bin_Linux: /usr/bin/lua
|
||||
|
||||
LUA_PATH_Linux: /usr/share/lua/5.4/?.lua;/usr/local/share/lua/5.4/?.lua;/usr/local/share/lua/5.4/?/init.lua;/usr/share/lua/5.4/?/init.lua;/usr/local/lib/lua/5.4/?.lua;/usr/local/lib/lua/5.4/?/init.lua;/usr/lib/lua/5.4/?.lua;/usr/lib/lua/5.4/?/init.lua;./?.lua;./?/init.lua;/home/francois/.luarocks/share/lua/5.4/?.lua;/home/francois/.luarocks/share/lua/5.4/?/init.lua
|
||||
LUA_CPATH_Linux: /usr/local/lib/lua/5.4/?.so;/usr/lib/lua/5.4/?.so;/usr/local/lib/lua/5.4/loadall.so;/usr/lib/lua/5.4/loadall.so;./?.so;/home/francois/.luarocks/lib/lua/5.4/?.so
|
||||
PATH_Linux:
|
||||
|
||||
LUA_PATH_Windows: ;.\?.lua;C:\Lua\5.1\lua\?.lua;C:\Lua\5.1\lua\?\init.lua;C:\Lua\5.1\?.lua;C:\Lua\5.1\?\init.lua;C:\Lua\5.1\lua\?.luac
|
||||
LUA_CPATH_Windows: .\?.dll;C:\Lua\5.1\?.dll;C:\Lua\5.1\loadall.dll;C:\Lua\5.1\clibs\?.dll;C:\Lua\5.1\clibs\loadall.dll;.\?51.dll;C:\Lua\5.1\?51.dll;C:\Lua\5.1\clibs\?51.dll
|
||||
PATH_Windows: ""
|
||||
|
||||
lua_env:
|
||||
PATH: $(PATH_$(os))
|
||||
LUA_PATH: $(LUA_PATH_$(os))
|
||||
LUA_CPATH: $(LUA_CPATH_$(os))
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
from time import sleep
|
||||
|
||||
def dummy_exit(useless1, useless2):
|
||||
return True
|
||||
|
||||
@@ -11,3 +13,6 @@ def funcToBeExecuted (bla):
|
||||
def funcToBeExecuted2 (bla):
|
||||
print(bla)
|
||||
return blo
|
||||
|
||||
def long_wait (sec):
|
||||
sleep(sec)
|
||||
Binary file not shown.
66
doc/quick_start.md
Normal file
66
doc/quick_start.md
Normal file
@@ -0,0 +1,66 @@
|
||||
# Quick start
|
||||
|
||||
Five minutes from zero to a passing test.
|
||||
|
||||
## Install
|
||||
|
||||
From a checkout of the repository:
|
||||
|
||||
```sh
|
||||
./run.sh --version # Linux
|
||||
run.bat # Windows cmd
|
||||
```
|
||||
|
||||
The wrapper creates a Python virtual environment on first run and verifies
|
||||
testium starts. If you prefer a manual install, see the README.
|
||||
|
||||
## Your first test
|
||||
|
||||
Create `hello.tum`:
|
||||
|
||||
```yaml
|
||||
main:
|
||||
name: hello world
|
||||
steps:
|
||||
- check:
|
||||
name: 1 + 1 makes 2
|
||||
values:
|
||||
- <| 1 + 1 == 2 |>
|
||||
```
|
||||
|
||||
Run it in batch mode:
|
||||
|
||||
```sh
|
||||
./run.sh -b -- hello.tum
|
||||
```
|
||||
|
||||
You should see something like:
|
||||
|
||||
```
|
||||
-----> step "1 + 1 makes 2" started
|
||||
Check passed
|
||||
<----- step "1 + 1 makes 2" finished: PASS
|
||||
Test run success.
|
||||
```
|
||||
|
||||
Replace `==` with `!=` and re-run — the step now ends with **FAIL** and
|
||||
the process exits with code 1.
|
||||
|
||||
## Open it in the GUI
|
||||
|
||||
```sh
|
||||
./run.sh hello.tum
|
||||
```
|
||||
|
||||
The test tree appears in the left panel; click *Run test* in the toolbar.
|
||||
Each item turns green or red live as it executes. Use `F1` on a selected
|
||||
item to open its detail panel.
|
||||
|
||||
## Where to go next
|
||||
|
||||
* [`doc/tutorial.md`](tutorial.md) — a guided walk-through of the most
|
||||
common test items (`py_func`, `let`, `group`, `condition`, `report`).
|
||||
* [`doc/examples/`](examples/) — runnable `.tum` snippets covering one
|
||||
feature each.
|
||||
* [`doc/manual/testium_manual.pdf`](manual/testium_manual.pdf) —
|
||||
full reference manual.
|
||||
223
doc/tutorial.md
Normal file
223
doc/tutorial.md
Normal file
@@ -0,0 +1,223 @@
|
||||
# Tutorial — testing a small Python utility
|
||||
|
||||
This walk-through builds, step by step, a testium campaign that exercises
|
||||
a small Python module. Each section adds one feature; you can follow
|
||||
along by editing a single `.tum` file and re-running it.
|
||||
|
||||
If you have not yet run testium, start with [`quick_start.md`](quick_start.md).
|
||||
|
||||
## The code under test
|
||||
|
||||
Create `calc.py` next to your `.tum` file:
|
||||
|
||||
```python
|
||||
def add(a, b):
|
||||
return a + b
|
||||
|
||||
def divide(a, b):
|
||||
return a / b
|
||||
```
|
||||
|
||||
## Step 1 — a static check
|
||||
|
||||
The simplest item is `check`: it evaluates an expression and the test
|
||||
passes iff the expression is truthy. Create `tutorial.tum`:
|
||||
|
||||
```yaml
|
||||
main:
|
||||
name: calc.py campaign
|
||||
steps:
|
||||
- check:
|
||||
name: addition is correct
|
||||
values:
|
||||
- <| 2 + 3 == 5 |>
|
||||
```
|
||||
|
||||
The `<| ... |>` markers turn the body into a Python expression evaluated
|
||||
at run time. Run it:
|
||||
|
||||
```sh
|
||||
./run.sh -b -- tutorial.tum
|
||||
```
|
||||
|
||||
## Step 2 — call your code with `py_func`
|
||||
|
||||
`check` only sees Python literals; to exercise `calc.py` we need a
|
||||
`py_func` item. Replace the step:
|
||||
|
||||
```yaml
|
||||
- py_func:
|
||||
name: add 2 and 3
|
||||
file: calc.py
|
||||
func_name: add
|
||||
param: [2, 3]
|
||||
expected_result: 5
|
||||
```
|
||||
|
||||
`expected_result` makes the item PASS only when the function returns
|
||||
exactly that value.
|
||||
|
||||
The result is also stored in the global dict under `pfn_<name>`
|
||||
(here `pfn_add 2 and 3`).
|
||||
|
||||
Anywhere in a `.tum`, `$(key)` is replaced at runtime by the value
|
||||
stored in the global dict under `key`. A subsequent step can read the
|
||||
result back with `$(pfn_<name>)`:
|
||||
|
||||
```yaml
|
||||
- check:
|
||||
name: result was 5
|
||||
values:
|
||||
- <| $(pfn_add 2 and 3) == 5 |>
|
||||
```
|
||||
|
||||
## Step 3 — group several checks
|
||||
|
||||
Wrap the steps in a `group` to keep them visually together and let
|
||||
testium report a per-group status:
|
||||
|
||||
```yaml
|
||||
main:
|
||||
name: calc.py campaign
|
||||
steps:
|
||||
- group:
|
||||
name: add
|
||||
steps:
|
||||
- py_func:
|
||||
name: 2 + 3
|
||||
file: calc.py
|
||||
func_name: add
|
||||
param: [2, 3]
|
||||
expected_result: 5
|
||||
- py_func:
|
||||
name: -1 + 1
|
||||
file: calc.py
|
||||
func_name: add
|
||||
param: [-1, 1]
|
||||
expected_result: 0
|
||||
- group:
|
||||
name: divide
|
||||
steps:
|
||||
- py_func:
|
||||
name: 6 / 2
|
||||
file: calc.py
|
||||
func_name: divide
|
||||
param: [6, 2]
|
||||
expected_result: 3.0
|
||||
```
|
||||
|
||||
A group fails as soon as one of its steps fails (set
|
||||
`stop_on_failure: false` to keep going).
|
||||
|
||||
## Step 4 — define a variable with `let`
|
||||
|
||||
Avoid hard-coding the same number twice with a variable:
|
||||
|
||||
```yaml
|
||||
- let:
|
||||
name: define numerator
|
||||
values:
|
||||
- num: 6
|
||||
- py_func:
|
||||
name: divide num by 2
|
||||
file: calc.py
|
||||
func_name: divide
|
||||
param:
|
||||
- $(num)
|
||||
- 2
|
||||
expected_result: 3.0
|
||||
```
|
||||
|
||||
`$(num)` expands to the global dict entry — when the stored value is a
|
||||
number it is substituted as a number, no need to wrap it in `<| ... |>`.
|
||||
|
||||
## Step 5 — conditional execution
|
||||
|
||||
Skip a step when a condition is false:
|
||||
|
||||
```yaml
|
||||
- py_func:
|
||||
name: divide by zero only on linux
|
||||
condition: <| "$(os)" == "Linux" |>
|
||||
file: calc.py
|
||||
func_name: divide
|
||||
param: [1, 0]
|
||||
```
|
||||
|
||||
Items skipped this way report `SKIP` and do not affect the overall
|
||||
result.
|
||||
|
||||
## Step 6 — generate a report
|
||||
|
||||
Add a `report` block at the root of the file:
|
||||
|
||||
```yaml
|
||||
main:
|
||||
name: calc.py campaign
|
||||
steps:
|
||||
# ... your steps here ...
|
||||
|
||||
report:
|
||||
enabled: true
|
||||
log_stored: true
|
||||
export:
|
||||
- junit:
|
||||
path: ./reports
|
||||
file_name: calc.xml
|
||||
- html:
|
||||
path: ./reports
|
||||
file_name: calc.html
|
||||
```
|
||||
|
||||
The `path` directory must exist before the test runs — testium does not
|
||||
create it. Create it once:
|
||||
|
||||
```sh
|
||||
mkdir -p reports
|
||||
```
|
||||
|
||||
Re-run the test — `./reports/calc.xml` (CI-friendly) and
|
||||
`./reports/calc.html` (human-friendly) are produced. Set
|
||||
`log_stored: true` to include each item's captured stdout.
|
||||
|
||||
## Step 7 — share state between calls
|
||||
|
||||
By default each `py_func` runs in its own short-lived subprocess.
|
||||
To keep state across calls, use `context_id`:
|
||||
|
||||
```yaml
|
||||
- py_func:
|
||||
name: open
|
||||
file: calc.py
|
||||
func_name: open_resource
|
||||
context_id: my_ctx
|
||||
- py_func:
|
||||
name: use
|
||||
file: calc.py
|
||||
func_name: use_resource
|
||||
context_id: my_ctx
|
||||
```
|
||||
|
||||
Both steps share the same persistent Python interpreter, so `calc.py`
|
||||
can store any object in module-level globals or in `tm.setgd()`.
|
||||
|
||||
To share data without `context_id`, write it to the testium global dict
|
||||
via the JSON-RPC bridge:
|
||||
|
||||
```python
|
||||
import py_func.tm as tm
|
||||
|
||||
def producer():
|
||||
tm.setgd("computed", 42)
|
||||
|
||||
def consumer():
|
||||
return tm.gd("computed")
|
||||
```
|
||||
|
||||
## Where to go next
|
||||
|
||||
* [`doc/examples/`](examples/) — one runnable `.tum` per feature
|
||||
(cycles, dialogs, console, plots, parallel, run-of-tum, …).
|
||||
* [`doc/manual/testium_manual.pdf`](manual/testium_manual.pdf) — full
|
||||
reference manual covering every test item, every attribute and the
|
||||
YAML syntax extensions.
|
||||
@@ -24,9 +24,8 @@ AppDir:
|
||||
|
||||
runtime:
|
||||
env:
|
||||
SEQUENCER_REV: '{{APP_VERSION}}'
|
||||
TESTIUM_VERSION: '{{APP_VERSION}}'
|
||||
PYTHONPATH: $APPDIR/usr/lib/python3.11/site-packages:$APPDIR/usr/lib/python3.11
|
||||
QT_QPA_PLATFORM: xcb
|
||||
|
||||
path_mappings:
|
||||
- /usr/share/matplotlib/mpl-data/matplotlibrc:$APPDIR/etc/matplotlibrc
|
||||
@@ -69,12 +68,13 @@ AppDir:
|
||||
|
||||
# Set python 3.11 as default
|
||||
ln -fs python3.11 $TARGET_APPDIR/usr/bin/python3
|
||||
# Install pip
|
||||
if [ ! -f "get-pip.py" ]; then curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py; fi
|
||||
|
||||
# Bootstrap pip into the AppDir Python
|
||||
if [ ! -f "get-pip.py" ]; then curl -sS https://bootstrap.pypa.io/get-pip.py -o get-pip.py; fi
|
||||
python3.11 get-pip.py --break-system-packages
|
||||
|
||||
# Install application dependencies in AppDir
|
||||
python3.11 -m pip install --break-system-packages --upgrade --isolated --no-input --ignore-installed --prefix=$TARGET_APPDIR/usr -r requirements.txt
|
||||
python3.11 -m pip install --break-system-packages --upgrade --isolated --no-input --ignore-installed --prefix=$TARGET_APPDIR/usr -r ../../src/requirements.txt
|
||||
|
||||
export PIP_CONFIG_FILE=$HOME/.pip/pip.conf
|
||||
python3.11 -m pip install --break-system-packages --upgrade --isolated --no-input --ignore-installed --prefix=$TARGET_APPDIR/usr ../../src/dist/testium-{{APP_VERSION}}-py3-none-any.whl
|
||||
|
||||
@@ -1,12 +1,54 @@
|
||||
#!/usr/bin/bash
|
||||
#!/bin/bash
|
||||
# Build the testium AppImage inside a Debian container (Podman or Docker).
|
||||
# The resulting .AppImage file is written to this directory.
|
||||
|
||||
export APP_VERSION=$(<../../src/VERSION)
|
||||
set -e
|
||||
|
||||
appimage-builder --recipe AppImageBuilder.yml
|
||||
REPO_ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
|
||||
APP_VERSION="$(<"$REPO_ROOT/src/VERSION")"
|
||||
|
||||
RESULT=$?
|
||||
if [ -n "$1" ] && [ "$1" = "install" ]; then
|
||||
if [ $RESULT -eq 0 ]; then
|
||||
install -v "testium-${APP_VERSION}-x86_64.AppImage" "${HOME}/.local/bin/testium"
|
||||
fi
|
||||
if command -v podman &>/dev/null; then
|
||||
RUNTIME=podman
|
||||
elif command -v docker &>/dev/null; then
|
||||
RUNTIME=docker
|
||||
else
|
||||
echo "Error: neither podman nor docker found." >&2
|
||||
exit 1
|
||||
fi
|
||||
echo "Using $RUNTIME — building testium $APP_VERSION AppImage..."
|
||||
|
||||
# APPIMAGE_EXTRACT_AND_RUN=1 lets appimagetool run without FUSE in the container.
|
||||
$RUNTIME run --rm \
|
||||
--privileged \
|
||||
-e APPIMAGE_EXTRACT_AND_RUN=1 \
|
||||
-v "$REPO_ROOT:/work" \
|
||||
-w /work/package/appimage \
|
||||
debian:bookworm bash -c "
|
||||
set -e
|
||||
export DEBIAN_FRONTEND=noninteractive
|
||||
|
||||
apt-get update -qq
|
||||
apt-get install -y -qq \
|
||||
python3 python3-pip python3-venv python3-build \
|
||||
dpkg-dev fakeroot squashfs-tools wget curl file binutils \
|
||||
libglib2.0-0 patchelf zsync > /dev/null
|
||||
|
||||
# Build the wheel
|
||||
cd /work/src
|
||||
python3 -m build --wheel --outdir dist/ > /dev/null
|
||||
cd /work/package/appimage
|
||||
|
||||
# Install appimage-builder
|
||||
pip3 install appimage-builder --quiet --break-system-packages
|
||||
|
||||
# Run the build
|
||||
export APP_VERSION=$APP_VERSION
|
||||
appimage-builder --recipe AppImageBuilder.yml --skip-test
|
||||
"
|
||||
|
||||
APPIMAGE_FILE=$(ls -1t Testium-*-x86_64.AppImage 2>/dev/null | head -1)
|
||||
echo "Done: ${APPIMAGE_FILE}"
|
||||
|
||||
if [ "${1}" = "install" ] && [ -n "${APPIMAGE_FILE}" ]; then
|
||||
install -v "${APPIMAGE_FILE}" "${HOME}/.local/bin/testium"
|
||||
fi
|
||||
|
||||
@@ -1,3 +0,0 @@
|
||||
tables
|
||||
pandas
|
||||
scapy
|
||||
@@ -5,4 +5,22 @@
|
||||
# flatpak install flathub org.kde.Sdk//6.10
|
||||
# flatpak install flathub io.qt.PySide.BaseApp//6.10
|
||||
|
||||
flatpak-builder --user --verbose --force-clean --install build org.testium.Testium.yaml
|
||||
set -e
|
||||
|
||||
# Build + install local
|
||||
flatpak-builder --user --verbose --force-clean --install --repo=repo build org.testium.Testium.yaml
|
||||
|
||||
# Génère le bundle distribuable
|
||||
flatpak build-bundle repo testium.flatpak org.testium.Testium
|
||||
echo "Bundle généré : $(pwd)/testium.flatpak"
|
||||
|
||||
# Crée ~/.local/bin/testium pour pouvoir taper "testium" en console
|
||||
WRAPPER="$HOME/.local/bin/testium"
|
||||
mkdir -p "$HOME/.local/bin"
|
||||
cat > "$WRAPPER" <<'EOF'
|
||||
#!/bin/sh
|
||||
exec flatpak run org.testium.Testium "$@"
|
||||
EOF
|
||||
chmod +x "$WRAPPER"
|
||||
echo "Wrapper installé : $WRAPPER"
|
||||
echo "Assurez-vous que ~/.local/bin est dans votre PATH."
|
||||
|
||||
7
package/flatpak/org.testium.Testium-mime.xml
Normal file
7
package/flatpak/org.testium.Testium-mime.xml
Normal file
@@ -0,0 +1,7 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<mime-info xmlns="http://www.freedesktop.org/standards/shared-mime-info">
|
||||
<mime-type type="application/x-testium">
|
||||
<comment>Testium test script</comment>
|
||||
<glob pattern="*.tum"/>
|
||||
</mime-type>
|
||||
</mime-info>
|
||||
10
package/flatpak/org.testium.Testium.desktop
Normal file
10
package/flatpak/org.testium.Testium.desktop
Normal file
@@ -0,0 +1,10 @@
|
||||
[Desktop Entry]
|
||||
Name=Testium
|
||||
GenericName=Test Sequencer
|
||||
Comment=YAML-based test sequencer and runner
|
||||
Exec=testium %f
|
||||
Icon=org.testium.Testium
|
||||
Type=Application
|
||||
Categories=Development;
|
||||
MimeType=application/x-testium;
|
||||
StartupNotify=true
|
||||
@@ -13,7 +13,14 @@ finish-args:
|
||||
- --socket=wayland
|
||||
- --device=dri
|
||||
- --share=network
|
||||
- --filesystem=home # Optionnel : si votre testium doit lire des fichiers utilisateurs
|
||||
- --filesystem=home
|
||||
- --filesystem=/tmp
|
||||
- --filesystem=host-os
|
||||
# Allow flatpak-spawn --host to launch host binaries (Python, Lua, git…)
|
||||
# outside the sandbox. Required because the sandbox glibc/ABI is
|
||||
# incompatible with arbitrary host shared libraries — we route py_func and
|
||||
# lua_func through the host instead.
|
||||
- --talk-name=org.freedesktop.Flatpak
|
||||
|
||||
build-options:
|
||||
build-args:
|
||||
@@ -41,18 +48,41 @@ modules:
|
||||
sources:
|
||||
- type: dir
|
||||
path: ../../src
|
||||
- type: file
|
||||
path: org.testium.Testium.desktop
|
||||
- type: file
|
||||
path: org.testium.Testium-mime.xml
|
||||
- type: file
|
||||
path: ../../package/testium.png
|
||||
build-commands:
|
||||
# On installe le code source dans /app/lib/testium
|
||||
# Code source
|
||||
- mkdir -p /app/lib
|
||||
- cp -r . /app/lib/
|
||||
- cp -r testium /app/lib/
|
||||
- cp VERSION /app/lib/testium/VERSION
|
||||
|
||||
# Création du launcher exécutable
|
||||
# Launcher exécutable
|
||||
- mkdir -p /app/bin
|
||||
- |
|
||||
cat <<EOF > /app/bin/testium
|
||||
#!/bin/sh
|
||||
# On ajoute le code source et l'extension PySide6 au PYTHONPATH
|
||||
export TESTIUM_VERSION="\$(cat /app/lib/testium/VERSION 2>/dev/null || echo unknown)"
|
||||
export PYTHONPATH="/app/lib/testium:/usr/lib/sdk/pyside6/lib/python3.13/site-packages:\$PYTHONPATH"
|
||||
exec python3 /app/lib/testium "\$@"
|
||||
# Expose host binaries (git, python3, lua, …) for subprocess lookups.
|
||||
# PATH is appended (not prepended) so the main process keeps the sandbox python3.
|
||||
export PATH="\$PATH:/run/host/usr/local/bin:/run/host/usr/bin:/run/host/bin"
|
||||
export GIT_PYTHON_GIT_EXECUTABLE="/run/host/usr/bin/git"
|
||||
exec /usr/bin/python3 /app/lib/testium "\$@"
|
||||
EOF
|
||||
- chmod +x /app/bin/testium
|
||||
|
||||
# Icône
|
||||
- mkdir -p /app/share/icons/hicolor/256x256/apps
|
||||
- cp testium.png /app/share/icons/hicolor/256x256/apps/org.testium.Testium.png
|
||||
|
||||
# Entrée menu
|
||||
- mkdir -p /app/share/applications
|
||||
- cp org.testium.Testium.desktop /app/share/applications/
|
||||
|
||||
# Type MIME pour .tum
|
||||
- mkdir -p /app/share/mime/packages
|
||||
- cp org.testium.Testium-mime.xml /app/share/mime/packages/
|
||||
|
||||
@@ -1,3 +1,55 @@
|
||||
version 0.2
|
||||
==============
|
||||
- Test items: each item type now declares its accepted parameters
|
||||
(``PARAMS = ParamSet(...)``). Typos in a ``.tum`` are surfaced as a
|
||||
WARN listing the accepted names instead of being silently ignored;
|
||||
missing required parameters error out at load time with the source
|
||||
``.tum`` file as context. No change to valid existing tests.
|
||||
|
||||
version 0.1.3
|
||||
==============
|
||||
- Stop interrupts engaged blocking steps (console, py_func, lua_func,
|
||||
json_rpc, sleep) within ~200 ms instead of waiting for the step
|
||||
to finish.
|
||||
- GUI Start / Stop / Pause flow simplified.
|
||||
- lua_func: a function returning nil is no longer reported as a failure.
|
||||
- ``-d python_bin=...`` and the GUI ``python_bin`` preference now reach
|
||||
the eval subprocess (used to be silently ignored). ``param.yaml`` can
|
||||
also override ``python_bin`` for py_func / cycle / post_exec.
|
||||
- Validation suite: ``test/validation/run.sh`` (and ``run.bat``)
|
||||
runs the suite inside a dedicated venv in the system temp dir.
|
||||
- build_all.sh: ``release_note.txt`` and the user manual copied into
|
||||
``dist/``; warning if the file has no entry for the version being built.
|
||||
- Flatpak: every GUI file/directory dialog (open test, save report, log
|
||||
path, default report/log dirs, python/lua interpreter pickers) now
|
||||
bypasses the XDG document portal — the v0.1.2 fix was only on the
|
||||
"open test" dialog.
|
||||
- Flatpak: py_func / lua_func / run sub-instance now execute on the host
|
||||
via flatpak-spawn, lifting the previous glibc/ABI incompatibility that
|
||||
prevented user-configured host Python or Lua interpreters from being
|
||||
reached from the sandbox.
|
||||
- Validation suite: single entry point with ``--mode source|wheel|
|
||||
pyinstaller|flatpak|appimage`` to validate any packaging channel
|
||||
against the same item set; reports are stamped per mode.
|
||||
- GUI: the "Run tum" test item now uses the testium logo.
|
||||
|
||||
version 0.1.2
|
||||
==============
|
||||
- Flatpak: opening a test from the GUI now correctly finds its companion
|
||||
files (param.yaml, .py scripts, ...).
|
||||
|
||||
version 0.1.1
|
||||
==============
|
||||
- New install channels: Flatpak bundle and AppImage. The AppImage runs
|
||||
on any distribution (built inside a Debian container).
|
||||
- About dialog: version is now correct in Flatpak and AppImage builds
|
||||
(used to display "unknown").
|
||||
- GUI dialogs no longer hang on pure-Wayland sessions.
|
||||
- Plot "last values" API: more tolerant timeout on loaded machines.
|
||||
- run item: `testium_path` and `python_bin` parameters removed —
|
||||
sub-instances are launched in the same packaging mode as the parent.
|
||||
- License: EUPL-1.2.
|
||||
|
||||
version 0.1
|
||||
==============
|
||||
- Start of the project
|
||||
|
||||
@@ -20,6 +20,12 @@ if [ "$?" -ne 0 ]; then
|
||||
echo "venv must be installed on the host distribution."
|
||||
exit -1
|
||||
fi
|
||||
# Check if venv is installed
|
||||
python3 -c "import ensurepip"
|
||||
if [ "$?" -ne 0 ]; then
|
||||
echo "ensurepip must be installed on the host distribution."
|
||||
exit -1
|
||||
fi
|
||||
|
||||
# Install the virtual environment if needed
|
||||
if [ ! -d "$PY_VENV_DIR" ]; then
|
||||
|
||||
315
src/LICENSE
Normal file
315
src/LICENSE
Normal file
@@ -0,0 +1,315 @@
|
||||
Copyright (c) 2025-2026 François Dausseur
|
||||
|
||||
Licensed under the EUPL
|
||||
|
||||
|
||||
EUROPEAN UNION PUBLIC LICENCE v. 1.2
|
||||
EUPL © the European Union 2007, 2016
|
||||
|
||||
This European Union Public Licence (the 'EUPL') applies to the Work (as
|
||||
defined below) which is provided under the terms of this Licence. Any use of
|
||||
the Work, other than as authorised under this Licence is prohibited (to the
|
||||
extent such use is covered by a right of the copyright holder of the Work).
|
||||
|
||||
The Work is provided under the terms of this Licence when the Licensor (as
|
||||
defined below) has placed the following notice immediately following the
|
||||
copyright notice for the Work:
|
||||
|
||||
Licensed under the EUPL
|
||||
|
||||
or has expressed by any other means his willingness to license under the EUPL.
|
||||
|
||||
|
||||
1. Definitions
|
||||
|
||||
In this Licence, the following terms have the following meaning:
|
||||
|
||||
- 'The Licence': this Licence.
|
||||
|
||||
- 'The Original Work': the work or software distributed or communicated by the
|
||||
Licensor under this Licence, available as Source Code and also as Executable
|
||||
Code as the case may be.
|
||||
|
||||
- 'Derivative Works': the works or software that could be created by the
|
||||
Licensee, based upon the Original Work or modifications thereof. This Licence
|
||||
does not define the extent of modification or dependence on the Original
|
||||
Work required in order to classify a work as a Derivative Work; this extent
|
||||
is determined by copyright law applicable in the country mentioned in
|
||||
Article 15.
|
||||
|
||||
- 'The Work': the Original Work or its Derivative Works.
|
||||
|
||||
- 'The Source Code': the human-readable form of the Work which is the most
|
||||
convenient for people to study and modify.
|
||||
|
||||
- 'The Executable Code': any code which has generally been compiled and which
|
||||
is meant to be interpreted by a computer as a program.
|
||||
|
||||
- 'The Licensor': the natural or legal person that distributes or communicates
|
||||
the Work under the Licence.
|
||||
|
||||
- 'Contributor(s)': any natural or legal person who modifies the Work under
|
||||
the Licence, or otherwise contributes to the creation of a Derivative Work.
|
||||
|
||||
- 'The Licensee' or 'You': any natural or legal person who makes any usage of
|
||||
the Work under the terms of the Licence.
|
||||
|
||||
- 'Distribution' or 'Communication': any act of selling, giving, lending,
|
||||
renting, distributing, communicating, transmitting, or otherwise making
|
||||
available, online or offline, copies of the Work or providing access to its
|
||||
essential functionalities at the disposal of any other natural or legal
|
||||
person.
|
||||
|
||||
|
||||
2. Scope of the rights granted by the Licence
|
||||
|
||||
The Licensor hereby grants You a worldwide, royalty-free, non-exclusive,
|
||||
sublicensable licence to do the following, for the duration of copyright
|
||||
vested in the Original Work:
|
||||
|
||||
- use the Work in any circumstance and for all usage,
|
||||
- reproduce the Work,
|
||||
- modify the Work, and make Derivative Works based upon the Work,
|
||||
- communicate to the public, including the right to make available or display
|
||||
the Work or copies thereof to the public and perform publicly, as the case
|
||||
may be, the Work,
|
||||
- distribute the Work or copies thereof,
|
||||
- lend and rent the Work or copies thereof,
|
||||
- sublicense rights in the Work or copies thereof.
|
||||
|
||||
Those rights can be exercised on any media, supports and formats, whether now
|
||||
known or later invented, as far as the applicable law permits so.
|
||||
|
||||
In the countries where moral rights apply, the Licensor waives his right to
|
||||
exercise his moral right to the extent allowed by law in order to make
|
||||
effective the licence of the economic rights here above listed.
|
||||
|
||||
The Licensor grants to the Licensee royalty-free, non-exclusive usage rights
|
||||
to any patents held by the Licensor, to the extent necessary to make use of
|
||||
the rights granted on the Work under this Licence.
|
||||
|
||||
|
||||
3. Communication of the Source Code
|
||||
|
||||
The Licensor may provide the Work either in its Source Code form, or as
|
||||
Executable Code. If the Work is provided as Executable Code, the Licensor
|
||||
provides in addition a machine-readable copy of the Source Code of the Work
|
||||
along with each copy of the Work that the Licensor distributes or indicates,
|
||||
in a notice following the copyright notice attached to the Work, a repository
|
||||
where the Source Code is easily and freely accessible for as long as the
|
||||
Licensor continues to distribute or communicate the Work.
|
||||
|
||||
|
||||
4. Limitations on copyright
|
||||
|
||||
Nothing in this Licence is intended to deprive the Licensee of the benefits
|
||||
from any exception or limitation to the exclusive rights of the rights owners
|
||||
in the Work, of the exhaustion of those rights or of other applicable
|
||||
limitations thereto.
|
||||
|
||||
|
||||
5. Obligations of the Licensee
|
||||
|
||||
The grant of the rights mentioned above is subject to some restrictions and
|
||||
obligations imposed on the Licensee. Those obligations are the following:
|
||||
|
||||
Attribution right: The Licensee shall keep intact all copyright, patent or
|
||||
trademarks notices and all notices that refer to the Licence and to the
|
||||
disclaimer of warranties. The Licensee must include a copy of such notices
|
||||
and a copy of the Licence with every copy of the Work he/she distributes or
|
||||
communicates. The Licensee must cause any Derivative Work to carry prominent
|
||||
notices stating that the Work has been modified and the date of modification.
|
||||
|
||||
Copyleft clause: If the Licensee distributes or communicates copies of the
|
||||
Original Works or Derivative Works, this Distribution or Communication will
|
||||
be done under the terms of this Licence or of a later version of this Licence
|
||||
unless the Original Work is expressly distributed only under this version of
|
||||
the Licence — for example by communicating 'EUPL v. 1.2 only'. The Licensee
|
||||
(becoming Licensor) cannot offer or impose any additional terms or conditions
|
||||
on the Work or Derivative Work that alter or restrict the terms of the
|
||||
Licence.
|
||||
|
||||
Compatibility clause: If the Licensee Distributes or Communicates Derivative
|
||||
Works or copies thereof based upon both the Work and another work licensed
|
||||
under a Compatible Licence, this Distribution or Communication can be done
|
||||
under the terms of this Compatible Licence. For the sake of this clause,
|
||||
'Compatible Licence' refers to the licences listed in the appendix attached
|
||||
to this Licence. Should the Licensee's obligations under the Compatible
|
||||
Licence conflict with his/her obligations under this Licence, the obligations
|
||||
of the Compatible Licence shall prevail.
|
||||
|
||||
Provision of Source Code: When distributing or communicating copies of the
|
||||
Work, the Licensee will provide a machine-readable copy of the Source Code or
|
||||
indicate a repository where this Source will be easily and freely available
|
||||
for as long as the Licensee continues to distribute or communicate the Work.
|
||||
|
||||
Legal Protection: This Licence does not grant permission to use the trade
|
||||
names, trademarks, service marks, or names of the Licensor, except as
|
||||
required for reasonable and customary use in describing the origin of the
|
||||
Work and reproducing the content of the copyright notice.
|
||||
|
||||
|
||||
6. Chain of Authorship
|
||||
|
||||
The original Licensor warrants that the copyright in the Original Work
|
||||
granted hereunder is owned by him/her or licensed to him/her and that he/she
|
||||
has the power and authority to grant the Licence.
|
||||
|
||||
Each Contributor warrants that the copyright in the modifications he/she
|
||||
brings to the Work are owned by him/her or licensed to him/her and that
|
||||
he/she has the power and authority to grant the Licence.
|
||||
|
||||
Each time You accept the Licence, the original Licensor and subsequent
|
||||
Contributors grant You a licence to their contributions to the Work, under
|
||||
the terms of this Licence.
|
||||
|
||||
|
||||
7. Disclaimer of Warranty
|
||||
|
||||
The Work is a work in progress, which is continuously improved by numerous
|
||||
Contributors. It is not a finished work and may therefore contain defects or
|
||||
'bugs' inherent to this type of development.
|
||||
|
||||
For the above reason, the Work is provided under the Licence on an 'as is'
|
||||
basis and without warranties of any kind concerning the Work, including
|
||||
without limitation merchantability, fitness for a particular purpose, absence
|
||||
of defects or errors, accuracy, non-infringement of intellectual property
|
||||
rights other than copyright as stated in Article 6 of this Licence.
|
||||
|
||||
This disclaimer of warranty is an essential part of the Licence and a
|
||||
condition for the grant of any rights to the Work.
|
||||
|
||||
|
||||
8. Disclaimer of Liability
|
||||
|
||||
Except in the cases of wilful misconduct or damages directly caused to
|
||||
natural persons, the Licensor will in no event be liable for any direct or
|
||||
indirect, material or moral, damages of any kind, arising out of the Licence
|
||||
or of the use of the Work, including without limitation, damages for loss of
|
||||
goodwill, work stoppage, computer failure or malfunction, loss of data or any
|
||||
commercial damage, even if the Licensor has been advised of the possibility
|
||||
of such damage. However, the Licensor will be liable under statutory product
|
||||
liability laws as far such laws apply to the Work.
|
||||
|
||||
|
||||
9. Additional agreements
|
||||
|
||||
While distributing the Work, You may choose to conclude an additional
|
||||
agreement, defining obligations or services consistent with this Licence.
|
||||
However, if accepting obligations, You may act only on your own behalf and on
|
||||
your sole responsibility, not on behalf of the original Licensor or any other
|
||||
Contributor, and only if You agree to indemnify, defend, and hold each
|
||||
Contributor harmless for any liability incurred by, or claims asserted
|
||||
against such Contributor by the fact You have accepted any warranty or
|
||||
additional liability.
|
||||
|
||||
|
||||
10. Acceptance of the Licence
|
||||
|
||||
The provisions of this Licence can be accepted by clicking on an icon 'I
|
||||
agree' placed under the bottom of a window displaying the text of this
|
||||
Licence or by affirming consent in any other similar way, in accordance with
|
||||
the rules of applicable law. Clicking on that icon indicates your clear and
|
||||
irrevocable acceptance of this Licence and all of its terms and conditions.
|
||||
|
||||
Similarly, you irrevocably accept this Licence and all of its terms and
|
||||
conditions by exercising any rights granted to You by Article 2 of this
|
||||
Licence, such as the use of the Work, the creation by You of a Derivative
|
||||
Work or the Distribution or Communication by You of the Work or copies
|
||||
thereof.
|
||||
|
||||
|
||||
11. Information to the public
|
||||
|
||||
In case of any Distribution or Communication of the Work by means of
|
||||
electronic communication by You (for example, by offering to download the
|
||||
Work from a remote location) the distribution channel or media (for example,
|
||||
a website) must at least provide to the public the information requested by
|
||||
the applicable law regarding the Licensor, the Licence and the way it may be
|
||||
accessible, concluded, stored and reproduced by the Licensee.
|
||||
|
||||
|
||||
12. Termination of the Licence
|
||||
|
||||
The Licence and the rights granted hereunder will terminate automatically
|
||||
upon any breach by the Licensee of the terms of the Licence.
|
||||
|
||||
Such a termination will not terminate the licences of any person who has
|
||||
received the Work from the Licensee under the Licence, provided such persons
|
||||
remain in full compliance with the Licence.
|
||||
|
||||
|
||||
13. Miscellaneous
|
||||
|
||||
Without prejudice of Article 9 above, the Licence represents the complete
|
||||
agreement between the Parties as to the Work.
|
||||
|
||||
If any provision of the Licence is invalid or unenforceable under applicable
|
||||
law, this will not affect the validity or enforceability of the Licence as a
|
||||
whole. Such provision will be construed or reformed so as necessary to make
|
||||
it valid and enforceable.
|
||||
|
||||
The European Commission may publish other linguistic versions or new versions
|
||||
of this Licence or updated versions of the Appendix, so far this is required
|
||||
and reasonable, without reducing the scope of the rights granted by the
|
||||
Licence. New versions of the Licence will be published with a unique version
|
||||
number.
|
||||
|
||||
All linguistic versions of this Licence, approved by the European Commission,
|
||||
have identical value. Parties can take advantage of the linguistic version of
|
||||
their choice.
|
||||
|
||||
|
||||
14. Jurisdiction
|
||||
|
||||
Without prejudice to specific agreement between parties,
|
||||
|
||||
- any litigation resulting from the interpretation of this License, arising
|
||||
between the European Union institutions, bodies, offices or agencies, as a
|
||||
Licensor, and any Licensee, will be subject to the jurisdiction of the
|
||||
Court of Justice of the European Union, as laid down in article 272 of the
|
||||
Treaty on the Functioning of the European Union,
|
||||
|
||||
- any litigation arising between other parties and resulting from the
|
||||
interpretation of this License, will be subject to the exclusive
|
||||
jurisdiction of the competent court where the Licensor resides or conducts
|
||||
its primary business.
|
||||
|
||||
|
||||
15. Applicable Law
|
||||
|
||||
Without prejudice to specific agreement between parties,
|
||||
|
||||
- this Licence shall be governed by the law of the European Union Member
|
||||
State where the Licensor has his seat, resides or has his registered
|
||||
office,
|
||||
|
||||
- this licence shall be governed by Belgian law if the Licensor has no seat,
|
||||
residence or registered office inside a European Union Member State.
|
||||
|
||||
|
||||
Appendix
|
||||
|
||||
|
||||
'Compatible Licences' according to Article 5 EUPL are:
|
||||
|
||||
- GNU General Public License (GPL) v. 2, v. 3
|
||||
- GNU Affero General Public License (AGPL) v. 3
|
||||
- Open Software License (OSL) v. 2.1, v. 3.0
|
||||
- Eclipse Public License (EPL) v. 1.0
|
||||
- CeCILL v. 2.0, v. 2.1
|
||||
- Mozilla Public Licence (MPL) v. 2
|
||||
- GNU Lesser General Public Licence (LGPL) v. 2.1, v. 3
|
||||
- Creative Commons Attribution-ShareAlike v. 3.0 Unported (CC BY-SA 3.0) for
|
||||
works other than software
|
||||
- European Union Public Licence (EUPL) v. 1.1, v. 1.2
|
||||
- Québec Free and Open-Source Licence — Reciprocity (LiLiQ-R) or Strong
|
||||
Reciprocity (LiLiQ-R+).
|
||||
|
||||
The European Commission may update this Appendix to later versions of the
|
||||
above licences without producing a new version of the EUPL, as long as they
|
||||
provide the rights granted in Article 2 of this Licence and protect the
|
||||
covered Source Code from exclusive appropriation.
|
||||
|
||||
All other changes or additions to this Appendix require the production of a
|
||||
new EUPL version.
|
||||
@@ -1 +1 @@
|
||||
0.1
|
||||
0.2
|
||||
|
||||
@@ -11,6 +11,7 @@ import threading
|
||||
from telnetlib3 import Telnet, DO, WILL, WONT, TTYPE, IAC, SB, SE, theNULL
|
||||
|
||||
TIMEOUT_NULL = 0.000001
|
||||
STOP_POLL_INTERVAL = 0.2
|
||||
|
||||
|
||||
class BytesStore(object):
|
||||
@@ -123,12 +124,14 @@ A {classname}.close() is missing somewhere in your code !'.format(classname=type
|
||||
# c = ''
|
||||
return c
|
||||
|
||||
def read_until(self, match, timeout=None, return_data=False, mute=False):
|
||||
def read_until(self, match, timeout=None, return_data=False, mute=False, should_stop=None):
|
||||
"""
|
||||
read until the string 'match is found
|
||||
If timeout is not set (None), this function runs indefinitely
|
||||
If timeout is set to zero, this function returns immediately
|
||||
If mute is set to True the characters read from the console will not be displayed
|
||||
If should_stop is a callable, it is polled between reads (every STOP_POLL_INTERVAL
|
||||
at most) and the loop exits early — like a timeout — when it returns True.
|
||||
|
||||
If function fails (because of a timeout) it will return a 'status' integer set to -1
|
||||
otherwise it will return 0.
|
||||
@@ -139,13 +142,6 @@ A {classname}.close() is missing somewhere in your code !'.format(classname=type
|
||||
status = -1
|
||||
if not match:
|
||||
raise ValueError('match parameter can not be empty')
|
||||
# replace all '\r' by '\n' as any '\r' read will undergo the same replacement
|
||||
# match = match.replace('\r\n', '\n')
|
||||
# match = match.replace('\r', '')
|
||||
|
||||
# update the console timeout in conformity with what is required.
|
||||
|
||||
self.set_read_timeout(timeout)
|
||||
|
||||
if timeout is None:
|
||||
timeout = 1000000
|
||||
@@ -159,6 +155,7 @@ A {classname}.close() is missing somewhere in your code !'.format(classname=type
|
||||
# buffer is empty
|
||||
# Otherwise we are waiting for the timeout to rise
|
||||
if timeout < TIMEOUT_NULL:
|
||||
self.set_read_timeout(0)
|
||||
data = self.readchar(0)
|
||||
|
||||
while (status < 0) and ((data is not None) and (data != b'')):
|
||||
@@ -191,39 +188,45 @@ A {classname}.close() is missing somewhere in your code !'.format(classname=type
|
||||
|
||||
# Timeout different than zero
|
||||
else:
|
||||
# Poll in short chunks so a stop request is honored within
|
||||
# STOP_POLL_INTERVAL, regardless of the per-protocol blocking
|
||||
# behavior of readchar().
|
||||
self.set_read_timeout(STOP_POLL_INTERVAL)
|
||||
|
||||
time_is_out = threading.Event()
|
||||
timer = threading.Timer(timeout, lambda: time_is_out.set())
|
||||
timer.start()
|
||||
|
||||
# We are waiting for the timeout to rise
|
||||
try:
|
||||
while (status < 0) and (not time_is_out.is_set()):
|
||||
if should_stop is not None and should_stop():
|
||||
break
|
||||
|
||||
while (status < 0) and (not time_is_out.isSet()):
|
||||
|
||||
data = self.readchar(timeout)
|
||||
if data is not None:
|
||||
data = self._compute_char(data)
|
||||
if data != '':
|
||||
if not mute:
|
||||
self.string_buffer += data
|
||||
read_data += data
|
||||
|
||||
search_deque.append(data)
|
||||
if search_deque == match_deque:
|
||||
timer.cancel()
|
||||
status = 0
|
||||
if (not mute) and (data != '\n'):
|
||||
self.string_buffer += '\n'
|
||||
|
||||
if data == '\n' or (status >= 0):
|
||||
# the datas are written line by line for display optimisation in GUI mode
|
||||
data = self.readchar(STOP_POLL_INTERVAL)
|
||||
if data is not None:
|
||||
data = self._compute_char(data)
|
||||
if data != '':
|
||||
if not mute:
|
||||
self.string_buffer = self.string_buffer.replace('\r\n', '\n')
|
||||
self.string_buffer = self.string_buffer.replace('\r', '')
|
||||
self.stream.write(self.string_buffer)
|
||||
self.string_buffer += data
|
||||
read_data += data
|
||||
|
||||
date_str = str(datetime.now()).split('.')[0].split(' ')[1]
|
||||
self.string_buffer = '[{} {}]'.format(date_str, self.name)
|
||||
search_deque.append(data)
|
||||
if search_deque == match_deque:
|
||||
status = 0
|
||||
if (not mute) and (data != '\n'):
|
||||
self.string_buffer += '\n'
|
||||
|
||||
if data == '\n' or (status >= 0):
|
||||
# the datas are written line by line for display optimisation in GUI mode
|
||||
if not mute:
|
||||
self.string_buffer = self.string_buffer.replace('\r\n', '\n')
|
||||
self.string_buffer = self.string_buffer.replace('\r', '')
|
||||
self.stream.write(self.string_buffer)
|
||||
|
||||
date_str = str(datetime.now()).split('.')[0].split(' ')[1]
|
||||
self.string_buffer = '[{} {}]'.format(date_str, self.name)
|
||||
finally:
|
||||
timer.cancel()
|
||||
|
||||
if return_data:
|
||||
return status, read_data
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
import sys
|
||||
import os
|
||||
import queue
|
||||
import multiprocessing as mp
|
||||
from threading import Timer
|
||||
from time import sleep, monotonic
|
||||
@@ -367,7 +368,7 @@ class RuntimePlot:
|
||||
self.msg_queue_in.get()
|
||||
self.msg_queue_out.put({"command": "last_values"})
|
||||
try:
|
||||
res = self.msg_queue_in.get(timeout=1)
|
||||
except:
|
||||
res = self.msg_queue_in.get(timeout=5)
|
||||
except queue.Empty:
|
||||
raise ETUMRuntimeError(f"Impossible to retrieve the last values of the \"{self.name}\" plot")
|
||||
return res
|
||||
|
||||
@@ -16,6 +16,7 @@ from interpreter.utils.test_init import (
|
||||
env_init,
|
||||
prepare_global,
|
||||
update_global,
|
||||
apply_overrides,
|
||||
set_standard_gd_keys,
|
||||
test_run_init,
|
||||
test_run_header,
|
||||
@@ -210,6 +211,19 @@ class TestProcess(Process):
|
||||
|
||||
env_init()
|
||||
|
||||
# Apply GUI defaults and CLI defines to the global dict
|
||||
# *before* eval_proc starts: bins.python_bin() reads
|
||||
# ``python_bin`` from gd on its very first call (during
|
||||
# eval_process_init) and caches the result. Without this,
|
||||
# ``-d python_bin=...`` and the GUI ``python_bin`` preference
|
||||
# would only take effect for items spawned *after* the cache
|
||||
# was already populated with the auto-discovered interpreter,
|
||||
# i.e. they would silently be ignored for eval_proc itself.
|
||||
# _load_initial_params re-applies the same overrides after
|
||||
# ``prepare_global()`` clears gd, so the gd value stays in
|
||||
# sync with the cached path.
|
||||
apply_overrides(self.__defs, self.__gui_defaults)
|
||||
|
||||
# Creation of the python evaluation process for loading of the complete test
|
||||
eval_proc = eval_process_init(api_request, 10, test_dir)
|
||||
eval_proc.start()
|
||||
|
||||
@@ -10,6 +10,8 @@ import os
|
||||
def setup():
|
||||
"""Configure the Qt environment for dialog subprocess usage."""
|
||||
if sys.platform.startswith('linux'):
|
||||
# On Linux/Wayland, force X11 (via XWayland) to avoid crashes
|
||||
# when Qt is initialized inside a multiprocessing subprocess.
|
||||
os.environ['QT_QPA_PLATFORM'] = 'xcb'
|
||||
if os.environ.get('DISPLAY'):
|
||||
# X11 available: force xcb to avoid crashes in multiprocessing subprocesses.
|
||||
os.environ['QT_QPA_PLATFORM'] = 'xcb'
|
||||
elif os.environ.get('WAYLAND_DISPLAY'):
|
||||
os.environ['QT_QPA_PLATFORM'] = 'wayland'
|
||||
|
||||
@@ -5,6 +5,9 @@ from copy import deepcopy
|
||||
from interpreter.test_items.test_result import TestResult, TestValue
|
||||
import api.testium as tm
|
||||
from interpreter.utils.params import TestItemParams
|
||||
from interpreter.utils.param_decl import (
|
||||
Param, ParamSet, LIST, BLOCK, unknown_keys, missing_required,
|
||||
)
|
||||
from interpreter.utils.constants import TestItemType as cst_type
|
||||
from interpreter.utils.eval import eval_to_boolean, evaluate, post_evaluate
|
||||
from runtime.tum_except import ETUMSyntaxError, item_load_context
|
||||
@@ -13,6 +16,32 @@ LOG_TEST_STOP = '<----- step "{}" finished'
|
||||
LOG_TEST_START = '-----> step "{}" started'
|
||||
|
||||
|
||||
# Parameters accepted by every test item, regardless of its type. Subclasses
|
||||
# concatenate their own ``PARAMS`` to this set; the merged result drives
|
||||
# unknown-param warnings and (later) the LSP schema export.
|
||||
COMMON_PARAMS = ParamSet(
|
||||
Param("name", doc="Display name shown in the GUI tree and reports."),
|
||||
Param("doc", doc="Free-form documentation; surfaced in tooltips."),
|
||||
Param("skipped", doc="If truthy, the step is skipped (static expression, "
|
||||
"evaluated at load time)."),
|
||||
Param("key", doc="Report key used to classify the result "
|
||||
"(typically <test>_PASS or <test>_FAIL)."),
|
||||
Param("stop_on_failure", doc="If true, abort the surrounding container on failure."),
|
||||
Param("execute_on_stop", doc="If true, run this step even when its container "
|
||||
"is being stopped (cleanup)."),
|
||||
Param("process_result", doc="Post-evaluation expression applied to the test result."),
|
||||
Param("store_result", doc="Global-dict key in which to store the test result."),
|
||||
Param("expected_result", doc="Expected outcome; the step is failed if it doesn't match."),
|
||||
Param("no_fail", doc="If truthy, never report a FAILURE for this step."),
|
||||
Param("report", doc="Per-step reporting override."),
|
||||
Param("condition", doc="Optional gating expression evaluated before each "
|
||||
"run; false ⇒ the step is skipped."),
|
||||
Param("steps", kind=LIST, doc="Children (for container items)."),
|
||||
Param("seq_filename", doc="(internal) source .tum file of this step; injected "
|
||||
"by the loader."),
|
||||
)
|
||||
|
||||
|
||||
class TestItem:
|
||||
pass
|
||||
|
||||
@@ -20,52 +49,64 @@ class TestItem:
|
||||
def test_run(f):
|
||||
@wraps(f)
|
||||
def wrapper(self):
|
||||
if not self.skipped:
|
||||
if self.enabled:
|
||||
self.run_test_init()
|
||||
# Conditional execution
|
||||
raw_condition = self._prms.getParam(
|
||||
"condition", default=None, processed=False
|
||||
)
|
||||
if raw_condition is None:
|
||||
condition = True
|
||||
else:
|
||||
c = self._prms.expanse(raw_condition)
|
||||
if isinstance(c, bool):
|
||||
condition = c
|
||||
else:
|
||||
condition = False
|
||||
c = False
|
||||
|
||||
if raw_condition == c:
|
||||
msg = f'"{c}"'
|
||||
else:
|
||||
msg = f'"{raw_condition}" --> "{c}"'
|
||||
|
||||
# Do we have to skip the test because of a true condition ?
|
||||
if condition:
|
||||
if not raw_condition is None:
|
||||
msg = "condition met: " + msg
|
||||
self.result.reported = {"input_condition": msg}
|
||||
print(msg)
|
||||
# Test preparation
|
||||
self.run_before_test()
|
||||
# Test execution
|
||||
f(self)
|
||||
else:
|
||||
msg = "condition not met: " + msg
|
||||
self.result.set(TestValue.NORUN, msg)
|
||||
self.result.reported = {"input_condition": msg}
|
||||
self.run_test_end()
|
||||
else:
|
||||
self.result.set(TestValue.NORUN, "test disabled")
|
||||
print("Test is disabled.")
|
||||
else:
|
||||
if self.skipped:
|
||||
self.result.set(TestValue.NORUN, "test skipped")
|
||||
print("Test is skipped.")
|
||||
return self.result
|
||||
|
||||
if not self.enabled:
|
||||
self.result.set(TestValue.NORUN, "test disabled")
|
||||
print("Test is disabled.")
|
||||
return self.result
|
||||
|
||||
self.run_test_init()
|
||||
|
||||
while self._is_paused:
|
||||
sleep(0.2)
|
||||
if self.isStopped() :
|
||||
self.result.set(TestValue.NORUN, "test stopped")
|
||||
print("Test is Stopped.")
|
||||
self._is_stopped = False # Restore state for next run
|
||||
return self.result
|
||||
|
||||
# Conditional execution
|
||||
raw_condition = self._prms.getParam(
|
||||
"condition", default=None, processed=False
|
||||
)
|
||||
if raw_condition is None:
|
||||
condition = True
|
||||
else:
|
||||
c = self._prms.expanse(raw_condition)
|
||||
if isinstance(c, bool):
|
||||
condition = c
|
||||
else:
|
||||
condition = False
|
||||
c = False
|
||||
|
||||
if raw_condition == c:
|
||||
msg = f'"{c}"'
|
||||
else:
|
||||
msg = f'"{raw_condition}" --> "{c}"'
|
||||
|
||||
# Do we have to skip the test because of a true condition ?
|
||||
if condition:
|
||||
if not raw_condition is None:
|
||||
msg = "condition met: " + msg
|
||||
self.result.reported = {"input_condition": msg}
|
||||
print(msg)
|
||||
# Test preparation
|
||||
self.run_before_test()
|
||||
# Test execution
|
||||
f(self)
|
||||
else:
|
||||
msg = "condition not met: " + msg
|
||||
self.result.set(TestValue.NORUN, msg)
|
||||
self.result.reported = {"input_condition": msg}
|
||||
self.run_test_end()
|
||||
|
||||
return self.result
|
||||
|
||||
|
||||
return wrapper
|
||||
|
||||
|
||||
@@ -85,6 +126,11 @@ def test_data(item: TestItem, child: dict) -> dict:
|
||||
|
||||
|
||||
class TestItem:
|
||||
# Subclasses override with their own ParamSet to opt into the declarative
|
||||
# validation. While ``PARAMS`` is empty / unset, the base class skips the
|
||||
# unknown-param check for this item type — keeps the migration incremental.
|
||||
PARAMS = None
|
||||
|
||||
def __init__(
|
||||
self, dict_item: dict = None, parent: TestItem = None, status_queue=None, filename = ""
|
||||
):
|
||||
@@ -122,6 +168,13 @@ class TestItem:
|
||||
# creation of the params object
|
||||
self._prms = TestItemParams(dict_item, parent)
|
||||
|
||||
# Declarative-params validation. Only kicks in when the concrete
|
||||
# subclass declares ``PARAMS`` — items not yet migrated stay
|
||||
# silent. Warnings (not errors) during the migration window so
|
||||
# existing .tum files don't break suddenly; will be flipped to
|
||||
# errors once every item has migrated.
|
||||
self._validate_declared_params(dict_item)
|
||||
|
||||
# getting parameters for the test item
|
||||
try:
|
||||
self._name = self._prms.getParam("name", default="", processed=True)
|
||||
@@ -178,6 +231,36 @@ class TestItem:
|
||||
|
||||
self.result = TestResult(self, TestValue.FAILURE, "Failure by default")
|
||||
|
||||
def _validate_declared_params(self, dict_item):
|
||||
"""Warn on unknown / missing-required params, if PARAMS is declared.
|
||||
|
||||
The check is opt-in per subclass: it only runs when the concrete
|
||||
class sets a non-empty ``PARAMS`` attribute. Items not yet migrated
|
||||
produce no diagnostics — preserving the historical "silently accept
|
||||
anything" behavior until they get their declaration.
|
||||
"""
|
||||
if not self.PARAMS:
|
||||
return
|
||||
# ``self._type`` is the parent root type at this point (subclasses set
|
||||
# it after super().__init__), so use the class name as a stable label
|
||||
# in diagnostics. ``self._name`` was preset to the type name by every
|
||||
# subclass before super() ran, which gives a useful prefix.
|
||||
label = f"{type(self).__name__} '{self._name}'"
|
||||
declared = COMMON_PARAMS + self.PARAMS
|
||||
unknown = unknown_keys(declared, dict_item)
|
||||
if unknown:
|
||||
accepted = ", ".join(sorted(declared.names()))
|
||||
for k in unknown:
|
||||
tm.print_warn(
|
||||
f"{label}: unknown parameter '{k}'. Accepted: {accepted}."
|
||||
)
|
||||
missing = missing_required(declared, dict_item)
|
||||
for k in missing:
|
||||
raise ETUMSyntaxError(
|
||||
f"{label}: required parameter '{k}' is missing.",
|
||||
self._seq_filename,
|
||||
)
|
||||
|
||||
def _filter_dict_item(self, dict_item):
|
||||
# Stores the content of the step to be displayed
|
||||
# in the GUI
|
||||
@@ -255,8 +338,6 @@ class TestItem:
|
||||
self._sendStatusStarted()
|
||||
if self._is_breakpoint:
|
||||
self._is_paused = True
|
||||
while self._is_paused:
|
||||
sleep(0.2)
|
||||
|
||||
if self.is_container:
|
||||
self.report.incLevel()
|
||||
@@ -274,9 +355,6 @@ class TestItem:
|
||||
if self.is_container:
|
||||
self.report.decLevel()
|
||||
|
||||
while self._is_paused:
|
||||
sleep(0.2)
|
||||
|
||||
# Post evaluation of the test result
|
||||
self.process_result()
|
||||
# expected_result treatment
|
||||
@@ -311,6 +389,7 @@ class TestItem:
|
||||
self.report.addTest(self, self.result, rk)
|
||||
self._sendStatusFinished()
|
||||
|
||||
|
||||
def process_result(self):
|
||||
if self._post_eval is None:
|
||||
return
|
||||
|
||||
@@ -5,11 +5,22 @@ from runtime.tum_except import ETUMSyntaxError, item_load_context
|
||||
import api.testium as tm
|
||||
from interpreter.utils.constants import TestItemType as cst
|
||||
from interpreter.utils.eval import evaluate
|
||||
from interpreter.utils.param_decl import Param, ParamSet, LIST
|
||||
|
||||
class TestItemCheckValue(TestItem):
|
||||
"""check item usage.
|
||||
check usage:{check: {name: check my func output, steps: ['$(pfn_echo) < 5']}}
|
||||
"""
|
||||
|
||||
PARAMS = ParamSet(
|
||||
Param("values", kind=LIST, required=True,
|
||||
doc="List of expressions to evaluate. Each is expanded then "
|
||||
"evaluated; non-truthy results fail the check."),
|
||||
# 'steps' is intentionally not redeclared here — it's the deprecated
|
||||
# alias of 'values' and is already accepted by COMMON_PARAMS for
|
||||
# container items. A runtime warning is emitted when 'steps' is used.
|
||||
)
|
||||
|
||||
def __init__(self, dict_item, parent = None, status_queue=None, filename=""):
|
||||
self._name = cst.TYPE_CHECK.item_name
|
||||
super().__init__(dict_item, parent, status_queue, filename=filename)
|
||||
|
||||
@@ -2,11 +2,25 @@ from interpreter.test_items.test_item import test_run
|
||||
from interpreter.test_items.test_result import TestValue
|
||||
from interpreter.test_items.test_item_dialog_base import TestItemDialogBase, _is_text_mode, _is_interactive
|
||||
from interpreter.utils.constants import TestItemType as cst
|
||||
from interpreter.utils.param_decl import Param, ParamSet, BLOCK
|
||||
from runtime.tum_except import item_load_context
|
||||
import api.testium as tm
|
||||
|
||||
|
||||
class TestItemChoicesDialog(TestItemDialogBase):
|
||||
|
||||
PARAMS = ParamSet(
|
||||
Param("question", required=True,
|
||||
doc="Prompt shown above the list of choices."),
|
||||
Param("choices", kind=BLOCK, required=True,
|
||||
doc="Tree of choices: either a list of strings, or a nested "
|
||||
"mapping {label: subchoices, ...} to build a multi-level menu."),
|
||||
Param("icon", default=None,
|
||||
doc="Default icon name shown next to each choice."),
|
||||
Param("auto_result", default=None,
|
||||
doc="Batch-mode selection (path or label). None ⇒ FAILURE."),
|
||||
)
|
||||
|
||||
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
|
||||
self._name = cst.TYPE_CHOICES_DLG.item_name
|
||||
super().__init__(dict_item, parent, status_queue, filename=filename)
|
||||
|
||||
@@ -10,6 +10,7 @@ from interpreter.test_items.test_item import test_run
|
||||
from interpreter.test_items.item_actions import TestItemActions
|
||||
from interpreter.test_items.item_actions.action import TestItemAction
|
||||
from interpreter.utils.constants import TestItemType as cst
|
||||
from interpreter.utils.param_decl import Param, ParamSet
|
||||
from interpreter.test_items.test_result import TestResult, TestValue
|
||||
|
||||
|
||||
@@ -21,6 +22,38 @@ class TestItemConsoleAction(TestItemAction):
|
||||
|
||||
|
||||
class TestItemConsoleOpen(TestItemConsoleAction):
|
||||
|
||||
PARAMS = ParamSet(
|
||||
Param("protocol", required=True,
|
||||
doc="Transport: 'telnet', 'ssh', 'rawtcp', 'serial' or 'terminal'."),
|
||||
Param("write_delay", default=0,
|
||||
doc="Inter-character write delay in ms (slow devices)."),
|
||||
Param("log", doc="Path to a log file capturing the console traffic."),
|
||||
Param("overwrite_log", default=True,
|
||||
doc="If true, truncate the log file at open; else append."),
|
||||
# telnet
|
||||
Param("telnet_host", doc="Hostname/IP for the telnet target."),
|
||||
Param("telnet_port", default=69, doc="TCP port for telnet."),
|
||||
# ssh
|
||||
Param("ssh_host", doc="Hostname/IP for the SSH target."),
|
||||
Param("ssh_user", doc="SSH login user."),
|
||||
Param("ssh_pwd", doc="SSH password (if key-based auth is not used)."),
|
||||
# rawtcp
|
||||
Param("tcp_host", doc="Hostname/IP for a raw-TCP connection."),
|
||||
Param("tcp_port", doc="TCP port for a raw-TCP connection."),
|
||||
# serial
|
||||
Param("serial_port", doc="Serial device path (e.g. /dev/ttyUSB0 or COM3)."),
|
||||
Param("serial_baudrate", doc="Serial baudrate."),
|
||||
Param("buffered", default=True,
|
||||
doc="If true, the serial console buffers received bytes between reads."),
|
||||
# terminal
|
||||
Param("terminal_path",
|
||||
doc="Working directory for the local terminal protocol."),
|
||||
Param("shell",
|
||||
doc="Shell command used for the local terminal protocol "
|
||||
"(default: 'cmd.exe' on Windows, '/usr/bin/env bash' elsewhere)."),
|
||||
)
|
||||
|
||||
def __init__(
|
||||
self, action_name, dict_item, parent=None, status_queue=None, filename=""
|
||||
):
|
||||
@@ -283,6 +316,17 @@ class TestItemConsoleWriteLn(TestItemConsoleAction):
|
||||
|
||||
|
||||
class TestItemConsoleReadUntil(TestItemConsoleAction):
|
||||
|
||||
PARAMS = ParamSet(
|
||||
Param("expected", required=True,
|
||||
doc="Regex matched against incoming console output until found "
|
||||
"or until timeout."),
|
||||
Param("timeout", default=-1,
|
||||
doc="Seconds before giving up. Negative means infinite."),
|
||||
Param("mute", default=False,
|
||||
doc="If true, don't echo received bytes to testium's stdout/log."),
|
||||
)
|
||||
|
||||
def __init__(
|
||||
self, action_name, dict_item, parent=None, status_queue=None, filename=""
|
||||
):
|
||||
@@ -307,11 +351,17 @@ class TestItemConsoleReadUntil(TestItemConsoleAction):
|
||||
|
||||
try:
|
||||
status, data = cons.read_until(
|
||||
ru, timeout=read_timeout, return_data=True, mute=mute
|
||||
ru, timeout=read_timeout, return_data=True, mute=mute,
|
||||
should_stop=self.isStopped,
|
||||
)
|
||||
if status == 0:
|
||||
self.result.set(TestValue.SUCCESS)
|
||||
self.result.value = data
|
||||
elif self.isStopped():
|
||||
self.result.set(
|
||||
result=TestValue.FAILURE,
|
||||
message="Console read aborted on stop request",
|
||||
)
|
||||
else:
|
||||
self.result.set(result=TestValue.FAILURE, message="No matching text")
|
||||
if mute:
|
||||
@@ -330,6 +380,14 @@ class TestItemConsoleReadUntil(TestItemConsoleAction):
|
||||
|
||||
|
||||
class TestItemConsole(TestItemActions):
|
||||
|
||||
PARAMS = ParamSet(
|
||||
Param("console_name", required=True,
|
||||
doc="Identifier of the console — used by every nested action to "
|
||||
"reach back the same transport. Multiple consoles can coexist "
|
||||
"as long as their names differ."),
|
||||
)
|
||||
|
||||
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
|
||||
super().__init__(
|
||||
cst.TYPE_CONSOLE, dict_item, parent, status_queue, filename=filename
|
||||
|
||||
@@ -8,9 +8,36 @@ from interpreter.test_items.test_result import TestResult, TestValue
|
||||
import api.testium as tm
|
||||
from interpreter.utils.params import TestItemParams
|
||||
from interpreter.utils.constants import TestItemType as cst
|
||||
from interpreter.utils.param_decl import Param, ParamSet, BLOCK
|
||||
|
||||
|
||||
# Sub-block validation: 'cycle' accepts an 'exit_condition:' mapping whose
|
||||
# own params are reported here so unknown keys inside it can be flagged
|
||||
# during a future Block-aware diagnostic pass. For now the parent only
|
||||
# declares that 'exit_condition' is an accepted top-level key.
|
||||
EXIT_CONDITION_PARAMS = ParamSet(
|
||||
Param("time", doc="HH:MM time of day after which the loop exits."),
|
||||
Param("value", doc="Expression; when truthy the loop exits."),
|
||||
Param("file", doc="Python file containing the exit-condition function."),
|
||||
Param("func_name", doc="Function name in 'file' returning the exit value."),
|
||||
Param("param", doc="Arguments passed to the exit function."),
|
||||
Param("eval", default="",
|
||||
doc="Post-evaluation expression applied to the function's return."),
|
||||
)
|
||||
|
||||
|
||||
class TestItemCycle(TestItem):
|
||||
|
||||
PARAMS = ParamSet(
|
||||
Param("iterator",
|
||||
doc="Iterable (or string expanding to one) driving the loop. "
|
||||
"The current value is exposed as $(loop_param)."),
|
||||
Param("exit_condition", kind=BLOCK,
|
||||
doc="Optional block stopping the loop early: combine 'time', "
|
||||
"'value', or a 'file'+'func_name' pair (with optional "
|
||||
"'param' and 'eval')."),
|
||||
)
|
||||
|
||||
def __init__(self, dict_cycle, parent=None, status_queue=None, filename=""):
|
||||
self._name = cst.TYPE_CYCLE.item_name
|
||||
super().__init__(dict_cycle, parent, status_queue, filename=filename)
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
from interpreter.test_items.test_item import (TestItem, test_run)
|
||||
from interpreter.test_items.test_result import (TestValue)
|
||||
from interpreter.utils.constants import TestItemType as cst
|
||||
from interpreter.utils.param_decl import Param, ParamSet, LIST
|
||||
from runtime.tum_except import ETUMParamError, ETUMSyntaxError
|
||||
import interpreter.utils.version as git
|
||||
|
||||
@@ -8,6 +9,13 @@ class TestItemGit(TestItem):
|
||||
"""
|
||||
This item expect only one parameter which is a string or list of string being the path to the git folder
|
||||
"""
|
||||
|
||||
PARAMS = ParamSet(
|
||||
Param("repo", kind=LIST, required=True,
|
||||
doc="Path to a git checkout, or list of such paths. Each is "
|
||||
"reported with its current version (tag + dirty state)."),
|
||||
)
|
||||
|
||||
def __init__(self, dict_item, parent = None, status_queue=None, filename=""):
|
||||
self._name = cst.TYPE_GIT.item_name
|
||||
super().__init__(dict_item, parent, status_queue, filename=filename)
|
||||
|
||||
@@ -1,10 +1,17 @@
|
||||
from interpreter.test_items.test_item import (TestItem, test_run)
|
||||
from interpreter.test_items.test_result import (TestResult, TestValue)
|
||||
from interpreter.utils.constants import TestItemType as cst
|
||||
from interpreter.utils.param_decl import ParamSet
|
||||
from runtime.tum_except import ETUMSyntaxError
|
||||
import api.testium as tm
|
||||
|
||||
class TestItemGroup(TestItem):
|
||||
|
||||
# 'group' has no item-specific parameters; 'steps' is handled by COMMON_PARAMS.
|
||||
# Declaring an empty ParamSet still opts in to unknown-param validation
|
||||
# (e.g. typo 'stop_on_failures').
|
||||
PARAMS = ParamSet()
|
||||
|
||||
def __init__(self, dict_cycle, parent = None, status_queue=None, filename=""):
|
||||
self._name = cst.TYPE_GROUP.item_name
|
||||
super().__init__(dict_cycle, parent, status_queue, filename=filename)
|
||||
|
||||
@@ -4,6 +4,7 @@ from interpreter.test_items.test_item import test_run
|
||||
from interpreter.test_items.test_result import TestValue
|
||||
from interpreter.test_items.test_item_dialog_base import TestItemDialogBase, _is_text_mode, _is_interactive
|
||||
from interpreter.utils.constants import TestItemType as cst
|
||||
from interpreter.utils.param_decl import Param, ParamSet
|
||||
from runtime.tum_except import item_load_context
|
||||
import api.testium as tm
|
||||
|
||||
@@ -12,6 +13,17 @@ class TestItemImageDialog(TestItemDialogBase):
|
||||
"""dialog_image item usage.
|
||||
dialog_image name: Nice image, question: could you press the red button, filename: img.jpg
|
||||
"""
|
||||
|
||||
PARAMS = ParamSet(
|
||||
Param("question", required=True,
|
||||
doc="Prompt shown above the image."),
|
||||
Param("filename", required=True,
|
||||
doc="Path to the image file (relative to the test directory or absolute)."),
|
||||
Param("auto_result", default=None,
|
||||
doc="Outcome used in batch/non-interactive mode. Truthy ⇒ SUCCESS, "
|
||||
"None ⇒ FAILURE."),
|
||||
)
|
||||
|
||||
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
|
||||
self._name = cst.TYPE_IMAGE_DLG.item_name
|
||||
super().__init__(dict_item, parent, status_queue, filename=filename)
|
||||
|
||||
@@ -11,6 +11,7 @@ from interpreter.test_items.item_actions.action import TestItemAction
|
||||
|
||||
from interpreter.utils.constants import TestItemType as cst
|
||||
from interpreter.utils.eval import evaluate
|
||||
from interpreter.utils.param_decl import Param, ParamSet, BLOCK
|
||||
|
||||
from interpreter.test_items.test_item_json_rpc.jsonrpc_adapters import (
|
||||
JrpcAdapter,
|
||||
@@ -76,6 +77,20 @@ class TestItemJSRPCActionClose(TestItemAction):
|
||||
|
||||
class TestItemJSRPCActionQuery(TestItemAction):
|
||||
|
||||
PARAMS = ParamSet(
|
||||
Param("method", required=True,
|
||||
doc="JSON-RPC method name to call."),
|
||||
Param("params",
|
||||
doc="Parameters payload (list, dict or scalar) sent to the method."),
|
||||
Param("id", default="rand",
|
||||
doc="JSON-RPC request id. 'rand' (default) ⇒ a random integer is used."),
|
||||
Param("no_wait", default=False,
|
||||
doc="If true, send the request without waiting for a response."),
|
||||
Param("timeout", default=None,
|
||||
doc="Seconds to wait for a response. None ⇒ inherits the transport "
|
||||
"default."),
|
||||
)
|
||||
|
||||
def __init__(
|
||||
self, action_name, dict_item, parent=None, status_queue=None, filename=""
|
||||
):
|
||||
@@ -105,6 +120,7 @@ class TestItemJSRPCActionQuery(TestItemAction):
|
||||
jrpc_id = randint(1, (2**32) - 1)
|
||||
send_only = self._prms.expanse(self._send_only)
|
||||
timeout = self._prms.expanse(self._timeout)
|
||||
self.token.set_should_stop(self.isStopped)
|
||||
try:
|
||||
success, result = self.token.query(
|
||||
meth, obj, jrpc_id, send_only, timeout=timeout
|
||||
@@ -128,6 +144,13 @@ class TestItemJSRPCActionQuery(TestItemAction):
|
||||
|
||||
class TestItemJSRPCActionReceive(TestItemAction):
|
||||
|
||||
PARAMS = ParamSet(
|
||||
Param("id", required=True,
|
||||
doc="JSON-RPC request id whose response we expect."),
|
||||
Param("timeout", default=None,
|
||||
doc="Seconds to wait for the response. None ⇒ transport default."),
|
||||
)
|
||||
|
||||
def __init__(
|
||||
self, action_name, dict_item, parent=None, status_queue=None, filename=""
|
||||
):
|
||||
@@ -146,6 +169,7 @@ class TestItemJSRPCActionReceive(TestItemAction):
|
||||
def execute(self):
|
||||
timeout = self._prms.expanse(self._timeout)
|
||||
jrpc_id = self._prms.expanse(self._jrpc_id)
|
||||
self.token.set_should_stop(self.isStopped)
|
||||
|
||||
try:
|
||||
success, result = self.token.receive(jrpc_id, timeout)
|
||||
@@ -170,6 +194,22 @@ class TestItemJSON_RPC(TestItemActions):
|
||||
This item TBD
|
||||
"""
|
||||
|
||||
PARAMS = ParamSet(
|
||||
Param("console", kind=BLOCK,
|
||||
doc="Console-transport block: {console_name, …}. Either 'console' "
|
||||
"or 'udp' must be set."),
|
||||
Param("udp", kind=BLOCK,
|
||||
doc="UDP-transport block: {host, port, …}. Either 'console' or "
|
||||
"'udp' must be set."),
|
||||
Param("version", default="1.0",
|
||||
doc="JSON-RPC protocol version ('1.0' or '2.0')."),
|
||||
Param("timeout", required=True,
|
||||
doc="Default seconds to wait for a JSON-RPC response across all "
|
||||
"child query/receive actions."),
|
||||
Param("mute", default=False,
|
||||
doc="If true, don't echo wire traffic to the log."),
|
||||
)
|
||||
|
||||
def __init__(
|
||||
self, dict_item: dict, parent: TestItem = None, status_queue=None, filename=""
|
||||
):
|
||||
|
||||
@@ -2,10 +2,11 @@ import json
|
||||
import socket
|
||||
import re
|
||||
import struct
|
||||
import time
|
||||
|
||||
from runtime.tum_except import ETUMRuntimeError
|
||||
import api.testium as tm
|
||||
from api.console import Console
|
||||
from api.console import Console, STOP_POLL_INTERVAL
|
||||
|
||||
|
||||
def is_ip_address(address):
|
||||
@@ -45,9 +46,16 @@ class JrpcAdapter:
|
||||
self._jrpc_version = version
|
||||
self._mute = mute
|
||||
self._timeout = timeout
|
||||
# Optional callable polled by _receive() implementations to abort
|
||||
# waits early when the test is being stopped. Set by the test item
|
||||
# action before each query/receive call.
|
||||
self._should_stop = None
|
||||
if not (version == "1.0" or version == "2.0"):
|
||||
raise ETUMRuntimeError("Invalid JSONRPC version passed.")
|
||||
|
||||
def set_should_stop(self, cb):
|
||||
self._should_stop = cb
|
||||
|
||||
@property
|
||||
def timeout(self):
|
||||
return self._timeout
|
||||
@@ -249,32 +257,38 @@ class JrpcUdpAdapter(JrpcAdapter):
|
||||
print(f" | sent to @{self._server}:{self._snd_port}")
|
||||
|
||||
def _receive(self, timeout: float) -> str:
|
||||
# Poll in short chunks so a stop request is honored within
|
||||
# STOP_POLL_INTERVAL.
|
||||
self.sock.settimeout(STOP_POLL_INTERVAL)
|
||||
deadline = time.monotonic() + float(timeout)
|
||||
data = None
|
||||
addr = None
|
||||
while True:
|
||||
if self._should_stop is not None and self._should_stop():
|
||||
raise ETUMRuntimeError("JSONRPC udp receive aborted on stop request.")
|
||||
try:
|
||||
data, addr = self.sock.recvfrom(self._bufsize)
|
||||
break
|
||||
except socket.timeout:
|
||||
if time.monotonic() >= deadline:
|
||||
raise ETUMRuntimeError(
|
||||
"JSONRPC udp answer took too long. Try to increase the timeout."
|
||||
)
|
||||
|
||||
# configures the reception timeout
|
||||
self.sock.settimeout(timeout)
|
||||
|
||||
# Receives the answer from the server
|
||||
try:
|
||||
data, addr = self.sock.recvfrom(self._bufsize)
|
||||
|
||||
# In case of buffer overload we chose to complain
|
||||
if len(data) >= self._bufsize:
|
||||
raise ETUMRuntimeError(
|
||||
"JSONRPC udp answer size overflow. Try to increase the bufsize"
|
||||
)
|
||||
|
||||
# Converts binary to string
|
||||
res = data.decode()
|
||||
|
||||
# Don't log if mute
|
||||
if not self._mute:
|
||||
print(f" | UDP answer: '{res}'")
|
||||
print(f" | received from @{addr[0]}:{addr[1]}")
|
||||
|
||||
except socket.timeout:
|
||||
# In case of buffer overload we chose to complain
|
||||
if len(data) >= self._bufsize:
|
||||
raise ETUMRuntimeError(
|
||||
"JSONRPC udp answer took too long. Try to increase the timeout."
|
||||
"JSONRPC udp answer size overflow. Try to increase the bufsize"
|
||||
)
|
||||
|
||||
# Converts binary to string
|
||||
res = data.decode()
|
||||
|
||||
# Don't log if mute
|
||||
if not self._mute:
|
||||
print(f" | UDP answer: '{res}'")
|
||||
print(f" | received from @{addr[0]}:{addr[1]}")
|
||||
|
||||
return res
|
||||
|
||||
def _build_query(self, method: str, obj, jrpc_id: int):
|
||||
@@ -339,11 +353,16 @@ class JrpcConsoleAdapter(JrpcAdapter):
|
||||
|
||||
def _receive(self, timeout: float) -> str:
|
||||
status, data = self._cons.read_until(
|
||||
self._endswith, timeout, return_data=True, mute=self._mute
|
||||
self._endswith, timeout, return_data=True, mute=self._mute,
|
||||
should_stop=self._should_stop,
|
||||
)
|
||||
|
||||
# if we did not receive anything, we complain
|
||||
if not status == 0:
|
||||
if self._should_stop is not None and self._should_stop():
|
||||
raise ETUMRuntimeError(
|
||||
f"JSONRPC console receive aborted on stop request."
|
||||
)
|
||||
raise ETUMRuntimeError(
|
||||
f"The '{self._cons.name}' console did not answer in the requested time."
|
||||
)
|
||||
|
||||
@@ -8,12 +8,20 @@ from interpreter.test_items.test_result import (TestResult, TestValue)
|
||||
from runtime.tum_except import ETUMSyntaxError, item_load_context
|
||||
import api.testium as tm
|
||||
from interpreter.utils.constants import TestItemType as cst
|
||||
from interpreter.utils.param_decl import Param, ParamSet, LIST
|
||||
|
||||
class TestItemLet(TestItem):
|
||||
"""let item usage.
|
||||
let values: {variable1: a, variable2: /dev/ttyUSB0, variable3: 115200}
|
||||
let eval: {conditional_exec: "random.randint(1, 4)"}
|
||||
"""
|
||||
|
||||
PARAMS = ParamSet(
|
||||
Param("values", kind=LIST, required=True,
|
||||
doc="Mapping (or list of single-pair mappings) of global-dict "
|
||||
"key → value to set. Values are expanded at execution time."),
|
||||
)
|
||||
|
||||
def __init__(self, dict_item, parent = None, status_queue=None, filename=""):
|
||||
self._name = cst.TYPE_LET.item_name
|
||||
super().__init__(dict_item, parent, status_queue, filename=filename)
|
||||
|
||||
@@ -11,6 +11,7 @@ import api.testium as tm
|
||||
from interpreter.utils.lua_func_exec import LuaFuncExecEngine
|
||||
from interpreter.utils.api_srv import api_request
|
||||
from interpreter.utils.constants import TestItemType as cst
|
||||
from interpreter.utils.param_decl import Param, ParamSet, LIST
|
||||
|
||||
_LUA_FUNC_CONTEXTS_KEY = "_lua_func_contexts"
|
||||
|
||||
@@ -21,6 +22,21 @@ class TestItemLuaFunc(TestItem):
|
||||
Optional: context_id: <id> — share a persistent process with other lua_func items using the same id.
|
||||
"""
|
||||
|
||||
PARAMS = ParamSet(
|
||||
Param("file", required=True,
|
||||
doc="Path to the .lua file containing the function."),
|
||||
Param("func_name", required=True,
|
||||
doc="Name of the function to call in the file."),
|
||||
Param("param", kind=LIST,
|
||||
doc="Arguments passed to the function. Each entry is expanded "
|
||||
"before the call. Special tokens $(loop_param) / $(loop_index) "
|
||||
"resolve from the surrounding cycle."),
|
||||
Param("context_id", default=None,
|
||||
doc="If set, the lua_func subprocess is kept alive and reused by "
|
||||
"every other lua_func item with the same context_id — enables "
|
||||
"shared in-memory state between successive calls."),
|
||||
)
|
||||
|
||||
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
|
||||
self._name = cst.TYPE_LUA_FUNCTION.item_name
|
||||
super().__init__(dict_item, parent, status_queue, filename=filename)
|
||||
@@ -45,6 +61,18 @@ class TestItemLuaFunc(TestItem):
|
||||
tm.setgd(_LUA_FUNC_CONTEXTS_KEY, contexts)
|
||||
return contexts[ctx_id], True
|
||||
|
||||
def stop(self):
|
||||
super().stop()
|
||||
# Tear down the worker so any in-flight func_call returns promptly.
|
||||
# join() clears _rpc/_process so a subsequent item reusing the same
|
||||
# context_id can restart the engine cleanly.
|
||||
try:
|
||||
engine, _ = self._get_engine()
|
||||
engine.stop()
|
||||
engine.join()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
@test_run
|
||||
def execute(self):
|
||||
self.result.set(
|
||||
@@ -96,9 +124,15 @@ Is the lua environnment well defined in the "LUA_PATH" and "LUA_CPATH" variables
|
||||
|
||||
return
|
||||
|
||||
except ConnectionAbortedError:
|
||||
self.result.set(TestValue.FAILURE, "lua_func aborted on stop request")
|
||||
print("lua_func aborted on stop request.")
|
||||
except:
|
||||
traceback.print_exception(*sys.exc_info())
|
||||
self.result.set(
|
||||
TestValue.FAILURE,
|
||||
'Unrecoverable "lua_func" item error from {}'.format(self.func_name),
|
||||
)
|
||||
if self.isStopped():
|
||||
self.result.set(TestValue.FAILURE, "lua_func aborted on stop request")
|
||||
else:
|
||||
self.result.set(
|
||||
TestValue.FAILURE,
|
||||
'Unrecoverable "lua_func" item error from {}'.format(self.func_name),
|
||||
)
|
||||
|
||||
@@ -5,6 +5,7 @@ from interpreter.test_items.test_item import test_run
|
||||
from interpreter.test_items.test_result import TestValue
|
||||
from interpreter.test_items.test_item_dialog_base import TestItemDialogBase, _is_text_mode, _is_interactive
|
||||
from interpreter.utils.constants import TestItemType as cst
|
||||
from interpreter.utils.param_decl import Param, ParamSet
|
||||
from runtime.tum_except import item_load_context
|
||||
|
||||
|
||||
@@ -12,6 +13,15 @@ class TestItemMsgDialog(TestItemDialogBase):
|
||||
"""dialog_message item usage.
|
||||
dialog_message name: Nice message, question: Open the door and press OK
|
||||
"""
|
||||
|
||||
PARAMS = ParamSet(
|
||||
Param("question", required=True,
|
||||
doc="Message body shown to the user. Multi-line strings are supported."),
|
||||
Param("auto_result", default=None,
|
||||
doc="Outcome used in batch/non-interactive mode instead of waiting "
|
||||
"for the user. Truthy ⇒ SUCCESS, None ⇒ FAILURE."),
|
||||
)
|
||||
|
||||
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
|
||||
self._name = cst.TYPE_MESSAGE_DLG.item_name
|
||||
super().__init__(dict_item, parent, status_queue, filename=filename)
|
||||
|
||||
@@ -2,11 +2,23 @@ from interpreter.test_items.test_item import test_run
|
||||
from interpreter.test_items.test_result import TestValue
|
||||
from interpreter.test_items.test_item_dialog_base import TestItemDialogBase, _is_text_mode, _is_interactive
|
||||
from interpreter.utils.constants import TestItemType as cst
|
||||
from interpreter.utils.param_decl import Param, ParamSet
|
||||
from runtime.tum_except import item_load_context
|
||||
import api.testium as tm
|
||||
|
||||
|
||||
class TestItemNoteDialog(TestItemDialogBase):
|
||||
|
||||
PARAMS = ParamSet(
|
||||
Param("question", required=True,
|
||||
doc="Prompt shown above the note input field."),
|
||||
Param("auto_result", default=None,
|
||||
doc="Batch-mode outcome: None ⇒ FAILURE, 'cancel' ⇒ cancelled, "
|
||||
"any other truthy ⇒ SUCCESS with auto_value."),
|
||||
Param("auto_value", default=None,
|
||||
doc="Note text used in batch mode when auto_result is set."),
|
||||
)
|
||||
|
||||
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
|
||||
self._name = cst.TYPE_NOTE_DLG.item_name
|
||||
super().__init__(dict_item, parent, status_queue, filename=filename)
|
||||
|
||||
@@ -6,6 +6,7 @@ from interpreter.test_items.test_item import test_run
|
||||
from interpreter.test_items.test_result import TestResult, TestValue
|
||||
from interpreter.utils.constants import TestItemType as cst
|
||||
from interpreter.utils.eval import eval_to_boolean
|
||||
from interpreter.utils.param_decl import Param, ParamSet, LIST, BLOCK, Enum
|
||||
from runtime.tum_except import ETUMSyntaxError
|
||||
from runtime.string_queue import StringQueue
|
||||
from runtime.stdout_redirect import stdio_redir
|
||||
@@ -15,6 +16,12 @@ class TestItemParallelBranch(TestItemContainer):
|
||||
"""One branch of a parallel item. Runs its children sequentially,
|
||||
optionally waiting for a condition before starting."""
|
||||
|
||||
PARAMS = ParamSet(
|
||||
Param("wait_for", kind=BLOCK,
|
||||
doc="Optional block {condition, timeout} that defers the branch "
|
||||
"start until the condition is truthy (or the timeout elapses)."),
|
||||
)
|
||||
|
||||
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
|
||||
super().__init__(cst.TYPE_PARALLEL_BRANCH, dict_item, parent, status_queue, filename=filename)
|
||||
self._wait_condition = None
|
||||
@@ -87,6 +94,15 @@ class TestItemParallel(TestItemContainer):
|
||||
- ...
|
||||
"""
|
||||
|
||||
PARAMS = ParamSet(
|
||||
Param("branches", kind=LIST, required=True,
|
||||
doc="List of branch blocks (each branch holds its own 'steps' "
|
||||
"and optional 'wait_for')."),
|
||||
Param("sync", kind=Enum("all", "any"), default="all",
|
||||
doc="'all' (default) waits for every branch; 'any' returns as "
|
||||
"soon as the first branch completes."),
|
||||
)
|
||||
|
||||
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
|
||||
branches = dict_item.get("branches", [])
|
||||
if not branches:
|
||||
|
||||
@@ -11,6 +11,7 @@ import api.testium as tm
|
||||
from interpreter.utils.py_func_exec import PyFuncExecEngine
|
||||
from interpreter.utils.api_srv import api_request
|
||||
from interpreter.utils.constants import TestItemType as cst
|
||||
from interpreter.utils.param_decl import Param, ParamSet, LIST
|
||||
|
||||
_PY_FUNC_CONTEXTS_KEY = "_py_func_contexts"
|
||||
|
||||
@@ -21,6 +22,21 @@ class TestItemPyFunc(TestItem):
|
||||
Optional: context_id: <id> — share a persistent process with other py_func items using the same id.
|
||||
"""
|
||||
|
||||
PARAMS = ParamSet(
|
||||
Param("file", required=True,
|
||||
doc="Path to the .py file containing the function."),
|
||||
Param("func_name", required=True,
|
||||
doc="Name of the function to call in the file."),
|
||||
Param("param", kind=LIST,
|
||||
doc="Arguments passed to the function. Each entry is expanded "
|
||||
"before the call. Special tokens $(loop_param) / $(loop_index) "
|
||||
"resolve from the surrounding cycle."),
|
||||
Param("context_id", default=None,
|
||||
doc="If set, the py_func subprocess is kept alive and reused by "
|
||||
"every other py_func item with the same context_id — enables "
|
||||
"shared in-memory state between successive calls."),
|
||||
)
|
||||
|
||||
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
|
||||
self._name = cst.TYPE_PY_FUNCTION.item_name
|
||||
super().__init__(dict_item, parent, status_queue, filename=filename)
|
||||
@@ -45,6 +61,18 @@ class TestItemPyFunc(TestItem):
|
||||
tm.setgd(_PY_FUNC_CONTEXTS_KEY, contexts)
|
||||
return contexts[ctx_id], True
|
||||
|
||||
def stop(self):
|
||||
super().stop()
|
||||
# Tear down the worker so any in-flight func_call returns promptly.
|
||||
# join() clears _rpc/_process so a subsequent item reusing the same
|
||||
# context_id can restart the engine cleanly.
|
||||
try:
|
||||
engine, _ = self._get_engine()
|
||||
engine.stop()
|
||||
engine.join()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
@test_run
|
||||
def execute(self):
|
||||
self.result.set(
|
||||
@@ -94,9 +122,15 @@ python_bin = {tm.gd("python_bin", "no python path defined")}"""
|
||||
|
||||
return
|
||||
|
||||
except ConnectionAbortedError:
|
||||
self.result.set(TestValue.FAILURE, "py_func aborted on stop request")
|
||||
print("py_func aborted on stop request.")
|
||||
except:
|
||||
traceback.print_exception(*sys.exc_info())
|
||||
self.result.set(
|
||||
TestValue.FAILURE,
|
||||
'Unrecoverable "py_func" item error from {}'.format(self.func_name),
|
||||
)
|
||||
if self.isStopped():
|
||||
self.result.set(TestValue.FAILURE, "py_func aborted on stop request")
|
||||
else:
|
||||
self.result.set(
|
||||
TestValue.FAILURE,
|
||||
'Unrecoverable "py_func" item error from {}'.format(self.func_name),
|
||||
)
|
||||
|
||||
@@ -2,6 +2,7 @@ from interpreter.test_items.test_item import test_run
|
||||
from interpreter.test_items.test_result import TestValue
|
||||
from interpreter.test_items.test_item_dialog_base import TestItemDialogBase, _is_text_mode, _is_interactive
|
||||
from interpreter.utils.constants import TestItemType as cst
|
||||
from interpreter.utils.param_decl import Param, ParamSet
|
||||
from runtime.tum_except import item_load_context
|
||||
|
||||
|
||||
@@ -9,6 +10,14 @@ class TestItemQuestionDialog(TestItemDialogBase):
|
||||
"""dialog_question item usage.
|
||||
dialog_question name: Nice question, question: "If OK, press OK, If not, press cancel"
|
||||
"""
|
||||
|
||||
PARAMS = ParamSet(
|
||||
Param("question", required=True,
|
||||
doc="Yes/No prompt presented to the user."),
|
||||
Param("auto_result", default=None,
|
||||
doc="Batch-mode answer ('yes'/'no' or truthy/falsy). None ⇒ FAILURE."),
|
||||
)
|
||||
|
||||
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
|
||||
self._name = cst.TYPE_QUESTION_DLG.item_name
|
||||
super().__init__(dict_item, parent, status_queue, filename=filename)
|
||||
|
||||
@@ -3,9 +3,17 @@ from interpreter.test_items.test_item import (TestItem, test_run)
|
||||
from interpreter.test_items.test_result import (TestValue)
|
||||
from runtime.tum_except import ETUMSyntaxError
|
||||
from interpreter.utils.constants import TestItemType as cst
|
||||
from interpreter.utils.param_decl import Param, ParamSet, LIST
|
||||
from interpreter.test_report.test_report import Export
|
||||
|
||||
class TestItemReport(TestItem):
|
||||
|
||||
PARAMS = ParamSet(
|
||||
Param("export", kind=LIST, required=True,
|
||||
doc="List of exporters to run (junit, sqlite, …). Each entry is a "
|
||||
"mapping describing the exporter type and its parameters."),
|
||||
)
|
||||
|
||||
def __init__(self, dict_item, parent = None, status_queue=None, filename=""):
|
||||
self._name = cst.TYPE_REPORT.item_name
|
||||
super().__init__(dict_item, parent, status_queue, filename=filename)
|
||||
|
||||
@@ -10,9 +10,42 @@ from interpreter.test_items.test_item import (TestItem, test_run)
|
||||
from interpreter.test_items.test_result import (TestValue)
|
||||
import api.testium as tm
|
||||
from interpreter.utils.constants import TestItemType as cst
|
||||
from interpreter.utils.param_decl import Param, ParamSet
|
||||
from runtime.tum_except import ETUMSyntaxError, ETUMRuntimeError, item_load_context
|
||||
|
||||
|
||||
def _testium_launch_cmd():
|
||||
"""Command prefix to launch a fresh testium instance, runtime-aware.
|
||||
|
||||
AppImage / Flatpak / PyInstaller / wheel / source all need a different
|
||||
entry point than just the path to __main__.py (which may be a .py inside
|
||||
a read-only bundle, or unreachable from the sub-instance's cwd).
|
||||
"""
|
||||
# AppImage: the env var holds the path to the .AppImage file itself.
|
||||
appimage = os.environ.get("APPIMAGE")
|
||||
if appimage:
|
||||
return [appimage]
|
||||
# Flatpak: re-launch via the Flatpak app id, but on the host side —
|
||||
# the `flatpak` CLI cannot run inside our sandbox (no D-Bus access to the
|
||||
# host Flatpak service, and the host binary would need host libs that are
|
||||
# ABI-incompatible with the sandbox runtime). flatpak-spawn proxies the
|
||||
# call to the host via org.freedesktop.Flatpak (allowed by --talk-name in
|
||||
# the manifest).
|
||||
if os.path.isfile("/.flatpak-info"):
|
||||
return ["flatpak-spawn", "--host",
|
||||
"flatpak", "run", "org.testium.Testium"]
|
||||
# PyInstaller frozen exe: sys.executable is the binary itself.
|
||||
if getattr(sys, "frozen", False):
|
||||
return [sys.executable]
|
||||
# Source / wheel: re-use the same Python with the same entry point that
|
||||
# launched this instance, made absolute so cwd changes in the sub-instance
|
||||
# don't break the lookup. argv[0] is either:
|
||||
# - the package directory (source: `python3 src/testium ...`)
|
||||
# - the console_scripts wrapper (wheel: `/usr/bin/testium`)
|
||||
# Both are runnable as `python <argv0>`.
|
||||
return [sys.executable, os.path.abspath(sys.argv[0])]
|
||||
|
||||
|
||||
def nowInBetween(start, end):
|
||||
"""
|
||||
Check wether current time is within boundaries
|
||||
@@ -25,6 +58,25 @@ def nowInBetween(start, end):
|
||||
|
||||
|
||||
class TestItemRun(TestItem):
|
||||
|
||||
PARAMS = ParamSet(
|
||||
Param("tum", required=True,
|
||||
doc="Path to the .tum file launched in a fresh testium instance."),
|
||||
Param("param_file", default="",
|
||||
doc="Optional path to a param.yaml passed to the sub-instance."),
|
||||
Param("log_file", default="",
|
||||
doc="Path where the sub-instance writes its log."),
|
||||
Param("report_file", default="",
|
||||
doc="Path where the sub-instance writes its report."),
|
||||
Param("start_time",
|
||||
doc="HH:MM time of day after which the sub-instance may run."),
|
||||
Param("end_time",
|
||||
doc="HH:MM time of day after which the sub-instance no longer runs."),
|
||||
Param("wait_for_exec",
|
||||
doc="If true, block until the time window opens. Requires both "
|
||||
"start_time and end_time."),
|
||||
)
|
||||
|
||||
def __init__(self, dict_item, parent = None, status_queue=None, filename=""):
|
||||
self._name = cst.TYPE_RUN.item_name
|
||||
super().__init__(dict_item, parent, status_queue, filename=filename)
|
||||
@@ -33,8 +85,6 @@ class TestItemRun(TestItem):
|
||||
with item_load_context(self.cmd(), self.name(), self.seqFilename()):
|
||||
self.tum_file = self._prms.getParam('tum', required=True)
|
||||
self.param_file = self._prms.getParam('param_file', default='')
|
||||
self.python_bin = self._prms.getParam('python_bin', default='')
|
||||
self.testium_path = self._prms.getParam('testium_path', default='')
|
||||
self.log_path = self._prms.getParam('log_file', default='')
|
||||
self.report_path = self._prms.getParam('report_file', default='')
|
||||
self.start_time = self._prms.getParam('start_time')
|
||||
@@ -52,18 +102,9 @@ class TestItemRun(TestItem):
|
||||
'"{}" file could not be found'.format(file_path))
|
||||
self.tum_file = file_path
|
||||
pf = self._prms.expanse(self.param_file)
|
||||
pp = self._prms.expanse(self.python_bin)
|
||||
sp = self._prms.expanse(self.testium_path)
|
||||
lp = self._prms.expanse(self.log_path)
|
||||
rp = self._prms.expanse(self.report_path)
|
||||
cmd = []
|
||||
if sp == '':
|
||||
sp = sys.argv[0]
|
||||
if pp != '':
|
||||
cmd.append(pp)
|
||||
elif not os.path.isfile(sp) or not os.access(sp, os.X_OK):
|
||||
cmd.append(sys.executable)
|
||||
cmd.append(sp)
|
||||
cmd = _testium_launch_cmd()
|
||||
if tm.text_mode():
|
||||
cmd.append("-b")
|
||||
else:
|
||||
|
||||
@@ -11,6 +11,7 @@ from interpreter.test_items.item_actions import TestItemActions
|
||||
from interpreter.test_items.item_actions.action import TestItemAction
|
||||
from interpreter.utils.constants import TestItemType as cst
|
||||
from interpreter.utils.eval import evaluate
|
||||
from interpreter.utils.param_decl import Param, ParamSet, LIST
|
||||
|
||||
|
||||
class TestItemPlotAction(TestItemAction):
|
||||
@@ -21,6 +22,12 @@ class TestItemPlotAction(TestItemAction):
|
||||
|
||||
|
||||
class TestItemPlotActionOpen(TestItemPlotAction):
|
||||
|
||||
PARAMS = ParamSet(
|
||||
Param("log_path", default=None,
|
||||
doc="Optional file to which the plot data are appended."),
|
||||
)
|
||||
|
||||
def __init__(
|
||||
self, action_name, dict_item, parent=None, status_queue=None, filename=""
|
||||
):
|
||||
@@ -57,6 +64,15 @@ class TestItemPlotActionOpen(TestItemPlotAction):
|
||||
|
||||
|
||||
class TestItemPlotActionClose(TestItemPlotAction):
|
||||
|
||||
PARAMS = ParamSet(
|
||||
Param("wait_dialog_exit", default=False,
|
||||
doc="If true, the close action blocks until the user closes the "
|
||||
"plot window (or timeout)."),
|
||||
Param("timeout", default=-1,
|
||||
doc="Seconds to wait when wait_dialog_exit is true. Negative ⇒ infinite."),
|
||||
)
|
||||
|
||||
def __init__(
|
||||
self, action_name, dict_item, parent=None, status_queue=None, filename=""
|
||||
):
|
||||
@@ -96,6 +112,20 @@ class TestItemPlotActionClose(TestItemPlotAction):
|
||||
|
||||
|
||||
class TestItemPlotActionPeriodic(TestItemPlotAction):
|
||||
|
||||
PARAMS = ParamSet(
|
||||
Param("period", required=True,
|
||||
doc="Seconds between two calls of the periodic function."),
|
||||
Param("file", required=True,
|
||||
doc="Path to the .py file holding the periodic function."),
|
||||
Param("func_name", required=True,
|
||||
doc="Name of the periodic function."),
|
||||
Param("param", kind=LIST,
|
||||
doc="Arguments passed to the periodic function on each call."),
|
||||
Param("eval", default="",
|
||||
doc="Post-evaluation applied to the function's return value."),
|
||||
)
|
||||
|
||||
def __init__(
|
||||
self, action_name, dict_item, parent=None, status_queue=None, filename=""
|
||||
):
|
||||
@@ -169,6 +199,13 @@ class TestItemPlotActionAdd(TestItemPlotAction):
|
||||
|
||||
|
||||
class TestItemPlotActionLastValues(TestItemPlotAction):
|
||||
|
||||
PARAMS = ParamSet(
|
||||
Param("name", kind=LIST,
|
||||
doc="List of plot variable names whose last sample is returned. "
|
||||
"Result is stored in $(plv_<plot_name>) as a dict."),
|
||||
)
|
||||
|
||||
def __init__(self, action_name, dict_item, parent=None, status_queue=None, filename=""):
|
||||
super().__init__(
|
||||
action_name, cst.TYPE_GRAPH_ACTION, dict_item, parent, status_queue, filename=filename
|
||||
@@ -219,6 +256,13 @@ class TestItemPlotActionExport(TestItemPlotAction):
|
||||
|
||||
|
||||
class TestItemPlot(TestItemActions):
|
||||
|
||||
PARAMS = ParamSet(
|
||||
Param("plot_name", required=True,
|
||||
doc="Identifier of the plot window — referenced by every nested "
|
||||
"action and by $(plv_<plot_name>) for last-values output."),
|
||||
)
|
||||
|
||||
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
|
||||
super().__init__(
|
||||
cst.TYPE_GRAPH, dict_item, parent, status_queue, filename=filename
|
||||
|
||||
@@ -7,6 +7,7 @@ import api.testium as tm
|
||||
from interpreter.test_items.test_item import (TestItem, test_run)
|
||||
from interpreter.test_items.test_result import (TestValue)
|
||||
from interpreter.utils.constants import TestItemType as cst
|
||||
from interpreter.utils.param_decl import Param, ParamSet
|
||||
from runtime.tum_except import ETUMSyntaxError, ETUMRuntimeError, item_load_context
|
||||
|
||||
class TestItemSleep(TestItem):
|
||||
@@ -14,6 +15,15 @@ class TestItemSleep(TestItem):
|
||||
sleep timeout: 10
|
||||
"""
|
||||
|
||||
PARAMS = ParamSet(
|
||||
Param("timeout", required=True,
|
||||
doc="Duration to sleep. Number of seconds, or a string "
|
||||
"like '1d 2h 30m 15s'."),
|
||||
Param("dialog", default=False,
|
||||
doc="If true, show a cancel dialog (GUI mode) or an interactive "
|
||||
"Ctrl+C-able countdown (text mode)."),
|
||||
)
|
||||
|
||||
def __init__(self, dict_item, parent = None, status_queue=None, filename=""):
|
||||
self._name = cst.TYPE_SLEEP.item_name
|
||||
super().__init__(dict_item, parent, status_queue, filename=filename)
|
||||
@@ -80,4 +90,7 @@ class TestItemSleep(TestItem):
|
||||
end_time = _time.time() + float(timeout)
|
||||
while _time.time() < end_time and not self._is_stopped:
|
||||
sleep(min(0.05, end_time - _time.time()))
|
||||
self.result.set(TestValue.SUCCESS, 'Sleep %s sec' % (str(timeout)))
|
||||
if self._is_stopped:
|
||||
self.result.set(TestValue.FAILURE, 'Sleep aborted on stop request')
|
||||
else:
|
||||
self.result.set(TestValue.SUCCESS, 'Sleep %s sec' % (str(timeout)))
|
||||
|
||||
@@ -2,11 +2,23 @@ from interpreter.test_items.test_item import test_run
|
||||
from interpreter.test_items.test_result import TestValue
|
||||
from interpreter.test_items.test_item_dialog_base import TestItemDialogBase, _is_text_mode, _is_interactive
|
||||
from interpreter.utils.constants import TestItemType as cst
|
||||
from interpreter.utils.param_decl import Param, ParamSet, LIST
|
||||
from runtime.tum_except import item_load_context
|
||||
import api.testium as tm
|
||||
|
||||
|
||||
class TestItemTestedRefsDialog(TestItemDialogBase):
|
||||
|
||||
PARAMS = ParamSet(
|
||||
Param("question", required=True,
|
||||
doc="Prompt asking the operator to enter the tested references."),
|
||||
Param("reference", kind=LIST,
|
||||
doc="Pre-filled list of references shown in the dialog."),
|
||||
Param("auto_result", default=None,
|
||||
doc="Batch-mode outcome: None ⇒ FAILURE, truthy ⇒ SUCCESS with "
|
||||
"the pre-filled references."),
|
||||
)
|
||||
|
||||
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
|
||||
self._name = cst.TYPE_REFERENCE_DLG.item_name
|
||||
super().__init__(dict_item, parent, status_queue, filename=filename)
|
||||
|
||||
@@ -11,6 +11,7 @@ from interpreter.test_items.test_item import (TestItem, test_run, LOG_TEST_STOP,
|
||||
from interpreter.test_items.test_result import (TestResult, TestValue)
|
||||
from interpreter.test_items.test_item import test_data
|
||||
from interpreter.utils.constants import TestItemType as cst
|
||||
from interpreter.utils.param_decl import Param, ParamSet, LIST
|
||||
from runtime.stdout_redirect import stdio_redir
|
||||
|
||||
class UnittestResult(TextTestResult):
|
||||
@@ -95,6 +96,15 @@ class TestItemUnittestElement(TestItem):
|
||||
|
||||
|
||||
class TestItemUnittestFile(TestItem):
|
||||
|
||||
PARAMS = ParamSet(
|
||||
Param("test_file", required=True,
|
||||
doc="Path to the Python unittest file (TestCase subclass)."),
|
||||
Param("test_method", kind=LIST,
|
||||
doc="Optional list of method names to restrict the run to. "
|
||||
"When empty, every test_* method in the file is run."),
|
||||
)
|
||||
|
||||
def __init__(self, dict_item, parent = None, status_queue=None, filename=""):
|
||||
self._name = cst.TYPE_UNITTEST.item_name
|
||||
super().__init__(dict_item, parent, status_queue, filename=filename)
|
||||
|
||||
@@ -2,6 +2,7 @@ from interpreter.test_items.test_item import test_run
|
||||
from interpreter.test_items.test_result import TestValue
|
||||
from interpreter.test_items.test_item_dialog_base import TestItemDialogBase, _is_text_mode, _is_interactive
|
||||
from interpreter.utils.constants import TestItemType as cst
|
||||
from interpreter.utils.param_decl import Param, ParamSet
|
||||
from runtime.tum_except import item_load_context
|
||||
import api.testium as tm
|
||||
|
||||
@@ -10,6 +11,19 @@ class TestItemValueDialog(TestItemDialogBase):
|
||||
"""dialog_value item usage.
|
||||
dialog_value name: Enter value, question: "Which value did you measure?"
|
||||
"""
|
||||
|
||||
PARAMS = ParamSet(
|
||||
Param("question", required=True,
|
||||
doc="Prompt shown above the value input field."),
|
||||
Param("default", default="",
|
||||
doc="Pre-filled value of the input field."),
|
||||
Param("auto_result", default=None,
|
||||
doc="Batch-mode outcome: None ⇒ FAILURE, 'cancel' ⇒ cancelled, "
|
||||
"any other truthy ⇒ SUCCESS with auto_value."),
|
||||
Param("auto_value", default=None,
|
||||
doc="Value used in batch mode when auto_result is set."),
|
||||
)
|
||||
|
||||
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
|
||||
self._name = cst.TYPE_VALUE_DLG.item_name
|
||||
super().__init__(dict_item, parent, status_queue, filename=filename)
|
||||
|
||||
@@ -17,8 +17,11 @@ Public API
|
||||
``reset()`` : clear the cache (mostly useful for tests)
|
||||
"""
|
||||
|
||||
import atexit
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
import tempfile
|
||||
|
||||
import api.testium as tm
|
||||
from interpreter.utils.paths import sys_app_path_lin, sys_app_path_win
|
||||
@@ -30,23 +33,235 @@ from runtime.tum_except import ETUMRuntimeError
|
||||
_PYTHON_CANDIDATES = ["python3", "python"]
|
||||
_LUA_CANDIDATES = ["lua", "lua5.5", "lua5.4", "lua5.3", "lua5.2", "lua5.1"]
|
||||
|
||||
# Inside an AppImage, AppRun prepends $APPDIR/usr/bin to PATH and exports a
|
||||
# bundle-local PYTHONHOME / PYTHONPATH / LD_LIBRARY_PATH. We want py_func and
|
||||
# lua_func to run under the *host* interpreter (not the bundled one), so we
|
||||
# probe standard host bin dirs directly and scrub APPDIR-prefixed entries from
|
||||
# the env passed to host subprocesses.
|
||||
_APPIMAGE_HOST_DIRS = [
|
||||
"/usr/local/bin",
|
||||
"/usr/bin",
|
||||
"/bin",
|
||||
]
|
||||
|
||||
|
||||
def _in_flatpak():
|
||||
return os.path.isfile("/.flatpak-info")
|
||||
|
||||
|
||||
def _in_appimage():
|
||||
return "APPIMAGE" in os.environ
|
||||
|
||||
|
||||
def apply_host_libs(env):
|
||||
"""Strip bundle-local entries from *env* so a host binary can run cleanly.
|
||||
|
||||
Only meaningful for AppImage: removes $APPDIR-prefixed entries from
|
||||
LD_LIBRARY_PATH / PYTHONPATH / PATH and drops PYTHONHOME, so the host
|
||||
interpreter doesn't try to load the bundled (incompatible) Python
|
||||
lib/site-packages. Flatpak is handled via flatpak-spawn --host instead
|
||||
(see flatpak_host_spawn), so the sandbox env is irrelevant there.
|
||||
"""
|
||||
if not _in_appimage():
|
||||
return
|
||||
appdir = os.environ.get("APPDIR", "")
|
||||
if appdir:
|
||||
for var, sep in (("LD_LIBRARY_PATH", ":"),
|
||||
("PYTHONPATH", os.pathsep),
|
||||
("PATH", os.pathsep)):
|
||||
cur = env.get(var, "")
|
||||
if not cur:
|
||||
continue
|
||||
cleaned = sep.join(
|
||||
p for p in cur.split(sep)
|
||||
if p and not p.startswith(appdir)
|
||||
)
|
||||
if cleaned:
|
||||
env[var] = cleaned
|
||||
else:
|
||||
env.pop(var, None)
|
||||
env.pop("PYTHONHOME", None)
|
||||
|
||||
|
||||
# ---------- Flatpak: spawn on host outside the sandbox -----------------------
|
||||
#
|
||||
# Inside a Flatpak the sandbox glibc is incompatible with host shared libraries,
|
||||
# so we can't run host Python/Lua under the sandbox runtime — `LD_LIBRARY_PATH`
|
||||
# tricks hit a `_dl_call_libc_early_init` assertion. The supported way out is
|
||||
# `flatpak-spawn --host`, which talks to the session-bus Flatpak D-Bus service
|
||||
# (org.freedesktop.Flatpak.Development) and asks it to spawn a process in the
|
||||
# host execution environment instead of inside our sandbox. The manifest must
|
||||
# grant `--talk-name=org.freedesktop.Flatpak` for the D-Bus call to be allowed.
|
||||
#
|
||||
# The host process can't see our /app/ contents (sandbox-only), so when we
|
||||
# spawn host Python/Lua to run `py_func` / `lua_func`, the cwd must be a
|
||||
# directory both sides can reach. /tmp is shared (--filesystem=/tmp), so we
|
||||
# stage the testium package there once per process and reuse it for every
|
||||
# spawn. In source mode (testium under $HOME) the host already sees the
|
||||
# original path, so we skip the copy.
|
||||
|
||||
_staged_testium_path = None
|
||||
|
||||
|
||||
def _get_host_testium_path():
|
||||
"""Return a path to the testium package that the host can read.
|
||||
|
||||
- Source / wheel / PyInstaller install under $HOME → return testium_path()
|
||||
as-is (host sees the same path via --filesystem=home).
|
||||
- Flatpak bundle (testium under /app/) → stage a copy under /tmp on first
|
||||
call and reuse it for the rest of the process.
|
||||
"""
|
||||
global _staged_testium_path
|
||||
if _staged_testium_path is not None:
|
||||
return _staged_testium_path
|
||||
|
||||
# Imported lazily to avoid a circular import (paths.py -> api.testium).
|
||||
from interpreter.utils.paths import testium_path
|
||||
tp = testium_path()
|
||||
|
||||
if not tp.startswith("/app/"):
|
||||
_staged_testium_path = tp
|
||||
return tp
|
||||
|
||||
staged = tempfile.mkdtemp(prefix="testium_host_", dir="/tmp")
|
||||
# copytree refuses to write into an existing dir unless dirs_exist_ok=True.
|
||||
# mkdtemp creates the dir, so we copy *into* it.
|
||||
for entry in os.listdir(tp):
|
||||
src = os.path.join(tp, entry)
|
||||
dst = os.path.join(staged, entry)
|
||||
if os.path.isdir(src):
|
||||
shutil.copytree(src, dst, symlinks=True)
|
||||
else:
|
||||
shutil.copy2(src, dst, follow_symlinks=False)
|
||||
_staged_testium_path = staged
|
||||
atexit.register(shutil.rmtree, staged, ignore_errors=True)
|
||||
return staged
|
||||
|
||||
|
||||
_FORWARDED_ENV_KEYS = (
|
||||
"HOME", "USER", "LOGNAME", "TMPDIR",
|
||||
"XDG_RUNTIME_DIR", "XDG_DATA_HOME", "XDG_CONFIG_HOME", "XDG_CACHE_HOME",
|
||||
"DBUS_SESSION_BUS_ADDRESS", "DISPLAY", "WAYLAND_DISPLAY",
|
||||
"LANG", "LC_ALL",
|
||||
)
|
||||
|
||||
|
||||
def flatpak_host_spawn(interp_bin, cmd_args, host_cwd, extra_env=None):
|
||||
"""Build a flatpak-spawn --host command vector.
|
||||
|
||||
Args:
|
||||
interp_bin: absolute path to the host interpreter (e.g. /usr/bin/python3).
|
||||
cmd_args: list of arguments passed to the interpreter.
|
||||
host_cwd: working directory on the host (must be reachable from host).
|
||||
extra_env: optional {name: value} of env vars to set on the host side
|
||||
in addition to the default forwarded set. Values of ""
|
||||
unset the variable on the host.
|
||||
|
||||
Returns a list suitable for subprocess.Popen.
|
||||
"""
|
||||
spawn = ["flatpak-spawn", "--host", f"--directory={host_cwd}"]
|
||||
forwarded = {}
|
||||
for key in _FORWARDED_ENV_KEYS:
|
||||
val = os.environ.get(key)
|
||||
if val:
|
||||
forwarded[key] = val
|
||||
if extra_env:
|
||||
forwarded.update(extra_env)
|
||||
for k, v in forwarded.items():
|
||||
if v == "":
|
||||
spawn.append(f"--unset-env={k}")
|
||||
else:
|
||||
spawn.append(f"--env={k}={v}")
|
||||
spawn.append(interp_bin)
|
||||
spawn.extend(cmd_args)
|
||||
return spawn
|
||||
|
||||
|
||||
def _which_host_flatpak(name):
|
||||
"""Resolve a binary name (or absolute path) on the host via flatpak-spawn.
|
||||
|
||||
We can't probe /run/host/... because (a) only host-os is mounted there,
|
||||
not arbitrary paths like /scratch, and (b) returning a /run/host path
|
||||
would be useless — the host-side spawn sees a different filesystem and
|
||||
needs the host-native path anyway.
|
||||
"""
|
||||
if os.path.isabs(name):
|
||||
cmd = flatpak_host_spawn("/bin/sh", ["-c", f'test -x "{name}"'],
|
||||
host_cwd="/tmp")
|
||||
try:
|
||||
r = subprocess.run(cmd, capture_output=True, timeout=10)
|
||||
except (FileNotFoundError, PermissionError, subprocess.TimeoutExpired):
|
||||
return ""
|
||||
return name if r.returncode == 0 else ""
|
||||
cmd = flatpak_host_spawn("/bin/sh", ["-c", f'command -v "{name}"'],
|
||||
host_cwd="/tmp")
|
||||
try:
|
||||
r = subprocess.run(cmd, capture_output=True, text=True, timeout=10)
|
||||
except (FileNotFoundError, PermissionError, subprocess.TimeoutExpired):
|
||||
return ""
|
||||
if r.returncode != 0:
|
||||
return ""
|
||||
return r.stdout.strip()
|
||||
|
||||
|
||||
def _which(name):
|
||||
func = sys_app_path_win if tm.OS() == "Windows" else sys_app_path_lin
|
||||
return func(name)
|
||||
if tm.OS() == "Windows":
|
||||
return sys_app_path_win(name)
|
||||
if _in_flatpak():
|
||||
return _which_host_flatpak(name)
|
||||
if _in_appimage():
|
||||
for d in _APPIMAGE_HOST_DIRS:
|
||||
p = os.path.join(d, name)
|
||||
if os.path.isfile(p) and os.access(p, os.X_OK):
|
||||
return p
|
||||
return ""
|
||||
return sys_app_path_lin(name)
|
||||
|
||||
|
||||
def _python_version(path):
|
||||
cmd = [path, "-c", "import sys; print(sys.version_info[:3])"]
|
||||
def _probe_env():
|
||||
"""Subprocess env for probing host binaries.
|
||||
|
||||
In AppImage we still need to scrub APPDIR-prefixed entries; in Flatpak we
|
||||
delegate execution to the host via flatpak-spawn so the sandbox env doesn't
|
||||
matter, but apply_host_libs is a no-op cost.
|
||||
"""
|
||||
env = os.environ.copy()
|
||||
apply_host_libs(env)
|
||||
return env
|
||||
|
||||
|
||||
def _run_probe(cmd):
|
||||
"""Run a probe command, dispatching through flatpak-spawn --host in Flatpak.
|
||||
|
||||
Returns (stdout, stderr) as str, or None on failure.
|
||||
"""
|
||||
if _in_flatpak():
|
||||
spawn = flatpak_host_spawn(cmd[0], cmd[1:], host_cwd="/tmp")
|
||||
try:
|
||||
r = subprocess.run(
|
||||
spawn, capture_output=True, text=True, timeout=10,
|
||||
)
|
||||
except (FileNotFoundError, PermissionError, subprocess.TimeoutExpired):
|
||||
return None
|
||||
if r.returncode != 0:
|
||||
return None
|
||||
return r.stdout, r.stderr
|
||||
try:
|
||||
r = subprocess.run(
|
||||
cmd, capture_output=True, text=True,
|
||||
encoding=tm.sys_encoding(), timeout=10,
|
||||
encoding=tm.sys_encoding(), timeout=10, env=_probe_env(),
|
||||
)
|
||||
except (FileNotFoundError, PermissionError, subprocess.TimeoutExpired):
|
||||
return None
|
||||
return r.stdout, r.stderr
|
||||
|
||||
|
||||
def _python_version(path):
|
||||
out = _run_probe([path, "-c", "import sys; print(sys.version_info[:3])"])
|
||||
if out is None:
|
||||
return None
|
||||
try:
|
||||
return eval(r.stdout)
|
||||
return eval(out[0])
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
@@ -57,14 +272,11 @@ def _is_python3(path):
|
||||
|
||||
|
||||
def _lua_version(path):
|
||||
try:
|
||||
r = subprocess.run(
|
||||
[path, "-v"], capture_output=True, text=True, timeout=10,
|
||||
)
|
||||
except (FileNotFoundError, PermissionError, subprocess.TimeoutExpired):
|
||||
out = _run_probe([path, "-v"])
|
||||
if out is None:
|
||||
return None
|
||||
# On Windows the version banner goes to stderr.
|
||||
line = r.stdout or r.stderr
|
||||
line = out[0] or out[1]
|
||||
try:
|
||||
major, minor, _patch = line.split(" ")[1].split(".")
|
||||
return (int(major), int(minor))
|
||||
@@ -85,20 +297,39 @@ _SPECS = {
|
||||
"lua": ("Lua 5.1+", "lua_bin", _LUA_CANDIDATES, _is_lua51),
|
||||
}
|
||||
|
||||
# Cached per (name, override) so that runtime changes to gd[gd_key] —
|
||||
# e.g. ``python_bin`` set from a YAML config file loaded *after*
|
||||
# eval_proc has already resolved its own interpreter — are picked up by
|
||||
# the next lookup instead of returning the stale, auto-discovered path.
|
||||
# Long-lived subprocesses (eval_proc) keep whatever they captured at
|
||||
# construction time, but every new PyProcessBase / FuncExecEngine spawned
|
||||
# afterwards sees the current override.
|
||||
_resolved = {}
|
||||
|
||||
|
||||
def _resolve(name):
|
||||
if name in _resolved:
|
||||
return _resolved[name]
|
||||
|
||||
display, gd_key, candidates, validator = _SPECS[name]
|
||||
override = tm.gd(gd_key, "") or ""
|
||||
|
||||
cached = _resolved.get(name)
|
||||
if cached is not None and cached[0] == override:
|
||||
return cached[1]
|
||||
|
||||
path = ""
|
||||
if override:
|
||||
if shutil.which(override) and validator(override):
|
||||
path = override
|
||||
# Absolute path: accept as-is (user knows exactly what they want).
|
||||
# Bare name: resolve via _which() so the override stays host-only in
|
||||
# Flatpak/AppImage instead of silently picking the bundled interpreter.
|
||||
# In Flatpak we always defer to _which() so even absolute paths are
|
||||
# checked from the host's perspective (the sandbox can't see e.g.
|
||||
# /scratch/... paths that the user may have configured).
|
||||
if os.path.isabs(override) and not _in_flatpak():
|
||||
resolved = override if (os.path.isfile(override)
|
||||
and os.access(override, os.X_OK)) else ""
|
||||
else:
|
||||
resolved = _which(override)
|
||||
if resolved and validator(resolved):
|
||||
path = resolved
|
||||
else:
|
||||
tm.print_warn(
|
||||
f"Configured {display} interpreter '{override}' is not usable; "
|
||||
@@ -114,7 +345,7 @@ def _resolve(name):
|
||||
path = p
|
||||
break
|
||||
|
||||
_resolved[name] = path
|
||||
_resolved[name] = (override, path)
|
||||
return path
|
||||
|
||||
|
||||
|
||||
@@ -8,6 +8,7 @@ from runtime.jrpc import JsonRpcClient
|
||||
from interpreter.utils.paths import subproc_path
|
||||
from runtime.tum_except import ETUMRuntimeError
|
||||
from interpreter.utils import bins
|
||||
from interpreter.utils.proc_drain import drain_to_log
|
||||
|
||||
|
||||
class LuaProcessBase:
|
||||
@@ -46,9 +47,16 @@ class LuaProcessBase:
|
||||
if self._process is not None:
|
||||
raise ETUMRuntimeError("The function subprocess has already been started.")
|
||||
|
||||
func_proc_path = os.path.realpath(
|
||||
os.path.join(subproc_path(), "lua_func")
|
||||
)
|
||||
# In Flatpak the host can't see /app/lib/testium/lua_func, so use a
|
||||
# staged copy under /tmp (shared between sandbox and host).
|
||||
if bins._in_flatpak():
|
||||
func_proc_path = os.path.join(
|
||||
bins._get_host_testium_path(), "lua_func"
|
||||
)
|
||||
else:
|
||||
func_proc_path = os.path.realpath(
|
||||
os.path.join(subproc_path(), "lua_func")
|
||||
)
|
||||
|
||||
# POpen config
|
||||
CUST_ENV = {
|
||||
@@ -59,6 +67,7 @@ class LuaProcessBase:
|
||||
|
||||
lua_env = tm.gd("lua_env", {})
|
||||
env = os.environ.copy()
|
||||
bins.apply_host_libs(env)
|
||||
if not isinstance(lua_env, dict):
|
||||
raise ETUMRuntimeError(f"The 'lua_env' global value should be a dictionary. But it is '{lua_env}'.")
|
||||
|
||||
@@ -76,8 +85,7 @@ class LuaProcessBase:
|
||||
sock.close()
|
||||
|
||||
# POpen params
|
||||
params = [
|
||||
self._lbin,
|
||||
cmd_args = [
|
||||
"main.lua",
|
||||
"--timeout",
|
||||
f"{self._timeout}",
|
||||
@@ -88,15 +96,36 @@ class LuaProcessBase:
|
||||
]
|
||||
|
||||
if tm.debug_enabled() and tm.gd("debug_rpc", False):
|
||||
params.append("--verbose")
|
||||
cmd_args.append("--verbose")
|
||||
|
||||
if bins._in_flatpak():
|
||||
# Run on the host outside the sandbox: avoids glibc ABI mismatches
|
||||
# between the Flatpak runtime and host shared libraries.
|
||||
host_env = {
|
||||
k: env[k] for k in ("LUA_PATH", "LUA_CPATH", "PATH")
|
||||
if k in env and env[k]
|
||||
}
|
||||
params = bins.flatpak_host_spawn(
|
||||
self._lbin, cmd_args, host_cwd=func_proc_path,
|
||||
extra_env=host_env,
|
||||
)
|
||||
popen_kwargs = {}
|
||||
else:
|
||||
params = [self._lbin, *cmd_args]
|
||||
popen_kwargs = {"env": env, "cwd": func_proc_path}
|
||||
|
||||
self._process = subprocess.Popen(
|
||||
params, env=env, cwd=func_proc_path,
|
||||
params,
|
||||
stdin=subprocess.DEVNULL,
|
||||
stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.DEVNULL,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.PIPE,
|
||||
restore_signals=False,
|
||||
**popen_kwargs,
|
||||
)
|
||||
# Route subprocess stdout/stderr (lua require failures, syntax
|
||||
# errors, anything written to fd 1/2 before the in-script
|
||||
# remote_print is set up) into the parent's log.
|
||||
drain_to_log(self._process, prefix="[lua_func] ")
|
||||
|
||||
self._rpc = JsonRpcClient(
|
||||
"localhost", self._port, req_handler=self._req_handler
|
||||
@@ -139,4 +168,12 @@ class LuaProcessBase:
|
||||
"""
|
||||
if self._rpc is not None:
|
||||
self._rpc.stop()
|
||||
# Force-kill the worker if it's still running. Needed when user code
|
||||
# in the worker is stuck and won't notice the parent closing the RPC
|
||||
# socket on its own.
|
||||
if self._process is not None and self._process.poll() is None:
|
||||
try:
|
||||
self._process.terminate()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
175
src/testium/interpreter/utils/param_decl.py
Normal file
175
src/testium/interpreter/utils/param_decl.py
Normal file
@@ -0,0 +1,175 @@
|
||||
"""Declarative description of a test item's accepted parameters.
|
||||
|
||||
Each ``TestItem`` subclass declares its parameter surface as a class
|
||||
attribute::
|
||||
|
||||
class TestItemFoo(TestItem):
|
||||
PARAMS = ParamSet(
|
||||
Param("bar", required=True, doc="The bar value."),
|
||||
Param("baz", default=0, doc="Optional baz."),
|
||||
Param("modes", kind=LIST, doc="Iterable of modes."),
|
||||
Param("strategy", kind=ENUM("a", "b"), doc="..."),
|
||||
Param("opts", kind=BLOCK, doc="Sub-block."),
|
||||
)
|
||||
|
||||
The base ``TestItem.__init__`` consumes both ``COMMON_PARAMS`` (defined
|
||||
in ``test_item.py``) and the subclass ``PARAMS`` to:
|
||||
|
||||
* warn on any key in the user's YAML that isn't declared anywhere
|
||||
(catches typos like ``param_filee``);
|
||||
* expose a machine-readable schema for documentation generation and,
|
||||
eventually, an LSP server.
|
||||
|
||||
The descriptor is **purely about shape and naming**. Type coercion and
|
||||
runtime checking of expanded values remain the responsibility of each
|
||||
item's ``execute()`` method — most parameters are expressions
|
||||
(``$(...)`` / ``<| ... |>``) whose effective type is only known after
|
||||
expansion, so a static type would be misleading.
|
||||
|
||||
Validation of *values* (e.g. ``start_time`` must match HH:MM) can be
|
||||
attached per-param via ``validate=lambda v: ...`` and is applied at
|
||||
execution time on the expanded value, not at load time.
|
||||
"""
|
||||
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any, Callable, Optional, Union
|
||||
|
||||
|
||||
# ---------- Parameter "kinds" -------------------------------------------------
|
||||
#
|
||||
# These describe the YAML *shape* expected for a parameter, not its
|
||||
# semantic type. They drive the LSP completion (do we suggest a single
|
||||
# value, a list, a sub-block, an enum picker?) and the unknown-param
|
||||
# diagnostic; nothing more.
|
||||
|
||||
SCALAR = "scalar" # single value (string, number, bool, expression, ...)
|
||||
LIST = "list" # YAML list — the historical ``getParamAll`` case
|
||||
BLOCK = "block" # nested dict — e.g. ``cycle.exit:``
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Enum:
|
||||
"""Closed enumeration of acceptable scalar values."""
|
||||
values: tuple
|
||||
|
||||
def __init__(self, *values):
|
||||
# frozen=True forbids assignment; bypass via object.__setattr__.
|
||||
object.__setattr__(self, "values", tuple(values))
|
||||
|
||||
def __repr__(self):
|
||||
return f"Enum({', '.join(repr(v) for v in self.values)})"
|
||||
|
||||
|
||||
Kind = Union[str, Enum]
|
||||
|
||||
|
||||
# ---------- The descriptor ----------------------------------------------------
|
||||
|
||||
_MISSING = object()
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Param:
|
||||
"""Declarative description of one accepted parameter.
|
||||
|
||||
Attributes
|
||||
----------
|
||||
name : str
|
||||
The YAML key.
|
||||
kind : ``SCALAR`` (default) | ``LIST`` | ``BLOCK`` | ``Enum(...)``
|
||||
The YAML shape expected.
|
||||
required : bool
|
||||
If True, missing the parameter is a load-time error.
|
||||
default : Any
|
||||
Default value when the parameter is absent. ``_MISSING`` when no
|
||||
default was set (used to distinguish "absent" from "None").
|
||||
doc : str
|
||||
Free-form description used for hover / generated documentation.
|
||||
validate : Optional[Callable[[Any], bool]]
|
||||
Optional post-expansion validator, evaluated at ``execute()``
|
||||
time on the effective (expanded) value. Returning ``False``
|
||||
raises a clear error pointing at the param.
|
||||
"""
|
||||
name: str
|
||||
kind: Kind = SCALAR
|
||||
required: bool = False
|
||||
default: Any = _MISSING
|
||||
doc: str = ""
|
||||
validate: Optional[Callable[[Any], bool]] = None
|
||||
|
||||
def has_default(self):
|
||||
return self.default is not _MISSING
|
||||
|
||||
def to_schema(self):
|
||||
"""Return a dict suitable for JSON Schema generation."""
|
||||
s = {"name": self.name, "required": self.required, "doc": self.doc}
|
||||
if isinstance(self.kind, Enum):
|
||||
s["kind"] = "enum"
|
||||
s["enum"] = list(self.kind.values)
|
||||
else:
|
||||
s["kind"] = self.kind
|
||||
if self.has_default():
|
||||
s["default"] = self.default
|
||||
return s
|
||||
|
||||
|
||||
class ParamSet:
|
||||
"""Ordered, name-indexed collection of ``Param`` descriptors.
|
||||
|
||||
Supports concatenation (``COMMON_PARAMS + SUBCLASS_PARAMS``) to
|
||||
merge the common surface with each item's own params. Later
|
||||
declarations override earlier ones (so a subclass can tighten a
|
||||
common param's docstring without redeclaring everything).
|
||||
"""
|
||||
|
||||
def __init__(self, *params):
|
||||
self._params = {}
|
||||
for p in params:
|
||||
self.add(p)
|
||||
|
||||
def add(self, param):
|
||||
if not isinstance(param, Param):
|
||||
raise TypeError(f"ParamSet only accepts Param instances, got {type(param).__name__}")
|
||||
self._params[param.name] = param
|
||||
|
||||
def __iter__(self):
|
||||
return iter(self._params.values())
|
||||
|
||||
def __contains__(self, name):
|
||||
return name in self._params
|
||||
|
||||
def __getitem__(self, name):
|
||||
return self._params[name]
|
||||
|
||||
def names(self):
|
||||
return tuple(self._params.keys())
|
||||
|
||||
def __add__(self, other):
|
||||
if not isinstance(other, ParamSet):
|
||||
return NotImplemented
|
||||
merged = ParamSet()
|
||||
merged._params = {**self._params, **other._params}
|
||||
return merged
|
||||
|
||||
def to_schema(self):
|
||||
return [p.to_schema() for p in self._params.values()]
|
||||
|
||||
|
||||
# ---------- Validation primitives --------------------------------------------
|
||||
|
||||
def unknown_keys(declared, user_dict):
|
||||
"""Return the user-provided keys that are not declared in *declared*.
|
||||
|
||||
*declared* is a ``ParamSet``; *user_dict* is the raw YAML mapping
|
||||
for the item. Unknown keys catch typos and obsolete parameters.
|
||||
"""
|
||||
if not isinstance(user_dict, dict):
|
||||
return ()
|
||||
return tuple(k for k in user_dict.keys() if k not in declared)
|
||||
|
||||
|
||||
def missing_required(declared, user_dict):
|
||||
"""Return the names of declared required params absent from *user_dict*."""
|
||||
if not isinstance(user_dict, dict):
|
||||
return tuple(p.name for p in declared if p.required)
|
||||
return tuple(p.name for p in declared if p.required and p.name not in user_dict)
|
||||
48
src/testium/interpreter/utils/proc_drain.py
Normal file
48
src/testium/interpreter/utils/proc_drain.py
Normal file
@@ -0,0 +1,48 @@
|
||||
"""Drain a subprocess stdout/stderr into testium's print pipeline.
|
||||
|
||||
Captured lines go through the parent's stdio_redir, so they reach the
|
||||
test log AND the live output (terminal in batch mode, GUI text panel
|
||||
in -r mode). This is essential for diagnosing early-startup errors
|
||||
of py_func / lua_func subprocesses (missing modules, unhandled
|
||||
exceptions before the in-process redirection kicks in, lua
|
||||
``require`` failures, anything written to fd 1/2 directly).
|
||||
"""
|
||||
import threading
|
||||
|
||||
|
||||
def _drain_pipe(pipe, prefix):
|
||||
try:
|
||||
for raw in iter(pipe.readline, b""):
|
||||
line = raw.decode("utf-8", errors="replace").rstrip("\r\n")
|
||||
if not line:
|
||||
continue
|
||||
if prefix:
|
||||
print(f"{prefix}{line}")
|
||||
else:
|
||||
print(line)
|
||||
finally:
|
||||
try:
|
||||
pipe.close()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
def drain_to_log(process, prefix=""):
|
||||
"""Spawn daemon threads that read ``process.stdout`` and
|
||||
``process.stderr`` line by line and print each line through the
|
||||
parent's stdout (so it reaches the log + live output).
|
||||
|
||||
Each thread exits cleanly when the subprocess closes the
|
||||
corresponding pipe (i.e. when it exits). Daemon flag ensures they
|
||||
do not block testium exit.
|
||||
"""
|
||||
threads = []
|
||||
for pipe in (process.stdout, process.stderr):
|
||||
if pipe is None:
|
||||
continue
|
||||
t = threading.Thread(
|
||||
target=_drain_pipe, args=(pipe, prefix), daemon=True,
|
||||
)
|
||||
t.start()
|
||||
threads.append(t)
|
||||
return threads
|
||||
@@ -7,6 +7,7 @@ import api.testium as tm
|
||||
from runtime.tum_except import ETUMRuntimeError
|
||||
from interpreter.utils.paths import testium_path, subproc_path
|
||||
from interpreter.utils import bins
|
||||
from interpreter.utils.proc_drain import drain_to_log
|
||||
|
||||
|
||||
class PyProcessBase:
|
||||
@@ -41,6 +42,10 @@ class PyProcessBase:
|
||||
raise ETUMRuntimeError(f"The 'py_env' global value should be a dictionary. But it is '{py_env}'.")
|
||||
|
||||
env = os.environ.copy()
|
||||
bins.apply_host_libs(env)
|
||||
# PYTHONUSERBASE is set by the Flatpak runtime to isolate sandbox
|
||||
# user packages; remove it so the host Python finds ~/.local packages.
|
||||
env.pop("PYTHONUSERBASE", None)
|
||||
for k, v in self.CUST_ENV.items():
|
||||
e = py_env.get(k, "")
|
||||
if e != "":
|
||||
@@ -56,14 +61,18 @@ class PyProcessBase:
|
||||
if sock is not None:
|
||||
sock.close()
|
||||
|
||||
# Add the path of the subprocess (root sources of testium)
|
||||
tstium_path = os.path.realpath(testium_path())
|
||||
func_proc_path = os.path.realpath(subproc_path())
|
||||
# In Flatpak the host can't see /app/lib/testium, so use a staged copy
|
||||
# under /tmp (shared between sandbox and host) for both cwd and as the
|
||||
# root in PYTHONPATH. Outside Flatpak the original paths are used.
|
||||
if bins._in_flatpak():
|
||||
tstium_path = bins._get_host_testium_path()
|
||||
func_proc_path = tstium_path
|
||||
else:
|
||||
tstium_path = os.path.realpath(testium_path())
|
||||
func_proc_path = os.path.realpath(subproc_path())
|
||||
env["PYTHONPATH"] = tstium_path + os.pathsep + self._ppath + os.pathsep + env.get("PYTHONPATH", "")
|
||||
|
||||
params = [
|
||||
self._pbin,
|
||||
# "-m",
|
||||
cmd_args = [
|
||||
"py_func",
|
||||
"-p",
|
||||
f"{self._port}",
|
||||
@@ -72,15 +81,37 @@ class PyProcessBase:
|
||||
]
|
||||
|
||||
if tm.debug_enabled() and tm.gd("debug_rpc", False):
|
||||
params.append("-v")
|
||||
cmd_args.append("-v")
|
||||
|
||||
if bins._in_flatpak():
|
||||
# Run on the host outside the sandbox: avoids glibc ABI mismatches
|
||||
# between the Flatpak runtime and host shared libraries.
|
||||
host_env = {
|
||||
k: env[k] for k in ("PYTHONPATH", "PATH")
|
||||
if k in env and env[k]
|
||||
}
|
||||
params = bins.flatpak_host_spawn(
|
||||
self._pbin, cmd_args, host_cwd=func_proc_path,
|
||||
extra_env=host_env,
|
||||
)
|
||||
popen_kwargs = {}
|
||||
else:
|
||||
params = [self._pbin, *cmd_args]
|
||||
popen_kwargs = {"env": env, "cwd": func_proc_path}
|
||||
|
||||
self._process = subprocess.Popen(
|
||||
params, env=env, cwd=func_proc_path,
|
||||
params,
|
||||
stdin=subprocess.DEVNULL,
|
||||
stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.DEVNULL,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.PIPE,
|
||||
restore_signals=False,
|
||||
**popen_kwargs,
|
||||
)
|
||||
# Route subprocess stdout/stderr (early-startup errors,
|
||||
# unhandled exceptions, anything written to fd 1/2 before the
|
||||
# in-process JSON-RPC stdio_redir kicks in) into the parent's
|
||||
# log.
|
||||
drain_to_log(self._process, prefix="[py_func] ")
|
||||
|
||||
self._rpc = JsonRpcClient(
|
||||
"localhost", self._port, req_handler=self._req_handler
|
||||
@@ -113,3 +144,11 @@ class PyProcessBase:
|
||||
def stop(self):
|
||||
if self._rpc is not None:
|
||||
self._rpc.stop()
|
||||
# Force-kill the worker if it's still running. Needed when user code
|
||||
# in the worker is stuck (e.g. sleep, blocking I/O) and won't notice
|
||||
# the parent closing the RPC socket on its own.
|
||||
if self._process is not None and self._process.poll() is None:
|
||||
try:
|
||||
self._process.terminate()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
@@ -165,11 +165,14 @@ def env_init():
|
||||
_constants_init()
|
||||
|
||||
|
||||
def update_global(config_files, defines, gui_defaults, silent=False):
|
||||
"""Global dict updated with the content of the config file and a dict provided.
|
||||
this function returns the resulting dict.
|
||||
def apply_overrides(defines, gui_defaults):
|
||||
"""Push GUI defaults then CLI defines into the global dict.
|
||||
|
||||
Extracted from update_global so it can be called *before* eval_proc
|
||||
starts: interpreter overrides (python_bin, lua_bin) must be visible
|
||||
to bins.python_bin() on its first lookup, which happens during
|
||||
eval_process_init.
|
||||
"""
|
||||
# GUI preferences applied first
|
||||
for k, v in gui_defaults.items():
|
||||
try:
|
||||
val = ast.literal_eval(v)
|
||||
@@ -177,7 +180,6 @@ def update_global(config_files, defines, gui_defaults, silent=False):
|
||||
val = v
|
||||
tm.setgd(k, val)
|
||||
|
||||
# Then command line defines
|
||||
for k, v in defines.items():
|
||||
try:
|
||||
val = ast.literal_eval(v)
|
||||
@@ -185,6 +187,14 @@ def update_global(config_files, defines, gui_defaults, silent=False):
|
||||
val = v
|
||||
tm.setgd(k, val)
|
||||
|
||||
|
||||
def update_global(config_files, defines, gui_defaults, silent=False):
|
||||
"""Global dict updated with the content of the config file and a dict provided.
|
||||
this function returns the resulting dict.
|
||||
"""
|
||||
# GUI preferences applied first, then command line defines
|
||||
apply_overrides(defines, gui_defaults)
|
||||
|
||||
# Then the configuration files
|
||||
# load global dic before test item
|
||||
_feed_gd_with_params(config_files, silent)
|
||||
|
||||
@@ -31,39 +31,47 @@ def get_version(path :str)-> str:
|
||||
return "Warning git not supported in your settings, version of {} unknown".format(path)
|
||||
|
||||
def get_testium_version():
|
||||
# case where we're executing from an Appimage
|
||||
if 'APPIMAGE' in os.environ:
|
||||
ver = 'unknown'
|
||||
if 'SEQUENCER_REV' in os.environ:
|
||||
ver = os.getenv('SEQUENCER_REV')
|
||||
return (ver + " (binary release)")
|
||||
# Flatpak bundle
|
||||
if os.path.isfile('/.flatpak-info'):
|
||||
ver = os.environ.get('TESTIUM_VERSION', '').strip()
|
||||
return (ver if ver else 'unknown') + " (flatpak release)"
|
||||
|
||||
# case where we're executing from pyinstaller exe
|
||||
# AppImage
|
||||
if 'APPIMAGE' in os.environ:
|
||||
ver = os.environ.get('TESTIUM_VERSION', '').strip()
|
||||
return (ver if ver else 'unknown') + " (binary release)"
|
||||
|
||||
# PyInstaller frozen exe
|
||||
if getattr(sys, 'frozen', False):
|
||||
file_path = os.path.join(sys._MEIPASS, "VERSION")
|
||||
with open(file_path, 'r') as file:
|
||||
ver = file.read()
|
||||
return (ver + " (binary release)")
|
||||
try:
|
||||
with open(file_path, 'r') as f:
|
||||
ver = f.read().strip()
|
||||
return ver + " (binary release)"
|
||||
except OSError:
|
||||
return "unknown (binary release)"
|
||||
|
||||
# Executed from sources
|
||||
try:
|
||||
if prefs.settings.git_supported:
|
||||
# Source checkout: prefer git revision when available
|
||||
if prefs.settings.git_supported:
|
||||
try:
|
||||
git = import_module("git")
|
||||
path = tm.get_main_dir()
|
||||
try:
|
||||
return repo_rev(path)
|
||||
except git.InvalidGitRepositoryError:
|
||||
pkg_rec = import_module("pkg_resources")
|
||||
try:
|
||||
ret = pkg_rec.get_distribution("testium").version
|
||||
_cached_versions.update({path: ret})
|
||||
return str(ret) + " (wheel release)"
|
||||
except:
|
||||
return "Warning : testium not versioned"
|
||||
else:
|
||||
return "Warning git not supported in your settings, version of testium is unknown."
|
||||
except:
|
||||
return ("Unknown")
|
||||
return repo_rev(tm.get_main_dir())
|
||||
except Exception:
|
||||
# Not a git repo (typical pip install): fall through.
|
||||
pass
|
||||
|
||||
# Pip-installed wheel: use the package metadata baked from VERSION
|
||||
try:
|
||||
from importlib.metadata import version as _pkg_version
|
||||
from importlib.metadata import PackageNotFoundError
|
||||
try:
|
||||
return _pkg_version("testium") + " (wheel release)"
|
||||
except PackageNotFoundError:
|
||||
pass
|
||||
except ImportError:
|
||||
pass
|
||||
|
||||
return "unknown"
|
||||
|
||||
def get_modifications(path : str)-> str:
|
||||
|
||||
|
||||
@@ -41,8 +41,7 @@ end
|
||||
--- INTERNAL: Handle requests from the client
|
||||
function JSONRPC:_handle_request(req)
|
||||
local method = self.methods[req.method]
|
||||
local ok, ret
|
||||
local res, err
|
||||
local ok, ret, err
|
||||
if not method then
|
||||
if req.id then self:_send_error(req.id, string.format("Method '%s' not registered in lua server")) end
|
||||
return
|
||||
@@ -52,15 +51,18 @@ function JSONRPC:_handle_request(req)
|
||||
|
||||
-- Only send response if it's not a Notification (notifications have no ID)
|
||||
if req.id then
|
||||
if ok then
|
||||
res = ret
|
||||
if res == nil then
|
||||
self:_send_error(req.id, tostring(err))
|
||||
else
|
||||
self:_send({ jsonrpc = "2.0", result = { returned_value = res }, id = req.id })
|
||||
end
|
||||
else
|
||||
if not ok then
|
||||
-- pcall trapped a runtime error in the method itself.
|
||||
self:_send_error(req.id, tostring(ret))
|
||||
elseif err ~= nil then
|
||||
-- Method ran but signaled a logical error via its 2nd return.
|
||||
self:_send_error(req.id, tostring(err))
|
||||
else
|
||||
-- Success. A user function returning nothing yields ret==nil;
|
||||
-- encode it as JSON null so "returned_value" stays present.
|
||||
local val = ret
|
||||
if val == nil then val = json.null end
|
||||
self:_send({ jsonrpc = "2.0", result = { returned_value = val }, id = req.id })
|
||||
end
|
||||
end
|
||||
end
|
||||
|
||||
21
src/testium/main_win/file_dialog.py
Normal file
21
src/testium/main_win/file_dialog.py
Normal file
@@ -0,0 +1,21 @@
|
||||
"""Helpers for Qt file/directory dialogs.
|
||||
|
||||
In Flatpak the native QFileDialog goes through the XDG document portal,
|
||||
which returns ``/run/user/UID/doc/.../<file>`` and only exposes the
|
||||
selected file — sibling files (param.yaml, scripts, recent paths in
|
||||
preferences, ...) are unreachable. Forcing Qt's own non-native dialog
|
||||
makes it walk the real filesystem mounted via ``--filesystem=home``
|
||||
and return a regular path.
|
||||
"""
|
||||
|
||||
import os
|
||||
|
||||
from PySide6.QtWidgets import QFileDialog
|
||||
|
||||
|
||||
def options():
|
||||
"""Default ``QFileDialog`` options for the current runtime."""
|
||||
opts = QFileDialog.Options()
|
||||
if os.path.isfile("/.flatpak-info"):
|
||||
opts |= QFileDialog.Option.DontUseNativeDialog
|
||||
return opts
|
||||
@@ -3,6 +3,7 @@ from PySide6.QtWidgets import QDialog, QFileDialog
|
||||
from PySide6.QtGui import QFont
|
||||
|
||||
from main_win.preference_win.preference_core_win import Ui_preferenceWindow
|
||||
from main_win import file_dialog
|
||||
|
||||
import interpreter.utils.settings as prefs
|
||||
|
||||
@@ -193,6 +194,7 @@ class PrefWindow(QDialog):
|
||||
self,
|
||||
caption="Select the default report directory",
|
||||
dir=self.ui.editDefaultReportPath.text(),
|
||||
options=file_dialog.options(),
|
||||
)
|
||||
if path:
|
||||
self.ui.editDefaultReportPath.setText(path)
|
||||
@@ -203,6 +205,7 @@ class PrefWindow(QDialog):
|
||||
self,
|
||||
caption="Select the default log directory",
|
||||
dir=self.ui.editDefaultLogPath.text(),
|
||||
options=file_dialog.options(),
|
||||
)
|
||||
if path:
|
||||
self.ui.editDefaultLogPath.setText(path)
|
||||
@@ -213,6 +216,7 @@ class PrefWindow(QDialog):
|
||||
self,
|
||||
caption="Select the python interpreter",
|
||||
dir=self.ui.editPythonPath.text(),
|
||||
options=file_dialog.options(),
|
||||
)
|
||||
if path:
|
||||
self.ui.editPythonPath.setText(path)
|
||||
@@ -220,7 +224,10 @@ class PrefWindow(QDialog):
|
||||
@Slot()
|
||||
def on_butLuaPath_pressed(self):
|
||||
path, _ = QFileDialog.getOpenFileName(
|
||||
self, caption="Select the lua interpreter", dir=self.ui.editLuaPath.text()
|
||||
self,
|
||||
caption="Select the lua interpreter",
|
||||
dir=self.ui.editLuaPath.text(),
|
||||
options=file_dialog.options(),
|
||||
)
|
||||
if path:
|
||||
self.ui.editLuaPath.setText(path)
|
||||
|
||||
@@ -9,6 +9,7 @@ from PySide6.QtWidgets import QApplication, QFileDialog, QProgressDialog
|
||||
from interpreter.process import TestProcess
|
||||
from interpreter.utils.test_ctrl import TestSetController
|
||||
from main_win.test_controller_service import TestControllerService
|
||||
from main_win import file_dialog
|
||||
import interpreter.utils.settings as prefs
|
||||
from runtime.tum_except import ETUMFileError, ETUMRuntimeError
|
||||
|
||||
@@ -213,7 +214,8 @@ class TestFileManager:
|
||||
if w.testFile is not None:
|
||||
d = os.path.dirname(w.testFile)
|
||||
file_name, _ = QFileDialog.getOpenFileName(
|
||||
w, "Open the test file", d, "testium file (*.tum);;All Files (*)"
|
||||
w, "Open the test file", d,
|
||||
"testium file (*.tum);;All Files (*)", options=file_dialog.options()
|
||||
)
|
||||
if file_name:
|
||||
self.reload(file_name)
|
||||
|
||||
@@ -176,7 +176,7 @@ class TestRunner:
|
||||
w.actionOpenTest.setDisabled(True)
|
||||
w.actionExit.setDisabled(True)
|
||||
icon = QtGui.QIcon()
|
||||
icon.addPixmap(QtGui.QPixmap(icon_prefix() + "/pause.png"), QtGui.QIcon.Normal, QtGui.QIcon.Off)
|
||||
icon.addPixmap(QtGui.QPixmap(icon_prefix() + "/pause2.png"), QtGui.QIcon.Normal, QtGui.QIcon.On)
|
||||
w.actionStart_test.setIcon(icon)
|
||||
w.actionStart_test.setText("Pause test")
|
||||
w.actionPreferences.setDisabled(True)
|
||||
|
||||
@@ -37,6 +37,7 @@ from interpreter.utils.icons import icon_prefix
|
||||
|
||||
from main_win.test_run.outlog import OutLog
|
||||
from main_win.test_run.test_run import ThreadTestStatus
|
||||
from main_win import file_dialog
|
||||
import interpreter.utils.settings as prefs
|
||||
from runtime.stdout_redirect import stdio_redir
|
||||
import api.testium as tm
|
||||
@@ -484,7 +485,8 @@ class MainWindow(QMainWindow, Ui_MainWindow):
|
||||
else:
|
||||
initialPath = None
|
||||
fileName, _ = QFileDialog.getSaveFileName(
|
||||
self, "Path to Log file", initialPath, "Log Files (*.log);;All Files (*)"
|
||||
self, "Path to Log file", initialPath, "Log Files (*.log);;All Files (*)",
|
||||
options=file_dialog.options(),
|
||||
)
|
||||
if fileName:
|
||||
shutil.copy(self.logFileName, fileName)
|
||||
@@ -525,7 +527,8 @@ class MainWindow(QMainWindow, Ui_MainWindow):
|
||||
else:
|
||||
initialPath = None
|
||||
fileName, _ = QFileDialog.getSaveFileName(
|
||||
self, "Path to log file", initialPath, "Log Files (*.log);;All Files (*)"
|
||||
self, "Path to log file", initialPath, "Log Files (*.log);;All Files (*)",
|
||||
options=file_dialog.options(),
|
||||
)
|
||||
if fileName:
|
||||
self.editLogFilePath.setText(fileName)
|
||||
|
||||
@@ -8,14 +8,18 @@ def exception_handler(typ_exc, value, trbk):
|
||||
print(f"Critical failure : '{value}'.")
|
||||
tb = traceback.format_exception(typ_exc, value, trbk)
|
||||
print("".join(tb))
|
||||
print(f" python : {sys.executable}")
|
||||
print(f" sys.path : {sys.path}")
|
||||
|
||||
sys.excepthook = exception_handler
|
||||
|
||||
p = Path(__file__)
|
||||
p = p.parent / ".."
|
||||
p = p.resolve()
|
||||
|
||||
sys.path.append(p)
|
||||
# Make the parent directory of py_func/ (= the testium package dir, which also
|
||||
# contains runtime/, lua_func/, …) the first entry on sys.path so `from py_func
|
||||
# import main` and `from runtime…` resolve regardless of cwd or how this script
|
||||
# was invoked. str() because some importers don't play well with PathLike entries.
|
||||
_pkg_parent = str((Path(__file__).resolve().parent / "..").resolve())
|
||||
if _pkg_parent not in sys.path:
|
||||
sys.path.insert(0, _pkg_parent)
|
||||
|
||||
from py_func import main
|
||||
|
||||
|
||||
@@ -200,6 +200,7 @@ class JsonRpcConnection:
|
||||
|
||||
Raises:
|
||||
TimeoutError: If no response is received within `timeout`.
|
||||
ConnectionAbortedError: If stop() was called while waiting.
|
||||
"""
|
||||
|
||||
req_id = next(self.id_gen)
|
||||
@@ -214,7 +215,12 @@ class JsonRpcConnection:
|
||||
self.pending.pop(req_id, None)
|
||||
raise TimeoutError("Timeout JSON-RPC")
|
||||
|
||||
return self.pending.pop(req_id)["response"]
|
||||
entry = self.pending.pop(req_id)
|
||||
if entry["response"] is None:
|
||||
# Woken by stop() (or by a malformed dispatch) rather than by a
|
||||
# real response — abort the call so callers don't block further.
|
||||
raise ConnectionAbortedError("JSON-RPC client stopped")
|
||||
return entry["response"]
|
||||
|
||||
def print_info(self, msg):
|
||||
if self.dbg_out is not None:
|
||||
@@ -223,6 +229,10 @@ class JsonRpcConnection:
|
||||
def stop(self):
|
||||
if self.running:
|
||||
self.running = False
|
||||
# Wake any in-flight call() so it doesn't sit on its (default 1h)
|
||||
# timeout. The response stays None and call() raises ConnectionAbortedError.
|
||||
for entry in list(self.pending.values()):
|
||||
entry["event"].set()
|
||||
|
||||
def join(self):
|
||||
self.recv_thread.join()
|
||||
|
||||
@@ -1,10 +1,76 @@
|
||||
# Validation
|
||||
|
||||
This directory contains the necessary material to run the testium validation.
|
||||
This directory contains the testium validation suite. A single set of
|
||||
items (`items/`), fixtures and post-processing (`post_execution.py`) is
|
||||
re-used across every packaging channel.
|
||||
|
||||
Here is the documentation on how to configure the validation, run it and check that the
|
||||
results are correct.
|
||||
## Running the suite
|
||||
|
||||
# Tests
|
||||
```sh
|
||||
./test/validation/run.sh # default mode = source
|
||||
./test/validation/run.sh --mode wheel
|
||||
./test/validation/run.sh --mode pyinstaller
|
||||
./test/validation/run.sh --mode flatpak
|
||||
./test/validation/run.sh --mode appimage
|
||||
```
|
||||
|
||||
TBD
|
||||
On Windows (only `source`, `wheel`, `pyinstaller` are supported):
|
||||
|
||||
```bat
|
||||
test\validation\run.bat --mode pyinstaller
|
||||
```
|
||||
|
||||
Pass `clean` as the **first** argument to recreate the validation venv
|
||||
from scratch (useful after a system Python upgrade):
|
||||
|
||||
```sh
|
||||
./test/validation/run.sh clean --mode flatpak
|
||||
```
|
||||
|
||||
Any extra arguments after the mode flag are forwarded to testium.
|
||||
|
||||
## Modes
|
||||
|
||||
| Mode | What it launches | Prerequisite |
|
||||
|---------------|-------------------------------------------------------------|------------------------------------------------------------------|
|
||||
| `source` | `python3 src/testium` via the project's `run.sh` | none — works straight out of the repo |
|
||||
| `wheel` | `python -m testium` inside a dedicated wheel venv | `./build_all.sh` produced `dist/testium-<v>-py3-none-any.whl` |
|
||||
| `pyinstaller` | `dist/testium-<v>` (frozen binary) | `./build_all.sh` produced the PyInstaller binary |
|
||||
| `flatpak` | `flatpak run --command=testium org.testium.Testium` | the Flatpak bundle is installed (`flatpak install --user dist/testium-<v>.flatpak`) |
|
||||
| `appimage` | `dist/Testium-<v>-x86_64.AppImage` | `./build_all.sh` produced the AppImage |
|
||||
|
||||
Each mode writes its results to a distinct report file
|
||||
(`validation-<mode>.sqlite` / `validation-<mode>-<item>.xml`), so you
|
||||
can run several modes in a row without clobbering previous reports.
|
||||
|
||||
## How `python_bin` is pinned
|
||||
|
||||
Every test-execution subprocess (inline `<| ... |>` evaluation,
|
||||
`py_func`, `cycle`, `post_execution`, …) is routed through a dedicated
|
||||
venv at `${TMPDIR:-/tmp}/testium-validation-venv`. The venv is created
|
||||
with `--system-site-packages` so existing system packages stay visible,
|
||||
then `junit-xml` is pip-installed for `post_execution.py`.
|
||||
|
||||
This is a **host** venv. In every mode (including Flatpak) the
|
||||
test-execution subprocesses end up running on the host — directly for
|
||||
source/wheel/pyinstaller/appimage, and via `flatpak-spawn --host` for
|
||||
Flatpak — so the same venv works across modes. The wheel mode
|
||||
additionally creates a separate `testium-wheel-venv-<v>` to hold the
|
||||
installed wheel; that one is only used to launch testium itself.
|
||||
|
||||
## What is checked
|
||||
|
||||
The `venv` item under `items/venv/` asserts that the validation venv is
|
||||
actually being used:
|
||||
|
||||
* `python_bin` is set in the global dict.
|
||||
* The eval subprocess (used for `<| ... |>` expressions) has
|
||||
`sys.executable == python_bin`, `sys.prefix == dirname(dirname(python_bin))`,
|
||||
and `sys.prefix != sys.base_prefix` (i.e. is actually inside a venv).
|
||||
* A `py_func` subprocess passes the same three checks.
|
||||
|
||||
These checks use `abspath`/`normpath` rather than `realpath` on
|
||||
purpose: the venv's `bin/python3` is a symlink to the host interpreter,
|
||||
so `realpath` would map both venv and non-venv interpreters to the same
|
||||
target. `sys.prefix != sys.base_prefix` is the venv-specific marker
|
||||
that distinguishes the two cases.
|
||||
|
||||
@@ -49,4 +49,12 @@ function module.test_delgd()
|
||||
return 0
|
||||
end
|
||||
|
||||
function module.return_nothing()
|
||||
-- Returns no value: ret is nil but no error.
|
||||
end
|
||||
|
||||
function module.return_explicit_nil()
|
||||
return nil
|
||||
end
|
||||
|
||||
return module
|
||||
@@ -186,6 +186,18 @@
|
||||
file: $(test_path)$(psep)lua_func.lua
|
||||
func_name: test_delgd
|
||||
|
||||
- lua_func:
|
||||
name: function returning nothing should succeed
|
||||
key: $(test)_PASS
|
||||
file: $(test_path)$(psep)lua_func.lua
|
||||
func_name: return_nothing
|
||||
|
||||
- lua_func:
|
||||
name: function returning explicit nil should succeed
|
||||
key: $(test)_PASS
|
||||
file: $(test_path)$(psep)lua_func.lua
|
||||
func_name: return_explicit_nil
|
||||
|
||||
- group:
|
||||
name: context_id tests
|
||||
steps:
|
||||
|
||||
@@ -54,3 +54,10 @@ def test_delgd():
|
||||
tm.delgd("_py_delgd_test")
|
||||
assert tm.gd("_py_delgd_test", None) is None
|
||||
return 0
|
||||
|
||||
def return_nothing():
|
||||
# Falls off the end: implicit None return, no error.
|
||||
pass
|
||||
|
||||
def return_explicit_none():
|
||||
return None
|
||||
|
||||
@@ -196,6 +196,18 @@
|
||||
file: $(test_path)$(psep)py_func.py
|
||||
func_name: test_delgd
|
||||
|
||||
- py_func:
|
||||
name: function returning nothing should succeed
|
||||
key: $(test)_PASS
|
||||
file: $(test_path)$(psep)py_func.py
|
||||
func_name: return_nothing
|
||||
|
||||
- py_func:
|
||||
name: function returning explicit None should succeed
|
||||
key: $(test)_PASS
|
||||
file: $(test_path)$(psep)py_func.py
|
||||
func_name: return_explicit_none
|
||||
|
||||
- group:
|
||||
name: context_id tests
|
||||
steps:
|
||||
|
||||
1
test/validation/items/venv/param.yaml
Normal file
1
test/validation/items/venv/param.yaml
Normal file
@@ -0,0 +1 @@
|
||||
no_param: Null
|
||||
53
test/validation/items/venv/test.tum
Normal file
53
test/validation/items/venv/test.tum
Normal file
@@ -0,0 +1,53 @@
|
||||
# venv test: assert that the dedicated validation venv is the python
|
||||
# being used for every test-execution subprocess (eval_proc / py_func /
|
||||
# cycle / ...). The venv path is pinned by ``-d python_bin=...`` in
|
||||
# test/validation/run.sh (or run.bat).
|
||||
#
|
||||
# We use ``abspath``/``normpath`` rather than ``realpath`` on purpose:
|
||||
# the venv's ``bin/python3`` is a symlink to the host python, so
|
||||
# realpath would map every venv interpreter to the same system path and
|
||||
# the comparison would silently pass even *without* a venv.
|
||||
# ``sys.prefix != sys.base_prefix`` is the venv-specific marker that
|
||||
# catches that case.
|
||||
|
||||
- check:
|
||||
name: python_bin is set in the global dict
|
||||
key: $(test)_PASS
|
||||
values:
|
||||
- <| bool(r"$(python_bin)") |>
|
||||
|
||||
- check:
|
||||
name: eval_proc subprocess runs in the validation venv
|
||||
key: $(test)_PASS
|
||||
values:
|
||||
- <| os.path.normpath(os.path.abspath(sys.executable)) == os.path.normpath(os.path.abspath(r"$(python_bin)")) |>
|
||||
|
||||
- check:
|
||||
name: eval_proc sys.prefix matches python_bin venv root
|
||||
key: $(test)_PASS
|
||||
values:
|
||||
- <| os.path.normpath(os.path.abspath(sys.prefix)) == os.path.dirname(os.path.dirname(os.path.normpath(os.path.abspath(r"$(python_bin)")))) |>
|
||||
|
||||
- check:
|
||||
name: eval_proc is actually inside a venv (sys.prefix != sys.base_prefix)
|
||||
key: $(test)_PASS
|
||||
values:
|
||||
- <| os.path.normpath(os.path.abspath(sys.prefix)) != os.path.normpath(os.path.abspath(sys.base_prefix)) |>
|
||||
|
||||
- py_func:
|
||||
name: py_func subprocess runs in the validation venv
|
||||
key: $(test)_PASS
|
||||
file: $(test_path)$(psep)verify_venv.py
|
||||
func_name: check_sys_executable
|
||||
|
||||
- py_func:
|
||||
name: py_func sys.prefix matches python_bin venv root
|
||||
key: $(test)_PASS
|
||||
file: $(test_path)$(psep)verify_venv.py
|
||||
func_name: check_sys_prefix_in_venv
|
||||
|
||||
- py_func:
|
||||
name: py_func is actually inside a venv
|
||||
key: $(test)_PASS
|
||||
file: $(test_path)$(psep)verify_venv.py
|
||||
func_name: check_is_venv
|
||||
62
test/validation/items/venv/verify_venv.py
Normal file
62
test/validation/items/venv/verify_venv.py
Normal file
@@ -0,0 +1,62 @@
|
||||
import os
|
||||
import sys
|
||||
|
||||
import py_func.tm as tm
|
||||
|
||||
|
||||
def _norm(p):
|
||||
# normpath + normcase, without resolving symlinks. realpath() would
|
||||
# follow the venv's ``python3`` symlink to ``/usr/bin/python3.X`` and
|
||||
# defeat the comparison.
|
||||
return os.path.normcase(os.path.normpath(os.path.abspath(p)))
|
||||
|
||||
|
||||
def _venv_dir():
|
||||
# python_bin is at ``<venv>/(bin|Scripts)/python*`` so the venv root
|
||||
# is two levels above the executable.
|
||||
exe = tm.gd("python_bin", "")
|
||||
if not exe:
|
||||
return ""
|
||||
return os.path.dirname(os.path.dirname(_norm(exe)))
|
||||
|
||||
|
||||
def check_sys_executable():
|
||||
"""py_func subprocess: sys.executable must match the configured python_bin."""
|
||||
expected = _norm(tm.gd("python_bin", ""))
|
||||
actual = _norm(sys.executable)
|
||||
if expected and actual == expected:
|
||||
return True
|
||||
return (
|
||||
-1,
|
||||
f"sys.executable={actual!r} differs from python_bin={expected!r}",
|
||||
)
|
||||
|
||||
|
||||
def check_sys_prefix_in_venv():
|
||||
"""py_func subprocess: sys.prefix must match the venv root derived
|
||||
from python_bin (two levels up from the executable)."""
|
||||
venv = _venv_dir()
|
||||
if not venv:
|
||||
return (-1, "python_bin is not set in the global dict")
|
||||
actual = _norm(sys.prefix)
|
||||
if actual == venv:
|
||||
return True
|
||||
return (
|
||||
-1,
|
||||
f"sys.prefix={actual!r} is not the validation venv {venv!r}",
|
||||
)
|
||||
|
||||
|
||||
def check_is_venv():
|
||||
"""py_func subprocess: confirm we are inside a venv, i.e. sys.prefix
|
||||
differs from sys.base_prefix. This catches the case where python_bin
|
||||
happens to be a system interpreter and the path-based check would
|
||||
pass trivially."""
|
||||
actual = _norm(sys.prefix)
|
||||
base = _norm(sys.base_prefix)
|
||||
if actual != base:
|
||||
return True
|
||||
return (
|
||||
-1,
|
||||
f"sys.prefix == sys.base_prefix == {actual!r}: not running in a venv",
|
||||
)
|
||||
131
test/validation/run.bat
Normal file
131
test/validation/run.bat
Normal file
@@ -0,0 +1,131 @@
|
||||
@echo off
|
||||
SETLOCAL EnableExtensions EnableDelayedExpansion
|
||||
|
||||
REM Runs the testium validation suite against any installable channel of
|
||||
REM testium on Windows (source, wheel, pyinstaller).
|
||||
REM
|
||||
REM Usage:
|
||||
REM test\validation\run.bat [clean] [--mode MODE] [extra testium args]
|
||||
REM
|
||||
REM clean remove the validation venv before recreating it
|
||||
REM (must be the first argument; useful after a Python upgrade)
|
||||
REM
|
||||
REM --mode MODE which testium build to validate. One of:
|
||||
REM source (default) project's run.bat (src\testium)
|
||||
REM wheel dist\testium-<v>-py3-none-any.whl
|
||||
REM pyinstaller dist\testium-<v>.exe (or dist\testium-<v>)
|
||||
REM
|
||||
REM Every test-execution subprocess runs in a dedicated host venv under
|
||||
REM %TEMP%\testium-validation-venv (created with --system-site-packages,
|
||||
REM then junit-xml is pip-installed for post_execution.py).
|
||||
REM
|
||||
REM The report file is suffixed with the mode so consecutive runs in
|
||||
REM different modes don't overwrite each other.
|
||||
|
||||
SET "SCRIPT_DIR=%~dp0"
|
||||
IF "%SCRIPT_DIR:~-1%"=="\" SET "SCRIPT_DIR=%SCRIPT_DIR:~0,-1%"
|
||||
SET "PROJECT_DIR=%SCRIPT_DIR%\..\.."
|
||||
SET /P VERSION=<"%PROJECT_DIR%\src\VERSION"
|
||||
|
||||
REM ---------- arg parsing ----------------------------------------------------
|
||||
|
||||
SET "MODE=source"
|
||||
SET "CLEAN=0"
|
||||
IF /I "%~1"=="clean" (
|
||||
SET "CLEAN=1"
|
||||
SHIFT
|
||||
)
|
||||
|
||||
SET "EXTRA="
|
||||
:PARSE_ARGS
|
||||
IF "%~1"=="" GOTO ARGS_DONE
|
||||
IF /I "%~1"=="--mode" (
|
||||
SET "MODE=%~2"
|
||||
SHIFT
|
||||
SHIFT
|
||||
GOTO PARSE_ARGS
|
||||
)
|
||||
SET "EXTRA=!EXTRA! "%~1""
|
||||
SHIFT
|
||||
GOTO PARSE_ARGS
|
||||
:ARGS_DONE
|
||||
|
||||
REM ---------- locate host python ---------------------------------------------
|
||||
|
||||
SET "PYTHON_EXE="
|
||||
py --version >nul 2>&1
|
||||
IF %ERRORLEVEL% EQU 0 (
|
||||
SET "PYTHON_EXE=py"
|
||||
GOTO PYTHON_FOUND
|
||||
)
|
||||
python --version >nul 2>&1
|
||||
IF %ERRORLEVEL% EQU 0 (
|
||||
SET "PYTHON_EXE=python"
|
||||
GOTO PYTHON_FOUND
|
||||
)
|
||||
echo ERROR: Python could not be found on this system.
|
||||
exit /b 1
|
||||
:PYTHON_FOUND
|
||||
|
||||
REM ---------- validation venv -------------------------------------------------
|
||||
|
||||
SET "VENV_DIR=%TEMP%\testium-validation-venv"
|
||||
IF "%CLEAN%"=="1" IF EXIST "%VENV_DIR%" rmdir /s /q "%VENV_DIR%"
|
||||
|
||||
IF NOT EXIST "%VENV_DIR%" (
|
||||
echo Creating validation venv at %VENV_DIR%
|
||||
%PYTHON_EXE% -m venv --system-site-packages "%VENV_DIR%"
|
||||
IF !ERRORLEVEL! NEQ 0 (
|
||||
echo ERROR while creating the validation venv.
|
||||
exit /b 1
|
||||
)
|
||||
call "%VENV_DIR%\Scripts\pip" install --quiet --upgrade pip
|
||||
call "%VENV_DIR%\Scripts\pip" install --quiet junit-xml
|
||||
)
|
||||
SET "VENV_PYTHON=%VENV_DIR%\Scripts\python.exe"
|
||||
|
||||
REM ---------- shared "tail" forwarded to every launcher -----------------------
|
||||
REM Reports are stamped with the mode so successive runs don't clobber each other.
|
||||
|
||||
SET "TAIL=-b -d "python_bin=%VENV_PYTHON%" -d "validation_report_file=validation-%MODE%" -- "%SCRIPT_DIR%\main.tum"%EXTRA%"
|
||||
|
||||
REM ---------- per-mode launcher ----------------------------------------------
|
||||
|
||||
echo -- validation mode: %MODE%
|
||||
|
||||
IF /I "%MODE%"=="source" GOTO MODE_SOURCE
|
||||
IF /I "%MODE%"=="wheel" GOTO MODE_WHEEL
|
||||
IF /I "%MODE%"=="pyinstaller" GOTO MODE_PYI
|
||||
echo ERROR: unknown --mode '%MODE%'. Expected: source ^| wheel ^| pyinstaller.
|
||||
exit /b 1
|
||||
|
||||
:MODE_SOURCE
|
||||
call "%PROJECT_DIR%\run.bat" %TAIL%
|
||||
exit /b %ERRORLEVEL%
|
||||
|
||||
:MODE_WHEEL
|
||||
SET "WHEEL=%PROJECT_DIR%\dist\testium-%VERSION%-py3-none-any.whl"
|
||||
IF NOT EXIST "%WHEEL%" (
|
||||
echo ERROR: wheel not found at %WHEEL% -- run build_all.sh first.
|
||||
exit /b 1
|
||||
)
|
||||
SET "WHEEL_VENV=%TEMP%\testium-wheel-venv-%VERSION%"
|
||||
IF "%CLEAN%"=="1" IF EXIST "%WHEEL_VENV%" rmdir /s /q "%WHEEL_VENV%"
|
||||
IF NOT EXIST "%WHEEL_VENV%" (
|
||||
echo Creating wheel venv at %WHEEL_VENV%
|
||||
%PYTHON_EXE% -m venv --system-site-packages "%WHEEL_VENV%"
|
||||
call "%WHEEL_VENV%\Scripts\pip" install --quiet --upgrade pip
|
||||
call "%WHEEL_VENV%\Scripts\pip" install --quiet "%WHEEL%"
|
||||
)
|
||||
"%WHEEL_VENV%\Scripts\python.exe" -m testium %TAIL%
|
||||
exit /b %ERRORLEVEL%
|
||||
|
||||
:MODE_PYI
|
||||
SET "PYI_BIN=%PROJECT_DIR%\dist\testium-%VERSION%.exe"
|
||||
IF NOT EXIST "%PYI_BIN%" SET "PYI_BIN=%PROJECT_DIR%\dist\testium-%VERSION%"
|
||||
IF NOT EXIST "%PYI_BIN%" (
|
||||
echo ERROR: PyInstaller binary not found in %PROJECT_DIR%\dist -- run build_all.sh first.
|
||||
exit /b 1
|
||||
)
|
||||
"%PYI_BIN%" %TAIL%
|
||||
exit /b %ERRORLEVEL%
|
||||
143
test/validation/run.sh
Executable file
143
test/validation/run.sh
Executable file
@@ -0,0 +1,143 @@
|
||||
#!/bin/bash
|
||||
# Runs the testium validation suite against any installable channel of
|
||||
# testium (source, wheel, pyinstaller, flatpak, appimage).
|
||||
#
|
||||
# Usage:
|
||||
# ./test/validation/run.sh [clean] [--mode MODE] [extra testium args]
|
||||
#
|
||||
# clean remove the validation venv before recreating it
|
||||
# (must be the first argument; useful after a Python upgrade)
|
||||
#
|
||||
# --mode MODE which testium build to validate. One of:
|
||||
# source (default) src/testium via project run.sh
|
||||
# wheel dist/testium-<v>-py3-none-any.whl
|
||||
# pyinstaller dist/testium-<v>
|
||||
# flatpak installed org.testium.Testium
|
||||
# appimage dist/Testium-<v>-*.AppImage
|
||||
#
|
||||
# Every test-execution subprocess (inline <| ... |>, py_func, cycle,
|
||||
# post_execution, ...) runs in a dedicated host venv under
|
||||
# /tmp/testium-validation-venv. That venv is shared across modes —
|
||||
# even Flatpak reaches it via flatpak-spawn --host. The validation venv
|
||||
# is created with --system-site-packages so existing system packages
|
||||
# (PySide6, lxml, ...) stay visible, then junit-xml is pip-installed
|
||||
# for post_execution.py.
|
||||
#
|
||||
# The report file is suffixed with the mode (e.g. validation-flatpak.sqlite)
|
||||
# so consecutive runs in different modes don't overwrite each other.
|
||||
|
||||
set -e
|
||||
|
||||
SCRIPT_PATH="$(readlink -f "$0")"
|
||||
SCRIPT_DIR="$(realpath "$(dirname "$SCRIPT_PATH")")"
|
||||
PROJECT_DIR="$(realpath "$SCRIPT_DIR/../..")"
|
||||
VERSION="$(cat "$PROJECT_DIR/src/VERSION")"
|
||||
|
||||
# ---------- arg parsing -------------------------------------------------------
|
||||
|
||||
MODE=source
|
||||
|
||||
if [ "${1:-}" = "clean" ]; then
|
||||
CLEAN=1
|
||||
shift
|
||||
else
|
||||
CLEAN=0
|
||||
fi
|
||||
|
||||
EXTRA=()
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
--mode)
|
||||
MODE="$2"
|
||||
shift 2
|
||||
;;
|
||||
--mode=*)
|
||||
MODE="${1#--mode=}"
|
||||
shift
|
||||
;;
|
||||
*)
|
||||
EXTRA+=("$1")
|
||||
shift
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
# ---------- validation venv ---------------------------------------------------
|
||||
|
||||
VENV_DIR="${TMPDIR:-/tmp}/testium-validation-venv"
|
||||
if [ "$CLEAN" -eq 1 ]; then
|
||||
rm -rf "$VENV_DIR"
|
||||
fi
|
||||
|
||||
if [ ! -d "$VENV_DIR" ]; then
|
||||
echo "Creating validation venv at $VENV_DIR"
|
||||
python3 -m venv --system-site-packages "$VENV_DIR"
|
||||
"$VENV_DIR/bin/pip" install --quiet --upgrade pip
|
||||
"$VENV_DIR/bin/pip" install --quiet junit-xml
|
||||
fi
|
||||
VENV_PYTHON="$VENV_DIR/bin/python3"
|
||||
|
||||
# ---------- per-mode launcher -------------------------------------------------
|
||||
|
||||
case "$MODE" in
|
||||
source)
|
||||
CMD=("$PROJECT_DIR/run.sh")
|
||||
;;
|
||||
wheel)
|
||||
WHEEL="$PROJECT_DIR/dist/testium-${VERSION}-py3-none-any.whl"
|
||||
if [ ! -f "$WHEEL" ]; then
|
||||
echo "ERROR: wheel not found at $WHEEL — run ./build_all.sh first." >&2
|
||||
exit 1
|
||||
fi
|
||||
WHEEL_VENV="${TMPDIR:-/tmp}/testium-wheel-venv-${VERSION}"
|
||||
if [ "$CLEAN" -eq 1 ]; then
|
||||
rm -rf "$WHEEL_VENV"
|
||||
fi
|
||||
if [ ! -d "$WHEEL_VENV" ]; then
|
||||
echo "Creating wheel venv at $WHEEL_VENV"
|
||||
python3 -m venv --system-site-packages "$WHEEL_VENV"
|
||||
"$WHEEL_VENV/bin/pip" install --quiet --upgrade pip
|
||||
"$WHEEL_VENV/bin/pip" install --quiet "$WHEEL"
|
||||
fi
|
||||
CMD=("$WHEEL_VENV/bin/python" -m testium)
|
||||
;;
|
||||
pyinstaller)
|
||||
PYI_BIN="$PROJECT_DIR/dist/testium-${VERSION}"
|
||||
if [ ! -x "$PYI_BIN" ]; then
|
||||
echo "ERROR: PyInstaller binary not found at $PYI_BIN — run ./build_all.sh first." >&2
|
||||
exit 1
|
||||
fi
|
||||
CMD=("$PYI_BIN")
|
||||
;;
|
||||
flatpak)
|
||||
if ! flatpak info --user org.testium.Testium &>/dev/null \
|
||||
&& ! flatpak info --system org.testium.Testium &>/dev/null; then
|
||||
echo "ERROR: org.testium.Testium is not installed." >&2
|
||||
echo " flatpak install --user $PROJECT_DIR/dist/testium-${VERSION}.flatpak" >&2
|
||||
exit 1
|
||||
fi
|
||||
CMD=(flatpak run --command=testium org.testium.Testium)
|
||||
;;
|
||||
appimage)
|
||||
APPIMAGE=$(ls -1t "$PROJECT_DIR/dist"/Testium-"${VERSION}"-*.AppImage 2>/dev/null | head -1)
|
||||
if [ -z "$APPIMAGE" ] || [ ! -x "$APPIMAGE" ]; then
|
||||
echo "ERROR: no AppImage for version $VERSION under $PROJECT_DIR/dist — run ./build_all.sh first." >&2
|
||||
exit 1
|
||||
fi
|
||||
CMD=("$APPIMAGE")
|
||||
;;
|
||||
*)
|
||||
echo "ERROR: unknown --mode '$MODE'. Expected: source|wheel|pyinstaller|flatpak|appimage." >&2
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
|
||||
# ---------- launch ------------------------------------------------------------
|
||||
|
||||
echo "-- validation mode: $MODE"
|
||||
echo "-- launch: ${CMD[*]}"
|
||||
|
||||
exec "${CMD[@]}" -b \
|
||||
-d "python_bin=$VENV_PYTHON" \
|
||||
-d "validation_report_file=validation-$MODE" \
|
||||
-- "$SCRIPT_DIR/main.tum" "${EXTRA[@]}"
|
||||
Reference in New Issue
Block a user