82 Commits

Author SHA1 Message Date
354c5e12e8 bump to 0.2: declarative test item parameters
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-29 13:16:45 +02:00
b1a7dac0f3 item params: migrate every structured item to declarative PARAMS
Migrates the remaining test items to the ParamSet/Param declaration
introduced in d0721af:
  - dialogs: image, question, value, choices, tested_references
  - actions: check, run, report
  - console: parent + open/read_until actions
  - py_func / lua_func
  - containers: group, parallel + parallel_branch, unittest
  - complex: cycle (sub-block exit_condition documented in
    EXIT_CONDITION_PARAMS), git
  - runtime_plot: parent + open/close/periodic/last_value actions
  - json_rpc: parent + query/receive actions

Items intentionally without PARAMS (and therefore not validated) are
those whose body is the unstructured user value: console write/writeln,
plot add/export, and the json_rpc/console open & close actions. Same
for the internally-instantiated TestItemUnittestElement which passes
dict_item=None.

Behavior on valid .tum files is unchanged (validation suite source
mode: SUCCESS). Typos on declared params now surface as warnings
listing the accepted names; missing required params surface as load-
time errors with file context.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-29 11:14:42 +02:00
d0721af719 item params: declarative descriptor foundation + 4 pilot items
Adds utils/param_decl.py with Param/ParamSet/kind descriptors. TestItem
declares COMMON_PARAMS (name, doc, condition, key, skipped, …) and a
new _validate_declared_params() method that warns on unknown keys and
errors on missing required ones — opt-in per subclass (skipped while
PARAMS is None to keep the migration incremental).

Migrates sleep, let, msg_dialog, note_dialog as pilots. Behavior is
unchanged for any well-formed .tum; typos like 'timeoot' on a sleep
item now produce a clear WARN listing the accepted parameters.

The descriptor intentionally carries no Python type information —
parameter values that are $(…) / <|…|> expressions only acquire their
effective type after expansion, so a static type would be misleading.
Per-param post-expansion validators stay opt-in via validate=lambda.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-29 10:45:57 +02:00
d4889c2a2e flatpak: run host interpreters via flatpak-spawn; validation --mode flag
py_func, lua_func and the run item now reach host binaries through
`flatpak-spawn --host` instead of trying to load them under the
sandbox runtime (which fails with a glibc ABI mismatch). Adds
`--talk-name=org.freedesktop.Flatpak` to the manifest, stages the
/app/lib/testium tree under /tmp so the host can read it, and drops
the dead `_FLATPAK_HOST_DIRS` / lib-injection code paths that the
new approach makes obsolete.

Validation suite gains a `--mode source|wheel|pyinstaller|flatpak|
appimage` flag so the same item set can run against every packaging
channel; per-mode report file names avoid clobbering.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-19 18:42:32 +02:00
a260e2a56c build_all: incremental build + per-step toolchain install
Skip steps whose dist/ artifact already exists; add --clean/-c to
force a full rebuild. Install sphinx/linuxdoc, build, pyinstaller
in their respective steps instead of upfront. Auto-add flathub
remote and install missing Flatpak SDK/runtime deps before step 4.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-19 13:51:22 +02:00
dd584c9064 gui: bypass XDG portal for all file/dir dialogs in Flatpak
The v0.1.2 fix that forced Qt's non-native dialog for the "open test"
dialog only covered one call site. The same XDG-portal-vs-sibling-files
problem applies to every other QFileDialog in the GUI (save report,
log file path, default report/log dirs in preferences, python/lua
interpreter pickers).

Extracted a single ``file_dialog.options()`` helper in main_win/ and
threaded it through every getOpenFileName / getSaveFileName /
getExistingDirectory call in main_win/. Outside Flatpak the helper
returns an empty Options(), so the native dialog stays in use on
KDE / GNOME / Windows / macOS.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-19 08:32:14 +02:00
4d8cafb5a0 validation: dedicated venv + fix python_bin override timing
eval_proc was started before -d/GUI defines reached gd, so
``-d python_bin=...`` and the GUI ``python_bin`` preference were
silently ignored by the very subprocess that runs ``<| ... |>`` evals
(and only took effect for later items once the discovery cache had
already been seeded with the system interpreter). apply_overrides() is
now applied before eval_process_init(), and bins._resolve()'s cache is
keyed by (name, override) so a later param.yaml change re-resolves on
the next lookup.

The validation suite now ships a wrapper (run.sh / run.bat) that
creates a dedicated venv in the system temp dir and pins it via
``-d python_bin=...``. A new ``venv`` item asserts the override took
effect for both eval_proc and py_func paths, with a
``sys.prefix != sys.base_prefix`` marker to catch the case where the
override happens to be a system interpreter (path-equality alone would
miss it, the venv's ``bin/python3`` being a symlink to the host).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-19 08:19:57 +02:00
6f832cd67b validation: cover nil/None return from lua_func/py_func
Two new steps per language: function returning nothing and function
returning explicit nil/None. Both tagged $(test)_PASS — they would
have failed before the lua nil fix (Lua side reported nil result as
error). Python side already worked but is covered for parity.
2026-05-17 18:13:03 +02:00
ff46886865 lua_func: nil return is not an error
_handle_request was using the 1st pcall return as the error
discriminator, so any Lua function returning nothing (e.g. long_wait
in the example) was reported as failed. Discriminate on the 2nd
return (err) instead, and encode nil result as cjson.null so the
returned_value field stays present in the JSON-RPC response.
2026-05-17 18:04:51 +02:00
50d183d191 removed lua param, useless. 2026-05-17 10:43:25 +02:00
2177715641 examples: long_wait py_func/lua_func to exercise Stop
Two extra steps in example_simple.tum that sleep for 10s, used to
verify that pressing Stop interrupts engaged blocking steps.
2026-05-17 10:42:49 +02:00
a728f561be Make Stop interrupt blocking steps promptly
console.read_until polls a should_stop callback in 0.2s chunks across
all protocols. py_func/lua_func override stop() to tear down the worker
and wake the parent RPC wait. json_rpc adapters honor should_stop too.
Engaged leaf steps now report FAILURE on stop (sleep no-dialog was
silently SUCCESS).
2026-05-17 10:42:40 +02:00
116e528a7d Simplify the Start Stop Pause process (v-and-v/testium#20) 2026-05-16 13:36:18 +02:00
cc744e17a1 Adding ensurepip verification for the build environnement (required by venv) 2026-05-16 13:29:37 +02:00
ab39b49558 now the release note and the manual are copied into dist with build_all 2026-05-13 21:24:35 +02:00
95275c4418 Merge branch 'main' of ssh://git.beafrancois.fr:8328/v-and-v/testium 2026-05-13 14:09:41 +02:00
0d614c2921 release: 0.1.2 2026-05-13 14:05:47 +02:00
9466b091dd docs: rebuild manual PDF 2026-05-13 14:05:47 +02:00
511288bd03 build_all.sh: build wheel + pyinstaller + flatpak + appimage in one go
Collects all four artifacts under <repo>/dist/ (PyInstaller and Flatpak
renamed to testium-<version>(.suff); wheel and AppImage keep PEP 427 /
appimage-builder original names). Re-uses scripts/build_env.sh and
set_env.sh, same venv as run.sh. AppImage build.sh now picks the actual
output file dynamically instead of a hardcoded lowercase name.
2026-05-13 14:03:20 +02:00
51b144f60c Flatpak: bypass XDG portal for .tum open dialog
Native file dialog routes through the XDG document portal, which exposes
only the selected file at /run/user/UID/doc/... — siblings (param.yaml,
.py) are unreachable. Force Qt's non-native dialog in Flatpak so it walks
the real filesystem via --filesystem=home and returns a usable path.
2026-05-13 12:49:46 +02:00
dee8d4a682 generic design elements 2026-05-10 17:41:43 +02:00
e726d47547 generic design elements 2026-05-10 17:40:52 +02:00
5fd50e1c85 release_note: add 0.1.1 entry 2026-05-07 10:09:44 +02:00
51939a566a chore: add src/LICENSE, sync VERSION to 0.1.1 2026-05-07 10:06:17 +02:00
26fccda6bf docs(CLAUDE.md): rewrite Packaging section, document run launcher
Adds Flatpak/AppImage to the channels table, documents the host-only
contract and bins.py mechanics, explains the runtime-aware run launcher.
2026-05-07 10:06:06 +02:00
405fb82fca AppImage packaging: containerized build, host-only py_func/lua_func
build.sh runs appimage-builder in a Debian Bookworm container (Podman or
Docker) so it works on Arch / non-Debian hosts. Uses single src/requirements.txt;
TESTIUM_VERSION exported in runtime.env.
2026-05-07 10:05:58 +02:00
6064d96138 Flatpak packaging: desktop entry, MIME, distributable bundle
org.testium.Testium.yaml uses host Python/Lua only (no bundled interpreter).
build.sh exports a .flatpak bundle. README documents the install procedure.
2026-05-07 10:05:44 +02:00
0658540cc2 run item: runtime-aware launcher, drop testium_path/python_bin params
_testium_launch_cmd() returns the right entry point per mode (AppImage,
Flatpak, PyInstaller, source/wheel). Fixes PermissionError on read-only
__main__.py inside the AppImage squashfs mount.
2026-05-07 10:05:09 +02:00
7bf946dabe py_func/__main__: robust sys.path + diagnostic on import failure
Insert str() parent dir at sys.path[0] (was appending a Path object);
exception handler prints sys.executable and sys.path.
2026-05-07 10:05:00 +02:00
f52d7bbe53 runtime_plot: bump last_values timeout 1s -> 5s, narrow except
Dispatch chain (queue poll + Qt signal/slot + main thread) can exceed 1s on
loaded machines; bare except masked everything other than queue.Empty.
2026-05-07 10:04:44 +02:00
c83ebccb55 version: read TESTIUM_VERSION env in Flatpak/AppImage
Both bundles export it from their launcher; previous fragile __file__-relative
VERSION lookup was reporting 'unknown'.
2026-05-07 10:04:08 +02:00
f17ef8a3a1 dialog_env: pick wayland/xcb from $DISPLAY/$WAYLAND_DISPLAY
Was forcing xcb unconditionally, which hung dialogs on pure-Wayland sessions.
2026-05-07 10:03:54 +02:00
ddb18abc21 bins.py: host-only Python/Lua in sandboxed bundles
_which() probes host dirs only in Flatpak (/run/host/usr/bin) and AppImage
(/usr/bin); apply_host_libs prepares env for host subprocesses (prepend host
libs in Flatpak, strip $APPDIR pollution + PYTHONHOME in AppImage); user
override resolved via _which() for bare names.
2026-05-07 10:03:30 +02:00
358ade8c98 Inc version 2026-05-05 09:21:43 +02:00
46bdb44cfb Route py_func/lua_func subprocess stdio into the parent log
stdout/stderr of the subprocesses were going to DEVNULL — early-startup
errors (lua require failures, exceptions before stdio_redir kicks in)
were lost.

New helper proc_drain.drain_to_log spawns a daemon thread per pipe that
print()s each line through stdio_redir, so it reaches the log + live
output. Used by py_process and lua_process with [py_func]/[lua_func]
prefixes.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-05 09:20:53 +02:00
41519c97cb Fix testium --version reporting "unknown" when installed from a wheel
get_testium_version() used pkg_resources (deprecated, slow to import)
and a narrow catch on git.InvalidGitRepositoryError; any other git
exception fell through to the outer except and returned "unknown".

- Use importlib.metadata.version("testium") to read the wheel
  version that setuptools bakes from src/VERSION at build time. Works
  out of any source checkout — pip-installed copies report
  "<x.y> (wheel release)" instead of "unknown".
- Source-checkout path: tried first when prefs.git_supported, broadly
  catches Exception so a missing repo / detached worktree / etc. no
  longer hides the wheel-metadata fallback.
- PyInstaller path: graceful "unknown (binary release)" if the bundled
  VERSION file is unreadable, instead of an unhandled exception.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-05 09:19:22 +02:00
b9475c6e9b docs: refocus README on users, add quick_start + tutorial, fill CONTRIBUTING
- README.md: pruned developer-oriented sections (Sphinx setup, Qt
  Creator workflow, VSCode debugging, release procedure, AppImage
  Wayland note) and replaced them with a user-facing layout: pre-built
  releases pointer, quick start, manual install, troubleshooting,
  licence.
- CONTRIBUTING.md: absorbed the developer content (debugging in VSCode,
  Qt GUI regen, Sphinx build, validation suite — batch + GUI variants,
  cross-distrib check, release procedure).
- doc/quick_start.md: 5-minute path from install to a passing test,
  in batch mode and in the GUI.
- doc/tutorial.md: guided walk-through against a small calc.py
  module — check, py_func, expected_result, $(...) expansion, group,
  let, condition, report (with the mkdir reminder), context_id.
- CLAUDE.md: subprocess API contract, bins.py, report-exporter
  plugin section, packaging matrix (wheel / PyInstaller / Flatpak /
  .deb work-in-progress), refreshed recent-fixes list. README/CLAUDE
  validation command no longer carries the spurious "-l" flag (which
  is GUI-only and a no-op in batch).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-05 09:18:59 +02:00
d3c5bd01e5 lua and python bin detection rationalized: bins.py module created.
Added some api accessible from python and lua sub_processes. Now the tests only access to py_func.tm instead of direct api.testium module access.

Corrected some f"xxx" to allow working with old python (bookworm).

Changed param.yaml of the test to allow lua to work in all situations.

Various other small fixes for frozen app, wheel.

Tested in all situations, and OK. Ready for tag !

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-03 10:16:56 +02:00
077e1a97c1 Update PyInstaller spec for the new package layout
- Drop the now-obsolete src/lib and src/py_func data entries (those
  paths no longer exist)
- Add src/testium/py_func and src/testium/runtime as bundle-root data
  dirs: the py_func subprocess is launched with the *host* Python
  (not the frozen interpreter), so it needs the source files on disk
  at cwd=subproc_path() to find py_func/__main__.py and import from
  runtime.*
- Hidden imports updated: libs.* → api.*, plus py_func.* explicitly
  declared so PyInstaller pulls them into the bundle even though
  they are loaded as data

Smoke-tested: built binary runs `testium -b`, py_func subprocess works.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-02 14:07:48 +02:00
35ca0a8b45 pyinstaller package updated 2026-05-02 09:58:46 +02:00
4529da7aee Restructure: consolidate everything inside testium/ package
Move src/lib/ → src/testium/runtime/ (internal plumbing)
Move src/testium/libs/ → src/testium/api/ (public SDK for test scripts)
Move src/py_func/ → src/testium/py_func/ (Python subprocess)
Move src/lua_func/ → src/testium/lua_func/ (Lua subprocess data)

The package now ships as a single coherent unit instead of four sibling
top-level packages (testium, lib, py_func, lua_func) — pip install
gives a clean site-packages/testium/ with no namespace pollution; .lua
files travel with the wheel via package_data; the wheel installs
cleanly and `testium -b` runs end-to-end including py_func subprocesses
and entry-point exporter plugins.

Naming:
- runtime/ (internal, no API guarantees) clearer than lib/
- api/ (public SDK consumed as `import api.testium as tm`) clearer than libs/

Imports updated en masse: from lib. → from runtime. and from libs. →
from api., plus the importlib.import_module("libs.*") strings in
test_item_console.py and test_item_runtime_plot.py. Test/example
scripts (helper_lib.py, parallel.py, post_execution.py) and the
fake_exporter test suite migrated too.

paths.py: subproc_path() now returns testium_path() — both point at
the testium package directory since the subprocesses live inside.

pyproject.toml: removed exclude=["lua_func", "py_func"] (no longer
needed), added package-data for testium.lua_func/*.lua, removed the
license classifier (PEP 639 conflict with license expression).

Subprocess isolation contract: py_func/ and lua_func/ may only import
runtime/ and their own modules — never interpreter/, main_win/, api/,
or testium/. Enforced by test/validation/items/isolation/ which runs a
py_func that statically scans subprocess source files for forbidden
imports. The contract holds today; the test prevents future drift.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-02 09:28:40 +02:00
8bd9b3e9d6 Add plugin registry for report exporters
Replace the hardcoded if/elif in Export.exec() with a dict registry.
Built-in formats (text, json, junit, html) are registered as lazy
loaders; missing optional deps (junit_xml, lxml) print a clear message
with a pip install hint instead of raising. Entry-points
(group "testium.exporters") are discovered at import time — installed
plugins are auto-detected with no extra config.

An unknown or unavailable format prints an info line and skips the
export; the test run is not interrupted.

Validation:
- New testium-fake-exporter package under test/validation/fake_exporter/
  installed automatically by scripts/build_env.sh on venv creation.
  It registers fake_format via entry-points and exports the tests
  table to CSV — a real, useful exporter that exercises the plugin
  contract end-to-end (entry-point discovery, dispatch, SQLite query).
- New dedicated items/report_plugin/ test exercises both the
  unknown-format skip path and the fake_format plugin path, with a
  py_func check (file_check.py) on the produced CSV. Runs once per
  validation suite.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-01 23:16:10 +02:00
a70b70db54 Rework About dialog: licence, copyright, proper version display
- about_win.ui: QVBoxLayout, version shown in a word-wrap QLabel
  (sized to content, no oversized text area), add labelCopyright
  (© 2025-2026 François Dausseur) and labelLicence (EUPL-1.2 link)
- about_win.py: regenerated from UI
- testium_win.py: set labelVersion from get_testium_version() (branch,
  dirty flag, commit or binary/Flatpak label)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-01 19:30:37 +02:00
d7f25718d0 CLAUDE.md: consolidate recent fixes, fix PASS/FAIL terminology
Merge the two "Recent fixes" sections into one (branches are gone),
add parallel_branch icon, F1 panel, test-tree state, unittest rename,
run item rename, licence. Fix SUCCESS/FAILURE → PASS/FAIL in run item.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-01 19:02:02 +02:00
9db0f89522 removed reference to "terminal" mode. Corrected errors in doc generation. 2026-05-01 08:20:29 +02:00
f38a24190d Adopt EUPL-1.2 licence for the project
Add LICENSE (full EUPL-1.2 text + project copyright), CONTRIBUTING.md
with the inbound = outbound rule, and declare the licence in
pyproject.toml and README.md.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-01 07:58:22 +02:00
b16494ef6d Preserve test tree check and fold state across reload
Reload (re-loading the same .tum file) was rebuilding the GUI tree
from scratch, resetting every checkbox and unfolding everything.
Snapshot the user's selection and fold state via the existing
getCheckList/restoreCheckList and getFoldList/restoreFoldList methods
(already used for session persistence through prefs), so a same-file
reload keeps both as well. A change in the total item count (file
edited between loads) skips the restore.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-30 23:51:36 +02:00
b175ff4189 Add distinct icon for parallel branch tree items
A parallel branch now displays a single-arrow icon (parallel_branch.png)
distinct from the parallel container's three-arrow icon, making the
tree hierarchy easier to read at a glance.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-30 23:41:31 +02:00
d66a46736f Hide branch steps in parallel item F1 panel
The 'branches' list shown in the F1 panel previously contained each
branch's full 'steps', duplicating what is already visible in the test
tree. Strip 'steps' inside each branch dict while keeping 'branches'
itself so per-branch attributes (name, wait_for, condition, ...) stay
visible.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-30 23:35:43 +02:00
1b2d427ced Add parallel test item with thread-aware stdout routing
The parallel item runs branches concurrently with sync:all or sync:any
policy and optional per-branch wait_for synchronization. Each branch
runs in its own daemon thread and produces a clean per-item entry in
the SQLite report; the live output is prefixed [<branch_name>] so
concurrent branches stay readable.

Supporting changes:
- StdoutProxy (lib/stdout_redirect.py): thread-aware sys.stdout/stderr
  with per-thread capture buffers and per-branch live-output prefix.
  Adds writeln() for Python 3.14 unittest compatibility.
- TestItemContainer: shared base extracted from Group/Cycle for the
  sequential children execution pattern.
- TestItemSleep: interruptible loop polling _is_stopped so sync:any
  can cancel slow branches quickly.
- TestReport: thread-safe SQLite (check_same_thread=False + lock).

Also drops the unused -m/--terminal mode and its module.

Validation: 11 scenarios in test/validation/items/parallel covering
sync:all/any, no_fail, wait_for + timeout, conditions, multi-branch,
nested parallel, parallel inside loop, real branch failure.

Documentation: new parallel_test_item.rst added to the manual.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-30 23:23:31 +02:00
be540cd304 text mode effort finished on batch. 2026-04-27 17:20:15 +02:00
476b59c6f7 icons 2026-04-27 17:19:14 +02:00
bcafbfae18 Changed the "run" icon by testium. 2026-04-27 12:04:19 +02:00
e56a1f72c8 requirements changed for doc in readme. 2026-04-27 08:07:53 +02:00
83411482b2 Rename unittest_file item to unittest
- constants.py: TYPE_UNITTEST_FILE → TYPE_UNITTEST, cmd "unittest_file" → "unittest"
- All Python files updated: test_item_unittest.py, test_set.py, test_init.py,
  terminal.py, report_export_txt.py, test_tree_item.py
- All .tum files updated (examples, validation, doc)
- Sphinx doc: unittest_file_test_item.rst → unittest_test_item.rst,
  all references updated

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 08:05:40 +02:00
a28e644621 doc: use PASS/FAIL terminology in run item doc
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 07:59:54 +02:00
4a4a70b5f6 pdf doc updated 2026-04-27 07:56:43 +02:00
06c4cc62c6 doc: update run item documentation
Clarify result semantics (SUCCESS on launch, not on sub-test result),
batch vs GUI mode behaviour, and clean up attribute descriptions.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 07:53:26 +02:00
60dbcf0252 Fix run item and batch mode robustness
- run item: rename tum_fime→tum, remove stdout=PIPE (deadlock with
  spawn), support batch mode (-b), SUCCESS on any completed subprocess
  regardless of sub-test result
- batch.py: fix control("loaded") deadlock via daemon thread + Event +
  is_alive() polling; fix premature finish on gd_update messages;
  propagate success flag from finished message; guard control("close")
- process.py: include success flag in send_finished message
- py_process/lua_process: add stdout/stderr=DEVNULL to Popen
- test_run.py: fix finished detection ("id" in m and m["id"] is None)
- testium_win.py: track run_exit_code, SIGABRT handler, clean exit
- __init__.py: sys.exit with batch success flag
- Add run item validation tests and CLAUDE.md documentation

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 07:49:16 +02:00
a3e449cc7d in batch mode, the dialog allways return FAIL, except if auto_result is defined (validation only).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-26 11:52:52 +02:00
95107117fa Color is automatically adapted to the theme of the console.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-26 11:50:28 +02:00
88cc410eed fix of blocking of the text output in batch mode.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-26 11:49:52 +02:00
fa7f8cef7c Text mode upgrade (to be fixed). 2026-04-26 09:20:39 +02:00
5a065128be Fix lua delgd test: use sentinel default instead of nil comparison
cjson decodes JSON null as cjson.null, not Lua nil, so == nil always fails.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-24 10:25:30 +02:00
b7b930aab1 Add validation tests for OS, get_main_dir, timestamp, timestamp_as_sec helper functions
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-24 10:21:56 +02:00
609ca57202 Add delgd validation test for py_func and lua_func
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-24 10:18:31 +02:00
d26b60435b Add auto_result param to dialog items for automated validation
Each dialog test item now accepts an optional auto_result parameter
(ok/cancel/yes/no) and auto_value for text dialogs. When set, the dialog
window opens, stays visible 2 seconds, then closes automatically with the
specified result — allowing the validation suite to run without manual
interaction.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-24 10:07:07 +02:00
de143b6cc3 Fix EOFError crash when dialog subprocess exits without sending a result
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-24 10:06:55 +02:00
d955ae81f9 Added variables list in "F1" dialog. They are modifiable. To be tested. 2026-04-20 23:27:39 +02:00
2cd3aa3305 updated doc + param 2026-04-20 22:55:09 +02:00
276d485905 Add store_result common attribute to test items
Allows any test item to store its result (or PASS/FAIL status when result
is None) into a named global variable, available to subsequent items via
$(variable_name). store_result runs after expected_result but before
no_fail so the real outcome is always captured.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-20 22:26:47 +02:00
95912dd3e1 Fix 'process_result must fail' test missing expected_result
Without expected_result, a False process_result value does not fail the
test. Adding expected_result: True makes the comparison fail as intended.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-20 21:43:33 +02:00
6d1fb6a6bc Add JSON-RPC echo server for validation suite
Replaces the external jrpces binary dependency with a self-contained
Python script. The server supports TCP (newline-delimited JSON, port 4321)
and UDP (port 4323), handles JSON-RPC 1.0 and 2.0, and implements:
  - echo(*args) -> [args, {}]
  - unknown methods -> error {code: -32000, message: "function not found"}

test.tum is updated to launch jrpc_echo_server.py via python3 and wait
for the "ready" readiness message before running tests.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-20 21:34:43 +02:00
2cc42e9065 Fix 15-second close delay after dialog tests
Dialog subprocesses were forked from TestProcess, inheriting its
multiprocessing Queue objects and their process-shared POSIX semaphores
(_wlock). If a fork happened while the feeder thread held _wlock, the
child exited without releasing it, permanently blocking the feeder
thread on the next wacquire() and stalling Python's atexit _finalize_join
— causing test_proc.join() (no timeout) to hang the app for ~15 seconds.

Fix: use multiprocessing.get_context('spawn') for dialog subprocesses so
they start with a clean interpreter and inherit no semaphores or Queue
state. Also add a terminate/kill fallback timeout to test_proc.join() as
a safety net, and fix the missing return in JsonRpcConnection.is_alive().

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-20 21:14:33 +02:00
2b7678c39e Fix dialog subprocesses: centralize Qt env setup and extract base class
- Add dialog_env.py service: forces QT_QPA_PLATFORM=xcb on Linux so Qt
  doesn't crash under Wayland in spawned subprocesses
- Use QMessageBox instance (instead of static methods) for msg/question
  dialogs so WindowStaysOnTopHint can be set, making them visible
- Add TestItemDialogBase with _run_dialog/_run_dialog_with_result/_cleanup_process,
  removing duplicated subprocess launch/poll/terminate logic from all 7 dialog items
- Reduce terminate() join timeout from 2s to 0.2s across all dialog items

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-19 20:01:56 +02:00
c72176d029 Improve loading error messages with item context and hierarchy path
Add item_load_context() context manager to tum_except.py that enriches
ETUMSyntaxError with the item type, name, and parent path instead of
replacing the original message with a vague 'missing or wrong parameter'.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-19 10:05:40 +02:00
617f599f86 Fix context subprocess leak and document py_func.tm helpers
- process.py: stop context_id engines in the inner finally block, before
  restore_gd() wipes _py_func_contexts/_lua_func_contexts from the global
  dict — engines were previously orphaned after every test run
- py_func/tm.py: add user-facing docstrings to gd/setgd/delgd; remove
  internal JSON-serialization details from the docs
- helper_lib.rst: auto-generate global variable helpers from py_func.tm
  (the actual subprocess API) instead of globdict
- conf.py: add src/ to Sphinx sys.path so py_func.tm is importable
- py_func_test_item.rst: simplify context sharing section, remove
  JSON-serializable/non-serializable distinction for end users
- Regenerated PDF manual

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-18 16:27:31 +02:00
d92f518e1e Add context_id to py_func and lua_func for shared persistent subprocess
- py_func and lua_func items accept a context_id parameter; items sharing
  the same id reuse the same subprocess for the duration of the test run
- Subprocess-side tm.setgd/tm.gd use a local fallback dict for non-JSON-
  serializable values (py_func only); serializable values reach the main
  process global dict and are accessible from any test item or subprocess
- Shared subprocess engines are cleaned up in process.py finally block
- LuaProcessBase gains is_alive() (was missing, broke all lua_func items)
- Validation tests cover serializable sharing across different context ids,
  non-serializable sharing within the same context_id, and cross-item access
- RST documentation updated for both py_func and lua_func items

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-18 16:02:36 +02:00
49354b8664 Merge branch 'feature/reload-without-restart'
Reload test without restarting testium (closes #18)
2026-04-18 15:24:11 +02:00
7383820aba Reload test without restarting testium (closes #18)
Replace os.execv restart in actionRefresh with file_manager.reload(),
leveraging the subprocess architecture so py_func modules are freshly
imported on each reload. Add a modal progress dialog with step labels
during loading. Fix checkbox reappearing on breakpoint with
show_checkboxes OFF.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-18 15:23:53 +02:00
67c879ab10 Fix checkboxes reappearing when setting a breakpoint with show_checkboxes OFF
itemChanged fires for any data change (including icon updates in the pause
column), causing on_testChecked to inadvertently restore CheckStateRole via
synchronizeEnabledState. Guard against non-checkbox column changes.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-18 14:55:18 +02:00
aa72e349c6 Merge branch 'refactor/ui-modularization'
- Extract TestControllerService service layer over TestSetController
- Split MainWindow into TestRunner and TestFileManager coordinators
- Merge 21 QTestTreeItem subclasses into a single factory
- Replace _test_started/_test_paused booleans with TestState enum
2026-04-18 14:46:06 +02:00
217 changed files with 9402 additions and 2686 deletions

3
.gitignore vendored
View File

@@ -8,6 +8,8 @@ dist
/.vscode
.venv/
.flatpak-builder/
package/flatpak/repo/
package/flatpak/*.flatpak
crash.tx*
report_test.tx*
*.autosave
@@ -24,6 +26,7 @@ package/appimage/*.AppImage
package/appimage/src
package/appimage/*.py
AppDir
*.squashfs
doc/manual/doxygen
doc/manual/sphinx/build/*
doc/manual/sphinx/source/_build/*

165
CONTRIBUTING.md Normal file
View File

@@ -0,0 +1,165 @@
# Contributing to testium
Thank you for your interest in contributing to testium.
## License of contributions
testium is licensed under the **European Union Public Licence v. 1.2 (EUPL-1.2)**
see the [LICENSE](LICENSE) file at the repository root.
By submitting a contribution to this project (pull request, patch, issue
attachment, or any other form of code, documentation or media), you agree
that your contribution is licensed to the project and to the public under the
**same EUPL-1.2** terms (or any later version of the EUPL approved by the
European Commission), and you certify that:
- you are the author of the contribution, or you have the right to submit it
under the EUPL-1.2;
- to the best of your knowledge, the contribution does not infringe any
third-party intellectual-property rights;
- the contribution may be redistributed by the project under the EUPL-1.2 and
any compatible licence listed in the EUPL-1.2 Appendix.
This is the **inbound = outbound** rule: contributions come in under the same
licence the project ships under.
You retain copyright on your contribution. The project does **not** ask you
to sign a CLA or assign your copyright.
## SPDX header in new source files
When creating a new source file, please include the following header at the
top of the file (adjust the comment marker to the file's language):
```python
# SPDX-License-Identifier: EUPL-1.2
# Copyright (c) <year> <your name>
```
For existing files, keep the header that is already there.
## How to contribute
1. Open an issue describing the change you want to make (bug, feature, doc).
2. Fork the repository, create a topic branch.
3. Commit with a clear message (one logical change per commit).
4. Make sure the validation suite still passes:
```
./run.sh -b -- test/validation/main.tum
```
5. Open a pull request against `main`.
## Coding conventions
- Python ≥ 3.11
- Follow existing style in the file you are modifying
- Add or update tests in `test/validation/` for new test items or behaviours
- Update `CLAUDE.md` and the Sphinx manual for user-visible changes
## Development
### Debugging in VSCode
The recommended workflow:
1. Add a debug configuration to `.vscode/launch.json`:
```json
{
"configurations": [
{
"name": "Python : testium",
"type": "python",
"request": "launch",
"program": "${workspaceFolder}/src/testium",
"console": "integratedTerminal",
"args": ["-g"],
"justMyCode": true
}
]
}
```
2. Install `debugpy` in the venv: `python -m pip install debugpy`.
3. Open the *Run and Debug* tab and press play. testium starts; load and
run a `.tum` file. Set breakpoints where you want to investigate.
### Qt GUI modification
UI files (`*.ui`) are edited in **Qt Creator**. After editing, regenerate
the corresponding Python and resource files:
```sh
scripts/qt_generate.sh
```
Icons come from <https://github.com/free-icons/free-icons>.
### Sphinx documentation
```sh
pip install sphinx linuxdoc
doc/manual/sphinx/build_doc.sh
```
PDF generation requires `texlive`:
```sh
sudo apt install texlive-full
```
### Validation suite
Batch mode (CI-friendly, headless):
```sh
./run.sh -b -- test/validation/main.tum
```
GUI mode (loads the suite, click *Run* to execute and inspect the tree):
```sh
./run.sh test/validation/main.tum
```
GUI run-and-close (executes the suite, then closes):
```sh
./run.sh -r -- test/validation/main.tum
```
Subset run via the `items` define (works in any mode):
```sh
./run.sh -b -d "items=['parallel','common']" -- test/validation/main.tum
```
### Cross-distribution check
`package/deb/test_distro.sh` spins up a Docker/Podman container of the
target image, installs the expected system Python deps via apt (with
pip fallback for what is missing), installs the testium wheel and runs
the validation suite end-to-end. Currently green on `debian:bookworm`,
`debian:trixie`, `ubuntu:24.04`.
```sh
./package/deb/test_distro.sh debian:trixie
```
## Release procedure
1. Update `release_note.txt`.
2. Bump the version in `src/VERSION`.
3. Make sure the documentation is up to date — rebuild with
`doc/manual/sphinx/build_doc.sh` if needed.
4. Push and tag the commit with the new version.
5. Build the binary release: `package/pyinstaller/build.sh`.
6. Run the validation suite against each generated binary.
7. Confirm all validation results are green before publishing.
## Reporting security issues
Please do **not** report security vulnerabilities through public GitHub
issues. Instead, send an email to the project maintainer directly.
## Questions
Open a GitHub Discussion or an issue tagged `question`.

288
DESIGN.md Normal file
View File

@@ -0,0 +1,288 @@
# Testium — Design Context
## What is testium
Testium is a test sequencer/runner written in Python. It executes YAML-based test scripts ("`.tum`" files) and supports two execution modes:
- **GUI mode** (default, no flag): PySide6 Qt application (`src/testium/main_win/`)
- **Batch mode** (`-b` / `--batch-execution`): headless, non-interactive, runs tests and exits
Run from repo root: `./run.sh` (Linux) or `run.bat` / `run.ps1` (Windows).
Direct invocation: `python3 -m src/testium [-b] <test_file.tum>`
## Architecture
### Entry point
`src/testium/__init__.py` — parses CLI args, dispatches to the two modes.
`multiprocessing.set_start_method('spawn')` is called early (required for Linux dialog subprocesses).
### Core execution
- `src/testium/interpreter/process.py``TestProcess(multiprocessing.Process)`: runs the test in a child process. Stdout is redirected via a `StringQueue` → pipe → parent thread (`capture_stdout`) that writes to real stdout.
- `src/testium/interpreter/batch.py``Batch`: parent-side orchestrator for `-b` mode. Creates the `msg_queue`, starts `TestProcess`, waits for the "finished" signal.
- `src/testium/interpreter/test_set.py``TestSet`: builds and executes the tree of test items.
- `src/testium/interpreter/test_items/test_item*.py` — one file per test item type (check, cycle, group, let, unittest, py_func, lua_func, console, git, dialogs, report, parallel, …).
### Communication channels (parent ↔ child process)
- `msg_queue` (`multiprocessing.Queue`): carries status messages from child to parent.
- Item status: `{"id": <non-None>, "name": ..., "status": "started"|"finished", ...}`
- Global dict updates: `{"type": "gd_update"|"gd_delete", "key": ..., "value": ...}`**no "id" key**
- Process finished: `{"id": None, "name": "test_process", "status": "finished"}` — id key present but `None`
- `tst_ctrl` (`TestSetController`): sends control commands (execute, stop, pause, close, …) from parent to child.
- stdout pipe (`multiprocessing.Pipe`): streams test output from child back to parent's `capture_stdout` thread.
### Stdout pipeline (batch mode)
```
test item print()
→ sys.stdout (StringQueue, in child)
→ send_stdout thread (child) → pipe → capture_stdout thread (parent)
→ print() → sys.stdout (TermLog wrapping real stdout, in parent)
→ terminal
```
### Global dictionary
`src/testium/interpreter/utils/globdict.py` — shared state accessible from test scripts via `tm.gd()` / `tm.setgd()`. When `set_update_queue()` is active (during test execution), every `setgd`/`delgd` on a non-`_`-prefixed key pushes a message to `msg_queue`.
### Coloring (`-o` disables it)
`src/testium/interpreter/utils/termlog.py``TermLog` wraps stdout with colorama-based line coloring (PASS=green, FAIL=red, WARN=yellow, …). Applied in parent process for batch mode. Auto-detects light/dark terminal background via (in order): `COLORFGBG` env var, OSC 11 query, default dark.
### Dialog items in batch mode
All dialog items (`dialog_image`, `dialog_question`, `dialog_references`, `dialog_value`, `dialog_message`, `dialog_choices`, `dialog_note`) follow this rule in non-interactive text mode (`-b`):
- `auto_result` defined in the `.tum` → result controlled by it (`ok`/`yes` → SUCCESS, `cancel`/`no` → FAIL)
- `auto_result` absent → FAIL with `"Dialog not supported in batch mode"`
- `sleep dialog: true` → exception: just sleeps normally, no GUI, no failure
`auto_result` (and `auto_value` for value/note dialogs) is intended for the validation test suite (`test/validation/`) only.
### `parallel` item
`src/testium/interpreter/test_items/test_item_parallel.py` — runs multiple branches concurrently.
```yaml
- parallel:
name: My parallel block
sync: all # all: wait for all; any: stop as soon as one finishes
no_fail: true # (optional) don't propagate branch failures to parent
branches:
- name: Branch A
wait_for: # (optional) poll condition before starting
condition: <| expr |>
timeout: 10
steps:
- ...
- name: Branch B
steps:
- ...
```
- `TestItemParallel(TestItemContainer)`: mutates `dict_item["steps"]` to inject synthetic `parallel_branch` items so `load_test_recursively` loads branches normally as children.
- `TestItemParallelBranch(TestItemContainer)`: container for one branch. `wait_for` polls every 0.1s up to `timeout` seconds before running steps.
- `sync: any` calls `_stop_branch_recursively()` on all other branches when one *actually runs* (SUCCESS/FAILURE). A `NORUN` branch (disabled, condition not met) never wins the race.
- Each branch runs in a daemon thread; the parent waits with `.join()`.
- Branches stopped late (e.g. user disabled them in the GUI, or another sync:any branch already won) go through the normal `branch.stop() + branch.execute()` path so they always produce a clean DB entry via `addTest()`.
- Exceptions raised in a branch's `execute()` are caught by `run_branch`, logged to stdout, and converted to a `FAILURE` result so they never disappear silently.
- `sync: all` ignores `NORUN` branches when computing success (matches Group/Cycle semantics): only an actual `FAILURE` fails the parallel.
- `TestItemSleep` is interruptible (polls `self._is_stopped` in a loop) so `sync: any` can stop slow branches quickly. `py_func` and `console` items are not interruptible; their full duration is observed before the branch returns.
### `TestItemContainer` base class
`src/testium/interpreter/test_items/test_item_container.py` — shared base for Group, Cycle, Parallel, and ParallelBranch. Provides `_run_children_sequentially()` which handles stop-on-failure, `executedOnStop` items, and returns `(TestResult, stopped_bool)`.
### Report threading
`src/testium/interpreter/test_report/test_report.py` — SQLite report with thread-safe writes:
- `sqlite3.connect(..., check_same_thread=False)`
- `self._lock = threading.Lock()` guards the SQLite `INSERT` only.
- Per-item log capture (`stdio_redir.read()`) is naturally race-free thanks to per-thread buffers (see `StdoutProxy`).
### Thread-aware stdout (`StdoutProxy`)
`src/testium/runtime/stdout_redirect.py` — when `log_stored: True`, `intercept()` installs a `StdoutProxy` as `sys.stdout`/`sys.stderr` instead of a single shared `StringQueue`. The proxy:
- Holds one `StringQueue` per thread (registered via `register_thread(buffer=...)`). The main thread uses a default buffer; each parallel branch's thread registers its own at start and unregisters at end. `stdio_redir.read()` reads the calling thread's buffer → `addTest()` of an item running in branch X reads X's clean, non-interleaved output.
- For the live stream (terminal in batch / GUI panel), prefixes every line emitted from a branch's thread with `[<branch_name>] ` so concurrent branches stay readable.
- Exposes `write` / `writeln` / `flush` (Python 3.14's `unittest` calls `stream.writeln()` directly without `_WritelnDecorator`).
### Subprocess API contract (py_func / lua_func)
User test scripts running inside a `py_func` or `lua_func` subprocess **must** use the JSON-RPC bridge to interact with testium state:
- Python: `import py_func.tm as tm` — auto-generates wrappers for every function in `runtime/api.py:SUPPORTED_API`. `tm.gd`/`tm.setgd`/`tm.delgd` go through JSON-RPC to the parent.
- Lua: `local tm = require("tm")` — same idea on the Lua side.
`api.testium` is the *main-process* implementation; it is **not** exposed to subprocesses by design (not bundled in PyInstaller, not on the subprocess `PYTHONPATH` in pip-installed mode either when isolation is preserved). An import attempt from a subprocess script is a code smell and is detected by `test/validation/items/isolation/`.
To add a new API call usable from subprocesses:
1. Add the function to `api/testium.py`
2. Add its name to `SUPPORTED_API` in `runtime/api.py`
3. It is auto-exposed via JSON-RPC by `interpreter/utils/api_srv.py` and auto-wrapped by `py_func/tm.py:_make_api`
### External interpreter resolution (`bins.py`)
`src/testium/interpreter/utils/bins.py` — single source of truth for the paths to the external Python and Lua interpreters used by subprocesses.
- `python_bin()` / `lua_bin()` : resolve and cache. The cache is keyed by `(name, override)` so that a later change to `gd[python_bin]` (typically when a `param.yaml` sets the key) triggers a re-resolution on the next lookup instead of returning the stale auto-discovered path. Falls back to discovery on PATH (candidates: `python3`/`python` and `lua`/`lua5.5`/`lua5.4`/`lua5.3`/`lua5.2`/`lua5.1`).
- `ensure(*names)` : called by `TestSet._validate_runtime_deps()` at test load. Always requires `python` (the eval engine always runs); requires `lua` only if a `lua_func` item is in the tree. Fails fast with a clear error citing tried candidates and override key.
Engines (`PyProcessBase`, `LuaProcessBase`, `EvalExecEngine`) call `bins.python_bin()`/`bins.lua_bin()` themselves — call sites never pass an explicit binary path.
#### Override-timing contract (`apply_overrides`)
`bins.python_bin()` is called for the **first** time inside `eval_process_init()` (the long-lived inline-`<| … |>` subprocess), which happens **before** the YAML param files are loaded. To make `-d python_bin=…` and the GUI `python_bin` preference take effect for `eval_proc` itself, `process.py:run()` applies them to gd **before** `eval_process_init()` via the `apply_overrides()` helper extracted from `update_global()`. The post-load `update_global()` call then re-applies the same overrides (after `prepare_global()` clears gd), keeping the gd value in sync with the cached resolution.
| Override source | `eval_proc` | `py_func` / `cycle` / `post_exec` |
|---|---|---|
| `-d python_bin=…` (CLI) | ✅ | ✅ |
| GUI `python_bin` preference | ✅ | ✅ |
| `python_bin: …` in `param.yaml` | ❌ (eval_proc already started) | ✅ (cache re-resolves on key change) |
## Key files
| Path | Role |
|------|------|
| `src/testium/__init__.py` | CLI entry, mode dispatch |
| `src/testium/interpreter/batch.py` | `-b` mode orchestrator |
| `src/testium/interpreter/process.py` | Child test process |
| `src/testium/interpreter/test_set.py` | Test tree builder/executor |
| `src/testium/interpreter/test_items/test_item_container.py` | Base class for container items |
| `src/testium/interpreter/test_items/test_item_parallel.py` | `parallel` and `parallel_branch` items |
| `src/testium/interpreter/utils/globdict.py` | Global variable dict |
| `src/testium/interpreter/utils/termlog.py` | Terminal color output |
| `src/testium/runtime/stdout_redirect.py` | `StdioRedirect` singleton (`stdio_redir`) |
| `src/testium/runtime/string_queue.py` | Thread-safe string buffer used for stdout redirection |
| `src/testium/api/testium.py` | Public API for test scripts (`tm.*`) |
| `src/testium/py_func/` | Python subprocess for `py_func` items (sandboxed: imports only `runtime/` and `py_func/`) |
| `src/testium/lua_func/` | Lua subprocess scripts for `lua_func` items |
## Package layout
The whole project is a single Python package under `src/testium/`:
```
src/testium/
├── __init__.py / __main__.py
├── runtime/ internal plumbing (jrpc, stdout_redirect, string_queue, tum_except, api)
├── api/ public SDK exposed to test scripts (`import api.testium as tm`)
├── interpreter/ test execution engine (NOT visible to py_func/lua_func)
├── main_win/ GUI (NOT visible to py_func/lua_func)
├── py_func/ subprocess code for python_func items
└── lua_func/ subprocess scripts for lua_func items (data files)
```
`subproc_path()` and `testium_path()` both return the package directory. The py_func subprocess is launched with cwd=that directory and `python3 py_func`. The contract that `py_func/` and `lua_func/` only depend on `runtime/` (no `interpreter`, `main_win`, `api`, `testium`) is enforced by `test/validation/items/isolation/`.
## GUI icons (main_win)
Icons live in `src/testium/main_win/resources/` with three theme variants:
| Folder | Theme index | Usage |
|--------|-------------|-------|
| `color/` | 0 (default) | Coloured icons |
| `black/` | 1 | Black silhouette on transparent |
| `white/` | 2 | White silhouette on transparent (LA mode) |
Icons are **64×64 PNG**. Black variants: RGBA with RGB=`(0,0,0)`, alpha varies. White variants: LA with luminance=`255`, alpha varies.
The mapping item-type → icon filename is in `_ITEM_CONFIG` (`src/testium/main_win/test_tree_items/test_tree_item.py`). At runtime, `icon_prefix()` returns `:/color`, `:/black`, or `:/white` (Qt resource prefix) based on the user preference.
All icons must be declared in `src/testium/main_win/resources/testium_core_win.qrc` (one entry per theme section). After any QRC change, regenerate the compiled resource file:
```
cd src/testium/main_win/resources
pyside6-rcc testium_core_win.qrc -o testium_core_win_rc.py
```
Icons are assigned once when the test file is loaded (not updated live on theme change — a file reload is required).
### `run` item
`src/testium/interpreter/test_items/test_item_run.py` — launches a `.tum` file in a new testium instance (`-b` in batch mode, `-r` in GUI mode). Result:
- **PASS** if the sub-instance launched and ran to completion (exit code is ignored)
- **FAIL** if the file is not found, `wait_for_exec` is set without `start_time`/`end_time`, the time window was not reached, or any other launch error
The sub-test's own pass/fail result is intentionally not propagated.
The interpreter and entry point used to spawn the sub-instance are picked automatically by `_testium_launch_cmd()` based on how the parent was started (AppImage → `$APPIMAGE`; Flatpak → `flatpak run`; PyInstaller → the frozen binary; source/wheel → `[sys.executable, abspath(sys.argv[0])]`). The user cannot override either via the YAML — selecting a different testium binary or Python from a sub-test was removed because it was either ill-defined (bundle modes have no separable Python) or could mismatch the parent's environment in surprising ways.
### Report exporters & plugins
`src/testium/interpreter/test_report/test_report.py``_EXPORTER_REGISTRY` dict maps a format name (cmd key in the YAML `report.export`) to a lazy loader. Built-ins: `text`, `json`, `junit` (needs `junit_xml`), `html` (needs `lxml`). `sqlite` is the storage layer, no-op as an export.
Third-party plugins are discovered at module import via `importlib.metadata.entry_points(group="testium.exporters")` — installing a wheel that declares such an entry point is enough, no testium config change needed:
```toml
[project.entry-points."testium.exporters"]
my_format = "my_pkg:MyExporter"
```
Exporter contract: `__init__(self, name, con, path, pats, keys, no_header=False)` — the class does its work in `__init__` and writes to `path`.
Behaviour on errors:
- Unknown format → info line `[report] Export skipped: format "X" not found. Available: ...`, run continues.
- Optional dependency missing → same info line with a pip-install hint, run continues.
A real-world test plugin lives at `test/validation/fake_exporter/` (CSV exporter, auto-installed by `scripts/build_env.sh` and exercised by `test/validation/items/report_plugin/`).
## Packaging
Four distribution channels coexist, all sharing the single `src/testium/` package and the single `src/requirements.txt` dependency list:
| Channel | Where | Build | Notes |
|---------|-------|-------|-------|
| Wheel (`pip install`) | `src/pyproject.toml` | `python -m build` | Vanilla Python package; entry point `testium = "testium:main"`. |
| PyInstaller binary | `package/pyinstaller/` | `build.sh` | Single ~130 MB binary. `py_func`, `runtime`, `lua_func` bundled at `_MEIPASS` root so the **host** Python can find them when launched as `python3 py_func`. `api`/`interpreter` are **not** exposed (subprocess isolation). |
| Flatpak | `package/flatpak/` | `build.sh` (uses `flatpak-builder`) | KDE 6.10 runtime. The bundled Python runs only the main process; `py_func` / `lua_func` MUST run under the **host** interpreter (no Python/Lua bundled). Produces a distributable `.flatpak` bundle. |
| AppImage | `package/appimage/` | `build.sh` (Debian Bookworm container via Podman/Docker) | Bundles Python 3.11 for the main process; `py_func` / `lua_func` MUST run under the **host** interpreter. Build runs in a container so it works on Arch / any non-Debian host. |
The `.deb` work-in-progress lives in `package/deb/`:
- `test_distro.sh debian:bookworm | debian:trixie | ubuntu:24.04` spins up a Docker/Podman container, reports system package availability, falls back to pip for what's missing (`pyside6` on bookworm/ubuntu, `telnetlib3`, `junit_xml`), runs the validation suite. Currently green on the three targets.
### Host-only py_func / lua_func in sandboxed bundles (Flatpak, AppImage)
The bundled Python (Flatpak's runtime python, AppImage's `python3.11`) is reserved for the **main process only**. Subprocesses (`py_func`, `lua_func`, `git`) must use the host's interpreters and tools so user-installed modules (pyserial, junit_xml, …) are visible. This is enforced by `interpreter/utils/bins.py`:
- `_in_flatpak()` (checks `/.flatpak-info`) and `_in_appimage()` (checks `APPIMAGE` env var) detect the sandbox.
- `_which(name)` probes only host bin dirs in those modes:
- Flatpak: `/run/host/usr/{local/,}bin`, `/run/host/bin` (host mounted via `--filesystem=host-os`).
- AppImage: `/usr/local/bin`, `/usr/bin`, `/bin` (we are directly on the host filesystem).
- If the host has no python3/lua, `ensure()` raises `ETUMRuntimeError` at test load with the candidate list — no silent fallback to a bundled interpreter.
- User overrides (`python_bin`/`lua_bin` in globdict): bare names are resolved through `_which()` (host-only), absolute paths are accepted as-is.
- `apply_host_libs(env)` is called by `py_process.py` / `lua_process.py` on the env passed to Popen:
- Flatpak: prepends host lib dirs to `LD_LIBRARY_PATH` so the dynamic linker finds host `.so`'s.
- AppImage: strips `$APPDIR`-prefixed entries from `LD_LIBRARY_PATH` / `PYTHONPATH` / `PATH` and drops `PYTHONHOME`, so the host Python doesn't try to load the bundled stdlib/site-packages.
- `apply_host_lua_paths(env)` (Flatpak only) prepends `/run/host/usr/{lib,share}/lua/X.Y` to `LUA_PATH` / `LUA_CPATH` so `cjson`, `socket`, etc. resolve. Must be called **after** user `lua_env` overrides so host paths win. AppImage relies on host Lua's compiled-in defaults.
- `py_process.py` additionally pops `PYTHONUSERBASE` (set to `/var/data/python` by the Flatpak runtime, which would hide `~/.local/lib/...`).
### Version reporting (`interpreter/utils/version.py`)
Both Flatpak and AppImage export `TESTIUM_VERSION` from a launcher (Flatpak: launcher script in `org.testium.Testium.yaml`; AppImage: `runtime.env` in `AppImageBuilder.yml`). `get_testium_version()` checks `/.flatpak-info` / `APPIMAGE` and reads `TESTIUM_VERSION` rather than relying on package metadata or repo introspection.
## Recent fixes / notable changes
- Restructure: single `src/testium/` Python package (was 4 sibling top-levels: `testium`, `lib`, `py_func`, `lua_func`). `lib/``runtime/`, `libs/``api/`. `pip install` now produces a clean `site-packages/testium/` with no top-level pollution; `.lua` files travel via `package_data`.
- `bins.py`: centralised resolution + cache of external `python3` / `lua` binaries. Replaces the scattered `tm.gd("python_bin")`/`tm.gd("lua_bin")` dance and the duplicated discovery logic in `py_process.py`/`lua_process.py`. Validates at test load via `TestSet._validate_runtime_deps()` so missing interpreters fail fast.
- Subprocess API contract: user scripts in `py_func`/`lua_func` use the JSON-RPC bridge (`py_func.tm` / Lua `tm`) — never `api.testium` / `interpreter.*` directly. `SUPPORTED_API` extended with `OS`, `get_main_dir`, `init_timestamp`, `timestamp`, `timestamp_as_sec` so subprocess scripts have the same surface as main-process code.
- Report exporter plugin registry (`test_report.py`): `_EXPORTER_REGISTRY` + `entry_points("testium.exporters")` discovery. Missing format → info line, run continues.
- About dialog rework: `QVBoxLayout` (resizable), version + dirty/branch info in a `QLabel` (auto-sized), copyright + clickable EUPL-1.2 link.
- `test_ctrl.control()`: drain stale responses (left over from polled `loaded()` after `clear()` race) instead of failing on a wrong cmd key — fixes a "Unexpected return error in test set controller" seen in GUI mode after a fast reload.
- `lua_process.py`: stderr no longer DEVNULL'd so actual Lua errors (missing `cjson`/`socket`) surface instead of "Connection refused".
- `run_post_exec`: failure message uses `print_warn` (was `print_debug` — silent in non-debug runs).
- Python 3.11 compat: replaced PEP 701 nested-quote f-strings (e.g. `f"... {d["k"]} ..."`) with single-quote inner strings or string concatenation.
- `parallel` item: new item with `sync: all|any`, `wait_for`, daemon threads, `_stop_branch_recursively()`. Each branch thread registers a per-thread stdout buffer.
- `parallel_branch` icon: distinct single-arrow icon (`parallel_branch.png`).
- `parallel` F1 panel: `steps` stripped from each branch dict.
- `test_item_container.py`: shared base class extracted from Group/Cycle.
- `test_item_sleep.py`: interruptible loop so `sync: any` can stop slow branches quickly.
- `stdout_redirect.py`: `StdoutProxy` (thread-aware buffers + branch-prefixed live output, `writeln()` for Python 3.14 unittest).
- `test_report.py`: thread-safe SQLite INSERT for parallel branch concurrency.
- `terminal.py`: deleted — `-m`/`--terminal` mode removed.
- `batch.py`: premature finish bug on `gd_update` (no `"id"` key) — fix uses `"id" in m and m["id"] is None`.
- `batch.py`: `control("loaded")` deadlock on TestProcess crash — fix uses daemon thread + `threading.Event` + `is_alive()` polling.
- `termlog.py`: light/dark terminal auto-detection (`COLORFGBG`, OSC 11) + write residue bug.
- Dialog items: `auto_result`/`auto_value` for non-interactive text mode; dialogs without `auto_result` FAIL immediately in batch.
- `run` item: renamed `tum_fime``tum`; removed `stdout=PIPE` deadlock; PASS on any completed subprocess.
- `unittest` item: renamed from `unittest_file`.
- GUI test tree: check and fold state preserved across same-file reloads.
- Licence: EUPL-1.2.
- Interpreter override timing: `apply_overrides()` extracted from `update_global()` and called by `process.py:run()` before `eval_process_init()`, so `-d python_bin=…` / GUI prefs reach `bins.python_bin()` on its first lookup. `bins._resolve()` cache is now keyed by `(name, override)` so later `param.yaml` changes are picked up by subsequently constructed engines.
## Validation tests
Located in `test/validation/`. Two entry points:
```
./test/validation/run.sh # wrapper — uses a dedicated venv (see below)
./run.sh -b -- test/validation/main.tum # direct — testium's own python is used for test execution
```
The `run.sh` / `run.bat` wrappers create a dedicated Python venv at `${TMPDIR:-/tmp}/testium-validation-venv` (Linux) or `%TEMP%\testium-validation-venv` (Windows), with `--system-site-packages` + `pip install junit-xml`, and run the suite with `-d python_bin=…` so every test-execution subprocess (eval_proc, py_func, cycle, post_exec) runs inside the venv. testium itself keeps running in the project's own environment. `clean` as the first argument recreates the venv.
The `venv` item (`test/validation/items/venv/`) asserts that the override actually took effect: `python_bin` is set, `sys.executable` matches it, `sys.prefix == dirname(dirname(python_bin))`, and `sys.prefix != sys.base_prefix` (the last marker catches the case where `python_bin` happens to be a system interpreter, which path-equality alone would miss because the venv's `bin/python3` is a symlink to the host). Both `eval_proc` (inline `<| … |>`) and `py_func` paths are exercised.
Parallel item tests: `test/validation/items/parallel/test.tum`
## Dependencies
See `src/requirements.txt`. Key ones: `pyside6`, `pyyaml`, `jinja2`, `colorama`, `gitpython`, `pexpect`, `matplotlib`.

315
LICENSE Normal file
View File

@@ -0,0 +1,315 @@
Copyright (c) 2025-2026 François Dausseur
Licensed under the EUPL
EUROPEAN UNION PUBLIC LICENCE v. 1.2
EUPL © the European Union 2007, 2016
This European Union Public Licence (the 'EUPL') applies to the Work (as
defined below) which is provided under the terms of this Licence. Any use of
the Work, other than as authorised under this Licence is prohibited (to the
extent such use is covered by a right of the copyright holder of the Work).
The Work is provided under the terms of this Licence when the Licensor (as
defined below) has placed the following notice immediately following the
copyright notice for the Work:
Licensed under the EUPL
or has expressed by any other means his willingness to license under the EUPL.
1. Definitions
In this Licence, the following terms have the following meaning:
- 'The Licence': this Licence.
- 'The Original Work': the work or software distributed or communicated by the
Licensor under this Licence, available as Source Code and also as Executable
Code as the case may be.
- 'Derivative Works': the works or software that could be created by the
Licensee, based upon the Original Work or modifications thereof. This Licence
does not define the extent of modification or dependence on the Original
Work required in order to classify a work as a Derivative Work; this extent
is determined by copyright law applicable in the country mentioned in
Article 15.
- 'The Work': the Original Work or its Derivative Works.
- 'The Source Code': the human-readable form of the Work which is the most
convenient for people to study and modify.
- 'The Executable Code': any code which has generally been compiled and which
is meant to be interpreted by a computer as a program.
- 'The Licensor': the natural or legal person that distributes or communicates
the Work under the Licence.
- 'Contributor(s)': any natural or legal person who modifies the Work under
the Licence, or otherwise contributes to the creation of a Derivative Work.
- 'The Licensee' or 'You': any natural or legal person who makes any usage of
the Work under the terms of the Licence.
- 'Distribution' or 'Communication': any act of selling, giving, lending,
renting, distributing, communicating, transmitting, or otherwise making
available, online or offline, copies of the Work or providing access to its
essential functionalities at the disposal of any other natural or legal
person.
2. Scope of the rights granted by the Licence
The Licensor hereby grants You a worldwide, royalty-free, non-exclusive,
sublicensable licence to do the following, for the duration of copyright
vested in the Original Work:
- use the Work in any circumstance and for all usage,
- reproduce the Work,
- modify the Work, and make Derivative Works based upon the Work,
- communicate to the public, including the right to make available or display
the Work or copies thereof to the public and perform publicly, as the case
may be, the Work,
- distribute the Work or copies thereof,
- lend and rent the Work or copies thereof,
- sublicense rights in the Work or copies thereof.
Those rights can be exercised on any media, supports and formats, whether now
known or later invented, as far as the applicable law permits so.
In the countries where moral rights apply, the Licensor waives his right to
exercise his moral right to the extent allowed by law in order to make
effective the licence of the economic rights here above listed.
The Licensor grants to the Licensee royalty-free, non-exclusive usage rights
to any patents held by the Licensor, to the extent necessary to make use of
the rights granted on the Work under this Licence.
3. Communication of the Source Code
The Licensor may provide the Work either in its Source Code form, or as
Executable Code. If the Work is provided as Executable Code, the Licensor
provides in addition a machine-readable copy of the Source Code of the Work
along with each copy of the Work that the Licensor distributes or indicates,
in a notice following the copyright notice attached to the Work, a repository
where the Source Code is easily and freely accessible for as long as the
Licensor continues to distribute or communicate the Work.
4. Limitations on copyright
Nothing in this Licence is intended to deprive the Licensee of the benefits
from any exception or limitation to the exclusive rights of the rights owners
in the Work, of the exhaustion of those rights or of other applicable
limitations thereto.
5. Obligations of the Licensee
The grant of the rights mentioned above is subject to some restrictions and
obligations imposed on the Licensee. Those obligations are the following:
Attribution right: The Licensee shall keep intact all copyright, patent or
trademarks notices and all notices that refer to the Licence and to the
disclaimer of warranties. The Licensee must include a copy of such notices
and a copy of the Licence with every copy of the Work he/she distributes or
communicates. The Licensee must cause any Derivative Work to carry prominent
notices stating that the Work has been modified and the date of modification.
Copyleft clause: If the Licensee distributes or communicates copies of the
Original Works or Derivative Works, this Distribution or Communication will
be done under the terms of this Licence or of a later version of this Licence
unless the Original Work is expressly distributed only under this version of
the Licence — for example by communicating 'EUPL v. 1.2 only'. The Licensee
(becoming Licensor) cannot offer or impose any additional terms or conditions
on the Work or Derivative Work that alter or restrict the terms of the
Licence.
Compatibility clause: If the Licensee Distributes or Communicates Derivative
Works or copies thereof based upon both the Work and another work licensed
under a Compatible Licence, this Distribution or Communication can be done
under the terms of this Compatible Licence. For the sake of this clause,
'Compatible Licence' refers to the licences listed in the appendix attached
to this Licence. Should the Licensee's obligations under the Compatible
Licence conflict with his/her obligations under this Licence, the obligations
of the Compatible Licence shall prevail.
Provision of Source Code: When distributing or communicating copies of the
Work, the Licensee will provide a machine-readable copy of the Source Code or
indicate a repository where this Source will be easily and freely available
for as long as the Licensee continues to distribute or communicate the Work.
Legal Protection: This Licence does not grant permission to use the trade
names, trademarks, service marks, or names of the Licensor, except as
required for reasonable and customary use in describing the origin of the
Work and reproducing the content of the copyright notice.
6. Chain of Authorship
The original Licensor warrants that the copyright in the Original Work
granted hereunder is owned by him/her or licensed to him/her and that he/she
has the power and authority to grant the Licence.
Each Contributor warrants that the copyright in the modifications he/she
brings to the Work are owned by him/her or licensed to him/her and that
he/she has the power and authority to grant the Licence.
Each time You accept the Licence, the original Licensor and subsequent
Contributors grant You a licence to their contributions to the Work, under
the terms of this Licence.
7. Disclaimer of Warranty
The Work is a work in progress, which is continuously improved by numerous
Contributors. It is not a finished work and may therefore contain defects or
'bugs' inherent to this type of development.
For the above reason, the Work is provided under the Licence on an 'as is'
basis and without warranties of any kind concerning the Work, including
without limitation merchantability, fitness for a particular purpose, absence
of defects or errors, accuracy, non-infringement of intellectual property
rights other than copyright as stated in Article 6 of this Licence.
This disclaimer of warranty is an essential part of the Licence and a
condition for the grant of any rights to the Work.
8. Disclaimer of Liability
Except in the cases of wilful misconduct or damages directly caused to
natural persons, the Licensor will in no event be liable for any direct or
indirect, material or moral, damages of any kind, arising out of the Licence
or of the use of the Work, including without limitation, damages for loss of
goodwill, work stoppage, computer failure or malfunction, loss of data or any
commercial damage, even if the Licensor has been advised of the possibility
of such damage. However, the Licensor will be liable under statutory product
liability laws as far such laws apply to the Work.
9. Additional agreements
While distributing the Work, You may choose to conclude an additional
agreement, defining obligations or services consistent with this Licence.
However, if accepting obligations, You may act only on your own behalf and on
your sole responsibility, not on behalf of the original Licensor or any other
Contributor, and only if You agree to indemnify, defend, and hold each
Contributor harmless for any liability incurred by, or claims asserted
against such Contributor by the fact You have accepted any warranty or
additional liability.
10. Acceptance of the Licence
The provisions of this Licence can be accepted by clicking on an icon 'I
agree' placed under the bottom of a window displaying the text of this
Licence or by affirming consent in any other similar way, in accordance with
the rules of applicable law. Clicking on that icon indicates your clear and
irrevocable acceptance of this Licence and all of its terms and conditions.
Similarly, you irrevocably accept this Licence and all of its terms and
conditions by exercising any rights granted to You by Article 2 of this
Licence, such as the use of the Work, the creation by You of a Derivative
Work or the Distribution or Communication by You of the Work or copies
thereof.
11. Information to the public
In case of any Distribution or Communication of the Work by means of
electronic communication by You (for example, by offering to download the
Work from a remote location) the distribution channel or media (for example,
a website) must at least provide to the public the information requested by
the applicable law regarding the Licensor, the Licence and the way it may be
accessible, concluded, stored and reproduced by the Licensee.
12. Termination of the Licence
The Licence and the rights granted hereunder will terminate automatically
upon any breach by the Licensee of the terms of the Licence.
Such a termination will not terminate the licences of any person who has
received the Work from the Licensee under the Licence, provided such persons
remain in full compliance with the Licence.
13. Miscellaneous
Without prejudice of Article 9 above, the Licence represents the complete
agreement between the Parties as to the Work.
If any provision of the Licence is invalid or unenforceable under applicable
law, this will not affect the validity or enforceability of the Licence as a
whole. Such provision will be construed or reformed so as necessary to make
it valid and enforceable.
The European Commission may publish other linguistic versions or new versions
of this Licence or updated versions of the Appendix, so far this is required
and reasonable, without reducing the scope of the rights granted by the
Licence. New versions of the Licence will be published with a unique version
number.
All linguistic versions of this Licence, approved by the European Commission,
have identical value. Parties can take advantage of the linguistic version of
their choice.
14. Jurisdiction
Without prejudice to specific agreement between parties,
- any litigation resulting from the interpretation of this License, arising
between the European Union institutions, bodies, offices or agencies, as a
Licensor, and any Licensee, will be subject to the jurisdiction of the
Court of Justice of the European Union, as laid down in article 272 of the
Treaty on the Functioning of the European Union,
- any litigation arising between other parties and resulting from the
interpretation of this License, will be subject to the exclusive
jurisdiction of the competent court where the Licensor resides or conducts
its primary business.
15. Applicable Law
Without prejudice to specific agreement between parties,
- this Licence shall be governed by the law of the European Union Member
State where the Licensor has his seat, resides or has his registered
office,
- this licence shall be governed by Belgian law if the Licensor has no seat,
residence or registered office inside a European Union Member State.
Appendix
'Compatible Licences' according to Article 5 EUPL are:
- GNU General Public License (GPL) v. 2, v. 3
- GNU Affero General Public License (AGPL) v. 3
- Open Software License (OSL) v. 2.1, v. 3.0
- Eclipse Public License (EPL) v. 1.0
- CeCILL v. 2.0, v. 2.1
- Mozilla Public Licence (MPL) v. 2
- GNU Lesser General Public Licence (LGPL) v. 2.1, v. 3
- Creative Commons Attribution-ShareAlike v. 3.0 Unported (CC BY-SA 3.0) for
works other than software
- European Union Public Licence (EUPL) v. 1.1, v. 1.2
- Québec Free and Open-Source Licence — Reciprocity (LiLiQ-R) or Strong
Reciprocity (LiLiQ-R+).
The European Commission may update this Appendix to later versions of the
above licences without producing a new version of the EUPL, as long as they
provide the rights granted in Article 2 of this Licence and protect the
covered Source Code from exclusive appropriation.
All other changes or additions to this Appendix require the production of a
new EUPL version.

225
README.md
View File

@@ -1,173 +1,122 @@
# Documentation
# testium
[See here](doc/manual/testium_manual.pdf).
testium is a YAML-driven test sequencer for hardware-in-the-loop and
integration testing. A test campaign is described in a `.tum` file as a tree
of items (checks, console interactions, Python/Lua functions, parallel blocks,
dialogs, …); testium executes the tree, captures results, and produces
reports in several formats.
# run testium
## Documentation
From the root path, on windows `cmd`:
* [Quick start](doc/quick_start.md) — install and run your first test in
five minutes.
* [Tutorial](doc/tutorial.md) — guided walk-through of the most common
test items with a runnable example.
* [User manual (PDF)](doc/manual/testium_manual.pdf) — full reference.
* [`doc/examples/`](doc/examples/) — runnable `.tum` snippets.
run.bat
## Pre-built releases
On windows powershell:
Pre-built artifacts are published at
<https://git.beafrancois.fr/v-and-v/testium/releases>:
run.ps1
* **Python wheel** (`testium-<version>-py3-none-any.whl`) — install with
`pip install testium-*.whl`. Lighter than the binary; pulls Python
dependencies from PyPI on install.
* **Self-contained Linux binary** (`testium`, built with PyInstaller) —
runnable directly, no Python installation required on the host. Lua
support still needs a system `lua` interpreter and the `lua-socket` /
`lua-cjson` modules.
* **Flatpak bundle** (`testium.flatpak`) — install with:
On linux:
```sh
# Add Flathub (once, to fetch the KDE/PySide runtimes)
flatpak remote-add --user --if-not-exists flathub https://flathub.org/repo/flathub.flatpakrepo
./run.sh
# Install the bundle
flatpak install --user testium.flatpak
```
The virtual environment is created if needed and *testium* is started.
After installation testium appears in the desktop application menu and the
`testium` command is available in the terminal (requires `~/.local/bin` in
`PATH`, which most modern distributions provide by default).
# Manual setup
## Quick start
A python virtual environment should be created:
From a checkout of the repository:
python3 -m venv <testium_venv>
| OS | Command |
|----|---------|
| Linux | `./run.sh` |
| Windows (cmd) | `run.bat` |
| Windows (PowerShell) | `run.ps1` |
## Requirements
The wrapper creates a Python virtual environment on first run and starts
testium in GUI mode. Add `-b path/to/test.tum` to run a test in batch mode.
In the virtual environment, the following modules must be installed:
## Manual installation
* pyside6
* pyserial
* pyyaml
* pexpect
* gitpython
* jinja2
* colorama
* matplotlib
* junit-xml
* lxml
If the wrapper script does not fit your environment, set up testium manually:
A `requirements.txt` file is also available in the git repository in the path `testium/src/`.
## run testium
from the testium path, execute
python3 -m src/testium
# Doc generation
## Install sphinx
pip install sphinx
## Generate the doc
Execute
doc/manual/sphinx/./build_doc.sh
This command works if texlive package has been installed on the system. It can be done by invoking the following command.
sudo apt install texlive-full
# QT GUI
## QT GUI modification
Open the ".ui" file with `qtcreator` and modify the gui. Then regenerate the python code.
On linux, a helper script has been created:
scripts/./qt_generate.sh
# Debugging
In order to debug testium or your python script executed within testium.
## In VSCODE
This is the prefered method :
1. Create a debug configuration like the following:
```
"configurations": [
{
"name": "Python : testium",
"type": "python",
"request": "launch",
"program": "${workspaceFolder}/src/testium",
"console": "integratedTerminal",
"args": ["-g"],
"justMyCode": true
},
]
```sh
python3 -m venv .venv
source .venv/bin/activate
pip install -r src/requirements.txt
```
2. Install debugpy module in python
Required Python packages (see `src/requirements.txt`):
`pyside6`, `pyserial`, `pyyaml`, `pexpect`, `gitpython`, `jinja2`, `colorama`,
`matplotlib`, `junit-xml`, `lxml`.
python -m pip install debugpy
3. Then get to the "RUN AND DEBUG" tab and press the play button.
4. A testium window will pops up ; start execution of your tum.
5. Do not forget to put breakpoints where you want to investigate.
For tests using `lua_func` items, install Lua (>= 5.1) plus the `socket` and
`cjson` modules. On Debian/Ubuntu:
## Icons
```sh
sudo apt install lua5.4 lua-socket lua-cjson
```
Icons are coming from the following site: https://github.com/free-icons/free-icons.git
Run testium:
# testium Release
```sh
python3 src/testium # GUI
python3 src/testium -b mytest.tum # batch
```
## Pre-requisite
## Troubleshooting
A `python` virtual environment must have been set as described above.
### `wl_proxy_marshal_flags` symbol error
### Install pyinstaller
```
testium: symbol lookup error: ... undefined symbol: wl_proxy_marshal_flags
```
Install `pyinstaller` package using pip.
Force the X11 Qt backend:
## Generate the binary package
```sh
export QT_QPA_PLATFORM=xcb
testium
```
The procedure for a binary release is as follows:
### `xcb plugin missing`
1. update the `release_note.txt` file
2. modify the version in `src/VERSION` file
3. be sure that the documentation is up to date, and if not execute `doc/manual/sphinx/build_doc.sh` script
4. push modifications and create a tag with the new version on the git repository
5. generate an executable file by calling `package/pyinstaller/./build.sh`
6. run the complete validation test for each generated binary
7. check that all the validation results are OK
```
qt.qpa.plugin: Could not load the Qt platform plugin "xcb"
```
# Troubleshooting
Install the missing system libraries:
## The testium exe crashes `wl_proxy_marshal_flags`
```sh
sudo apt install libxcb-cursor0 libicu-dev libxcb-cursor-dev
```
### Error message
## License
/testium: symbol lookup error: /tmp/_MEIOhDCPF/libQt6WaylandClient.so.6: undefined symbol: wl_proxy_marshal_flags
Copyright © 2025-2026 François Dausseur.
### Solution
testium is distributed under the **European Union Public Licence v. 1.2
(EUPL-1.2)** — see [`LICENSE`](LICENSE) for the full text. SPDX:
`EUPL-1.2`.
Set the appropriate environment variable
export QT_QPA_PLATFORM=xcb
testium
## xcb plugin missing
### Error message
qt.qpa.plugin: Could not load the Qt platform plugin "xcb" in "" even though it was found.
### Solution
A package is missing
sudo apt install libxcb-cursor0
sudo apt-get install libicu-dev
sudo apt-get install libxcb-cursor-dev
## The testium appimage crashes when opening a file
This is usually because wayland is defined as the default X server.
To change it :
* Disable Wayland by uncommenting WaylandEnable=false in the `/etc/gdm3/daemon.conf`
* Add `QT_QPA_PLATFORM=xcb` in `/etc/environment`
* After a reboot, check that the environment variable value returns `x11`:
$ echo $XDG_SESSION_TYPE
x11
Contributions are accepted under the same licence (inbound = outbound).
See [`CONTRIBUTING.md`](CONTRIBUTING.md) for development setup, debugging
workflow, and the release procedure.

164
build_all.sh Executable file
View File

@@ -0,0 +1,164 @@
#!/bin/bash
# Build every distribution channel of testium, in order:
# 1. Manual PDF -> dist/testium-manual-<v>.pdf
# 2. Wheel -> dist/testium-<v>-py3-none-any.whl (PEP 427 name)
# 3. PyInstaller binary -> dist/testium-<v>
# 4. Flatpak bundle -> dist/testium-<v>.flatpak
# 5. AppImage -> dist/Testium-<v>-x86_64.AppImage (original name)
# release_note.txt is copied to dist/ up front (with a warning if it has no
# entry for the current version).
#
# By default, a step is skipped if its artifact already exists in dist/.
# Pass --clean to remove existing dist/ artifacts and rebuild everything.
#
# All artifacts are collected (copied) under <repo>/dist/. Original outputs in
# src/dist/, package/*/dist/, doc/manual/ are left in place. Wheel and AppImage
# keep their original names (which already contain the version); manual,
# pyinstaller and flatpak are renamed to testium(-manual)-<version>(.suff).
#
# Re-uses scripts/build_env.sh and scripts/set_env.sh — the same pair invoked
# by run.sh — so the venv at test/tmp/.venv stays the single source of Python
# dependencies. `build`, `pyinstaller`, `sphinx` and `linuxdoc` are installed
# into that venv on demand if not already there. Flatpak and AppImage build in
# their own container/sandbox; their build.sh scripts have their own toolchain
# checks.
set -e
CLEAN=0
for arg in "$@"; do
case "$arg" in
--clean|-c) CLEAN=1 ;;
*) echo "Unknown option: $arg" >&2; exit 1 ;;
esac
done
SCRIPT_DIR=$(realpath "$(dirname "$0")")
VERSION=$(cat "$SCRIPT_DIR/src/VERSION")
DIST_DIR="$SCRIPT_DIR/dist"
mkdir -p "$DIST_DIR"
if [ "$CLEAN" -eq 1 ]; then
echo "-- clean: removing existing dist artifacts for version $VERSION"
rm -f "$DIST_DIR/testium-manual-${VERSION}.pdf"
rm -f "$DIST_DIR"/testium-${VERSION}-*.whl
rm -f "$DIST_DIR/testium-${VERSION}"
rm -f "$DIST_DIR/testium-${VERSION}.flatpak"
rm -f "$DIST_DIR"/Testium-${VERSION}-*.AppImage
fi
# Release note: copy it to dist/ and warn (but don't fail) if it has no entry
# for the current version.
RELEASE_NOTE_SRC="$SCRIPT_DIR/release_note.txt"
RELEASE_NOTE="$DIST_DIR/release_note.txt"
cp -f "$RELEASE_NOTE_SRC" "$RELEASE_NOTE"
if ! grep -qE "^version $VERSION([^.0-9]|$)" "$RELEASE_NOTE_SRC"; then
echo "WARNING: release_note.txt has no entry for version $VERSION." >&2
fi
export PY_VENV_NAME=".venv"
export PY_VENV_DIR="$SCRIPT_DIR/test/tmp/$PY_VENV_NAME"
export REQ_PATH="$SCRIPT_DIR/src/requirements.txt"
bash "$SCRIPT_DIR/scripts/build_env.sh"
source "$SCRIPT_DIR/scripts/set_env.sh"
step() {
echo
echo "================================================================"
echo " $1"
echo "================================================================"
}
skip() { echo " (already built — skipping)"; }
# 1. Manual PDF
MANUAL="$DIST_DIR/testium-manual-${VERSION}.pdf"
step "1/5 Manual PDF (version $VERSION)"
if [ ! -f "$MANUAL" ]; then
python -m pip install --quiet --upgrade sphinx linuxdoc
bash "$SCRIPT_DIR/doc/manual/sphinx/build_doc.sh"
cp -f "$SCRIPT_DIR/doc/manual/testium_manual.pdf" "$MANUAL"
else
skip
fi
# 2. Wheel — PEP 427 name kept (already contains version)
step "2/5 Wheel (version $VERSION)"
WHEEL=$(ls -1t "$DIST_DIR"/testium-${VERSION}-*.whl 2>/dev/null | head -1)
if [ -z "$WHEEL" ]; then
python -m pip install --quiet --upgrade build
(
cd "$SCRIPT_DIR/src"
rm -rf dist build *.egg-info
python -m build --wheel
)
WHEEL_SRC=$(ls -1t "$SCRIPT_DIR/src/dist"/*.whl | head -1)
WHEEL="$DIST_DIR/$(basename "$WHEEL_SRC")"
cp -f "$WHEEL_SRC" "$WHEEL"
else
skip
fi
# 3. PyInstaller binary
PYI_BIN="$DIST_DIR/testium-${VERSION}"
step "3/5 PyInstaller binary (version $VERSION)"
if [ ! -f "$PYI_BIN" ]; then
python -m pip install --quiet --upgrade pyinstaller
bash "$SCRIPT_DIR/package/pyinstaller/build.sh"
cp -f "$SCRIPT_DIR/package/pyinstaller/dist/testium" "$PYI_BIN"
else
skip
fi
# 4. Flatpak bundle
FLATPAK_BUNDLE="$DIST_DIR/testium-${VERSION}.flatpak"
step "4/5 Flatpak bundle (version $VERSION)"
if [ ! -f "$FLATPAK_BUNDLE" ]; then
FLATPAK_DEPS=(
"org.kde.Platform//6.10"
"org.kde.Sdk//6.10"
"io.qt.PySide.BaseApp//6.10"
)
if ! flatpak remotes --user | grep -q "^flathub"; then
echo " Adding Flathub remote"
flatpak remote-add --user --if-not-exists flathub https://dl.flathub.org/repo/flathub.flatpakrepo
fi
for dep in "${FLATPAK_DEPS[@]}"; do
if ! flatpak info --user "$dep" &>/dev/null && ! flatpak info --system "$dep" &>/dev/null; then
echo " Installing Flatpak dependency: $dep"
flatpak install --user --noninteractive flathub "$dep"
fi
done
(
cd "$SCRIPT_DIR/package/flatpak"
bash build.sh
)
cp -f "$SCRIPT_DIR/package/flatpak/testium.flatpak" "$FLATPAK_BUNDLE"
else
skip
fi
# 5. AppImage
step "5/5 AppImage (version $VERSION)"
APPIMAGE=$(ls -1t "$DIST_DIR"/Testium-${VERSION}-*.AppImage 2>/dev/null | head -1)
if [ -z "$APPIMAGE" ]; then
(
cd "$SCRIPT_DIR/package/appimage"
bash build.sh
)
APPIMAGE_SRC=$(ls -1t "$SCRIPT_DIR/package/appimage"/*.AppImage 2>/dev/null | head -1)
APPIMAGE="$DIST_DIR/$(basename "$APPIMAGE_SRC")"
cp -f "$APPIMAGE_SRC" "$APPIMAGE"
chmod +x "$APPIMAGE"
else
skip
fi
step "All packages built"
printf " manual : %s\n" "$MANUAL"
printf " wheel : %s\n" "$WHEEL"
printf " pyinstaller : %s\n" "$PYI_BIN"
printf " flatpak : %s\n" "$FLATPAK_BUNDLE"
printf " appimage : %s\n" "$APPIMAGE"
printf " release_note : %s\n" "$RELEASE_NOTE"

View File

@@ -1,4 +1,4 @@
- unittest_file:
- unittest:
name: Test 5
test_file: dummy.py

View File

@@ -3,7 +3,7 @@ sequence: &endurance_test
!include endurance.tum
sequence:
- unittest_file:
- unittest:
name: Test 3
test_file: dummy.py
test_method: test_01_pass
@@ -11,6 +11,6 @@ sequence:
iterator: 10
steps:
*endurance_test
- unittest_file:
- unittest:
name: Test 4
test_file: dummy.py

View File

@@ -15,7 +15,7 @@ main:
- $(reference_1)
- $(reference_2)
report_show_success: true
- unittest_file:
- unittest:
name: Test 1
test_file: dummy.py
doc: |
@@ -23,7 +23,7 @@ main:
Voilà...
- sleep:
{name: Sleep between one and two, timeout: 10, dialog: true}
- unittest_file:
- unittest:
{name: Test 2, test_file: dummy.py,execute_on_stop: true}
- loop:
name: Cycle Temperature

View File

@@ -1,5 +1,5 @@
import libs.testium as tm
import py_func.tm as tm
def post_exec():
print('Success !!!!')

View File

@@ -14,7 +14,7 @@ main:
key: report-key-2
stop_on_failure: True
steps:
- unittest_file:
- unittest:
name: unittest item
doc: |
The purpose of this unittest test item is to demonstrate
@@ -41,7 +41,7 @@ main:
param:
- 123
- unittest_file:
- unittest:
name: Unittest item
test_file: dummy/dummy.py
test_method:
@@ -98,7 +98,7 @@ main:
name: Infine loop unittest step crashes
stop_on_failure: True
steps:
- unittest_file:
- unittest:
name: Unittest item
test_file: dummy/dummy.py
test_method:
@@ -243,7 +243,7 @@ main:
name: Infinite loop
skipped: True
steps:
- unittest_file:
- unittest:
name: Unittest item
test_file: dummy/dummy.py
test_method: test_01_pass

View File

@@ -9,7 +9,7 @@ main:
name: Test Sample number one
version: 0.1
steps:
- unittest_file:
- unittest:
name: Unittest item
test_file: dummy/unittest_str.py
doc: Unittest test
@@ -88,7 +88,7 @@ main:
name: cycle item
iterator : 3
steps:
- unittest_file:
- unittest:
name: Unittest item
test_file: dummy/dummy.py
test_method: test_01_pass
@@ -99,7 +99,7 @@ main:
name: cycle item
iterator : 3
steps:
- unittest_file:
- unittest:
name: Unittest item
test_file: dummy/dummy.py
test_method: test_01_pass

View File

@@ -20,6 +20,22 @@ main:
param:
- 123
- py_func:
name: python long wait
doc: The purpose of this step is to try the tasks "stop" interruption
file: utils.py
func_name: long_wait
param:
- 10
- lua_func:
name: lua long wait
doc: The purpose of this step is to try the tasks "stop" interruption
file: lua_func.lua
func_name: long_wait
param:
- 10
- sleep:
name: sleep item
dialog: true

View File

@@ -1,4 +1,5 @@
tm = require("tm")
socket = require("socket")
local module = {}
@@ -7,4 +8,8 @@ function module.func_to_be_executed(param)
return param
end
function module.long_wait(sec)
socket.sleep(sec)
end
return module

View File

@@ -17,18 +17,3 @@ plot_log_path: /tmp/testium_plot/$(testrun_date)/$(testrun_time)/
python_path_Windows: C:\Users\François\Applications\Python313\python.exe
python_path_Linux: $(home)/tmp/tum_venv/bin/python3
# lua_bin_Windows: C:\Lua\5.1
# lua_bin_Linux: /usr/bin/lua
LUA_PATH_Linux: /usr/share/lua/5.4/?.lua;/usr/local/share/lua/5.4/?.lua;/usr/local/share/lua/5.4/?/init.lua;/usr/share/lua/5.4/?/init.lua;/usr/local/lib/lua/5.4/?.lua;/usr/local/lib/lua/5.4/?/init.lua;/usr/lib/lua/5.4/?.lua;/usr/lib/lua/5.4/?/init.lua;./?.lua;./?/init.lua;/home/francois/.luarocks/share/lua/5.4/?.lua;/home/francois/.luarocks/share/lua/5.4/?/init.lua
LUA_CPATH_Linux: /usr/local/lib/lua/5.4/?.so;/usr/lib/lua/5.4/?.so;/usr/local/lib/lua/5.4/loadall.so;/usr/lib/lua/5.4/loadall.so;./?.so;/home/francois/.luarocks/lib/lua/5.4/?.so
PATH_Linux:
LUA_PATH_Windows: ;.\?.lua;C:\Lua\5.1\lua\?.lua;C:\Lua\5.1\lua\?\init.lua;C:\Lua\5.1\?.lua;C:\Lua\5.1\?\init.lua;C:\Lua\5.1\lua\?.luac
LUA_CPATH_Windows: .\?.dll;C:\Lua\5.1\?.dll;C:\Lua\5.1\loadall.dll;C:\Lua\5.1\clibs\?.dll;C:\Lua\5.1\clibs\loadall.dll;.\?51.dll;C:\Lua\5.1\?51.dll;C:\Lua\5.1\clibs\?51.dll
PATH_Windows: ""
lua_env:
PATH: $(PATH_$(os))
LUA_PATH: $(LUA_PATH_$(os))
LUA_CPATH: $(LUA_CPATH_$(os))

View File

@@ -1,3 +1,5 @@
from time import sleep
def dummy_exit(useless1, useless2):
return True
@@ -11,3 +13,6 @@ def funcToBeExecuted (bla):
def funcToBeExecuted2 (bla):
print(bla)
return blo
def long_wait (sec):
sleep(sec)

View File

@@ -3,7 +3,7 @@ Command Line Interface
.. code-block:: text
usage: testium.pyw [-h] [--version] [-b] [-m] [-c CONFIG_FILE [CONFIG_FILE ...]] [-r] [-l LOG_FILE]
usage: testium.pyw [-h] [--version] [-b] [-c CONFIG_FILE [CONFIG_FILE ...]] [-r] [-l LOG_FILE]
[-d DEFINE [DEFINE ...]] [-p REPORT_FILE] [-t {sqlite,json,junit,html,text}]
[-n REPORT_PATTERN [REPORT_PATTERN ...]] [-i INCLUDE_PATH [INCLUDE_PATH ...]] [-o] [-g]
[test_file]
@@ -16,9 +16,8 @@ Command Line Interface
--version Returns the version of testium
-b, --batch-execution
Executes the test in batch mode
-m, --terminal Starts terminal mode
-c CONFIG_FILE [CONFIG_FILE ...], --config-file CONFIG_FILE [CONFIG_FILE ...]
-o, --no-color Deactivates stdout colors in batch and terminal mode
-o, --no-color Deactivates stdout colors in batch mode
Configuration file
-r, --run-and-close Runs the test then closes the application
-l LOG_FILE, --log-file LOG_FILE
@@ -45,17 +44,10 @@ Returns what's in the previous section.
Executes the test in text mode. No need to have QT installed in that case.
``-m, --terminal``
------------------
Starts a testium interactive console. It allows to run commands and sub-tests manually
in a console.
``-o, --no-color``
------------------
Switch allowing to disable the colored output in terminal or batch modes.
Switch allowing to disable the colored output in batch mode.
``-c, --config-file``
---------------------

View File

@@ -14,6 +14,7 @@ import os
import sys
sys.path.insert(0, os.path.abspath('../../../../src/testium/'))
sys.path.insert(0, os.path.abspath('../../../../src/'))
# -- Project information -----------------------------------------------------

View File

@@ -4,7 +4,12 @@ Python helper library
======================
A python library including helper function for python modules called from
testium.
testium ``py_func`` items.
User scripts run inside the ``py_func`` subprocess and interact with testium
through a JSON-RPC bridge — the ``py_func.tm`` module. They must **not**
import ``api.testium`` or ``interpreter.*`` directly: those are main-process
modules and may not even be reachable in a packaged build (PyInstaller, .deb).
To include the support of this library in a python script, the following
line must be included in the script header:
@@ -18,58 +23,38 @@ line must be included in the script header:
Global variables helper functions
----------------------------------
To manage values in the global variables dataset, the following testium library API
must be used:
To manage values in the global variables dataset:
.. automodule:: interpreter.utils.globdict
.. automodule:: py_func.tm
:members: gd, setgd, delgd
:undoc-members:
:no-index:
Console helper functions
------------------------
Every opened console instance is added to a list with the
key ``console_instances`` of the global variables.
The instance is removed from the list on close step of the ``console`` test item.
To manage consoles from within ``py_func`` python functions,
the following testium library API can be used:
.. automodule:: libs.testium
:members: add_console, remove_console, console
:undoc-members:
:no-index:
Plot helper functions
------------------------
Every opened plot window instance is added to a list with the
key ``plot_instances`` of the global variables.
Add values to a running plot or read the last value from it:
The instance is removed from the list on close step of the ``plot`` test item.
To manage plots from within ``py_func`` python functions,
the following testium library API can be used:
.. automodule:: libs.testium
:members: add_plot, remove_plot, plot, add_plot_values, last_plot_value
.. automodule:: py_func.tm
:members: add_plot_values, last_plot_value
:undoc-members:
:no-index:
Console and plot **lifecycle** management (``add_console``, ``remove_console``,
``console``, ``add_plot``, ``remove_plot``, ``plot``) is performed by the
``console`` and ``plot`` test items themselves — not from user ``py_func``
scripts. Use those test items to open/close consoles and plots.
Other helper functions
------------------------
.. automodule:: libs.testium
:members: OS, get_main_dir, timestamp, timestamp_as_sec
.. automodule:: py_func.tm
:members: OS, get_main_dir, init_timestamp, timestamp, timestamp_as_sec, text_mode
:undoc-members:
:no-index:
Debug mode
------------------------
.. automodule:: libs.testium
:members: debug_enabled, enable_debug, print_debug, print_info, print_warn
:undoc-members:
:no-index:
The ``test_debug`` global variable controls debug-only output. Read or write
it via ``tm.gd("test_debug")`` / ``tm.setgd("test_debug", True)``.

View File

@@ -23,23 +23,3 @@ graphical interface.
:caption: call a test in batch mode
testium -b test/my_test/main.tum
Terminal mode
-------------
The terminal mode starts *testium* in interactive mode. From this console, some tests and
sequences of tests can be called interactively.
.. code-block:: text
:caption: call a test in terminal mode
$ testium -m
Configuration file loaded: /my/execution/path/param.yaml
[...]
================================================================================
====== Test configuration
================================================================================
Test executed with testium : 2.4.0 (binary release)
(testium)~

View File

@@ -75,7 +75,7 @@ a tooltip on the test row.
name: Test example
steps:
- unittest_file:
- unittest:
name: unittest item
doc: |
The purpose of this unittest test item is to demonstrate
@@ -93,4 +93,4 @@ See illustration in :numref:`Figure %s<doc-illustration>`.
Unittest
^^^^^^^^^
For ``unittest_file`` type test items, the python docstring of the test method is used as documentation.
For ``unittest`` type test items, the python docstring of the test method is used as documentation.

View File

@@ -6,7 +6,7 @@ This software is developed in python and it implements the Qt6 graphical framewo
It has been developed since 2013 with production and development testing in mind.
It's function is to automate the execution of tests. It can be invoked either as command line terminal application or as a graphical interface application.
It's function is to automate the execution of tests. It can be invoked either as command line application or as a graphical interface application.
Tests reports generation and customization are also in this tool's scope.

View File

@@ -6,18 +6,25 @@ Reports
If a report is required (in addition to the log), the ``report`` YAML element
must be added at the root of the TUM main test file.
The ``report`` YAML element has the following form:
The ``report`` element accepts a single export or a list of them under the
``export`` key. Each export entry uses the format name as its key:
.. code-block:: yaml
:caption: reports global settings
:caption: reports global settings — multiple exports
report:
enabled: True
file_name: $(test_name).rep
path: $(home)/reports
pattern: "Console%"
export: junit
log_stored: False
log_stored: True
export:
- sqlite:
path: $(home)/reports
file_name: $(test_name).db
- junit:
path: $(home)/reports
file_name: $(test_name).xml
- html:
path: $(home)/reports
file_name: $(test_name).html
.. table:: report attributes
:widths: 20, 30, 50
@@ -27,21 +34,93 @@ The ``report`` YAML element has the following form:
+-----------------+-----------------------+-------------------------------------------+
| ``enabled`` | ``True`` | Report activated |
+-----------------+-----------------------+-------------------------------------------+
| ``file_name`` | / | Report file name |
| ``log_stored`` | ``False`` | When ``True``, captures stdout per test |
| | | item so exports (html, json) can include |
| | | the log of each item. |
+-----------------+-----------------------+-------------------------------------------+
| ``path`` | ``$(report_path)`` | Report storage path By default, it uses |
| | | the default one set in the |
| | | preferences. |
| ``export`` | / | One export entry or a list of them. Each |
| | | entry's key is the format name (see |
| | | below). |
+-----------------+-----------------------+-------------------------------------------+
| ``pattern`` | / | The pattern in SQL wildachars syntax |
| | | to be applied on test names to |
| | | selected reported tests. |
Each export entry supports the following sub-attributes:
.. table:: export attributes
:widths: 20, 30, 50
+-----------------+-----------------------+-------------------------------------------+
| ``export`` | / | The type of export. For exemple junit. |
| | | By default, the sqlite format is |
| | | used to generate reports. |
| Attribute | default value | Description |
+-----------------+-----------------------+-------------------------------------------+
| ``log_stored`` | / | Defines if the output log of each |
| | | test is accessible to generate the |
| | | report export. |
| ``path`` | ``$(report_path)`` | Output directory. |
+-----------------+-----------------------+-------------------------------------------+
| ``file_name`` | / | Output file name. May include |
| | | ``$(...)`` global-dict expansions. |
+-----------------+-----------------------+-------------------------------------------+
| ``pattern`` | / | One or more SQL ``LIKE`` patterns |
| | | applied on the test ``name``. |
+-----------------+-----------------------+-------------------------------------------+
| ``key`` | / | One or more SQL ``LIKE`` patterns |
| | | applied on the test ``key`` |
| | | (the per-item ``key`` attribute). |
+-----------------+-----------------------+-------------------------------------------+
Built-in formats
^^^^^^^^^^^^^^^^
* ``sqlite`` — raw SQLite database (storage layer; selecting it persists the run).
* ``text`` — simple indented text dump of the test tree.
* ``json`` — full report as JSON: ``{"header": {...}, "tests": [...]}``.
* ``junit`` — JUnit XML (requires the ``junit_xml`` Python package).
* ``html`` — single HTML page with header, results table and per-item logs (requires ``lxml``).
If a format is unknown or its optional dependency is missing, the export is
skipped with an ``[report] Export skipped: ...`` info line on stdout — the
test run is **not** interrupted.
.. _sec_reports_plugins:
Custom export formats (plugins)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
A third-party Python package can register additional export formats via the
``testium.exporters`` setuptools entry point group. Once installed in the same
Python environment as testium, the format is auto-detected at startup and can
be referenced from the YAML by its declared name.
Plugin contract — a class with this constructor signature:
.. code-block:: python
:caption: minimal exporter contract
class MyExporter:
def __init__(self, name, con, path, pats, keys, no_header=False):
# name : str — report name
# con : sqlite3.Connection (read) — tables: header, tests
# path : str — output file path (already expansed)
# pats : list[str] — LIKE filters on test_name (may be empty)
# keys : list[str] — LIKE filters on report_key (may be empty)
# no_header : bool — skip header section (set by the inline
# `report` test item)
... # do the work in __init__ and write to `path`
Tables and columns of the SQLite report:
* ``header(key TEXT, value TEXT)`` — keys: ``report_version``, ``test_file``,
``test_name``, ``test_result``, ``test_revision``, ``testium_version``,
``testrun_date``, ``testrun_time``, ``test_duration``.
* ``tests`` — 12 columns: ``timestamp_start``, ``test_id``, ``parent_id``,
``level``, ``test_name``, ``test_type``, ``report_key``, ``result``
(``PASS``/``FAIL``/``SKIP``), ``message``, ``duration`` (ms),
``log`` (captured stdout when ``log_stored: True``), ``data`` (JSON of
values reported via ``self.reportValue(...)``).
Declaration in the plugin's ``pyproject.toml``:
.. code-block:: toml
:caption: registering an exporter via entry-points
[project.entry-points."testium.exporters"]
my_format = "my_pkg:MyExporter"
The plugin is then usable in any ``.tum`` report block as ``my_format:``
no testium configuration change required.

View File

@@ -10,7 +10,7 @@ This element is of the following form:
name: Group Item
condition: <| "$(OS)" == "Linux" |>
steps:
- unittest_file:
- unittest:
test_file: test_prod_alpha_13.py
test_method:
...

View File

@@ -33,7 +33,7 @@ if not provided is given in the table as well.
| | | It depends on the test item to take it |
| | | into account or not. |
| | | For example it makes sense to use it |
| | | for ``unittest_file`` test type |
| | | for ``unittest`` test type |
| | | because it can contain many sub-tests, |
| | | but not for sleep test type. |
| | | In cycles, it means that the child |
@@ -87,6 +87,10 @@ if not provided is given in the table as well.
| | | see :ref:`Expected result<sec_expected_result>` |
| | | for details. |
+-----------------------+-------------------+-------------------------------------------------------+
| ``store_result`` | / | Store the test result in a global variable. |
| | | see :ref:`Store result<sec_store_result>` |
| | | for details. |
+-----------------------+-------------------+-------------------------------------------------------+
last test result
@@ -183,6 +187,61 @@ If the result and the expected_result is equal, the test will be *PASSED* if ``T
The special ``$(result)`` variable is replaced in the ``expected_result`` attribute content with the test result value.
.. _sec_store_result:
Store result
-----------------------------------------------
The ``store_result`` attribute stores the test result into a named global variable,
making it available to subsequent test items via ``$(variable_name)``.
If the test item returns a value (e.g. ``py_func``, ``json_rpc``), that value is stored.
If ``process_result`` is also specified, the stored value is the post-processed result.
If the test item produces no value (result is ``None``), the stored value is the
test status string: ``"PASS"`` or ``"FAIL"``, evaluated after ``expected_result``
but **before** ``no_fail``. This ensures the real outcome is captured even when
``no_fail: True`` would otherwise mask a failure.
.. code-block:: yaml
:caption: Store a function return value
- py_func:
name: Read sensor
func_name: read_temperature
store_result: temperature
- py_func:
name: Check temperature in range
func_name: check_range
param: [$(temperature), 20, 30]
.. code-block:: yaml
:caption: Store a post-processed value
- py_func:
name: Get firmware version string
func_name: get_version
process_result: "'$(result)'.split('.')[0]"
store_result: fw_major
.. code-block:: yaml
:caption: Store the pass/fail status of a test with no return value
- console:
name: Send command
console_name: device
steps:
- writeln: reboot
- read_until: {expected: "ready", timeout: 10}
store_result: reboot_status
- py_func:
name: Use reboot status
func_name: log_status
param: [$(reboot_status)]
Export attribute
-----------------------------------------------

View File

@@ -12,7 +12,7 @@ This element is of the following form:
name: Cycle Temperature
iterator: 10
steps:
- unittest_file:
- unittest:
test_file: test_prod_rio6_8093.py
- py_func:
name: function test item

View File

@@ -39,11 +39,14 @@ The ``lua_func`` test item is of the form:
Beside common test items attributes, lua_func item has specific attribute, some of which being mandatory.
* ``file``: the script file name that contains the function to be executed.
Only python script format is supported.
Only Lua script format is supported.
* ``func_name``: The function name to be executed.
* ``param``: This is a list of parameters that are passed to the function
in the order they are presented in the script. These parameters are not
mandatory and are highly dependent of the function prototype.
* ``context_id``: Optional. When set, all ``lua_func`` items sharing the same
``context_id`` value run inside the same persistent Lua subprocess for the
duration of the test. See :ref:`lua_func context<sec_lua_func_context>` for details.
.. code-block:: yaml
:caption: ``lua_func`` test item example of usage
@@ -56,16 +59,71 @@ Beside common test items attributes, lua_func item has specific attribute, some
- $(my_param)
The result of the function (after eventual post treatment) is stored in the global
variable named ``pfn_<func_name>``
variable named ``lfn_<item_name>``
(See :ref:`global variables<sec_global_variables>` for more detail
on how to access to global variables from test items and scripts).
In the example above, the global variable ``$(lfn_activity)``
would be created at the end of the item execution. It would contain the resulting
value of the funcToBeExecuted python function.
value of the methodName Lua function.
The ``lua_func`` will always result ``PASS``, except if the called function raises
and exception or if the ``expected_result`` attribute is used.
an exception or if the ``expected_result`` attribute is used.
.. _sec_lua_func_context:
Sharing state between ``lua_func`` calls
------------------------------------------
Each ``lua_func`` item without a ``context_id`` runs in a dedicated subprocess that
is started and stopped around the call. Module-level variables are not preserved
between two such items.
Inside a ``lua_func`` script, the ``tm`` module exposes ``tm.setgd`` and ``tm.gd``
to read and write the testium global dictionary of the test process. Values stored
this way are accessible from any subsequent test item without requiring a shared
subprocess.
.. code-block:: lua
:caption: sharing a value via the global dictionary
local tm = require("tm")
local module = {}
function module.produce(val)
tm.setgd("my_shared_value", val)
return val
end
function module.consume()
return tm.gd("my_shared_value")
end
return module
When ``context_id`` is set, all ``lua_func`` items that share the same identifier
reuse the same persistent subprocess. This allows Lua-side state (upvalues, module
cache) to be retained across calls beyond what ``tm.setgd`` persists.
.. code-block:: yaml
:caption: ``lua_func`` items sharing a persistent subprocess
- lua_func:
name: produce value
file: my_script.lua
func_name: produce
context_id: my_context
param:
- hello
- lua_func:
name: consume value
file: my_script.lua
func_name: consume
context_id: my_context
expected_result: hello
The shared subprocess is automatically stopped at the end of the test run.
**Lua Interpreter environment setup**

View File

@@ -0,0 +1,97 @@
.. _sec_parallel_item:
**parallel** test item
============================================================
This element is of the following form:
.. code-block:: yaml
:caption: ``parallel`` test item usage example
- parallel:
name: My parallel block
sync: all
branches:
- name: Branch A
steps:
- py_func:
name: Long operation
file: long_op.py
func_name: do_work
- name: Branch B
wait_for:
condition: <| "$(ready_flag)" == "True" |>
timeout: 30
steps:
- let:
name: Mark done
values:
- branch_b_done: true
The ``parallel`` element runs several sequences of items concurrently. Each
inner sequence is called a *branch* and runs in its own thread. The parent
test item waits for branches to finish according to the ``sync`` policy.
Attributes
--------------------
* ``branches``: required. A list of branches to execute concurrently. Each
branch has a ``name`` and a ``steps`` list (same structure as a ``group``
item). It can also declare a ``wait_for`` precondition (see below).
* ``sync``: optional, defaults to ``all``.
* ``all``: the parallel item completes when *every* branch has finished.
The result is ``PASS`` if no branch returned ``FAIL`` (skipped or
disabled branches are ignored, like in ``group``); otherwise ``FAIL``.
* ``any``: the parallel item completes as soon as the *first* branch
finishes. The remaining branches are stopped (their next test items
are not executed). The result is ``PASS`` if at least one branch
succeeded.
* ``no_fail``: optional. When ``true``, a ``FAIL`` result is forced to
``PASS`` for the parallel item itself (same semantics as for any test
item). Branches keep their own result.
Branch attributes
--------------------
Each entry of ``branches`` is a dict with the following attributes:
* ``name``: required. The branch name. Used in reports and as a prefix
in the live log output (each line printed by the branch is prefixed
with the branch name in square brackets, e.g. ``[Branch A]``, so
concurrent branches stay readable).
* ``steps``: required. The list of test items executed sequentially
inside the branch.
* ``wait_for``: optional. Forces the branch to wait until a condition is
met before running its steps. If the timeout elapses, the branch
returns ``FAIL`` (the steps are not run). Sub-attributes:
* ``condition``: a testium expression evaluated repeatedly (every
100 ms) until it returns ``True``.
* ``timeout``: maximum wait, in seconds. Defaults to 30.
Reporting
--------------------
Each branch produces its own row in the SQLite report (with type
``Parallel branch``), in addition to the parent ``Parallel`` row. The
``log`` column of each row contains only the output emitted from that
branch's thread, so logs are never mixed between concurrent branches.
In the live (terminal / GUI) output, lines emitted from a branch are
prefixed with the branch name in square brackets (e.g. ``[Branch A]``).
The prefix is not stored in the SQLite log column.
Notes
--------------------
* A ``sleep`` item inside a branch is interruptible: if another
``sync: any`` branch wins the race, slow ``sleep`` items are aborted
within ~50 ms.
* A ``py_func`` or ``console`` item inside a branch is **not**
interruptible: a ``sync: any`` stop will only take effect after the
current item returns. The branch will then skip its remaining steps.
* When a user disables a branch in the GUI tree, the branch returns
``SKIP`` instantly without affecting the others (it does *not* win a
``sync: any`` race).

View File

@@ -13,7 +13,7 @@ class ``py_func`` item
This is the normal way of calling some custom python code.
A class must be defined and derived from ``FunctionItem`` from the ``libs.testium`` module.
A class must be defined and derived from ``FunctionItem`` from the ``py_func.tm`` module.
From this class it is possible to define some custom reported values with the following API
@@ -89,6 +89,9 @@ some of which being mandatory.
* ``param``: This is a list of parameters that are passed to the function
in the order they are presented in the script. These parameters are not
mandatory and are highly dependent of the function prototype.
* ``context_id``: Optional. When set, all ``py_func`` items sharing the same
``context_id`` value run inside the same persistent Python subprocess for the
duration of the test. See :ref:`py_func context<sec_py_func_context>` for details.
.. code-block:: yaml
:caption: ``py_func`` test item example of usage
@@ -111,6 +114,67 @@ value of the funcToBeExecuted python function.
The ``py_func`` will always result ``PASS``, except if the called function raises
and exception or if the ``expected_result`` attribute is used.
.. _sec_py_func_context:
Sharing state between ``py_func`` calls
------------------------------------------
Each ``py_func`` item without a ``context_id`` runs in a dedicated subprocess that
is started and stopped around the call. State cannot be shared between two such
items using module-level variables.
Inside a ``py_func`` script, ``tm.setgd`` and ``tm.gd`` read and write the testium
global dictionary. Values stored this way are accessible from any subsequent test
item, including other ``py_func`` items, without requiring a shared subprocess.
.. code-block:: python
:caption: sharing a value via the global dictionary
import py_func.tm as tm
def produce(val):
tm.setgd("my_shared_value", val)
return val
def consume():
return tm.gd("my_shared_value", None)
When ``context_id`` is set, all ``py_func`` items that share the same identifier
reuse the same persistent subprocess. This allows sharing any Python object across
calls — including objects that cannot be transmitted to other processes.
.. code-block:: python
:caption: sharing an object via ``context_id``
import py_func.tm as tm
def open_connection():
tm.setgd("conn", MyConnection())
return "ok"
def use_connection():
conn = tm.gd("conn")
return conn.status()
.. code-block:: yaml
:caption: ``py_func`` items sharing a persistent subprocess
- py_func:
name: open connection
file: my_script.py
func_name: open_connection
context_id: my_context
expected_result: ok
- py_func:
name: use connection
file: my_script.py
func_name: use_connection
context_id: my_context
expected_result: open
The shared subprocess is automatically stopped at the end of the test run.
**Python Interpreter environment setup**
Some global variables have an impact on the ``py_func`` test item behavior:

View File

@@ -1,16 +1,23 @@
**run** test item
============================================================
This test item executes a new instance of testium.
This test item executes a new instance of testium with the specified ``.tum`` file.
* In **batch mode** (``-b``): the sub-instance is started with ``-b``.
* In **GUI mode**: the sub-instance is started with ``-r`` (run and close).
The item result is **PASS** if the sub-instance launched and ran to completion,
regardless of whether the sub-tests passed or failed.
It is **FAIL** if the file could not be found, the sub-instance could not be
launched, or the time window was not reached (see ``start_time`` / ``end_time``).
.. code-block:: yaml
:caption: ``run`` test item usage example
- run:
name: Execute TUM
tum_fime: example_cycle.tum
tum: example_cycle.tum
python_bin: python3
testium_path: /home/francois/projets/testium-new-report/testium.pyw
log_file: $(home)/reports/test.log
report_file: $(home)/reports/test.rep
@@ -19,12 +26,12 @@ Attributes
run test item has the following specific attributes:
* ``tum_fime``: mandatory the path of the file to execute, it can be relative to current execution folder,
* ``param_file`` (optional) the path of the parameter file to use, otherwise default parameter file is used.
* ``python_bin`` (optional) the path of a specific python to run your scripts,
* ``testium_path`` (optional) the path of a specific testium to run your scripts,
* ``log_file`` (optional) the path of log file to register, if not provided a file is created with timestamp at the location of TUM file.
* ``report_file`` (optional), the path of report file to create
* ``start_time`` (optional), start time for the script execution, in HH:MM format.
* ``end_time`` (optional), end time for an execution within a time frame, in HH:MM format.
* ``wait_for_exec`` (optional). True or False, wait to be in the execution window defined by start_time and end_time to run the script.
* ``tum``: mandatory, the path of the file to execute. Can be relative to the current execution folder.
* ``param_file`` (optional): the path of the parameter file to use; otherwise the default parameter file is used.
* ``python_bin`` (optional): the path of a specific Python interpreter to use.
* ``testium_path`` (optional): the path of a specific testium executable to use.
* ``log_file`` (optional): the path of the log file. In GUI mode, if not provided, a file is created with a timestamp next to the ``.tum`` file. Not used in batch mode.
* ``report_file`` (optional): the path of the report file to create.
* ``start_time`` (optional): earliest time to execute the sub-instance, in ``HH:MM`` format.
* ``end_time`` (optional): latest time for execution within a time frame, in ``HH:MM`` format.
* ``wait_for_exec`` (optional): ``true`` to wait until the time window defined by ``start_time`` and ``end_time`` is reached before running. Requires both ``start_time`` and ``end_time``.

View File

@@ -1,15 +1,15 @@
**unittest_file** test item
**unittest** test item
============================================================
unittest_file test item allows the execution of unittest test script which
unittest test item allows the execution of unittest test script which
is part of python standard libraries.
The tum file prototype is as followed:
.. code-block:: yaml
:caption: ``unittest_file`` test item usage example
:caption: ``unittest`` test item usage example
- unittest_file:
- unittest:
name: unitTest test item
test_file: unitTestScript.py
test_method:
@@ -23,7 +23,7 @@ Beside common test items attributes, unittest test item has specific attribute,
* ``test_file``: it is the name (and eventually path) of the unittest file
to be processed.
* ``test_method``: it is an optional unittest_file test sub-item. If one or more
* ``test_method``: it is an optional unittest test sub-item. If one or more
elements are present, the unittest python script file is parsed and only
the corresponding methods are included in the test tree. Otherwise, all
the test methods are included in the test tree.

View File

@@ -255,11 +255,12 @@ step list attributes.
test_items/let_test_item.rst
test_items/loop_test_item.rst
test_items/lua_func_test_item.rst
test_items/parallel_test_item.rst
test_items/plot_test_item.rst
test_items/report_test_item.rst
test_items/run_test_item.rst
test_items/sleep_test_item.rst
test_items/unittest_file_test_item.rst
test_items/unittest_test_item.rst

Binary file not shown.

66
doc/quick_start.md Normal file
View File

@@ -0,0 +1,66 @@
# Quick start
Five minutes from zero to a passing test.
## Install
From a checkout of the repository:
```sh
./run.sh --version # Linux
run.bat # Windows cmd
```
The wrapper creates a Python virtual environment on first run and verifies
testium starts. If you prefer a manual install, see the README.
## Your first test
Create `hello.tum`:
```yaml
main:
name: hello world
steps:
- check:
name: 1 + 1 makes 2
values:
- <| 1 + 1 == 2 |>
```
Run it in batch mode:
```sh
./run.sh -b -- hello.tum
```
You should see something like:
```
-----> step "1 + 1 makes 2" started
Check passed
<----- step "1 + 1 makes 2" finished: PASS
Test run success.
```
Replace `==` with `!=` and re-run — the step now ends with **FAIL** and
the process exits with code 1.
## Open it in the GUI
```sh
./run.sh hello.tum
```
The test tree appears in the left panel; click *Run test* in the toolbar.
Each item turns green or red live as it executes. Use `F1` on a selected
item to open its detail panel.
## Where to go next
* [`doc/tutorial.md`](tutorial.md) — a guided walk-through of the most
common test items (`py_func`, `let`, `group`, `condition`, `report`).
* [`doc/examples/`](examples/) — runnable `.tum` snippets covering one
feature each.
* [`doc/manual/testium_manual.pdf`](manual/testium_manual.pdf) —
full reference manual.

223
doc/tutorial.md Normal file
View File

@@ -0,0 +1,223 @@
# Tutorial — testing a small Python utility
This walk-through builds, step by step, a testium campaign that exercises
a small Python module. Each section adds one feature; you can follow
along by editing a single `.tum` file and re-running it.
If you have not yet run testium, start with [`quick_start.md`](quick_start.md).
## The code under test
Create `calc.py` next to your `.tum` file:
```python
def add(a, b):
return a + b
def divide(a, b):
return a / b
```
## Step 1 — a static check
The simplest item is `check`: it evaluates an expression and the test
passes iff the expression is truthy. Create `tutorial.tum`:
```yaml
main:
name: calc.py campaign
steps:
- check:
name: addition is correct
values:
- <| 2 + 3 == 5 |>
```
The `<| ... |>` markers turn the body into a Python expression evaluated
at run time. Run it:
```sh
./run.sh -b -- tutorial.tum
```
## Step 2 — call your code with `py_func`
`check` only sees Python literals; to exercise `calc.py` we need a
`py_func` item. Replace the step:
```yaml
- py_func:
name: add 2 and 3
file: calc.py
func_name: add
param: [2, 3]
expected_result: 5
```
`expected_result` makes the item PASS only when the function returns
exactly that value.
The result is also stored in the global dict under `pfn_<name>`
(here `pfn_add 2 and 3`).
Anywhere in a `.tum`, `$(key)` is replaced at runtime by the value
stored in the global dict under `key`. A subsequent step can read the
result back with `$(pfn_<name>)`:
```yaml
- check:
name: result was 5
values:
- <| $(pfn_add 2 and 3) == 5 |>
```
## Step 3 — group several checks
Wrap the steps in a `group` to keep them visually together and let
testium report a per-group status:
```yaml
main:
name: calc.py campaign
steps:
- group:
name: add
steps:
- py_func:
name: 2 + 3
file: calc.py
func_name: add
param: [2, 3]
expected_result: 5
- py_func:
name: -1 + 1
file: calc.py
func_name: add
param: [-1, 1]
expected_result: 0
- group:
name: divide
steps:
- py_func:
name: 6 / 2
file: calc.py
func_name: divide
param: [6, 2]
expected_result: 3.0
```
A group fails as soon as one of its steps fails (set
`stop_on_failure: false` to keep going).
## Step 4 — define a variable with `let`
Avoid hard-coding the same number twice with a variable:
```yaml
- let:
name: define numerator
values:
- num: 6
- py_func:
name: divide num by 2
file: calc.py
func_name: divide
param:
- $(num)
- 2
expected_result: 3.0
```
`$(num)` expands to the global dict entry — when the stored value is a
number it is substituted as a number, no need to wrap it in `<| ... |>`.
## Step 5 — conditional execution
Skip a step when a condition is false:
```yaml
- py_func:
name: divide by zero only on linux
condition: <| "$(os)" == "Linux" |>
file: calc.py
func_name: divide
param: [1, 0]
```
Items skipped this way report `SKIP` and do not affect the overall
result.
## Step 6 — generate a report
Add a `report` block at the root of the file:
```yaml
main:
name: calc.py campaign
steps:
# ... your steps here ...
report:
enabled: true
log_stored: true
export:
- junit:
path: ./reports
file_name: calc.xml
- html:
path: ./reports
file_name: calc.html
```
The `path` directory must exist before the test runs — testium does not
create it. Create it once:
```sh
mkdir -p reports
```
Re-run the test — `./reports/calc.xml` (CI-friendly) and
`./reports/calc.html` (human-friendly) are produced. Set
`log_stored: true` to include each item's captured stdout.
## Step 7 — share state between calls
By default each `py_func` runs in its own short-lived subprocess.
To keep state across calls, use `context_id`:
```yaml
- py_func:
name: open
file: calc.py
func_name: open_resource
context_id: my_ctx
- py_func:
name: use
file: calc.py
func_name: use_resource
context_id: my_ctx
```
Both steps share the same persistent Python interpreter, so `calc.py`
can store any object in module-level globals or in `tm.setgd()`.
To share data without `context_id`, write it to the testium global dict
via the JSON-RPC bridge:
```python
import py_func.tm as tm
def producer():
tm.setgd("computed", 42)
def consumer():
return tm.gd("computed")
```
## Where to go next
* [`doc/examples/`](examples/) — one runnable `.tum` per feature
(cycles, dialogs, console, plots, parallel, run-of-tum, …).
* [`doc/manual/testium_manual.pdf`](manual/testium_manual.pdf) — full
reference manual covering every test item, every attribute and the
YAML syntax extensions.

View File

@@ -24,9 +24,8 @@ AppDir:
runtime:
env:
SEQUENCER_REV: '{{APP_VERSION}}'
TESTIUM_VERSION: '{{APP_VERSION}}'
PYTHONPATH: $APPDIR/usr/lib/python3.11/site-packages:$APPDIR/usr/lib/python3.11
QT_QPA_PLATFORM: xcb
path_mappings:
- /usr/share/matplotlib/mpl-data/matplotlibrc:$APPDIR/etc/matplotlibrc
@@ -69,12 +68,13 @@ AppDir:
# Set python 3.11 as default
ln -fs python3.11 $TARGET_APPDIR/usr/bin/python3
# Install pip
if [ ! -f "get-pip.py" ]; then curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py; fi
# Bootstrap pip into the AppDir Python
if [ ! -f "get-pip.py" ]; then curl -sS https://bootstrap.pypa.io/get-pip.py -o get-pip.py; fi
python3.11 get-pip.py --break-system-packages
# Install application dependencies in AppDir
python3.11 -m pip install --break-system-packages --upgrade --isolated --no-input --ignore-installed --prefix=$TARGET_APPDIR/usr -r requirements.txt
python3.11 -m pip install --break-system-packages --upgrade --isolated --no-input --ignore-installed --prefix=$TARGET_APPDIR/usr -r ../../src/requirements.txt
export PIP_CONFIG_FILE=$HOME/.pip/pip.conf
python3.11 -m pip install --break-system-packages --upgrade --isolated --no-input --ignore-installed --prefix=$TARGET_APPDIR/usr ../../src/dist/testium-{{APP_VERSION}}-py3-none-any.whl

View File

@@ -1,12 +1,54 @@
#!/usr/bin/bash
#!/bin/bash
# Build the testium AppImage inside a Debian container (Podman or Docker).
# The resulting .AppImage file is written to this directory.
export APP_VERSION=$(<../../src/VERSION)
set -e
appimage-builder --recipe AppImageBuilder.yml
REPO_ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
APP_VERSION="$(<"$REPO_ROOT/src/VERSION")"
RESULT=$?
if [ -n "$1" ] && [ "$1" = "install" ]; then
if [ $RESULT -eq 0 ]; then
install -v "testium-${APP_VERSION}-x86_64.AppImage" "${HOME}/.local/bin/testium"
fi
if command -v podman &>/dev/null; then
RUNTIME=podman
elif command -v docker &>/dev/null; then
RUNTIME=docker
else
echo "Error: neither podman nor docker found." >&2
exit 1
fi
echo "Using $RUNTIME — building testium $APP_VERSION AppImage..."
# APPIMAGE_EXTRACT_AND_RUN=1 lets appimagetool run without FUSE in the container.
$RUNTIME run --rm \
--privileged \
-e APPIMAGE_EXTRACT_AND_RUN=1 \
-v "$REPO_ROOT:/work" \
-w /work/package/appimage \
debian:bookworm bash -c "
set -e
export DEBIAN_FRONTEND=noninteractive
apt-get update -qq
apt-get install -y -qq \
python3 python3-pip python3-venv python3-build \
dpkg-dev fakeroot squashfs-tools wget curl file binutils \
libglib2.0-0 patchelf zsync > /dev/null
# Build the wheel
cd /work/src
python3 -m build --wheel --outdir dist/ > /dev/null
cd /work/package/appimage
# Install appimage-builder
pip3 install appimage-builder --quiet --break-system-packages
# Run the build
export APP_VERSION=$APP_VERSION
appimage-builder --recipe AppImageBuilder.yml --skip-test
"
APPIMAGE_FILE=$(ls -1t Testium-*-x86_64.AppImage 2>/dev/null | head -1)
echo "Done: ${APPIMAGE_FILE}"
if [ "${1}" = "install" ] && [ -n "${APPIMAGE_FILE}" ]; then
install -v "${APPIMAGE_FILE}" "${HOME}/.local/bin/testium"
fi

View File

@@ -1,3 +0,0 @@
tables
pandas
scapy

141
package/deb/test_distro.sh Executable file
View File

@@ -0,0 +1,141 @@
#!/usr/bin/env bash
# test_distro.sh — verify testium runs on a target Debian/Ubuntu distrib.
#
# Spins up a Docker container of the requested image, checks which expected
# system Python packages are available (apt), installs them, installs the
# testium wheel, and runs a smoke test that exercises batch mode + py_func
# subprocess.
#
# Usage:
# ./test_distro.sh debian:bookworm
# ./test_distro.sh debian:trixie
# ./test_distro.sh ubuntu:24.04
set -euo pipefail
IMAGE="${1:?Usage: $0 <image> e.g. debian:bookworm | debian:trixie | ubuntu:24.04}"
ROOT=$(realpath "$(dirname "$0")/../..")
# Container runtime: prefer docker if available, fall back to podman
if command -v docker >/dev/null 2>&1; then
CTR=docker
elif command -v podman >/dev/null 2>&1; then
CTR=podman
else
echo "ERROR: neither docker nor podman is installed" >&2
exit 1
fi
echo "[host] Using $CTR"
# --- Build the wheel on the host if it does not already exist
WHEEL_DIR="$ROOT/src/dist"
PYTHON_HOST="$ROOT/test/tmp/.venv/bin/python3"
[ -x "$PYTHON_HOST" ] || PYTHON_HOST=python3
if ! ls "$WHEEL_DIR"/testium-*.whl >/dev/null 2>&1; then
echo "[host] Building wheel..."
(cd "$ROOT/src" && "$PYTHON_HOST" -m build --wheel >/dev/null)
fi
WHEEL=$(ls "$WHEEL_DIR"/testium-*.whl | head -1)
WHEEL_NAME=$(basename "$WHEEL")
echo "[host] Using $WHEEL_NAME"
# Expected system Python packages on the target distrib
APT_PACKAGES=(
python3
python3-pip
python3-setuptools
python3-pyside6.qtwidgets
python3-yaml
python3-jinja2
python3-colorama
python3-git
python3-pexpect
python3-matplotlib
python3-lxml
python3-serial
python3-telnetlib3
lua5.4
lua-cjson
lua-socket
git
)
echo "=== Testing on $IMAGE ==="
$CTR run --rm \
-v "$ROOT:/testium:ro" \
-e WHEEL_NAME="$WHEEL_NAME" \
-e PACKAGES="${APT_PACKAGES[*]}" \
"$IMAGE" \
bash -c '
set -e
export DEBIAN_FRONTEND=noninteractive
apt-get update -qq
# 1. Availability check
echo
echo "--- System package availability ---"
AVAILABLE=()
MISSING=()
for pkg in $PACKAGES; do
if apt-cache show "$pkg" >/dev/null 2>&1; then
AVAILABLE+=("$pkg")
echo " OK $pkg"
else
MISSING+=("$pkg")
echo " MISSING $pkg"
fi
done
echo
# 2. Install available packages
echo "--- Installing system packages ---"
apt-get install -qq -y --no-install-recommends "${AVAILABLE[@]}" ca-certificates >/dev/null
# 3. Map missing apt packages to their PyPI equivalents and pip-install
# them as a fallback (kept minimal so the run is still a "system"
# install for the most part)
declare -A PIP_FALLBACK=(
[python3-pyside6.qtwidgets]=pyside6
[python3-telnetlib3]=telnetlib3
)
# junit_xml has no Debian package — install it via pip so the
# validation post_execution.py can import it.
EXTRA_PIP=(junit-xml)
PIP_PKGS=()
for m in "${MISSING[@]}"; do
fallback="${PIP_FALLBACK[$m]:-}"
if [ -n "$fallback" ]; then
PIP_PKGS+=("$fallback")
fi
done
PIP_PKGS+=("${EXTRA_PIP[@]}")
if [ ${#PIP_PKGS[@]} -gt 0 ]; then
echo "--- Installing missing deps via pip: ${PIP_PKGS[*]} ---"
pip install --break-system-packages "${PIP_PKGS[@]}" >/dev/null
fi
# 4. Install testium wheel
echo "--- Installing testium wheel ---"
pip install --break-system-packages --no-deps "/testium/src/dist/$WHEEL_NAME" >/dev/null
# 5. Install the fake_exporter plugin (needed by the report_plugin
# validation test which exercises entry-points discovery).
# Copy it first because /testium is mounted read-only and the
# setuptools backend touches its build dir.
echo "--- Installing testium-fake-exporter (test plugin) ---"
cp -r /testium/test/validation/fake_exporter /tmp/fake_exporter
pip install --break-system-packages /tmp/fake_exporter >/dev/null
# 6. Run the full validation suite. Outputs are streamed live so
# progress is visible — the suite takes a couple of minutes.
# Reports go to /tmp/testium-validation since /testium is RO.
echo "--- Running validation suite ---"
mkdir -p /tmp/testium-validation
cd /testium
testium -b -o \
-d "validation_report_path=/tmp/testium-validation/" \
-- test/validation/main.tum
'
echo "=== $IMAGE: PASS ==="

View File

@@ -5,4 +5,22 @@
# flatpak install flathub org.kde.Sdk//6.10
# flatpak install flathub io.qt.PySide.BaseApp//6.10
flatpak-builder --user --verbose --force-clean --install build org.testium.Testium.yaml
set -e
# Build + install local
flatpak-builder --user --verbose --force-clean --install --repo=repo build org.testium.Testium.yaml
# Génère le bundle distribuable
flatpak build-bundle repo testium.flatpak org.testium.Testium
echo "Bundle généré : $(pwd)/testium.flatpak"
# Crée ~/.local/bin/testium pour pouvoir taper "testium" en console
WRAPPER="$HOME/.local/bin/testium"
mkdir -p "$HOME/.local/bin"
cat > "$WRAPPER" <<'EOF'
#!/bin/sh
exec flatpak run org.testium.Testium "$@"
EOF
chmod +x "$WRAPPER"
echo "Wrapper installé : $WRAPPER"
echo "Assurez-vous que ~/.local/bin est dans votre PATH."

View File

@@ -0,0 +1,7 @@
<?xml version="1.0" encoding="UTF-8"?>
<mime-info xmlns="http://www.freedesktop.org/standards/shared-mime-info">
<mime-type type="application/x-testium">
<comment>Testium test script</comment>
<glob pattern="*.tum"/>
</mime-type>
</mime-info>

View File

@@ -0,0 +1,10 @@
[Desktop Entry]
Name=Testium
GenericName=Test Sequencer
Comment=YAML-based test sequencer and runner
Exec=testium %f
Icon=org.testium.Testium
Type=Application
Categories=Development;
MimeType=application/x-testium;
StartupNotify=true

View File

@@ -13,7 +13,14 @@ finish-args:
- --socket=wayland
- --device=dri
- --share=network
- --filesystem=home # Optionnel : si votre testium doit lire des fichiers utilisateurs
- --filesystem=home
- --filesystem=/tmp
- --filesystem=host-os
# Allow flatpak-spawn --host to launch host binaries (Python, Lua, git…)
# outside the sandbox. Required because the sandbox glibc/ABI is
# incompatible with arbitrary host shared libraries — we route py_func and
# lua_func through the host instead.
- --talk-name=org.freedesktop.Flatpak
build-options:
build-args:
@@ -41,18 +48,41 @@ modules:
sources:
- type: dir
path: ../../src
- type: file
path: org.testium.Testium.desktop
- type: file
path: org.testium.Testium-mime.xml
- type: file
path: ../../package/testium.png
build-commands:
# On installe le code source dans /app/lib/testium
# Code source
- mkdir -p /app/lib
- cp -r . /app/lib/
- cp -r testium /app/lib/
- cp VERSION /app/lib/testium/VERSION
# Création du launcher exécutable
# Launcher exécutable
- mkdir -p /app/bin
- |
cat <<EOF > /app/bin/testium
#!/bin/sh
# On ajoute le code source et l'extension PySide6 au PYTHONPATH
export TESTIUM_VERSION="\$(cat /app/lib/testium/VERSION 2>/dev/null || echo unknown)"
export PYTHONPATH="/app/lib/testium:/usr/lib/sdk/pyside6/lib/python3.13/site-packages:\$PYTHONPATH"
exec python3 /app/lib/testium "\$@"
# Expose host binaries (git, python3, lua, …) for subprocess lookups.
# PATH is appended (not prepended) so the main process keeps the sandbox python3.
export PATH="\$PATH:/run/host/usr/local/bin:/run/host/usr/bin:/run/host/bin"
export GIT_PYTHON_GIT_EXECUTABLE="/run/host/usr/bin/git"
exec /usr/bin/python3 /app/lib/testium "\$@"
EOF
- chmod +x /app/bin/testium
# Icône
- mkdir -p /app/share/icons/hicolor/256x256/apps
- cp testium.png /app/share/icons/hicolor/256x256/apps/org.testium.Testium.png
# Entrée menu
- mkdir -p /app/share/applications
- cp org.testium.Testium.desktop /app/share/applications/
# Type MIME pour .tum
- mkdir -p /app/share/mime/packages
- cp org.testium.Testium-mime.xml /app/share/mime/packages/

View File

@@ -1,23 +1,50 @@
# -*- mode: python ; coding: utf-8 -*-
import os
# junit_xml is imported by post_exec scripts running under the *host* Python,
# not the frozen interpreter — so bundling it via hiddenimports alone is not
# enough. We also drop its source files at the _MEIPASS root so the host
# python3 finds them via the PYTHONPATH that py_process.py sets to
# tstium_path (= _MEIPASS when frozen).
import junit_xml as _junit_xml
JUNIT_XML_DIR = os.path.dirname(_junit_xml.__file__)
a = Analysis(
['../../src/testium/__main__.py'],
pathex=['../../src/testium',
'../../src/testium/main_win/resources'],
binaries=[],
datas=[ ('../../src/VERSION', '.'),
('../../src/lua_func', 'lua_func'),
('../../src/py_func', 'py_func'),
('../../src/lib', 'lib')],
# py_func/ and runtime/ are bundled at the _MEIPASS root because the
# py_func subprocess is launched with the *host* Python (not the
# frozen interpreter): it needs the source files on disk to find them
# via cwd=subproc_path() and `python3 py_func` + `from runtime.*`.
# py_func/, lua_func/ and runtime/ are bundled at the _MEIPASS root
# because the py_func subprocess is launched with the *host* Python
# (not the frozen interpreter): it needs the source files on disk to
# find them via cwd=subproc_path() and `python3 py_func` +
# `from runtime.*`. api/ and interpreter/ are intentionally NOT
# exposed: user py_func scripts must go through py_func.tm
# (JSON-RPC bridge) for any testium API call.
datas=[('../../src/VERSION', '.'),
('../../src/testium/lua_func', 'lua_func'),
('../../src/testium/py_func', 'py_func'),
('../../src/testium/runtime', 'runtime'),
(JUNIT_XML_DIR, 'junit_xml')],
hiddenimports=["git",
"interpreter",
"main_win",
"libs",
"libs.console",
"libs.termconsole",
"libs.console_ssh",
"libs.raw_tcp_console",
"libs.runtime_plot",
"runtime",
"py_func",
"py_func.tm",
"py_func.handle",
"py_func.func_call",
"api",
"api.console",
"api.termconsole",
"api.console_ssh",
"api.raw_tcp_console",
"api.runtime_plot",
"api.testium",
"matplotlib.backends.backend_pdf",
"telnetlib3",
"serial",

View File

@@ -1,3 +1,55 @@
version 0.2
==============
- Test items: each item type now declares its accepted parameters
(``PARAMS = ParamSet(...)``). Typos in a ``.tum`` are surfaced as a
WARN listing the accepted names instead of being silently ignored;
missing required parameters error out at load time with the source
``.tum`` file as context. No change to valid existing tests.
version 0.1.3
==============
- Stop interrupts engaged blocking steps (console, py_func, lua_func,
json_rpc, sleep) within ~200 ms instead of waiting for the step
to finish.
- GUI Start / Stop / Pause flow simplified.
- lua_func: a function returning nil is no longer reported as a failure.
- ``-d python_bin=...`` and the GUI ``python_bin`` preference now reach
the eval subprocess (used to be silently ignored). ``param.yaml`` can
also override ``python_bin`` for py_func / cycle / post_exec.
- Validation suite: ``test/validation/run.sh`` (and ``run.bat``)
runs the suite inside a dedicated venv in the system temp dir.
- build_all.sh: ``release_note.txt`` and the user manual copied into
``dist/``; warning if the file has no entry for the version being built.
- Flatpak: every GUI file/directory dialog (open test, save report, log
path, default report/log dirs, python/lua interpreter pickers) now
bypasses the XDG document portal — the v0.1.2 fix was only on the
"open test" dialog.
- Flatpak: py_func / lua_func / run sub-instance now execute on the host
via flatpak-spawn, lifting the previous glibc/ABI incompatibility that
prevented user-configured host Python or Lua interpreters from being
reached from the sandbox.
- Validation suite: single entry point with ``--mode source|wheel|
pyinstaller|flatpak|appimage`` to validate any packaging channel
against the same item set; reports are stamped per mode.
- GUI: the "Run tum" test item now uses the testium logo.
version 0.1.2
==============
- Flatpak: opening a test from the GUI now correctly finds its companion
files (param.yaml, .py scripts, ...).
version 0.1.1
==============
- New install channels: Flatpak bundle and AppImage. The AppImage runs
on any distribution (built inside a Debian container).
- About dialog: version is now correct in Flatpak and AppImage builds
(used to display "unknown").
- GUI dialogs no longer hang on pure-Wayland sessions.
- Plot "last values" API: more tolerant timeout on loaded machines.
- run item: `testium_path` and `python_bin` parameters removed —
sub-instances are launched in the same packaging mode as the parent.
- License: EUPL-1.2.
version 0.1
==============
- Start of the project

View File

@@ -20,6 +20,12 @@ if [ "$?" -ne 0 ]; then
echo "venv must be installed on the host distribution."
exit -1
fi
# Check if venv is installed
python3 -c "import ensurepip"
if [ "$?" -ne 0 ]; then
echo "ensurepip must be installed on the host distribution."
exit -1
fi
# Install the virtual environment if needed
if [ ! -d "$PY_VENV_DIR" ]; then
@@ -27,4 +33,10 @@ if [ ! -d "$PY_VENV_DIR" ]; then
python3 -m venv "$PY_VENV_DIR"
source "$PY_VENV_DIR/bin/activate"
pip install --extra-index-url https://pypi.python.org/pypi -r $REQ_PATH
# Validation suite plugin used to verify the report-exporter
# entry-points discovery end-to-end.
FAKE_EXPORTER_DIR="$(dirname "$REQ_PATH")/../test/validation/fake_exporter"
if [ -d "$FAKE_EXPORTER_DIR" ]; then
pip install -e "$FAKE_EXPORTER_DIR"
fi
fi

315
src/LICENSE Normal file
View File

@@ -0,0 +1,315 @@
Copyright (c) 2025-2026 François Dausseur
Licensed under the EUPL
EUROPEAN UNION PUBLIC LICENCE v. 1.2
EUPL © the European Union 2007, 2016
This European Union Public Licence (the 'EUPL') applies to the Work (as
defined below) which is provided under the terms of this Licence. Any use of
the Work, other than as authorised under this Licence is prohibited (to the
extent such use is covered by a right of the copyright holder of the Work).
The Work is provided under the terms of this Licence when the Licensor (as
defined below) has placed the following notice immediately following the
copyright notice for the Work:
Licensed under the EUPL
or has expressed by any other means his willingness to license under the EUPL.
1. Definitions
In this Licence, the following terms have the following meaning:
- 'The Licence': this Licence.
- 'The Original Work': the work or software distributed or communicated by the
Licensor under this Licence, available as Source Code and also as Executable
Code as the case may be.
- 'Derivative Works': the works or software that could be created by the
Licensee, based upon the Original Work or modifications thereof. This Licence
does not define the extent of modification or dependence on the Original
Work required in order to classify a work as a Derivative Work; this extent
is determined by copyright law applicable in the country mentioned in
Article 15.
- 'The Work': the Original Work or its Derivative Works.
- 'The Source Code': the human-readable form of the Work which is the most
convenient for people to study and modify.
- 'The Executable Code': any code which has generally been compiled and which
is meant to be interpreted by a computer as a program.
- 'The Licensor': the natural or legal person that distributes or communicates
the Work under the Licence.
- 'Contributor(s)': any natural or legal person who modifies the Work under
the Licence, or otherwise contributes to the creation of a Derivative Work.
- 'The Licensee' or 'You': any natural or legal person who makes any usage of
the Work under the terms of the Licence.
- 'Distribution' or 'Communication': any act of selling, giving, lending,
renting, distributing, communicating, transmitting, or otherwise making
available, online or offline, copies of the Work or providing access to its
essential functionalities at the disposal of any other natural or legal
person.
2. Scope of the rights granted by the Licence
The Licensor hereby grants You a worldwide, royalty-free, non-exclusive,
sublicensable licence to do the following, for the duration of copyright
vested in the Original Work:
- use the Work in any circumstance and for all usage,
- reproduce the Work,
- modify the Work, and make Derivative Works based upon the Work,
- communicate to the public, including the right to make available or display
the Work or copies thereof to the public and perform publicly, as the case
may be, the Work,
- distribute the Work or copies thereof,
- lend and rent the Work or copies thereof,
- sublicense rights in the Work or copies thereof.
Those rights can be exercised on any media, supports and formats, whether now
known or later invented, as far as the applicable law permits so.
In the countries where moral rights apply, the Licensor waives his right to
exercise his moral right to the extent allowed by law in order to make
effective the licence of the economic rights here above listed.
The Licensor grants to the Licensee royalty-free, non-exclusive usage rights
to any patents held by the Licensor, to the extent necessary to make use of
the rights granted on the Work under this Licence.
3. Communication of the Source Code
The Licensor may provide the Work either in its Source Code form, or as
Executable Code. If the Work is provided as Executable Code, the Licensor
provides in addition a machine-readable copy of the Source Code of the Work
along with each copy of the Work that the Licensor distributes or indicates,
in a notice following the copyright notice attached to the Work, a repository
where the Source Code is easily and freely accessible for as long as the
Licensor continues to distribute or communicate the Work.
4. Limitations on copyright
Nothing in this Licence is intended to deprive the Licensee of the benefits
from any exception or limitation to the exclusive rights of the rights owners
in the Work, of the exhaustion of those rights or of other applicable
limitations thereto.
5. Obligations of the Licensee
The grant of the rights mentioned above is subject to some restrictions and
obligations imposed on the Licensee. Those obligations are the following:
Attribution right: The Licensee shall keep intact all copyright, patent or
trademarks notices and all notices that refer to the Licence and to the
disclaimer of warranties. The Licensee must include a copy of such notices
and a copy of the Licence with every copy of the Work he/she distributes or
communicates. The Licensee must cause any Derivative Work to carry prominent
notices stating that the Work has been modified and the date of modification.
Copyleft clause: If the Licensee distributes or communicates copies of the
Original Works or Derivative Works, this Distribution or Communication will
be done under the terms of this Licence or of a later version of this Licence
unless the Original Work is expressly distributed only under this version of
the Licence — for example by communicating 'EUPL v. 1.2 only'. The Licensee
(becoming Licensor) cannot offer or impose any additional terms or conditions
on the Work or Derivative Work that alter or restrict the terms of the
Licence.
Compatibility clause: If the Licensee Distributes or Communicates Derivative
Works or copies thereof based upon both the Work and another work licensed
under a Compatible Licence, this Distribution or Communication can be done
under the terms of this Compatible Licence. For the sake of this clause,
'Compatible Licence' refers to the licences listed in the appendix attached
to this Licence. Should the Licensee's obligations under the Compatible
Licence conflict with his/her obligations under this Licence, the obligations
of the Compatible Licence shall prevail.
Provision of Source Code: When distributing or communicating copies of the
Work, the Licensee will provide a machine-readable copy of the Source Code or
indicate a repository where this Source will be easily and freely available
for as long as the Licensee continues to distribute or communicate the Work.
Legal Protection: This Licence does not grant permission to use the trade
names, trademarks, service marks, or names of the Licensor, except as
required for reasonable and customary use in describing the origin of the
Work and reproducing the content of the copyright notice.
6. Chain of Authorship
The original Licensor warrants that the copyright in the Original Work
granted hereunder is owned by him/her or licensed to him/her and that he/she
has the power and authority to grant the Licence.
Each Contributor warrants that the copyright in the modifications he/she
brings to the Work are owned by him/her or licensed to him/her and that
he/she has the power and authority to grant the Licence.
Each time You accept the Licence, the original Licensor and subsequent
Contributors grant You a licence to their contributions to the Work, under
the terms of this Licence.
7. Disclaimer of Warranty
The Work is a work in progress, which is continuously improved by numerous
Contributors. It is not a finished work and may therefore contain defects or
'bugs' inherent to this type of development.
For the above reason, the Work is provided under the Licence on an 'as is'
basis and without warranties of any kind concerning the Work, including
without limitation merchantability, fitness for a particular purpose, absence
of defects or errors, accuracy, non-infringement of intellectual property
rights other than copyright as stated in Article 6 of this Licence.
This disclaimer of warranty is an essential part of the Licence and a
condition for the grant of any rights to the Work.
8. Disclaimer of Liability
Except in the cases of wilful misconduct or damages directly caused to
natural persons, the Licensor will in no event be liable for any direct or
indirect, material or moral, damages of any kind, arising out of the Licence
or of the use of the Work, including without limitation, damages for loss of
goodwill, work stoppage, computer failure or malfunction, loss of data or any
commercial damage, even if the Licensor has been advised of the possibility
of such damage. However, the Licensor will be liable under statutory product
liability laws as far such laws apply to the Work.
9. Additional agreements
While distributing the Work, You may choose to conclude an additional
agreement, defining obligations or services consistent with this Licence.
However, if accepting obligations, You may act only on your own behalf and on
your sole responsibility, not on behalf of the original Licensor or any other
Contributor, and only if You agree to indemnify, defend, and hold each
Contributor harmless for any liability incurred by, or claims asserted
against such Contributor by the fact You have accepted any warranty or
additional liability.
10. Acceptance of the Licence
The provisions of this Licence can be accepted by clicking on an icon 'I
agree' placed under the bottom of a window displaying the text of this
Licence or by affirming consent in any other similar way, in accordance with
the rules of applicable law. Clicking on that icon indicates your clear and
irrevocable acceptance of this Licence and all of its terms and conditions.
Similarly, you irrevocably accept this Licence and all of its terms and
conditions by exercising any rights granted to You by Article 2 of this
Licence, such as the use of the Work, the creation by You of a Derivative
Work or the Distribution or Communication by You of the Work or copies
thereof.
11. Information to the public
In case of any Distribution or Communication of the Work by means of
electronic communication by You (for example, by offering to download the
Work from a remote location) the distribution channel or media (for example,
a website) must at least provide to the public the information requested by
the applicable law regarding the Licensor, the Licence and the way it may be
accessible, concluded, stored and reproduced by the Licensee.
12. Termination of the Licence
The Licence and the rights granted hereunder will terminate automatically
upon any breach by the Licensee of the terms of the Licence.
Such a termination will not terminate the licences of any person who has
received the Work from the Licensee under the Licence, provided such persons
remain in full compliance with the Licence.
13. Miscellaneous
Without prejudice of Article 9 above, the Licence represents the complete
agreement between the Parties as to the Work.
If any provision of the Licence is invalid or unenforceable under applicable
law, this will not affect the validity or enforceability of the Licence as a
whole. Such provision will be construed or reformed so as necessary to make
it valid and enforceable.
The European Commission may publish other linguistic versions or new versions
of this Licence or updated versions of the Appendix, so far this is required
and reasonable, without reducing the scope of the rights granted by the
Licence. New versions of the Licence will be published with a unique version
number.
All linguistic versions of this Licence, approved by the European Commission,
have identical value. Parties can take advantage of the linguistic version of
their choice.
14. Jurisdiction
Without prejudice to specific agreement between parties,
- any litigation resulting from the interpretation of this License, arising
between the European Union institutions, bodies, offices or agencies, as a
Licensor, and any Licensee, will be subject to the jurisdiction of the
Court of Justice of the European Union, as laid down in article 272 of the
Treaty on the Functioning of the European Union,
- any litigation arising between other parties and resulting from the
interpretation of this License, will be subject to the exclusive
jurisdiction of the competent court where the Licensor resides or conducts
its primary business.
15. Applicable Law
Without prejudice to specific agreement between parties,
- this Licence shall be governed by the law of the European Union Member
State where the Licensor has his seat, resides or has his registered
office,
- this licence shall be governed by Belgian law if the Licensor has no seat,
residence or registered office inside a European Union Member State.
Appendix
'Compatible Licences' according to Article 5 EUPL are:
- GNU General Public License (GPL) v. 2, v. 3
- GNU Affero General Public License (AGPL) v. 3
- Open Software License (OSL) v. 2.1, v. 3.0
- Eclipse Public License (EPL) v. 1.0
- CeCILL v. 2.0, v. 2.1
- Mozilla Public Licence (MPL) v. 2
- GNU Lesser General Public Licence (LGPL) v. 2.1, v. 3
- Creative Commons Attribution-ShareAlike v. 3.0 Unported (CC BY-SA 3.0) for
works other than software
- European Union Public Licence (EUPL) v. 1.1, v. 1.2
- Québec Free and Open-Source Licence — Reciprocity (LiLiQ-R) or Strong
Reciprocity (LiLiQ-R+).
The European Commission may update this Appendix to later versions of the
above licences without producing a new version of the EUPL, as long as they
provide the rights granted in Article 2 of this Licence and protect the
covered Source Code from exclusive appropriation.
All other changes or additions to this Appendix require the production of a
new EUPL version.

View File

@@ -1 +1 @@
0.1
0.2

View File

@@ -1,9 +0,0 @@
SUPPORTED_API = [
"gd",
"setgd",
"delgd",
"add_plot_values",
"last_plot_value"
]

View File

@@ -1,75 +0,0 @@
import sys
from threading import (Thread, Event)
from lib.string_queue import StringQueue
from time import (sleep)
class StdioRedirect:
def __init__(self):
self.redirect_enabled = False
self.spy_enabled = False
self.ini_stdout = sys.stdout
self.ini_stderr = sys.stderr
self.stream = self.ini_stdout
def redirect(self, stream):
if not self.spy_enabled:
self.out_stream = stream
self.stream = self.out_stream
sys.stdout = self.out_stream
sys.stderr = self.out_stream
self.redirect_enabled = True
def restore(self):
if not self.spy_enabled and self.redirect_enabled:
sys.stdout = self.ini_stdout
sys.stderr = self.ini_stderr
self.redirect_enabled = False
def intercept(self):
if not self.spy_enabled:
self.thr_started = Event()
self.log_buf = StringQueue()
self.in_stream = StringQueue()
self.stop_output = Event()
self.thrd_out = Thread(target=self.interceptStdOut)
self.thrd_out.daemon = True
sys.stdout = self.in_stream
sys.stderr = self.in_stream
self.stream = self.in_stream
self.thrd_out.start()
self.thr_started.wait()
self.spy_enabled = True
def stop(self):
if self.spy_enabled:
sys.stdout = self.out_stream
sys.stderr = self.out_stream
self.stream = self.out_stream
self.stop_output.set()
self.thrd_out.join()
del self.log_buf
del self.in_stream
del self.stop_output
del self.thrd_out
del self.thr_started
self.spy_enabled = False
def interceptStdOut(self):
self.thr_started.set()
while not self.stop_output.is_set():
data = self.in_stream.read()
self.log_buf.write(data)
self.out_stream.write(data)
if data == '':
sleep(0.1)
def read(self):
ret = ''
if self.spy_enabled:
ret = self.log_buf.read()
return ret
stdio_redir = StdioRedirect()

View File

@@ -1,23 +0,0 @@
from pathlib import Path
import sys
import traceback
def exception_handler(typ_exc, value, trbk):
"""Testium Exception handling"""
print("An unmanaged exception occured")
print(f"Critical failure : '{value}'.")
tb = traceback.format_exception(typ_exc, value, trbk)
print("".join(tb))
sys.excepthook = exception_handler
p = Path(__file__)
p = p.parent / ".."
p = p.resolve()
sys.path.append(p)
from py_func import main
if __name__ == '__main__':
main()

View File

@@ -1,85 +0,0 @@
import sys
from py_func.handle import FuncHandler
from lib.tum_except import ETUMRuntimeError
from lib.api import SUPPORTED_API
thismodule = sys.modules[__name__]
# Shared FuncHandler instance used to forward API calls. Remains None
# until `_init_api` is invoked.
_func_call_thread = None
###############################################################################
# Dynamically create module-level functions for each supported API name.
# Each generated function shares the implementation of `api_call` but
# has a distinct name used as the remote action identifier.
def _make_api(name):
def _wrapper(*params):
if _func_call_thread is not None:
res = _func_call_thread.call(name, params)
if "result" in res:
ret_val = res["result"]
elif "error" in res:
raise ETUMRuntimeError(f"api call to 'tm.{name}' failed with error '{res["error"]}'")
else:
raise ETUMRuntimeError("api call failure in jrpc client to be reported to testium support team.")
return ret_val
else:
raise ETUMRuntimeError("api not initialized")
_wrapper.__name__ = name
return _wrapper
for k in SUPPORTED_API:
setattr(thismodule, k, _make_api(k))
def _init_api(host, port, timeout):
"""Start and initialize the remote function handler.
Starts a ``FuncHandler`` bound to ``port``, runs it and blocks until
it signals readiness.
Args:
port: port number or identifier passed to ``FuncHandler``.
Returns:
The initialized ``FuncHandler`` instance assigned to
``_func_call_thread``.
"""
global _func_call_thread
_func_call_thread = FuncHandler(host, port, timeout=timeout)
return _func_call_thread
###############################################################################
def _remote_print(*values):
"""Forward print-like output to the remote handler.
If a ``_func_call_thread`` is available, this function calls the
handler with action name ``"print"`` and the provided values. Errors
during forwarding are ignored because printing is best-effort.
"""
if _func_call_thread is not None:
try:
_func_call_thread.call("print", values)
except:
# Best-effort: ignore forwarding failures
pass
###############################################################################
class FunctionItem():
"""Class allowing extended capabilities of function."""
module_count = 0
def __init__(self):
self._reported_value = {}
def reportValue(self, key, value):
self._reported_value[key] = value
def reportedValues(self):
return self._reported_value
def exec(self):
pass

View File

@@ -8,9 +8,11 @@ requires-python = ">=3.11"
authors = [
{name = "François Dausseur", email = "francois@beafrancois.fr"},
]
license = "EUPL-1.2"
license-files = ["../LICENSE"]
classifiers = [
"Development Status :: 5 - Production/Stable",
"Programming Language :: Python"
"Programming Language :: Python",
]
dependencies = [
"setuptools",
@@ -33,7 +35,9 @@ testium = "testium:main"
[tool.setuptools.packages.find]
where=["."]
exclude=["lua_func", "py_func"]
[tool.setuptools.package-data]
"testium.lua_func" = ["*.lua"]
[tool.setuptools.dynamic]
version = {file = ["VERSION"]}

View File

@@ -21,10 +21,8 @@ def main():
help="Returns the version of testium", action='store_true')
parser.add_argument("-b", "--batch-execution",
help="Executes the test in batch mode", action='store_true')
parser.add_argument("-m", "--terminal",
help="Starts terminal mode", action='store_true')
parser.add_argument("-o", "--no-color",
help="Deactivates stdout colors in batch and terminal mode", action='store_true')
help="Deactivates stdout colors in batch mode", action='store_true')
parser.add_argument("-c", "--config-file", help="Configuration file",
nargs='+',
default=[])
@@ -95,36 +93,13 @@ def main():
from interpreter.utils.version import get_testium_version
print(get_testium_version())
elif args.terminal:
import select
from interpreter.terminal import Terminal
if (lf != '') or (rf != '') or (tf != '') or (pn != []):
print('"-l", "-p", "-t", "-n" options are not supported in this mode.')
t = Terminal(os.getcwd(), cf, defines, args.no_color)
loop = 1
while loop:
try:
loop = 0
t.cmdloop()
except KeyboardInterrupt:
print("\n<ctrl-c>")
loop = 1
except Exception as exc:
if str(exc) == 'quit':
break
print(exc)
loop = 1
elif args.batch_execution:
if (lf != ''):
print('"-l" option is not supported in this mode.')
from interpreter.batch import Batch
b = Batch(tf, cf, defines, rf, args.report_type, pn, args.no_color)
b = Batch(tf, cf, defines, rf, args.report_type, pn, args.no_color, text_mode=True)
sys.exit(0 if b.success else 1)
else:
from main_win.testium_win import MainWin

View File

@@ -11,6 +11,7 @@ import threading
from telnetlib3 import Telnet, DO, WILL, WONT, TTYPE, IAC, SB, SE, theNULL
TIMEOUT_NULL = 0.000001
STOP_POLL_INTERVAL = 0.2
class BytesStore(object):
@@ -123,12 +124,14 @@ A {classname}.close() is missing somewhere in your code !'.format(classname=type
# c = ''
return c
def read_until(self, match, timeout=None, return_data=False, mute=False):
def read_until(self, match, timeout=None, return_data=False, mute=False, should_stop=None):
"""
read until the string 'match is found
If timeout is not set (None), this function runs indefinitely
If timeout is set to zero, this function returns immediately
If mute is set to True the characters read from the console will not be displayed
If should_stop is a callable, it is polled between reads (every STOP_POLL_INTERVAL
at most) and the loop exits early like a timeout when it returns True.
If function fails (because of a timeout) it will return a 'status' integer set to -1
otherwise it will return 0.
@@ -139,13 +142,6 @@ A {classname}.close() is missing somewhere in your code !'.format(classname=type
status = -1
if not match:
raise ValueError('match parameter can not be empty')
# replace all '\r' by '\n' as any '\r' read will undergo the same replacement
# match = match.replace('\r\n', '\n')
# match = match.replace('\r', '')
# update the console timeout in conformity with what is required.
self.set_read_timeout(timeout)
if timeout is None:
timeout = 1000000
@@ -159,6 +155,7 @@ A {classname}.close() is missing somewhere in your code !'.format(classname=type
# buffer is empty
# Otherwise we are waiting for the timeout to rise
if timeout < TIMEOUT_NULL:
self.set_read_timeout(0)
data = self.readchar(0)
while (status < 0) and ((data is not None) and (data != b'')):
@@ -191,39 +188,45 @@ A {classname}.close() is missing somewhere in your code !'.format(classname=type
# Timeout different than zero
else:
# Poll in short chunks so a stop request is honored within
# STOP_POLL_INTERVAL, regardless of the per-protocol blocking
# behavior of readchar().
self.set_read_timeout(STOP_POLL_INTERVAL)
time_is_out = threading.Event()
timer = threading.Timer(timeout, lambda: time_is_out.set())
timer.start()
# We are waiting for the timeout to rise
try:
while (status < 0) and (not time_is_out.is_set()):
if should_stop is not None and should_stop():
break
while (status < 0) and (not time_is_out.isSet()):
data = self.readchar(timeout)
if data is not None:
data = self._compute_char(data)
if data != '':
if not mute:
self.string_buffer += data
read_data += data
search_deque.append(data)
if search_deque == match_deque:
timer.cancel()
status = 0
if (not mute) and (data != '\n'):
self.string_buffer += '\n'
if data == '\n' or (status >= 0):
# the datas are written line by line for display optimisation in GUI mode
data = self.readchar(STOP_POLL_INTERVAL)
if data is not None:
data = self._compute_char(data)
if data != '':
if not mute:
self.string_buffer = self.string_buffer.replace('\r\n', '\n')
self.string_buffer = self.string_buffer.replace('\r', '')
self.stream.write(self.string_buffer)
self.string_buffer += data
read_data += data
date_str = str(datetime.now()).split('.')[0].split(' ')[1]
self.string_buffer = '[{} {}]'.format(date_str, self.name)
search_deque.append(data)
if search_deque == match_deque:
status = 0
if (not mute) and (data != '\n'):
self.string_buffer += '\n'
if data == '\n' or (status >= 0):
# the datas are written line by line for display optimisation in GUI mode
if not mute:
self.string_buffer = self.string_buffer.replace('\r\n', '\n')
self.string_buffer = self.string_buffer.replace('\r', '')
self.stream.write(self.string_buffer)
date_str = str(datetime.now()).split('.')[0].split(' ')[1]
self.string_buffer = '[{} {}]'.format(date_str, self.name)
finally:
timer.cancel()
if return_data:
return status, read_data
@@ -245,7 +248,7 @@ A {classname}.close() is missing somewhere in your code !'.format(classname=type
if not sys.platform.startswith('win'):
# import SshConsole if pexpect is installed
try:
from libs.console_ssh import SshConsole
from api.console_ssh import SshConsole
except ImportError:
pass

View File

@@ -8,7 +8,7 @@ import os
import pexpect
from pexpect import ExceptionPexpect, TIMEOUT, EOF, spawn
from libs.console import Console
from api.console import Console
# Exception classes used by this module.

View File

@@ -3,7 +3,7 @@ import sys
import socket
import traceback
from libs.console import *
from api.console import *
class RawTCPConsole(Console):
TYPE = 'rawtcp'

View File

@@ -1,5 +1,6 @@
import sys
import os
import queue
import multiprocessing as mp
from threading import Timer
from time import sleep, monotonic
@@ -16,9 +17,9 @@ import numpy as np
import matplotlib.dates as mdates
from datetime import datetime, timedelta, timezone
import libs.testium as tm
import api.testium as tm
from interpreter.test_items.test_result import TestValue
from lib.tum_except import ETUMRuntimeError
from runtime.tum_except import ETUMRuntimeError
from interpreter.utils.py_func_exec import PyFuncExecEngine
from interpreter.utils.api_srv import api_request
from interpreter.utils.eval import post_evaluate
@@ -270,7 +271,7 @@ class RuntimePlotPeriodic(PeriodicTimer):
self.func_name = func_name
self.args = args
self.post_eval = post_eval
self.proc = PyFuncExecEngine(tm.gd("python_bin", ""), api_request, 10)
self.proc = PyFuncExecEngine(api_request, 10)
self.proc.start()
if not self.proc.wait_ready(10):
raise ETUMRuntimeError(
@@ -367,7 +368,7 @@ class RuntimePlot:
self.msg_queue_in.get()
self.msg_queue_out.put({"command": "last_values"})
try:
res = self.msg_queue_in.get(timeout=1)
except:
res = self.msg_queue_in.get(timeout=5)
except queue.Empty:
raise ETUMRuntimeError(f"Impossible to retrieve the last values of the \"{self.name}\" plot")
return res

View File

@@ -10,7 +10,7 @@ import os
ourPath = os.path.dirname(__file__)
sys.path.append(ourPath)
from libs.console import (Console, BytesStore, TIMEOUT_NULL)
from api.console import (Console, BytesStore, TIMEOUT_NULL)
class TermConsole(Console):
TYPE = 'term'

View File

@@ -4,7 +4,7 @@ import sys
import textwrap
from time import monotonic
import interpreter.utils.globdict as globdict
from lib.tum_except import (ETUMSyntaxError)
from runtime.tum_except import (ETUMSyntaxError)
###############################################################################
# Console helper functions
@@ -14,7 +14,7 @@ def add_console(console):
''' Function which adds a ``Console`` class instance to *testium*
:param console: The ``Console`` instance.
:type console: ``libs.console.Console`` or child class instance
:type console: ``api.console.Console`` or child class instance
:return: No returned value
'''
@@ -48,7 +48,7 @@ def console(name):
:param name: The name of the ``Console`` instance.
:type name: str
:return: The ``Console`` or child class object
:rtype: ``libs.console.Console`` or child class instance
:rtype: ``api.console.Console`` or child class instance
"""
cons = None
for c in globdict.gd('console_instances', []):
@@ -65,7 +65,7 @@ def add_plot(plot: object) -> None:
''' Function which adds a ``RuntimePlot`` class instance to *testium*
:param plot: The ``RuntimePlot`` instance.
:type plot: ``libs.runtime_plot.RuntimePlot`` or child class instance
:type plot: ``api.runtime_plot.RuntimePlot`` or child class instance
:return: No returned value
'''
@@ -99,7 +99,7 @@ def plot(name: str) -> object:
:param name: The name of the ``RuntimePlot`` instance.
:type name: str
:return: The ``RuntimePlot`` or child class object
:rtype: ``libs.runtime_plot.RuntimePlot`` or child class instance
:rtype: ``api.runtime_plot.RuntimePlot`` or child class instance
"""
plot = None
for g in globdict.gd('plot_instances', []):
@@ -209,6 +209,15 @@ def OS():
return platform.system()
def text_mode():
"""Whether testium is running in text mode (batch ``-b`` or terminal ``-m``).
:return: ``True`` if running in text mode, ``False`` otherwise.
:rtype: bool
"""
return bool(globdict.gd("_text_mode", False))
def sys_encoding():
if OS() == "Windows":
enc = "oem"

View File

@@ -1,6 +1,7 @@
import os
import sys
import platform
import threading
from time import sleep
from signal import signal, SIGINT
from queue import Empty
@@ -8,8 +9,8 @@ from multiprocessing import Queue
from interpreter.process import TestProcess
from interpreter.utils.test_ctrl import TestSetController
from lib.tum_except import ETUMFileError
from lib.stdout_redirect import stdio_redir
from runtime.tum_except import ETUMFileError, ETUMRuntimeError
from runtime.stdout_redirect import stdio_redir
class Batch:
@@ -22,6 +23,7 @@ class Batch:
report_type,
report_pattern,
no_color,
text_mode=False,
):
try:
try:
@@ -51,6 +53,7 @@ class Batch:
signal(SIGINT, self.sigint_handler)
self._success = False
msg_queue = Queue()
self.tst_ctrl = TestSetController()
tst_proc = TestProcess(
@@ -59,11 +62,21 @@ class Batch:
self.tst_ctrl,
config_files,
defines,
text_mode=text_mode,
)
tst_proc.start()
while not self.tst_ctrl.control("loaded"):
sleep(0.1)
# Wait for TestProcess to finish loading.
# Run the blocking control("loaded") in a daemon thread so we
# can watch for unexpected process death in the main thread.
_loaded_event = threading.Event()
def _wait_loaded():
self.tst_ctrl.control("loaded")
_loaded_event.set()
threading.Thread(target=_wait_loaded, daemon=True).start()
while not _loaded_event.wait(timeout=0.1):
if not tst_proc.is_alive():
raise ETUMRuntimeError("TestProcess terminated unexpectedly during load")
self.tst_ctrl.control(
"report",
@@ -78,14 +91,18 @@ class Batch:
while True:
try:
m = msg_queue.get(timeout=0.2)
if m.get("id", None) is None:
# No id -> finished
if "id" in m and m["id"] is None:
# id key present and None -> finished
self._success = m.get("success", False)
break
except Empty:
if not tst_proc.is_alive():
break
continue
# Close the process and wait for termination
self.tst_ctrl.control("close")
if tst_proc.is_alive():
self.tst_ctrl.control("close")
tst_proc.join()
except Exception as e:
@@ -94,5 +111,12 @@ class Batch:
finally:
stdio_redir.restore()
@property
def success(self):
return self._success
def sigint_handler(self, signal_received, frame):
self.tst_ctrl.control("stop")
try:
self.tst_ctrl.control("stop", timeout=5)
except Exception:
pass

View File

@@ -1,19 +1,22 @@
import os
import signal
from multiprocessing import Process, Queue, Pipe
from queue import Empty
from threading import Thread
from time import sleep
import copy
from lib.string_queue import StringQueue
from lib.tum_except import print_exception, ETUMRuntimeError, ETUMSyntaxError
import libs.testium as tm
from runtime.string_queue import StringQueue
from runtime.tum_except import print_exception, ETUMRuntimeError, ETUMSyntaxError
import api.testium as tm
import interpreter.utils.globdict as globdict
from interpreter.utils.params import expanse
from interpreter.utils.test_ctrl import TestSetController
from interpreter.utils.test_init import (
env_init,
prepare_global,
update_global,
apply_overrides,
set_standard_gd_keys,
test_run_init,
test_run_header,
@@ -24,7 +27,7 @@ from interpreter.utils.test_init import (
from interpreter.utils.constants import TestItemType as cst_type
from interpreter.test_set import TestSet
from interpreter.utils.include import TUMLoader, TUMLoaderNoIncludes, TUMLoaderRawIncludes
from lib.stdout_redirect import stdio_redir
from runtime.stdout_redirect import stdio_redir
from interpreter.utils.template import template_to_test
from interpreter.utils.yaml_load import yaml_load
from interpreter.utils.py_eval import eval_process_init
@@ -40,6 +43,7 @@ class TestProcess(Process):
config_files,
defines,
gui_defaults={},
text_mode=False,
) -> None:
super().__init__()
self.__fname = file_name
@@ -48,6 +52,7 @@ class TestProcess(Process):
self.__cfgf = config_files
self.__defs = defines
self.__gui_defaults = gui_defaults # default values coming from GUI prefs
self.__text_mode = text_mode
self.__exec = False
self.__loaded = False
self.__closed = False
@@ -193,6 +198,7 @@ class TestProcess(Process):
def run(self):
signal.signal(signal.SIGINT, signal.SIG_IGN)
try:
try:
# Thread for stdout redirection
@@ -205,8 +211,21 @@ class TestProcess(Process):
env_init()
# Apply GUI defaults and CLI defines to the global dict
# *before* eval_proc starts: bins.python_bin() reads
# ``python_bin`` from gd on its very first call (during
# eval_process_init) and caches the result. Without this,
# ``-d python_bin=...`` and the GUI ``python_bin`` preference
# would only take effect for items spawned *after* the cache
# was already populated with the auto-discovered interpreter,
# i.e. they would silently be ignored for eval_proc itself.
# _load_initial_params re-applies the same overrides after
# ``prepare_global()`` clears gd, so the gd value stays in
# sync with the cached path.
apply_overrides(self.__defs, self.__gui_defaults)
# Creation of the python evaluation process for loading of the complete test
eval_proc = eval_process_init("", api_request, 10, test_dir)
eval_proc = eval_process_init(api_request, 10, test_dir)
eval_proc.start()
tm.print_debug(f"python bin is: '{eval_proc.python_bin}'.")
if not eval_proc.wait_ready(10):
@@ -223,6 +242,10 @@ Is the python exec path correct ?"""
# Load the test file
test_dict, param_files = self._load_test(init_param_files, glob_variables)
if self.__text_mode:
tm.setgd("_text_mode", True)
tm.setgd("_interactive", False)
# Backup the global dict in case of restart of the test
gdict = backup_gd()
@@ -255,6 +278,7 @@ Is the python exec path correct ?"""
try:
test_run_init()
print(test_run_header())
globdict.set_update_queue(self.__squeue)
test_set.execute()
finally:
if test_set.success():
@@ -265,8 +289,16 @@ Is the python exec path correct ?"""
test_set.run_post_exec()
finally:
self.__exec = False
# Stop shared context engines before restore_gd wipes them
for engine in tm.gd("_py_func_contexts", {}).values():
engine.stop()
engine.join()
for engine in tm.gd("_lua_func_contexts", {}).values():
engine.stop()
engine.join()
# Sends signal to the GUI
self.send_finished()
self.send_finished(success=test_set.success())
globdict.set_update_queue(None)
restore_gd(gdict)
except Exception as e:
print_exception(e)
@@ -275,6 +307,13 @@ Is the python exec path correct ?"""
# Stop python eval execution process
eval_proc.stop()
eval_proc.join()
# Stop shared func context engines (keep_context_id)
for engine in tm.gd("_py_func_contexts", {}).values():
engine.stop()
engine.join()
for engine in tm.gd("_lua_func_contexts", {}).values():
engine.stop()
engine.join()
except Exception as e:
print_exception(e)
@@ -297,6 +336,9 @@ Is the python exec path correct ?"""
"enabled_state": test_set.getEnabledState,
"process_param": self.process_param,
"set_test_outputs": self.set_test_outputs,
"get_gd_vars": self.get_gd_vars,
"set_gd_var": self.set_gd_var,
"del_gd_var": self.del_gd_var,
"set_enabled_state": test_set.setEnabledState,
"check_uncheck_all": test_set.checkUncheckAll,
"get_folded": test_set.getFolded,
@@ -311,8 +353,10 @@ Is the python exec path correct ?"""
stdio_redir.restore()
stdio_redir.stop()
def send_finished(self):
def send_finished(self, success=None):
status = {"id": None, "name": "test_process", "status": "finished"}
if success is not None:
status["success"] = success
self.__squeue.put(status)
def execute(self):
@@ -330,6 +374,25 @@ Is the python exec path correct ?"""
def set_test_outputs(self, outputs: list):
tm.setgd("test_outputs", outputs)
def get_gd_vars(self):
import json
result = {}
for k, v in globdict.global_dict.items():
if k.startswith("_"):
continue
try:
json.dumps(v)
result[k] = v
except (TypeError, ValueError):
pass
return result
def set_gd_var(self, name: str, value):
tm.setgd(name, value)
def del_gd_var(self, name: str):
tm.delgd(name)
def process_control_commands(self, tctrl):
term = False
while (not term) and (not self.__closed):
@@ -382,7 +445,7 @@ Is the python exec path correct ?"""
try:
# read the pipe data
data = cconn.recv()
print(data, end="")
print(data, end="", flush=True)
except EOFError:
# exit the loop is the pipe is closed
break

View File

@@ -1,244 +0,0 @@
try:
import readline
except:
pass
from cmd import Cmd
import os
import sys
from yaml import load, Loader
import functools
import platform
import types
import inspect
# test modules
from interpreter.utils.test_init import (
env_init, prepare_global, set_standard_gd_keys,
update_global, test_run_init, test_run_header, load_test)
from interpreter.utils.globdict import (global_dict)
import libs.testium as tm
from interpreter.utils.constants import TestItemType as cst
from interpreter.test_report.test_report import TestReport
class FakeQueue:
def put(self, arg):
pass
def func(self, args):
if not args.startswith("{"):
args = "{"+args+"}"
y = load(args, Loader)
obj = self.current_item(y, status_queue=FakeQueue())
obj.report = self.report
res = obj.execute()
if not (res.value is None):
print('result : {}'.format(res.value))
print(res.test_result)
class Terminal(Cmd):
SUPPORTED_TESTS = [
cst.TYPE_SLEEP,
cst.TYPE_LET,
cst.TYPE_PY_FUNCTION,
cst.TYPE_LUA_FUNCTION,
cst.TYPE_CONSOLE,
cst.TYPE_IMAGE_DLG,
cst.TYPE_MESSAGE_DLG,
cst.TYPE_QUESTION_DLG,
cst.TYPE_VALUE_DLG,
]
SUPPORTED_GROUPS = [
cst.TYPE_GROUP,
cst.TYPE_CYCLE
]
def __init__(self, working_dir, config_files, defines, no_color):
super().__init__()
self.working_dir = working_dir
self.config_files = config_files
self.current_item = None
report = TestReport(None)
self.report = report
env_init()
prepare_global()
# Define the builtin variables
set_standard_gd_keys("Unnamed", self.working_dir, '', config_files)
update_global([], defines)
# creation of the functions
for tst in self.SUPPORTED_TESTS:
meth_name = "do_" + tst.item_cmd
# copy of the function
f = types.FunctionType(func.__code__, func.__globals__, name=meth_name,
argdefs=func.__defaults__,
closure=func.__closure__)
f = functools.update_wrapper(f, func)
f.__kwdefaults__ = func.__kwdefaults__
f.__doc__ = tst.item_class.__doc__
setattr(self, meth_name, types.MethodType(f, self))
test_run_init()
self.prompt = "(testium)~ "
# display header
print(test_run_header())
# redirect output
if 'Linux' in platform.system() and not no_color:
from lib.stdout_redirect import stdio_redir
try:
from interpreter.utils.termlog import TermLog
stdio_redir.redirect(TermLog(sys.stdout))
except ModuleNotFoundError:
tm.print_info('Colored console not supported by the system.' +
' If you want it, please install colorama module')
def precmd(self, line: str) -> str:
c = line.split(" ", 1)[0].strip()
self.current_item = None
for tst in self.SUPPORTED_TESTS:
if c == tst.item_cmd:
self.current_item = tst.item_class
break
return line
def load_test_recursively(self, tree_parent, parent_seq, status_queue):
try:
parent_seq_name = parent_seq['name']
except KeyError:
parent_seq['name'] = "sequence"
except TypeError:
raise Exception("Syntax error in an item of type {} which is a child of {}".format(
tree_parent.type(), tree_parent.parent().name()))
try:
parent_seq_actions = parent_seq['steps']
except KeyError:
raise Exception(' No action list found for "%s" sequence'
% (parent_seq_name))
# if action is a dictionary , we assume it is a single action
# that has not been nested in a list, so do it
if isinstance(parent_seq_actions, (dict)):
parent_seq_actions = [parent_seq_actions]
if not isinstance(parent_seq_actions, (list, tuple)):
raise Exception('Actions list not valid.')
# first we merged to the same level 'sequence dict entries and list within the list
counter = 0
test_dir = tm.gd('test_directory')
while (counter < len(parent_seq_actions)):
action = parent_seq_actions[counter]
# if action is a list raise up to the the same level,
# ie insert action element into the parent_seq_actions
if isinstance(action, (list, tuple)):
parent_seq_actions[counter:counter+1] = action
continue
# if action is a NoneType skip and continue
# (when pointing to an unused alias for instance)
if action is None:
counter += 1
continue
# if action is a sequence we insert its entry into the action list
if 'sequence' in action:
parent_seq_actions[counter:counter+1] = action['sequence']
continue
else:
executed = False
for it in [*self.SUPPORTED_TESTS, *self.SUPPORTED_GROUPS]:
if it.item_cmd in action:
executed = True
item = (it.item_class)(action[it.item_cmd],
tree_parent,
status_queue)
# check for sequence type:
if it.item_cmd == cst.TYPE_UNITTEST_FILE.item_cmd:
item.setTestDir(test_dir)
item.load()
elif ((it.item_cmd == cst.TYPE_CYCLE.item_cmd) or
(it.item_cmd == cst.TYPE_GROUP.item_cmd)):
self.load_test_recursively(
item, action[it.item_cmd], status_queue)
if not executed:
raise Exception('action type is not known "{}"'.format(
list(action.keys())[0]))
counter += 1
def __setReportRecursively(self, parent):
for i in range(parent.childCount()):
parent.child(i).report = self.report
self.__setReportRecursively(parent.child(i))
def setReport(self, root_item):
root_item.report = self.report
self.__setReportRecursively(root_item)
def get_names(self):
memb = inspect.getmembers(self)
return [n[0] for n in memb if (inspect.ismethod(n[1]) and n[0].startswith("do_"))]
def do_load(self, args):
"""load function.
This function loads and executes a testium sub-script.
The loaded sequence can't be a main testium script ("testium -b" option is
defined for such a usage).
Accepted files are with extension "*.tum".
usage:
load path/to/my/sequence.tum
"""
file = args.strip()
suff = file[-4:]
if not suff in ['.tum']:
raise Exception('Wrong input file extension')
if not (os.path.exists(file) and os.path.isfile(file)):
raise Exception(
'"{}" does not exist or is not a file.'.format(file))
d, _ = load_test(file)
if not isinstance(d, list):
raise Exception(
"The file root object must be a list. A \"main\" tum can't be loaded from here (use batch mode instead).")
if (len(d) == 1) and isinstance(d[0], dict) and (not d[0].get('sequence', None) is None):
d = d[0]['sequence']
sq = FakeQueue()
root_item = (cst.TYPE_ROOT.item_class)(
dict_item={'steps': d}, status_queue=sq)
self.load_test_recursively(root_item, {'steps': d}, sq)
self.setReport(root_item)
res = root_item.execute()
if not (res.value is None):
print('"{}" execution overall result: {}'.format(file, res.value))
print(res.test_result)
def do_gd(self, args):
"""Variables lists and values.
usage:
gd
gd home
"""
if args != '':
res = tm.gd(args, None)
if res is None:
raise Exception(
'the variable: "{}" has not been found.'.format(args))
print(res)
return
for k in global_dict.keys():
print('{}: {}'.format(str(k), str(global_dict[k])))
def do_quit(self, args):
'''Quit the application.'''
raise Exception('quit')

View File

@@ -5,7 +5,7 @@ from itertools import chain
from PySide6.QtGui import QIcon, QPixmap
from PySide6.QtWidgets import QApplication, QDialog, QDialogButtonBox
from PySide6.QtCore import Qt, QSettings, QSize
from PySide6.QtCore import Qt, QSettings, QTimer, QSize
from PySide6.QtGui import QFont, QFontInfo
from PySide6.QtWidgets import QTreeWidgetItem
@@ -185,7 +185,9 @@ def main(args, conn=None):
SettingsApplication = "testium_choices_dlg_" + args[0]
SettingsLastChoices = "last_choice"
success = True
app = QApplication()
from interpreter.test_items import dialog_env
dialog_env.setup()
app = QApplication(['testium'])
d = ChoicesDialog()
d.setFixedSize(800, 600)
d.setWindowFlags(Qt.WindowStaysOnTopHint)
@@ -205,6 +207,9 @@ def main(args, conn=None):
d.connect_checked()
d.choicesView.setFocus()
auto_result = args[4] if len(args) > 4 else None
if auto_result is not None:
QTimer.singleShot(2000, lambda: d.accept() if auto_result.lower() == 'ok' else d.reject())
dres = d.exec()
if dres == QDialog.Rejected:

View File

@@ -3,7 +3,7 @@
################################################################################
## Form generated from reading UI file 'choices_dialog_win.ui'
##
## Created by: Qt User Interface Compiler version 6.10.1
## Created by: Qt User Interface Compiler version 6.11.0
##
## WARNING! All changes made in this file will be lost when recompiling UI file!
################################################################################

View File

@@ -0,0 +1,17 @@
"""Qt platform environment setup for dialog subprocesses.
Call setup() at the start of every dialog subprocess main() function
to ensure the correct Qt platform plugin is selected.
"""
import sys
import os
def setup():
"""Configure the Qt environment for dialog subprocess usage."""
if sys.platform.startswith('linux'):
if os.environ.get('DISPLAY'):
# X11 available: force xcb to avoid crashes in multiprocessing subprocesses.
os.environ['QT_QPA_PLATFORM'] = 'xcb'
elif os.environ.get('WAYLAND_DISPLAY'):
os.environ['QT_QPA_PLATFORM'] = 'wayland'

View File

@@ -1,7 +1,6 @@
import sys
import os
from PySide6.QtCore import (Qt)
from PySide6.QtCore import Qt, QTimer
from PySide6.QtWidgets import (QApplication, QDialog)
from PySide6 import (QtGui)
@@ -18,7 +17,9 @@ class TestDialogWindow(QDialog, dialog_image_win.Ui_Dialog):
def main(args, conn):
success = True
app = QApplication(args)
from interpreter.test_items import dialog_env
dialog_env.setup()
app = QApplication(['testium'])
d = TestDialogWindow()
d.setFixedSize(700,600)
d.setWindowFlags(Qt.WindowStaysOnTopHint)
@@ -37,6 +38,10 @@ def main(args, conn):
d.labelImage.setPixmap(QtGui.QPixmap.fromImage(image2))
auto_result = args[3] if len(args) > 3 else None
if auto_result is not None:
QTimer.singleShot(2000, lambda: d.accept() if auto_result.lower() == 'ok' else d.reject())
dres = d.exec()
if dres == QDialog.Rejected:

View File

@@ -3,7 +3,7 @@
################################################################################
## Form generated from reading UI file 'dialog_image_win.ui'
##
## Created by: Qt User Interface Compiler version 6.10.1
## Created by: Qt User Interface Compiler version 6.11.0
##
## WARNING! All changes made in this file will be lost when recompiling UI file!
################################################################################

View File

@@ -1,26 +1,32 @@
import sys
import os
from multiprocessing import freeze_support
from PySide6.QtWidgets import (QApplication, QMessageBox)
from PySide6.QtCore import Qt, QTimer
from PySide6.QtWidgets import (QApplication, QDialog)
from PySide6.QtCore import (Qt)
from PySide6.QtWidgets import QMessageBox
from multiprocessing import freeze_support
def main(args):
app = QApplication(sys.argv)
reply = QMessageBox.information(None, args[0], args[1], QMessageBox.Ok)
from interpreter.test_items import dialog_env
dialog_env.setup()
app = QApplication(['testium'])
msg = QMessageBox()
msg.setWindowFlags(Qt.WindowStaysOnTopHint)
msg.setWindowTitle(args[0])
msg.setText(args[1])
msg.setIcon(QMessageBox.Information)
msg.setStandardButtons(QMessageBox.Ok)
if len(args) > 2:
QTimer.singleShot(2000, lambda: msg.button(QMessageBox.Ok).click())
msg.exec()
if hasattr(sys, "frozen"):
#all standard streams are replaced by dummy one to avoid cx_freeze flushing bug.
class dummyStream:
''' dummyStream behaves like a stream but does nothing. '''
def __init__(self): pass
def write(self,data): pass
def read(self,data): pass
def write(self, data): pass
def read(self, data): pass
def flush(self): pass
def close(self): pass
# and now redirect all default streams to this dummyStream:
sys.stdout = dummyStream()
sys.stderr = dummyStream()
sys.stdin = dummyStream()
@@ -31,6 +37,3 @@ def main(args):
if __name__ == '__main__':
main(sys.argv[1:])

View File

@@ -3,7 +3,7 @@
################################################################################
## Form generated from reading UI file 'dialog_note_win.ui'
##
## Created by: Qt User Interface Compiler version 6.10.1
## Created by: Qt User Interface Compiler version 6.11.0
##
## WARNING! All changes made in this file will be lost when recompiling UI file!
################################################################################

View File

@@ -2,7 +2,7 @@ import sys
import os
from PySide6.QtWidgets import (QApplication, QDialog)
from PySide6.QtCore import (Qt)
from PySide6.QtCore import Qt, QTimer
from interpreter.test_items.dialog_note_files import dialog_note_win
from multiprocessing import freeze_support
@@ -14,13 +14,23 @@ class TestDialogWindow(QDialog, dialog_note_win.Ui_Dialog):
def main(args, conn=None):
success = True
app = QApplication(args)
from interpreter.test_items import dialog_env
dialog_env.setup()
app = QApplication(['testium'])
d = TestDialogWindow()
d.setFixedSize(387,224)
d.setWindowFlags(Qt.WindowStaysOnTopHint)
d.setWindowTitle(args[0])
d.labelDialog.setText(args[1])
d.textEdit.setFocus()
auto_result = args[2] if len(args) > 2 else None
if auto_result is not None:
auto_value = args[3] if len(args) > 3 else None
def _auto_close():
if auto_value is not None:
d.textEdit.setPlainText(auto_value)
d.accept() if auto_result.lower() == 'ok' else d.reject()
QTimer.singleShot(2000, _auto_close)
dres = d.exec()
if dres == QDialog.Rejected:

View File

@@ -1,29 +1,40 @@
import sys
import os
from multiprocessing import freeze_support
from PySide6.QtWidgets import (QApplication, QMessageBox)
from PySide6.QtCore import Qt, QTimer
from PySide6.QtWidgets import (QApplication, QDialog)
from PySide6.QtCore import (Qt)
from PySide6.QtWidgets import QMessageBox
from multiprocessing import freeze_support
def main(args, conn):
app = QApplication(sys.argv)
reply = QMessageBox.question(None, args[0], args[1], QMessageBox.Yes|QMessageBox.No)
conn.send(reply)
conn.close()
try:
from interpreter.test_items import dialog_env
dialog_env.setup()
app = QApplication(['testium'])
msg = QMessageBox()
msg.setWindowFlags(Qt.WindowStaysOnTopHint)
msg.setWindowTitle(args[0])
msg.setText(args[1])
msg.setIcon(QMessageBox.Question)
msg.setStandardButtons(QMessageBox.Yes | QMessageBox.No)
auto_result = args[2] if len(args) > 2 else None
if auto_result is not None:
btn = QMessageBox.Yes if auto_result.lower() == 'yes' else QMessageBox.No
QTimer.singleShot(2000, lambda: msg.button(btn).click())
reply = msg.exec()
conn.send(reply)
except Exception as e:
print(f"dialog_question error: {e}", file=sys.stderr)
finally:
conn.close()
if hasattr(sys, "frozen"):
#all standard streams are replaced by dummy one to avoid cx_freeze flushing bug.
class dummyStream:
''' dummyStream behaves like a stream but does nothing. '''
def __init__(self): pass
def write(self,data): pass
def read(self,data): pass
def write(self, data): pass
def read(self, data): pass
def flush(self): pass
def close(self): pass
# and now redirect all default streams to this dummyStream:
sys.stdout = dummyStream()
sys.stderr = dummyStream()
sys.stdin = dummyStream()

View File

@@ -39,7 +39,9 @@ class DialogSleepWindow(QDialog, dialog_sleep_win.Ui_SleepDialogWindow):
def main(args, conn=None):
success = True
app = QApplication(sys.argv)
from interpreter.test_items import dialog_env
dialog_env.setup()
app = QApplication(['testium'])
d = DialogSleepWindow()
d.setFixedSize(379,129)
d.setWindowFlags(Qt.WindowStaysOnTopHint)

View File

@@ -3,7 +3,7 @@
################################################################################
## Form generated from reading UI file 'dialog_sleep_win.ui'
##
## Created by: Qt User Interface Compiler version 6.10.1
## Created by: Qt User Interface Compiler version 6.11.0
##
## WARNING! All changes made in this file will be lost when recompiling UI file!
################################################################################

View File

@@ -3,7 +3,7 @@
################################################################################
## Form generated from reading UI file 'dialog_value_win.ui'
##
## Created by: Qt User Interface Compiler version 6.10.1
## Created by: Qt User Interface Compiler version 6.11.0
##
## WARNING! All changes made in this file will be lost when recompiling UI file!
################################################################################

View File

@@ -2,7 +2,7 @@ import sys
import os
from PySide6.QtWidgets import (QApplication, QDialog)
from PySide6.QtCore import (Qt)
from PySide6.QtCore import Qt, QTimer
from interpreter.test_items.dialog_value_files import dialog_value_win
from multiprocessing import freeze_support
@@ -15,7 +15,9 @@ class TestDialogWindow(QDialog, dialog_value_win.Ui_Dialog):
def main(args, conn=None):
success = True
app = QApplication(args)
from interpreter.test_items import dialog_env
dialog_env.setup()
app = QApplication(['testium'])
d = TestDialogWindow()
d.setFixedSize(387,224)
d.setWindowFlags(Qt.WindowStaysOnTopHint)
@@ -23,6 +25,14 @@ def main(args, conn=None):
d.labelDialog.setText(args[1])
d.lineEdit.setText(args[2])
d.lineEdit.setFocus()
auto_result = args[3] if len(args) > 3 else None
if auto_result is not None:
auto_value = args[4] if len(args) > 4 else None
def _auto_close():
if auto_value is not None:
d.lineEdit.setText(auto_value)
d.accept() if auto_result.lower() == 'ok' else d.reject()
QTimer.singleShot(2000, _auto_close)
dres = d.exec()
if dres == QDialog.Rejected:

View File

@@ -1,4 +1,4 @@
from lib.tum_except import ETUMSyntaxError
from runtime.tum_except import ETUMSyntaxError
from interpreter.test_items.test_item import TestItem, test_run, test_data
from interpreter.test_items.test_result import TestResult, TestValue
from interpreter.test_items.item_actions.action import TestItemAction

View File

@@ -3,16 +3,45 @@ from time import sleep
import yaml
from copy import deepcopy
from interpreter.test_items.test_result import TestResult, TestValue
import libs.testium as tm
import api.testium as tm
from interpreter.utils.params import TestItemParams
from interpreter.utils.param_decl import (
Param, ParamSet, LIST, BLOCK, unknown_keys, missing_required,
)
from interpreter.utils.constants import TestItemType as cst_type
from interpreter.utils.eval import eval_to_boolean, evaluate, post_evaluate
from lib.tum_except import ETUMSyntaxError
from runtime.tum_except import ETUMSyntaxError, item_load_context
LOG_TEST_STOP = '<----- step "{}" finished'
LOG_TEST_START = '-----> step "{}" started'
# Parameters accepted by every test item, regardless of its type. Subclasses
# concatenate their own ``PARAMS`` to this set; the merged result drives
# unknown-param warnings and (later) the LSP schema export.
COMMON_PARAMS = ParamSet(
Param("name", doc="Display name shown in the GUI tree and reports."),
Param("doc", doc="Free-form documentation; surfaced in tooltips."),
Param("skipped", doc="If truthy, the step is skipped (static expression, "
"evaluated at load time)."),
Param("key", doc="Report key used to classify the result "
"(typically <test>_PASS or <test>_FAIL)."),
Param("stop_on_failure", doc="If true, abort the surrounding container on failure."),
Param("execute_on_stop", doc="If true, run this step even when its container "
"is being stopped (cleanup)."),
Param("process_result", doc="Post-evaluation expression applied to the test result."),
Param("store_result", doc="Global-dict key in which to store the test result."),
Param("expected_result", doc="Expected outcome; the step is failed if it doesn't match."),
Param("no_fail", doc="If truthy, never report a FAILURE for this step."),
Param("report", doc="Per-step reporting override."),
Param("condition", doc="Optional gating expression evaluated before each "
"run; false ⇒ the step is skipped."),
Param("steps", kind=LIST, doc="Children (for container items)."),
Param("seq_filename", doc="(internal) source .tum file of this step; injected "
"by the loader."),
)
class TestItem:
pass
@@ -20,52 +49,64 @@ class TestItem:
def test_run(f):
@wraps(f)
def wrapper(self):
if not self.skipped:
if self.enabled:
self.run_test_init()
# Conditional execution
raw_condition = self._prms.getParam(
"condition", default=None, processed=False
)
if raw_condition is None:
condition = True
else:
c = self._prms.expanse(raw_condition)
if isinstance(c, bool):
condition = c
else:
condition = False
c = False
if raw_condition == c:
msg = f'"{c}"'
else:
msg = f'"{raw_condition}" --> "{c}"'
# Do we have to skip the test because of a true condition ?
if condition:
if not raw_condition is None:
msg = "condition met: " + msg
self.result.reported = {"input_condition": msg}
print(msg)
# Test preparation
self.run_before_test()
# Test execution
f(self)
else:
msg = "condition not met: " + msg
self.result.set(TestValue.NORUN, msg)
self.result.reported = {"input_condition": msg}
self.run_test_end()
else:
self.result.set(TestValue.NORUN, "test disabled")
print("Test is disabled.")
else:
if self.skipped:
self.result.set(TestValue.NORUN, "test skipped")
print("Test is skipped.")
return self.result
if not self.enabled:
self.result.set(TestValue.NORUN, "test disabled")
print("Test is disabled.")
return self.result
self.run_test_init()
while self._is_paused:
sleep(0.2)
if self.isStopped() :
self.result.set(TestValue.NORUN, "test stopped")
print("Test is Stopped.")
self._is_stopped = False # Restore state for next run
return self.result
# Conditional execution
raw_condition = self._prms.getParam(
"condition", default=None, processed=False
)
if raw_condition is None:
condition = True
else:
c = self._prms.expanse(raw_condition)
if isinstance(c, bool):
condition = c
else:
condition = False
c = False
if raw_condition == c:
msg = f'"{c}"'
else:
msg = f'"{raw_condition}" --> "{c}"'
# Do we have to skip the test because of a true condition ?
if condition:
if not raw_condition is None:
msg = "condition met: " + msg
self.result.reported = {"input_condition": msg}
print(msg)
# Test preparation
self.run_before_test()
# Test execution
f(self)
else:
msg = "condition not met: " + msg
self.result.set(TestValue.NORUN, msg)
self.result.reported = {"input_condition": msg}
self.run_test_end()
return self.result
return wrapper
@@ -85,6 +126,11 @@ def test_data(item: TestItem, child: dict) -> dict:
class TestItem:
# Subclasses override with their own ParamSet to opt into the declarative
# validation. While ``PARAMS`` is empty / unset, the base class skips the
# unknown-param check for this item type — keeps the migration incremental.
PARAMS = None
def __init__(
self, dict_item: dict = None, parent: TestItem = None, status_queue=None, filename = ""
):
@@ -101,6 +147,7 @@ class TestItem:
self.status_queue = status_queue
self._execute_on_stop = False
self._post_eval = None
self._store_result = None
self._expected_result = None
self._no_fail = None
self._is_stopped = False
@@ -121,6 +168,13 @@ class TestItem:
# creation of the params object
self._prms = TestItemParams(dict_item, parent)
# Declarative-params validation. Only kicks in when the concrete
# subclass declares ``PARAMS`` — items not yet migrated stay
# silent. Warnings (not errors) during the migration window so
# existing .tum files don't break suddenly; will be flipped to
# errors once every item has migrated.
self._validate_declared_params(dict_item)
# getting parameters for the test item
try:
self._name = self._prms.getParam("name", default="", processed=True)
@@ -131,11 +185,11 @@ class TestItem:
if s:
try:
self.skipped = eval_to_boolean(s)
except:
except Exception as e:
raise ETUMSyntaxError(
f"'{self.cmd()}' test item named '{self.name()}':\nskipped expresion can only be a static expression as it is evaluated during loading of TUM : {s}",
self.seqFilename(),
)
) from e
# This allow disabling test item directly by using its name inside param.yaml file
elif self._name in tm.gd("skipped_test_item", []):
self.skipped = True
@@ -155,6 +209,9 @@ class TestItem:
if "process_result" in dict_item:
self._post_eval = dict_item["process_result"]
if "store_result" in dict_item:
self._store_result = dict_item["store_result"]
if "expected_result" in dict_item:
self._expected_result = dict_item["expected_result"]
@@ -164,14 +221,46 @@ class TestItem:
self.banner = LOG_TEST_START.format(self._name)
self.footer = LOG_TEST_STOP.format(self._name)
except:
except ETUMSyntaxError:
raise
except Exception as e:
raise ETUMSyntaxError(
f"The '{self.cmd()}' test item named '{self.name()}' has a missing or wrong parameter",
f"The '{self.cmd()}' test item named '{self.name()}' has an unexpected loading error: {e}",
self.seqFilename(),
)
) from e
self.result = TestResult(self, TestValue.FAILURE, "Failure by default")
def _validate_declared_params(self, dict_item):
"""Warn on unknown / missing-required params, if PARAMS is declared.
The check is opt-in per subclass: it only runs when the concrete
class sets a non-empty ``PARAMS`` attribute. Items not yet migrated
produce no diagnostics — preserving the historical "silently accept
anything" behavior until they get their declaration.
"""
if not self.PARAMS:
return
# ``self._type`` is the parent root type at this point (subclasses set
# it after super().__init__), so use the class name as a stable label
# in diagnostics. ``self._name`` was preset to the type name by every
# subclass before super() ran, which gives a useful prefix.
label = f"{type(self).__name__} '{self._name}'"
declared = COMMON_PARAMS + self.PARAMS
unknown = unknown_keys(declared, dict_item)
if unknown:
accepted = ", ".join(sorted(declared.names()))
for k in unknown:
tm.print_warn(
f"{label}: unknown parameter '{k}'. Accepted: {accepted}."
)
missing = missing_required(declared, dict_item)
for k in missing:
raise ETUMSyntaxError(
f"{label}: required parameter '{k}' is missing.",
self._seq_filename,
)
def _filter_dict_item(self, dict_item):
# Stores the content of the step to be displayed
# in the GUI
@@ -249,8 +338,6 @@ class TestItem:
self._sendStatusStarted()
if self._is_breakpoint:
self._is_paused = True
while self._is_paused:
sleep(0.2)
if self.is_container:
self.report.incLevel()
@@ -268,13 +355,13 @@ class TestItem:
if self.is_container:
self.report.decLevel()
while self._is_paused:
sleep(0.2)
# Post evaluation of the test result
self.process_result()
# expected_result treatment
self.result_expected()
# Store result in a global variable if requested (before no_fail so
# the real outcome is captured when result.value is None)
self.store_result()
# Case of the no_fail true parameter
self.process_no_fail()
@@ -302,6 +389,7 @@ class TestItem:
self.report.addTest(self, self.result, rk)
self._sendStatusFinished()
def process_result(self):
if self._post_eval is None:
return
@@ -317,6 +405,17 @@ class TestItem:
print(e)
self.result.set(TestValue.FAILURE, "Result processing failed")
def store_result(self):
if self._store_result is None:
return
var_name = self._prms.expanse(self._store_result)
if self.result.value is None:
value = str(self.result.test_result)
else:
value = self.result.value
tm.setgd(var_name, value)
print(f"Stored result in '$({var_name})': {value}")
def process_report(self, report_eval):
tm.print_debug(f"Export reported values:")
rep_eval = self._prms.expanse(report_eval)

View File

@@ -1,35 +1,41 @@
from interpreter.test_items.test_item import (TestItem, test_run)
from interpreter.test_items.test_result import TestValue
from lib.tum_except import ETUMSyntaxError
import libs.testium as tm
from runtime.tum_except import ETUMSyntaxError, item_load_context
import api.testium as tm
from interpreter.utils.constants import TestItemType as cst
from interpreter.utils.eval import evaluate
from interpreter.utils.param_decl import Param, ParamSet, LIST
class TestItemCheckValue(TestItem):
"""check item usage.
check usage:{check: {name: check my func output, steps: ['$(pfn_echo) < 5']}}
"""
PARAMS = ParamSet(
Param("values", kind=LIST, required=True,
doc="List of expressions to evaluate. Each is expanded then "
"evaluated; non-truthy results fail the check."),
# 'steps' is intentionally not redeclared here — it's the deprecated
# alias of 'values' and is already accepted by COMMON_PARAMS for
# container items. A runtime warning is emitted when 'steps' is used.
)
def __init__(self, dict_item, parent = None, status_queue=None, filename=""):
self._name = cst.TYPE_CHECK.item_name
super().__init__(dict_item, parent, status_queue, filename=filename)
self._type = cst.TYPE_CHECK
self.is_container = False
try:
with item_load_context(self.cmd(), self.name(), self.seqFilename()):
self._action_list = self._prms.getParamAll('steps', default=[], required=False)
if len(self._action_list) > 0:
tm.print_warn("'steps' argument of check test item is deprecated and is replaced by 'values'")
self._action_list += self._prms.getParamAll('values', default=[], required=False)
if len(self._action_list) <= 0:
raise ETUMSyntaxError(
f" The '{self.cmd()}' test item named '{self.name()}' must have a 'values' parameter",
f"Missing required 'values' parameter",
self.seqFilename()
)
except:
raise ETUMSyntaxError(
f"The '{self.cmd()}' test item named '{self.name()}' (a child of: '{self.parent().name()}') has a missing or wrong parameter",
self.seqFilename(),
)
@test_run
def execute(self):

View File

@@ -1,48 +1,104 @@
from multiprocessing import Process, Pipe
from interpreter.test_items.test_item import TestItem, test_run
from interpreter.test_items.test_result import TestResult, TestValue
from interpreter.test_items.dialog_choices_files import choices_dialog
import libs.testium as tm
from lib.tum_except import ETUMSyntaxError
from interpreter.test_items.test_item import test_run
from interpreter.test_items.test_result import TestValue
from interpreter.test_items.test_item_dialog_base import TestItemDialogBase, _is_text_mode, _is_interactive
from interpreter.utils.constants import TestItemType as cst
from interpreter.utils.param_decl import Param, ParamSet, BLOCK
from runtime.tum_except import item_load_context
import api.testium as tm
class TestItemChoicesDialog(TestItem):
class TestItemChoicesDialog(TestItemDialogBase):
PARAMS = ParamSet(
Param("question", required=True,
doc="Prompt shown above the list of choices."),
Param("choices", kind=BLOCK, required=True,
doc="Tree of choices: either a list of strings, or a nested "
"mapping {label: subchoices, ...} to build a multi-level menu."),
Param("icon", default=None,
doc="Default icon name shown next to each choice."),
Param("auto_result", default=None,
doc="Batch-mode selection (path or label). None ⇒ FAILURE."),
)
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
self._name = cst.TYPE_CHOICES_DLG.item_name
super().__init__(dict_item, parent, status_queue, filename=filename)
self._type = cst.TYPE_CHOICES_DLG
self.is_container = False
try:
with item_load_context(self.cmd(), self.name(), self.seqFilename()):
self._question = self._prms.getParam("question", required=True)
self._choices = self._prms.getParam("choices", required=True)
self._default_icon = self._prms.getParam(
"icon", required=False, default=None
)
except:
raise ETUMSyntaxError(
f"The '{self.cmd()}' test item named '{self.name()}' (a child of: '{self.parent().name()}') has a missing or wrong parameter",
self.seqFilename()
)
self._default_icon = self._prms.getParam("icon", required=False, default=None)
self._auto_result = self._prms.getParam("auto_result", required=False, default=None)
def _print_choices(self, choices, indent=0):
if not isinstance(choices, list):
return
for choice in choices:
name = choice.get("name", "")
desc = choice.get("description", "")
line = " " * indent + f"- {name}"
if desc:
line += f": {desc}"
print(line)
sub = choice.get("choices", None)
if sub:
self._print_choices(sub, indent + 1)
def _all_checked(self, choices):
result = []
if not isinstance(choices, list):
return result
for choice in choices:
item = {"name": choice.get("name", ""), "checked": True}
sub = choice.get("choices", None)
if sub is not None:
item["choices"] = self._all_checked(sub)
result.append(item)
return result
@test_run
def execute(self):
q = self._prms.expanse(self._question)
choices = self._prms.expanse(self._choices)
icon = self._prms.expanse(self._default_icon)
parent_conn, child_conn = Pipe()
p = Process(
target=choices_dialog.main, args=([self.name(), q, choices, icon], child_conn)
)
p.start()
val, succ = parent_conn.recv()
p.join()
if _is_text_mode():
print(f"Choices: {q}")
self._print_choices(choices)
if _is_interactive():
ans = input("Accept all? (y/n) [default: y]: ").strip().lower()
if ans in ('n', 'no'):
tm.delgd("cs_" + self._name)
self.result.set(TestValue.FAILURE, "Cancelled")
else:
val = self._all_checked(choices)
self.result.value = val
tm.setgd("cs_" + self._name, val)
self.result.set(TestValue.SUCCESS, str(val))
else:
ar = self._prms.expanse(self._auto_result) if self._auto_result is not None else None
if ar is None:
self.result.set(TestValue.FAILURE, 'Dialog not supported in batch mode')
elif ar == 'cancel':
tm.delgd("cs_" + self._name)
self.result.set(TestValue.FAILURE, "Cancelled")
else:
val = self._all_checked(choices)
self.result.value = val
tm.setgd("cs_" + self._name, val)
self.result.set(TestValue.SUCCESS, str(val))
return
from interpreter.test_items.dialog_choices_files import choices_dialog
ar = self._prms.expanse(self._auto_result) if self._auto_result is not None else None
args = [self.name(), q, choices, icon] + ([ar] if ar is not None else [])
result = self._run_dialog_with_result(choices_dialog.main, args)
if result is None:
self.result.set(TestValue.FAILURE, "Dialog subprocess exited without returning a result")
return
val, succ = result
self.result.value = val
if succ:
# The result of the test item is put into the global dict
tm.setgd("cs_" + self._name, val)
self.result.set(TestValue.SUCCESS, str(val))
else:

View File

@@ -3,13 +3,14 @@ import os
import importlib
import traceback
import libs.testium as tm
from lib.tum_except import ETUMSyntaxError
from lib.stdout_redirect import stdio_redir
import api.testium as tm
from runtime.tum_except import ETUMSyntaxError
from runtime.stdout_redirect import stdio_redir
from interpreter.test_items.test_item import test_run
from interpreter.test_items.item_actions import TestItemActions
from interpreter.test_items.item_actions.action import TestItemAction
from interpreter.utils.constants import TestItemType as cst
from interpreter.utils.param_decl import Param, ParamSet
from interpreter.test_items.test_result import TestResult, TestValue
@@ -21,6 +22,38 @@ class TestItemConsoleAction(TestItemAction):
class TestItemConsoleOpen(TestItemConsoleAction):
PARAMS = ParamSet(
Param("protocol", required=True,
doc="Transport: 'telnet', 'ssh', 'rawtcp', 'serial' or 'terminal'."),
Param("write_delay", default=0,
doc="Inter-character write delay in ms (slow devices)."),
Param("log", doc="Path to a log file capturing the console traffic."),
Param("overwrite_log", default=True,
doc="If true, truncate the log file at open; else append."),
# telnet
Param("telnet_host", doc="Hostname/IP for the telnet target."),
Param("telnet_port", default=69, doc="TCP port for telnet."),
# ssh
Param("ssh_host", doc="Hostname/IP for the SSH target."),
Param("ssh_user", doc="SSH login user."),
Param("ssh_pwd", doc="SSH password (if key-based auth is not used)."),
# rawtcp
Param("tcp_host", doc="Hostname/IP for a raw-TCP connection."),
Param("tcp_port", doc="TCP port for a raw-TCP connection."),
# serial
Param("serial_port", doc="Serial device path (e.g. /dev/ttyUSB0 or COM3)."),
Param("serial_baudrate", doc="Serial baudrate."),
Param("buffered", default=True,
doc="If true, the serial console buffers received bytes between reads."),
# terminal
Param("terminal_path",
doc="Working directory for the local terminal protocol."),
Param("shell",
doc="Shell command used for the local terminal protocol "
"(default: 'cmd.exe' on Windows, '/usr/bin/env bash' elsewhere)."),
)
def __init__(
self, action_name, dict_item, parent=None, status_queue=None, filename=""
):
@@ -283,6 +316,17 @@ class TestItemConsoleWriteLn(TestItemConsoleAction):
class TestItemConsoleReadUntil(TestItemConsoleAction):
PARAMS = ParamSet(
Param("expected", required=True,
doc="Regex matched against incoming console output until found "
"or until timeout."),
Param("timeout", default=-1,
doc="Seconds before giving up. Negative means infinite."),
Param("mute", default=False,
doc="If true, don't echo received bytes to testium's stdout/log."),
)
def __init__(
self, action_name, dict_item, parent=None, status_queue=None, filename=""
):
@@ -307,11 +351,17 @@ class TestItemConsoleReadUntil(TestItemConsoleAction):
try:
status, data = cons.read_until(
ru, timeout=read_timeout, return_data=True, mute=mute
ru, timeout=read_timeout, return_data=True, mute=mute,
should_stop=self.isStopped,
)
if status == 0:
self.result.set(TestValue.SUCCESS)
self.result.value = data
elif self.isStopped():
self.result.set(
result=TestValue.FAILURE,
message="Console read aborted on stop request",
)
else:
self.result.set(result=TestValue.FAILURE, message="No matching text")
if mute:
@@ -330,6 +380,14 @@ class TestItemConsoleReadUntil(TestItemConsoleAction):
class TestItemConsole(TestItemActions):
PARAMS = ParamSet(
Param("console_name", required=True,
doc="Identifier of the console — used by every nested action to "
"reach back the same transport. Multiple consoles can coexist "
"as long as their names differ."),
)
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
super().__init__(
cst.TYPE_CONSOLE, dict_item, parent, status_queue, filename=filename
@@ -345,17 +403,17 @@ class TestItemConsole(TestItemActions):
self.actions_token = {}
global console
console = importlib.import_module("libs.console")
console = importlib.import_module("api.console")
if not sys.platform.startswith("win"):
global console_ssh
console_ssh = importlib.import_module("libs.console_ssh")
console_ssh = importlib.import_module("api.console_ssh")
global termconsole
termconsole = importlib.import_module("libs.termconsole")
termconsole = importlib.import_module("api.termconsole")
global raw_tcp_console
raw_tcp_console = importlib.import_module("libs.raw_tcp_console")
raw_tcp_console = importlib.import_module("api.raw_tcp_console")
self.actions_token["console_name"] = self._prms.getParam(
"console_name", required=True

View File

@@ -0,0 +1,37 @@
from interpreter.test_items.test_item import TestItem, test_run
from interpreter.test_items.test_result import TestResult, TestValue
class TestItemContainer(TestItem):
"""Base class for items that run a sequence of children sequentially."""
def __init__(self, item_type, dict_item, parent=None, status_queue=None, filename=""):
self._name = item_type.item_name
super().__init__(dict_item, parent, status_queue, filename=filename)
self._type = item_type
self.is_container = True
def _run_children_sequentially(self):
"""Execute all children in order, respecting stop_on_failure and stop requests.
Returns a TestResult aggregating all children outcomes."""
i = 0
to_be_stopped = False
while not self.isStopped() and i < self.childCount() and not to_be_stopped:
result = self.child(i).execute()
if result.test_result == TestValue.FAILURE and self._stop_on_failure:
to_be_stopped = True
i += 1
if self.isStopped() or to_be_stopped:
for j in range(self.childCount()):
if self.child(j).executedOnStop() and j >= i:
self.child(j).execute()
success = TestValue.SUCCESS
for j in range(i):
if self.child(j).result.test_result == TestValue.FAILURE:
success = TestValue.FAILURE
break
stopped = self.isStopped() or to_be_stopped
return TestResult(None, success, ""), stopped

View File

@@ -1,16 +1,43 @@
import traceback
from lib.tum_except import ETUMSyntaxError, ETUMRuntimeError
from runtime.tum_except import ETUMSyntaxError, ETUMRuntimeError
from interpreter.utils.py_func_exec import PyFuncExecEngine
from interpreter.utils.api_srv import api_request
from interpreter.test_items.test_item import TestItem, test_run
from interpreter.test_items.test_result import TestResult, TestValue
import libs.testium as tm
import api.testium as tm
from interpreter.utils.params import TestItemParams
from interpreter.utils.constants import TestItemType as cst
from interpreter.utils.param_decl import Param, ParamSet, BLOCK
# Sub-block validation: 'cycle' accepts an 'exit_condition:' mapping whose
# own params are reported here so unknown keys inside it can be flagged
# during a future Block-aware diagnostic pass. For now the parent only
# declares that 'exit_condition' is an accepted top-level key.
EXIT_CONDITION_PARAMS = ParamSet(
Param("time", doc="HH:MM time of day after which the loop exits."),
Param("value", doc="Expression; when truthy the loop exits."),
Param("file", doc="Python file containing the exit-condition function."),
Param("func_name", doc="Function name in 'file' returning the exit value."),
Param("param", doc="Arguments passed to the exit function."),
Param("eval", default="",
doc="Post-evaluation expression applied to the function's return."),
)
class TestItemCycle(TestItem):
PARAMS = ParamSet(
Param("iterator",
doc="Iterable (or string expanding to one) driving the loop. "
"The current value is exposed as $(loop_param)."),
Param("exit_condition", kind=BLOCK,
doc="Optional block stopping the loop early: combine 'time', "
"'value', or a 'file'+'func_name' pair (with optional "
"'param' and 'eval')."),
)
def __init__(self, dict_cycle, parent=None, status_queue=None, filename=""):
self._name = cst.TYPE_CYCLE.item_name
super().__init__(dict_cycle, parent, status_queue, filename=filename)
@@ -207,7 +234,7 @@ then considered as 'False'""")
else:
pl = [self._currentLoop]
proc = PyFuncExecEngine(tm.gd("python_bin", ""), api_request, 10)
proc = PyFuncExecEngine(api_request, 10)
proc.start()
if not proc.wait_ready(10):
raise ETUMRuntimeError(

View File

@@ -0,0 +1,58 @@
import multiprocessing
import api.testium as tm
from interpreter.test_items.test_item import TestItem
def _is_text_mode():
return tm.text_mode()
def _is_interactive():
return bool(tm.gd("_interactive", True))
_spawn_ctx = multiprocessing.get_context('spawn')
class TestItemDialogBase(TestItem):
"""Base class for test items that launch a Qt dialog in a subprocess."""
def _cleanup_process(self, p):
if p.is_alive():
p.terminate()
p.join(timeout=0.2)
if p.is_alive():
p.kill()
p.join()
def _run_dialog(self, target, args):
"""Launch target(args) in a subprocess with no return value.
Returns the subprocess exit code.
"""
p = _spawn_ctx.Process(target=target, args=(args,))
p.start()
while p.is_alive() and not self._is_stopped:
p.join(timeout=0.5)
self._cleanup_process(p)
return p.exitcode
def _run_dialog_with_result(self, target, args):
"""Launch target(args, child_conn) in a subprocess and return what it sends.
Returns the received value, or None if stopped or if the subprocess crashed.
"""
parent_conn, child_conn = _spawn_ctx.Pipe()
p = _spawn_ctx.Process(target=target, args=(args, child_conn))
p.start()
child_conn.close()
result = None
while p.is_alive() and not self._is_stopped:
if parent_conn.poll(0.5):
try:
result = parent_conn.recv()
except EOFError:
pass
break
self._cleanup_process(p)
return result

View File

@@ -1,13 +1,21 @@
from interpreter.test_items.test_item import (TestItem, test_run)
from interpreter.test_items.test_result import (TestValue)
from interpreter.utils.constants import TestItemType as cst
from lib.tum_except import ETUMParamError, ETUMSyntaxError
from interpreter.utils.param_decl import Param, ParamSet, LIST
from runtime.tum_except import ETUMParamError, ETUMSyntaxError
import interpreter.utils.version as git
class TestItemGit(TestItem):
"""
This item expect only one parameter which is a string or list of string being the path to the git folder
"""
PARAMS = ParamSet(
Param("repo", kind=LIST, required=True,
doc="Path to a git checkout, or list of such paths. Each is "
"reported with its current version (tag + dirty state)."),
)
def __init__(self, dict_item, parent = None, status_queue=None, filename=""):
self._name = cst.TYPE_GIT.item_name
super().__init__(dict_item, parent, status_queue, filename=filename)

View File

@@ -1,10 +1,17 @@
from interpreter.test_items.test_item import (TestItem, test_run)
from interpreter.test_items.test_result import (TestResult, TestValue)
from interpreter.utils.constants import TestItemType as cst
from lib.tum_except import ETUMSyntaxError
import libs.testium as tm
from interpreter.utils.param_decl import ParamSet
from runtime.tum_except import ETUMSyntaxError
import api.testium as tm
class TestItemGroup(TestItem):
# 'group' has no item-specific parameters; 'steps' is handled by COMMON_PARAMS.
# Declaring an empty ParamSet still opts in to unknown-param validation
# (e.g. typo 'stop_on_failures').
PARAMS = ParamSet()
def __init__(self, dict_cycle, parent = None, status_queue=None, filename=""):
self._name = cst.TYPE_GROUP.item_name
super().__init__(dict_cycle, parent, status_queue, filename=filename)

View File

@@ -1,41 +1,41 @@
import os
import sys
from multiprocessing import Process, Pipe
from interpreter.test_items.test_item import TestItem, test_run
from interpreter.test_items.test_result import TestResult, TestValue
from interpreter.test_items.dialog_image_files import dialog_image
import libs.testium as tm
from interpreter.test_items.test_item import test_run
from interpreter.test_items.test_result import TestValue
from interpreter.test_items.test_item_dialog_base import TestItemDialogBase, _is_text_mode, _is_interactive
from interpreter.utils.constants import TestItemType as cst
from lib.tum_except import ETUMSyntaxError
from interpreter.utils.param_decl import Param, ParamSet
from runtime.tum_except import item_load_context
import api.testium as tm
class TestItemImageDialog(TestItem):
class TestItemImageDialog(TestItemDialogBase):
"""dialog_image item usage.
dialog_image name: Nice image, question: could you press the red button, filename: img.jpg
"""
PARAMS = ParamSet(
Param("question", required=True,
doc="Prompt shown above the image."),
Param("filename", required=True,
doc="Path to the image file (relative to the test directory or absolute)."),
Param("auto_result", default=None,
doc="Outcome used in batch/non-interactive mode. Truthy ⇒ SUCCESS, "
"None ⇒ FAILURE."),
)
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
self._name = cst.TYPE_IMAGE_DLG.item_name
super().__init__(dict_item, parent, status_queue, filename=filename)
self._type = cst.TYPE_IMAGE_DLG
self.is_container = False
try:
with item_load_context(self.cmd(), self.name(), self.seqFilename()):
self._question = self._prms.getParam("question", required=True)
self._filename = self._prms.getParam("filename", required=True)
except:
raise ETUMSyntaxError(
f"The '{self.cmd()}' test item named '{self.name()}' has a missing or wrong parameter",
self.seqFilename(),
)
self._auto_result = self._prms.getParam("auto_result", required=False, default=None)
@test_run
def execute(self):
ourpath = __file__
test_file = os.path.join(
os.path.dirname(ourpath), "dialog_image_files", "dialog_image.py"
)
q = self._prms.expanse(self._question)
image_path = self._prms.expanse(self._filename)
print("Image Displayed:\n" + q + "\n" + image_path)
@@ -43,29 +43,26 @@ class TestItemImageDialog(TestItem):
image_path = os.path.normpath(
os.path.join(tm.gd("test_directory"), image_path)
)
parent_conn, child_conn = Pipe()
p = Process(
target=dialog_image.main, args=([self.name(), q, image_path], child_conn)
)
p.start()
succ = parent_conn.recv()
p.join()
if succ:
if _is_text_mode():
if _is_interactive():
ans = input("Accept? (y/n) [default: y]: ").strip().lower()
self.result.set(TestValue.FAILURE if ans in ('n', 'no') else TestValue.SUCCESS)
else:
ar = self._prms.expanse(self._auto_result) if self._auto_result is not None else None
if ar is None:
self.result.set(TestValue.FAILURE, 'Dialog not supported in batch mode')
elif ar == 'cancel':
self.result.set(TestValue.FAILURE)
else:
self.result.set(TestValue.SUCCESS)
return
from interpreter.test_items.dialog_image_files import dialog_image
ar = self._prms.expanse(self._auto_result) if self._auto_result is not None else None
args = [self.name(), q, image_path] + ([ar] if ar is not None else [])
succ = self._run_dialog_with_result(dialog_image.main, args)
if succ is None:
self.result.set(TestValue.FAILURE, "Dialog subprocess exited without returning a result")
elif succ:
self.result.set(TestValue.SUCCESS)
else:
self.result.set(TestValue.FAILURE)
def mypath():
if hasattr(sys, "frozen"):
return os.path.dirname(sys.executable)
return os.path.dirname(__file__)
from multiprocessing import Process
if __name__ == "__main__":
p = Process(target=test_dialog.main, args=(["bob", "bab"],))
p.start()
p.join()

View File

@@ -2,7 +2,7 @@ import sys
import traceback
from random import randint
from lib.tum_except import ETUMSyntaxError
from runtime.tum_except import ETUMSyntaxError
from interpreter.test_items.test_item import TestItem, test_run
from interpreter.test_items.test_result import TestResult, TestValue
@@ -11,6 +11,7 @@ from interpreter.test_items.item_actions.action import TestItemAction
from interpreter.utils.constants import TestItemType as cst
from interpreter.utils.eval import evaluate
from interpreter.utils.param_decl import Param, ParamSet, BLOCK
from interpreter.test_items.test_item_json_rpc.jsonrpc_adapters import (
JrpcAdapter,
@@ -76,6 +77,20 @@ class TestItemJSRPCActionClose(TestItemAction):
class TestItemJSRPCActionQuery(TestItemAction):
PARAMS = ParamSet(
Param("method", required=True,
doc="JSON-RPC method name to call."),
Param("params",
doc="Parameters payload (list, dict or scalar) sent to the method."),
Param("id", default="rand",
doc="JSON-RPC request id. 'rand' (default) ⇒ a random integer is used."),
Param("no_wait", default=False,
doc="If true, send the request without waiting for a response."),
Param("timeout", default=None,
doc="Seconds to wait for a response. None ⇒ inherits the transport "
"default."),
)
def __init__(
self, action_name, dict_item, parent=None, status_queue=None, filename=""
):
@@ -105,6 +120,7 @@ class TestItemJSRPCActionQuery(TestItemAction):
jrpc_id = randint(1, (2**32) - 1)
send_only = self._prms.expanse(self._send_only)
timeout = self._prms.expanse(self._timeout)
self.token.set_should_stop(self.isStopped)
try:
success, result = self.token.query(
meth, obj, jrpc_id, send_only, timeout=timeout
@@ -128,6 +144,13 @@ class TestItemJSRPCActionQuery(TestItemAction):
class TestItemJSRPCActionReceive(TestItemAction):
PARAMS = ParamSet(
Param("id", required=True,
doc="JSON-RPC request id whose response we expect."),
Param("timeout", default=None,
doc="Seconds to wait for the response. None ⇒ transport default."),
)
def __init__(
self, action_name, dict_item, parent=None, status_queue=None, filename=""
):
@@ -146,6 +169,7 @@ class TestItemJSRPCActionReceive(TestItemAction):
def execute(self):
timeout = self._prms.expanse(self._timeout)
jrpc_id = self._prms.expanse(self._jrpc_id)
self.token.set_should_stop(self.isStopped)
try:
success, result = self.token.receive(jrpc_id, timeout)
@@ -170,6 +194,22 @@ class TestItemJSON_RPC(TestItemActions):
This item TBD
"""
PARAMS = ParamSet(
Param("console", kind=BLOCK,
doc="Console-transport block: {console_name, …}. Either 'console' "
"or 'udp' must be set."),
Param("udp", kind=BLOCK,
doc="UDP-transport block: {host, port, …}. Either 'console' or "
"'udp' must be set."),
Param("version", default="1.0",
doc="JSON-RPC protocol version ('1.0' or '2.0')."),
Param("timeout", required=True,
doc="Default seconds to wait for a JSON-RPC response across all "
"child query/receive actions."),
Param("mute", default=False,
doc="If true, don't echo wire traffic to the log."),
)
def __init__(
self, dict_item: dict, parent: TestItem = None, status_queue=None, filename=""
):

View File

@@ -2,10 +2,11 @@ import json
import socket
import re
import struct
import time
from lib.tum_except import ETUMRuntimeError
import libs.testium as tm
from libs.console import Console
from runtime.tum_except import ETUMRuntimeError
import api.testium as tm
from api.console import Console, STOP_POLL_INTERVAL
def is_ip_address(address):
@@ -45,9 +46,16 @@ class JrpcAdapter:
self._jrpc_version = version
self._mute = mute
self._timeout = timeout
# Optional callable polled by _receive() implementations to abort
# waits early when the test is being stopped. Set by the test item
# action before each query/receive call.
self._should_stop = None
if not (version == "1.0" or version == "2.0"):
raise ETUMRuntimeError("Invalid JSONRPC version passed.")
def set_should_stop(self, cb):
self._should_stop = cb
@property
def timeout(self):
return self._timeout
@@ -249,32 +257,38 @@ class JrpcUdpAdapter(JrpcAdapter):
print(f" | sent to @{self._server}:{self._snd_port}")
def _receive(self, timeout: float) -> str:
# Poll in short chunks so a stop request is honored within
# STOP_POLL_INTERVAL.
self.sock.settimeout(STOP_POLL_INTERVAL)
deadline = time.monotonic() + float(timeout)
data = None
addr = None
while True:
if self._should_stop is not None and self._should_stop():
raise ETUMRuntimeError("JSONRPC udp receive aborted on stop request.")
try:
data, addr = self.sock.recvfrom(self._bufsize)
break
except socket.timeout:
if time.monotonic() >= deadline:
raise ETUMRuntimeError(
"JSONRPC udp answer took too long. Try to increase the timeout."
)
# configures the reception timeout
self.sock.settimeout(timeout)
# Receives the answer from the server
try:
data, addr = self.sock.recvfrom(self._bufsize)
# In case of buffer overload we chose to complain
if len(data) >= self._bufsize:
raise ETUMRuntimeError(
"JSONRPC udp answer size overflow. Try to increase the bufsize"
)
# Converts binary to string
res = data.decode()
# Don't log if mute
if not self._mute:
print(f" | UDP answer: '{res}'")
print(f" | received from @{addr[0]}:{addr[1]}")
except socket.timeout:
# In case of buffer overload we chose to complain
if len(data) >= self._bufsize:
raise ETUMRuntimeError(
"JSONRPC udp answer took too long. Try to increase the timeout."
"JSONRPC udp answer size overflow. Try to increase the bufsize"
)
# Converts binary to string
res = data.decode()
# Don't log if mute
if not self._mute:
print(f" | UDP answer: '{res}'")
print(f" | received from @{addr[0]}:{addr[1]}")
return res
def _build_query(self, method: str, obj, jrpc_id: int):
@@ -339,11 +353,16 @@ class JrpcConsoleAdapter(JrpcAdapter):
def _receive(self, timeout: float) -> str:
status, data = self._cons.read_until(
self._endswith, timeout, return_data=True, mute=self._mute
self._endswith, timeout, return_data=True, mute=self._mute,
should_stop=self._should_stop,
)
# if we did not receive anything, we complain
if not status == 0:
if self._should_stop is not None and self._should_stop():
raise ETUMRuntimeError(
f"JSONRPC console receive aborted on stop request."
)
raise ETUMRuntimeError(
f"The '{self._cons.name}' console did not answer in the requested time."
)

View File

@@ -5,32 +5,35 @@ import time
from interpreter.test_items.test_item import (TestItem, test_run)
from interpreter.test_items.test_result import (TestResult, TestValue)
from lib.tum_except import ETUMSyntaxError
import libs.testium as tm
from runtime.tum_except import ETUMSyntaxError, item_load_context
import api.testium as tm
from interpreter.utils.constants import TestItemType as cst
from interpreter.utils.param_decl import Param, ParamSet, LIST
class TestItemLet(TestItem):
"""let item usage.
let values: {variable1: a, variable2: /dev/ttyUSB0, variable3: 115200}
let eval: {conditional_exec: "random.randint(1, 4)"}
"""
PARAMS = ParamSet(
Param("values", kind=LIST, required=True,
doc="Mapping (or list of single-pair mappings) of global-dict "
"key → value to set. Values are expanded at execution time."),
)
def __init__(self, dict_item, parent = None, status_queue=None, filename=""):
self._name = cst.TYPE_LET.item_name
super().__init__(dict_item, parent, status_queue, filename=filename)
self._type = cst.TYPE_LET
self.is_container = False
try:
with item_load_context(self.cmd(), self.name(), self.seqFilename()):
self._values_list = self._prms.getParamAll('values', default=[], required=False)
if len(self._values_list) <= 0:
raise ETUMSyntaxError(
f"The '{self.cmd()}' test item named '{self.name()}' must have a 'values' parameter",
f"Missing required 'values' parameter",
self.seqFilename(),
)
except:
raise ETUMSyntaxError(
f"The '{self.cmd()}' test item named '{self.name()}' has a missing or wrong parameter",
self.seqFilename(),
)
@test_run
def execute(self):

View File

@@ -4,36 +4,74 @@ import traceback
import pprint
import textwrap
from lib.tum_except import ETUMSyntaxError, ETUMRuntimeError
from runtime.tum_except import ETUMSyntaxError, ETUMRuntimeError, item_load_context
from interpreter.test_items.test_item import TestItem, test_run
from interpreter.test_items.test_result import TestValue
import libs.testium as tm
import api.testium as tm
from interpreter.utils.lua_func_exec import LuaFuncExecEngine
from interpreter.utils.api_srv import api_request
from interpreter.utils.constants import TestItemType as cst
from interpreter.utils.param_decl import Param, ParamSet, LIST
_LUA_FUNC_CONTEXTS_KEY = "_lua_func_contexts"
class TestItemLuaFunc(TestItem):
"""lua_func item usage.
func file: func_file.lua, func_name: func, param: [$(variable1), [1, 2, 3], true]
Optional: context_id: <id> — share a persistent process with other lua_func items using the same id.
"""
PARAMS = ParamSet(
Param("file", required=True,
doc="Path to the .lua file containing the function."),
Param("func_name", required=True,
doc="Name of the function to call in the file."),
Param("param", kind=LIST,
doc="Arguments passed to the function. Each entry is expanded "
"before the call. Special tokens $(loop_param) / $(loop_index) "
"resolve from the surrounding cycle."),
Param("context_id", default=None,
doc="If set, the lua_func subprocess is kept alive and reused by "
"every other lua_func item with the same context_id — enables "
"shared in-memory state between successive calls."),
)
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
self._name = cst.TYPE_LUA_FUNCTION.item_name
super().__init__(dict_item, parent, status_queue, filename=filename)
self._type = cst.TYPE_LUA_FUNCTION
self.is_container = False
try:
with item_load_context(self.cmd(), self.name(), self.seqFilename()):
self.file_name = self._prms.getParam("file", required=True)
self.func_name = self._prms.getParam("func_name", required=True)
self.params = self._prms.getParamAll("param")
except:
raise ETUMSyntaxError(
f"The '{self.cmd()}' test item named '{self.name()}' (child of '{self.parent.name()}') has a missing or wrong parameter",
self.seqFilename(),
)
# Lua functions call subprocess initialization
self._lua_func_proc = LuaFuncExecEngine(tm.gd("lua_bin", ""), api_request, 10)
self._context_id = self._prms.getParam("context_id", default=None, processed=False)
self._lua_func_proc = LuaFuncExecEngine(api_request, 10)
def _get_engine(self):
"""Return (engine, persistent). If context_id is set, use a shared persistent engine."""
if self._context_id is None:
return self._lua_func_proc, False
ctx_id = self._prms.expanse(self._context_id)
contexts = tm.gd(_LUA_FUNC_CONTEXTS_KEY, {})
if ctx_id not in contexts:
contexts[ctx_id] = LuaFuncExecEngine(api_request, 10)
tm.setgd(_LUA_FUNC_CONTEXTS_KEY, contexts)
return contexts[ctx_id], True
def stop(self):
super().stop()
# Tear down the worker so any in-flight func_call returns promptly.
# join() clears _rpc/_process so a subsequent item reusing the same
# context_id can restart the engine cleanly.
try:
engine, _ = self._get_engine()
engine.stop()
engine.join()
except Exception:
pass
@test_run
def execute(self):
@@ -48,22 +86,25 @@ class TestItemLuaFunc(TestItem):
print("Parameters list:")
print(textwrap.indent(pprint.pformat(pl), " |"))
self._lua_func_proc.start()
if not self._lua_func_proc.wait_ready(10):
raise ETUMRuntimeError(
f"""Impossible to start the external lua execution process.
engine, persistent = self._get_engine()
if not engine.is_alive():
engine.start()
if not engine.wait_ready(10):
raise ETUMRuntimeError(
f"""Impossible to start the external lua execution process.
Is the lua path correct ?
lua_bin = {tm.gd("lua_bin", "no lua path defined")}
Are "lua-sockets" and "lua-cjson" installed ?
Is the lua environnment well defined in the "LUA_PATH" and "LUA_CPATH" variables ?"""
)
)
try:
success, ret = self._lua_func_proc.func_call(self.file_name, self.func_name, pl)
success, ret = engine.func_call(self.file_name, self.func_name, pl)
finally:
# Stops lua function execution process
self._lua_func_proc.stop()
self._lua_func_proc.join()
if not persistent:
engine.stop()
engine.join()
if success == TestValue.SUCCESS:
self.result.set(TestValue.SUCCESS)
@@ -73,7 +114,6 @@ Is the lua environnment well defined in the "LUA_PATH" and "LUA_CPATH" variables
print("Returned value:")
print(textwrap.indent(pprint.pformat(res), " |"))
# The result of the func test item is put in global dir and result
tm.setgd("lfn_" + self._name, res)
self.result.value = res
@@ -84,9 +124,15 @@ Is the lua environnment well defined in the "LUA_PATH" and "LUA_CPATH" variables
return
except ConnectionAbortedError:
self.result.set(TestValue.FAILURE, "lua_func aborted on stop request")
print("lua_func aborted on stop request.")
except:
traceback.print_exception(*sys.exc_info())
self.result.set(
TestValue.FAILURE,
'Unrecoverable "py_func" item error from {}'.format(self.func_name),
)
if self.isStopped():
self.result.set(TestValue.FAILURE, "lua_func aborted on stop request")
else:
self.result.set(
TestValue.FAILURE,
'Unrecoverable "lua_func" item error from {}'.format(self.func_name),
)

View File

@@ -1,54 +1,56 @@
import os
import sys
from multiprocessing import Process, Pipe
from interpreter.test_items.test_item import (TestItem, test_run)
from interpreter.test_items.test_result import (TestValue)
from interpreter.test_items.dialog_msg_files import msg_dialog
from interpreter.test_items.test_item import test_run
from interpreter.test_items.test_result import TestValue
from interpreter.test_items.test_item_dialog_base import TestItemDialogBase, _is_text_mode, _is_interactive
from interpreter.utils.constants import TestItemType as cst
from lib.tum_except import ETUMSyntaxError
from interpreter.utils.param_decl import Param, ParamSet
from runtime.tum_except import item_load_context
class TestItemMsgDialog(TestItem):
class TestItemMsgDialog(TestItemDialogBase):
"""dialog_message item usage.
dialog_message name: Nice message, question: Open the door and press OK
"""
def __init__(self, dict_item, parent = None, status_queue=None, filename=""):
PARAMS = ParamSet(
Param("question", required=True,
doc="Message body shown to the user. Multi-line strings are supported."),
Param("auto_result", default=None,
doc="Outcome used in batch/non-interactive mode instead of waiting "
"for the user. Truthy ⇒ SUCCESS, None ⇒ FAILURE."),
)
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
self._name = cst.TYPE_MESSAGE_DLG.item_name
super().__init__(dict_item, parent, status_queue, filename=filename)
self._type = cst.TYPE_MESSAGE_DLG
self.is_container = False
try:
self._question = self._prms.getParam('question', required = True)
except:
raise ETUMSyntaxError(
f"The '{self.cmd()}' test item named '{self.name()}' has a missing or wrong parameter",
self.seqFilename(),
)
with item_load_context(self.cmd(), self.name(), self.seqFilename()):
self._question = self._prms.getParam('question', required=True)
self._auto_result = self._prms.getParam('auto_result', required=False, default=None)
@test_run
def execute(self):
ourpath = __file__
test_file = os.path.join(os.path.dirname(ourpath),
'dialog_msg_files',
'msg_dialog.py')
q = self._prms.expanse(self._question)
print("Message Displayed:\n" + q)
parent_conn, child_conn = Pipe()
p=Process(target=msg_dialog.main,
args=([self.name(), q],))
p.start()
p.join()
self.result.set(TestValue.SUCCESS)
def mypath():
if hasattr(sys, "frozen"):
return os.path.dirname(sys.executable)
return os.path.dirname(__file__)
from multiprocessing import Process
if __name__=='__main__':
p=Process(target=msg_dialog.main, args=(['bob', 'bab'],))
p.start()
p.join()
if _is_text_mode():
if _is_interactive():
input("Press Enter to continue...")
self.result.set(TestValue.SUCCESS)
else:
ar = self._prms.expanse(self._auto_result) if self._auto_result is not None else None
if ar is not None:
self.result.set(TestValue.SUCCESS)
else:
self.result.set(TestValue.FAILURE, 'Dialog not supported in batch mode')
return
from interpreter.test_items.dialog_msg_files import msg_dialog
ar = self._prms.expanse(self._auto_result) if self._auto_result is not None else None
args = [self.name(), q] + ([ar] if ar is not None else [])
exitcode = self._run_dialog(msg_dialog.main, args)
if exitcode == 0:
self.result.set(TestValue.SUCCESS)
else:
self.result.set(TestValue.FAILURE, f"Dialog subprocess exited with code {exitcode}")

View File

@@ -1,42 +1,79 @@
import os
import sys
from multiprocessing import Process, Pipe
from interpreter.test_items.test_item import (TestItem, test_run)
from interpreter.test_items.test_result import (TestResult, TestValue)
from interpreter.test_items.dialog_note_files import test_dialog
from lib.tum_except import ETUMSyntaxError
import libs.testium as tm
from interpreter.test_items.test_item import test_run
from interpreter.test_items.test_result import TestValue
from interpreter.test_items.test_item_dialog_base import TestItemDialogBase, _is_text_mode, _is_interactive
from interpreter.utils.constants import TestItemType as cst
from interpreter.utils.param_decl import Param, ParamSet
from runtime.tum_except import item_load_context
import api.testium as tm
class TestItemNoteDialog(TestItem):
def __init__(self, dict_item, parent = None, status_queue=None, filename=""):
class TestItemNoteDialog(TestItemDialogBase):
PARAMS = ParamSet(
Param("question", required=True,
doc="Prompt shown above the note input field."),
Param("auto_result", default=None,
doc="Batch-mode outcome: None ⇒ FAILURE, 'cancel' ⇒ cancelled, "
"any other truthy ⇒ SUCCESS with auto_value."),
Param("auto_value", default=None,
doc="Note text used in batch mode when auto_result is set."),
)
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
self._name = cst.TYPE_NOTE_DLG.item_name
super().__init__(dict_item, parent, status_queue, filename=filename)
self._type = cst.TYPE_NOTE_DLG
self.is_container = False
try:
self._question = self._prms.getParam('question', required = True)
except:
raise ETUMSyntaxError(
f"The '{self.cmd()}' test item named '{self.name()}' has a missing or wrong parameter",
self.seqFilename(),
)
with item_load_context(self.cmd(), self.name(), self.seqFilename()):
self._question = self._prms.getParam('question', required=True)
self._auto_result = self._prms.getParam('auto_result', required=False, default=None)
self._auto_value = self._prms.getParam('auto_value', required=False, default=None)
@test_run
def execute(self):
ourpath = __file__
test_file = os.path.join(os.path.dirname(ourpath),
'dialog_note_files',
'test_dialog.py')
q = self._prms.expanse(self._question)
print("Question:\n" + q)
parent_conn, child_conn = Pipe()
p=Process(target=test_dialog.main, args=([self.name(), q],child_conn))
p.start()
val, succ = parent_conn.recv()
p.join()
if _is_text_mode():
if _is_interactive():
print("Enter your note (type '.' on a new line to finish, empty line to cancel):")
lines = []
while True:
line = input()
if line == '.':
break
lines.append(line)
val = '\n'.join(lines)
else:
ar = self._prms.expanse(self._auto_result) if self._auto_result is not None else None
av = self._prms.expanse(self._auto_value) if self._auto_value is not None else None
if ar is None:
self.result.set(TestValue.FAILURE, 'Dialog not supported in batch mode')
return
if ar == 'cancel':
self.result.set(TestValue.FAILURE, 'Dialog cancelled')
return
val = av if av is not None else ''
tm.setgd(self.name(), val)
print("\n" + ("-" * 80) + "\n")
print("- Test note\n")
print("-" * 80 + "\n")
print(val)
print("-" * 80 + "\n")
self.result.reported = {'note': val}
if val:
self.result.set(TestValue.SUCCESS, val)
else:
self.result.set(TestValue.FAILURE, val)
return
from interpreter.test_items.dialog_note_files import test_dialog
ar = self._prms.expanse(self._auto_result) if self._auto_result is not None else None
av = self._prms.expanse(self._auto_value) if self._auto_value is not None else None
args = [self.name(), q] + ([ar, av] if ar is not None else [])
result = self._run_dialog_with_result(test_dialog.main, args)
if result is None:
self.result.set(TestValue.FAILURE, "Dialog subprocess exited without returning a result")
return
val, succ = result
tm.setgd(self.name(), val)
print("\n" + ("-" * 80) + "\n")
print("- Test note\n")
@@ -48,15 +85,3 @@ class TestItemNoteDialog(TestItem):
self.result.set(TestValue.SUCCESS, val)
else:
self.result.set(TestValue.FAILURE, val)
def mypath():
if hasattr(sys, "frozen"):
return os.path.dirname(sys.executable)
return os.path.dirname(__file__)
from multiprocessing import Process
if __name__=='__main__':
p=Process(target=test_dialog.main, args=(['bob', 'bab'],))
p.start()
p.join()

View File

@@ -0,0 +1,209 @@
import threading
from time import sleep, time
from interpreter.test_items.test_item_container import TestItemContainer
from interpreter.test_items.test_item import test_run
from interpreter.test_items.test_result import TestResult, TestValue
from interpreter.utils.constants import TestItemType as cst
from interpreter.utils.eval import eval_to_boolean
from interpreter.utils.param_decl import Param, ParamSet, LIST, BLOCK, Enum
from runtime.tum_except import ETUMSyntaxError
from runtime.string_queue import StringQueue
from runtime.stdout_redirect import stdio_redir
class TestItemParallelBranch(TestItemContainer):
"""One branch of a parallel item. Runs its children sequentially,
optionally waiting for a condition before starting."""
PARAMS = ParamSet(
Param("wait_for", kind=BLOCK,
doc="Optional block {condition, timeout} that defers the branch "
"start until the condition is truthy (or the timeout elapses)."),
)
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
super().__init__(cst.TYPE_PARALLEL_BRANCH, dict_item, parent, status_queue, filename=filename)
self._wait_condition = None
self._wait_timeout = 30
if "wait_for" in dict_item:
wf = dict_item["wait_for"]
if not isinstance(wf, dict):
raise ETUMSyntaxError(
f"'wait_for' in branch '{self.name()}' must be a dict with 'condition' and optional 'timeout'",
self.seqFilename(),
)
self._wait_condition = wf.get("condition", None)
self._wait_timeout = float(wf.get("timeout", 30))
def _wait_start(self):
"""Block until wait_for condition is True, or timeout. Returns False on timeout."""
if self._wait_condition is None:
return True
deadline = time() + self._wait_timeout
while time() < deadline:
if self.isStopped():
return False
try:
c = self._prms.expanse(self._wait_condition)
if eval_to_boolean(c):
return True
except Exception:
pass
sleep(0.1)
return False
@test_run
def execute(self):
if not self._wait_start():
self.result.set(
TestValue.FAILURE,
f"wait_for timeout ({self._wait_timeout}s): condition '{self._wait_condition}' not met",
)
return
result, stopped = self._run_children_sequentially()
if stopped:
if result.test_result == TestValue.FAILURE:
self.result.set(TestValue.FAILURE, "Branch aborted on failure")
else:
self.result.set(TestValue.NORUN, "Branch aborted on user request")
else:
self.result.set(result.test_result, "")
class TestItemParallel(TestItemContainer):
"""Runs multiple branches concurrently.
YAML:
parallel:
name: ...
sync: all # all (default): wait for every branch
# any: stop as soon as one branch finishes
stop_on_failure: false
branches:
- name: Branch A
wait_for:
condition: "'$(ready)' == 'True'"
timeout: 30
steps:
- ...
- name: Branch B
steps:
- ...
"""
PARAMS = ParamSet(
Param("branches", kind=LIST, required=True,
doc="List of branch blocks (each branch holds its own 'steps' "
"and optional 'wait_for')."),
Param("sync", kind=Enum("all", "any"), default="all",
doc="'all' (default) waits for every branch; 'any' returns as "
"soon as the first branch completes."),
)
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
branches = dict_item.get("branches", [])
if not branches:
raise ETUMSyntaxError(
f"'parallel' item requires at least one branch in 'branches'",
dict_item.get("seq_filename", ""),
)
# Inject a synthetic 'steps' key so load_test_recursively can load branches
# as TestItemParallelBranch children. The base class' _filter_dict_item
# drops 'steps'; we also drop 'branches' (overridden below) so the F1
# panel shows only the parallel's own attributes, not the duplicated
# tree of branches/steps already displayed in the test tree.
dict_item["steps"] = [{"parallel_branch": b} for b in branches]
super().__init__(cst.TYPE_PARALLEL, dict_item, parent, status_queue, filename=filename)
self._sync = str(dict_item.get("sync", "all")).lower()
if self._sync not in ("all", "any"):
raise ETUMSyntaxError(
f"'sync' must be 'all' or 'any', got '{self._sync}'",
self.seqFilename(),
)
def _filter_dict_item(self, dict_item):
c = super()._filter_dict_item(dict_item)
# Keep 'branches' so the F1 panel shows the branch list and their
# per-branch attributes (name, wait_for, condition, ...), but strip
# the 'steps' inside each branch — the steps are already visible as
# children in the test tree and would just duplicate the information.
if isinstance(c, dict) and isinstance(c.get("branches"), list):
stripped = []
for b in c["branches"]:
if isinstance(b, dict):
stripped.append({k: v for k, v in b.items() if k != "steps"})
else:
stripped.append(b)
c["branches"] = stripped
return c
def _stop_branch_recursively(self, item):
item.stop()
for i in range(item.childCount()):
self._stop_branch_recursively(item.child(i))
@test_run
def execute(self):
branch_results = [None] * self.childCount()
any_done = threading.Event()
def run_branch(idx):
branch = self.child(idx)
stdio_redir.register_thread(buffer=StringQueue(), branch=branch.name())
try:
# sync:any: if another branch already won the race, mark this
# branch as stopped so its execute() skips children but still
# goes through the normal addTest path (clean DB entry).
if self._sync == "any" and any_done.is_set():
branch.stop()
try:
result = branch.execute()
except Exception as e:
import traceback
print(f"[parallel] Branch '{branch.name()}' crashed: {e}")
traceback.print_exc()
branch.result.set(TestValue.FAILURE, f"Branch crashed: {e}")
result = branch.result
branch_results[idx] = result
# Only a branch that actually ran (SUCCESS or FAILURE) wins the
# sync:any race. A disabled or skipped branch returns NORUN
# almost instantly and must not stop legitimate branches.
if self._sync == "any" and result.test_result != TestValue.NORUN:
any_done.set()
for j in range(self.childCount()):
if j != idx:
self._stop_branch_recursively(self.child(j))
finally:
stdio_redir.unregister_thread()
threads = [
threading.Thread(target=run_branch, args=(i,), daemon=True)
for i in range(self.childCount())
]
for t in threads:
t.start()
for t in threads:
t.join()
if self._sync == "all":
# Pass if no branch failed; disabled/skipped branches (NORUN) are
# ignored, matching how Group/Cycle treat disabled children.
success = all(
r is not None and r.test_result != TestValue.FAILURE
for r in branch_results
)
else:
# Pass if at least one branch ran and succeeded.
success = any(
r is not None and r.test_result == TestValue.SUCCESS
for r in branch_results
)
self.result.set(
TestValue.SUCCESS if success else TestValue.FAILURE,
f"parallel sync={self._sync}",
)

View File

@@ -4,35 +4,74 @@ import time
import pprint
import textwrap
from lib.tum_except import ETUMSyntaxError, ETUMRuntimeError
from runtime.tum_except import ETUMSyntaxError, ETUMRuntimeError, item_load_context
from interpreter.test_items.test_item import TestItem, test_run
from interpreter.test_items.test_result import TestValue
import libs.testium as tm
import api.testium as tm
from interpreter.utils.py_func_exec import PyFuncExecEngine
from interpreter.utils.api_srv import api_request
from interpreter.utils.constants import TestItemType as cst
from interpreter.utils.param_decl import Param, ParamSet, LIST
_PY_FUNC_CONTEXTS_KEY = "_py_func_contexts"
class TestItemPyFunc(TestItem):
"""py_func item usage.
func file: func_file.py, func_name: func, param: [$(variable1), [1, 2, 3], true]
Optional: context_id: <id> — share a persistent process with other py_func items using the same id.
"""
PARAMS = ParamSet(
Param("file", required=True,
doc="Path to the .py file containing the function."),
Param("func_name", required=True,
doc="Name of the function to call in the file."),
Param("param", kind=LIST,
doc="Arguments passed to the function. Each entry is expanded "
"before the call. Special tokens $(loop_param) / $(loop_index) "
"resolve from the surrounding cycle."),
Param("context_id", default=None,
doc="If set, the py_func subprocess is kept alive and reused by "
"every other py_func item with the same context_id — enables "
"shared in-memory state between successive calls."),
)
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
self._name = cst.TYPE_PY_FUNCTION.item_name
super().__init__(dict_item, parent, status_queue, filename=filename)
self._type = cst.TYPE_PY_FUNCTION
self.is_container = False
try:
with item_load_context(self.cmd(), self.name(), self.seqFilename()):
self.file_name = self._prms.getParam("file", required=True)
self.func_name = self._prms.getParam("func_name", required=True)
self.params = self._prms.getParamAll("param")
except:
raise ETUMSyntaxError(
f"The '{self.cmd()}' test item named '{self.name()}' (child of '{self.parent.name()}') has a missing or wrong parameter",
self.seqFilename(),
)
self._py_func_proc = PyFuncExecEngine(tm.gd("python_bin", ""), api_request, 10)
self._context_id = self._prms.getParam("context_id", default=None, processed=False)
self._py_func_proc = PyFuncExecEngine(api_request, 10)
def _get_engine(self):
"""Return (engine, persistent). If context_id is set, use a shared persistent engine."""
if self._context_id is None:
return self._py_func_proc, False
ctx_id = self._prms.expanse(self._context_id)
contexts = tm.gd(_PY_FUNC_CONTEXTS_KEY, {})
if ctx_id not in contexts:
contexts[ctx_id] = PyFuncExecEngine(api_request, 10)
tm.setgd(_PY_FUNC_CONTEXTS_KEY, contexts)
return contexts[ctx_id], True
def stop(self):
super().stop()
# Tear down the worker so any in-flight func_call returns promptly.
# join() clears _rpc/_process so a subsequent item reusing the same
# context_id can restart the engine cleanly.
try:
engine, _ = self._get_engine()
engine.stop()
engine.join()
except Exception:
pass
@test_run
def execute(self):
@@ -47,20 +86,23 @@ class TestItemPyFunc(TestItem):
print("Parameters list:")
print(textwrap.indent(pprint.pformat(pl), " |"))
# start the process for executing external python
self._py_func_proc.start()
if not self._py_func_proc.wait_ready():
raise ETUMRuntimeError(
f"""Impossible to start the external python execution process.
engine, persistent = self._get_engine()
if not engine.is_alive():
engine.start()
if not engine.wait_ready():
raise ETUMRuntimeError(
f"""Impossible to start the external python execution process.
Is the python path correct ?
python_bin = {tm.gd("python_bin", "no python path defined")}"""
)
)
try:
success, ret = self._py_func_proc.func_call(self.file_name, self.func_name, pl)
success, ret = engine.func_call(self.file_name, self.func_name, pl)
finally:
# Stops python function execution process
self._py_func_proc.stop()
self._py_func_proc.join()
if not persistent:
engine.stop()
engine.join()
if success == TestValue.SUCCESS:
self.result.set(TestValue.SUCCESS)
@@ -70,7 +112,6 @@ python_bin = {tm.gd("python_bin", "no python path defined")}"""
print("Returned value:")
print(textwrap.indent(pprint.pformat(res), " |"))
# The result of the func test item is put in global dir and result
tm.setgd("pfn_" + self._name, res)
self.result.value = res
@@ -81,9 +122,15 @@ python_bin = {tm.gd("python_bin", "no python path defined")}"""
return
except ConnectionAbortedError:
self.result.set(TestValue.FAILURE, "py_func aborted on stop request")
print("py_func aborted on stop request.")
except:
traceback.print_exception(*sys.exc_info())
self.result.set(
TestValue.FAILURE,
'Unrecoverable "py_func" item error from {}'.format(self.func_name),
)
if self.isStopped():
self.result.set(TestValue.FAILURE, "py_func aborted on stop request")
else:
self.result.set(
TestValue.FAILURE,
'Unrecoverable "py_func" item error from {}'.format(self.func_name),
)

View File

@@ -1,62 +1,67 @@
import os
import sys
from multiprocessing import Process, Pipe
from PySide6.QtWidgets import QMessageBox
from interpreter.test_items.test_item import (TestItem, test_run)
from interpreter.test_items.test_result import (TestResult, TestValue)
from interpreter.test_items.dialog_question_files import question_dialog
from lib.tum_except import ETUMSyntaxError
from interpreter.test_items.test_item import test_run
from interpreter.test_items.test_result import TestValue
from interpreter.test_items.test_item_dialog_base import TestItemDialogBase, _is_text_mode, _is_interactive
from interpreter.utils.constants import TestItemType as cst
from interpreter.utils.param_decl import Param, ParamSet
from runtime.tum_except import item_load_context
class TestItemQuestionDialog(TestItem):
class TestItemQuestionDialog(TestItemDialogBase):
"""dialog_question item usage.
dialog_question name: Nice question, question: "If OK, press OK, If not, press cancel"
"""
def __init__(self, dict_item, parent = None, status_queue=None, filename=""):
PARAMS = ParamSet(
Param("question", required=True,
doc="Yes/No prompt presented to the user."),
Param("auto_result", default=None,
doc="Batch-mode answer ('yes'/'no' or truthy/falsy). None ⇒ FAILURE."),
)
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
self._name = cst.TYPE_QUESTION_DLG.item_name
super().__init__(dict_item, parent, status_queue, filename=filename)
self._type = cst.TYPE_QUESTION_DLG
self.is_container = False
try:
self._question = self._prms.getParam('question', required = True)
except:
raise ETUMSyntaxError(
f"The '{self.cmd()}' test item named '{self.name()}' has a missing or wrong parameter",
self.seqFilename(),
)
with item_load_context(self.cmd(), self.name(), self.seqFilename()):
self._question = self._prms.getParam('question', required=True)
self._auto_result = self._prms.getParam('auto_result', required=False, default=None)
@test_run
def execute(self):
ourpath = __file__
test_file = os.path.join(os.path.dirname(ourpath),
'dialog_question_files',
'question_dialog.py')
q = self._prms.expanse(self._question)
print('Question asked:\n' + q + '\n')
parent_conn, child_conn = Pipe()
p=Process(target=question_dialog.main,
args=([self.name(), q],child_conn))
p.start()
succ = parent_conn.recv()
p.join()
if _is_text_mode():
if _is_interactive():
ans = input("Answer yes (y) or no (n) [default: y]: ").strip().lower()
if ans in ('n', 'no'):
self.result.set(TestValue.FAILURE)
print('Answer: NO\n')
else:
self.result.set(TestValue.SUCCESS)
print('Answer: YES\n')
else:
ar = self._prms.expanse(self._auto_result) if self._auto_result is not None else None
if ar is None:
self.result.set(TestValue.FAILURE, 'Dialog not supported in batch mode')
elif ar in ('no', 'cancel'):
self.result.set(TestValue.FAILURE)
print('Answer: NO\n')
else:
self.result.set(TestValue.SUCCESS)
print('Answer: YES\n')
return
from interpreter.test_items.dialog_question_files import question_dialog
ar = self._prms.expanse(self._auto_result) if self._auto_result is not None else None
args = [self.name(), q] + ([ar] if ar is not None else [])
succ = self._run_dialog_with_result(question_dialog.main, args)
if succ is None:
self.result.set(TestValue.FAILURE, "Dialog subprocess exited without returning a result")
return
from PySide6.QtWidgets import QMessageBox
if succ == QMessageBox.Yes:
self.result.set(TestValue.SUCCESS)
print('Answer: YES\n')
else:
self.result.set(TestValue.FAILURE)
print('Answer: NO\n')
def mypath():
if hasattr(sys, "frozen"):
return os.path.dirname(sys.executable)
return os.path.dirname(__file__)
from multiprocessing import Process
if __name__=='__main__':
p=Process(target=test_dialog.main, args=(['bob', 'bab'],))
p.start()
p.join()

View File

@@ -1,11 +1,19 @@
from interpreter.test_items.test_item import (TestItem, test_run)
from interpreter.test_items.test_result import (TestValue)
from lib.tum_except import ETUMSyntaxError
from runtime.tum_except import ETUMSyntaxError
from interpreter.utils.constants import TestItemType as cst
from interpreter.utils.param_decl import Param, ParamSet, LIST
from interpreter.test_report.test_report import Export
class TestItemReport(TestItem):
PARAMS = ParamSet(
Param("export", kind=LIST, required=True,
doc="List of exporters to run (junit, sqlite, …). Each entry is a "
"mapping describing the exporter type and its parameters."),
)
def __init__(self, dict_item, parent = None, status_queue=None, filename=""):
self._name = cst.TYPE_REPORT.item_name
super().__init__(dict_item, parent, status_queue, filename=filename)

View File

@@ -8,9 +8,42 @@ import traceback
from interpreter.test_items.test_item import (TestItem, test_run)
from interpreter.test_items.test_result import (TestValue)
import libs.testium as tm
import api.testium as tm
from interpreter.utils.constants import TestItemType as cst
from lib.tum_except import ETUMSyntaxError, ETUMRuntimeError
from interpreter.utils.param_decl import Param, ParamSet
from runtime.tum_except import ETUMSyntaxError, ETUMRuntimeError, item_load_context
def _testium_launch_cmd():
"""Command prefix to launch a fresh testium instance, runtime-aware.
AppImage / Flatpak / PyInstaller / wheel / source all need a different
entry point than just the path to __main__.py (which may be a .py inside
a read-only bundle, or unreachable from the sub-instance's cwd).
"""
# AppImage: the env var holds the path to the .AppImage file itself.
appimage = os.environ.get("APPIMAGE")
if appimage:
return [appimage]
# Flatpak: re-launch via the Flatpak app id, but on the host side —
# the `flatpak` CLI cannot run inside our sandbox (no D-Bus access to the
# host Flatpak service, and the host binary would need host libs that are
# ABI-incompatible with the sandbox runtime). flatpak-spawn proxies the
# call to the host via org.freedesktop.Flatpak (allowed by --talk-name in
# the manifest).
if os.path.isfile("/.flatpak-info"):
return ["flatpak-spawn", "--host",
"flatpak", "run", "org.testium.Testium"]
# PyInstaller frozen exe: sys.executable is the binary itself.
if getattr(sys, "frozen", False):
return [sys.executable]
# Source / wheel: re-use the same Python with the same entry point that
# launched this instance, made absolute so cwd changes in the sub-instance
# don't break the lookup. argv[0] is either:
# - the package directory (source: `python3 src/testium ...`)
# - the console_scripts wrapper (wheel: `/usr/bin/testium`)
# Both are runnable as `python <argv0>`.
return [sys.executable, os.path.abspath(sys.argv[0])]
def nowInBetween(start, end):
@@ -25,62 +58,69 @@ def nowInBetween(start, end):
class TestItemRun(TestItem):
PARAMS = ParamSet(
Param("tum", required=True,
doc="Path to the .tum file launched in a fresh testium instance."),
Param("param_file", default="",
doc="Optional path to a param.yaml passed to the sub-instance."),
Param("log_file", default="",
doc="Path where the sub-instance writes its log."),
Param("report_file", default="",
doc="Path where the sub-instance writes its report."),
Param("start_time",
doc="HH:MM time of day after which the sub-instance may run."),
Param("end_time",
doc="HH:MM time of day after which the sub-instance no longer runs."),
Param("wait_for_exec",
doc="If true, block until the time window opens. Requires both "
"start_time and end_time."),
)
def __init__(self, dict_item, parent = None, status_queue=None, filename=""):
self._name = cst.TYPE_RUN.item_name
super().__init__(dict_item, parent, status_queue, filename=filename)
self._type = cst.TYPE_RUN
self.is_container = False
try:
self.tum_fime = self._prms.getParam('tum_fime', required=True)
with item_load_context(self.cmd(), self.name(), self.seqFilename()):
self.tum_file = self._prms.getParam('tum', required=True)
self.param_file = self._prms.getParam('param_file', default='')
self.python_bin = self._prms.getParam('python_bin', default='')
self.testium_path = self._prms.getParam('testium_path', default='')
self.log_path = self._prms.getParam('log_file', default='')
self.report_path = self._prms.getParam('report_file', default='')
self.start_time = self._prms.getParam('start_time')
self.end_time = self._prms.getParam('end_time')
self.wait_for_exec = self._prms.getParam('wait_for_exec')
except:
raise ETUMSyntaxError(
f"The '{self.cmd()}' test item named '{self.name()}' has a missing or wrong parameter",
self.seqFilename(),
)
@test_run
def execute(self):
res = -1
try:
file_path = self._prms.expanse(self.tum_fime)
file_path = self._prms.expanse(self.tum_file)
if not os.path.exists(file_path) and not os.path.isabs(file_path):
file_path = os.path.join(tm.gd('test_directory'), self.tum_fime)
file_path = os.path.join(tm.gd('test_directory'), file_path)
if not os.path.isfile(file_path):
raise ETUMRuntimeError(
'"{}" file could not be found'.format(file_path))
self.tum_fime = file_path
self.tum_file = file_path
pf = self._prms.expanse(self.param_file)
pp = self._prms.expanse(self.python_bin)
sp = self._prms.expanse(self.testium_path)
lp = self._prms.expanse(self.log_path)
rp = self._prms.expanse(self.report_path)
cmd = []
if pp != '':
cmd.append(pp)
if sp == '':
sp = os.path.join(tm.get_main_dir(), "testium.pyw")
cmd.append(sp)
if lp == '':
lp = os.path.splitext(self.tum_fime)[0] + "_" + \
datetime.utcnow().isoformat(timespec='seconds') + '.log'
cmd.append("-r")
cmd = _testium_launch_cmd()
if tm.text_mode():
cmd.append("-b")
else:
cmd.append("-r")
if lp == '':
lp = os.path.splitext(self.tum_file)[0] + "_" + \
datetime.utcnow().isoformat(timespec='seconds') + '.log'
cmd.append("-l")
cmd.append('"' + lp + '"')
if pf != '':
cmd.append("-c")
cmd.append('"' + pf + '"')
cmd.append("-l")
cmd.append('"' + lp + '"')
if rp != '':
cmd.append("-p")
cmd.append('"' + rp + '"')
cmd.append(self.tum_fime)
cmd.append(self.tum_file)
for c in cmd:
print(c, end = ' ')
@@ -95,31 +135,23 @@ class TestItemRun(TestItem):
raise ETUMRuntimeError(
'"wait_for_exec" set but not start_time or end_time')
r = None
if self.wait_for_exec:
while not nowInBetween(self.start_time, self.end_time):
sleep(60)
r = subprocess.run(
cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
r = subprocess.run(cmd)
elif self.start_time is not None and self.end_time is not None:
if nowInBetween(self.start_time, self.end_time):
r = subprocess.run(
cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
r = subprocess.run(cmd)
elif self.start_time is not None:
if self.start_time < datetime.now().time():
r = subprocess.run(
cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
r = subprocess.run(cmd)
else:
r = subprocess.run(
cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
r = subprocess.run(cmd)
if isinstance(r, subprocess.CompletedProcess):
print((r.stdout).decode())
print(r.stderr.decode())
res = r.returncode
if res >= 0:
self.result.set(TestValue.SUCCESS)
else:
self.result.set(TestValue.FAILURE,
'Test execution returned negative value.')
self.result.set(TestValue.FAILURE, 'Sub-test did not execute')
except:
traceback.print_exception(*sys.exc_info())
self.result.set(TestValue.FAILURE, 'Unrecoverable "run" item error')

View File

@@ -3,14 +3,15 @@ import importlib
import traceback
from functools import wraps
import libs.testium as tm
from lib.tum_except import ETUMSyntaxError
import api.testium as tm
from runtime.tum_except import ETUMSyntaxError, item_load_context
from interpreter.test_items.test_item import TestItem, test_run
from interpreter.test_items.test_result import TestResult, TestValue
from interpreter.test_items.item_actions import TestItemActions
from interpreter.test_items.item_actions.action import TestItemAction
from interpreter.utils.constants import TestItemType as cst
from interpreter.utils.eval import evaluate
from interpreter.utils.param_decl import Param, ParamSet, LIST
class TestItemPlotAction(TestItemAction):
@@ -21,6 +22,12 @@ class TestItemPlotAction(TestItemAction):
class TestItemPlotActionOpen(TestItemPlotAction):
PARAMS = ParamSet(
Param("log_path", default=None,
doc="Optional file to which the plot data are appended."),
)
def __init__(
self, action_name, dict_item, parent=None, status_queue=None, filename=""
):
@@ -40,6 +47,7 @@ class TestItemPlotActionOpen(TestItemPlotAction):
try:
gname = self._prms.expanse(self.token)
lpath = self._prms.expanse(self._log_path)
runtime_plot = importlib.import_module("api.runtime_plot")
gr = runtime_plot.RuntimePlot(gname, lpath)
tm.add_plot(gr)
@@ -56,6 +64,15 @@ class TestItemPlotActionOpen(TestItemPlotAction):
class TestItemPlotActionClose(TestItemPlotAction):
PARAMS = ParamSet(
Param("wait_dialog_exit", default=False,
doc="If true, the close action blocks until the user closes the "
"plot window (or timeout)."),
Param("timeout", default=-1,
doc="Seconds to wait when wait_dialog_exit is true. Negative ⇒ infinite."),
)
def __init__(
self, action_name, dict_item, parent=None, status_queue=None, filename=""
):
@@ -95,6 +112,20 @@ class TestItemPlotActionClose(TestItemPlotAction):
class TestItemPlotActionPeriodic(TestItemPlotAction):
PARAMS = ParamSet(
Param("period", required=True,
doc="Seconds between two calls of the periodic function."),
Param("file", required=True,
doc="Path to the .py file holding the periodic function."),
Param("func_name", required=True,
doc="Name of the periodic function."),
Param("param", kind=LIST,
doc="Arguments passed to the periodic function on each call."),
Param("eval", default="",
doc="Post-evaluation applied to the function's return value."),
)
def __init__(
self, action_name, dict_item, parent=None, status_queue=None, filename=""
):
@@ -108,17 +139,12 @@ class TestItemPlotActionPeriodic(TestItemPlotAction):
)
# Periodic function call
try:
with item_load_context(self.cmd(), self.name(), self.seqFilename()):
self.period = self._prms.getParam("period", required=True)
self.file_name = self._prms.getParam("file", required=True)
self.func_name = self._prms.getParam("func_name", required=True)
self.params = self._prms.getParamAll("param")
self.post_eval = self._prms.getParam("eval", default="")
except:
raise ETUMSyntaxError(
f"The '{self.cmd()}' test item named '{self.name()}' 'periodic' action settings syntax error",
self.seqFilename(),
)
@test_run
def execute(self):
@@ -173,6 +199,13 @@ class TestItemPlotActionAdd(TestItemPlotAction):
class TestItemPlotActionLastValues(TestItemPlotAction):
PARAMS = ParamSet(
Param("name", kind=LIST,
doc="List of plot variable names whose last sample is returned. "
"Result is stored in $(plv_<plot_name>) as a dict."),
)
def __init__(self, action_name, dict_item, parent=None, status_queue=None, filename=""):
super().__init__(
action_name, cst.TYPE_GRAPH_ACTION, dict_item, parent, status_queue, filename=filename
@@ -223,6 +256,13 @@ class TestItemPlotActionExport(TestItemPlotAction):
class TestItemPlot(TestItemActions):
PARAMS = ParamSet(
Param("plot_name", required=True,
doc="Identifier of the plot window — referenced by every nested "
"action and by $(plv_<plot_name>) for last-values output."),
)
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
super().__init__(
cst.TYPE_GRAPH, dict_item, parent, status_queue, filename=filename
@@ -238,6 +278,3 @@ class TestItemPlot(TestItemActions):
)
self.actions_token = self._prms.getParam("plot_name", required=True)
global runtime_plot
runtime_plot = importlib.import_module("libs.runtime_plot")

Some files were not shown because too many files have changed in this diff Show More