Compare commits
13 Commits
text_no_py
...
9db0f89522
| Author | SHA1 | Date | |
|---|---|---|---|
| 9db0f89522 | |||
| f38a24190d | |||
| b16494ef6d | |||
| b175ff4189 | |||
| d66a46736f | |||
| 1b2d427ced | |||
| be540cd304 | |||
| 476b59c6f7 | |||
| bcafbfae18 | |||
| e56a1f72c8 | |||
| 83411482b2 | |||
| a28e644621 | |||
| 4a4a70b5f6 |
97
CLAUDE.md
@@ -2,26 +2,25 @@
|
||||
|
||||
## What is testium
|
||||
|
||||
Testium is a test sequencer/runner written in Python. It executes YAML-based test scripts ("`.tum`" files) and supports three execution modes:
|
||||
Testium is a test sequencer/runner written in Python. It executes YAML-based test scripts ("`.tum`" files) and supports two execution modes:
|
||||
|
||||
- **GUI mode** (default, no flag): PySide6 Qt application (`src/testium/main_win/`)
|
||||
- **Batch mode** (`-b` / `--batch-execution`): headless, non-interactive, runs tests and exits
|
||||
- **Terminal mode** (`-m` / `--terminal`): interactive REPL in the terminal (partially implemented / work-in-progress)
|
||||
|
||||
Run from repo root: `./run.sh` (Linux) or `run.bat` / `run.ps1` (Windows).
|
||||
Direct invocation: `python3 -m src/testium [-b|-m] <test_file.tum>`
|
||||
Direct invocation: `python3 -m src/testium [-b] <test_file.tum>`
|
||||
|
||||
## Architecture
|
||||
|
||||
### Entry point
|
||||
`src/testium/__init__.py` — parses CLI args, dispatches to the three modes.
|
||||
`src/testium/__init__.py` — parses CLI args, dispatches to the two modes.
|
||||
`multiprocessing.set_start_method('spawn')` is called early (required for Linux dialog subprocesses).
|
||||
|
||||
### Core execution
|
||||
- `src/testium/interpreter/process.py` — `TestProcess(multiprocessing.Process)`: runs the test in a child process. Stdout is redirected via a `StringQueue` → pipe → parent thread (`capture_stdout`) that writes to real stdout.
|
||||
- `src/testium/interpreter/batch.py` — `Batch`: parent-side orchestrator for `-b` mode. Creates the `msg_queue`, starts `TestProcess`, waits for the "finished" signal.
|
||||
- `src/testium/interpreter/test_set.py` — `TestSet`: builds and executes the tree of test items.
|
||||
- `src/testium/interpreter/test_items/test_item*.py` — one file per test item type (check, cycle, group, let, unittest, py_func, lua_func, console, git, dialogs, report, …).
|
||||
- `src/testium/interpreter/test_items/test_item*.py` — one file per test item type (check, cycle, group, let, unittest, py_func, lua_func, console, git, dialogs, report, parallel, …).
|
||||
|
||||
### Communication channels (parent ↔ child process)
|
||||
- `msg_queue` (`multiprocessing.Queue`): carries status messages from child to parent.
|
||||
@@ -44,7 +43,7 @@ test item print()
|
||||
`src/testium/interpreter/utils/globdict.py` — shared state accessible from test scripts via `tm.gd()` / `tm.setgd()`. When `set_update_queue()` is active (during test execution), every `setgd`/`delgd` on a non-`_`-prefixed key pushes a message to `msg_queue`.
|
||||
|
||||
### Coloring (`-o` disables it)
|
||||
`src/testium/interpreter/utils/termlog.py` — `TermLog` wraps stdout with colorama-based line coloring (PASS=green, FAIL=red, WARN=yellow, …). Applied in parent process for batch/terminal modes. Auto-detects light/dark terminal background via (in order): `COLORFGBG` env var, OSC 11 query, default dark.
|
||||
`src/testium/interpreter/utils/termlog.py` — `TermLog` wraps stdout with colorama-based line coloring (PASS=green, FAIL=red, WARN=yellow, …). Applied in parent process for batch mode. Auto-detects light/dark terminal background via (in order): `COLORFGBG` env var, OSC 11 query, default dark.
|
||||
|
||||
### Dialog items in batch mode
|
||||
All dialog items (`dialog_image`, `dialog_question`, `dialog_references`, `dialog_value`, `dialog_message`, `dialog_choices`, `dialog_note`) follow this rule in non-interactive text mode (`-b`):
|
||||
@@ -54,6 +53,50 @@ All dialog items (`dialog_image`, `dialog_question`, `dialog_references`, `dialo
|
||||
|
||||
`auto_result` (and `auto_value` for value/note dialogs) is intended for the validation test suite (`test/validation/`) only.
|
||||
|
||||
### `parallel` item
|
||||
`src/testium/interpreter/test_items/test_item_parallel.py` — runs multiple branches concurrently.
|
||||
|
||||
```yaml
|
||||
- parallel:
|
||||
name: My parallel block
|
||||
sync: all # all: wait for all; any: stop as soon as one finishes
|
||||
no_fail: true # (optional) don't propagate branch failures to parent
|
||||
branches:
|
||||
- name: Branch A
|
||||
wait_for: # (optional) poll condition before starting
|
||||
condition: <| expr |>
|
||||
timeout: 10
|
||||
steps:
|
||||
- ...
|
||||
- name: Branch B
|
||||
steps:
|
||||
- ...
|
||||
```
|
||||
|
||||
- `TestItemParallel(TestItemContainer)`: mutates `dict_item["steps"]` to inject synthetic `parallel_branch` items so `load_test_recursively` loads branches normally as children.
|
||||
- `TestItemParallelBranch(TestItemContainer)`: container for one branch. `wait_for` polls every 0.1s up to `timeout` seconds before running steps.
|
||||
- `sync: any` calls `_stop_branch_recursively()` on all other branches when one *actually runs* (SUCCESS/FAILURE). A `NORUN` branch (disabled, condition not met) never wins the race.
|
||||
- Each branch runs in a daemon thread; the parent waits with `.join()`.
|
||||
- Branches stopped late (e.g. user disabled them in the GUI, or another sync:any branch already won) go through the normal `branch.stop() + branch.execute()` path so they always produce a clean DB entry via `addTest()`.
|
||||
- Exceptions raised in a branch's `execute()` are caught by `run_branch`, logged to stdout, and converted to a `FAILURE` result so they never disappear silently.
|
||||
- `sync: all` ignores `NORUN` branches when computing success (matches Group/Cycle semantics): only an actual `FAILURE` fails the parallel.
|
||||
- `TestItemSleep` is interruptible (polls `self._is_stopped` in a loop) so `sync: any` can stop slow branches quickly. `py_func` and `console` items are not interruptible; their full duration is observed before the branch returns.
|
||||
|
||||
### `TestItemContainer` base class
|
||||
`src/testium/interpreter/test_items/test_item_container.py` — shared base for Group, Cycle, Parallel, and ParallelBranch. Provides `_run_children_sequentially()` which handles stop-on-failure, `executedOnStop` items, and returns `(TestResult, stopped_bool)`.
|
||||
|
||||
### Report threading
|
||||
`src/testium/interpreter/test_report/test_report.py` — SQLite report with thread-safe writes:
|
||||
- `sqlite3.connect(..., check_same_thread=False)`
|
||||
- `self._lock = threading.Lock()` guards the SQLite `INSERT` only.
|
||||
- Per-item log capture (`stdio_redir.read()`) is naturally race-free thanks to per-thread buffers (see `StdoutProxy`).
|
||||
|
||||
### Thread-aware stdout (`StdoutProxy`)
|
||||
`src/lib/stdout_redirect.py` — when `log_stored: True`, `intercept()` installs a `StdoutProxy` as `sys.stdout`/`sys.stderr` instead of a single shared `StringQueue`. The proxy:
|
||||
- Holds one `StringQueue` per thread (registered via `register_thread(buffer=...)`). The main thread uses a default buffer; each parallel branch's thread registers its own at start and unregisters at end. `stdio_redir.read()` reads the calling thread's buffer → `addTest()` of an item running in branch X reads X's clean, non-interleaved output.
|
||||
- For the live stream (terminal in batch / GUI panel), prefixes every line emitted from a branch's thread with `[<branch_name>] ` so concurrent branches stay readable.
|
||||
- Exposes `write` / `writeln` / `flush` (Python 3.14's `unittest` calls `stream.writeln()` directly without `_WritelnDecorator`).
|
||||
|
||||
## Key files
|
||||
|
||||
| Path | Role |
|
||||
@@ -61,17 +104,36 @@ All dialog items (`dialog_image`, `dialog_question`, `dialog_references`, `dialo
|
||||
| `src/testium/__init__.py` | CLI entry, mode dispatch |
|
||||
| `src/testium/interpreter/batch.py` | `-b` mode orchestrator |
|
||||
| `src/testium/interpreter/process.py` | Child test process |
|
||||
| `src/testium/interpreter/terminal.py` | `-m` mode (WIP) |
|
||||
| `src/testium/interpreter/test_set.py` | Test tree builder/executor |
|
||||
| `src/testium/interpreter/test_items/test_item_container.py` | Base class for container items |
|
||||
| `src/testium/interpreter/test_items/test_item_parallel.py` | `parallel` and `parallel_branch` items |
|
||||
| `src/testium/interpreter/utils/globdict.py` | Global variable dict |
|
||||
| `src/testium/interpreter/utils/termlog.py` | Terminal color output |
|
||||
| `src/lib/stdout_redirect.py` | `StdioRedirect` singleton (`stdio_redir`) |
|
||||
| `src/lib/string_queue.py` | Thread-safe string buffer used for stdout redirection |
|
||||
| `src/testium/libs/testium.py` | Public API for test scripts (`tm.*`) |
|
||||
|
||||
## Known issues / WIP
|
||||
- `-m` (terminal mode) is not functional yet.
|
||||
- `text_no_pyside` branch: work in progress on text mode improvements.
|
||||
## GUI icons (main_win)
|
||||
|
||||
Icons live in `src/testium/main_win/resources/` with three theme variants:
|
||||
|
||||
| Folder | Theme index | Usage |
|
||||
|--------|-------------|-------|
|
||||
| `color/` | 0 (default) | Coloured icons |
|
||||
| `black/` | 1 | Black silhouette on transparent |
|
||||
| `white/` | 2 | White silhouette on transparent (LA mode) |
|
||||
|
||||
Icons are **64×64 PNG**. Black variants: RGBA with RGB=`(0,0,0)`, alpha varies. White variants: LA with luminance=`255`, alpha varies.
|
||||
|
||||
The mapping item-type → icon filename is in `_ITEM_CONFIG` (`src/testium/main_win/test_tree_items/test_tree_item.py`). At runtime, `icon_prefix()` returns `:/color`, `:/black`, or `:/white` (Qt resource prefix) based on the user preference.
|
||||
|
||||
All icons must be declared in `src/testium/main_win/resources/testium_core_win.qrc` (one entry per theme section). After any QRC change, regenerate the compiled resource file:
|
||||
```
|
||||
cd src/testium/main_win/resources
|
||||
pyside6-rcc testium_core_win.qrc -o testium_core_win_rc.py
|
||||
```
|
||||
|
||||
Icons are assigned once when the test file is loaded (not updated live on theme change — a file reload is required).
|
||||
|
||||
### `run` item
|
||||
`src/testium/interpreter/test_items/test_item_run.py` — launches a `.tum` file in a new testium instance (`-b` in batch mode, `-r` in GUI mode). Result:
|
||||
@@ -80,6 +142,15 @@ All dialog items (`dialog_image`, `dialog_question`, `dialog_references`, `dialo
|
||||
|
||||
The sub-test's own pass/fail result is intentionally not propagated.
|
||||
|
||||
## Recent fixes (branch `parallel_execution`)
|
||||
- `test_item_parallel.py`: new `parallel` item with `sync: all|any`, `wait_for`, daemon threads, `_stop_branch_recursively()`. Each branch thread registers a per-thread stdout buffer with `stdio_redir.register_thread(...)` so its log capture and live-output prefix work in isolation.
|
||||
- `test_item_container.py`: new `TestItemContainer` base class extracted from Group/Cycle patterns
|
||||
- `test_item_sleep.py`: interruptible loop (checks `self._is_stopped`) instead of blocking `time.sleep()` so `sync: any` can stop slow branches quickly
|
||||
- `stdout_redirect.py`: rewrote `intercept()` to install a `StdoutProxy` (thread-aware: per-thread capture buffers + branch-prefixed live output). Adds `writeln()` for Python 3.14 unittest compatibility.
|
||||
- `test_report.py`: `check_same_thread=False` + lock around the SQLite `INSERT` for parallel branch concurrency. Log capture itself is race-free thanks to per-thread buffers.
|
||||
- `__init__.py`: removed `-m`/`--terminal` mode
|
||||
- `terminal.py`: deleted
|
||||
|
||||
## Recent fixes (branch `text_no_pyside`)
|
||||
- `batch.py`: premature loop exit when `gd_update` messages (no `"id"` key) were mistaken for the "finished" signal — fix: `"id" in m and m["id"] is None`
|
||||
- `batch.py`: `control("loaded")` deadlock if `TestProcess` crashed before `cmd_th` started — fix: daemon thread + `threading.Event` + `is_alive()` polling
|
||||
@@ -88,7 +159,11 @@ The sub-test's own pass/fail result is intentionally not propagated.
|
||||
- `run` item: removed `stdout=PIPE` (caused deadlock with `multiprocessing` spawn); simplified result to SUCCESS on any completed subprocess.
|
||||
|
||||
## Validation tests
|
||||
Located in `test/validation/`. Run with `-b` flag against each `.tum` file.
|
||||
Located in `test/validation/`. Run with `-b` flag:
|
||||
```
|
||||
./run.sh -b -l mon_log.log -- test/validation/main.tum
|
||||
```
|
||||
Parallel item tests: `test/validation/items/parallel/test.tum`
|
||||
|
||||
## Dependencies
|
||||
See `src/requirements.txt`. Key ones: `pyside6`, `pyyaml`, `jinja2`, `colorama`, `gitpython`, `pexpect`, `matplotlib`.
|
||||
|
||||
66
CONTRIBUTING.md
Normal file
@@ -0,0 +1,66 @@
|
||||
# Contributing to testium
|
||||
|
||||
Thank you for your interest in contributing to testium.
|
||||
|
||||
## License of contributions
|
||||
|
||||
testium is licensed under the **European Union Public Licence v. 1.2 (EUPL-1.2)** —
|
||||
see the [LICENSE](LICENSE) file at the repository root.
|
||||
|
||||
By submitting a contribution to this project (pull request, patch, issue
|
||||
attachment, or any other form of code, documentation or media), you agree
|
||||
that your contribution is licensed to the project and to the public under the
|
||||
**same EUPL-1.2** terms (or any later version of the EUPL approved by the
|
||||
European Commission), and you certify that:
|
||||
|
||||
- you are the author of the contribution, or you have the right to submit it
|
||||
under the EUPL-1.2;
|
||||
- to the best of your knowledge, the contribution does not infringe any
|
||||
third-party intellectual-property rights;
|
||||
- the contribution may be redistributed by the project under the EUPL-1.2 and
|
||||
any compatible licence listed in the EUPL-1.2 Appendix.
|
||||
|
||||
This is the **inbound = outbound** rule: contributions come in under the same
|
||||
licence the project ships under.
|
||||
|
||||
You retain copyright on your contribution. The project does **not** ask you
|
||||
to sign a CLA or assign your copyright.
|
||||
|
||||
## SPDX header in new source files
|
||||
|
||||
When creating a new source file, please include the following header at the
|
||||
top of the file (adjust the comment marker to the file's language):
|
||||
|
||||
```python
|
||||
# SPDX-License-Identifier: EUPL-1.2
|
||||
# Copyright (c) <year> <your name>
|
||||
```
|
||||
|
||||
For existing files, keep the header that is already there.
|
||||
|
||||
## How to contribute
|
||||
|
||||
1. Open an issue describing the change you want to make (bug, feature, doc).
|
||||
2. Fork the repository, create a topic branch.
|
||||
3. Commit with a clear message (one logical change per commit).
|
||||
4. Make sure the validation suite still passes:
|
||||
```
|
||||
./run.sh -b -l mon_log.log -- test/validation/main.tum
|
||||
```
|
||||
5. Open a pull request against `main`.
|
||||
|
||||
## Coding conventions
|
||||
|
||||
- Python ≥ 3.11
|
||||
- Follow existing style in the file you are modifying
|
||||
- Add or update tests in `test/validation/` for new test items or behaviours
|
||||
- Update `CLAUDE.md` and the Sphinx manual for user-visible changes
|
||||
|
||||
## Reporting security issues
|
||||
|
||||
Please do **not** report security vulnerabilities through public GitHub
|
||||
issues. Instead, send an email to the project maintainer directly.
|
||||
|
||||
## Questions
|
||||
|
||||
Open a GitHub Discussion or an issue tagged `question`.
|
||||
315
LICENSE
Normal file
@@ -0,0 +1,315 @@
|
||||
Copyright (c) 2025-2026 François Dausseur
|
||||
|
||||
Licensed under the EUPL
|
||||
|
||||
|
||||
EUROPEAN UNION PUBLIC LICENCE v. 1.2
|
||||
EUPL © the European Union 2007, 2016
|
||||
|
||||
This European Union Public Licence (the 'EUPL') applies to the Work (as
|
||||
defined below) which is provided under the terms of this Licence. Any use of
|
||||
the Work, other than as authorised under this Licence is prohibited (to the
|
||||
extent such use is covered by a right of the copyright holder of the Work).
|
||||
|
||||
The Work is provided under the terms of this Licence when the Licensor (as
|
||||
defined below) has placed the following notice immediately following the
|
||||
copyright notice for the Work:
|
||||
|
||||
Licensed under the EUPL
|
||||
|
||||
or has expressed by any other means his willingness to license under the EUPL.
|
||||
|
||||
|
||||
1. Definitions
|
||||
|
||||
In this Licence, the following terms have the following meaning:
|
||||
|
||||
- 'The Licence': this Licence.
|
||||
|
||||
- 'The Original Work': the work or software distributed or communicated by the
|
||||
Licensor under this Licence, available as Source Code and also as Executable
|
||||
Code as the case may be.
|
||||
|
||||
- 'Derivative Works': the works or software that could be created by the
|
||||
Licensee, based upon the Original Work or modifications thereof. This Licence
|
||||
does not define the extent of modification or dependence on the Original
|
||||
Work required in order to classify a work as a Derivative Work; this extent
|
||||
is determined by copyright law applicable in the country mentioned in
|
||||
Article 15.
|
||||
|
||||
- 'The Work': the Original Work or its Derivative Works.
|
||||
|
||||
- 'The Source Code': the human-readable form of the Work which is the most
|
||||
convenient for people to study and modify.
|
||||
|
||||
- 'The Executable Code': any code which has generally been compiled and which
|
||||
is meant to be interpreted by a computer as a program.
|
||||
|
||||
- 'The Licensor': the natural or legal person that distributes or communicates
|
||||
the Work under the Licence.
|
||||
|
||||
- 'Contributor(s)': any natural or legal person who modifies the Work under
|
||||
the Licence, or otherwise contributes to the creation of a Derivative Work.
|
||||
|
||||
- 'The Licensee' or 'You': any natural or legal person who makes any usage of
|
||||
the Work under the terms of the Licence.
|
||||
|
||||
- 'Distribution' or 'Communication': any act of selling, giving, lending,
|
||||
renting, distributing, communicating, transmitting, or otherwise making
|
||||
available, online or offline, copies of the Work or providing access to its
|
||||
essential functionalities at the disposal of any other natural or legal
|
||||
person.
|
||||
|
||||
|
||||
2. Scope of the rights granted by the Licence
|
||||
|
||||
The Licensor hereby grants You a worldwide, royalty-free, non-exclusive,
|
||||
sublicensable licence to do the following, for the duration of copyright
|
||||
vested in the Original Work:
|
||||
|
||||
- use the Work in any circumstance and for all usage,
|
||||
- reproduce the Work,
|
||||
- modify the Work, and make Derivative Works based upon the Work,
|
||||
- communicate to the public, including the right to make available or display
|
||||
the Work or copies thereof to the public and perform publicly, as the case
|
||||
may be, the Work,
|
||||
- distribute the Work or copies thereof,
|
||||
- lend and rent the Work or copies thereof,
|
||||
- sublicense rights in the Work or copies thereof.
|
||||
|
||||
Those rights can be exercised on any media, supports and formats, whether now
|
||||
known or later invented, as far as the applicable law permits so.
|
||||
|
||||
In the countries where moral rights apply, the Licensor waives his right to
|
||||
exercise his moral right to the extent allowed by law in order to make
|
||||
effective the licence of the economic rights here above listed.
|
||||
|
||||
The Licensor grants to the Licensee royalty-free, non-exclusive usage rights
|
||||
to any patents held by the Licensor, to the extent necessary to make use of
|
||||
the rights granted on the Work under this Licence.
|
||||
|
||||
|
||||
3. Communication of the Source Code
|
||||
|
||||
The Licensor may provide the Work either in its Source Code form, or as
|
||||
Executable Code. If the Work is provided as Executable Code, the Licensor
|
||||
provides in addition a machine-readable copy of the Source Code of the Work
|
||||
along with each copy of the Work that the Licensor distributes or indicates,
|
||||
in a notice following the copyright notice attached to the Work, a repository
|
||||
where the Source Code is easily and freely accessible for as long as the
|
||||
Licensor continues to distribute or communicate the Work.
|
||||
|
||||
|
||||
4. Limitations on copyright
|
||||
|
||||
Nothing in this Licence is intended to deprive the Licensee of the benefits
|
||||
from any exception or limitation to the exclusive rights of the rights owners
|
||||
in the Work, of the exhaustion of those rights or of other applicable
|
||||
limitations thereto.
|
||||
|
||||
|
||||
5. Obligations of the Licensee
|
||||
|
||||
The grant of the rights mentioned above is subject to some restrictions and
|
||||
obligations imposed on the Licensee. Those obligations are the following:
|
||||
|
||||
Attribution right: The Licensee shall keep intact all copyright, patent or
|
||||
trademarks notices and all notices that refer to the Licence and to the
|
||||
disclaimer of warranties. The Licensee must include a copy of such notices
|
||||
and a copy of the Licence with every copy of the Work he/she distributes or
|
||||
communicates. The Licensee must cause any Derivative Work to carry prominent
|
||||
notices stating that the Work has been modified and the date of modification.
|
||||
|
||||
Copyleft clause: If the Licensee distributes or communicates copies of the
|
||||
Original Works or Derivative Works, this Distribution or Communication will
|
||||
be done under the terms of this Licence or of a later version of this Licence
|
||||
unless the Original Work is expressly distributed only under this version of
|
||||
the Licence — for example by communicating 'EUPL v. 1.2 only'. The Licensee
|
||||
(becoming Licensor) cannot offer or impose any additional terms or conditions
|
||||
on the Work or Derivative Work that alter or restrict the terms of the
|
||||
Licence.
|
||||
|
||||
Compatibility clause: If the Licensee Distributes or Communicates Derivative
|
||||
Works or copies thereof based upon both the Work and another work licensed
|
||||
under a Compatible Licence, this Distribution or Communication can be done
|
||||
under the terms of this Compatible Licence. For the sake of this clause,
|
||||
'Compatible Licence' refers to the licences listed in the appendix attached
|
||||
to this Licence. Should the Licensee's obligations under the Compatible
|
||||
Licence conflict with his/her obligations under this Licence, the obligations
|
||||
of the Compatible Licence shall prevail.
|
||||
|
||||
Provision of Source Code: When distributing or communicating copies of the
|
||||
Work, the Licensee will provide a machine-readable copy of the Source Code or
|
||||
indicate a repository where this Source will be easily and freely available
|
||||
for as long as the Licensee continues to distribute or communicate the Work.
|
||||
|
||||
Legal Protection: This Licence does not grant permission to use the trade
|
||||
names, trademarks, service marks, or names of the Licensor, except as
|
||||
required for reasonable and customary use in describing the origin of the
|
||||
Work and reproducing the content of the copyright notice.
|
||||
|
||||
|
||||
6. Chain of Authorship
|
||||
|
||||
The original Licensor warrants that the copyright in the Original Work
|
||||
granted hereunder is owned by him/her or licensed to him/her and that he/she
|
||||
has the power and authority to grant the Licence.
|
||||
|
||||
Each Contributor warrants that the copyright in the modifications he/she
|
||||
brings to the Work are owned by him/her or licensed to him/her and that
|
||||
he/she has the power and authority to grant the Licence.
|
||||
|
||||
Each time You accept the Licence, the original Licensor and subsequent
|
||||
Contributors grant You a licence to their contributions to the Work, under
|
||||
the terms of this Licence.
|
||||
|
||||
|
||||
7. Disclaimer of Warranty
|
||||
|
||||
The Work is a work in progress, which is continuously improved by numerous
|
||||
Contributors. It is not a finished work and may therefore contain defects or
|
||||
'bugs' inherent to this type of development.
|
||||
|
||||
For the above reason, the Work is provided under the Licence on an 'as is'
|
||||
basis and without warranties of any kind concerning the Work, including
|
||||
without limitation merchantability, fitness for a particular purpose, absence
|
||||
of defects or errors, accuracy, non-infringement of intellectual property
|
||||
rights other than copyright as stated in Article 6 of this Licence.
|
||||
|
||||
This disclaimer of warranty is an essential part of the Licence and a
|
||||
condition for the grant of any rights to the Work.
|
||||
|
||||
|
||||
8. Disclaimer of Liability
|
||||
|
||||
Except in the cases of wilful misconduct or damages directly caused to
|
||||
natural persons, the Licensor will in no event be liable for any direct or
|
||||
indirect, material or moral, damages of any kind, arising out of the Licence
|
||||
or of the use of the Work, including without limitation, damages for loss of
|
||||
goodwill, work stoppage, computer failure or malfunction, loss of data or any
|
||||
commercial damage, even if the Licensor has been advised of the possibility
|
||||
of such damage. However, the Licensor will be liable under statutory product
|
||||
liability laws as far such laws apply to the Work.
|
||||
|
||||
|
||||
9. Additional agreements
|
||||
|
||||
While distributing the Work, You may choose to conclude an additional
|
||||
agreement, defining obligations or services consistent with this Licence.
|
||||
However, if accepting obligations, You may act only on your own behalf and on
|
||||
your sole responsibility, not on behalf of the original Licensor or any other
|
||||
Contributor, and only if You agree to indemnify, defend, and hold each
|
||||
Contributor harmless for any liability incurred by, or claims asserted
|
||||
against such Contributor by the fact You have accepted any warranty or
|
||||
additional liability.
|
||||
|
||||
|
||||
10. Acceptance of the Licence
|
||||
|
||||
The provisions of this Licence can be accepted by clicking on an icon 'I
|
||||
agree' placed under the bottom of a window displaying the text of this
|
||||
Licence or by affirming consent in any other similar way, in accordance with
|
||||
the rules of applicable law. Clicking on that icon indicates your clear and
|
||||
irrevocable acceptance of this Licence and all of its terms and conditions.
|
||||
|
||||
Similarly, you irrevocably accept this Licence and all of its terms and
|
||||
conditions by exercising any rights granted to You by Article 2 of this
|
||||
Licence, such as the use of the Work, the creation by You of a Derivative
|
||||
Work or the Distribution or Communication by You of the Work or copies
|
||||
thereof.
|
||||
|
||||
|
||||
11. Information to the public
|
||||
|
||||
In case of any Distribution or Communication of the Work by means of
|
||||
electronic communication by You (for example, by offering to download the
|
||||
Work from a remote location) the distribution channel or media (for example,
|
||||
a website) must at least provide to the public the information requested by
|
||||
the applicable law regarding the Licensor, the Licence and the way it may be
|
||||
accessible, concluded, stored and reproduced by the Licensee.
|
||||
|
||||
|
||||
12. Termination of the Licence
|
||||
|
||||
The Licence and the rights granted hereunder will terminate automatically
|
||||
upon any breach by the Licensee of the terms of the Licence.
|
||||
|
||||
Such a termination will not terminate the licences of any person who has
|
||||
received the Work from the Licensee under the Licence, provided such persons
|
||||
remain in full compliance with the Licence.
|
||||
|
||||
|
||||
13. Miscellaneous
|
||||
|
||||
Without prejudice of Article 9 above, the Licence represents the complete
|
||||
agreement between the Parties as to the Work.
|
||||
|
||||
If any provision of the Licence is invalid or unenforceable under applicable
|
||||
law, this will not affect the validity or enforceability of the Licence as a
|
||||
whole. Such provision will be construed or reformed so as necessary to make
|
||||
it valid and enforceable.
|
||||
|
||||
The European Commission may publish other linguistic versions or new versions
|
||||
of this Licence or updated versions of the Appendix, so far this is required
|
||||
and reasonable, without reducing the scope of the rights granted by the
|
||||
Licence. New versions of the Licence will be published with a unique version
|
||||
number.
|
||||
|
||||
All linguistic versions of this Licence, approved by the European Commission,
|
||||
have identical value. Parties can take advantage of the linguistic version of
|
||||
their choice.
|
||||
|
||||
|
||||
14. Jurisdiction
|
||||
|
||||
Without prejudice to specific agreement between parties,
|
||||
|
||||
- any litigation resulting from the interpretation of this License, arising
|
||||
between the European Union institutions, bodies, offices or agencies, as a
|
||||
Licensor, and any Licensee, will be subject to the jurisdiction of the
|
||||
Court of Justice of the European Union, as laid down in article 272 of the
|
||||
Treaty on the Functioning of the European Union,
|
||||
|
||||
- any litigation arising between other parties and resulting from the
|
||||
interpretation of this License, will be subject to the exclusive
|
||||
jurisdiction of the competent court where the Licensor resides or conducts
|
||||
its primary business.
|
||||
|
||||
|
||||
15. Applicable Law
|
||||
|
||||
Without prejudice to specific agreement between parties,
|
||||
|
||||
- this Licence shall be governed by the law of the European Union Member
|
||||
State where the Licensor has his seat, resides or has his registered
|
||||
office,
|
||||
|
||||
- this licence shall be governed by Belgian law if the Licensor has no seat,
|
||||
residence or registered office inside a European Union Member State.
|
||||
|
||||
|
||||
Appendix
|
||||
|
||||
|
||||
'Compatible Licences' according to Article 5 EUPL are:
|
||||
|
||||
- GNU General Public License (GPL) v. 2, v. 3
|
||||
- GNU Affero General Public License (AGPL) v. 3
|
||||
- Open Software License (OSL) v. 2.1, v. 3.0
|
||||
- Eclipse Public License (EPL) v. 1.0
|
||||
- CeCILL v. 2.0, v. 2.1
|
||||
- Mozilla Public Licence (MPL) v. 2
|
||||
- GNU Lesser General Public Licence (LGPL) v. 2.1, v. 3
|
||||
- Creative Commons Attribution-ShareAlike v. 3.0 Unported (CC BY-SA 3.0) for
|
||||
works other than software
|
||||
- European Union Public Licence (EUPL) v. 1.1, v. 1.2
|
||||
- Québec Free and Open-Source Licence — Reciprocity (LiLiQ-R) or Strong
|
||||
Reciprocity (LiLiQ-R+).
|
||||
|
||||
The European Commission may update this Appendix to later versions of the
|
||||
above licences without producing a new version of the EUPL, as long as they
|
||||
provide the rights granted in Article 2 of this Licence and protect the
|
||||
covered Source Code from exclusive appropriation.
|
||||
|
||||
All other changes or additions to this Appendix require the production of a
|
||||
new EUPL version.
|
||||
14
README.md
@@ -2,6 +2,18 @@
|
||||
|
||||
[See here](doc/manual/testium_manual.pdf).
|
||||
|
||||
# License
|
||||
|
||||
Copyright (c) 2025-2026 François Dausseur.
|
||||
|
||||
testium is distributed under the **European Union Public Licence v. 1.2
|
||||
(EUPL-1.2)** — see the [LICENSE](LICENSE) file for the full text.
|
||||
|
||||
SPDX identifier: `EUPL-1.2`
|
||||
|
||||
Contributions are accepted under the same licence (inbound = outbound). See
|
||||
[CONTRIBUTING.md](CONTRIBUTING.md) for details.
|
||||
|
||||
# run testium
|
||||
|
||||
From the root path, on windows `cmd`:
|
||||
@@ -52,7 +64,7 @@ from the testium path, execute
|
||||
|
||||
## Install sphinx
|
||||
|
||||
pip install sphinx
|
||||
pip install sphinx linuxdoc
|
||||
|
||||
## Generate the doc
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
|
||||
- unittest_file:
|
||||
- unittest:
|
||||
name: Test 5
|
||||
test_file: dummy.py
|
||||
|
||||
@@ -3,7 +3,7 @@ sequence: &endurance_test
|
||||
!include endurance.tum
|
||||
|
||||
sequence:
|
||||
- unittest_file:
|
||||
- unittest:
|
||||
name: Test 3
|
||||
test_file: dummy.py
|
||||
test_method: test_01_pass
|
||||
@@ -11,6 +11,6 @@ sequence:
|
||||
iterator: 10
|
||||
steps:
|
||||
*endurance_test
|
||||
- unittest_file:
|
||||
- unittest:
|
||||
name: Test 4
|
||||
test_file: dummy.py
|
||||
|
||||
@@ -15,7 +15,7 @@ main:
|
||||
- $(reference_1)
|
||||
- $(reference_2)
|
||||
report_show_success: true
|
||||
- unittest_file:
|
||||
- unittest:
|
||||
name: Test 1
|
||||
test_file: dummy.py
|
||||
doc: |
|
||||
@@ -23,7 +23,7 @@ main:
|
||||
Voilà...
|
||||
- sleep:
|
||||
{name: Sleep between one and two, timeout: 10, dialog: true}
|
||||
- unittest_file:
|
||||
- unittest:
|
||||
{name: Test 2, test_file: dummy.py,execute_on_stop: true}
|
||||
- loop:
|
||||
name: Cycle Temperature
|
||||
|
||||
@@ -14,7 +14,7 @@ main:
|
||||
key: report-key-2
|
||||
stop_on_failure: True
|
||||
steps:
|
||||
- unittest_file:
|
||||
- unittest:
|
||||
name: unittest item
|
||||
doc: |
|
||||
The purpose of this unittest test item is to demonstrate
|
||||
@@ -41,7 +41,7 @@ main:
|
||||
param:
|
||||
- 123
|
||||
|
||||
- unittest_file:
|
||||
- unittest:
|
||||
name: Unittest item
|
||||
test_file: dummy/dummy.py
|
||||
test_method:
|
||||
@@ -98,7 +98,7 @@ main:
|
||||
name: Infine loop unittest step crashes
|
||||
stop_on_failure: True
|
||||
steps:
|
||||
- unittest_file:
|
||||
- unittest:
|
||||
name: Unittest item
|
||||
test_file: dummy/dummy.py
|
||||
test_method:
|
||||
@@ -243,7 +243,7 @@ main:
|
||||
name: Infinite loop
|
||||
skipped: True
|
||||
steps:
|
||||
- unittest_file:
|
||||
- unittest:
|
||||
name: Unittest item
|
||||
test_file: dummy/dummy.py
|
||||
test_method: test_01_pass
|
||||
|
||||
@@ -9,7 +9,7 @@ main:
|
||||
name: Test Sample number one
|
||||
version: 0.1
|
||||
steps:
|
||||
- unittest_file:
|
||||
- unittest:
|
||||
name: Unittest item
|
||||
test_file: dummy/unittest_str.py
|
||||
doc: Unittest test
|
||||
@@ -88,7 +88,7 @@ main:
|
||||
name: cycle item
|
||||
iterator : 3
|
||||
steps:
|
||||
- unittest_file:
|
||||
- unittest:
|
||||
name: Unittest item
|
||||
test_file: dummy/dummy.py
|
||||
test_method: test_01_pass
|
||||
@@ -99,7 +99,7 @@ main:
|
||||
name: cycle item
|
||||
iterator : 3
|
||||
steps:
|
||||
- unittest_file:
|
||||
- unittest:
|
||||
name: Unittest item
|
||||
test_file: dummy/dummy.py
|
||||
test_method: test_01_pass
|
||||
|
||||
@@ -3,7 +3,7 @@ Command Line Interface
|
||||
|
||||
.. code-block:: text
|
||||
|
||||
usage: testium.pyw [-h] [--version] [-b] [-m] [-c CONFIG_FILE [CONFIG_FILE ...]] [-r] [-l LOG_FILE]
|
||||
usage: testium.pyw [-h] [--version] [-b] [-c CONFIG_FILE [CONFIG_FILE ...]] [-r] [-l LOG_FILE]
|
||||
[-d DEFINE [DEFINE ...]] [-p REPORT_FILE] [-t {sqlite,json,junit,html,text}]
|
||||
[-n REPORT_PATTERN [REPORT_PATTERN ...]] [-i INCLUDE_PATH [INCLUDE_PATH ...]] [-o] [-g]
|
||||
[test_file]
|
||||
@@ -16,9 +16,8 @@ Command Line Interface
|
||||
--version Returns the version of testium
|
||||
-b, --batch-execution
|
||||
Executes the test in batch mode
|
||||
-m, --terminal Starts terminal mode
|
||||
-c CONFIG_FILE [CONFIG_FILE ...], --config-file CONFIG_FILE [CONFIG_FILE ...]
|
||||
-o, --no-color Deactivates stdout colors in batch and terminal mode
|
||||
-o, --no-color Deactivates stdout colors in batch mode
|
||||
Configuration file
|
||||
-r, --run-and-close Runs the test then closes the application
|
||||
-l LOG_FILE, --log-file LOG_FILE
|
||||
@@ -45,17 +44,10 @@ Returns what's in the previous section.
|
||||
|
||||
Executes the test in text mode. No need to have QT installed in that case.
|
||||
|
||||
``-m, --terminal``
|
||||
------------------
|
||||
|
||||
Starts a testium interactive console. It allows to run commands and sub-tests manually
|
||||
in a console.
|
||||
|
||||
|
||||
``-o, --no-color``
|
||||
------------------
|
||||
|
||||
Switch allowing to disable the colored output in terminal or batch modes.
|
||||
Switch allowing to disable the colored output in batch mode.
|
||||
|
||||
``-c, --config-file``
|
||||
---------------------
|
||||
|
||||
@@ -23,23 +23,3 @@ graphical interface.
|
||||
:caption: call a test in batch mode
|
||||
|
||||
testium -b test/my_test/main.tum
|
||||
|
||||
Terminal mode
|
||||
-------------
|
||||
|
||||
The terminal mode starts *testium* in interactive mode. From this console, some tests and
|
||||
sequences of tests can be called interactively.
|
||||
|
||||
.. code-block:: text
|
||||
:caption: call a test in terminal mode
|
||||
|
||||
$ testium -m
|
||||
Configuration file loaded: /my/execution/path/param.yaml
|
||||
[...]
|
||||
================================================================================
|
||||
====== Test configuration
|
||||
================================================================================
|
||||
Test executed with testium : 2.4.0 (binary release)
|
||||
|
||||
|
||||
(testium)~
|
||||
|
||||
@@ -75,7 +75,7 @@ a tooltip on the test row.
|
||||
name: Test example
|
||||
steps:
|
||||
|
||||
- unittest_file:
|
||||
- unittest:
|
||||
name: unittest item
|
||||
doc: |
|
||||
The purpose of this unittest test item is to demonstrate
|
||||
@@ -93,4 +93,4 @@ See illustration in :numref:`Figure %s<doc-illustration>`.
|
||||
Unittest
|
||||
^^^^^^^^^
|
||||
|
||||
For ``unittest_file`` type test items, the python docstring of the test method is used as documentation.
|
||||
For ``unittest`` type test items, the python docstring of the test method is used as documentation.
|
||||
|
||||
@@ -6,7 +6,7 @@ This software is developed in python and it implements the Qt6 graphical framewo
|
||||
|
||||
It has been developed since 2013 with production and development testing in mind.
|
||||
|
||||
It's function is to automate the execution of tests. It can be invoked either as command line terminal application or as a graphical interface application.
|
||||
It's function is to automate the execution of tests. It can be invoked either as command line application or as a graphical interface application.
|
||||
|
||||
Tests reports generation and customization are also in this tool's scope.
|
||||
|
||||
|
||||
@@ -10,7 +10,7 @@ This element is of the following form:
|
||||
name: Group Item
|
||||
condition: <| "$(OS)" == "Linux" |>
|
||||
steps:
|
||||
- unittest_file:
|
||||
- unittest:
|
||||
test_file: test_prod_alpha_13.py
|
||||
test_method:
|
||||
...
|
||||
|
||||
@@ -33,7 +33,7 @@ if not provided is given in the table as well.
|
||||
| | | It depends on the test item to take it |
|
||||
| | | into account or not. |
|
||||
| | | For example it makes sense to use it |
|
||||
| | | for ``unittest_file`` test type |
|
||||
| | | for ``unittest`` test type |
|
||||
| | | because it can contain many sub-tests, |
|
||||
| | | but not for sleep test type. |
|
||||
| | | In cycles, it means that the child |
|
||||
|
||||
@@ -12,7 +12,7 @@ This element is of the following form:
|
||||
name: Cycle Temperature
|
||||
iterator: 10
|
||||
steps:
|
||||
- unittest_file:
|
||||
- unittest:
|
||||
test_file: test_prod_rio6_8093.py
|
||||
- py_func:
|
||||
name: function test item
|
||||
|
||||
97
doc/manual/sphinx/source/test_items/parallel_test_item.rst
Normal file
@@ -0,0 +1,97 @@
|
||||
.. _sec_parallel_item:
|
||||
|
||||
**parallel** test item
|
||||
============================================================
|
||||
|
||||
This element is of the following form:
|
||||
|
||||
.. code-block:: yaml
|
||||
:caption: ``parallel`` test item usage example
|
||||
|
||||
- parallel:
|
||||
name: My parallel block
|
||||
sync: all
|
||||
branches:
|
||||
- name: Branch A
|
||||
steps:
|
||||
- py_func:
|
||||
name: Long operation
|
||||
file: long_op.py
|
||||
func_name: do_work
|
||||
- name: Branch B
|
||||
wait_for:
|
||||
condition: <| "$(ready_flag)" == "True" |>
|
||||
timeout: 30
|
||||
steps:
|
||||
- let:
|
||||
name: Mark done
|
||||
values:
|
||||
- branch_b_done: true
|
||||
|
||||
The ``parallel`` element runs several sequences of items concurrently. Each
|
||||
inner sequence is called a *branch* and runs in its own thread. The parent
|
||||
test item waits for branches to finish according to the ``sync`` policy.
|
||||
|
||||
Attributes
|
||||
--------------------
|
||||
|
||||
* ``branches``: required. A list of branches to execute concurrently. Each
|
||||
branch has a ``name`` and a ``steps`` list (same structure as a ``group``
|
||||
item). It can also declare a ``wait_for`` precondition (see below).
|
||||
* ``sync``: optional, defaults to ``all``.
|
||||
|
||||
* ``all``: the parallel item completes when *every* branch has finished.
|
||||
The result is ``PASS`` if no branch returned ``FAIL`` (skipped or
|
||||
disabled branches are ignored, like in ``group``); otherwise ``FAIL``.
|
||||
* ``any``: the parallel item completes as soon as the *first* branch
|
||||
finishes. The remaining branches are stopped (their next test items
|
||||
are not executed). The result is ``PASS`` if at least one branch
|
||||
succeeded.
|
||||
|
||||
* ``no_fail``: optional. When ``true``, a ``FAIL`` result is forced to
|
||||
``PASS`` for the parallel item itself (same semantics as for any test
|
||||
item). Branches keep their own result.
|
||||
|
||||
Branch attributes
|
||||
--------------------
|
||||
|
||||
Each entry of ``branches`` is a dict with the following attributes:
|
||||
|
||||
* ``name``: required. The branch name. Used in reports and as a prefix
|
||||
in the live log output (each line printed by the branch is prefixed
|
||||
with the branch name in square brackets, e.g. ``[Branch A]``, so
|
||||
concurrent branches stay readable).
|
||||
* ``steps``: required. The list of test items executed sequentially
|
||||
inside the branch.
|
||||
* ``wait_for``: optional. Forces the branch to wait until a condition is
|
||||
met before running its steps. If the timeout elapses, the branch
|
||||
returns ``FAIL`` (the steps are not run). Sub-attributes:
|
||||
|
||||
* ``condition``: a testium expression evaluated repeatedly (every
|
||||
100 ms) until it returns ``True``.
|
||||
* ``timeout``: maximum wait, in seconds. Defaults to 30.
|
||||
|
||||
Reporting
|
||||
--------------------
|
||||
|
||||
Each branch produces its own row in the SQLite report (with type
|
||||
``Parallel branch``), in addition to the parent ``Parallel`` row. The
|
||||
``log`` column of each row contains only the output emitted from that
|
||||
branch's thread, so logs are never mixed between concurrent branches.
|
||||
|
||||
In the live (terminal / GUI) output, lines emitted from a branch are
|
||||
prefixed with the branch name in square brackets (e.g. ``[Branch A]``).
|
||||
The prefix is not stored in the SQLite log column.
|
||||
|
||||
Notes
|
||||
--------------------
|
||||
|
||||
* A ``sleep`` item inside a branch is interruptible: if another
|
||||
``sync: any`` branch wins the race, slow ``sleep`` items are aborted
|
||||
within ~50 ms.
|
||||
* A ``py_func`` or ``console`` item inside a branch is **not**
|
||||
interruptible: a ``sync: any`` stop will only take effect after the
|
||||
current item returns. The branch will then skip its remaining steps.
|
||||
* When a user disables a branch in the GUI tree, the branch returns
|
||||
``SKIP`` instantly without affecting the others (it does *not* win a
|
||||
``sync: any`` race).
|
||||
@@ -6,9 +6,9 @@ This test item executes a new instance of testium with the specified ``.tum`` fi
|
||||
* In **batch mode** (``-b``): the sub-instance is started with ``-b``.
|
||||
* In **GUI mode**: the sub-instance is started with ``-r`` (run and close).
|
||||
|
||||
The item result is **SUCCESS** if the sub-instance launched and ran to completion,
|
||||
The item result is **PASS** if the sub-instance launched and ran to completion,
|
||||
regardless of whether the sub-tests passed or failed.
|
||||
It is **FAILURE** if the file could not be found, the sub-instance could not be
|
||||
It is **FAIL** if the file could not be found, the sub-instance could not be
|
||||
launched, or the time window was not reached (see ``start_time`` / ``end_time``).
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
@@ -1,15 +1,15 @@
|
||||
**unittest_file** test item
|
||||
**unittest** test item
|
||||
============================================================
|
||||
|
||||
unittest_file test item allows the execution of unittest test script which
|
||||
unittest test item allows the execution of unittest test script which
|
||||
is part of python standard libraries.
|
||||
|
||||
The tum file prototype is as followed:
|
||||
|
||||
.. code-block:: yaml
|
||||
:caption: ``unittest_file`` test item usage example
|
||||
:caption: ``unittest`` test item usage example
|
||||
|
||||
- unittest_file:
|
||||
- unittest:
|
||||
name: unitTest test item
|
||||
test_file: unitTestScript.py
|
||||
test_method:
|
||||
@@ -23,7 +23,7 @@ Beside common test items attributes, unittest test item has specific attribute,
|
||||
|
||||
* ``test_file``: it is the name (and eventually path) of the unittest file
|
||||
to be processed.
|
||||
* ``test_method``: it is an optional unittest_file test sub-item. If one or more
|
||||
* ``test_method``: it is an optional unittest test sub-item. If one or more
|
||||
elements are present, the unittest python script file is parsed and only
|
||||
the corresponding methods are included in the test tree. Otherwise, all
|
||||
the test methods are included in the test tree.
|
||||
@@ -78,4 +78,4 @@ Here is an example how to use the console module from python ``unittest``.
|
||||
|
||||
@classmethod
|
||||
def tearDownClass(cls):
|
||||
cls.consA0.close()
|
||||
cls.consA0.close()
|
||||
@@ -255,11 +255,12 @@ step list attributes.
|
||||
test_items/let_test_item.rst
|
||||
test_items/loop_test_item.rst
|
||||
test_items/lua_func_test_item.rst
|
||||
test_items/parallel_test_item.rst
|
||||
test_items/plot_test_item.rst
|
||||
test_items/report_test_item.rst
|
||||
test_items/run_test_item.rst
|
||||
test_items/sleep_test_item.rst
|
||||
test_items/unittest_file_test_item.rst
|
||||
test_items/unittest_test_item.rst
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -1,8 +1,88 @@
|
||||
import sys
|
||||
import threading
|
||||
from threading import (Thread, Event)
|
||||
from lib.string_queue import StringQueue
|
||||
from time import (sleep)
|
||||
|
||||
|
||||
class StdoutProxy:
|
||||
"""Thread-aware stdout proxy.
|
||||
|
||||
Each writing thread can be associated with:
|
||||
- a per-thread buffer (StringQueue) where its writes are captured for the
|
||||
per-item SQLite log column;
|
||||
- a 'branch' label, used to prefix each line in the live (parent-visible)
|
||||
output stream so concurrent branches are easy to read.
|
||||
|
||||
Threads with no association fall back to the default buffer (the "main"
|
||||
thread's buffer) and write to live output without prefix.
|
||||
"""
|
||||
|
||||
def __init__(self, live_stream, default_buffer):
|
||||
self.live_stream = live_stream
|
||||
self.default_buffer = default_buffer
|
||||
self._buffers = {}
|
||||
self._branches = {}
|
||||
self._lock = threading.Lock()
|
||||
|
||||
def register(self, tid=None, buffer=None, branch=None):
|
||||
if tid is None:
|
||||
tid = threading.get_ident()
|
||||
with self._lock:
|
||||
if buffer is not None:
|
||||
self._buffers[tid] = buffer
|
||||
if branch is not None:
|
||||
self._branches[tid] = branch
|
||||
|
||||
def unregister(self, tid=None):
|
||||
if tid is None:
|
||||
tid = threading.get_ident()
|
||||
with self._lock:
|
||||
self._buffers.pop(tid, None)
|
||||
self._branches.pop(tid, None)
|
||||
|
||||
def get_buffer(self, tid=None):
|
||||
if tid is None:
|
||||
tid = threading.get_ident()
|
||||
with self._lock:
|
||||
return self._buffers.get(tid, self.default_buffer)
|
||||
|
||||
def write(self, s):
|
||||
if not s:
|
||||
return
|
||||
tid = threading.get_ident()
|
||||
with self._lock:
|
||||
buf = self._buffers.get(tid, self.default_buffer)
|
||||
branch = self._branches.get(tid)
|
||||
# Per-thread capture: clean, no prefix
|
||||
buf.write(s)
|
||||
# Live stream: prefix each line with the branch label
|
||||
if branch:
|
||||
self.live_stream.write(self._prefix(s, f'[{branch}] '))
|
||||
else:
|
||||
self.live_stream.write(s)
|
||||
|
||||
@staticmethod
|
||||
def _prefix(s, prefix):
|
||||
ends_nl = s.endswith('\n')
|
||||
body = s[:-1] if ends_nl else s
|
||||
if body == '':
|
||||
return s
|
||||
prefixed = '\n'.join(prefix + line for line in body.split('\n'))
|
||||
if ends_nl:
|
||||
prefixed += '\n'
|
||||
return prefixed
|
||||
|
||||
def writeln(self, s=''):
|
||||
self.write(s + '\n')
|
||||
|
||||
def flush(self):
|
||||
try:
|
||||
self.live_stream.flush()
|
||||
except AttributeError:
|
||||
pass
|
||||
|
||||
|
||||
class StdioRedirect:
|
||||
|
||||
def __init__(self):
|
||||
@@ -28,48 +108,38 @@ class StdioRedirect:
|
||||
|
||||
def intercept(self):
|
||||
if not self.spy_enabled:
|
||||
self.thr_started = Event()
|
||||
self.log_buf = StringQueue()
|
||||
self.in_stream = StringQueue()
|
||||
self.stop_output = Event()
|
||||
self.thrd_out = Thread(target=self.interceptStdOut)
|
||||
self.thrd_out.daemon = True
|
||||
sys.stdout = self.in_stream
|
||||
sys.stderr = self.in_stream
|
||||
self.stream = self.in_stream
|
||||
self.thrd_out.start()
|
||||
self.thr_started.wait()
|
||||
self.log_buf = StringQueue() # default buffer (main thread)
|
||||
self.proxy = StdoutProxy(self.out_stream, self.log_buf)
|
||||
sys.stdout = self.proxy
|
||||
sys.stderr = self.proxy
|
||||
self.stream = self.proxy
|
||||
self.spy_enabled = True
|
||||
|
||||
|
||||
def stop(self):
|
||||
if self.spy_enabled:
|
||||
sys.stdout = self.out_stream
|
||||
sys.stderr = self.out_stream
|
||||
self.stream = self.out_stream
|
||||
self.stop_output.set()
|
||||
self.thrd_out.join()
|
||||
del self.log_buf
|
||||
del self.in_stream
|
||||
del self.stop_output
|
||||
del self.thrd_out
|
||||
del self.thr_started
|
||||
del self.proxy
|
||||
|
||||
self.spy_enabled = False
|
||||
|
||||
def interceptStdOut(self):
|
||||
self.thr_started.set()
|
||||
while not self.stop_output.is_set():
|
||||
data = self.in_stream.read()
|
||||
self.log_buf.write(data)
|
||||
self.out_stream.write(data)
|
||||
if data == '':
|
||||
sleep(0.1)
|
||||
|
||||
def read(self):
|
||||
ret = ''
|
||||
"""Read accumulated content from the calling thread's buffer."""
|
||||
if not self.spy_enabled:
|
||||
return ''
|
||||
return self.proxy.get_buffer().read()
|
||||
|
||||
def register_thread(self, buffer=None, branch=None):
|
||||
"""Register the calling thread's per-thread buffer and/or branch label."""
|
||||
if self.spy_enabled:
|
||||
ret = self.log_buf.read()
|
||||
return ret
|
||||
self.proxy.register(buffer=buffer, branch=branch)
|
||||
|
||||
def unregister_thread(self):
|
||||
"""Drop the calling thread's registration."""
|
||||
if self.spy_enabled:
|
||||
self.proxy.unregister()
|
||||
|
||||
|
||||
stdio_redir = StdioRedirect()
|
||||
|
||||
@@ -1,8 +1,5 @@
|
||||
# from io import (StringIO, SEEK_SET, SEEK_CUR, SEEK_END)
|
||||
from multiprocessing import Queue
|
||||
from queue import (Empty)
|
||||
from threading import (Thread, Event, Condition)
|
||||
from threading import Lock
|
||||
|
||||
|
||||
class StringQueue(object):
|
||||
""" Class used to store the buffered consoles data:
|
||||
|
||||
@@ -8,9 +8,12 @@ requires-python = ">=3.11"
|
||||
authors = [
|
||||
{name = "François Dausseur", email = "francois@beafrancois.fr"},
|
||||
]
|
||||
license = "EUPL-1.2"
|
||||
license-files = ["../LICENSE"]
|
||||
classifiers = [
|
||||
"Development Status :: 5 - Production/Stable",
|
||||
"Programming Language :: Python"
|
||||
"Programming Language :: Python",
|
||||
"License :: OSI Approved :: European Union Public Licence 1.2 (EUPL 1.2)",
|
||||
]
|
||||
dependencies = [
|
||||
"setuptools",
|
||||
|
||||
@@ -21,10 +21,8 @@ def main():
|
||||
help="Returns the version of testium", action='store_true')
|
||||
parser.add_argument("-b", "--batch-execution",
|
||||
help="Executes the test in batch mode", action='store_true')
|
||||
parser.add_argument("-m", "--terminal",
|
||||
help="Starts terminal mode", action='store_true')
|
||||
parser.add_argument("-o", "--no-color",
|
||||
help="Deactivates stdout colors in batch and terminal mode", action='store_true')
|
||||
help="Deactivates stdout colors in batch mode", action='store_true')
|
||||
parser.add_argument("-c", "--config-file", help="Configuration file",
|
||||
nargs='+',
|
||||
default=[])
|
||||
@@ -95,30 +93,6 @@ def main():
|
||||
from interpreter.utils.version import get_testium_version
|
||||
print(get_testium_version())
|
||||
|
||||
elif args.terminal:
|
||||
import select
|
||||
from interpreter.terminal import Terminal
|
||||
|
||||
if (lf != '') or (rf != '') or (tf != '') or (pn != []):
|
||||
print('"-l", "-p", "-t", "-n" options are not supported in this mode.')
|
||||
|
||||
t = Terminal(os.getcwd(), cf, defines, args.no_color, text_mode=True)
|
||||
|
||||
loop = 1
|
||||
while loop:
|
||||
try:
|
||||
loop = 0
|
||||
t.cmdloop()
|
||||
except KeyboardInterrupt:
|
||||
print("\n<ctrl-c>")
|
||||
loop = 1
|
||||
except Exception as exc:
|
||||
if str(exc) == 'quit':
|
||||
break
|
||||
print(exc)
|
||||
loop = 1
|
||||
|
||||
|
||||
elif args.batch_execution:
|
||||
if (lf != ''):
|
||||
print('"-l" option is not supported in this mode.')
|
||||
|
||||
@@ -1,246 +0,0 @@
|
||||
try:
|
||||
import readline
|
||||
except:
|
||||
pass
|
||||
from cmd import Cmd
|
||||
import os
|
||||
import sys
|
||||
from yaml import load, Loader
|
||||
import functools
|
||||
import platform
|
||||
import types
|
||||
import inspect
|
||||
|
||||
# test modules
|
||||
from interpreter.utils.test_init import (
|
||||
env_init, prepare_global, set_standard_gd_keys,
|
||||
update_global, test_run_init, test_run_header, load_test)
|
||||
from interpreter.utils.globdict import (global_dict)
|
||||
import libs.testium as tm
|
||||
from interpreter.utils.constants import TestItemType as cst
|
||||
from interpreter.test_report.test_report import TestReport
|
||||
|
||||
|
||||
class FakeQueue:
|
||||
def put(self, arg):
|
||||
pass
|
||||
|
||||
|
||||
def func(self, args):
|
||||
if not args.startswith("{"):
|
||||
args = "{"+args+"}"
|
||||
y = load(args, Loader)
|
||||
obj = self.current_item(y, status_queue=FakeQueue())
|
||||
obj.report = self.report
|
||||
res = obj.execute()
|
||||
if not (res.value is None):
|
||||
print('result : {}'.format(res.value))
|
||||
print(res.test_result)
|
||||
|
||||
|
||||
class Terminal(Cmd):
|
||||
SUPPORTED_TESTS = [
|
||||
cst.TYPE_SLEEP,
|
||||
cst.TYPE_LET,
|
||||
cst.TYPE_PY_FUNCTION,
|
||||
cst.TYPE_LUA_FUNCTION,
|
||||
cst.TYPE_CONSOLE,
|
||||
cst.TYPE_IMAGE_DLG,
|
||||
cst.TYPE_MESSAGE_DLG,
|
||||
cst.TYPE_QUESTION_DLG,
|
||||
cst.TYPE_VALUE_DLG,
|
||||
]
|
||||
|
||||
SUPPORTED_GROUPS = [
|
||||
cst.TYPE_GROUP,
|
||||
cst.TYPE_CYCLE
|
||||
]
|
||||
|
||||
def __init__(self, working_dir, config_files, defines, no_color, text_mode=False):
|
||||
super().__init__()
|
||||
self.working_dir = working_dir
|
||||
self.config_files = config_files
|
||||
self.current_item = None
|
||||
report = TestReport(None)
|
||||
self.report = report
|
||||
|
||||
env_init()
|
||||
prepare_global()
|
||||
# Define the builtin variables
|
||||
set_standard_gd_keys("Unnamed", self.working_dir, '', config_files)
|
||||
update_global([], defines)
|
||||
if text_mode:
|
||||
tm.setgd("_text_mode", True)
|
||||
|
||||
# creation of the functions
|
||||
for tst in self.SUPPORTED_TESTS:
|
||||
meth_name = "do_" + tst.item_cmd
|
||||
# copy of the function
|
||||
f = types.FunctionType(func.__code__, func.__globals__, name=meth_name,
|
||||
argdefs=func.__defaults__,
|
||||
closure=func.__closure__)
|
||||
f = functools.update_wrapper(f, func)
|
||||
f.__kwdefaults__ = func.__kwdefaults__
|
||||
f.__doc__ = tst.item_class.__doc__
|
||||
setattr(self, meth_name, types.MethodType(f, self))
|
||||
|
||||
test_run_init()
|
||||
self.prompt = "(testium)~ "
|
||||
|
||||
# display header
|
||||
print(test_run_header())
|
||||
# redirect output
|
||||
|
||||
if 'Linux' in platform.system() and not no_color:
|
||||
from lib.stdout_redirect import stdio_redir
|
||||
try:
|
||||
from interpreter.utils.termlog import TermLog
|
||||
stdio_redir.redirect(TermLog(sys.stdout))
|
||||
except ModuleNotFoundError:
|
||||
tm.print_info('Colored console not supported by the system.' +
|
||||
' If you want it, please install colorama module')
|
||||
|
||||
def precmd(self, line: str) -> str:
|
||||
c = line.split(" ", 1)[0].strip()
|
||||
self.current_item = None
|
||||
for tst in self.SUPPORTED_TESTS:
|
||||
if c == tst.item_cmd:
|
||||
self.current_item = tst.item_class
|
||||
break
|
||||
return line
|
||||
|
||||
def load_test_recursively(self, tree_parent, parent_seq, status_queue):
|
||||
try:
|
||||
parent_seq_name = parent_seq['name']
|
||||
except KeyError:
|
||||
parent_seq['name'] = "sequence"
|
||||
except TypeError:
|
||||
raise Exception("Syntax error in an item of type {} which is a child of {}".format(
|
||||
tree_parent.type(), tree_parent.parent().name()))
|
||||
try:
|
||||
parent_seq_actions = parent_seq['steps']
|
||||
except KeyError:
|
||||
raise Exception(' No action list found for "%s" sequence'
|
||||
% (parent_seq_name))
|
||||
# if action is a dictionary , we assume it is a single action
|
||||
# that has not been nested in a list, so do it
|
||||
if isinstance(parent_seq_actions, (dict)):
|
||||
parent_seq_actions = [parent_seq_actions]
|
||||
if not isinstance(parent_seq_actions, (list, tuple)):
|
||||
raise Exception('Actions list not valid.')
|
||||
# first we merged to the same level 'sequence dict entries and list within the list
|
||||
counter = 0
|
||||
test_dir = tm.gd('test_directory')
|
||||
while (counter < len(parent_seq_actions)):
|
||||
action = parent_seq_actions[counter]
|
||||
# if action is a list raise up to the the same level,
|
||||
# ie insert action element into the parent_seq_actions
|
||||
if isinstance(action, (list, tuple)):
|
||||
parent_seq_actions[counter:counter+1] = action
|
||||
continue
|
||||
# if action is a NoneType skip and continue
|
||||
# (when pointing to an unused alias for instance)
|
||||
if action is None:
|
||||
counter += 1
|
||||
continue
|
||||
# if action is a sequence we insert its entry into the action list
|
||||
if 'sequence' in action:
|
||||
parent_seq_actions[counter:counter+1] = action['sequence']
|
||||
continue
|
||||
else:
|
||||
executed = False
|
||||
for it in [*self.SUPPORTED_TESTS, *self.SUPPORTED_GROUPS]:
|
||||
if it.item_cmd in action:
|
||||
executed = True
|
||||
item = (it.item_class)(action[it.item_cmd],
|
||||
tree_parent,
|
||||
status_queue)
|
||||
# check for sequence type:
|
||||
if it.item_cmd == cst.TYPE_UNITTEST_FILE.item_cmd:
|
||||
item.setTestDir(test_dir)
|
||||
item.load()
|
||||
elif ((it.item_cmd == cst.TYPE_CYCLE.item_cmd) or
|
||||
(it.item_cmd == cst.TYPE_GROUP.item_cmd)):
|
||||
self.load_test_recursively(
|
||||
item, action[it.item_cmd], status_queue)
|
||||
|
||||
if not executed:
|
||||
raise Exception('action type is not known "{}"'.format(
|
||||
list(action.keys())[0]))
|
||||
|
||||
counter += 1
|
||||
|
||||
def __setReportRecursively(self, parent):
|
||||
for i in range(parent.childCount()):
|
||||
parent.child(i).report = self.report
|
||||
self.__setReportRecursively(parent.child(i))
|
||||
|
||||
def setReport(self, root_item):
|
||||
root_item.report = self.report
|
||||
self.__setReportRecursively(root_item)
|
||||
|
||||
def get_names(self):
|
||||
memb = inspect.getmembers(self)
|
||||
return [n[0] for n in memb if (inspect.ismethod(n[1]) and n[0].startswith("do_"))]
|
||||
|
||||
def do_load(self, args):
|
||||
"""load function.
|
||||
|
||||
This function loads and executes a testium sub-script.
|
||||
|
||||
The loaded sequence can't be a main testium script ("testium -b" option is
|
||||
defined for such a usage).
|
||||
|
||||
Accepted files are with extension "*.tum".
|
||||
|
||||
usage:
|
||||
load path/to/my/sequence.tum
|
||||
"""
|
||||
file = args.strip()
|
||||
suff = file[-4:]
|
||||
if not suff in ['.tum']:
|
||||
raise Exception('Wrong input file extension')
|
||||
|
||||
if not (os.path.exists(file) and os.path.isfile(file)):
|
||||
raise Exception(
|
||||
'"{}" does not exist or is not a file.'.format(file))
|
||||
|
||||
d, _ = load_test(file)
|
||||
if not isinstance(d, list):
|
||||
raise Exception(
|
||||
"The file root object must be a list. A \"main\" tum can't be loaded from here (use batch mode instead).")
|
||||
|
||||
if (len(d) == 1) and isinstance(d[0], dict) and (not d[0].get('sequence', None) is None):
|
||||
d = d[0]['sequence']
|
||||
|
||||
sq = FakeQueue()
|
||||
root_item = (cst.TYPE_ROOT.item_class)(
|
||||
dict_item={'steps': d}, status_queue=sq)
|
||||
self.load_test_recursively(root_item, {'steps': d}, sq)
|
||||
self.setReport(root_item)
|
||||
res = root_item.execute()
|
||||
if not (res.value is None):
|
||||
print('"{}" execution overall result: {}'.format(file, res.value))
|
||||
print(res.test_result)
|
||||
|
||||
def do_gd(self, args):
|
||||
"""Variables lists and values.
|
||||
|
||||
usage:
|
||||
gd
|
||||
gd home
|
||||
"""
|
||||
if args != '':
|
||||
res = tm.gd(args, None)
|
||||
if res is None:
|
||||
raise Exception(
|
||||
'the variable: "{}" has not been found.'.format(args))
|
||||
print(res)
|
||||
return
|
||||
|
||||
for k in global_dict.keys():
|
||||
print('{}: {}'.format(str(k), str(global_dict[k])))
|
||||
|
||||
def do_quit(self, args):
|
||||
'''Quit the application.'''
|
||||
raise Exception('quit')
|
||||
@@ -3,7 +3,7 @@
|
||||
################################################################################
|
||||
## Form generated from reading UI file 'choices_dialog_win.ui'
|
||||
##
|
||||
## Created by: Qt User Interface Compiler version 6.10.1
|
||||
## Created by: Qt User Interface Compiler version 6.11.0
|
||||
##
|
||||
## WARNING! All changes made in this file will be lost when recompiling UI file!
|
||||
################################################################################
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
################################################################################
|
||||
## Form generated from reading UI file 'dialog_image_win.ui'
|
||||
##
|
||||
## Created by: Qt User Interface Compiler version 6.10.1
|
||||
## Created by: Qt User Interface Compiler version 6.11.0
|
||||
##
|
||||
## WARNING! All changes made in this file will be lost when recompiling UI file!
|
||||
################################################################################
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
################################################################################
|
||||
## Form generated from reading UI file 'dialog_note_win.ui'
|
||||
##
|
||||
## Created by: Qt User Interface Compiler version 6.10.1
|
||||
## Created by: Qt User Interface Compiler version 6.11.0
|
||||
##
|
||||
## WARNING! All changes made in this file will be lost when recompiling UI file!
|
||||
################################################################################
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
################################################################################
|
||||
## Form generated from reading UI file 'dialog_sleep_win.ui'
|
||||
##
|
||||
## Created by: Qt User Interface Compiler version 6.10.1
|
||||
## Created by: Qt User Interface Compiler version 6.11.0
|
||||
##
|
||||
## WARNING! All changes made in this file will be lost when recompiling UI file!
|
||||
################################################################################
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
################################################################################
|
||||
## Form generated from reading UI file 'dialog_value_win.ui'
|
||||
##
|
||||
## Created by: Qt User Interface Compiler version 6.10.1
|
||||
## Created by: Qt User Interface Compiler version 6.11.0
|
||||
##
|
||||
## WARNING! All changes made in this file will be lost when recompiling UI file!
|
||||
################################################################################
|
||||
|
||||
37
src/testium/interpreter/test_items/test_item_container.py
Normal file
@@ -0,0 +1,37 @@
|
||||
from interpreter.test_items.test_item import TestItem, test_run
|
||||
from interpreter.test_items.test_result import TestResult, TestValue
|
||||
|
||||
|
||||
class TestItemContainer(TestItem):
|
||||
"""Base class for items that run a sequence of children sequentially."""
|
||||
|
||||
def __init__(self, item_type, dict_item, parent=None, status_queue=None, filename=""):
|
||||
self._name = item_type.item_name
|
||||
super().__init__(dict_item, parent, status_queue, filename=filename)
|
||||
self._type = item_type
|
||||
self.is_container = True
|
||||
|
||||
def _run_children_sequentially(self):
|
||||
"""Execute all children in order, respecting stop_on_failure and stop requests.
|
||||
Returns a TestResult aggregating all children outcomes."""
|
||||
i = 0
|
||||
to_be_stopped = False
|
||||
while not self.isStopped() and i < self.childCount() and not to_be_stopped:
|
||||
result = self.child(i).execute()
|
||||
if result.test_result == TestValue.FAILURE and self._stop_on_failure:
|
||||
to_be_stopped = True
|
||||
i += 1
|
||||
|
||||
if self.isStopped() or to_be_stopped:
|
||||
for j in range(self.childCount()):
|
||||
if self.child(j).executedOnStop() and j >= i:
|
||||
self.child(j).execute()
|
||||
|
||||
success = TestValue.SUCCESS
|
||||
for j in range(i):
|
||||
if self.child(j).result.test_result == TestValue.FAILURE:
|
||||
success = TestValue.FAILURE
|
||||
break
|
||||
|
||||
stopped = self.isStopped() or to_be_stopped
|
||||
return TestResult(None, success, ""), stopped
|
||||
193
src/testium/interpreter/test_items/test_item_parallel.py
Normal file
@@ -0,0 +1,193 @@
|
||||
import threading
|
||||
from time import sleep, time
|
||||
|
||||
from interpreter.test_items.test_item_container import TestItemContainer
|
||||
from interpreter.test_items.test_item import test_run
|
||||
from interpreter.test_items.test_result import TestResult, TestValue
|
||||
from interpreter.utils.constants import TestItemType as cst
|
||||
from interpreter.utils.eval import eval_to_boolean
|
||||
from lib.tum_except import ETUMSyntaxError
|
||||
from lib.string_queue import StringQueue
|
||||
from lib.stdout_redirect import stdio_redir
|
||||
|
||||
|
||||
class TestItemParallelBranch(TestItemContainer):
|
||||
"""One branch of a parallel item. Runs its children sequentially,
|
||||
optionally waiting for a condition before starting."""
|
||||
|
||||
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
|
||||
super().__init__(cst.TYPE_PARALLEL_BRANCH, dict_item, parent, status_queue, filename=filename)
|
||||
self._wait_condition = None
|
||||
self._wait_timeout = 30
|
||||
if "wait_for" in dict_item:
|
||||
wf = dict_item["wait_for"]
|
||||
if not isinstance(wf, dict):
|
||||
raise ETUMSyntaxError(
|
||||
f"'wait_for' in branch '{self.name()}' must be a dict with 'condition' and optional 'timeout'",
|
||||
self.seqFilename(),
|
||||
)
|
||||
self._wait_condition = wf.get("condition", None)
|
||||
self._wait_timeout = float(wf.get("timeout", 30))
|
||||
|
||||
def _wait_start(self):
|
||||
"""Block until wait_for condition is True, or timeout. Returns False on timeout."""
|
||||
if self._wait_condition is None:
|
||||
return True
|
||||
deadline = time() + self._wait_timeout
|
||||
while time() < deadline:
|
||||
if self.isStopped():
|
||||
return False
|
||||
try:
|
||||
c = self._prms.expanse(self._wait_condition)
|
||||
if eval_to_boolean(c):
|
||||
return True
|
||||
except Exception:
|
||||
pass
|
||||
sleep(0.1)
|
||||
return False
|
||||
|
||||
@test_run
|
||||
def execute(self):
|
||||
if not self._wait_start():
|
||||
self.result.set(
|
||||
TestValue.FAILURE,
|
||||
f"wait_for timeout ({self._wait_timeout}s): condition '{self._wait_condition}' not met",
|
||||
)
|
||||
return
|
||||
|
||||
result, stopped = self._run_children_sequentially()
|
||||
|
||||
if stopped:
|
||||
if result.test_result == TestValue.FAILURE:
|
||||
self.result.set(TestValue.FAILURE, "Branch aborted on failure")
|
||||
else:
|
||||
self.result.set(TestValue.NORUN, "Branch aborted on user request")
|
||||
else:
|
||||
self.result.set(result.test_result, "")
|
||||
|
||||
|
||||
class TestItemParallel(TestItemContainer):
|
||||
"""Runs multiple branches concurrently.
|
||||
|
||||
YAML:
|
||||
parallel:
|
||||
name: ...
|
||||
sync: all # all (default): wait for every branch
|
||||
# any: stop as soon as one branch finishes
|
||||
stop_on_failure: false
|
||||
branches:
|
||||
- name: Branch A
|
||||
wait_for:
|
||||
condition: "'$(ready)' == 'True'"
|
||||
timeout: 30
|
||||
steps:
|
||||
- ...
|
||||
- name: Branch B
|
||||
steps:
|
||||
- ...
|
||||
"""
|
||||
|
||||
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
|
||||
branches = dict_item.get("branches", [])
|
||||
if not branches:
|
||||
raise ETUMSyntaxError(
|
||||
f"'parallel' item requires at least one branch in 'branches'",
|
||||
dict_item.get("seq_filename", ""),
|
||||
)
|
||||
# Inject a synthetic 'steps' key so load_test_recursively can load branches
|
||||
# as TestItemParallelBranch children. The base class' _filter_dict_item
|
||||
# drops 'steps'; we also drop 'branches' (overridden below) so the F1
|
||||
# panel shows only the parallel's own attributes, not the duplicated
|
||||
# tree of branches/steps already displayed in the test tree.
|
||||
dict_item["steps"] = [{"parallel_branch": b} for b in branches]
|
||||
|
||||
super().__init__(cst.TYPE_PARALLEL, dict_item, parent, status_queue, filename=filename)
|
||||
self._sync = str(dict_item.get("sync", "all")).lower()
|
||||
if self._sync not in ("all", "any"):
|
||||
raise ETUMSyntaxError(
|
||||
f"'sync' must be 'all' or 'any', got '{self._sync}'",
|
||||
self.seqFilename(),
|
||||
)
|
||||
|
||||
def _filter_dict_item(self, dict_item):
|
||||
c = super()._filter_dict_item(dict_item)
|
||||
# Keep 'branches' so the F1 panel shows the branch list and their
|
||||
# per-branch attributes (name, wait_for, condition, ...), but strip
|
||||
# the 'steps' inside each branch — the steps are already visible as
|
||||
# children in the test tree and would just duplicate the information.
|
||||
if isinstance(c, dict) and isinstance(c.get("branches"), list):
|
||||
stripped = []
|
||||
for b in c["branches"]:
|
||||
if isinstance(b, dict):
|
||||
stripped.append({k: v for k, v in b.items() if k != "steps"})
|
||||
else:
|
||||
stripped.append(b)
|
||||
c["branches"] = stripped
|
||||
return c
|
||||
|
||||
def _stop_branch_recursively(self, item):
|
||||
item.stop()
|
||||
for i in range(item.childCount()):
|
||||
self._stop_branch_recursively(item.child(i))
|
||||
|
||||
@test_run
|
||||
def execute(self):
|
||||
branch_results = [None] * self.childCount()
|
||||
any_done = threading.Event()
|
||||
|
||||
def run_branch(idx):
|
||||
branch = self.child(idx)
|
||||
stdio_redir.register_thread(buffer=StringQueue(), branch=branch.name())
|
||||
try:
|
||||
# sync:any: if another branch already won the race, mark this
|
||||
# branch as stopped so its execute() skips children but still
|
||||
# goes through the normal addTest path (clean DB entry).
|
||||
if self._sync == "any" and any_done.is_set():
|
||||
branch.stop()
|
||||
try:
|
||||
result = branch.execute()
|
||||
except Exception as e:
|
||||
import traceback
|
||||
print(f"[parallel] Branch '{branch.name()}' crashed: {e}")
|
||||
traceback.print_exc()
|
||||
branch.result.set(TestValue.FAILURE, f"Branch crashed: {e}")
|
||||
result = branch.result
|
||||
branch_results[idx] = result
|
||||
# Only a branch that actually ran (SUCCESS or FAILURE) wins the
|
||||
# sync:any race. A disabled or skipped branch returns NORUN
|
||||
# almost instantly and must not stop legitimate branches.
|
||||
if self._sync == "any" and result.test_result != TestValue.NORUN:
|
||||
any_done.set()
|
||||
for j in range(self.childCount()):
|
||||
if j != idx:
|
||||
self._stop_branch_recursively(self.child(j))
|
||||
finally:
|
||||
stdio_redir.unregister_thread()
|
||||
|
||||
threads = [
|
||||
threading.Thread(target=run_branch, args=(i,), daemon=True)
|
||||
for i in range(self.childCount())
|
||||
]
|
||||
for t in threads:
|
||||
t.start()
|
||||
for t in threads:
|
||||
t.join()
|
||||
|
||||
if self._sync == "all":
|
||||
# Pass if no branch failed; disabled/skipped branches (NORUN) are
|
||||
# ignored, matching how Group/Cycle treat disabled children.
|
||||
success = all(
|
||||
r is not None and r.test_result != TestValue.FAILURE
|
||||
for r in branch_results
|
||||
)
|
||||
else:
|
||||
# Pass if at least one branch ran and succeeded.
|
||||
success = any(
|
||||
r is not None and r.test_result == TestValue.SUCCESS
|
||||
for r in branch_results
|
||||
)
|
||||
|
||||
self.result.set(
|
||||
TestValue.SUCCESS if success else TestValue.FAILURE,
|
||||
f"parallel sync={self._sync}",
|
||||
)
|
||||
@@ -76,5 +76,8 @@ class TestItemSleep(TestItem):
|
||||
else:
|
||||
if not isinstance(timeout, (int, float)):
|
||||
raise ETUMRuntimeError(f"Timeout value of sleep test item \"{self.name}\" is not valid: \"{timeout}\".")
|
||||
sleep(timeout)
|
||||
import time as _time
|
||||
end_time = _time.time() + float(timeout)
|
||||
while _time.time() < end_time and not self._is_stopped:
|
||||
sleep(min(0.05, end_time - _time.time()))
|
||||
self.result.set(TestValue.SUCCESS, 'Sleep %s sec' % (str(timeout)))
|
||||
|
||||
@@ -96,10 +96,10 @@ class TestItemUnittestElement(TestItem):
|
||||
|
||||
class TestItemUnittestFile(TestItem):
|
||||
def __init__(self, dict_item, parent = None, status_queue=None, filename=""):
|
||||
self._name = cst.TYPE_UNITTEST_FILE.item_name
|
||||
self._name = cst.TYPE_UNITTEST.item_name
|
||||
super().__init__(dict_item, parent, status_queue, filename=filename)
|
||||
self.is_container = True
|
||||
self._type = cst.TYPE_UNITTEST_FILE
|
||||
self._type = cst.TYPE_UNITTEST
|
||||
self._fileName = self._prms.getParam('test_file', required = True, processed = True)
|
||||
self._testDir = ''
|
||||
self._test_methods = self._prms.getParamAll('test_method', processed=True)
|
||||
@@ -161,7 +161,7 @@ class TestItemUnittestFile(TestItem):
|
||||
if self.isStopped():
|
||||
self.result.set(TestValue.NORUN, 'Group execution aborted on user request')
|
||||
else:
|
||||
self.result.set(result.test_result, 'unittest file ' + str(result.test_result))
|
||||
self.result.set(result.test_result, 'unittest ' + str(result.test_result))
|
||||
|
||||
def load(self):
|
||||
ret = {}
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
################################################################################
|
||||
## Form generated from reading UI file 'tested_refs_win.ui'
|
||||
##
|
||||
## Created by: Qt User Interface Compiler version 6.10.1
|
||||
## Created by: Qt User Interface Compiler version 6.11.0
|
||||
##
|
||||
## WARNING! All changes made in this file will be lost when recompiling UI file!
|
||||
################################################################################
|
||||
@@ -44,13 +44,13 @@ class Ui_Dialog(object):
|
||||
font1 = QFont()
|
||||
font1.setPointSize(10)
|
||||
__qtablewidgetitem = QTableWidgetItem()
|
||||
__qtablewidgetitem.setFont(font1);
|
||||
__qtablewidgetitem.setFont(font1)
|
||||
self.tableReferences.setHorizontalHeaderItem(0, __qtablewidgetitem)
|
||||
__qtablewidgetitem1 = QTableWidgetItem()
|
||||
__qtablewidgetitem1.setFont(font1);
|
||||
__qtablewidgetitem1.setFont(font1)
|
||||
self.tableReferences.setHorizontalHeaderItem(1, __qtablewidgetitem1)
|
||||
__qtablewidgetitem2 = QTableWidgetItem()
|
||||
__qtablewidgetitem2.setFont(font1);
|
||||
__qtablewidgetitem2.setFont(font1)
|
||||
self.tableReferences.setHorizontalHeaderItem(2, __qtablewidgetitem2)
|
||||
self.tableReferences.setObjectName(u"tableReferences")
|
||||
self.tableReferences.setGeometry(QRect(10, 130, 461, 211))
|
||||
@@ -70,10 +70,10 @@ class Ui_Dialog(object):
|
||||
Dialog.setWindowTitle(QCoreApplication.translate("Dialog", u"Dialog", None))
|
||||
self.labelDialog.setText(QCoreApplication.translate("Dialog", u"TextLabel", None))
|
||||
___qtablewidgetitem = self.tableReferences.horizontalHeaderItem(0)
|
||||
___qtablewidgetitem.setText(QCoreApplication.translate("Dialog", u"Reference", None));
|
||||
___qtablewidgetitem.setText(QCoreApplication.translate("Dialog", u"Reference", None))
|
||||
___qtablewidgetitem1 = self.tableReferences.horizontalHeaderItem(1)
|
||||
___qtablewidgetitem1.setText(QCoreApplication.translate("Dialog", u"Revision", None));
|
||||
___qtablewidgetitem1.setText(QCoreApplication.translate("Dialog", u"Revision", None))
|
||||
___qtablewidgetitem2 = self.tableReferences.horizontalHeaderItem(2)
|
||||
___qtablewidgetitem2.setText(QCoreApplication.translate("Dialog", u"Serial number", None));
|
||||
___qtablewidgetitem2.setText(QCoreApplication.translate("Dialog", u"Serial number", None))
|
||||
# retranslateUi
|
||||
|
||||
|
||||
@@ -10,7 +10,7 @@ class ReportExportTxt(rpe.ReportExport):
|
||||
no_value_types = [cst_type.TYPE_CONSOLE.item_name, cst_type.TYPE_SLEEP.item_name,
|
||||
cst_type.TYPE_IMAGE_DLG.item_name, cst_type.TYPE_LET.item_name, cst_type.TYPE_CHECK,
|
||||
cst_type.TYPE_CYCLE.item_name, cst_type.TYPE_GROUP.item_name,
|
||||
cst_type.TYPE_UNITTEST_FILE.item_name, cst_type.TYPE_MESSAGE_DLG.item_name,
|
||||
cst_type.TYPE_UNITTEST.item_name, cst_type.TYPE_MESSAGE_DLG.item_name,
|
||||
cst_type.TYPE_QUESTION_DLG.item_name]
|
||||
|
||||
def __init__(self, name, report_db, report_file, pattern, key, no_header=False):
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import os
|
||||
import threading
|
||||
from functools import wraps
|
||||
import sqlite3
|
||||
from time import (time, sleep)
|
||||
@@ -143,6 +144,7 @@ class TestReport:
|
||||
self._level = 0
|
||||
self._log_stored = False
|
||||
self._con = None
|
||||
self._lock = threading.Lock()
|
||||
|
||||
if dict_report is None:
|
||||
self._active = False
|
||||
@@ -231,7 +233,7 @@ class TestReport:
|
||||
prepare_file_to_save(rep_path)
|
||||
if not os.path.exists(os.path.dirname(rep_path)):
|
||||
raise ETUMRuntimeError("Report path does not exist: " + rep_path)
|
||||
self._con = sqlite3.connect(rep_path)
|
||||
self._con = sqlite3.connect(rep_path, check_same_thread=False)
|
||||
self.createHeader(header)
|
||||
self.createTestTable()
|
||||
self._con.commit()
|
||||
@@ -334,7 +336,8 @@ class TestReport:
|
||||
req = req + '?,'
|
||||
req = req[:-1] + ')'
|
||||
|
||||
self._con.execute(req, param)
|
||||
with self._lock:
|
||||
self._con.execute(req, param)
|
||||
|
||||
def incLevel(self):
|
||||
self._level = self._level + 1
|
||||
|
||||
@@ -504,7 +504,7 @@ class TestSet:
|
||||
item.is_folded = is_folded
|
||||
child = {}
|
||||
# case where the test item loads itself its descendants
|
||||
if it == cst_type.TYPE_UNITTEST_FILE:
|
||||
if it == cst_type.TYPE_UNITTEST:
|
||||
item.setTestDir(test_dir)
|
||||
child = item.load()
|
||||
elif issubclass(it.item_class, TestItemActions):
|
||||
|
||||
@@ -8,7 +8,7 @@ class TestItemEnum():
|
||||
self.item_class = item_class
|
||||
|
||||
class TestItemType(Enum):
|
||||
TYPE_UNITTEST_FILE = TestItemEnum("unittest_file", "unittest file")
|
||||
TYPE_UNITTEST = TestItemEnum("unittest", "unittest")
|
||||
TYPE_UNITTEST_STEP = TestItemEnum("unittest_step", "unittest step")
|
||||
TYPE_CONSOLE = TestItemEnum("console", "Console")
|
||||
TYPE_CONSOLE_ACTION = TestItemEnum("console_action", "Console action")
|
||||
@@ -33,6 +33,8 @@ class TestItemType(Enum):
|
||||
TYPE_RUN = TestItemEnum("run", "Run tum")
|
||||
TYPE_JSON_RPC = TestItemEnum("json_rpc", "JSON-RPC")
|
||||
TYPE_JSON_RPC_ACTION = TestItemEnum("json_rpc_action", "JSON-RPC action")
|
||||
TYPE_PARALLEL = TestItemEnum("parallel", "Parallel")
|
||||
TYPE_PARALLEL_BRANCH = TestItemEnum("parallel_branch", "Parallel branch")
|
||||
TYPE_ROOT = TestItemEnum("default", "default")
|
||||
|
||||
@staticmethod
|
||||
|
||||
@@ -44,6 +44,7 @@ from interpreter.test_items.test_item_choices_dialog import TestItemChoicesDialo
|
||||
from interpreter.test_items.test_item_console import TestItemConsole
|
||||
from interpreter.test_items.test_item_run import TestItemRun
|
||||
from interpreter.test_items.test_item_report import TestItemReport
|
||||
from interpreter.test_items.test_item_parallel import TestItemParallel, TestItemParallelBranch
|
||||
|
||||
|
||||
def _constants_init():
|
||||
@@ -67,8 +68,10 @@ def _constants_init():
|
||||
cst.TYPE_ROOT.item_class = TestItem
|
||||
cst.TYPE_RUN.item_class = TestItemRun
|
||||
cst.TYPE_SLEEP.item_class = TestItemSleep
|
||||
cst.TYPE_UNITTEST_FILE.item_class = TestItemUnittestFile
|
||||
cst.TYPE_UNITTEST.item_class = TestItemUnittestFile
|
||||
cst.TYPE_VALUE_DLG.item_class = TestItemValueDialog
|
||||
cst.TYPE_PARALLEL.item_class = TestItemParallel
|
||||
cst.TYPE_PARALLEL_BRANCH.item_class = TestItemParallelBranch
|
||||
|
||||
|
||||
def locate_report_file(rep_file):
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
################################################################################
|
||||
## Form generated from reading UI file 'about_win.ui'
|
||||
##
|
||||
## Created by: Qt User Interface Compiler version 6.10.1
|
||||
## Created by: Qt User Interface Compiler version 6.11.0
|
||||
##
|
||||
## WARNING! All changes made in this file will be lost when recompiling UI file!
|
||||
################################################################################
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
################################################################################
|
||||
## Form generated from reading UI file 'preference_core_win.ui'
|
||||
##
|
||||
## Created by: Qt User Interface Compiler version 6.10.1
|
||||
## Created by: Qt User Interface Compiler version 6.11.0
|
||||
##
|
||||
## WARNING! All changes made in this file will be lost when recompiling UI file!
|
||||
################################################################################
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Resource object code (Python 3)
|
||||
# Created by: object code
|
||||
# Created by: The Resource Compiler for Qt version 6.10.1
|
||||
# Created by: The Resource Compiler for Qt version 6.11.0
|
||||
# WARNING! All changes made in this file will be lost!
|
||||
|
||||
from PySide6 import QtCore
|
||||
@@ -1826,7 +1826,7 @@ qt_resource_struct = b"\
|
||||
\x00\x00\x00\x00\x00\x02\x00\x00\x00\x01\x00\x00\x00\x01\
|
||||
\x00\x00\x00\x00\x00\x00\x00\x00\
|
||||
\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x00\
|
||||
\x00\x00\x01\x9bi\x96\x0e\x1c\
|
||||
\x00\x00\x01\x9b\x8f'M\xbb\
|
||||
"
|
||||
|
||||
def qInitResources():
|
||||
|
||||
BIN
src/testium/main_win/resources/black/parallel.png
Normal file
|
After Width: | Height: | Size: 654 B |
BIN
src/testium/main_win/resources/black/parallel_branch.png
Normal file
|
After Width: | Height: | Size: 245 B |
BIN
src/testium/main_win/resources/black/run.png
Normal file
|
After Width: | Height: | Size: 3.0 KiB |
BIN
src/testium/main_win/resources/color/parallel.png
Normal file
|
After Width: | Height: | Size: 1.2 KiB |
BIN
src/testium/main_win/resources/color/parallel_branch.png
Normal file
|
After Width: | Height: | Size: 313 B |
BIN
src/testium/main_win/resources/color/run.png
Normal file
|
After Width: | Height: | Size: 5.5 KiB |
@@ -1,6 +1,6 @@
|
||||
# Resource object code (Python 3)
|
||||
# Created by: object code
|
||||
# Created by: The Resource Compiler for Qt version 6.10.1
|
||||
# Created by: The Resource Compiler for Qt version 6.11.0
|
||||
# WARNING! All changes made in this file will be lost!
|
||||
|
||||
from PySide6 import QtCore
|
||||
@@ -1832,7 +1832,7 @@ qt_resource_struct = b"\
|
||||
\x00\x00\x00\x00\x00\x02\x00\x00\x00\x01\x00\x00\x00\x02\
|
||||
\x00\x00\x00\x00\x00\x00\x00\x00\
|
||||
\x00\x00\x00\x10\x00\x00\x00\x00\x00\x01\x00\x00\x00\x00\
|
||||
\x00\x00\x01\x9bi\x96\x0e\x1c\
|
||||
\x00\x00\x01\x9b\x8f'M\xbb\
|
||||
"
|
||||
|
||||
def qInitResources():
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
<RCC>
|
||||
<qresource prefix="/black">
|
||||
<file alias="testium_logo.png">testium_logo.png</file>
|
||||
<file alias="run.png">black/run.png</file>
|
||||
<file alias="red.png">red.png</file>
|
||||
<file alias="gray.png">gray.png</file>
|
||||
<file alias="green.png">green.png</file>
|
||||
@@ -44,9 +45,12 @@
|
||||
<file alias="lua.png">black/lua.png</file>
|
||||
<file alias="verif.png">black/verif.png</file>
|
||||
<file alias="view-refresh.png">black/view-refresh.png</file>
|
||||
<file alias="parallel.png">black/parallel.png</file>
|
||||
<file alias="parallel_branch.png">black/parallel_branch.png</file>
|
||||
</qresource>
|
||||
<qresource prefix="/white">
|
||||
<file alias="testium_logo.png">testium_logo.png</file>
|
||||
<file alias="run.png">white/run.png</file>
|
||||
<file alias="red.png">red.png</file>
|
||||
<file alias="gray.png">gray.png</file>
|
||||
<file alias="green.png">green.png</file>
|
||||
@@ -90,9 +94,12 @@
|
||||
<file alias="lua.png">white/lua.png</file>
|
||||
<file alias="verif.png">white/verif.png</file>
|
||||
<file alias="view-refresh.png">white/view-refresh.png</file>
|
||||
<file alias="parallel.png">white/parallel.png</file>
|
||||
<file alias="parallel_branch.png">white/parallel_branch.png</file>
|
||||
</qresource>
|
||||
<qresource prefix="/color">
|
||||
<file alias="testium_logo.png">testium_logo.png</file>
|
||||
<file alias="run.png">color/run.png</file>
|
||||
<file alias="red.png">red.png</file>
|
||||
<file alias="gray.png">gray.png</file>
|
||||
<file alias="green.png">green.png</file>
|
||||
@@ -136,5 +143,7 @@
|
||||
<file alias="lua.png">color/lua.png</file>
|
||||
<file alias="verif.png">color/verif.png</file>
|
||||
<file alias="view-refresh.png">color/view-refresh.png</file>
|
||||
<file alias="parallel.png">color/parallel.png</file>
|
||||
<file alias="parallel_branch.png">color/parallel_branch.png</file>
|
||||
</qresource>
|
||||
</RCC>
|
||||
|
||||
BIN
src/testium/main_win/resources/white/parallel.png
Normal file
|
After Width: | Height: | Size: 480 B |
BIN
src/testium/main_win/resources/white/parallel_branch.png
Normal file
|
After Width: | Height: | Size: 245 B |
BIN
src/testium/main_win/resources/white/run.png
Normal file
|
After Width: | Height: | Size: 2.6 KiB |
@@ -48,8 +48,16 @@ class TestFileManager:
|
||||
def reload(self, file_name: str):
|
||||
w = self._win
|
||||
w.disconnect_signals()
|
||||
# Snapshot user-selected checkboxes and fold state so they survive a
|
||||
# reload of the same file (same logic as session-restore through prefs).
|
||||
previous_check_list = w.treeTests.getCheckList()
|
||||
previous_fold_list = w.treeTests.getFoldList()
|
||||
previous_count = w.treeTests.getItemCount()
|
||||
self.clear_process()
|
||||
self.load(file_name)
|
||||
if self.load(file_name) and w.test_service is not None:
|
||||
if w.treeTests.getItemCount() == previous_count:
|
||||
w.treeTests.restoreCheckList(previous_check_list, w.test_service)
|
||||
w.treeTests.restoreFoldList(previous_fold_list)
|
||||
w.reconnect_signals()
|
||||
|
||||
def _make_progress(self, w):
|
||||
|
||||
@@ -10,7 +10,7 @@ from libs.testium import print_warn
|
||||
# Maps item_name (from TestItemType.item_name) to visual config.
|
||||
# Keys: icon (required), icon_on (optional 2nd state), expanded, unfoldable, no_breakpoint
|
||||
_ITEM_CONFIG = {
|
||||
"unittest file": {"icon": "folder.png", "icon_on": "folder-open.png", "expanded": True, "no_breakpoint": True},
|
||||
"unittest": {"icon": "folder.png", "icon_on": "folder-open.png", "expanded": True, "no_breakpoint": True},
|
||||
"unittest step": {"icon": "document.png", "no_breakpoint": True},
|
||||
"Console": {"icon": "terminal.png", "unfoldable": False},
|
||||
"Console action": {"icon": "terminal.png"},
|
||||
@@ -32,9 +32,11 @@ _ITEM_CONFIG = {
|
||||
"References Dialog": {"icon": "label.png"},
|
||||
"Value Dialog": {"icon": "question.png"},
|
||||
"Choices Dialog": {"icon": "label.png"},
|
||||
"Run tum": {"icon": "testium_logo.svg"},
|
||||
"Run tum": {"icon": "run.png"},
|
||||
"JSON-RPC": {"icon": "json.png", "unfoldable": False},
|
||||
"JSON-RPC action": {"icon": "json.png"},
|
||||
"Parallel": {"icon": "parallel.png", "expanded": True},
|
||||
"Parallel branch": {"icon": "parallel_branch.png", "expanded": True},
|
||||
}
|
||||
|
||||
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
################################################################################
|
||||
## Form generated from reading UI file 'testium_core_win.ui'
|
||||
##
|
||||
## Created by: Qt User Interface Compiler version 6.10.1
|
||||
## Created by: Qt User Interface Compiler version 6.11.0
|
||||
##
|
||||
## WARNING! All changes made in this file will be lost when recompiling UI file!
|
||||
################################################################################
|
||||
|
||||
16
test/validation/items/parallel/parallel.py
Normal file
@@ -0,0 +1,16 @@
|
||||
import time
|
||||
import libs.testium as tm
|
||||
|
||||
|
||||
def sleep_func(duration):
|
||||
time.sleep(float(duration))
|
||||
return 0
|
||||
|
||||
|
||||
def check_duration(item_name, max_duration):
|
||||
t0 = tm.gd(f"ts_start_{item_name}")
|
||||
t1 = tm.gd(f"ts_end_{item_name}")
|
||||
duration = tm.timestamp_as_sec(t1 - t0)
|
||||
if duration < float(max_duration):
|
||||
return 0
|
||||
return 1
|
||||
1
test/validation/items/parallel/param.yaml
Normal file
@@ -0,0 +1 @@
|
||||
no_param: Null
|
||||
343
test/validation/items/parallel/test.tum
Normal file
@@ -0,0 +1,343 @@
|
||||
# --- Test 1: both branches succeed, sync:all ---
|
||||
- parallel:
|
||||
name: Both branches pass
|
||||
key: $(test)_PASS
|
||||
sync: all
|
||||
branches:
|
||||
- name: Branch A
|
||||
steps:
|
||||
- let:
|
||||
name: Set A done
|
||||
values:
|
||||
- branch_a_done: true
|
||||
- name: Branch B
|
||||
steps:
|
||||
- let:
|
||||
name: Set B done
|
||||
values:
|
||||
- branch_b_done: true
|
||||
|
||||
- check:
|
||||
name: Both branches ran
|
||||
key: $(test)_PASS
|
||||
values:
|
||||
- <| $(branch_a_done) == True |>
|
||||
- <| $(branch_b_done) == True |>
|
||||
|
||||
# --- Test 2: one branch fails, sync:all + no_fail → parallel forced to PASS ---
|
||||
- parallel:
|
||||
name: One branch fails
|
||||
key: $(test)_PASS
|
||||
sync: all
|
||||
no_fail: true
|
||||
branches:
|
||||
- name: Pass branch
|
||||
steps:
|
||||
- let:
|
||||
name: Set pass flag
|
||||
values:
|
||||
- pass_branch_ran: true
|
||||
- name: Fail branch
|
||||
steps:
|
||||
- py_func:
|
||||
name: Raise exception
|
||||
file: $(test_path)$(psep)parallel.py
|
||||
func_name: sleep_func
|
||||
param: [0]
|
||||
expected_result: fail
|
||||
|
||||
- check:
|
||||
name: Pass branch still ran
|
||||
key: $(test)_PASS
|
||||
values:
|
||||
- <| $(pass_branch_ran) == True |>
|
||||
|
||||
# --- Test 3: sync:any — first branch done stops the rest ---
|
||||
- let:
|
||||
name: Reset slow flag
|
||||
values:
|
||||
- slow_done: false
|
||||
|
||||
- parallel:
|
||||
name: sync any - first wins
|
||||
key: $(test)_PASS
|
||||
sync: any
|
||||
branches:
|
||||
- name: Fast branch
|
||||
steps:
|
||||
- let:
|
||||
name: Fast done
|
||||
values:
|
||||
- fast_done: true
|
||||
- name: Slow branch
|
||||
steps:
|
||||
- py_func:
|
||||
name: Sleep 2s
|
||||
file: $(test_path)$(psep)parallel.py
|
||||
func_name: sleep_func
|
||||
param: [2]
|
||||
- let:
|
||||
name: Slow done
|
||||
values:
|
||||
- slow_done: true
|
||||
|
||||
- check:
|
||||
name: Fast branch ran, slow branch was stopped
|
||||
key: $(test)_PASS
|
||||
values:
|
||||
- <| $(fast_done) == True |>
|
||||
- <| $(slow_done) == False |>
|
||||
|
||||
# --- Test 4: wait_for — branch B waits for A to set a flag ---
|
||||
- let:
|
||||
name: Reset sync flag
|
||||
values:
|
||||
- sync_flag: ""
|
||||
- waiter_ran: false
|
||||
|
||||
- parallel:
|
||||
name: wait_for synchronization
|
||||
key: $(test)_PASS
|
||||
sync: all
|
||||
branches:
|
||||
- name: Setter branch
|
||||
steps:
|
||||
- py_func:
|
||||
name: Sleep 0.3s then set flag
|
||||
file: $(test_path)$(psep)parallel.py
|
||||
func_name: sleep_func
|
||||
param: [0.3]
|
||||
- let:
|
||||
name: Set sync flag
|
||||
values:
|
||||
- sync_flag: ready
|
||||
- name: Waiter branch
|
||||
wait_for:
|
||||
condition: <| "$(sync_flag)" == "ready" |>
|
||||
timeout: 10
|
||||
steps:
|
||||
- let:
|
||||
name: Got flag
|
||||
values:
|
||||
- waiter_ran: true
|
||||
|
||||
- check:
|
||||
name: Waiter branch ran after flag was set
|
||||
key: $(test)_PASS
|
||||
values:
|
||||
- <| $(waiter_ran) == True |>
|
||||
|
||||
# --- Test 5: parallel is faster than sequential (timing) ---
|
||||
# Two 1s sleeps in parallel → ~1s total, not ~2s sequential
|
||||
- parallel:
|
||||
name: Timing test
|
||||
key: $(test)_PASS
|
||||
sync: all
|
||||
branches:
|
||||
- name: Sleep A
|
||||
steps:
|
||||
- sleep:
|
||||
name: Sleep 1s A
|
||||
timeout: 1
|
||||
- name: Sleep B
|
||||
steps:
|
||||
- sleep:
|
||||
name: Sleep 1s B
|
||||
timeout: 1
|
||||
|
||||
- let:
|
||||
name: Capture parallel duration
|
||||
values:
|
||||
- parallel_duration: $(ts_duration_Timing test)
|
||||
|
||||
- check:
|
||||
name: Duration < 1.8s (would be 2s if sequential)
|
||||
key: $(test)_PASS
|
||||
values:
|
||||
- <| float("$(parallel_duration)") < 1.8 |>
|
||||
|
||||
# --- Test 6: more than two branches ---
|
||||
- let:
|
||||
name: Reset N flags
|
||||
values:
|
||||
- n_a: false
|
||||
- n_b: false
|
||||
- n_c: false
|
||||
- n_d: false
|
||||
|
||||
- parallel:
|
||||
name: Four branches
|
||||
key: $(test)_PASS
|
||||
sync: all
|
||||
branches:
|
||||
- name: NA
|
||||
steps:
|
||||
- let: {name: set n_a, values: [{n_a: true}]}
|
||||
- name: NB
|
||||
steps:
|
||||
- let: {name: set n_b, values: [{n_b: true}]}
|
||||
- name: NC
|
||||
steps:
|
||||
- let: {name: set n_c, values: [{n_c: true}]}
|
||||
- name: ND
|
||||
steps:
|
||||
- let: {name: set n_d, values: [{n_d: true}]}
|
||||
|
||||
- check:
|
||||
name: Four branches all set their flag
|
||||
key: $(test)_PASS
|
||||
values:
|
||||
- <| $(n_a) == True |>
|
||||
- <| $(n_b) == True |>
|
||||
- <| $(n_c) == True |>
|
||||
- <| $(n_d) == True |>
|
||||
|
||||
# --- Test 7: nested parallel ---
|
||||
- let:
|
||||
name: Reset nested flags
|
||||
values:
|
||||
- outer_x: false
|
||||
- inner_x_1: false
|
||||
- inner_x_2: false
|
||||
|
||||
- parallel:
|
||||
name: Outer parallel
|
||||
key: $(test)_PASS
|
||||
sync: all
|
||||
branches:
|
||||
- name: Outer X
|
||||
steps:
|
||||
- let: {name: set outer_x, values: [{outer_x: true}]}
|
||||
- parallel:
|
||||
name: Inner parallel
|
||||
sync: all
|
||||
branches:
|
||||
- name: Inner X1
|
||||
steps:
|
||||
- let: {name: set inner_x_1, values: [{inner_x_1: true}]}
|
||||
- name: Inner X2
|
||||
steps:
|
||||
- let: {name: set inner_x_2, values: [{inner_x_2: true}]}
|
||||
- name: Outer Y
|
||||
steps:
|
||||
- sleep:
|
||||
name: brief sleep
|
||||
timeout: 0
|
||||
|
||||
- check:
|
||||
name: Nested parallel set all flags
|
||||
key: $(test)_PASS
|
||||
values:
|
||||
- <| $(outer_x) == True |>
|
||||
- <| $(inner_x_1) == True |>
|
||||
- <| $(inner_x_2) == True |>
|
||||
|
||||
# --- Test 9: wait_for timeout ---
|
||||
- let:
|
||||
name: Reset waiter timeout flag
|
||||
values:
|
||||
- waiter_timeout_ran: false
|
||||
|
||||
- parallel:
|
||||
name: wait_for timeout
|
||||
key: $(test)_PASS
|
||||
sync: all
|
||||
no_fail: true
|
||||
branches:
|
||||
- name: Quick branch
|
||||
steps:
|
||||
- sleep:
|
||||
name: brief sleep
|
||||
timeout: 0
|
||||
- name: Doomed waiter
|
||||
wait_for:
|
||||
condition: <| "never" == "ready" |>
|
||||
timeout: 1
|
||||
steps:
|
||||
- let: {name: should not run, values: [{waiter_timeout_ran: true}]}
|
||||
|
||||
- check:
|
||||
name: Doomed waiter never ran its steps
|
||||
key: $(test)_PASS
|
||||
values:
|
||||
- <| $(waiter_timeout_ran) == False |>
|
||||
|
||||
# --- Test 10: sync:all with a real branch failure (parallel must FAIL) ---
|
||||
- parallel:
|
||||
name: One branch really fails
|
||||
key: $(test)_FAIL
|
||||
sync: all
|
||||
branches:
|
||||
- name: ok branch
|
||||
steps:
|
||||
- let: {name: noop, values: [{noop_var: 1}]}
|
||||
- name: broken branch
|
||||
steps:
|
||||
- py_func:
|
||||
name: Forced fail
|
||||
file: $(test_path)$(psep)parallel.py
|
||||
func_name: sleep_func
|
||||
param: [0]
|
||||
expected_result: fail
|
||||
|
||||
# --- Test 11: branch with unmet condition is skipped, not failing the parallel ---
|
||||
- let:
|
||||
name: Reset branch condition flag
|
||||
values:
|
||||
- cond_branch_ran: false
|
||||
- other_branch_ran: false
|
||||
|
||||
- parallel:
|
||||
name: Condition-skipped branch
|
||||
key: $(test)_PASS
|
||||
sync: all
|
||||
branches:
|
||||
- name: Skipped branch
|
||||
condition: <| "always" == "false" |>
|
||||
steps:
|
||||
- let: {name: should not run, values: [{cond_branch_ran: true}]}
|
||||
- name: Other branch
|
||||
steps:
|
||||
- let: {name: ran, values: [{other_branch_ran: true}]}
|
||||
|
||||
- check:
|
||||
name: Skipped condition branch did not run
|
||||
key: $(test)_PASS
|
||||
values:
|
||||
- <| $(cond_branch_ran) == False |>
|
||||
- <| $(other_branch_ran) == True |>
|
||||
|
||||
# --- Test 8: parallel inside loop (re-execution) ---
|
||||
- let:
|
||||
name: Reset loop counters
|
||||
values:
|
||||
- loop_count_a: 0
|
||||
- loop_count_b: 0
|
||||
|
||||
- loop:
|
||||
name: Loop wrapping parallel
|
||||
iterator: 3
|
||||
steps:
|
||||
- parallel:
|
||||
name: Per-iteration parallel
|
||||
sync: all
|
||||
branches:
|
||||
- name: LA
|
||||
steps:
|
||||
- let:
|
||||
name: bump A
|
||||
values:
|
||||
- loop_count_a: <| int("$(loop_count_a)") + 1 |>
|
||||
- name: LB
|
||||
steps:
|
||||
- let:
|
||||
name: bump B
|
||||
values:
|
||||
- loop_count_b: <| int("$(loop_count_b)") + 1 |>
|
||||
|
||||
- check:
|
||||
name: Both branches ran 3 times
|
||||
key: $(test)_PASS
|
||||
values:
|
||||
- <| int("$(loop_count_a)") == 3 |>
|
||||
- <| int("$(loop_count_b)") == 3 |>
|
||||
@@ -1,10 +1,10 @@
|
||||
- unittest_file:
|
||||
- unittest:
|
||||
name: Unittest item
|
||||
test_file: {{include_directory}}/unittest.py
|
||||
key: $(test)_PASS
|
||||
test_method: test_01_pass
|
||||
|
||||
- unittest_file:
|
||||
- unittest:
|
||||
name: Unittest item
|
||||
test_file: {{include_directory}}/unittest.py
|
||||
key: $(test)_FAIL
|
||||
|
||||