Merge branch 'feat/graceful-item-load' into feat/pytest-item
# Conflicts: # DESIGN.md
This commit is contained in:
@@ -313,6 +313,7 @@ Both Flatpak and AppImage export `TESTIUM_VERSION` from a launcher (Flatpak: lau
|
||||
|
||||
## Recent fixes / notable changes
|
||||
- `pytest` item: pytest analogue of `unittest`, but runs on the **host interpreter in a subprocess** (`bins.python_bin()`, like `py_func`) so it works across every packaging channel. A stdlib-only pytest plugin streams collected node-ids + per-test results back over stdout via sentinels; each test becomes a child item with its own PASS/FAIL/SKIP, duration and failure message. Params: `test_file`, `test_method`. Validation item: `test/validation/items/pytest/` (the validation venv now pip-installs `pytest`). See "### `pytest` item".
|
||||
- Graceful item load: a self-loading item that fails to load its module/file (e.g. a `unittest` test file importing a missing module, or `pytest` not installed on the host) no longer aborts the **whole** test load. `TestSet._load_item()` wraps the item's `load()`, emits a `tm.print_warn(...)` at load time and records the reason in `item._load_error`; the `@test_run` wrapper turns a non-None `_load_error` into a clean run-time `FAILURE` (message printed once via `write_footer`). The rest of the campaign loads and runs normally. Applies to module-loading items (`unittest`, `pytest`); structural action loading stays fail-fast.
|
||||
- `console` item — serial robustness + richer `read_until`: (1) a failed serial `open()` now raises a clear `ETUMRuntimeError` ("Serial device '…' does not exist." / permission hint) instead of dumping a pyserial traceback, and a console whose open failed is safely "not open" (init `_thd=None` + `isOpened` guards in `readchar`/`read_nowait`/`close`) so later reads no longer crash with `AttributeError: '_thd'`; the action handlers show a one-liner for expected (`ETUMRuntimeError`) errors and keep the full traceback for unexpected ones. (2) `read_until`'s `expected` now accepts a **list of values** (match any) and a new `regex: true` flag treats each pattern as a Python regex (`re.search` over a bounded tail — `Console.REGEX_WINDOW`; limitation: cost/memory bounded, so a match only after a very long stream or beyond the window won't fire). Flatpak manifest now grants `--device=all` so serial adapters (`/dev/ttyUSB*`, `/dev/ttyACM*`) are visible in the sandbox. Validation: new `read_until` list/regex cases in `test/validation/items/console/test.tum`.
|
||||
- Parameters are expanded at **run time**, never at load: control-flow flags (`stop_on_failure`/`execute_on_stop`) resolve via properties at run, `cycle` iterator / `git` repo / `tested_references` references and `console` `telnet_port` are no longer (incorrectly) expanded or left unexpanded at load. Justified load-time exceptions: `name`, `doc`, `skipped`, and `unittest`/`pytest` `test_method`.
|
||||
- Subprocess RPC startup handshake: the `py_func`/`lua_func`/`eval_proc` worker now picks its own port (`bind 0`), announces it on stdout (`__TESTIUM_RPC_PORT__=`), and the parent connects only after reading it. Fixes intermittent Windows `failed to connect : timeout` and the matching `wait_ready()` hang; removes the reserve/close/rebind race and `SO_REUSEADDR`. See "Subprocess RPC startup handshake".
|
||||
|
||||
@@ -61,6 +61,13 @@ def test_run(f):
|
||||
|
||||
self.run_test_init()
|
||||
|
||||
# The item could not be loaded (e.g. a missing module): FAIL at run.
|
||||
# run_test_end -> write_footer prints the message.
|
||||
if self._load_error is not None:
|
||||
self.result.set(TestValue.FAILURE, self._load_error)
|
||||
self.run_test_end()
|
||||
return self.result
|
||||
|
||||
while self._is_paused:
|
||||
sleep(0.2)
|
||||
if self.isStopped() :
|
||||
@@ -151,6 +158,7 @@ class TestItem:
|
||||
self._expected_result = None
|
||||
self._no_fail = None
|
||||
self._is_stopped = False
|
||||
self._load_error = None
|
||||
self._is_running = False
|
||||
self._is_breakpoint = False
|
||||
self._is_paused = False
|
||||
|
||||
@@ -451,6 +451,20 @@ class TestSet:
|
||||
def rootItem(self):
|
||||
return self._rootItem
|
||||
|
||||
def _load_item(self, item):
|
||||
"""Run an item's self-load, deferring a failure (e.g. a missing module)
|
||||
to a run-time FAILURE instead of aborting the whole test load."""
|
||||
try:
|
||||
return item.load()
|
||||
except Exception as e:
|
||||
msg = getattr(e, "_message", None) or str(e)
|
||||
item._load_error = msg
|
||||
tm.print_warn(
|
||||
f"'{item.cmd()}' item '{item.name()}' could not be loaded: "
|
||||
f"{msg} (it will FAIL at run)."
|
||||
)
|
||||
return {}
|
||||
|
||||
def load_test_recursively(self, tree_parent, parent_seq, file_name):
|
||||
ret = {}
|
||||
try:
|
||||
@@ -534,7 +548,7 @@ class TestSet:
|
||||
# case where the test item loads itself its descendants
|
||||
if it in (cst_type.TYPE_UNITTEST, cst_type.TYPE_PYTEST):
|
||||
item.setTestDir(test_dir)
|
||||
child = item.load()
|
||||
child = self._load_item(item)
|
||||
elif issubclass(it.item_class, TestItemActions):
|
||||
child = item.load()
|
||||
# case where the test item is an items container
|
||||
|
||||
Reference in New Issue
Block a user