feat(items): add pytest test item
Run a user pytest file as a testium item, surfacing each collected test as a child with its own PASS/FAIL/SKIP, duration and failure message. Mirrors the unittest item but runs pytest in a subprocess on the host interpreter (bins.python_bin(), like py_func/lua_func) so it works across every packaging channel. A stdlib-only pytest plugin streams collected node-ids and per-test results over stdout via sentinels; the parent parses them live. Params: test_file, test_method. stop_on_failure maps to -x; disabled children are reported NORUN without running. Wiring: TYPE_PYTEST / TYPE_PYTEST_STEP constants, test_init registration, self-loading branch in test_set, GUI tree icon. Schema/LSP pick it up automatically from the declarative PARAMS. Validation: test/validation/items/pytest/ (validation venv now installs pytest). WIP: paused mid-feature (DESIGN.md documented; manual section pending). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
11
DESIGN.md
11
DESIGN.md
@@ -203,6 +203,16 @@ The sub-test's own pass/fail result is intentionally not propagated.
|
||||
|
||||
The interpreter and entry point used to spawn the sub-instance are picked automatically by `_testium_launch_cmd()` based on how the parent was started (AppImage → `$APPIMAGE`; Flatpak → `flatpak run`; PyInstaller → the frozen binary; source/wheel → `[sys.executable, abspath(sys.argv[0])]`). The user cannot override either via the YAML — selecting a different testium binary or Python from a sub-test was removed because it was either ill-defined (bundle modes have no separable Python) or could mismatch the parent's environment in surprising ways.
|
||||
|
||||
### `pytest` item
|
||||
`src/testium/interpreter/test_items/test_item_pytest.py` — the pytest analogue of the `unittest` item: runs a user pytest file and surfaces every collected test as a child item (one PASS/FAIL/SKIP per test, with duration + failure message in the report).
|
||||
|
||||
Unlike `unittest` (which runs in-process), pytest runs in a **subprocess on the host interpreter** (`bins.python_bin()`), like `py_func`/`lua_func` — so the user's pytest install and test dependencies live on the host and the item works across every packaging channel (incl. Flatpak via the same staging used by `py_func`).
|
||||
|
||||
- A stdlib-only pytest plugin (`_PLUGIN_SOURCE`, written to a temp dir and loaded with `-p`) streams sentinel-prefixed lines back over the subprocess stdout: `__TESTIUM_PYTEST_COLLECTED__` (node-id list, at collection), `__TESTIUM_PYTEST_START__` / `__TESTIUM_PYTEST_RESULT__` (per test). The parent parses them live; non-sentinel lines are forwarded to the log.
|
||||
- `load()` runs `pytest --collect-only` once to build the child tree; `execute()` runs the enabled node-ids once and maps results back by node-id.
|
||||
- pytest is invoked with `--capture=no` (so plugin sentinels + test prints reach our pipe), `-o addopts=` (neutralise user addopts — xdist/cov would break the per-test hook parsing), `-p no:cacheprovider`. `stop_on_failure` → `-x`; disabled children → NORUN without running.
|
||||
- Params: `test_file` (required), `test_method` (optional list of function names, matched against the node-id function segment with the parametrisation suffix stripped). Registered as `cst.TYPE_PYTEST` / `TYPE_PYTEST_STEP`, loaded via the same self-loading branch as `unittest` in `test_set.load_test_recursively`.
|
||||
|
||||
### Report exporters & plugins
|
||||
`src/testium/interpreter/test_report/test_report.py` — `_EXPORTER_REGISTRY` dict maps a format name (cmd key in the YAML `report.export`) to a lazy loader. Built-ins: `text`, `json`, `junit` (needs `junit_xml`), `html` (needs `lxml`). `sqlite` is the storage layer, no-op as an export.
|
||||
|
||||
@@ -302,6 +312,7 @@ The `testium_assist` editor extension is a thin LSP client that spawns `testium
|
||||
Both Flatpak and AppImage export `TESTIUM_VERSION` from a launcher (Flatpak: launcher script in `org.testium.Testium.yaml`; AppImage: `runtime.env` in `AppImageBuilder.yml`). `get_testium_version()` checks `/.flatpak-info` / `APPIMAGE` and reads `TESTIUM_VERSION` rather than relying on package metadata or repo introspection.
|
||||
|
||||
## Recent fixes / notable changes
|
||||
- `pytest` item: pytest analogue of `unittest`, but runs on the **host interpreter in a subprocess** (`bins.python_bin()`, like `py_func`) so it works across every packaging channel. A stdlib-only pytest plugin streams collected node-ids + per-test results back over stdout via sentinels; each test becomes a child item with its own PASS/FAIL/SKIP, duration and failure message. Params: `test_file`, `test_method`. Validation item: `test/validation/items/pytest/` (the validation venv now pip-installs `pytest`). See "### `pytest` item".
|
||||
- Subprocess RPC startup handshake: the `py_func`/`lua_func`/`eval_proc` worker now picks its own port (`bind 0`), announces it on stdout (`__TESTIUM_RPC_PORT__=`), and the parent connects only after reading it. Fixes intermittent Windows `failed to connect : timeout` and the matching `wait_ready()` hang; removes the reserve/close/rebind race and `SO_REUSEADDR`. See "Subprocess RPC startup handshake".
|
||||
- `build_all.sh`: builds the four heavy channels in parallel (serial prep for the shared venv + wheel), results in completion order, Ctrl+C kills the whole job tree; `--ram` puts the build scratch on tmpfs (`/dev/shm`) + skips UPX for fast builds on USB/SD storage (Flatpak excluded — rofiles-fuse can't mount tmpfs). See the "Building all channels" section.
|
||||
- LSP across packaging channels: `testium lsp` (and the `testium_assist` editor extension that spawns it) now works from source, wheel, PyInstaller, Flatpak and AppImage. Two enablers — (1) action items declare a class-level `ACTIONS = {key: class}` registry (like `PARAMS`), so `lsp/schema.py` builds the full schema from class attributes with no `inspect.getsource`/AST (which broke under frozen PyInstaller); (2) the `[lsp]` extra (pygls) is wired into every full-app channel. `test/validation/lsp_check.py`, run by `run.sh` before the suite, asserts per-channel that `schema` keeps its actions and `lsp` answers `initialize`. See the matching architecture sections.
|
||||
|
||||
Reference in New Issue
Block a user