8 Commits
v0.3.1 ... main

Author SHA1 Message Date
c313e1431b fix(gui): Flatpak Show Results opens the log via host xdg-open
fix(gui): keep Show Results enabled during a run

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 11:54:11 +02:00
7edfc25a1f docs: note F1 variable filter under 0.3.2
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 23:38:45 +02:00
7a732c0d04 feat(gui): filter variables in the F1 window (name, optional value)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 23:36:57 +02:00
f62ea10d24 test(validation): make immediate read_until deterministic (drop prompt race)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 23:17:14 +02:00
51068c881f chore(release): 0.3.2
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 22:53:30 +02:00
83475dd215 docs: run item capture + batch param
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 22:53:30 +02:00
4fe23518a0 test(validation): run capture via store_result
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 22:53:30 +02:00
87e62a7f2e feat(run): capture sub-instance output, add batch param
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 22:53:30 +02:00
13 changed files with 185 additions and 48 deletions

View File

@@ -202,7 +202,7 @@ A find bar (Ctrl+F) over the `QTestTree` (`src/testium/main_win/test_tree.py`) h
- `QTestTreeItem._refresh_highlight()` is the single source of truth for the name-column colours: the **search** highlight (pastel amber bg + forced black text, readable in light *and* dark themes) and the green **run** highlight (`setHighlighted`) are recomputed from state flags with precedence **run > search > default**. No brush is saved/restored, so the two layers never leave a stale/permanent colour when they overlap (e.g. searching while a test runs).
### `run` item
`src/testium/interpreter/test_items/test_item_run.py` — launches a `.tum` file in a new testium instance (`-b` in batch mode, `-r` in GUI mode). Result:
`src/testium/interpreter/test_items/test_item_run.py` — launches a `.tum` file in a new testium instance. Child mode: `-b` in batch, `-r` (own window) in the GUI, or forced `-b` by `batch: true`. A `-b` child is **captured** (launched `-o`, no colour): its stdout/stderr stream through `proc_drain.drain_to_log()` into this test's log/report, and the full text is kept as the result value, so `store_result` pushes it to the gdict and `expected_result`/`process_result`/a py_func can post-process it. Stop kills the child via the poll loop. Result:
- **PASS** if the sub-instance launched and ran to completion (exit code is ignored)
- **FAIL** if the file is not found, `wait_for_exec` is set without `start_time`/`end_time`, the time window was not reached, or any other launch error
@@ -323,6 +323,7 @@ The `testium_assist` editor extension is a thin LSP client that spawns `testium
Both Flatpak and AppImage export `TESTIUM_VERSION` from a launcher (Flatpak: launcher script in `org.testium.Testium.yaml`; AppImage: `runtime.env` in `AppImageBuilder.yml`). `get_testium_version()` checks `/.flatpak-info` / `APPIMAGE` and reads `TESTIUM_VERSION` rather than relying on package metadata or repo introspection.
## Recent fixes / notable changes
- Show Results (GUI): the toolbar action stays enabled during a run (the log grows live, so it is useful mid-test), not just after. In Flatpak `QDesktopServices.openUrl` routes through the OpenURI portal and often opens no editor for a `.log`; `bins.host_open_path()` now spawns `xdg-open` on the host via `flatpak-spawn --host` (returns False outside Flatpak so the caller falls back to `openUrl`).
- Test-tree search (GUI): a Ctrl+F find bar highlights + navigates matching items, with Name/Type/Doc field checkboxes. Search modifications run under `blockSignals` (else `setBackground``itemChanged``on_testChecked` storms the controller), and the search/run highlights share one flag-driven `_refresh_highlight()` (run > search > default) so overlapping layers never leave a stale colour. See "## Test-tree search (GUI)".
- `pytest` item: pytest analogue of `unittest`, but runs on the **host interpreter in a subprocess** (`bins.python_bin()`, like `py_func`) so it works across every packaging channel. A stdlib-only pytest plugin streams collected node-ids + per-test results back over stdout via sentinels; each test becomes a child item with its own PASS/FAIL/SKIP, duration and failure message. Params: `test_file`, `test_method`. Validation item: `test/validation/items/pytest/` (the validation venv now pip-installs `pytest`). See "### `pytest` item".
- Graceful item load: a self-loading item that fails to load its module/file (e.g. a `unittest` test file importing a missing module, or `pytest` not installed on the host) no longer aborts the **whole** test load. `TestSet._load_item()` wraps the item's `load()`, emits a `tm.print_warn(...)` at load time and records the reason in `item._load_error`; the `@test_run` wrapper turns a non-None `_load_error` into a clean run-time `FAILURE` (message printed once via `write_footer`). The rest of the campaign loads and runs normally. Applies to module-loading items (`unittest`, `pytest`); structural action loading stays fail-fast.

View File

@@ -4,7 +4,13 @@
This test item executes a new instance of testium with the specified ``.tum`` file.
* In **batch mode** (``-b``): the sub-instance is started with ``-b``.
* In **GUI mode**: the sub-instance is started with ``-r`` (run and close).
* In **GUI mode**: the sub-instance opens its own window with ``-r`` (run and close).
* ``batch: true`` forces the sub-instance to run headless (``-b``) even in the GUI.
A sub-instance started with ``-b`` is **captured**: its output is streamed into this
test's log and report, and kept as the item's result value, so it can be stored
with ``store_result`` and post-processed (``expected_result``, ``process_result``,
or a ``py_func`` reading the global variable).
The item result is **PASS** if the sub-instance launched and ran to completion,
regardless of whether the sub-tests passed or failed.
@@ -17,7 +23,6 @@ launched, or the time window was not reached (see ``start_time`` / ``end_time``)
- run:
name: Execute TUM
tum: example_cycle.tum
python_bin: python3
log_file: $(home)/reports/test.log
report_file: $(home)/reports/test.rep
@@ -28,9 +33,8 @@ run test item has the following specific attributes:
* ``tum``: mandatory, the path of the file to execute. Can be relative to the current execution folder.
* ``param_file`` (optional): the path of the parameter file to use; otherwise the default parameter file is used.
* ``python_bin`` (optional): the path of a specific Python interpreter to use.
* ``testium_path`` (optional): the path of a specific testium executable to use.
* ``log_file`` (optional): the path of the log file. In GUI mode, if not provided, a file is created with a timestamp next to the ``.tum`` file. Not used in batch mode.
* ``batch`` (optional): ``true`` to run the sub-instance headless (``-b``) and capture its output even in GUI mode (see above).
* ``log_file`` (optional): the path of the log file. In GUI mode, if not provided, a file is created with a timestamp next to the ``.tum`` file. Not used in batch mode (the output is captured instead).
* ``report_file`` (optional): the path of the report file to create.
* ``start_time`` (optional): earliest time to execute the sub-instance, in ``HH:MM`` format.
* ``end_time`` (optional): latest time for execution within a time frame, in ``HH:MM`` format.

View File

@@ -1,3 +1,14 @@
version 0.3.2
==============
- The variables window (F1) now has a filter box: type to show only the
variables whose name matches. Tick "values" to also match on the value.
- The ``run`` item now captures the output of the test it launches into your
log and report (it used to go only to the terminal and was lost from the
report). New ``batch: true`` option runs that test headless (and captured)
even when testium is in the GUI.
- The captured output of a ``run`` step can be saved with ``store_result`` and
inspected afterwards (for example with ``expected_result`` or a ``py_func``).
version 0.3.1
==============
- Clearer errors when a test file fails to load. The message now names the

View File

@@ -1 +1 @@
0.3.1
0.3.2

View File

@@ -11,6 +11,7 @@ from interpreter.test_items.test_result import (TestValue)
import api.testium as tm
from interpreter.utils.constants import TestItemType as cst
from interpreter.utils.param_decl import Param, ParamSet
from interpreter.utils.proc_drain import drain_to_log
from runtime.tum_except import ETUMSyntaxError, ETUMRuntimeError, item_load_context
@@ -75,6 +76,9 @@ class TestItemRun(TestItem):
Param("wait_for_exec",
doc="If true, block until the time window opens. Requires both "
"start_time and end_time."),
Param("batch", default=False,
doc="Run the sub-instance headless (-b) with its output captured "
"into this test's log/report and result value, even in the GUI."),
)
def __init__(self, dict_item, parent = None, status_queue=None, filename=""):
@@ -90,6 +94,38 @@ class TestItemRun(TestItem):
self.start_time = self._prms.getParam('start_time')
self.end_time = self._prms.getParam('end_time')
self.wait_for_exec = self._prms.getParam('wait_for_exec')
self.batch = self._prms.getParam('batch', default=False)
def _launch(self, cmd, capture):
"""Run the sub-instance once. When *capture*, stream its output to the
log/report, keep it as the result value, and let Stop kill the child."""
if not capture:
subprocess.run(cmd)
return
sink = []
prefix = f"[{os.path.basename(self.tum_file)}] "
proc = subprocess.Popen(
cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE
)
threads = drain_to_log(proc, prefix=prefix, sink=sink)
try:
while True:
try:
proc.wait(timeout=0.2)
break
except subprocess.TimeoutExpired:
if self.isStopped():
proc.terminate()
try:
proc.wait(timeout=2)
except subprocess.TimeoutExpired:
proc.kill()
break
finally:
for t in threads:
t.join(timeout=2)
# Captured log -> result value (store_result / expected_result).
self.result.value = "\n".join(sink)
@test_run
def execute(self):
@@ -104,25 +140,26 @@ class TestItemRun(TestItem):
pf = self._prms.expanse(self.param_file)
lp = self._prms.expanse(self.log_path)
rp = self._prms.expanse(self.report_path)
# Capture (headless -b) in batch or when `batch: true`; else open
# the child's own GUI window (-r).
capture = bool(self.batch) or tm.text_mode()
cmd = _testium_launch_cmd()
if tm.text_mode():
cmd.append("-b")
if capture:
cmd += ["-b", "-o"] # -o: no colour codes in the captured log
else:
cmd.append("-r")
if lp == '':
lp = os.path.splitext(self.tum_file)[0] + "_" + \
datetime.utcnow().isoformat(timespec='seconds') + '.log'
cmd.append("-l")
cmd.append('"' + lp + '"')
cmd += ["-l", '"' + lp + '"']
if pf != '':
cmd.append("-c")
cmd.append('"' + pf + '"')
cmd += ["-c", '"' + pf + '"']
if rp != '':
cmd.append("-p")
cmd.append('"' + rp + '"')
cmd += ["-p", '"' + rp + '"']
cmd.append(self.tum_file)
for c in cmd:
print(c, end = ' ')
print(" ".join(cmd))
if self.start_time is not None:
self.start_time = datetime.strptime(
@@ -135,20 +172,24 @@ class TestItemRun(TestItem):
raise ETUMRuntimeError(
'"wait_for_exec" set but not start_time or end_time')
r = None
ran = False
if self.wait_for_exec:
while not nowInBetween(self.start_time, self.end_time):
sleep(60)
r = subprocess.run(cmd)
self._launch(cmd, capture)
ran = True
elif self.start_time is not None and self.end_time is not None:
if nowInBetween(self.start_time, self.end_time):
r = subprocess.run(cmd)
self._launch(cmd, capture)
ran = True
elif self.start_time is not None:
if self.start_time < datetime.now().time():
r = subprocess.run(cmd)
self._launch(cmd, capture)
ran = True
else:
r = subprocess.run(cmd)
if isinstance(r, subprocess.CompletedProcess):
self._launch(cmd, capture)
ran = True
if ran:
self.result.set(TestValue.SUCCESS)
else:
self.result.set(TestValue.FAILURE, 'Sub-test did not execute')

View File

@@ -199,6 +199,23 @@ def host_console_command(shell_cmd, cwd):
return ["flatpak-spawn", "--host", f"--directory={cwd}", *argv]
def host_open_path(path):
"""Open *path* with the host default application (Flatpak only).
QDesktopServices/openUrl routes through the OpenURI portal inside Flatpak,
which often fails to open a plain editor for a log file. Spawn xdg-open on
the host so the user's real default app is used. Returns True on dispatch;
False (incl. outside Flatpak) so the caller can fall back to openUrl.
"""
if not _in_flatpak():
return False
try:
subprocess.Popen(["flatpak-spawn", "--host", "xdg-open", path])
return True
except (FileNotFoundError, PermissionError):
return False
def _which_host_flatpak(name):
"""Resolve a binary name (or absolute path) on the host via flatpak-spawn.

View File

@@ -13,7 +13,7 @@ from time import monotonic
from runtime.jrpc import RPC_PORT_SENTINEL
def _drain_pipe(pipe, prefix):
def _drain_pipe(pipe, prefix, sink=None):
try:
for raw in iter(pipe.readline, b""):
line = raw.decode("utf-8", errors="replace").rstrip("\r\n")
@@ -23,6 +23,9 @@ def _drain_pipe(pipe, prefix):
print(f"{prefix}{line}")
else:
print(line)
# sink keeps the clean (unprefixed) line for reuse as a result value
if sink is not None:
sink.append(line)
finally:
try:
pipe.close()
@@ -30,21 +33,16 @@ def _drain_pipe(pipe, prefix):
pass
def drain_to_log(process, prefix=""):
"""Spawn daemon threads that read ``process.stdout`` and
``process.stderr`` line by line and print each line through the
parent's stdout (so it reaches the log + live output).
Each thread exits cleanly when the subprocess closes the
corresponding pipe (i.e. when it exits). Daemon flag ensures they
do not block testium exit.
"""
def drain_to_log(process, prefix="", sink=None):
"""Stream the subprocess stdout/stderr line by line through the parent's
print pipeline (log + live output). If ``sink`` is a list, each clean line
is also appended to it (GIL-atomic, shared by both threads). Daemon threads."""
threads = []
for pipe in (process.stdout, process.stderr):
if pipe is None:
continue
t = threading.Thread(
target=_drain_pipe, args=(pipe, prefix), daemon=True,
target=_drain_pipe, args=(pipe, prefix, sink), daemon=True,
)
t.start()
threads.append(t)

View File

@@ -6,8 +6,8 @@ import subprocess
import sys
from PySide6.QtWidgets import (
QDialog, QDialogButtonBox, QHeaderView, QMenu, QMessageBox,
QPushButton, QTextEdit, QVBoxLayout,
QCheckBox, QDialog, QDialogButtonBox, QHBoxLayout, QHeaderView, QLineEdit,
QMenu, QMessageBox, QPushButton, QTextEdit, QVBoxLayout,
)
from PySide6.QtGui import QSyntaxHighlighter, QTextCharFormat, QColor, QFont, QDesktopServices
from PySide6.QtCore import Qt, QUrl, Slot
@@ -119,6 +119,43 @@ class DialogF1(QDialog):
self.ui.addVarButton.setEnabled(False)
self.ui.addVarButton.clicked.connect(self._on_add_var)
# Filter box above the table: hides rows whose name doesn't match.
# The optional "values" checkbox extends the match to the value column.
self._filter_text = ""
self._filter_edit = QLineEdit(self.ui.tabVariables)
self._filter_edit.setPlaceholderText("Filter variables by name")
self._filter_edit.setClearButtonEnabled(True)
self._filter_edit.textChanged.connect(self._on_filter_changed)
self._filter_values_cb = QCheckBox("values", self.ui.tabVariables)
self._filter_values_cb.setToolTip("Also match on the variable value")
self._filter_values_cb.toggled.connect(lambda _checked: self._apply_filter())
filter_row = QHBoxLayout()
filter_row.addWidget(self._filter_edit)
filter_row.addWidget(self._filter_values_cb)
self.ui.verticalLayout_tab1.insertLayout(0, filter_row)
def _on_filter_changed(self, text):
self._filter_text = text.strip().lower()
self._apply_filter()
def _apply_filter(self):
for row in range(self.ui.varsTable.rowCount()):
self._apply_filter_row(row)
def _apply_filter_row(self, row):
needle = self._filter_text
if not needle:
self.ui.varsTable.setRowHidden(row, False)
return
table = self.ui.varsTable
key_item = table.item(row, 0)
hay = key_item.text().lower() if key_item else ""
if self._filter_values_cb.isChecked():
val_item = table.item(row, 1)
if val_item is not None:
hay += "\n" + val_item.text().lower()
table.setRowHidden(row, needle not in hay)
def load_initial_vars(self, vars_dict: dict):
for key, value in vars_dict.items():
self.gd_var_updated(key, value)
@@ -149,6 +186,7 @@ class DialogF1(QDialog):
self._updating = False
self._key_rows[key] = row
self._refresh_row(row, key, value)
self._apply_filter_row(self._key_rows[key])
@Slot(str)
def gd_var_deleted(self, key):
@@ -161,6 +199,7 @@ class DialogF1(QDialog):
finally:
self._updating = False
self._key_rows = {k: (r - 1 if r > row else r) for k, r in self._key_rows.items()}
self._apply_filter()
def _refresh_row(self, row, key, value):
from PySide6.QtWidgets import QTableWidgetItem

View File

@@ -181,7 +181,8 @@ class TestRunner:
w.actionStart_test.setText("Pause test")
w.actionPreferences.setDisabled(True)
w.actionRefresh_test.setDisabled(True)
w.actionShow_Results.setDisabled(True)
# Show Results stays available during the run (log grows live).
w.actionShow_Results.setEnabled(True)
w.actionSave_report.setDisabled(True)
w.logSettingsBox.setDisabled(True)
w.actionStop_test.setEnabled(True)

View File

@@ -40,6 +40,7 @@ from runtime.string_queue import StringQueue
from interpreter.process import TestProcess
from interpreter.utils.test_ctrl import TestSetController
from interpreter.utils.icons import icon_prefix
from interpreter.utils import bins
from main_win.test_run.outlog import OutLog
from main_win.test_run.test_run import ThreadTestStatus
@@ -639,7 +640,8 @@ class MainWindow(QMainWindow, Ui_MainWindow):
self.statusBar().showMessage(
"Opening the logfile (" + s + "): " + self.logFileName, 100000
)
QDesktopServices.openUrl(QUrl.fromLocalFile(self.logFileName))
if not bins.host_open_path(self.logFileName):
QDesktopServices.openUrl(QUrl.fromLocalFile(self.logFileName))
@Slot()
def on_actionHelp_triggered(self):

View File

@@ -47,13 +47,15 @@
{% endif %}
- read_until: {expected: terminal loaded, timeout: 5}
# Echo two tokens on one line so both are buffered together; the immediate
# (timeout 0) reads below match buffered data with no race on the async prompt.
- console:
name: Console write
condition: <| $(conditional_exec) == 1 |>
console_name: consname
key: $(test)_PASS
steps:
- writeln: echo 0
- writeln: echo ALPHA BETA
- sleep:
name: sleep item
@@ -67,9 +69,9 @@
key: $(test)_PASS
steps:
{% if os == "Windows" %}
- read_until: {expected: echo 0, timeout: 0}
- read_until: {expected: echo ALPHA BETA, timeout: 0}
{% endif %}
- read_until: {expected: "0", timeout: 0}
- read_until: {expected: ALPHA, timeout: 0}
- console:
name: Console read_until immediate (2)
@@ -77,7 +79,7 @@
console_name: consname
key: $(test)_PASS
steps:
- read_until: {expected: "$(terminal_prompt)", timeout: 0}
- read_until: {expected: BETA, timeout: 0}
- console:
name: Console closure

View File

@@ -0,0 +1,9 @@
import py_func.tm as tm
def assert_captured():
"""The sub-run log stored by `run` via store_result must be in the gdict."""
log = tm.gd("captured_log", "")
assert "Test run success." in log, \
"captured sub-run log not reachable from the gdict (store_result)"
return 0

View File

@@ -1,10 +1,7 @@
# run item: launches a .tum file in a new testium instance.
# In batch mode the sub-instance runs with -b; in GUI mode with -r.
# The run item result is SUCCESS if the sub-instance launched successfully,
# regardless of its own test result.
#
# log_file points the sub-instance log at the throwaway report dir (gitignored)
# so a GUI run does not litter the repo with sub_*.log files.
# Child mode: -b in batch / -r in the GUI, or forced -b (captured) by batch: true.
# Result is SUCCESS if the sub-instance launched, regardless of its own result.
# log_file (GUI -r only) goes to the gitignored report dir to avoid repo litter.
- run:
name: run PASS (valid file, passing sub-test)
@@ -30,3 +27,18 @@
tum: $(test_path)$(psep)sub_pass.tum
wait_for_exec: true
log_file: $(validation_report_path)$(psep)run_sub.log
# batch: true forces a headless, captured sub-run even in the GUI; its log is
# kept as the result value and pushed to the gdict by store_result.
- run:
name: run batch (capture sub-run log to the gdict)
key: $(test)_PASS
tum: $(test_path)$(psep)sub_pass.tum
batch: true
store_result: captured_log
- py_func:
name: captured sub-run log is post-processable from the gdict
key: $(test)_PASS
file: $(test_path)$(psep)check_capture.py
func_name: assert_captured