14 Commits
v0.3 ... main

Author SHA1 Message Date
c313e1431b fix(gui): Flatpak Show Results opens the log via host xdg-open
fix(gui): keep Show Results enabled during a run

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 11:54:11 +02:00
7edfc25a1f docs: note F1 variable filter under 0.3.2
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 23:38:45 +02:00
7a732c0d04 feat(gui): filter variables in the F1 window (name, optional value)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 23:36:57 +02:00
f62ea10d24 test(validation): make immediate read_until deterministic (drop prompt race)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 23:17:14 +02:00
51068c881f chore(release): 0.3.2
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 22:53:30 +02:00
83475dd215 docs: run item capture + batch param
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 22:53:30 +02:00
4fe23518a0 test(validation): run capture via store_result
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 22:53:30 +02:00
87e62a7f2e feat(run): capture sub-instance output, add batch param
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 22:53:30 +02:00
5b5792a296 Merge branch 'main' of ssh://cahute.beafrancois.fr:8327/v-and-v/testium 2026-06-15 14:40:50 +02:00
087aa93a16 chore(release): 0.3.1
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 14:40:36 +02:00
7abd8c07a6 test(validation): negative load-error checks; keep run logs out of the repo
load_errors_check.py loads deliberately broken .tum fixtures in batch on the
build under test (like lsp_check.py) and asserts each fails with its specific
located message and without a raw traceback. Wired into run.sh just before the
main suite, so it runs for every channel.

The run validation items now point their sub-instance log at the gitignored
report dir, so a GUI run no longer litters the tree with sub_*.log files.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 14:40:24 +02:00
1ea360e5a5 fix(load): report every test-load error with file, item path and cause
A structural mistake in a .tum (unknown item or action, a step holding two
items, a missing 'steps:' list, a scalar where a mapping is expected, ...)
used to surface as a bare Python traceback. At worst the unknown-action
formatter itself crashed with "'dict_keys' object is not subscriptable"
(action.keys()[0]), masking the real cause and leaving only the generic
"test process crashed for any reason".

The load path now validates each step and funnels every failure through a
located TUM file syntax error: the file, a breadcrumb to the item, the
offending value and the list of valid names. A problem inside an !include-d
file points to that file. A last-resort net in __loadTestTree turns any
unforeseen exception into a located error too.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 14:40:15 +02:00
d5154348f6 Midified the package for windows (not a monolitic bin). And added the option to remove older versions in the setup. 2026-06-15 09:08:28 +02:00
6dc473de41 test(validation): --gui option to run the suite in the GUI
Adds `--gui` to test/validation/run.sh: drops `-b` so testium opens the GUI
with the validation suite loaded instead of running headless. The run is
started manually and the window stays open — handy to inspect the test
tree, try the Ctrl+F search, etc. Works with any --mode.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 00:25:53 +02:00
29 changed files with 657 additions and 129 deletions

View File

@@ -202,7 +202,7 @@ A find bar (Ctrl+F) over the `QTestTree` (`src/testium/main_win/test_tree.py`) h
- `QTestTreeItem._refresh_highlight()` is the single source of truth for the name-column colours: the **search** highlight (pastel amber bg + forced black text, readable in light *and* dark themes) and the green **run** highlight (`setHighlighted`) are recomputed from state flags with precedence **run > search > default**. No brush is saved/restored, so the two layers never leave a stale/permanent colour when they overlap (e.g. searching while a test runs).
### `run` item
`src/testium/interpreter/test_items/test_item_run.py` — launches a `.tum` file in a new testium instance (`-b` in batch mode, `-r` in GUI mode). Result:
`src/testium/interpreter/test_items/test_item_run.py` — launches a `.tum` file in a new testium instance. Child mode: `-b` in batch, `-r` (own window) in the GUI, or forced `-b` by `batch: true`. A `-b` child is **captured** (launched `-o`, no colour): its stdout/stderr stream through `proc_drain.drain_to_log()` into this test's log/report, and the full text is kept as the result value, so `store_result` pushes it to the gdict and `expected_result`/`process_result`/a py_func can post-process it. Stop kills the child via the poll loop. Result:
- **PASS** if the sub-instance launched and ran to completion (exit code is ignored)
- **FAIL** if the file is not found, `wait_for_exec` is set without `start_time`/`end_time`, the time window was not reached, or any other launch error
@@ -323,6 +323,7 @@ The `testium_assist` editor extension is a thin LSP client that spawns `testium
Both Flatpak and AppImage export `TESTIUM_VERSION` from a launcher (Flatpak: launcher script in `org.testium.Testium.yaml`; AppImage: `runtime.env` in `AppImageBuilder.yml`). `get_testium_version()` checks `/.flatpak-info` / `APPIMAGE` and reads `TESTIUM_VERSION` rather than relying on package metadata or repo introspection.
## Recent fixes / notable changes
- Show Results (GUI): the toolbar action stays enabled during a run (the log grows live, so it is useful mid-test), not just after. In Flatpak `QDesktopServices.openUrl` routes through the OpenURI portal and often opens no editor for a `.log`; `bins.host_open_path()` now spawns `xdg-open` on the host via `flatpak-spawn --host` (returns False outside Flatpak so the caller falls back to `openUrl`).
- Test-tree search (GUI): a Ctrl+F find bar highlights + navigates matching items, with Name/Type/Doc field checkboxes. Search modifications run under `blockSignals` (else `setBackground``itemChanged``on_testChecked` storms the controller), and the search/run highlights share one flag-driven `_refresh_highlight()` (run > search > default) so overlapping layers never leave a stale colour. See "## Test-tree search (GUI)".
- `pytest` item: pytest analogue of `unittest`, but runs on the **host interpreter in a subprocess** (`bins.python_bin()`, like `py_func`) so it works across every packaging channel. A stdlib-only pytest plugin streams collected node-ids + per-test results back over stdout via sentinels; each test becomes a child item with its own PASS/FAIL/SKIP, duration and failure message. Params: `test_file`, `test_method`. Validation item: `test/validation/items/pytest/` (the validation venv now pip-installs `pytest`). See "### `pytest` item".
- Graceful item load: a self-loading item that fails to load its module/file (e.g. a `unittest` test file importing a missing module, or `pytest` not installed on the host) no longer aborts the **whole** test load. `TestSet._load_item()` wraps the item's `load()`, emits a `tm.print_warn(...)` at load time and records the reason in `item._load_error`; the `@test_run` wrapper turns a non-None `_load_error` into a clean run-time `FAILURE` (message printed once via `write_footer`). The rest of the campaign loads and runs normally. Applies to module-loading items (`unittest`, `pytest`); structural action loading stays fail-fast.

View File

@@ -4,7 +4,13 @@
This test item executes a new instance of testium with the specified ``.tum`` file.
* In **batch mode** (``-b``): the sub-instance is started with ``-b``.
* In **GUI mode**: the sub-instance is started with ``-r`` (run and close).
* In **GUI mode**: the sub-instance opens its own window with ``-r`` (run and close).
* ``batch: true`` forces the sub-instance to run headless (``-b``) even in the GUI.
A sub-instance started with ``-b`` is **captured**: its output is streamed into this
test's log and report, and kept as the item's result value, so it can be stored
with ``store_result`` and post-processed (``expected_result``, ``process_result``,
or a ``py_func`` reading the global variable).
The item result is **PASS** if the sub-instance launched and ran to completion,
regardless of whether the sub-tests passed or failed.
@@ -17,7 +23,6 @@ launched, or the time window was not reached (see ``start_time`` / ``end_time``)
- run:
name: Execute TUM
tum: example_cycle.tum
python_bin: python3
log_file: $(home)/reports/test.log
report_file: $(home)/reports/test.rep
@@ -28,9 +33,8 @@ run test item has the following specific attributes:
* ``tum``: mandatory, the path of the file to execute. Can be relative to the current execution folder.
* ``param_file`` (optional): the path of the parameter file to use; otherwise the default parameter file is used.
* ``python_bin`` (optional): the path of a specific Python interpreter to use.
* ``testium_path`` (optional): the path of a specific testium executable to use.
* ``log_file`` (optional): the path of the log file. In GUI mode, if not provided, a file is created with a timestamp next to the ``.tum`` file. Not used in batch mode.
* ``batch`` (optional): ``true`` to run the sub-instance headless (``-b``) and capture its output even in GUI mode (see above).
* ``log_file`` (optional): the path of the log file. In GUI mode, if not provided, a file is created with a timestamp next to the ``.tum`` file. Not used in batch mode (the output is captured instead).
* ``report_file`` (optional): the path of the report file to create.
* ``start_time`` (optional): earliest time to execute the sub-instance, in ``HH:MM`` format.
* ``end_time`` (optional): latest time for execution within a time frame, in ``HH:MM`` format.

View File

@@ -1,16 +1,36 @@
# Build the Testium installer from testium.iss (needs Inno Setup 6 / ISCC.exe).
# Build the Windows installer: PyInstaller one-folder build (fast start) + Inno Setup.
# Install ISCC without admin: winget install --id JRSoftware.InnoSetup -e
$ErrorActionPreference = 'Stop'
$scriptDir = Split-Path -Parent $MyInvocation.MyCommand.Path
$repoRoot = Resolve-Path (Join-Path $scriptDir '..\..')
$pyiDir = Join-Path $repoRoot 'package\pyinstaller'
# The PyInstaller exe must exist first.
$exe = Join-Path $scriptDir '..\pyinstaller\dist\testium.exe'
if (-not (Test-Path $exe)) {
throw "PyInstaller build not found: $exe`nRun package\pyinstaller\build first."
# Locate PyInstaller: PATH first, then the known project venvs.
$pyi = (Get-Command pyinstaller.exe -ErrorAction SilentlyContinue).Source
if (-not $pyi) {
foreach ($p in @(
(Join-Path $repoRoot 'test\tmp\testium_venv\Scripts\pyinstaller.exe'),
(Join-Path $repoRoot 'test\tmp\.venv\Scripts\pyinstaller.exe'))) {
if (Test-Path $p) { $pyi = $p; break }
}
}
if (-not $pyi) { throw "pyinstaller.exe not found (PATH or project venv)." }
# One-folder PyInstaller build => dist\testium\testium.exe + dist\testium\_internal\.
Write-Host "Building one-folder exe with: $pyi"
Remove-Item -Recurse -Force (Join-Path $pyiDir 'build'), (Join-Path $pyiDir 'dist') -ErrorAction SilentlyContinue
Push-Location $pyiDir
try {
$env:TESTIUM_ONEDIR = '1'
& $pyi 'testium.spec'
if ($LASTEXITCODE -ne 0) { throw "pyinstaller failed with exit code $LASTEXITCODE" }
} finally {
Remove-Item Env:\TESTIUM_ONEDIR -ErrorAction SilentlyContinue
Pop-Location
}
# Locate ISCC.exe: PATH, then the usual install dirs.
# Locate ISCC: PATH, then the usual install dirs.
$iscc = (Get-Command ISCC.exe -ErrorAction SilentlyContinue).Source
if (-not $iscc) {
foreach ($p in @(

View File

@@ -49,9 +49,12 @@ Name: "english"; MessagesFile: "compiler:Default.isl"
Name: "desktopicon"; Description: "{cm:CreateDesktopIcon}"; GroupDescription: "{cm:AdditionalIcons}"; Flags: unchecked
; PATH off by default: the exe is windowed (console=False), so CLI shows no output.
Name: "addtopath"; Description: "Ajouter Testium au PATH (usage en ligne de commande)"; Flags: unchecked
; Shown only if another version is already installed; unchecked => keep it.
Name: "removeold"; Description: "Désinstaller les autres versions de Testium déjà installées"; Check: OtherVersionsExist; Flags: unchecked
[Files]
Source: "..\pyinstaller\dist\{#MyAppExeName}"; DestDir: "{app}"; Flags: ignoreversion
; One-folder build: the exe plus its _internal\ tree (fast startup, no re-extract).
Source: "..\pyinstaller\dist\testium\*"; DestDir: "{app}"; Flags: recursesubdirs ignoreversion
; Ship the .ico so shortcuts/uninstall reference it directly, not the embedded one.
Source: "..\testium.ico"; DestDir: "{app}"; Flags: ignoreversion
@@ -67,6 +70,54 @@ Filename: "{app}\{#MyAppExeName}"; Description: "{cm:LaunchProgram,{#MyAppName}}
[Code]
const
EnvKey = 'Environment';
UninstallRoot = 'Software\Microsoft\Windows\CurrentVersion\Uninstall';
AppGuid = '{B7E6F1C2-9A4D-4E3B-8F71-7C2D5A6E0B14}';
// Inno's uninstall subkey for *this* version: "{AppId}_is1".
function CurrentUninstallSubkey(): string;
begin
Result := AppGuid + '_{#MyAppVersion}_is1';
end;
// Uninstall subkeys of every installed Testium version except this one.
function OtherTestiumSubkeys(): TArrayOfString;
var
names: TArrayOfString;
i: Integer;
prefix, cur: string;
begin
SetArrayLength(Result, 0);
prefix := Uppercase(AppGuid + '_');
cur := Uppercase(CurrentUninstallSubkey());
if RegGetSubkeyNames(HKEY_CURRENT_USER, UninstallRoot, names) then
for i := 0 to GetArrayLength(names) - 1 do
if (Pos(prefix, Uppercase(names[i])) = 1) and (Uppercase(names[i]) <> cur) then
begin
SetArrayLength(Result, GetArrayLength(Result) + 1);
Result[GetArrayLength(Result) - 1] := names[i];
end;
end;
// Drives the "removeold" task: only offered when another version exists.
function OtherVersionsExist(): Boolean;
begin
Result := GetArrayLength(OtherTestiumSubkeys()) > 0;
end;
// Silently run each other version's uninstaller.
procedure RemoveOtherVersions();
var
subs: TArrayOfString;
i, rc: Integer;
cmd: string;
begin
subs := OtherTestiumSubkeys();
for i := 0 to GetArrayLength(subs) - 1 do
if RegQueryStringValue(HKEY_CURRENT_USER, UninstallRoot + '\' + subs[i],
'UninstallString', cmd) and (cmd <> '') then
Exec(RemoveQuotes(cmd), '/VERYSILENT /SUPPRESSMSGBOXES /NORESTART',
'', SW_HIDE, ewWaitUntilTerminated, rc);
end;
// True if Param is not already a full segment of the per-user PATH.
function NeedsAddPath(Param: string): Boolean;
@@ -97,6 +148,8 @@ begin
Path := Path + ExpandConstant('{app}');
RegWriteStringValue(HKEY_CURRENT_USER, EnvKey, 'Path', Path);
end;
if WizardIsTaskSelected('removeold') then
RemoveOtherVersions();
end;
end;

View File

@@ -79,26 +79,60 @@ a = Analysis(
)
pyz = PYZ(a.pure)
exe = EXE(
pyz,
a.scripts,
a.binaries,
a.datas,
[],
name='testium',
debug=False,
bootloader_ignore_signals=False,
strip=False,
# UPX is CPU+IO heavy for a marginal size gain — build_all --ram sets
# TESTIUM_NO_UPX=1 to skip it (much faster on slow/flash storage).
upx=not os.environ.get("TESTIUM_NO_UPX"),
upx_exclude=[],
runtime_tmpdir=None,
console=False,
disable_windowed_traceback=False,
argv_emulation=False,
target_arch=None,
codesign_identity=None,
entitlements_file=None,
ico='../testium.ico'
)
# TESTIUM_ONEDIR=1 => one-folder build (fast startup), used by the Windows
# installer; default one-file keeps the Linux build_all portable binary.
ONEDIR = bool(os.environ.get("TESTIUM_ONEDIR"))
# UPX skipped via TESTIUM_NO_UPX (build_all --ram) — slow for a marginal gain.
_upx = not os.environ.get("TESTIUM_NO_UPX")
if ONEDIR:
exe = EXE(
pyz,
a.scripts,
[],
exclude_binaries=True,
name='testium',
debug=False,
bootloader_ignore_signals=False,
strip=False,
upx=_upx,
upx_exclude=[],
console=False,
disable_windowed_traceback=False,
argv_emulation=False,
target_arch=None,
codesign_identity=None,
entitlements_file=None,
ico='../testium.ico'
)
coll = COLLECT(
exe,
a.binaries,
a.datas,
strip=False,
upx=_upx,
upx_exclude=[],
name='testium',
)
else:
exe = EXE(
pyz,
a.scripts,
a.binaries,
a.datas,
[],
name='testium',
debug=False,
bootloader_ignore_signals=False,
strip=False,
upx=_upx,
upx_exclude=[],
runtime_tmpdir=None,
console=False,
disable_windowed_traceback=False,
argv_emulation=False,
target_arch=None,
codesign_identity=None,
entitlements_file=None,
ico='../testium.ico'
)

View File

@@ -1,3 +1,22 @@
version 0.3.2
==============
- The variables window (F1) now has a filter box: type to show only the
variables whose name matches. Tick "values" to also match on the value.
- The ``run`` item now captures the output of the test it launches into your
log and report (it used to go only to the terminal and was lost from the
report). New ``batch: true`` option runs that test headless (and captured)
even when testium is in the GUI.
- The captured output of a ``run`` step can be saved with ``store_result`` and
inspected afterwards (for example with ``expected_result`` or a ``py_func``).
version 0.3.1
==============
- Clearer errors when a test file fails to load. The message now names the
exact file and item and explains the problem (unknown item or action, a
step holding two items, a missing ``steps:`` list, a misplaced value, ...)
and lists the valid names, instead of a cryptic failure. A problem inside
an ``!include``-d file points to that file.
version 0.3
==============
- New ``pytest`` test item: run your pytest files as a test step; each

View File

@@ -1 +1 @@
0.3
0.3.2

View File

@@ -39,20 +39,36 @@ class TestItemActions(TestItem):
def load(self):
ret = {}
if self.dict_actions is None:
self.dict_actions = []
if not isinstance(self.dict_actions, (list, tuple)):
raise ETUMSyntaxError(
f"The '{self.cmd()}' test item named '{self.name()}' expects a "
f"list of actions under 'steps' but got "
f"{type(self.dict_actions).__name__} ({self.dict_actions!r}).",
self.seqFilename()
)
known_actions = ", ".join(sorted(self.action_classes.keys())) or "(none)"
for action in self.dict_actions:
# Action should be only dict of length 1
if not isinstance(action, dict) or (not len(action) == 1):
# Each action must be a single-key mapping ``{action_name: {...}}``.
if not isinstance(action, dict) or len(action) != 1:
raise ETUMSyntaxError(
f"The '{self.cmd()}' test item named '{self.name()}' action should be only dict of length = 1.",
f"The '{self.cmd()}' test item named '{self.name()}' has an "
f"invalid action: each action must be a single-key mapping "
f"('<action>: ...'), got {type(action).__name__} ({action!r}).",
self.seqFilename()
)
action_name = list(action.keys())[0]
if not (action_name in self.action_classes.keys()):
if action_name not in self.action_classes:
raise ETUMSyntaxError(
f"The '{self.cmd()}' test item named '{self.name()}' has an unknown action '{action.keys()[0]}'.",
f"The '{self.cmd()}' test item named '{self.name()}' has an "
f"unknown action '{action_name}'.\n"
f"Known actions: {known_actions}.",
self.seqFilename()
)
# NB: an action body is not necessarily a mapping — several actions
# accept a scalar shorthand (e.g. ``writeln: 'echo hi'``); the action
# class validates its own body. Pass it through untouched.
item = (self.action_classes[action_name])(
action_name,
action[action_name],

View File

@@ -11,6 +11,7 @@ from interpreter.test_items.test_result import (TestValue)
import api.testium as tm
from interpreter.utils.constants import TestItemType as cst
from interpreter.utils.param_decl import Param, ParamSet
from interpreter.utils.proc_drain import drain_to_log
from runtime.tum_except import ETUMSyntaxError, ETUMRuntimeError, item_load_context
@@ -75,6 +76,9 @@ class TestItemRun(TestItem):
Param("wait_for_exec",
doc="If true, block until the time window opens. Requires both "
"start_time and end_time."),
Param("batch", default=False,
doc="Run the sub-instance headless (-b) with its output captured "
"into this test's log/report and result value, even in the GUI."),
)
def __init__(self, dict_item, parent = None, status_queue=None, filename=""):
@@ -90,6 +94,38 @@ class TestItemRun(TestItem):
self.start_time = self._prms.getParam('start_time')
self.end_time = self._prms.getParam('end_time')
self.wait_for_exec = self._prms.getParam('wait_for_exec')
self.batch = self._prms.getParam('batch', default=False)
def _launch(self, cmd, capture):
"""Run the sub-instance once. When *capture*, stream its output to the
log/report, keep it as the result value, and let Stop kill the child."""
if not capture:
subprocess.run(cmd)
return
sink = []
prefix = f"[{os.path.basename(self.tum_file)}] "
proc = subprocess.Popen(
cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE
)
threads = drain_to_log(proc, prefix=prefix, sink=sink)
try:
while True:
try:
proc.wait(timeout=0.2)
break
except subprocess.TimeoutExpired:
if self.isStopped():
proc.terminate()
try:
proc.wait(timeout=2)
except subprocess.TimeoutExpired:
proc.kill()
break
finally:
for t in threads:
t.join(timeout=2)
# Captured log -> result value (store_result / expected_result).
self.result.value = "\n".join(sink)
@test_run
def execute(self):
@@ -104,25 +140,26 @@ class TestItemRun(TestItem):
pf = self._prms.expanse(self.param_file)
lp = self._prms.expanse(self.log_path)
rp = self._prms.expanse(self.report_path)
# Capture (headless -b) in batch or when `batch: true`; else open
# the child's own GUI window (-r).
capture = bool(self.batch) or tm.text_mode()
cmd = _testium_launch_cmd()
if tm.text_mode():
cmd.append("-b")
if capture:
cmd += ["-b", "-o"] # -o: no colour codes in the captured log
else:
cmd.append("-r")
if lp == '':
lp = os.path.splitext(self.tum_file)[0] + "_" + \
datetime.utcnow().isoformat(timespec='seconds') + '.log'
cmd.append("-l")
cmd.append('"' + lp + '"')
cmd += ["-l", '"' + lp + '"']
if pf != '':
cmd.append("-c")
cmd.append('"' + pf + '"')
cmd += ["-c", '"' + pf + '"']
if rp != '':
cmd.append("-p")
cmd.append('"' + rp + '"')
cmd += ["-p", '"' + rp + '"']
cmd.append(self.tum_file)
for c in cmd:
print(c, end = ' ')
print(" ".join(cmd))
if self.start_time is not None:
self.start_time = datetime.strptime(
@@ -135,20 +172,24 @@ class TestItemRun(TestItem):
raise ETUMRuntimeError(
'"wait_for_exec" set but not start_time or end_time')
r = None
ran = False
if self.wait_for_exec:
while not nowInBetween(self.start_time, self.end_time):
sleep(60)
r = subprocess.run(cmd)
self._launch(cmd, capture)
ran = True
elif self.start_time is not None and self.end_time is not None:
if nowInBetween(self.start_time, self.end_time):
r = subprocess.run(cmd)
self._launch(cmd, capture)
ran = True
elif self.start_time is not None:
if self.start_time < datetime.now().time():
r = subprocess.run(cmd)
self._launch(cmd, capture)
ran = True
else:
r = subprocess.run(cmd)
if isinstance(r, subprocess.CompletedProcess):
self._launch(cmd, capture)
ran = True
if ran:
self.result.set(TestValue.SUCCESS)
else:
self.result.set(TestValue.FAILURE, 'Sub-test did not execute')

View File

@@ -3,7 +3,7 @@ import datetime
from queue import Queue
from interpreter.utils.params import expanse
import api.testium as tm
from runtime.tum_except import ETUMSyntaxError
from runtime.tum_except import ETUMSyntaxError, ETUMError
import interpreter.utils.settings as prefs
from interpreter.test_report.test_report import TestReport
from interpreter.utils.py_func_exec import PyFuncExecEngine
@@ -65,9 +65,22 @@ def _flatten_actions(actions, out, parent_seq_name):
f"Syntax error in '{parent_seq_name}' step number {idx+1}. Sequence definition: '{str(action)}'",
f
)
if not isinstance(sequence, list):
raise ETUMSyntaxError(
f"Invalid included sequence in '{parent_seq_name}' "
f"(step {idx+1}): expected a list of steps, got "
f"{type(sequence).__name__}.",
f
)
for s in sequence:
if isinstance(s, dict) and s:
s[list(s.keys())[0]]["seq_filename"] = f
# Propagate the source filename onto each included step. Only a
# single-key mapping with a mapping body can carry it; malformed
# entries are left untouched and reported by the loader below,
# with their real location.
if isinstance(s, dict) and len(s) == 1:
body = s[next(iter(s))]
if isinstance(body, dict):
body["seq_filename"] = f
_flatten_actions(sequence, out, parent_seq_name)
continue
@@ -390,7 +403,19 @@ class TestSet:
self._rootItem = (cst_type.TYPE_ROOT.item_class)(
dict_item=dict_main, status_queue=self.status_queue
)
ret = self.load_test_recursively(self._rootItem, dict_main, filename)
try:
ret = self.load_test_recursively(self._rootItem, dict_main, filename)
except ETUMError:
# Already a located, user-readable testium error.
raise
except Exception as e:
# Last-resort net: turn any unforeseen failure into a located error
# rather than a bare traceback / 'crashed for any reason'.
raise ETUMSyntaxError(
f"Unexpected error while building the test tree: "
f"{type(e).__name__}: {e}",
filename
) from e
self.set_post_exec()
return ret
@@ -467,30 +492,43 @@ class TestSet:
def load_test_recursively(self, tree_parent, parent_seq, file_name):
ret = {}
path = _build_item_path(tree_parent)
if not isinstance(parent_seq, dict):
raise ETUMSyntaxError(
f"In: {path}\n"
f"The body of '{tree_parent.cmd()}' must be a mapping (with a "
f"'steps' list) but is {type(parent_seq).__name__} "
f"({parent_seq!r}).",
file_name
)
try:
parent_seq_name = parent_seq["name"]
except KeyError:
parent_seq["name"] = "sequence"
except TypeError:
raise ETUMSyntaxError(
f"No 'name' attribute in '{tree_parent.type()}' (a child of '{tree_parent.parent().name()}')",
file_name
)
parent_seq_name = "sequence"
try:
parent_seq_actions = parent_seq["steps"]
except KeyError:
raise ETUMSyntaxError(
f"No step list found for '{parent_seq_name}' sequence. \n" +
f"Check the syntax of the 'steps' parameter of the '{tree_parent.cmd()}' test item definition.",
f"In: {path}\n"
f"No 'steps' list found for the '{tree_parent.cmd()}' item "
f"'{parent_seq_name}'.\n"
f"A container item must declare its children under 'steps:'.",
file_name
)
# if action is a dictionary , we assume it is a single action
# that has not been nested in a list, so do it
if isinstance(parent_seq_actions, (dict)):
parent_seq_actions = [parent_seq_actions]
# an empty 'steps:' (None) is a valid, empty sequence
if parent_seq_actions is None:
parent_seq_actions = []
if not isinstance(parent_seq_actions, (list, tuple)):
raise ETUMSyntaxError(
f"No valid list of actions in sequence {parent_seq_name}",
f"In: {path}\n"
f"The 'steps' of '{parent_seq_name}' must be a list of test "
f"items but is {type(parent_seq_actions).__name__} "
f"({parent_seq_actions!r}).",
file_name
)
test_dir = tm.gd("test_directory")
@@ -502,10 +540,50 @@ class TestSet:
_flatten_actions(parent_seq_actions, flat_actions, parent_seq_name)
for action in flat_actions:
# Action is now for sure a dict of length 1
# After flattening, each step must be a single-key mapping
# '{item_cmd: {params...}}'. Anything else is a structural mistake
# in the .tum (a stray scalar, a missing '-' marker, an over- or
# under-indented block) — report it with its location instead of
# crashing on it below.
if not isinstance(action, dict):
raise ETUMSyntaxError(
f"In: {path}\n"
f"A step is not a valid test item: expected a "
f"'<item>: ...' mapping but got {type(action).__name__} "
f"({action!r}).\n"
f"Check the indentation and the '-' list markers of 'steps'.",
file_name
)
if len(action) != 1:
raise ETUMSyntaxError(
f"In: {path}\n"
f"A step must define exactly one test item but defines "
f"{len(action)}: {sorted(map(str, action.keys()))}.\n"
f"Each '-' step holds a single '<item>:'; the lines below it "
f"are probably its parameters and need one more indent level.",
file_name
)
k = list(action.keys())[0]
if action[k].get("seq_filename", None) is None:
action[k]["seq_filename"] = file_name
# The body of an item is its parameter mapping. A bare '<item>:'
# (None) is tolerated as an empty parameter set; a scalar or list is
# a structural mistake and is reported with its location.
body = action[k]
if body is None:
body = {}
action[k] = body
if not isinstance(body, dict):
raise ETUMSyntaxError(
f"In: {path}\n"
f"The body of test item '{k}' must be a mapping of "
f"parameters but is {type(body).__name__} ({body!r}).",
file_name
)
if body.get("seq_filename", None) is None:
body["seq_filename"] = file_name
seq_filename = body["seq_filename"]
executed = False
for it in TEST_TYPE_LIST:
@@ -517,32 +595,18 @@ class TestSet:
(it.item_class is None)
):
continue
if (it.item_cmd in action) or (
(cst.FOLDED_CHAR + it.item_cmd) in action
):
executed = True
is_folded = False
action_name = it.item_cmd
# Check if a "." is before the cmd_name (meaning folded)
if (cst.FOLDED_CHAR + it.item_cmd) in action:
is_folded = True
action_name = cst.FOLDED_CHAR + it.item_cmd
seq_filename = action[action_name]["seq_filename"]
try:
item = (it.item_class)(
action[action_name],
tree_parent,
self.status_queue,
filename=seq_filename
)
except ETUMSyntaxError as e:
path = _build_item_path(tree_parent)
raise ETUMSyntaxError(
f"In: {path}\n{e._message}",
e._file or seq_filename,
) from e
if k not in (it.item_cmd, cst.FOLDED_CHAR + it.item_cmd):
continue
executed = True
# A "." before the cmd name means the item is folded in the GUI
is_folded = k.startswith(cst.FOLDED_CHAR)
try:
item = (it.item_class)(
body,
tree_parent,
self.status_queue,
filename=seq_filename
)
item.is_folded = is_folded
child = {}
# case where the test item loads itself its descendants
@@ -554,15 +618,42 @@ class TestSet:
# case where the test item is an items container
elif item.is_container:
child = self.load_test_recursively(
item, action[action_name], seq_filename
item, body, seq_filename
)
except ETUMSyntaxError as e:
# Already a syntax error: prepend the breadcrumb to its
# location (unless it already carries one from a deeper level).
msg = e._message
if not msg.lstrip().startswith("In:"):
msg = f"In: {path} > {k}\n{msg}"
raise ETUMSyntaxError(msg, e._file or seq_filename) from e
except ETUMError:
# Other testium errors (missing parameter, runtime, I/O)
# already carry structured context (item type, name,
# parameter, ...): let them through unchanged.
raise
except Exception as e:
# Anything unexpected: never let a raw Python error reach the
# user as 'crashed for any reason' — locate it precisely.
raise ETUMSyntaxError(
f"In: {path} > {k}\n"
f"Unexpected error while loading this item: "
f"{type(e).__name__}: {e}",
seq_filename
) from e
ret.update(test_data(item, child))
ret.update(test_data(item, child))
if not executed:
known = ", ".join(
t.item_cmd for t in TEST_TYPE_LIST
if t is not cst_type.TYPE_ROOT and t.item_class is not None
)
raise ETUMSyntaxError(
f"test item '{k}' is not known.",
action[k]["seq_filename"]
f"In: {path}\n"
f"'{k}' is not a known test item.\n"
f"Known items: {known}.",
seq_filename
)
return ret

View File

@@ -199,6 +199,23 @@ def host_console_command(shell_cmd, cwd):
return ["flatpak-spawn", "--host", f"--directory={cwd}", *argv]
def host_open_path(path):
"""Open *path* with the host default application (Flatpak only).
QDesktopServices/openUrl routes through the OpenURI portal inside Flatpak,
which often fails to open a plain editor for a log file. Spawn xdg-open on
the host so the user's real default app is used. Returns True on dispatch;
False (incl. outside Flatpak) so the caller can fall back to openUrl.
"""
if not _in_flatpak():
return False
try:
subprocess.Popen(["flatpak-spawn", "--host", "xdg-open", path])
return True
except (FileNotFoundError, PermissionError):
return False
def _which_host_flatpak(name):
"""Resolve a binary name (or absolute path) on the host via flatpak-spawn.

View File

@@ -13,7 +13,7 @@ from time import monotonic
from runtime.jrpc import RPC_PORT_SENTINEL
def _drain_pipe(pipe, prefix):
def _drain_pipe(pipe, prefix, sink=None):
try:
for raw in iter(pipe.readline, b""):
line = raw.decode("utf-8", errors="replace").rstrip("\r\n")
@@ -23,6 +23,9 @@ def _drain_pipe(pipe, prefix):
print(f"{prefix}{line}")
else:
print(line)
# sink keeps the clean (unprefixed) line for reuse as a result value
if sink is not None:
sink.append(line)
finally:
try:
pipe.close()
@@ -30,21 +33,16 @@ def _drain_pipe(pipe, prefix):
pass
def drain_to_log(process, prefix=""):
"""Spawn daemon threads that read ``process.stdout`` and
``process.stderr`` line by line and print each line through the
parent's stdout (so it reaches the log + live output).
Each thread exits cleanly when the subprocess closes the
corresponding pipe (i.e. when it exits). Daemon flag ensures they
do not block testium exit.
"""
def drain_to_log(process, prefix="", sink=None):
"""Stream the subprocess stdout/stderr line by line through the parent's
print pipeline (log + live output). If ``sink`` is a list, each clean line
is also appended to it (GIL-atomic, shared by both threads). Daemon threads."""
threads = []
for pipe in (process.stdout, process.stderr):
if pipe is None:
continue
t = threading.Thread(
target=_drain_pipe, args=(pipe, prefix), daemon=True,
target=_drain_pipe, args=(pipe, prefix, sink), daemon=True,
)
t.start()
threads.append(t)

View File

@@ -6,8 +6,8 @@ import subprocess
import sys
from PySide6.QtWidgets import (
QDialog, QDialogButtonBox, QHeaderView, QMenu, QMessageBox,
QPushButton, QTextEdit, QVBoxLayout,
QCheckBox, QDialog, QDialogButtonBox, QHBoxLayout, QHeaderView, QLineEdit,
QMenu, QMessageBox, QPushButton, QTextEdit, QVBoxLayout,
)
from PySide6.QtGui import QSyntaxHighlighter, QTextCharFormat, QColor, QFont, QDesktopServices
from PySide6.QtCore import Qt, QUrl, Slot
@@ -119,6 +119,43 @@ class DialogF1(QDialog):
self.ui.addVarButton.setEnabled(False)
self.ui.addVarButton.clicked.connect(self._on_add_var)
# Filter box above the table: hides rows whose name doesn't match.
# The optional "values" checkbox extends the match to the value column.
self._filter_text = ""
self._filter_edit = QLineEdit(self.ui.tabVariables)
self._filter_edit.setPlaceholderText("Filter variables by name")
self._filter_edit.setClearButtonEnabled(True)
self._filter_edit.textChanged.connect(self._on_filter_changed)
self._filter_values_cb = QCheckBox("values", self.ui.tabVariables)
self._filter_values_cb.setToolTip("Also match on the variable value")
self._filter_values_cb.toggled.connect(lambda _checked: self._apply_filter())
filter_row = QHBoxLayout()
filter_row.addWidget(self._filter_edit)
filter_row.addWidget(self._filter_values_cb)
self.ui.verticalLayout_tab1.insertLayout(0, filter_row)
def _on_filter_changed(self, text):
self._filter_text = text.strip().lower()
self._apply_filter()
def _apply_filter(self):
for row in range(self.ui.varsTable.rowCount()):
self._apply_filter_row(row)
def _apply_filter_row(self, row):
needle = self._filter_text
if not needle:
self.ui.varsTable.setRowHidden(row, False)
return
table = self.ui.varsTable
key_item = table.item(row, 0)
hay = key_item.text().lower() if key_item else ""
if self._filter_values_cb.isChecked():
val_item = table.item(row, 1)
if val_item is not None:
hay += "\n" + val_item.text().lower()
table.setRowHidden(row, needle not in hay)
def load_initial_vars(self, vars_dict: dict):
for key, value in vars_dict.items():
self.gd_var_updated(key, value)
@@ -149,6 +186,7 @@ class DialogF1(QDialog):
self._updating = False
self._key_rows[key] = row
self._refresh_row(row, key, value)
self._apply_filter_row(self._key_rows[key])
@Slot(str)
def gd_var_deleted(self, key):
@@ -161,6 +199,7 @@ class DialogF1(QDialog):
finally:
self._updating = False
self._key_rows = {k: (r - 1 if r > row else r) for k, r in self._key_rows.items()}
self._apply_filter()
def _refresh_row(self, row, key, value):
from PySide6.QtWidgets import QTableWidgetItem

View File

@@ -127,7 +127,8 @@ class TestFileManager:
del w.ts_controller
w.ts_controller = None
raise ETUMRuntimeError(
"Test could not be loaded (test process crashed for any reason)"
"Test could not be loaded. See the log above for the cause "
"(syntax error, missing file, missing module, ...)."
)
progress.setLabelText("Building test tree…")

View File

@@ -181,7 +181,8 @@ class TestRunner:
w.actionStart_test.setText("Pause test")
w.actionPreferences.setDisabled(True)
w.actionRefresh_test.setDisabled(True)
w.actionShow_Results.setDisabled(True)
# Show Results stays available during the run (log grows live).
w.actionShow_Results.setEnabled(True)
w.actionSave_report.setDisabled(True)
w.logSettingsBox.setDisabled(True)
w.actionStop_test.setEnabled(True)

View File

@@ -40,6 +40,7 @@ from runtime.string_queue import StringQueue
from interpreter.process import TestProcess
from interpreter.utils.test_ctrl import TestSetController
from interpreter.utils.icons import icon_prefix
from interpreter.utils import bins
from main_win.test_run.outlog import OutLog
from main_win.test_run.test_run import ThreadTestStatus
@@ -639,7 +640,8 @@ class MainWindow(QMainWindow, Ui_MainWindow):
self.statusBar().showMessage(
"Opening the logfile (" + s + "): " + self.logFileName, 100000
)
QDesktopServices.openUrl(QUrl.fromLocalFile(self.logFileName))
if not bins.host_open_path(self.logFileName):
QDesktopServices.openUrl(QUrl.fromLocalFile(self.logFileName))
@Slot()
def on_actionHelp_triggered(self):

View File

@@ -47,13 +47,15 @@
{% endif %}
- read_until: {expected: terminal loaded, timeout: 5}
# Echo two tokens on one line so both are buffered together; the immediate
# (timeout 0) reads below match buffered data with no race on the async prompt.
- console:
name: Console write
condition: <| $(conditional_exec) == 1 |>
console_name: consname
key: $(test)_PASS
steps:
- writeln: echo 0
- writeln: echo ALPHA BETA
- sleep:
name: sleep item
@@ -67,9 +69,9 @@
key: $(test)_PASS
steps:
{% if os == "Windows" %}
- read_until: {expected: echo 0, timeout: 0}
- read_until: {expected: echo ALPHA BETA, timeout: 0}
{% endif %}
- read_until: {expected: "0", timeout: 0}
- read_until: {expected: ALPHA, timeout: 0}
- console:
name: Console read_until immediate (2)
@@ -77,7 +79,7 @@
console_name: consname
key: $(test)_PASS
steps:
- read_until: {expected: "$(terminal_prompt)", timeout: 0}
- read_until: {expected: BETA, timeout: 0}
- console:
name: Console closure

View File

@@ -0,0 +1,9 @@
import py_func.tm as tm
def assert_captured():
"""The sub-run log stored by `run` via store_result must be in the gdict."""
log = tm.gd("captured_log", "")
assert "Test run success." in log, \
"captured sub-run log not reachable from the gdict (store_result)"
return 0

View File

@@ -1,25 +1,44 @@
# run item: launches a .tum file in a new testium instance.
# In batch mode the sub-instance runs with -b; in GUI mode with -r.
# The run item result is SUCCESS if the sub-instance launched successfully,
# regardless of its own test result.
# Child mode: -b in batch / -r in the GUI, or forced -b (captured) by batch: true.
# Result is SUCCESS if the sub-instance launched, regardless of its own result.
# log_file (GUI -r only) goes to the gitignored report dir to avoid repo litter.
- run:
name: run PASS (valid file, passing sub-test)
key: $(test)_PASS
tum: $(test_path)$(psep)sub_pass.tum
log_file: $(validation_report_path)$(psep)run_sub.log
- run:
name: run PASS (valid file, failing sub-test)
key: $(test)_PASS
tum: $(test_path)$(psep)sub_fail.tum
log_file: $(validation_report_path)$(psep)run_sub.log
- run:
name: run FAIL (file not found)
key: $(test)_FAIL
tum: $(test_path)$(psep)non_existent.tum
log_file: $(validation_report_path)$(psep)run_sub.log
- run:
name: run FAIL (wait_for_exec without time window)
key: $(test)_FAIL
tum: $(test_path)$(psep)sub_pass.tum
wait_for_exec: true
log_file: $(validation_report_path)$(psep)run_sub.log
# batch: true forces a headless, captured sub-run even in the GUI; its log is
# kept as the result value and pushed to the gdict by store_result.
- run:
name: run batch (capture sub-run log to the gdict)
key: $(test)_PASS
tum: $(test_path)$(psep)sub_pass.tum
batch: true
store_result: captured_log
- py_func:
name: captured sub-run log is post-processable from the gdict
key: $(test)_PASS
file: $(test_path)$(psep)check_capture.py
func_name: assert_captured

View File

@@ -0,0 +1,9 @@
main:
name: root
steps:
- sleep:
name: ok
timeout: 0
# The structural error lives in the included file; the message must point
# the user at that file, not at this one.
- !include bad_include_inc.tum

View File

@@ -0,0 +1,4 @@
# Included as a bare list of steps. The unknown item below must be reported
# with THIS file as the location.
- frobnicate_in_include:
name: nope

View File

@@ -0,0 +1,6 @@
main:
name: root
steps:
# A container item (group) without its mandatory 'steps:' list.
- group:
name: g

View File

@@ -0,0 +1,5 @@
main:
name: root
steps:
# The body of an item must be a mapping of parameters, not a scalar.
- sleep: 5

View File

@@ -0,0 +1,5 @@
main:
name: root
steps:
# A step that is a bare scalar instead of a '<item>: ...' mapping.
- just some text

View File

@@ -0,0 +1,11 @@
main:
name: root
steps:
# Two items wrongly packed under a single '-' marker (a frequent indent
# mistake): the second key belongs one '-' lower.
- sleep:
name: s
timeout: 0
group:
name: g
steps: []

View File

@@ -0,0 +1,8 @@
main:
name: root
steps:
- console:
console_name: c1
steps:
- opens:
device: /dev/ttyUSB0

View File

@@ -0,0 +1,5 @@
main:
name: root
steps:
- frobnicate:
name: nope

View File

@@ -0,0 +1,87 @@
#!/usr/bin/env python3
"""Per-channel check of test-load error reporting.
Given the channel's testium invocation as argv (e.g. ``flatpak run
--command=testium org.testium.Testium``, a PyInstaller binary path, or
``python -m testium``), load each deliberately broken ``.tum`` under
``load_errors/`` in batch mode and verify that:
1. the load FAILS (non-zero exit), and
2. the output carries the *specific, located* message we expect — not a bare
Python traceback and not the generic 'crashed for any reason'.
This guards the load-time error handling in ``test_set.load_test_recursively``
and ``item_actions.load`` (a structural mistake in a ``.tum`` must always reach
the user as a readable ``TUM file syntax error`` naming the offending file,
item path and value). The historical failure mode was an unknown console
action crashing the error formatter itself with ``'dict_keys' object is not
subscriptable``.
Exits non-zero (with a diagnostic) on the first failure so the validation run
fails loudly. Used by ``run.sh`` before launching the main suite.
"""
import os
import re
import subprocess
import sys
HERE = os.path.dirname(os.path.abspath(__file__))
FIXTURES = os.path.join(HERE, "load_errors")
# testium colourises its log; strip the ANSI escapes before matching messages.
_ANSI = re.compile(r"\x1b\[[0-9;]*m")
# fixture file -> substrings that must all appear in the load output.
CASES = [
("unknown_item.tum", ["TUM file syntax error", "is not a known test item",
"frobnicate", "Known items:"]),
("unknown_action.tum", ["unknown action", "opens", "Known actions:"]),
("two_steps.tum", ["must define exactly one test item"]),
("scalar_body.tum", ["body of test item 'sleep'", "must be a mapping"]),
("group_no_steps.tum", ["No 'steps' list found", "'group' item 'g'"]),
("step_not_mapping.tum", ["is not a valid test item"]),
# The error is inside the included file: the message must name that file.
("bad_include.tum", ["bad_include_inc.tum", "frobnicate_in_include",
"is not a known test item"]),
]
def fail(msg):
print(f"LOAD-ERROR CHECK: FAIL — {msg}", file=sys.stderr)
sys.exit(1)
def check_case(cmd, fixture, needles):
path = os.path.join(FIXTURES, fixture)
try:
out = subprocess.run(cmd + ["-b", path], capture_output=True, timeout=120)
except Exception as e: # noqa: BLE001
fail(f"`{' '.join(cmd)} -b {fixture}` could not run: {e}")
blob = _ANSI.sub("", (out.stdout + out.stderr).decode(errors="replace"))
if out.returncode == 0 or "Test run success." in blob:
fail(f"{fixture}: load was expected to fail but succeeded "
f"(exit {out.returncode}).")
# A raw Python traceback reaching the user is exactly what we are guarding
# against: every load error must be funnelled through a TUM*Error.
if "Traceback (most recent call last)" in blob:
fail(f"{fixture}: a raw Python traceback leaked to the user:\n"
f"{blob[-600:]}")
missing = [n for n in needles if n not in blob]
if missing:
fail(f"{fixture}: load message is missing {missing}.\n"
f"--- got ---\n{blob[-800:]}")
print(f"LOAD-ERROR CHECK: {fixture} OK")
def main():
cmd = sys.argv[1:]
if not cmd:
fail("usage: load_errors_check.py <testium-invocation...>")
for fixture, needles in CASES:
check_case(cmd, fixture, needles)
print("LOAD-ERROR CHECK: PASS")
if __name__ == "__main__":
main()

View File

@@ -3,11 +3,15 @@
# testium (source, wheel, pyinstaller, flatpak, appimage).
#
# Usage:
# ./test/validation/run.sh [clean] [--mode MODE] [extra testium args]
# ./test/validation/run.sh [clean] [--mode MODE] [--gui] [extra testium args]
#
# clean remove the validation venv before recreating it
# (must be the first argument; useful after a Python upgrade)
#
# --gui open the GUI with the suite loaded instead of running in
# batch; run it manually from the window, which stays open
# (handy to inspect the tree, try the Ctrl+F search, ...)
#
# --mode MODE which testium build to validate. One of:
# source (default) src/testium via project run.sh
# wheel dist/testium-<v>-py3-none-any.whl
@@ -45,6 +49,8 @@ else
fi
EXTRA=()
RUN_FLAGS=(-b) # batch by default; --gui opens the GUI and stays open
GUI=0
while [ $# -gt 0 ]; do
case "$1" in
--mode)
@@ -55,6 +61,11 @@ while [ $# -gt 0 ]; do
MODE="${1#--mode=}"
shift
;;
--gui)
GUI=1
RUN_FLAGS=() # no -b: launch the GUI with the suite loaded,
shift # run it manually; the window does not auto-close
;;
*)
EXTRA+=("$1")
shift
@@ -147,7 +158,17 @@ echo "-- launch: ${CMD[*]}"
echo "-- LSP check ($MODE)"
"$VENV_PYTHON" "$SCRIPT_DIR/lsp_check.py" "${CMD[@]}"
exec "${CMD[@]}" -b \
# ---------- load-error check (this exact channel) -----------------------------
# Deliberately broken .tum files must fail to load with a specific, located
# message (not a raw traceback): guards the load-time error handling.
echo "-- load-error check ($MODE)"
"$VENV_PYTHON" "$SCRIPT_DIR/load_errors_check.py" "${CMD[@]}"
if [ "$GUI" -eq 1 ]; then
echo "-- GUI mode: the suite is loaded; press Start to run. Window stays open."
fi
exec "${CMD[@]}" "${RUN_FLAGS[@]}" \
-d "python_bin=$VENV_PYTHON" \
-d "validation_report_file=validation-$MODE" \
-- "$SCRIPT_DIR/main.tum" "${EXTRA[@]}"