refactor(settings): defaults carried by SettingsItem, getters/setters via _pref
refactor(pref-win): declarative Field table + _FIELD bridge + merged file pickers
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
load_errors_check.py loads deliberately broken .tum fixtures in batch on the
build under test (like lsp_check.py) and asserts each fails with its specific
located message and without a raw traceback. Wired into run.sh just before the
main suite, so it runs for every channel.
The run validation items now point their sub-instance log at the gitignored
report dir, so a GUI run no longer litters the tree with sub_*.log files.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A structural mistake in a .tum (unknown item or action, a step holding two
items, a missing 'steps:' list, a scalar where a mapping is expected, ...)
used to surface as a bare Python traceback. At worst the unknown-action
formatter itself crashed with "'dict_keys' object is not subscriptable"
(action.keys()[0]), masking the real cause and leaving only the generic
"test process crashed for any reason".
The load path now validates each step and funnels every failure through a
located TUM file syntax error: the file, a breadcrumb to the item, the
offending value and the list of valid names. A problem inside an !include-d
file points to that file. A last-resort net in __loadTestTree turns any
unforeseen exception into a located error too.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds `--gui` to test/validation/run.sh: drops `-b` so testium opens the GUI
with the validation suite loaded instead of running headless. The run is
started manually and the window stays open — handy to inspect the test
tree, try the Ctrl+F search, etc. Works with any --mode.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Document the Ctrl+F find bar: field checkboxes, the blockSignals pass that
avoids the on_testChecked controller storm, and the flag-driven
_refresh_highlight (run > search > default) that keeps the run and search
highlight layers from clobbering each other.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Find bar over the test tree: highlight matches and navigate them
(Enter / ◂ ▸), with Name/Type/Doc checkboxes to choose the searched
fields. Ctrl+F toggles the bar (clearing the highlight); Esc / ✕ close.
- QTestTreeItem: matches_search(needle, fields) + a search highlight that
shares one _refresh_highlight() with the green run highlight, recomputed
from state flags (run > search > default) so the two layers never leave a
stale/permanent colour. Amber bg + forced black text → readable in any
theme.
- QTestTree.search()/clear_search(): single signal-blocked pass (setBackground
fires itemChanged → on_testChecked, a controller storm otherwise); expands
ancestors of matches; returns matches in visual order.
- MainWindow: the find bar widget + Ctrl+F shortcut + navigation; search is
reset when a new test file is loaded.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
"pytest step" children now show the pytest logo (pytest.png), matching the
parent "pytest" item, instead of the generic document icon. Icon already
embedded in the compiled resources — no rcc regen needed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The plugin was delivered by writing it to a temp dir, putting that dir on
PYTHONPATH and loading it with `python -m pytest -p _testium_pytest_plugin`.
That import-by-name failed in the AppImage runtime (ModuleNotFoundError:
_testium_pytest_plugin) so collection returned nothing and the item FAILed
— while wheel/pyinstaller/flatpak worked. Local sims forcing the AppImage
env path (apply_host_libs) passed, ruling out the env scrubbing.
Ship the plugin as a self-contained launcher run directly
(`python launcher.py ...`) that registers it as a plugin object via
pytest.main(plugins=[sys.modules[__name__]]): no PYTHONPATH, no `-p`, no
import-by-name. apply_host_libs is untouched. Verified on source, wheel,
pyinstaller, flatpak and AppImage.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
When collection finds nothing because pytest is missing on the host
interpreter, load() raises a dedicated message ("pytest is not installed
... pip install pytest") instead of the raw pytest output. The graceful
load mechanism surfaces it as a WARN at load + a clean FAIL at run, the
rest of the campaign keeps running.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A self-loading item that can't load its module/file (unittest test file
with a missing import, pytest not installed on the host, ...) no longer
aborts the whole test load. TestSet._load_item() wraps load(), warns at
load time and records item._load_error; @test_run turns it into a clean
run-time FAILURE. The rest of the campaign loads and runs.
Scoped to module-loading items (unittest; pytest once merged). Structural
action loading stays fail-fast.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Use the official pytest logo (devicon) for the pytest item instead of
reusing python.png — visually distinct from the py_func (Python logo)
item. Three 64x64 theme variants (color RGBA, black RGBA silhouette,
white LA), declared in the QRC and compiled into testium_core_win_rc.py;
_ITEM_CONFIG "pytest" now points to pytest.png.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
New test_items/pytest_test_item.rst (params, host-subprocess execution,
pytest-must-be-installed note), wired into the toctree. Regenerated PDF.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
0.3 is reserved for the upcoming pytest item release. Regenerated the
manual PDF so its version stamp matches.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Variable substitution ($(...)) must use the runtime global dict, so it
must happen at run time (execute), never at load (__init__).
- console telnet_port: was never expanded — `telnet_port: $(port)` stayed
literal. Now expanded at run (processed=True in execute, like the other
host/port params).
- test_item base: stop_on_failure / execute_on_stop are now stored raw and
resolved at run time via properties (so a $(...) flag reflects the
runtime value, not the load-time one).
- cycle iterator and git repo: drop the redundant load-time expansion
(execute() already re-expands them).
- tested_references: fetch 'reference' raw, expand each value in execute().
Justified load-time exceptions kept: name, doc, skipped (static/GUI at
load) and unittest test_method (drives child loading at load).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Document that read_until's 'expected' accepts a list (match any) and the
new 'regex' flag, with examples and the bounded-window limitation note.
Regenerated manual PDF.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
read_until:
- 'expected' now accepts a list of values (succeeds on any match).
- new 'regex: true' flag: each pattern is a Python regex (re.search over a
bounded tail, Console.REGEX_WINDOW). Reports which pattern matched.
Serial console robustness & clarity:
- failed open() raises a clear ETUMRuntimeError ("Serial device '…' does not
exist." / permission hint) instead of a raw pyserial traceback.
- a console whose open failed is safely "not open" (init _thd=None +
isOpened guards in readchar/read_nowait/close) — no more cascading
AttributeError: '_thd' on subsequent read steps.
- action handlers: one-liner for expected (ETUMRuntimeError) errors, full
traceback kept for unexpected ones. All console errors use testium
exceptions (ETUMRuntimeError).
Flatpak: grant --device=all so serial adapters (/dev/ttyUSB*, /dev/ttyACM*)
are visible in the sandbox.
Validation: new read_until list/regex (match + no-match) cases in
items/console/test.tum.
Version: 0.3.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Run a user pytest file as a testium item, surfacing each collected test
as a child with its own PASS/FAIL/SKIP, duration and failure message.
Mirrors the unittest item but runs pytest in a subprocess on the host
interpreter (bins.python_bin(), like py_func/lua_func) so it works across
every packaging channel. A stdlib-only pytest plugin streams collected
node-ids and per-test results over stdout via sentinels; the parent parses
them live. Params: test_file, test_method. stop_on_failure maps to -x;
disabled children are reported NORUN without running.
Wiring: TYPE_PYTEST / TYPE_PYTEST_STEP constants, test_init registration,
self-loading branch in test_set, GUI tree icon. Schema/LSP pick it up
automatically from the declarative PARAMS.
Validation: test/validation/items/pytest/ (validation venv now installs
pytest).
WIP: paused mid-feature (DESIGN.md documented; manual section pending).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- PyInstaller exe built windowed (console=False) with package/testium.ico
as the embedded icon (BMP entries for shell compatibility).
- Suppress stray subprocess console windows in the frozen Windows build via
paths.no_window_kwargs() (CREATE_NO_WINDOW); wheel/source unchanged.
Applied to py_process, lua_process, bins probes, sys_app_path_win.
- New per-user Inno Setup installer (package/innosetup/): no admin,
version-scoped AppId/dir so versions install side-by-side, one Start
Menu entry per version, .ico shipped for shortcut/uninstall icons.
- DESIGN.md + release_note.txt updated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
bins.ensure() now stores the resolved interpreter path under
python_bin / lua_bin when the key is unset, so test scripts can use
$(python_bin) / $(lua_bin) in GUI mode (no -d override). Restores the
behaviour lost when bins.py centralised resolution. A user-provided
value is left untouched.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The subprocess now binds port 0, prints the bound port on stdout after
listen(), and the parent connects only once it reads that port. Removes
the reserve/close/rebind race and SO_REUSEADDR, and the connect-before-
ready timing guess that failed intermittently on Windows. wait_ready()
no longer hangs when a connection attempt fails.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Make the suite run cleanly on Windows.
Product code:
- __init__.py: force UTF-8 on stdout/stderr. The Windows console code
page (cp1252) cannot encode the box-drawing/accented characters the
runner prints, which crashed the parent capture_stdout thread. Only
the stream encoders are reconfigured; the locale default used to read
cp1252 config files is left untouched.
- report_export_junit/html: open the report file with encoding="utf-8"
(XML/HTML are UTF-8) instead of the platform default, matching the
txt/json exporters.
Validation:
- run.bat: source mode now sets up its own venv and runs testium from
src\ directly instead of delegating to the project run.bat (which
launches the GUI and drops its arguments). Installs the fake_exporter
entry-point plugin (report_plugin) and the [lsp] extra, and runs the
same lsp_check.py pre-flight as run.sh.
- jsonrpc/test.tum: launch the echo server via "$(python_bin)" instead
of "python3" (the Microsoft Store stub on Windows).
- post_execution.py: write the JUnit XML with encoding="utf-8".
- restore items/run/sub_pass.tum and sub_fail.tum, deleted by mistake in
d97d00c "removed test logs".
- term console via flatpak-spawn --host so host venvs resolve (bins.host_console_command)
- QSettings sync() before subprocess kill in choices/tested-refs dialogs
- console regression test: fails on the in-sandbox 0.2.1 console
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Manual (modes.rst) and README: install the extension from Open VSX in
VSCodium/Cursor/etc., and as a .vsix by hand in Microsoft VSCode; note
that testium must be on PATH or set via testium.serverPath.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Set the app id via setDesktopFileName so the window stops inheriting the
launcher's class ("python3" under the AppImage), which is what GNOME was
keying the wrong icon off. On native Wayland the task-bar icon comes from
an installed desktop file matched to the app id (setWindowIcon is
ignored there), so on Linux drop an idempotent desktop entry + 256px icon
under ~/.local/share. Flatpak keeps its own id/desktop; Windows / macOS
use the window icon. No-op off Linux.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
load_test_recursively expanded nested lists and included 'sequence'
entries by splicing each into the step list and rebuilding the whole
list every time (O(n^2)). The list branch also rebuilt after an in-place
splice, duplicating entries when a nested list held more than one item.
Replace both with a single linear _flatten_actions pass. Build phase
~12% faster at 6k items; the real fix is the duplication (a nested
2-element list now yields a,b,c,d not a,b,c,c,d). Validation suite
identical (post-exec SUCCESS, same verdicts/tracebacks).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
