Adds `--gui` to test/validation/run.sh: drops `-b` so testium opens the GUI
with the validation suite loaded instead of running headless. The run is
started manually and the window stays open — handy to inspect the test
tree, try the Ctrl+F search, etc. Works with any --mode.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Document the Ctrl+F find bar: field checkboxes, the blockSignals pass that
avoids the on_testChecked controller storm, and the flag-driven
_refresh_highlight (run > search > default) that keeps the run and search
highlight layers from clobbering each other.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Find bar over the test tree: highlight matches and navigate them
(Enter / ◂ ▸), with Name/Type/Doc checkboxes to choose the searched
fields. Ctrl+F toggles the bar (clearing the highlight); Esc / ✕ close.
- QTestTreeItem: matches_search(needle, fields) + a search highlight that
shares one _refresh_highlight() with the green run highlight, recomputed
from state flags (run > search > default) so the two layers never leave a
stale/permanent colour. Amber bg + forced black text → readable in any
theme.
- QTestTree.search()/clear_search(): single signal-blocked pass (setBackground
fires itemChanged → on_testChecked, a controller storm otherwise); expands
ancestors of matches; returns matches in visual order.
- MainWindow: the find bar widget + Ctrl+F shortcut + navigation; search is
reset when a new test file is loaded.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
"pytest step" children now show the pytest logo (pytest.png), matching the
parent "pytest" item, instead of the generic document icon. Icon already
embedded in the compiled resources — no rcc regen needed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The plugin was delivered by writing it to a temp dir, putting that dir on
PYTHONPATH and loading it with `python -m pytest -p _testium_pytest_plugin`.
That import-by-name failed in the AppImage runtime (ModuleNotFoundError:
_testium_pytest_plugin) so collection returned nothing and the item FAILed
— while wheel/pyinstaller/flatpak worked. Local sims forcing the AppImage
env path (apply_host_libs) passed, ruling out the env scrubbing.
Ship the plugin as a self-contained launcher run directly
(`python launcher.py ...`) that registers it as a plugin object via
pytest.main(plugins=[sys.modules[__name__]]): no PYTHONPATH, no `-p`, no
import-by-name. apply_host_libs is untouched. Verified on source, wheel,
pyinstaller, flatpak and AppImage.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
When collection finds nothing because pytest is missing on the host
interpreter, load() raises a dedicated message ("pytest is not installed
... pip install pytest") instead of the raw pytest output. The graceful
load mechanism surfaces it as a WARN at load + a clean FAIL at run, the
rest of the campaign keeps running.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A self-loading item that can't load its module/file (unittest test file
with a missing import, pytest not installed on the host, ...) no longer
aborts the whole test load. TestSet._load_item() wraps load(), warns at
load time and records item._load_error; @test_run turns it into a clean
run-time FAILURE. The rest of the campaign loads and runs.
Scoped to module-loading items (unittest; pytest once merged). Structural
action loading stays fail-fast.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Use the official pytest logo (devicon) for the pytest item instead of
reusing python.png — visually distinct from the py_func (Python logo)
item. Three 64x64 theme variants (color RGBA, black RGBA silhouette,
white LA), declared in the QRC and compiled into testium_core_win_rc.py;
_ITEM_CONFIG "pytest" now points to pytest.png.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
New test_items/pytest_test_item.rst (params, host-subprocess execution,
pytest-must-be-installed note), wired into the toctree. Regenerated PDF.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
0.3 is reserved for the upcoming pytest item release. Regenerated the
manual PDF so its version stamp matches.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Variable substitution ($(...)) must use the runtime global dict, so it
must happen at run time (execute), never at load (__init__).
- console telnet_port: was never expanded — `telnet_port: $(port)` stayed
literal. Now expanded at run (processed=True in execute, like the other
host/port params).
- test_item base: stop_on_failure / execute_on_stop are now stored raw and
resolved at run time via properties (so a $(...) flag reflects the
runtime value, not the load-time one).
- cycle iterator and git repo: drop the redundant load-time expansion
(execute() already re-expands them).
- tested_references: fetch 'reference' raw, expand each value in execute().
Justified load-time exceptions kept: name, doc, skipped (static/GUI at
load) and unittest test_method (drives child loading at load).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Document that read_until's 'expected' accepts a list (match any) and the
new 'regex' flag, with examples and the bounded-window limitation note.
Regenerated manual PDF.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
read_until:
- 'expected' now accepts a list of values (succeeds on any match).
- new 'regex: true' flag: each pattern is a Python regex (re.search over a
bounded tail, Console.REGEX_WINDOW). Reports which pattern matched.
Serial console robustness & clarity:
- failed open() raises a clear ETUMRuntimeError ("Serial device '…' does not
exist." / permission hint) instead of a raw pyserial traceback.
- a console whose open failed is safely "not open" (init _thd=None +
isOpened guards in readchar/read_nowait/close) — no more cascading
AttributeError: '_thd' on subsequent read steps.
- action handlers: one-liner for expected (ETUMRuntimeError) errors, full
traceback kept for unexpected ones. All console errors use testium
exceptions (ETUMRuntimeError).
Flatpak: grant --device=all so serial adapters (/dev/ttyUSB*, /dev/ttyACM*)
are visible in the sandbox.
Validation: new read_until list/regex (match + no-match) cases in
items/console/test.tum.
Version: 0.3.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Run a user pytest file as a testium item, surfacing each collected test
as a child with its own PASS/FAIL/SKIP, duration and failure message.
Mirrors the unittest item but runs pytest in a subprocess on the host
interpreter (bins.python_bin(), like py_func/lua_func) so it works across
every packaging channel. A stdlib-only pytest plugin streams collected
node-ids and per-test results over stdout via sentinels; the parent parses
them live. Params: test_file, test_method. stop_on_failure maps to -x;
disabled children are reported NORUN without running.
Wiring: TYPE_PYTEST / TYPE_PYTEST_STEP constants, test_init registration,
self-loading branch in test_set, GUI tree icon. Schema/LSP pick it up
automatically from the declarative PARAMS.
Validation: test/validation/items/pytest/ (validation venv now installs
pytest).
WIP: paused mid-feature (DESIGN.md documented; manual section pending).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- PyInstaller exe built windowed (console=False) with package/testium.ico
as the embedded icon (BMP entries for shell compatibility).
- Suppress stray subprocess console windows in the frozen Windows build via
paths.no_window_kwargs() (CREATE_NO_WINDOW); wheel/source unchanged.
Applied to py_process, lua_process, bins probes, sys_app_path_win.
- New per-user Inno Setup installer (package/innosetup/): no admin,
version-scoped AppId/dir so versions install side-by-side, one Start
Menu entry per version, .ico shipped for shortcut/uninstall icons.
- DESIGN.md + release_note.txt updated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
bins.ensure() now stores the resolved interpreter path under
python_bin / lua_bin when the key is unset, so test scripts can use
$(python_bin) / $(lua_bin) in GUI mode (no -d override). Restores the
behaviour lost when bins.py centralised resolution. A user-provided
value is left untouched.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The subprocess now binds port 0, prints the bound port on stdout after
listen(), and the parent connects only once it reads that port. Removes
the reserve/close/rebind race and SO_REUSEADDR, and the connect-before-
ready timing guess that failed intermittently on Windows. wait_ready()
no longer hangs when a connection attempt fails.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Make the suite run cleanly on Windows.
Product code:
- __init__.py: force UTF-8 on stdout/stderr. The Windows console code
page (cp1252) cannot encode the box-drawing/accented characters the
runner prints, which crashed the parent capture_stdout thread. Only
the stream encoders are reconfigured; the locale default used to read
cp1252 config files is left untouched.
- report_export_junit/html: open the report file with encoding="utf-8"
(XML/HTML are UTF-8) instead of the platform default, matching the
txt/json exporters.
Validation:
- run.bat: source mode now sets up its own venv and runs testium from
src\ directly instead of delegating to the project run.bat (which
launches the GUI and drops its arguments). Installs the fake_exporter
entry-point plugin (report_plugin) and the [lsp] extra, and runs the
same lsp_check.py pre-flight as run.sh.
- jsonrpc/test.tum: launch the echo server via "$(python_bin)" instead
of "python3" (the Microsoft Store stub on Windows).
- post_execution.py: write the JUnit XML with encoding="utf-8".
- restore items/run/sub_pass.tum and sub_fail.tum, deleted by mistake in
d97d00c "removed test logs".
- term console via flatpak-spawn --host so host venvs resolve (bins.host_console_command)
- QSettings sync() before subprocess kill in choices/tested-refs dialogs
- console regression test: fails on the in-sandbox 0.2.1 console
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Manual (modes.rst) and README: install the extension from Open VSX in
VSCodium/Cursor/etc., and as a .vsix by hand in Microsoft VSCode; note
that testium must be on PATH or set via testium.serverPath.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Set the app id via setDesktopFileName so the window stops inheriting the
launcher's class ("python3" under the AppImage), which is what GNOME was
keying the wrong icon off. On native Wayland the task-bar icon comes from
an installed desktop file matched to the app id (setWindowIcon is
ignored there), so on Linux drop an idempotent desktop entry + 256px icon
under ~/.local/share. Flatpak keeps its own id/desktop; Windows / macOS
use the window icon. No-op off Linux.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
load_test_recursively expanded nested lists and included 'sequence'
entries by splicing each into the step list and rebuilding the whole
list every time (O(n^2)). The list branch also rebuilt after an in-place
splice, duplicating entries when a nested list held more than one item.
Replace both with a single linear _flatten_actions pass. Build phase
~12% faster at 6k items; the real fix is the duplication (a nested
2-element list now yields a,b,c,d not a,b,c,c,d). Validation suite
identical (post-exec SUCCESS, same verdicts/tracebacks).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Base the TUM loaders (and the param-file load) on yaml.CLoader when
PyYAML is built with libyaml, falling back to the pure-Python Loader
otherwise. Same ParserError/ScannerError, same custom !include
constructors. YAML parse time ~8x lower; validation suite identical
(same verdicts, same 8 expected-fail tracebacks, post-exec SUCCESS).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Shared jinja Environment + compiled-template cache keyed on (path, mtime,
size), and render to an in-memory StringIO instead of a temp file.
Behaviour unchanged (validation suite passes); template time -10..40x,
total load -20..30% on template-heavy trees.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Generator + in-process harness timing the real loader's three stages and
template/YAML call counts, across tunable profiles. cases/ git-ignored;
see test/benchmark/README.md.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The wheel channel was installed without [lsp], so 'testium lsp' reported pygls
missing. Install testium[lsp] in the wheel venv — validating the wheel's
language-server-capable form, like the AppImage installs ...whl[lsp].
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
modes.rst: new 'Language server (editor support)' section — testium lsp /
testium schema, the testium_assist client, the [lsp] extra. tum_syntax.rst:
a note on parameter validation (unknown-key warning, missing-required error).
Builds clean (sphinx html, no RST errors).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The 'main' merge (bd1cd03) brought a more detailed AppImage entry (with the
libfuse2 requirement); my earlier branch had added a simpler one, leaving two.
Keep main's, drop the duplicate, retain the language-server note.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
README: add the AppImage release, an 'Editor support' section (testium lsp /
schema, [lsp] extra, the testium_assist client), note the LSP ships in every
channel. DESIGN.md: document build_all parallelism, --ram tmpfs mode and the
Ctrl+C job-tree kill. release_note.txt: 0.2 entries for the language server and
the build_all parallel/--ram work.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Previously the reaping loop waited on jobs in array order, so a finished build's
OK/FAILED line was delayed until the loop reached its PID (e.g. appimage done
but unreported while flatpak still ran). Use 'wait -n -p' to print each result
as soon as that build finishes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
flatpak-builder mounts its state dir via rofiles-fuse; FUSE mounts fail on
/dev/shm ('fusermount: Permission denied'). So --ram no longer redirects the
flatpak dirs — it builds on disk as before. PyInstaller workpath, AppImage
AppDir and TMPDIR/PIP_CACHE_DIR still go to tmpfs.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
flatpak-builder hardlinks between its state dir and the build dir, so they must
share a filesystem. With only the build dir on tmpfs it errored ('state dir not
on the same filesystem as the target dir'). Move .flatpak-builder to tmpfs as
well via FLATPAK_STATEDIR; its download cache no longer persists across --ram
runs, which is the accepted trade for the tmpfs speedup.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Trap INT/TERM around the parallel wait recursively kills each job's process
tree (subshell + grandchildren: podman container, flatpak-builder, pyinstaller),
then exits 130 — the EXIT trap frees the tmpfs scratch. Verified: SIGINT leaves
no orphan processes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Redirect the per-channel build scratch to /dev/shm and skip UPX, a big win when
building from a USB stick / SD card (I/O-bound on flash):
- TMPDIR + PIP_CACHE_DIR -> tmpfs
- PyInstaller: --workpath -> tmpfs (PYI_WORKPATH); UPX off via TESTIUM_NO_UPX
- Flatpak: build dir + ostree repo -> tmpfs (FLATPAK_BUILDDIR/REPODIR); the
.flatpak-builder download cache stays on disk
- AppImage: bind-mount a tmpfs dir at the in-container AppDir path
(APPIMAGE_APPDIR_TMPFS)
Scratch is freed on exit. Each build.sh honors the env vars with on-disk
defaults, so behavior is unchanged without --ram. With --ram, prefer --serial
on RAM-limited machines (flatpak+appimage are ~1 GB each).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Serial prep (venv tool installs + flatpak runtimes + wheel, which the AppImage
depends on), then the four heavy builds run concurrently. The shared venv is
only written during prep, so the parallel builds (read-only on the venv) don't
race on pip. Per-step logs under dist/.build-logs/; failing logs are printed.
--serial falls back to one-at-a-time.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
