Find bar over the test tree: highlight matches and navigate them
(Enter / ◂ ▸), with Name/Type/Doc checkboxes to choose the searched
fields. Ctrl+F toggles the bar (clearing the highlight); Esc / ✕ close.
- QTestTreeItem: matches_search(needle, fields) + a search highlight that
shares one _refresh_highlight() with the green run highlight, recomputed
from state flags (run > search > default) so the two layers never leave a
stale/permanent colour. Amber bg + forced black text → readable in any
theme.
- QTestTree.search()/clear_search(): single signal-blocked pass (setBackground
fires itemChanged → on_testChecked, a controller storm otherwise); expands
ancestors of matches; returns matches in visual order.
- MainWindow: the find bar widget + Ctrl+F shortcut + navigation; search is
reset when a new test file is loaded.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
"pytest step" children now show the pytest logo (pytest.png), matching the
parent "pytest" item, instead of the generic document icon. Icon already
embedded in the compiled resources — no rcc regen needed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The plugin was delivered by writing it to a temp dir, putting that dir on
PYTHONPATH and loading it with `python -m pytest -p _testium_pytest_plugin`.
That import-by-name failed in the AppImage runtime (ModuleNotFoundError:
_testium_pytest_plugin) so collection returned nothing and the item FAILed
— while wheel/pyinstaller/flatpak worked. Local sims forcing the AppImage
env path (apply_host_libs) passed, ruling out the env scrubbing.
Ship the plugin as a self-contained launcher run directly
(`python launcher.py ...`) that registers it as a plugin object via
pytest.main(plugins=[sys.modules[__name__]]): no PYTHONPATH, no `-p`, no
import-by-name. apply_host_libs is untouched. Verified on source, wheel,
pyinstaller, flatpak and AppImage.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
When collection finds nothing because pytest is missing on the host
interpreter, load() raises a dedicated message ("pytest is not installed
... pip install pytest") instead of the raw pytest output. The graceful
load mechanism surfaces it as a WARN at load + a clean FAIL at run, the
rest of the campaign keeps running.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A self-loading item that can't load its module/file (unittest test file
with a missing import, pytest not installed on the host, ...) no longer
aborts the whole test load. TestSet._load_item() wraps load(), warns at
load time and records item._load_error; @test_run turns it into a clean
run-time FAILURE. The rest of the campaign loads and runs.
Scoped to module-loading items (unittest; pytest once merged). Structural
action loading stays fail-fast.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Use the official pytest logo (devicon) for the pytest item instead of
reusing python.png — visually distinct from the py_func (Python logo)
item. Three 64x64 theme variants (color RGBA, black RGBA silhouette,
white LA), declared in the QRC and compiled into testium_core_win_rc.py;
_ITEM_CONFIG "pytest" now points to pytest.png.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
0.3 is reserved for the upcoming pytest item release. Regenerated the
manual PDF so its version stamp matches.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Variable substitution ($(...)) must use the runtime global dict, so it
must happen at run time (execute), never at load (__init__).
- console telnet_port: was never expanded — `telnet_port: $(port)` stayed
literal. Now expanded at run (processed=True in execute, like the other
host/port params).
- test_item base: stop_on_failure / execute_on_stop are now stored raw and
resolved at run time via properties (so a $(...) flag reflects the
runtime value, not the load-time one).
- cycle iterator and git repo: drop the redundant load-time expansion
(execute() already re-expands them).
- tested_references: fetch 'reference' raw, expand each value in execute().
Justified load-time exceptions kept: name, doc, skipped (static/GUI at
load) and unittest test_method (drives child loading at load).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
read_until:
- 'expected' now accepts a list of values (succeeds on any match).
- new 'regex: true' flag: each pattern is a Python regex (re.search over a
bounded tail, Console.REGEX_WINDOW). Reports which pattern matched.
Serial console robustness & clarity:
- failed open() raises a clear ETUMRuntimeError ("Serial device '…' does not
exist." / permission hint) instead of a raw pyserial traceback.
- a console whose open failed is safely "not open" (init _thd=None +
isOpened guards in readchar/read_nowait/close) — no more cascading
AttributeError: '_thd' on subsequent read steps.
- action handlers: one-liner for expected (ETUMRuntimeError) errors, full
traceback kept for unexpected ones. All console errors use testium
exceptions (ETUMRuntimeError).
Flatpak: grant --device=all so serial adapters (/dev/ttyUSB*, /dev/ttyACM*)
are visible in the sandbox.
Validation: new read_until list/regex (match + no-match) cases in
items/console/test.tum.
Version: 0.3.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Run a user pytest file as a testium item, surfacing each collected test
as a child with its own PASS/FAIL/SKIP, duration and failure message.
Mirrors the unittest item but runs pytest in a subprocess on the host
interpreter (bins.python_bin(), like py_func/lua_func) so it works across
every packaging channel. A stdlib-only pytest plugin streams collected
node-ids and per-test results over stdout via sentinels; the parent parses
them live. Params: test_file, test_method. stop_on_failure maps to -x;
disabled children are reported NORUN without running.
Wiring: TYPE_PYTEST / TYPE_PYTEST_STEP constants, test_init registration,
self-loading branch in test_set, GUI tree icon. Schema/LSP pick it up
automatically from the declarative PARAMS.
Validation: test/validation/items/pytest/ (validation venv now installs
pytest).
WIP: paused mid-feature (DESIGN.md documented; manual section pending).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- PyInstaller exe built windowed (console=False) with package/testium.ico
as the embedded icon (BMP entries for shell compatibility).
- Suppress stray subprocess console windows in the frozen Windows build via
paths.no_window_kwargs() (CREATE_NO_WINDOW); wheel/source unchanged.
Applied to py_process, lua_process, bins probes, sys_app_path_win.
- New per-user Inno Setup installer (package/innosetup/): no admin,
version-scoped AppId/dir so versions install side-by-side, one Start
Menu entry per version, .ico shipped for shortcut/uninstall icons.
- DESIGN.md + release_note.txt updated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
bins.ensure() now stores the resolved interpreter path under
python_bin / lua_bin when the key is unset, so test scripts can use
$(python_bin) / $(lua_bin) in GUI mode (no -d override). Restores the
behaviour lost when bins.py centralised resolution. A user-provided
value is left untouched.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The subprocess now binds port 0, prints the bound port on stdout after
listen(), and the parent connects only once it reads that port. Removes
the reserve/close/rebind race and SO_REUSEADDR, and the connect-before-
ready timing guess that failed intermittently on Windows. wait_ready()
no longer hangs when a connection attempt fails.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Make the suite run cleanly on Windows.
Product code:
- __init__.py: force UTF-8 on stdout/stderr. The Windows console code
page (cp1252) cannot encode the box-drawing/accented characters the
runner prints, which crashed the parent capture_stdout thread. Only
the stream encoders are reconfigured; the locale default used to read
cp1252 config files is left untouched.
- report_export_junit/html: open the report file with encoding="utf-8"
(XML/HTML are UTF-8) instead of the platform default, matching the
txt/json exporters.
Validation:
- run.bat: source mode now sets up its own venv and runs testium from
src\ directly instead of delegating to the project run.bat (which
launches the GUI and drops its arguments). Installs the fake_exporter
entry-point plugin (report_plugin) and the [lsp] extra, and runs the
same lsp_check.py pre-flight as run.sh.
- jsonrpc/test.tum: launch the echo server via "$(python_bin)" instead
of "python3" (the Microsoft Store stub on Windows).
- post_execution.py: write the JUnit XML with encoding="utf-8".
- restore items/run/sub_pass.tum and sub_fail.tum, deleted by mistake in
d97d00c "removed test logs".
- term console via flatpak-spawn --host so host venvs resolve (bins.host_console_command)
- QSettings sync() before subprocess kill in choices/tested-refs dialogs
- console regression test: fails on the in-sandbox 0.2.1 console
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Set the app id via setDesktopFileName so the window stops inheriting the
launcher's class ("python3" under the AppImage), which is what GNOME was
keying the wrong icon off. On native Wayland the task-bar icon comes from
an installed desktop file matched to the app id (setWindowIcon is
ignored there), so on Linux drop an idempotent desktop entry + 256px icon
under ~/.local/share. Flatpak keeps its own id/desktop; Windows / macOS
use the window icon. No-op off Linux.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
load_test_recursively expanded nested lists and included 'sequence'
entries by splicing each into the step list and rebuilding the whole
list every time (O(n^2)). The list branch also rebuilt after an in-place
splice, duplicating entries when a nested list held more than one item.
Replace both with a single linear _flatten_actions pass. Build phase
~12% faster at 6k items; the real fix is the duplication (a nested
2-element list now yields a,b,c,d not a,b,c,c,d). Validation suite
identical (post-exec SUCCESS, same verdicts/tracebacks).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Base the TUM loaders (and the param-file load) on yaml.CLoader when
PyYAML is built with libyaml, falling back to the pure-Python Loader
otherwise. Same ParserError/ScannerError, same custom !include
constructors. YAML parse time ~8x lower; validation suite identical
(same verdicts, same 8 expected-fail tracebacks, post-exec SUCCESS).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Shared jinja Environment + compiled-template cache keyed on (path, mtime,
size), and render to an in-memory StringIO instead of a temp file.
Behaviour unchanged (validation suite passes); template time -10..40x,
total load -20..30% on template-heavy trees.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Make `testium lsp` (and the testium_assist editor extension that spawns it)
work from every distribution channel: source, wheel, PyInstaller, Flatpak,
AppImage.
Two enablers:
1. Declarative ACTIONS registry. The TestItemActions parents (console, plot,
json_rpc) now declare their nested actions as a class attribute
`ACTIONS = {yaml_key: class}`, mirroring PARAMS. The base __init__ seeds
action_classes from type(self).ACTIONS; register_actions() is kept only as
an imperative escape hatch. lsp/schema.py reads ACTIONS directly, dropping
the inspect.getsource/AST walk that returned no actions in a frozen
PyInstaller build (no .py source on disk).
2. pygls bundled per channel. Kept as the pyproject [lsp] extra (lean
`pip install testium`), layered into each full-app channel:
- build_env.sh installs pygls into test/tmp/.venv (source run + PyInstaller
build env)
- AppImage installs the wheel as `…whl[lsp]`
- Flatpak adds a python3-lsp network-pip module (matches the manifest's
global --share=network)
- PyInstaller .spec collect_submodules(pygls/lsprotocol) + hiddenimports for
the lazily-imported lsp/lsp.server/lsp.schema
test/validation/lsp_smoke.py (run by run.sh before the suite) enforces both
per channel: `<channel> schema` must keep console/plot/json_rpc actions and
`<channel> lsp` must answer an initialize request without reporting pygls
missing. Verified for source mode; the other channels need a rebuild to verify.
DESIGN.md updated (declarative section + new "Language server across channels"
subsection + Recent fixes).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds two new LSP features that share the schema with completion:
- textDocument/hover — when the cursor is on the item-type word of a
step line (`- sleep:`), the server renders the same Markdown doc
used by the completion item, listing required/optional params.
Other words (string values, YAML keys other than item types) don't
trigger the popup.
- textDocument/documentSymbol — the outline view now contains one
entry per step, nested by leading-dash indentation so container
items (group, parallel, cycle, console, plot, json_rpc) display
their children as a subtree. Each symbol's `detail` shows the
YAML `name:` field if present nearby — found via a small forward
scan, no YAML parsing yet.
Action item types (console open/close/…, plot open/close/…, json_rpc
query/receive/…) are accepted by hover and outline too, so the
outline doesn't stop at the parent.
Markdown rendering is now shared by completion and hover via
`_render_item_markdown(cmd, entry)`; both surfaces show the same
description regardless of how the user reached it.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds two new testium CLI subcommands:
testium schema dump the JSON-shaped schema of every test item type
(PARAMS merged with the common ones, console / plot /
json_rpc actions nested under their parent's "actions"
block). Zero runtime dependencies — usable by any editor
that speaks the YAML JSON Schema extension to get static
completion straight away.
testium lsp start a pygls LSP server over stdio. First feature:
completion of test item type names when the user starts
a new step (`- |`). The completion item carries the
item's display name and a hover doc listing its required
and optional non-common parameters.
pygls is declared as an optional extra ([project.optional-dependencies]
lsp = ["pygls>=1.3"]) so the core install isn't enlarged for users who
don't need the server. The import compatibility shim picks
pygls.lsp.server.LanguageServer (pygls 2.x) first and falls back to
pygls.server.LanguageServer (pygls 1.x).
The subcommands intercept argv[1] before argparse runs so they don't
share the GUI/batch flag surface.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Migrates the remaining test items to the ParamSet/Param declaration
introduced in d0721af:
- dialogs: image, question, value, choices, tested_references
- actions: check, run, report
- console: parent + open/read_until actions
- py_func / lua_func
- containers: group, parallel + parallel_branch, unittest
- complex: cycle (sub-block exit_condition documented in
EXIT_CONDITION_PARAMS), git
- runtime_plot: parent + open/close/periodic/last_value actions
- json_rpc: parent + query/receive actions
Items intentionally without PARAMS (and therefore not validated) are
those whose body is the unstructured user value: console write/writeln,
plot add/export, and the json_rpc/console open & close actions. Same
for the internally-instantiated TestItemUnittestElement which passes
dict_item=None.
Behavior on valid .tum files is unchanged (validation suite source
mode: SUCCESS). Typos on declared params now surface as warnings
listing the accepted names; missing required params surface as load-
time errors with file context.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds utils/param_decl.py with Param/ParamSet/kind descriptors. TestItem
declares COMMON_PARAMS (name, doc, condition, key, skipped, …) and a
new _validate_declared_params() method that warns on unknown keys and
errors on missing required ones — opt-in per subclass (skipped while
PARAMS is None to keep the migration incremental).
Migrates sleep, let, msg_dialog, note_dialog as pilots. Behavior is
unchanged for any well-formed .tum; typos like 'timeoot' on a sleep
item now produce a clear WARN listing the accepted parameters.
The descriptor intentionally carries no Python type information —
parameter values that are $(…) / <|…|> expressions only acquire their
effective type after expansion, so a static type would be misleading.
Per-param post-expansion validators stay opt-in via validate=lambda.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Inside a Flatpak the host /usr/bin/git is reachable at
/run/host/usr/bin/git but linked against host glibc/zlib, which the
sandbox cannot load (libz-ng.so.2 missing). gitpython resolves git
eagerly on import and crashed the whole validation run.
Install a tiny shell wrapper under /tmp at module load
(``exec flatpak-spawn --host git "$@"``) and point gitpython at it via
GIT_PYTHON_GIT_EXECUTABLE so test_version / test_modifs work in
flatpak mode. No-op outside Flatpak.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
py_func, lua_func and the run item now reach host binaries through
`flatpak-spawn --host` instead of trying to load them under the
sandbox runtime (which fails with a glibc ABI mismatch). Adds
`--talk-name=org.freedesktop.Flatpak` to the manifest, stages the
/app/lib/testium tree under /tmp so the host can read it, and drops
the dead `_FLATPAK_HOST_DIRS` / lib-injection code paths that the
new approach makes obsolete.
Validation suite gains a `--mode source|wheel|pyinstaller|flatpak|
appimage` flag so the same item set can run against every packaging
channel; per-mode report file names avoid clobbering.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Skip steps whose dist/ artifact already exists; add --clean/-c to
force a full rebuild. Install sphinx/linuxdoc, build, pyinstaller
in their respective steps instead of upfront. Auto-add flathub
remote and install missing Flatpak SDK/runtime deps before step 4.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The v0.1.2 fix that forced Qt's non-native dialog for the "open test"
dialog only covered one call site. The same XDG-portal-vs-sibling-files
problem applies to every other QFileDialog in the GUI (save report,
log file path, default report/log dirs in preferences, python/lua
interpreter pickers).
Extracted a single ``file_dialog.options()`` helper in main_win/ and
threaded it through every getOpenFileName / getSaveFileName /
getExistingDirectory call in main_win/. Outside Flatpak the helper
returns an empty Options(), so the native dialog stays in use on
KDE / GNOME / Windows / macOS.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
eval_proc was started before -d/GUI defines reached gd, so
``-d python_bin=...`` and the GUI ``python_bin`` preference were
silently ignored by the very subprocess that runs ``<| ... |>`` evals
(and only took effect for later items once the discovery cache had
already been seeded with the system interpreter). apply_overrides() is
now applied before eval_process_init(), and bins._resolve()'s cache is
keyed by (name, override) so a later param.yaml change re-resolves on
the next lookup.
The validation suite now ships a wrapper (run.sh / run.bat) that
creates a dedicated venv in the system temp dir and pins it via
``-d python_bin=...``. A new ``venv`` item asserts the override took
effect for both eval_proc and py_func paths, with a
``sys.prefix != sys.base_prefix`` marker to catch the case where the
override happens to be a system interpreter (path-equality alone would
miss it, the venv's ``bin/python3`` being a symlink to the host).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
_handle_request was using the 1st pcall return as the error
discriminator, so any Lua function returning nothing (e.g. long_wait
in the example) was reported as failed. Discriminate on the 2nd
return (err) instead, and encode nil result as cjson.null so the
returned_value field stays present in the JSON-RPC response.
console.read_until polls a should_stop callback in 0.2s chunks across
all protocols. py_func/lua_func override stop() to tear down the worker
and wake the parent RPC wait. json_rpc adapters honor should_stop too.
Engaged leaf steps now report FAILURE on stop (sleep no-dialog was
silently SUCCESS).
Native file dialog routes through the XDG document portal, which exposes
only the selected file at /run/user/UID/doc/... — siblings (param.yaml,
.py) are unreachable. Force Qt's non-native dialog in Flatpak so it walks
the real filesystem via --filesystem=home and returns a usable path.
_testium_launch_cmd() returns the right entry point per mode (AppImage,
Flatpak, PyInstaller, source/wheel). Fixes PermissionError on read-only
__main__.py inside the AppImage squashfs mount.
_which() probes host dirs only in Flatpak (/run/host/usr/bin) and AppImage
(/usr/bin); apply_host_libs prepares env for host subprocesses (prepend host
libs in Flatpak, strip $APPDIR pollution + PYTHONHOME in AppImage); user
override resolved via _which() for bare names.
