Make `testium lsp` (and the testium_assist editor extension that spawns it)
work from every distribution channel: source, wheel, PyInstaller, Flatpak,
AppImage.
Two enablers:
1. Declarative ACTIONS registry. The TestItemActions parents (console, plot,
json_rpc) now declare their nested actions as a class attribute
`ACTIONS = {yaml_key: class}`, mirroring PARAMS. The base __init__ seeds
action_classes from type(self).ACTIONS; register_actions() is kept only as
an imperative escape hatch. lsp/schema.py reads ACTIONS directly, dropping
the inspect.getsource/AST walk that returned no actions in a frozen
PyInstaller build (no .py source on disk).
2. pygls bundled per channel. Kept as the pyproject [lsp] extra (lean
`pip install testium`), layered into each full-app channel:
- build_env.sh installs pygls into test/tmp/.venv (source run + PyInstaller
build env)
- AppImage installs the wheel as `…whl[lsp]`
- Flatpak adds a python3-lsp network-pip module (matches the manifest's
global --share=network)
- PyInstaller .spec collect_submodules(pygls/lsprotocol) + hiddenimports for
the lazily-imported lsp/lsp.server/lsp.schema
test/validation/lsp_smoke.py (run by run.sh before the suite) enforces both
per channel: `<channel> schema` must keep console/plot/json_rpc actions and
`<channel> lsp` must answer an initialize request without reporting pygls
missing. Verified for source mode; the other channels need a rebuild to verify.
DESIGN.md updated (declarative section + new "Language server across channels"
subsection + Recent fixes).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds two new LSP features that share the schema with completion:
- textDocument/hover — when the cursor is on the item-type word of a
step line (`- sleep:`), the server renders the same Markdown doc
used by the completion item, listing required/optional params.
Other words (string values, YAML keys other than item types) don't
trigger the popup.
- textDocument/documentSymbol — the outline view now contains one
entry per step, nested by leading-dash indentation so container
items (group, parallel, cycle, console, plot, json_rpc) display
their children as a subtree. Each symbol's `detail` shows the
YAML `name:` field if present nearby — found via a small forward
scan, no YAML parsing yet.
Action item types (console open/close/…, plot open/close/…, json_rpc
query/receive/…) are accepted by hover and outline too, so the
outline doesn't stop at the parent.
Markdown rendering is now shared by completion and hover via
`_render_item_markdown(cmd, entry)`; both surfaces show the same
description regardless of how the user reached it.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds two new testium CLI subcommands:
testium schema dump the JSON-shaped schema of every test item type
(PARAMS merged with the common ones, console / plot /
json_rpc actions nested under their parent's "actions"
block). Zero runtime dependencies — usable by any editor
that speaks the YAML JSON Schema extension to get static
completion straight away.
testium lsp start a pygls LSP server over stdio. First feature:
completion of test item type names when the user starts
a new step (`- |`). The completion item carries the
item's display name and a hover doc listing its required
and optional non-common parameters.
pygls is declared as an optional extra ([project.optional-dependencies]
lsp = ["pygls>=1.3"]) so the core install isn't enlarged for users who
don't need the server. The import compatibility shim picks
pygls.lsp.server.LanguageServer (pygls 2.x) first and falls back to
pygls.server.LanguageServer (pygls 1.x).
The subcommands intercept argv[1] before argparse runs so they don't
share the GUI/batch flag surface.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Migrates the remaining test items to the ParamSet/Param declaration
introduced in d0721af:
- dialogs: image, question, value, choices, tested_references
- actions: check, run, report
- console: parent + open/read_until actions
- py_func / lua_func
- containers: group, parallel + parallel_branch, unittest
- complex: cycle (sub-block exit_condition documented in
EXIT_CONDITION_PARAMS), git
- runtime_plot: parent + open/close/periodic/last_value actions
- json_rpc: parent + query/receive actions
Items intentionally without PARAMS (and therefore not validated) are
those whose body is the unstructured user value: console write/writeln,
plot add/export, and the json_rpc/console open & close actions. Same
for the internally-instantiated TestItemUnittestElement which passes
dict_item=None.
Behavior on valid .tum files is unchanged (validation suite source
mode: SUCCESS). Typos on declared params now surface as warnings
listing the accepted names; missing required params surface as load-
time errors with file context.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds utils/param_decl.py with Param/ParamSet/kind descriptors. TestItem
declares COMMON_PARAMS (name, doc, condition, key, skipped, …) and a
new _validate_declared_params() method that warns on unknown keys and
errors on missing required ones — opt-in per subclass (skipped while
PARAMS is None to keep the migration incremental).
Migrates sleep, let, msg_dialog, note_dialog as pilots. Behavior is
unchanged for any well-formed .tum; typos like 'timeoot' on a sleep
item now produce a clear WARN listing the accepted parameters.
The descriptor intentionally carries no Python type information —
parameter values that are $(…) / <|…|> expressions only acquire their
effective type after expansion, so a static type would be misleading.
Per-param post-expansion validators stay opt-in via validate=lambda.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
py_func, lua_func and the run item now reach host binaries through
`flatpak-spawn --host` instead of trying to load them under the
sandbox runtime (which fails with a glibc ABI mismatch). Adds
`--talk-name=org.freedesktop.Flatpak` to the manifest, stages the
/app/lib/testium tree under /tmp so the host can read it, and drops
the dead `_FLATPAK_HOST_DIRS` / lib-injection code paths that the
new approach makes obsolete.
Validation suite gains a `--mode source|wheel|pyinstaller|flatpak|
appimage` flag so the same item set can run against every packaging
channel; per-mode report file names avoid clobbering.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Skip steps whose dist/ artifact already exists; add --clean/-c to
force a full rebuild. Install sphinx/linuxdoc, build, pyinstaller
in their respective steps instead of upfront. Auto-add flathub
remote and install missing Flatpak SDK/runtime deps before step 4.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The v0.1.2 fix that forced Qt's non-native dialog for the "open test"
dialog only covered one call site. The same XDG-portal-vs-sibling-files
problem applies to every other QFileDialog in the GUI (save report,
log file path, default report/log dirs in preferences, python/lua
interpreter pickers).
Extracted a single ``file_dialog.options()`` helper in main_win/ and
threaded it through every getOpenFileName / getSaveFileName /
getExistingDirectory call in main_win/. Outside Flatpak the helper
returns an empty Options(), so the native dialog stays in use on
KDE / GNOME / Windows / macOS.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
eval_proc was started before -d/GUI defines reached gd, so
``-d python_bin=...`` and the GUI ``python_bin`` preference were
silently ignored by the very subprocess that runs ``<| ... |>`` evals
(and only took effect for later items once the discovery cache had
already been seeded with the system interpreter). apply_overrides() is
now applied before eval_process_init(), and bins._resolve()'s cache is
keyed by (name, override) so a later param.yaml change re-resolves on
the next lookup.
The validation suite now ships a wrapper (run.sh / run.bat) that
creates a dedicated venv in the system temp dir and pins it via
``-d python_bin=...``. A new ``venv`` item asserts the override took
effect for both eval_proc and py_func paths, with a
``sys.prefix != sys.base_prefix`` marker to catch the case where the
override happens to be a system interpreter (path-equality alone would
miss it, the venv's ``bin/python3`` being a symlink to the host).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
_handle_request was using the 1st pcall return as the error
discriminator, so any Lua function returning nothing (e.g. long_wait
in the example) was reported as failed. Discriminate on the 2nd
return (err) instead, and encode nil result as cjson.null so the
returned_value field stays present in the JSON-RPC response.
console.read_until polls a should_stop callback in 0.2s chunks across
all protocols. py_func/lua_func override stop() to tear down the worker
and wake the parent RPC wait. json_rpc adapters honor should_stop too.
Engaged leaf steps now report FAILURE on stop (sleep no-dialog was
silently SUCCESS).
Native file dialog routes through the XDG document portal, which exposes
only the selected file at /run/user/UID/doc/... — siblings (param.yaml,
.py) are unreachable. Force Qt's non-native dialog in Flatpak so it walks
the real filesystem via --filesystem=home and returns a usable path.
_testium_launch_cmd() returns the right entry point per mode (AppImage,
Flatpak, PyInstaller, source/wheel). Fixes PermissionError on read-only
__main__.py inside the AppImage squashfs mount.
_which() probes host dirs only in Flatpak (/run/host/usr/bin) and AppImage
(/usr/bin); apply_host_libs prepares env for host subprocesses (prepend host
libs in Flatpak, strip $APPDIR pollution + PYTHONHOME in AppImage); user
override resolved via _which() for bare names.
stdout/stderr of the subprocesses were going to DEVNULL — early-startup
errors (lua require failures, exceptions before stdio_redir kicks in)
were lost.
New helper proc_drain.drain_to_log spawns a daemon thread per pipe that
print()s each line through stdio_redir, so it reaches the log + live
output. Used by py_process and lua_process with [py_func]/[lua_func]
prefixes.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
get_testium_version() used pkg_resources (deprecated, slow to import)
and a narrow catch on git.InvalidGitRepositoryError; any other git
exception fell through to the outer except and returned "unknown".
- Use importlib.metadata.version("testium") to read the wheel
version that setuptools bakes from src/VERSION at build time. Works
out of any source checkout — pip-installed copies report
"<x.y> (wheel release)" instead of "unknown".
- Source-checkout path: tried first when prefs.git_supported, broadly
catches Exception so a missing repo / detached worktree / etc. no
longer hides the wheel-metadata fallback.
- PyInstaller path: graceful "unknown (binary release)" if the bundled
VERSION file is unreadable, instead of an unhandled exception.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Added some api accessible from python and lua sub_processes. Now the tests only access to py_func.tm instead of direct api.testium module access.
Corrected some f"xxx" to allow working with old python (bookworm).
Changed param.yaml of the test to allow lua to work in all situations.
Various other small fixes for frozen app, wheel.
Tested in all situations, and OK. Ready for tag !
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Move src/lib/ → src/testium/runtime/ (internal plumbing)
Move src/testium/libs/ → src/testium/api/ (public SDK for test scripts)
Move src/py_func/ → src/testium/py_func/ (Python subprocess)
Move src/lua_func/ → src/testium/lua_func/ (Lua subprocess data)
The package now ships as a single coherent unit instead of four sibling
top-level packages (testium, lib, py_func, lua_func) — pip install
gives a clean site-packages/testium/ with no namespace pollution; .lua
files travel with the wheel via package_data; the wheel installs
cleanly and `testium -b` runs end-to-end including py_func subprocesses
and entry-point exporter plugins.
Naming:
- runtime/ (internal, no API guarantees) clearer than lib/
- api/ (public SDK consumed as `import api.testium as tm`) clearer than libs/
Imports updated en masse: from lib. → from runtime. and from libs. →
from api., plus the importlib.import_module("libs.*") strings in
test_item_console.py and test_item_runtime_plot.py. Test/example
scripts (helper_lib.py, parallel.py, post_execution.py) and the
fake_exporter test suite migrated too.
paths.py: subproc_path() now returns testium_path() — both point at
the testium package directory since the subprocesses live inside.
pyproject.toml: removed exclude=["lua_func", "py_func"] (no longer
needed), added package-data for testium.lua_func/*.lua, removed the
license classifier (PEP 639 conflict with license expression).
Subprocess isolation contract: py_func/ and lua_func/ may only import
runtime/ and their own modules — never interpreter/, main_win/, api/,
or testium/. Enforced by test/validation/items/isolation/ which runs a
py_func that statically scans subprocess source files for forbidden
imports. The contract holds today; the test prevents future drift.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Replace the hardcoded if/elif in Export.exec() with a dict registry.
Built-in formats (text, json, junit, html) are registered as lazy
loaders; missing optional deps (junit_xml, lxml) print a clear message
with a pip install hint instead of raising. Entry-points
(group "testium.exporters") are discovered at import time — installed
plugins are auto-detected with no extra config.
An unknown or unavailable format prints an info line and skips the
export; the test run is not interrupted.
Validation:
- New testium-fake-exporter package under test/validation/fake_exporter/
installed automatically by scripts/build_env.sh on venv creation.
It registers fake_format via entry-points and exports the tests
table to CSV — a real, useful exporter that exercises the plugin
contract end-to-end (entry-point discovery, dispatch, SQLite query).
- New dedicated items/report_plugin/ test exercises both the
unknown-format skip path and the fake_format plugin path, with a
py_func check (file_check.py) on the produced CSV. Runs once per
validation suite.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add LICENSE (full EUPL-1.2 text + project copyright), CONTRIBUTING.md
with the inbound = outbound rule, and declare the licence in
pyproject.toml and README.md.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Reload (re-loading the same .tum file) was rebuilding the GUI tree
from scratch, resetting every checkbox and unfolding everything.
Snapshot the user's selection and fold state via the existing
getCheckList/restoreCheckList and getFoldList/restoreFoldList methods
(already used for session persistence through prefs), so a same-file
reload keeps both as well. A change in the total item count (file
edited between loads) skips the restore.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
A parallel branch now displays a single-arrow icon (parallel_branch.png)
distinct from the parallel container's three-arrow icon, making the
tree hierarchy easier to read at a glance.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The 'branches' list shown in the F1 panel previously contained each
branch's full 'steps', duplicating what is already visible in the test
tree. Strip 'steps' inside each branch dict while keeping 'branches'
itself so per-branch attributes (name, wait_for, condition, ...) stay
visible.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The parallel item runs branches concurrently with sync:all or sync:any
policy and optional per-branch wait_for synchronization. Each branch
runs in its own daemon thread and produces a clean per-item entry in
the SQLite report; the live output is prefixed [<branch_name>] so
concurrent branches stay readable.
Supporting changes:
- StdoutProxy (lib/stdout_redirect.py): thread-aware sys.stdout/stderr
with per-thread capture buffers and per-branch live-output prefix.
Adds writeln() for Python 3.14 unittest compatibility.
- TestItemContainer: shared base extracted from Group/Cycle for the
sequential children execution pattern.
- TestItemSleep: interruptible loop polling _is_stopped so sync:any
can cancel slow branches quickly.
- TestReport: thread-safe SQLite (check_same_thread=False + lock).
Also drops the unused -m/--terminal mode and its module.
Validation: 11 scenarios in test/validation/items/parallel covering
sync:all/any, no_fail, wait_for + timeout, conditions, multi-branch,
nested parallel, parallel inside loop, real branch failure.
Documentation: new parallel_test_item.rst added to the manual.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- run item: rename tum_fime→tum, remove stdout=PIPE (deadlock with
spawn), support batch mode (-b), SUCCESS on any completed subprocess
regardless of sub-test result
- batch.py: fix control("loaded") deadlock via daemon thread + Event +
is_alive() polling; fix premature finish on gd_update messages;
propagate success flag from finished message; guard control("close")
- process.py: include success flag in send_finished message
- py_process/lua_process: add stdout/stderr=DEVNULL to Popen
- test_run.py: fix finished detection ("id" in m and m["id"] is None)
- testium_win.py: track run_exit_code, SIGABRT handler, clean exit
- __init__.py: sys.exit with batch success flag
- Add run item validation tests and CLAUDE.md documentation
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Each dialog test item now accepts an optional auto_result parameter
(ok/cancel/yes/no) and auto_value for text dialogs. When set, the dialog
window opens, stays visible 2 seconds, then closes automatically with the
specified result — allowing the validation suite to run without manual
interaction.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Allows any test item to store its result (or PASS/FAIL status when result
is None) into a named global variable, available to subsequent items via
$(variable_name). store_result runs after expected_result but before
no_fail so the real outcome is always captured.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Dialog subprocesses were forked from TestProcess, inheriting its
multiprocessing Queue objects and their process-shared POSIX semaphores
(_wlock). If a fork happened while the feeder thread held _wlock, the
child exited without releasing it, permanently blocking the feeder
thread on the next wacquire() and stalling Python's atexit _finalize_join
— causing test_proc.join() (no timeout) to hang the app for ~15 seconds.
Fix: use multiprocessing.get_context('spawn') for dialog subprocesses so
they start with a clean interpreter and inherit no semaphores or Queue
state. Also add a terminate/kill fallback timeout to test_proc.join() as
a safety net, and fix the missing return in JsonRpcConnection.is_alive().
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add dialog_env.py service: forces QT_QPA_PLATFORM=xcb on Linux so Qt
doesn't crash under Wayland in spawned subprocesses
- Use QMessageBox instance (instead of static methods) for msg/question
dialogs so WindowStaysOnTopHint can be set, making them visible
- Add TestItemDialogBase with _run_dialog/_run_dialog_with_result/_cleanup_process,
removing duplicated subprocess launch/poll/terminate logic from all 7 dialog items
- Reduce terminate() join timeout from 2s to 0.2s across all dialog items
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add item_load_context() context manager to tum_except.py that enriches
ETUMSyntaxError with the item type, name, and parent path instead of
replacing the original message with a vague 'missing or wrong parameter'.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- process.py: stop context_id engines in the inner finally block, before
restore_gd() wipes _py_func_contexts/_lua_func_contexts from the global
dict — engines were previously orphaned after every test run
- py_func/tm.py: add user-facing docstrings to gd/setgd/delgd; remove
internal JSON-serialization details from the docs
- helper_lib.rst: auto-generate global variable helpers from py_func.tm
(the actual subprocess API) instead of globdict
- conf.py: add src/ to Sphinx sys.path so py_func.tm is importable
- py_func_test_item.rst: simplify context sharing section, remove
JSON-serializable/non-serializable distinction for end users
- Regenerated PDF manual
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- py_func and lua_func items accept a context_id parameter; items sharing
the same id reuse the same subprocess for the duration of the test run
- Subprocess-side tm.setgd/tm.gd use a local fallback dict for non-JSON-
serializable values (py_func only); serializable values reach the main
process global dict and are accessible from any test item or subprocess
- Shared subprocess engines are cleaned up in process.py finally block
- LuaProcessBase gains is_alive() (was missing, broke all lua_func items)
- Validation tests cover serializable sharing across different context ids,
non-serializable sharing within the same context_id, and cross-item access
- RST documentation updated for both py_func and lua_func items
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
