Make `testium lsp` (and the testium_assist editor extension that spawns it)
work from every distribution channel: source, wheel, PyInstaller, Flatpak,
AppImage.
Two enablers:
1. Declarative ACTIONS registry. The TestItemActions parents (console, plot,
json_rpc) now declare their nested actions as a class attribute
`ACTIONS = {yaml_key: class}`, mirroring PARAMS. The base __init__ seeds
action_classes from type(self).ACTIONS; register_actions() is kept only as
an imperative escape hatch. lsp/schema.py reads ACTIONS directly, dropping
the inspect.getsource/AST walk that returned no actions in a frozen
PyInstaller build (no .py source on disk).
2. pygls bundled per channel. Kept as the pyproject [lsp] extra (lean
`pip install testium`), layered into each full-app channel:
- build_env.sh installs pygls into test/tmp/.venv (source run + PyInstaller
build env)
- AppImage installs the wheel as `…whl[lsp]`
- Flatpak adds a python3-lsp network-pip module (matches the manifest's
global --share=network)
- PyInstaller .spec collect_submodules(pygls/lsprotocol) + hiddenimports for
the lazily-imported lsp/lsp.server/lsp.schema
test/validation/lsp_smoke.py (run by run.sh before the suite) enforces both
per channel: `<channel> schema` must keep console/plot/json_rpc actions and
`<channel> lsp` must answer an initialize request without reporting pygls
missing. Verified for source mode; the other channels need a rebuild to verify.
DESIGN.md updated (declarative section + new "Language server across channels"
subsection + Recent fixes).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds two new LSP features that share the schema with completion:
- textDocument/hover — when the cursor is on the item-type word of a
step line (`- sleep:`), the server renders the same Markdown doc
used by the completion item, listing required/optional params.
Other words (string values, YAML keys other than item types) don't
trigger the popup.
- textDocument/documentSymbol — the outline view now contains one
entry per step, nested by leading-dash indentation so container
items (group, parallel, cycle, console, plot, json_rpc) display
their children as a subtree. Each symbol's `detail` shows the
YAML `name:` field if present nearby — found via a small forward
scan, no YAML parsing yet.
Action item types (console open/close/…, plot open/close/…, json_rpc
query/receive/…) are accepted by hover and outline too, so the
outline doesn't stop at the parent.
Markdown rendering is now shared by completion and hover via
`_render_item_markdown(cmd, entry)`; both surfaces show the same
description regardless of how the user reached it.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds two new testium CLI subcommands:
testium schema dump the JSON-shaped schema of every test item type
(PARAMS merged with the common ones, console / plot /
json_rpc actions nested under their parent's "actions"
block). Zero runtime dependencies — usable by any editor
that speaks the YAML JSON Schema extension to get static
completion straight away.
testium lsp start a pygls LSP server over stdio. First feature:
completion of test item type names when the user starts
a new step (`- |`). The completion item carries the
item's display name and a hover doc listing its required
and optional non-common parameters.
pygls is declared as an optional extra ([project.optional-dependencies]
lsp = ["pygls>=1.3"]) so the core install isn't enlarged for users who
don't need the server. The import compatibility shim picks
pygls.lsp.server.LanguageServer (pygls 2.x) first and falls back to
pygls.server.LanguageServer (pygls 1.x).
The subcommands intercept argv[1] before argparse runs so they don't
share the GUI/batch flag surface.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Brings DESIGN.md in sync with the v0.2 changes:
- new section describing the PARAMS/ParamSet/Param descriptor on every
TestItem subclass and the unknown/missing-param diagnostics;
- rewrites the Flatpak section so it matches the flatpak-spawn --host
pipeline instead of the obsolete LD_LIBRARY_PATH/apply_host_lua_paths
injection;
- documents the --mode flag in the validation suite section.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Migrates the remaining test items to the ParamSet/Param declaration
introduced in d0721af:
- dialogs: image, question, value, choices, tested_references
- actions: check, run, report
- console: parent + open/read_until actions
- py_func / lua_func
- containers: group, parallel + parallel_branch, unittest
- complex: cycle (sub-block exit_condition documented in
EXIT_CONDITION_PARAMS), git
- runtime_plot: parent + open/close/periodic/last_value actions
- json_rpc: parent + query/receive actions
Items intentionally without PARAMS (and therefore not validated) are
those whose body is the unstructured user value: console write/writeln,
plot add/export, and the json_rpc/console open & close actions. Same
for the internally-instantiated TestItemUnittestElement which passes
dict_item=None.
Behavior on valid .tum files is unchanged (validation suite source
mode: SUCCESS). Typos on declared params now surface as warnings
listing the accepted names; missing required params surface as load-
time errors with file context.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds utils/param_decl.py with Param/ParamSet/kind descriptors. TestItem
declares COMMON_PARAMS (name, doc, condition, key, skipped, …) and a
new _validate_declared_params() method that warns on unknown keys and
errors on missing required ones — opt-in per subclass (skipped while
PARAMS is None to keep the migration incremental).
Migrates sleep, let, msg_dialog, note_dialog as pilots. Behavior is
unchanged for any well-formed .tum; typos like 'timeoot' on a sleep
item now produce a clear WARN listing the accepted parameters.
The descriptor intentionally carries no Python type information —
parameter values that are $(…) / <|…|> expressions only acquire their
effective type after expansion, so a static type would be misleading.
Per-param post-expansion validators stay opt-in via validate=lambda.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
py_func, lua_func and the run item now reach host binaries through
`flatpak-spawn --host` instead of trying to load them under the
sandbox runtime (which fails with a glibc ABI mismatch). Adds
`--talk-name=org.freedesktop.Flatpak` to the manifest, stages the
/app/lib/testium tree under /tmp so the host can read it, and drops
the dead `_FLATPAK_HOST_DIRS` / lib-injection code paths that the
new approach makes obsolete.
Validation suite gains a `--mode source|wheel|pyinstaller|flatpak|
appimage` flag so the same item set can run against every packaging
channel; per-mode report file names avoid clobbering.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Skip steps whose dist/ artifact already exists; add --clean/-c to
force a full rebuild. Install sphinx/linuxdoc, build, pyinstaller
in their respective steps instead of upfront. Auto-add flathub
remote and install missing Flatpak SDK/runtime deps before step 4.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The v0.1.2 fix that forced Qt's non-native dialog for the "open test"
dialog only covered one call site. The same XDG-portal-vs-sibling-files
problem applies to every other QFileDialog in the GUI (save report,
log file path, default report/log dirs in preferences, python/lua
interpreter pickers).
Extracted a single ``file_dialog.options()`` helper in main_win/ and
threaded it through every getOpenFileName / getSaveFileName /
getExistingDirectory call in main_win/. Outside Flatpak the helper
returns an empty Options(), so the native dialog stays in use on
KDE / GNOME / Windows / macOS.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
eval_proc was started before -d/GUI defines reached gd, so
``-d python_bin=...`` and the GUI ``python_bin`` preference were
silently ignored by the very subprocess that runs ``<| ... |>`` evals
(and only took effect for later items once the discovery cache had
already been seeded with the system interpreter). apply_overrides() is
now applied before eval_process_init(), and bins._resolve()'s cache is
keyed by (name, override) so a later param.yaml change re-resolves on
the next lookup.
The validation suite now ships a wrapper (run.sh / run.bat) that
creates a dedicated venv in the system temp dir and pins it via
``-d python_bin=...``. A new ``venv`` item asserts the override took
effect for both eval_proc and py_func paths, with a
``sys.prefix != sys.base_prefix`` marker to catch the case where the
override happens to be a system interpreter (path-equality alone would
miss it, the venv's ``bin/python3`` being a symlink to the host).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Two new steps per language: function returning nothing and function
returning explicit nil/None. Both tagged $(test)_PASS — they would
have failed before the lua nil fix (Lua side reported nil result as
error). Python side already worked but is covered for parity.
_handle_request was using the 1st pcall return as the error
discriminator, so any Lua function returning nothing (e.g. long_wait
in the example) was reported as failed. Discriminate on the 2nd
return (err) instead, and encode nil result as cjson.null so the
returned_value field stays present in the JSON-RPC response.
console.read_until polls a should_stop callback in 0.2s chunks across
all protocols. py_func/lua_func override stop() to tear down the worker
and wake the parent RPC wait. json_rpc adapters honor should_stop too.
Engaged leaf steps now report FAILURE on stop (sleep no-dialog was
silently SUCCESS).
Collects all four artifacts under <repo>/dist/ (PyInstaller and Flatpak
renamed to testium-<version>(.suff); wheel and AppImage keep PEP 427 /
appimage-builder original names). Re-uses scripts/build_env.sh and
set_env.sh, same venv as run.sh. AppImage build.sh now picks the actual
output file dynamically instead of a hardcoded lowercase name.
Native file dialog routes through the XDG document portal, which exposes
only the selected file at /run/user/UID/doc/... — siblings (param.yaml,
.py) are unreachable. Force Qt's non-native dialog in Flatpak so it walks
the real filesystem via --filesystem=home and returns a usable path.
build.sh runs appimage-builder in a Debian Bookworm container (Podman or
Docker) so it works on Arch / non-Debian hosts. Uses single src/requirements.txt;
TESTIUM_VERSION exported in runtime.env.
org.testium.Testium.yaml uses host Python/Lua only (no bundled interpreter).
build.sh exports a .flatpak bundle. README documents the install procedure.
_testium_launch_cmd() returns the right entry point per mode (AppImage,
Flatpak, PyInstaller, source/wheel). Fixes PermissionError on read-only
__main__.py inside the AppImage squashfs mount.
_which() probes host dirs only in Flatpak (/run/host/usr/bin) and AppImage
(/usr/bin); apply_host_libs prepares env for host subprocesses (prepend host
libs in Flatpak, strip $APPDIR pollution + PYTHONHOME in AppImage); user
override resolved via _which() for bare names.
stdout/stderr of the subprocesses were going to DEVNULL — early-startup
errors (lua require failures, exceptions before stdio_redir kicks in)
were lost.
New helper proc_drain.drain_to_log spawns a daemon thread per pipe that
print()s each line through stdio_redir, so it reaches the log + live
output. Used by py_process and lua_process with [py_func]/[lua_func]
prefixes.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
get_testium_version() used pkg_resources (deprecated, slow to import)
and a narrow catch on git.InvalidGitRepositoryError; any other git
exception fell through to the outer except and returned "unknown".
- Use importlib.metadata.version("testium") to read the wheel
version that setuptools bakes from src/VERSION at build time. Works
out of any source checkout — pip-installed copies report
"<x.y> (wheel release)" instead of "unknown".
- Source-checkout path: tried first when prefs.git_supported, broadly
catches Exception so a missing repo / detached worktree / etc. no
longer hides the wheel-metadata fallback.
- PyInstaller path: graceful "unknown (binary release)" if the bundled
VERSION file is unreadable, instead of an unhandled exception.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- README.md: pruned developer-oriented sections (Sphinx setup, Qt
Creator workflow, VSCode debugging, release procedure, AppImage
Wayland note) and replaced them with a user-facing layout: pre-built
releases pointer, quick start, manual install, troubleshooting,
licence.
- CONTRIBUTING.md: absorbed the developer content (debugging in VSCode,
Qt GUI regen, Sphinx build, validation suite — batch + GUI variants,
cross-distrib check, release procedure).
- doc/quick_start.md: 5-minute path from install to a passing test,
in batch mode and in the GUI.
- doc/tutorial.md: guided walk-through against a small calc.py
module — check, py_func, expected_result, $(...) expansion, group,
let, condition, report (with the mkdir reminder), context_id.
- CLAUDE.md: subprocess API contract, bins.py, report-exporter
plugin section, packaging matrix (wheel / PyInstaller / Flatpak /
.deb work-in-progress), refreshed recent-fixes list. README/CLAUDE
validation command no longer carries the spurious "-l" flag (which
is GUI-only and a no-op in batch).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Added some api accessible from python and lua sub_processes. Now the tests only access to py_func.tm instead of direct api.testium module access.
Corrected some f"xxx" to allow working with old python (bookworm).
Changed param.yaml of the test to allow lua to work in all situations.
Various other small fixes for frozen app, wheel.
Tested in all situations, and OK. Ready for tag !
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Drop the now-obsolete src/lib and src/py_func data entries (those
paths no longer exist)
- Add src/testium/py_func and src/testium/runtime as bundle-root data
dirs: the py_func subprocess is launched with the *host* Python
(not the frozen interpreter), so it needs the source files on disk
at cwd=subproc_path() to find py_func/__main__.py and import from
runtime.*
- Hidden imports updated: libs.* → api.*, plus py_func.* explicitly
declared so PyInstaller pulls them into the bundle even though
they are loaded as data
Smoke-tested: built binary runs `testium -b`, py_func subprocess works.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Move src/lib/ → src/testium/runtime/ (internal plumbing)
Move src/testium/libs/ → src/testium/api/ (public SDK for test scripts)
Move src/py_func/ → src/testium/py_func/ (Python subprocess)
Move src/lua_func/ → src/testium/lua_func/ (Lua subprocess data)
The package now ships as a single coherent unit instead of four sibling
top-level packages (testium, lib, py_func, lua_func) — pip install
gives a clean site-packages/testium/ with no namespace pollution; .lua
files travel with the wheel via package_data; the wheel installs
cleanly and `testium -b` runs end-to-end including py_func subprocesses
and entry-point exporter plugins.
Naming:
- runtime/ (internal, no API guarantees) clearer than lib/
- api/ (public SDK consumed as `import api.testium as tm`) clearer than libs/
Imports updated en masse: from lib. → from runtime. and from libs. →
from api., plus the importlib.import_module("libs.*") strings in
test_item_console.py and test_item_runtime_plot.py. Test/example
scripts (helper_lib.py, parallel.py, post_execution.py) and the
fake_exporter test suite migrated too.
paths.py: subproc_path() now returns testium_path() — both point at
the testium package directory since the subprocesses live inside.
pyproject.toml: removed exclude=["lua_func", "py_func"] (no longer
needed), added package-data for testium.lua_func/*.lua, removed the
license classifier (PEP 639 conflict with license expression).
Subprocess isolation contract: py_func/ and lua_func/ may only import
runtime/ and their own modules — never interpreter/, main_win/, api/,
or testium/. Enforced by test/validation/items/isolation/ which runs a
py_func that statically scans subprocess source files for forbidden
imports. The contract holds today; the test prevents future drift.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Replace the hardcoded if/elif in Export.exec() with a dict registry.
Built-in formats (text, json, junit, html) are registered as lazy
loaders; missing optional deps (junit_xml, lxml) print a clear message
with a pip install hint instead of raising. Entry-points
(group "testium.exporters") are discovered at import time — installed
plugins are auto-detected with no extra config.
An unknown or unavailable format prints an info line and skips the
export; the test run is not interrupted.
Validation:
- New testium-fake-exporter package under test/validation/fake_exporter/
installed automatically by scripts/build_env.sh on venv creation.
It registers fake_format via entry-points and exports the tests
table to CSV — a real, useful exporter that exercises the plugin
contract end-to-end (entry-point discovery, dispatch, SQLite query).
- New dedicated items/report_plugin/ test exercises both the
unknown-format skip path and the fake_format plugin path, with a
py_func check (file_check.py) on the produced CSV. Runs once per
validation suite.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Merge the two "Recent fixes" sections into one (branches are gone),
add parallel_branch icon, F1 panel, test-tree state, unittest rename,
run item rename, licence. Fix SUCCESS/FAILURE → PASS/FAIL in run item.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
