10 Commits

Author SHA1 Message Date
097b17124b docs: README editor-support + AppImage, DESIGN build_all, release note 0.2
README: add the AppImage release, an 'Editor support' section (testium lsp /
schema, [lsp] extra, the testium_assist client), note the LSP ships in every
channel. DESIGN.md: document build_all parallelism, --ram tmpfs mode and the
Ctrl+C job-tree kill. release_note.txt: 0.2 entries for the language server and
the build_all parallel/--ram work.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-05-30 13:18:32 +02:00
c950b8f3ca build_all: report parallel results in completion order (wait -n -p)
Previously the reaping loop waited on jobs in array order, so a finished build's
OK/FAILED line was delayed until the loop reached its PID (e.g. appimage done
but unreported while flatpak still ran). Use 'wait -n -p' to print each result
as soon as that build finishes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-05-30 11:12:37 +02:00
523a69698b build_all --ram: exclude flatpak from tmpfs (rofiles-fuse can't mount /dev/shm)
flatpak-builder mounts its state dir via rofiles-fuse; FUSE mounts fail on
/dev/shm ('fusermount: Permission denied'). So --ram no longer redirects the
flatpak dirs — it builds on disk as before. PyInstaller workpath, AppImage
AppDir and TMPDIR/PIP_CACHE_DIR still go to tmpfs.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-05-30 10:56:16 +02:00
ab3058d789 build_all --ram: move flatpak state dir to tmpfs too (same-fs requirement)
flatpak-builder hardlinks between its state dir and the build dir, so they must
share a filesystem. With only the build dir on tmpfs it errored ('state dir not
on the same filesystem as the target dir'). Move .flatpak-builder to tmpfs as
well via FLATPAK_STATEDIR; its download cache no longer persists across --ram
runs, which is the accepted trade for the tmpfs speedup.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-05-30 10:50:57 +02:00
f748dae369 build_all: clean Ctrl+C in parallel mode (kill job trees on INT/TERM)
Trap INT/TERM around the parallel wait recursively kills each job's process
tree (subshell + grandchildren: podman container, flatpak-builder, pyinstaller),
then exits 130 — the EXIT trap frees the tmpfs scratch. Verified: SIGINT leaves
no orphan processes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-05-30 10:46:11 +02:00
46583f5622 build_all: --ram mode (build scratch on tmpfs) for slow storage
Redirect the per-channel build scratch to /dev/shm and skip UPX, a big win when
building from a USB stick / SD card (I/O-bound on flash):
- TMPDIR + PIP_CACHE_DIR -> tmpfs
- PyInstaller: --workpath -> tmpfs (PYI_WORKPATH); UPX off via TESTIUM_NO_UPX
- Flatpak: build dir + ostree repo -> tmpfs (FLATPAK_BUILDDIR/REPODIR); the
  .flatpak-builder download cache stays on disk
- AppImage: bind-mount a tmpfs dir at the in-container AppDir path
  (APPIMAGE_APPDIR_TMPFS)
Scratch is freed on exit. Each build.sh honors the env vars with on-disk
defaults, so behavior is unchanged without --ram. With --ram, prefer --serial
on RAM-limited machines (flatpak+appimage are ~1 GB each).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-05-30 10:42:46 +02:00
262dfd0240 build_all: parallelize manual/pyinstaller/flatpak/appimage
Serial prep (venv tool installs + flatpak runtimes + wheel, which the AppImage
depends on), then the four heavy builds run concurrently. The shared venv is
only written during prep, so the parallel builds (read-only on the venv) don't
race on pip. Per-step logs under dist/.build-logs/; failing logs are printed.
--serial falls back to one-at-a-time.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-05-30 10:30:38 +02:00
06cfaf33b7 flatpak: quote python3-lsp pip command (YAML parsed ':all: ' as a mapping)
The unquoted build-command was parsed by YAML as a dict because of the
':all: ' colon-space, so flatpak-builder ran an empty module and pygls was
never installed into the bundle.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-05-30 10:24:47 +02:00
c14a671b45 lsp: rename validation lsp_smoke -> lsp_check (clearer name)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-05-30 10:13:20 +02:00
8ab53f470d lsp: declarative action registry + cross-channel language server
Make `testium lsp` (and the testium_assist editor extension that spawns it)
work from every distribution channel: source, wheel, PyInstaller, Flatpak,
AppImage.

Two enablers:

1. Declarative ACTIONS registry. The TestItemActions parents (console, plot,
   json_rpc) now declare their nested actions as a class attribute
   `ACTIONS = {yaml_key: class}`, mirroring PARAMS. The base __init__ seeds
   action_classes from type(self).ACTIONS; register_actions() is kept only as
   an imperative escape hatch. lsp/schema.py reads ACTIONS directly, dropping
   the inspect.getsource/AST walk that returned no actions in a frozen
   PyInstaller build (no .py source on disk).

2. pygls bundled per channel. Kept as the pyproject [lsp] extra (lean
   `pip install testium`), layered into each full-app channel:
   - build_env.sh installs pygls into test/tmp/.venv (source run + PyInstaller
     build env)
   - AppImage installs the wheel as `…whl[lsp]`
   - Flatpak adds a python3-lsp network-pip module (matches the manifest's
     global --share=network)
   - PyInstaller .spec collect_submodules(pygls/lsprotocol) + hiddenimports for
     the lazily-imported lsp/lsp.server/lsp.schema

test/validation/lsp_smoke.py (run by run.sh before the suite) enforces both
per channel: `<channel> schema` must keep console/plot/json_rpc actions and
`<channel> lsp` must answer an initialize request without reporting pygls
missing. Verified for source mode; the other channels need a rebuild to verify.

DESIGN.md updated (declarative section + new "Language server across channels"
subsection + Recent fixes).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-05-29 23:17:59 +02:00
18 changed files with 482 additions and 119 deletions

View File

@@ -224,6 +224,13 @@ Four distribution channels coexist, all sharing the single `src/testium/` packag
The `.deb` work-in-progress lives in `package/deb/`:
- `test_distro.sh debian:bookworm | debian:trixie | ubuntu:24.04` spins up a Docker/Podman container, reports system package availability, falls back to pip for what's missing (`pyside6` on bookworm/ubuntu, `telnetlib3`, `junit_xml`), runs the validation suite. Currently green on the three targets.
### Building all channels (`build_all.sh`)
`build_all.sh` builds every artifact into `dist/` (manual PDF, wheel, PyInstaller binary, Flatpak bundle, AppImage). It reuses `scripts/build_env.sh` + `set_env.sh` so the venv at `test/tmp/.venv` stays the single source of Python deps; `build`/`pyinstaller`/`sphinx`/`linuxdoc` (and `pygls`, via the `[lsp]` extra) are installed there on demand. A step is skipped if its artifact already exists; `--clean` forces a rebuild.
- **Parallelism (default).** A serial *prep* phase does everything that writes the shared venv (the `pip install`s) plus the Flatpak runtime install and the wheel (the AppImage installs it). Then manual + PyInstaller + Flatpak + AppImage build concurrently — they only *read* the venv, so there is no concurrent-pip race. Per-step output goes to `dist/.build-logs/<step>.log`; results print in completion order (`wait -n`), and a failing step's log is dumped at the end. `--serial` builds one at a time. Ctrl+C is trapped to kill each job's whole process tree (subshell + grandchildren: podman container, flatpak-builder, pyinstaller), so no orphans survive.
- **`--ram` (slow/flash storage).** Redirects the build scratch to `/dev/shm` and skips UPX, a large win when building from a USB stick / SD card (I/O-bound on flash): `TMPDIR` + `PIP_CACHE_DIR`, the PyInstaller `--workpath` (`PYI_WORKPATH`), and a tmpfs bind-mount at the in-container AppImage AppDir (`APPIMAGE_APPDIR_TMPFS`); UPX is disabled via `TESTIUM_NO_UPX` (read by the `.spec`). **Flatpak is excluded**`flatpak-builder` mounts its state dir with `rofiles-fuse` and FUSE cannot mount on `/dev/shm` (`fusermount: Permission denied`), so it builds on disk. Each `package/*/build.sh` honours these env vars with on-disk defaults, so behaviour is unchanged without `--ram`; the tmpfs scratch is freed on exit. On a RAM-limited machine combine with `--serial`.
### Host-only py_func / lua_func in sandboxed bundles (Flatpak, AppImage)
The bundled Python (Flatpak's runtime python, AppImage's `python3.11`) is reserved for the **main process only**. Subprocesses (`py_func`, `lua_func`, `git`, the `run` item's sub-instance) must use the host's interpreters and tools so user-installed modules (pyserial, junit_xml, …) are visible. This is enforced by `interpreter/utils/bins.py`:
@@ -254,13 +261,26 @@ Validation is **opt-in per subclass**: while a subclass keeps `PARAMS = None` (t
Diagnostics are currently **warnings** for unknown params so an out-of-tree `.tum` with a pre-existing typo doesn't suddenly fail. The flip to a hard error is a one-line change in `_validate_declared_params()` once the user is comfortable.
The schema is also designed to be the source of truth for a future LSP server and for auto-generated manual sections: `ParamSet.to_schema()` returns the JSON-Schema-shaped representation that those consumers will read.
Action items follow the same declarative principle. A `TestItemActions` parent (`console`, `plot`, `json_rpc`) declares its nested actions as a class attribute `ACTIONS = {yaml_key: action_class}` (e.g. `{"open": TestItemConsoleOpen, "write": …}`), mirroring `PARAMS`. The base `TestItemActions.__init__` seeds `self.action_classes` from `type(self).ACTIONS`; the imperative `register_actions(**…)` method is retained only as an escape hatch for actions that can't be known at class-definition time (none today). Because the action classes are always defined above their parent in the module, the class-level dict resolves without forward-reference gymnastics.
The schema is the realized source of truth for the LSP server (`testium lsp`), the `testium schema` CLI dump, and future auto-generated manual sections: `ParamSet.to_schema()` returns the JSON-Schema-shaped representation, and `lsp/schema.py` reads both `PARAMS` and `ACTIONS` **purely from class attributes** — no `inspect.getsource`/AST parsing. This is what lets the full schema (including nested actions) survive a frozen PyInstaller build where the `.py` source isn't on disk.
### Language server (`testium lsp`) across channels
The `testium_assist` editor extension is a thin LSP client that spawns `testium lsp` and talks JSON-RPC over stdio, so the language server must work from *every* distribution channel. Two requirements:
1. **`pygls` (+ `lsprotocol`, `cattrs`, `attrs`, `typing_extensions`) must be bundled.** It is the pyproject `[lsp]` extra (kept optional so a plain `pip install testium` stays lean), wired into each full-app channel: `build_env.sh` installs it into the shared `test/tmp/.venv` (covers **source run** and the **PyInstaller** build env); the **AppImage** installs the wheel as `…whl[lsp]`; the **Flatpak** adds a `python3-lsp` pip module (network-at-build, consistent with the manifest's global `--share=network`); the **PyInstaller** `.spec` force-collects the submodules via `collect_submodules` + explicit `hiddenimports` (including the lazily-imported `lsp`, `lsp.server`, `lsp.schema`).
2. **The schema must build without source** — handled by the declarative `PARAMS`/`ACTIONS` above; PyInstaller is the only channel that strips `.py` source, and it no longer matters.
`test/validation/lsp_check.py` enforces both per channel: `run.sh` calls it before launching the suite, asserting that `<channel> schema` returns JSON whose `console`/`plot`/`json_rpc` items still carry their actions, and that `<channel> lsp` answers an `initialize` request with capabilities (and never reports the pygls dependency missing). So `./test/validation/run.sh --mode flatpak|pyinstaller|appimage` now fails loudly if a channel ships a broken or pygls-less language server.
### Version reporting (`interpreter/utils/version.py`)
Both Flatpak and AppImage export `TESTIUM_VERSION` from a launcher (Flatpak: launcher script in `org.testium.Testium.yaml`; AppImage: `runtime.env` in `AppImageBuilder.yml`). `get_testium_version()` checks `/.flatpak-info` / `APPIMAGE` and reads `TESTIUM_VERSION` rather than relying on package metadata or repo introspection.
## Recent fixes / notable changes
- `build_all.sh`: builds the four heavy channels in parallel (serial prep for the shared venv + wheel), results in completion order, Ctrl+C kills the whole job tree; `--ram` puts the build scratch on tmpfs (`/dev/shm`) + skips UPX for fast builds on USB/SD storage (Flatpak excluded — rofiles-fuse can't mount tmpfs). See the "Building all channels" section.
- LSP across packaging channels: `testium lsp` (and the `testium_assist` editor extension that spawns it) now works from source, wheel, PyInstaller, Flatpak and AppImage. Two enablers — (1) action items declare a class-level `ACTIONS = {key: class}` registry (like `PARAMS`), so `lsp/schema.py` builds the full schema from class attributes with no `inspect.getsource`/AST (which broke under frozen PyInstaller); (2) the `[lsp]` extra (pygls) is wired into every full-app channel. `test/validation/lsp_check.py`, run by `run.sh` before the suite, asserts per-channel that `schema` keeps its actions and `lsp` answers `initialize`. See the matching architecture sections.
- Declarative test item parameters (v0.2): each `TestItem` subclass exposes a `PARAMS = ParamSet(...)` class attribute consumed by the base `__init__`. Catches unknown YAML keys (typo warnings listing the accepted names) and missing required params (load-time errors with `.tum` context). Lays the schema foundation for a future LSP server and auto-generated manual sections. See the matching architecture section.
- Flatpak: `py_func` / `lua_func` / `run` sub-instance now execute on the host via `flatpak-spawn --host`. The previous attempt to inject host lib dirs into the sandbox's `LD_LIBRARY_PATH` was abandoned — host shared libs are ABI-incompatible with the Flatpak runtime's glibc and would trip `_dl_call_libc_early_init`. The manifest gained `--talk-name=org.freedesktop.Flatpak` so the spawn proxy call is allowed. The testium package is staged once per process under `/tmp` (shared with the host) so the host interpreter can locate `py_func` / `lua_func`.
- Validation suite: single entry point with `--mode source|wheel|pyinstaller|flatpak|appimage` to validate every packaging channel against the same items. Per-mode report filenames prevent clobbering.

View File

@@ -41,6 +41,20 @@ Pre-built artifacts are published at
`testium` command is available in the terminal (requires `~/.local/bin` in
`PATH`, which most modern distributions provide by default).
* **AppImage** (`Testium-<version>-x86_64.AppImage`) — a single self-contained
executable bundling its own Python. Make it executable and run it:
```sh
chmod +x Testium-*-x86_64.AppImage
./Testium-*-x86_64.AppImage -b mytest.tum
```
As with the binary and Flatpak, `py_func` / `lua_func` items run under the
*host* Python / Lua so your own modules stay visible.
Every channel ships the language server, so `testium lsp` (see
[Editor support](#editor-support)) works out of the box from any of them.
## Quick start
From a checkout of the repository:
@@ -82,6 +96,29 @@ python3 src/testium # GUI
python3 src/testium -b mytest.tum # batch
```
## Editor support
testium ships a Language Server Protocol (LSP) server that gives `.tum` files
completion of item types, hover documentation, and an outline view in any
LSP-capable editor:
```sh
testium lsp # speaks LSP over stdio; an editor's LSP client drives it
testium schema # dumps the item/parameter schema as JSON (what the LSP serves)
```
The server is bundled in every pre-built release (wheel, binary, Flatpak,
AppImage). For a source / wheel install, pull the language-server extra:
```sh
pip install 'testium[lsp]' # from PyPI / a wheel
pip install -e /path/to/testium/src[lsp] # from a source checkout
```
A VSCode / VSCodium client extension (`testium_assist`) wraps `testium lsp`;
the schema is built from testium itself, so new item types and parameters
appear in the editor on the next testium upgrade with no client change.
## Troubleshooting
### `wl_proxy_marshal_flags` symbol error

View File

@@ -1,5 +1,5 @@
#!/bin/bash
# Build every distribution channel of testium, in order:
# Build every distribution channel of testium:
# 1. Manual PDF -> dist/testium-manual-<v>.pdf
# 2. Wheel -> dist/testium-<v>-py3-none-any.whl (PEP 427 name)
# 3. PyInstaller binary -> dist/testium-<v>
@@ -11,6 +11,21 @@
# By default, a step is skipped if its artifact already exists in dist/.
# Pass --clean to remove existing dist/ artifacts and rebuild everything.
#
# Parallelism: the wheel is built first (the AppImage installs it), then the
# manual, PyInstaller, Flatpak and AppImage builds run concurrently. The shared
# venv at test/tmp/.venv is only WRITTEN during the serial prep phase (the
# `pip install` of build/sphinx/pyinstaller); the parallel builds only read it,
# so there is no concurrent-pip race. Pass --serial to build one step at a time
# (useful when debugging or on a resource-constrained machine). Per-step output
# of the parallel phase is captured under dist/.build-logs/<step>.log and the
# log of any failing step is printed at the end.
#
# Pass --ram to redirect the per-channel build scratch (PyInstaller workpath,
# AppImage AppDir) and TMPDIR/PIP_CACHE_DIR to /dev/shm, and skip UPX. Big
# speedup on slow/flash storage. Flatpak is excluded (its rofiles-fuse can't
# mount on /dev/shm), so it still builds on disk. On a RAM-limited machine
# combine with --serial (e.g. ./build_all.sh --ram --serial).
#
# All artifacts are collected (copied) under <repo>/dist/. Original outputs in
# src/dist/, package/*/dist/, doc/manual/ are left in place. Wheel and AppImage
# keep their original names (which already contain the version); manual,
@@ -26,9 +41,13 @@
set -e
CLEAN=0
SERIAL=0
RAM=0
for arg in "$@"; do
case "$arg" in
--clean|-c) CLEAN=1 ;;
--serial) SERIAL=1 ;;
--ram) RAM=1 ;;
*) echo "Unknown option: $arg" >&2; exit 1 ;;
esac
done
@@ -63,6 +82,31 @@ export REQ_PATH="$SCRIPT_DIR/src/requirements.txt"
bash "$SCRIPT_DIR/scripts/build_env.sh"
source "$SCRIPT_DIR/scripts/set_env.sh"
# ---------- RAM mode: put build scratch on tmpfs (--ram) ----------------------
# On slow storage (USB stick, SD card) the per-channel build dirs and temp
# churn dominate. --ram redirects the PyInstaller workpath, the AppImage AppDir
# and TMPDIR/PIP_CACHE_DIR to /dev/shm, and skips UPX. Flatpak is intentionally
# NOT moved: flatpak-builder mounts its state dir with rofiles-fuse, and FUSE
# can't mount on /dev/shm (fusermount: Permission denied) — so it builds on
# disk. The tmpfs scratch is freed on exit.
if [ "$RAM" -eq 1 ]; then
RAMROOT="/dev/shm/testium-build-${VERSION}"
echo "-- RAM mode: build scratch under $RAMROOT (tmpfs), freed on exit"
echo " (flatpak builds on disk — rofiles-fuse can't mount on /dev/shm)"
rm -rf "$RAMROOT"
mkdir -p "$RAMROOT"/{tmp,pip,pyi-work,appdir}
export TMPDIR="$RAMROOT/tmp"
export PIP_CACHE_DIR="$RAMROOT/pip"
export PYI_WORKPATH="$RAMROOT/pyi-work" # pyinstaller --workpath
export APPIMAGE_APPDIR_TMPFS="$RAMROOT/appdir" # AppDir bind-mount
export TESTIUM_NO_UPX=1 # skip slow UPX in the spec
trap 'rm -rf "$RAMROOT"' EXIT
if [ "$SERIAL" -ne 1 ]; then
echo " note: with --ram, prefer adding --serial so each step gets the"
echo " full tmpfs and you don't risk OOM (flatpak+appimage are ~1 GB each)."
fi
fi
step() {
echo
echo "================================================================"
@@ -70,50 +114,100 @@ step() {
echo "================================================================"
}
skip() { echo " (already built — skipping)"; }
# Kill a process and its whole descendant tree (children first) — used by the
# interrupt handler so SIGINT also stops grandchildren the parallel jobs spawned
# (podman container, flatpak-builder, pyinstaller …), not just the subshells.
_kill_tree() {
local pid=$1 c
for c in $(pgrep -P "$pid" 2>/dev/null); do
_kill_tree "$c"
done
kill -TERM "$pid" 2>/dev/null || true
}
# Set as INT/TERM handler around the parallel wait. Stops every running build
# tree, then exits — the EXIT trap (set under --ram) frees the tmpfs scratch.
_interrupt() {
echo >&2
echo "-- interrupted: stopping running builds…" >&2
local pid
for pid in "${!PID2NAME[@]}"; do
_kill_tree "$pid"
done
exit 130
}
# ---------- artifact paths ----------------------------------------------------
# 1. Manual PDF
MANUAL="$DIST_DIR/testium-manual-${VERSION}.pdf"
step "1/5 Manual PDF (version $VERSION)"
if [ ! -f "$MANUAL" ]; then
python -m pip install --quiet --upgrade sphinx linuxdoc
bash "$SCRIPT_DIR/doc/manual/sphinx/build_doc.sh"
cp -f "$SCRIPT_DIR/doc/manual/testium_manual.pdf" "$MANUAL"
else
skip
fi
PYI_BIN="$DIST_DIR/testium-${VERSION}"
FLATPAK_BUNDLE="$DIST_DIR/testium-${VERSION}.flatpak"
wheel_in_dist() { ls -1t "$DIST_DIR"/testium-${VERSION}-*.whl 2>/dev/null | head -1; }
appimage_in_dist() { ls -1t "$DIST_DIR"/Testium-${VERSION}-*.AppImage 2>/dev/null | head -1; }
# 2. Wheel — PEP 427 name kept (already contains version)
step "2/5 Wheel (version $VERSION)"
WHEEL=$(ls -1t "$DIST_DIR"/testium-${VERSION}-*.whl 2>/dev/null | head -1)
if [ -z "$WHEEL" ]; then
python -m pip install --quiet --upgrade build
# ---------- per-step build functions (assume tools are installed) -------------
build_wheel() {
if [ -n "$(wheel_in_dist)" ]; then echo "wheel: already built — skipping"; return 0; fi
echo "wheel: building"
(
cd "$SCRIPT_DIR/src"
rm -rf dist build *.egg-info
python -m build --wheel
)
WHEEL_SRC=$(ls -1t "$SCRIPT_DIR/src/dist"/*.whl | head -1)
WHEEL="$DIST_DIR/$(basename "$WHEEL_SRC")"
cp -f "$WHEEL_SRC" "$WHEEL"
else
skip
fi
local src; src=$(ls -1t "$SCRIPT_DIR/src/dist"/*.whl | head -1)
cp -f "$src" "$DIST_DIR/$(basename "$src")"
echo "wheel: done"
}
# 3. PyInstaller binary
PYI_BIN="$DIST_DIR/testium-${VERSION}"
step "3/5 PyInstaller binary (version $VERSION)"
if [ ! -f "$PYI_BIN" ]; then
python -m pip install --quiet --upgrade pyinstaller
build_manual() {
if [ -f "$MANUAL" ]; then echo "manual: already built — skipping"; return 0; fi
echo "manual: building"
bash "$SCRIPT_DIR/doc/manual/sphinx/build_doc.sh"
cp -f "$SCRIPT_DIR/doc/manual/testium_manual.pdf" "$MANUAL"
echo "manual: done"
}
build_pyinstaller() {
if [ -f "$PYI_BIN" ]; then echo "pyinstaller: already built — skipping"; return 0; fi
echo "pyinstaller: building"
bash "$SCRIPT_DIR/package/pyinstaller/build.sh"
cp -f "$SCRIPT_DIR/package/pyinstaller/dist/testium" "$PYI_BIN"
else
skip
fi
echo "pyinstaller: done"
}
build_flatpak() {
if [ -f "$FLATPAK_BUNDLE" ]; then echo "flatpak: already built — skipping"; return 0; fi
echo "flatpak: building"
(
cd "$SCRIPT_DIR/package/flatpak"
bash build.sh
)
cp -f "$SCRIPT_DIR/package/flatpak/testium.flatpak" "$FLATPAK_BUNDLE"
echo "flatpak: done"
}
build_appimage() {
if [ -n "$(appimage_in_dist)" ]; then echo "appimage: already built — skipping"; return 0; fi
echo "appimage: building"
(
cd "$SCRIPT_DIR/package/appimage"
bash build.sh
)
local src; src=$(ls -1t "$SCRIPT_DIR/package/appimage"/*.AppImage 2>/dev/null | head -1)
cp -f "$src" "$DIST_DIR/$(basename "$src")"
chmod +x "$DIST_DIR/$(basename "$src")"
echo "appimage: done"
}
# ---------- serial prep: tool installs (shared venv) + flatpak runtimes -------
step "Prep: build tools + runtimes (serial — shared venv)"
[ -f "$MANUAL" ] || python -m pip install --quiet --upgrade sphinx linuxdoc
[ -n "$(wheel_in_dist)" ] || python -m pip install --quiet --upgrade build
[ -f "$PYI_BIN" ] || python -m pip install --quiet --upgrade pyinstaller
# 4. Flatpak bundle
FLATPAK_BUNDLE="$DIST_DIR/testium-${VERSION}.flatpak"
step "4/5 Flatpak bundle (version $VERSION)"
if [ ! -f "$FLATPAK_BUNDLE" ]; then
FLATPAK_DEPS=(
"org.kde.Platform//6.10"
@@ -130,35 +224,76 @@ if [ ! -f "$FLATPAK_BUNDLE" ]; then
flatpak install --user --noninteractive flathub "$dep"
fi
done
(
cd "$SCRIPT_DIR/package/flatpak"
bash build.sh
)
cp -f "$SCRIPT_DIR/package/flatpak/testium.flatpak" "$FLATPAK_BUNDLE"
else
skip
fi
# 5. AppImage
step "5/5 AppImage (version $VERSION)"
APPIMAGE=$(ls -1t "$DIST_DIR"/Testium-${VERSION}-*.AppImage 2>/dev/null | head -1)
if [ -z "$APPIMAGE" ]; then
(
cd "$SCRIPT_DIR/package/appimage"
bash build.sh
)
APPIMAGE_SRC=$(ls -1t "$SCRIPT_DIR/package/appimage"/*.AppImage 2>/dev/null | head -1)
APPIMAGE="$DIST_DIR/$(basename "$APPIMAGE_SRC")"
cp -f "$APPIMAGE_SRC" "$APPIMAGE"
chmod +x "$APPIMAGE"
# ---------- serial: wheel (the AppImage installs it) --------------------------
step "1/5 Wheel (version $VERSION)"
build_wheel
# ---------- build the rest --------------------------------------------------
REST=(manual pyinstaller flatpak appimage)
if [ "$SERIAL" -eq 1 ]; then
n=2
for name in "${REST[@]}"; do
step "$n/5 $name (version $VERSION)"
"build_$name"
n=$((n + 1))
done
else
skip
step "2-5/5 manual + pyinstaller + flatpak + appimage (parallel)"
LOGDIR="$DIST_DIR/.build-logs"
mkdir -p "$LOGDIR"
declare -A PID2NAME
for name in "${REST[@]}"; do
log="$LOGDIR/$name.log"
echo " -> launching $name (log: $log)"
( "build_$name" ) >"$log" 2>&1 &
PID2NAME[$!]="$name"
done
# From here until all jobs are reaped, Ctrl+C stops every build tree.
trap _interrupt INT TERM
# Reap in completion order (wait -n) so each result prints the moment that
# build finishes, not when its slot comes up in the array.
FAILED=()
remaining=${#PID2NAME[@]}
while [ "$remaining" -gt 0 ]; do
if wait -n -p donepid; then rc=0; else rc=$?; fi
name="${PID2NAME[$donepid]:-}"
[ -z "$name" ] && continue
if [ "$rc" -eq 0 ]; then
echo " -> $name: OK"
else
echo " -> $name: FAILED (rc=$rc)"
FAILED+=("$name")
fi
remaining=$((remaining - 1))
done
trap - INT TERM
if [ "${#FAILED[@]}" -gt 0 ]; then
for name in "${FAILED[@]}"; do
echo
echo "===================== $name log ====================="
cat "$LOGDIR/$name.log"
done
echo >&2
echo "BUILD FAILED: ${FAILED[*]} (logs under $LOGDIR)" >&2
exit 1
fi
fi
# ---------- summary -----------------------------------------------------------
step "All packages built"
printf " manual : %s\n" "$MANUAL"
printf " wheel : %s\n" "$WHEEL"
printf " wheel : %s\n" "$(wheel_in_dist)"
printf " pyinstaller : %s\n" "$PYI_BIN"
printf " flatpak : %s\n" "$FLATPAK_BUNDLE"
printf " appimage : %s\n" "$APPIMAGE"
printf " appimage : %s\n" "$(appimage_in_dist)"
printf " release_note : %s\n" "$RELEASE_NOTE"

View File

@@ -77,7 +77,10 @@ AppDir:
python3.11 -m pip install --break-system-packages --upgrade --isolated --no-input --ignore-installed --prefix=$TARGET_APPDIR/usr -r ../../src/requirements.txt
export PIP_CONFIG_FILE=$HOME/.pip/pip.conf
python3.11 -m pip install --break-system-packages --upgrade --isolated --no-input --ignore-installed --prefix=$TARGET_APPDIR/usr ../../src/dist/testium-{{APP_VERSION}}-py3-none-any.whl
# Install the wheel with the [lsp] extra so `testium lsp` (pygls) works
# from the AppImage. The extra pulls pygls/lsprotocol/cattrs/attrs from
# the index (network is available at build time, see get-pip above).
python3.11 -m pip install --break-system-packages --upgrade --isolated --no-input --ignore-installed --prefix=$TARGET_APPDIR/usr "../../src/dist/testium-{{APP_VERSION}}-py3-none-any.whl[lsp]"
AppImage:

View File

@@ -17,11 +17,20 @@ else
fi
echo "Using $RUNTIME — building testium $APP_VERSION AppImage..."
# APPIMAGE_APPDIR_TMPFS (set by build_all --ram) bind-mounts a host tmpfs dir at
# the AppDir build path, keeping the ~1 GB AppDir churn off slow storage.
APPDIR_MOUNT=""
if [ -n "$APPIMAGE_APPDIR_TMPFS" ]; then
mkdir -p "$APPIMAGE_APPDIR_TMPFS"
APPDIR_MOUNT="-v $APPIMAGE_APPDIR_TMPFS:/work/package/appimage/AppDir"
fi
# APPIMAGE_EXTRACT_AND_RUN=1 lets appimagetool run without FUSE in the container.
$RUNTIME run --rm \
--privileged \
-e APPIMAGE_EXTRACT_AND_RUN=1 \
-v "$REPO_ROOT:/work" \
$APPDIR_MOUNT \
-w /work/package/appimage \
debian:bookworm bash -c "
set -e

View File

@@ -7,11 +7,19 @@
set -e
# Build + install local
flatpak-builder --user --verbose --force-clean --install --repo=repo build org.testium.Testium.yaml
# Build + install local. FLATPAK_BUILDDIR / FLATPAK_STATEDIR / FLATPAK_REPODIR
# (set by build_all --ram) redirect the build dir, the state dir
# (.flatpak-builder) and the ostree repo to tmpfs. flatpak-builder hardlinks
# between the state dir and the build dir, so they MUST be on the same
# filesystem — hence the state dir moves to tmpfs too (its download cache then
# doesn't persist across --ram runs).
BUILDDIR="${FLATPAK_BUILDDIR:-build}"
STATEDIR="${FLATPAK_STATEDIR:-.flatpak-builder}"
REPODIR="${FLATPAK_REPODIR:-repo}"
flatpak-builder --user --verbose --force-clean --install --state-dir="$STATEDIR" --repo="$REPODIR" "$BUILDDIR" org.testium.Testium.yaml
# Génère le bundle distribuable
flatpak build-bundle repo testium.flatpak org.testium.Testium
flatpak build-bundle "$REPODIR" testium.flatpak org.testium.Testium
echo "Bundle généré : $(pwd)/testium.flatpak"
# Crée ~/.local/bin/testium pour pouvoir taper "testium" en console

View File

@@ -28,6 +28,23 @@ build-options:
modules:
- python3-requirements.json
# Language-server deps for `testium lsp` (pygls + lsprotocol + cattrs + attrs
# + typing_extensions). Installed from PyPI at build time — the build already
# runs with --share=network (see build-options). The core runtime deps stay
# offline-pinned in python3-requirements.json; these are pure-python wheels,
# hence --only-binary=:all: (no compilation, deterministic).
- name: python3-lsp
buildsystem: simple
build-options:
build-args:
- --share=network
build-commands:
# Whole command single-quoted: the ':all: ' colon-space would otherwise
# make YAML parse this list item as a mapping, silently dropping the
# command (flatpak-builder then runs an empty module — installs nothing).
- 'pip3 install --prefix=${FLATPAK_DEST} --only-binary=:all: "pygls>=1.3"'
# 1. Dépendances Python tierces (HORS PySide6)
# Utilisez flatpak-pip-generator pour vos autres libs (ex: pyserial, requests, etc.)
# - name: python3-requirements

View File

@@ -2,11 +2,15 @@
SCRIPT_DIR=$(realpath $( dirname "$0"))
rm -r "${SCRIPT_DIR}/build" "${SCRIPT_DIR}/dist"
rm -rf "${SCRIPT_DIR}/build" "${SCRIPT_DIR}/dist"
pwd=$(pwd)
cd ${SCRIPT_DIR}
pyinstaller testium.spec
# PYI_WORKPATH (set by build_all --ram) puts the big intermediate build tree on
# tmpfs; dist/ stays local so build_all can collect the binary.
WORKARG=""
[ -n "$PYI_WORKPATH" ] && WORKARG="--workpath $PYI_WORKPATH"
pyinstaller $WORKARG testium.spec
RESULT=$?
if [ -n "$1" ] && [ "$1" = "install" ]; then
if [ $RESULT -eq 0 ]; then

View File

@@ -1,5 +1,21 @@
# -*- mode: python ; coding: utf-8 -*-
import os
from PyInstaller.utils.hooks import collect_submodules
# Language-server dependencies for `testium lsp`. pygls/lsprotocol register
# converters and features dynamically, so we collect their submodules wholesale
# and force-import their pure-python deps (cattrs/attrs/typing_extensions).
# The testium lsp modules are imported lazily by the CLI dispatch
# (`from lsp.server import serve`), which PyInstaller's static analysis misses —
# hence the explicit names. No source files need bundling: the schema export is
# now fully declarative (PARAMS + ACTIONS class attributes), so it no longer
# reads .py source via inspect.getsource (which fails in a frozen build).
_LSP_HIDDEN = (
collect_submodules("pygls")
+ collect_submodules("lsprotocol")
+ ["cattrs", "attr", "attrs", "typing_extensions",
"lsp", "lsp.server", "lsp.schema"]
)
# junit_xml is imported by post_exec scripts running under the *host* Python,
# not the frozen interpreter — so bundling it via hiddenimports alone is not
@@ -54,7 +70,7 @@ a = Analysis(
"colorama",
"matplotlib",
"junit_xml",
"lxml"],
"lxml"] + _LSP_HIDDEN,
hookspath=[],
hooksconfig={},
runtime_hooks=[],
@@ -73,7 +89,9 @@ exe = EXE(
debug=False,
bootloader_ignore_signals=False,
strip=False,
upx=True,
# UPX is CPU+IO heavy for a marginal size gain — build_all --ram sets
# TESTIUM_NO_UPX=1 to skip it (much faster on slow/flash storage).
upx=not os.environ.get("TESTIUM_NO_UPX"),
upx_exclude=[],
runtime_tmpdir=None,
console=True,

View File

@@ -5,6 +5,17 @@ version 0.2
WARN listing the accepted names instead of being silently ignored;
missing required parameters error out at load time with the source
``.tum`` file as context. No change to valid existing tests.
- Editor support: testium now ships a language server. ``testium lsp``
gives ``.tum`` files item-type completion, hover documentation and an
outline view in any LSP-capable editor (a VSCode / VSCodium client is
provided separately). ``testium schema`` dumps the item/parameter
schema as JSON. The server works from every channel — bundled in the
binary / Flatpak / AppImage, and pulled by ``pip install testium[lsp]``
for wheel installs.
- build_all.sh: the four heavy channels now build in parallel (results
reported as each finishes; Ctrl+C stops them cleanly). New ``--ram``
option builds on a tmpfs (``/dev/shm``) and skips UPX for much faster
packaging on USB-stick / SD-card storage.
version 0.1.3
==============

View File

@@ -33,6 +33,11 @@ if [ ! -d "$PY_VENV_DIR" ]; then
python3 -m venv "$PY_VENV_DIR"
source "$PY_VENV_DIR/bin/activate"
pip install --extra-index-url https://pypi.python.org/pypi -r $REQ_PATH
# Language-server deps (the pyproject [lsp] extra). Installed here so the
# source run AND the PyInstaller build — both of which use this venv — can
# start / collect the `testium lsp` server. pip-installed wheel users get
# them via `pip install testium[lsp]` instead.
pip install --extra-index-url https://pypi.python.org/pypi "pygls>=1.3"
# Validation suite plugin used to verify the report-exporter
# entry-points discovery end-to-end.
FAKE_EXPORTER_DIR="$(dirname "$REQ_PATH")/../test/validation/fake_exporter"

View File

@@ -5,6 +5,13 @@ from interpreter.test_items.item_actions.action import TestItemAction
class TestItemActions(TestItem):
# Declarative action registry: subclasses set ``ACTIONS = {yaml_key: class}``
# as a class attribute (mirroring ``PARAMS``). It is read here to populate
# the runtime registry, and read identically by the schema export — no
# instantiation or source inspection required. ``register_actions()`` stays
# available as an imperative escape hatch for dynamic/conditional cases.
ACTIONS = {}
def __init__(
self, item_type, dict_actions, parent=None, status_queue=None, filename=""
):
@@ -12,7 +19,7 @@ class TestItemActions(TestItem):
super().__init__(dict_actions, parent, status_queue, filename=filename)
self._type = item_type
self.is_container = False
self.action_classes = {}
self.action_classes = dict(type(self).ACTIONS)
self.actions_token = None
self.actions = []
try:
@@ -24,6 +31,9 @@ class TestItemActions(TestItem):
)
def register_actions(self, **args: TestItemAction):
# Imperative escape hatch. The declarative ``ACTIONS`` class attribute
# covers every current subclass; use this only to add actions that
# can't be known at class-definition time (e.g. platform-conditional).
for action_name, action_class in args.items():
self.action_classes.update({action_name: action_class})

View File

@@ -388,18 +388,19 @@ class TestItemConsole(TestItemActions):
"as long as their names differ."),
)
ACTIONS = {
"open": TestItemConsoleOpen,
"close": TestItemConsoleClose,
"write": TestItemConsoleWrite,
"writeln": TestItemConsoleWriteLn,
"read_until": TestItemConsoleReadUntil,
}
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
super().__init__(
cst.TYPE_CONSOLE, dict_item, parent, status_queue, filename=filename
)
self.register_actions(
open=TestItemConsoleOpen,
close=TestItemConsoleClose,
write=TestItemConsoleWrite,
writeln=TestItemConsoleWriteLn,
read_until=TestItemConsoleReadUntil,
)
self.actions_token = {}
global console

View File

@@ -210,6 +210,13 @@ class TestItemJSON_RPC(TestItemActions):
doc="If true, don't echo wire traffic to the log."),
)
ACTIONS = {
"open": TestItemJSRPCActionOpen,
"close": TestItemJSRPCActionClose,
"query": TestItemJSRPCActionQuery,
"receive": TestItemJSRPCActionReceive,
}
def __init__(
self, dict_item: dict, parent: TestItem = None, status_queue=None, filename=""
):
@@ -217,13 +224,6 @@ class TestItemJSON_RPC(TestItemActions):
cst.TYPE_JSON_RPC, dict_item, parent, status_queue, filename=filename
)
self.register_actions(
open=TestItemJSRPCActionOpen,
close=TestItemJSRPCActionClose,
query=TestItemJSRPCActionQuery,
receive=TestItemJSRPCActionReceive,
)
# Console specific params
self._console = self._prms.getParam("console", required=False)
# UDP specific params

View File

@@ -263,18 +263,18 @@ class TestItemPlot(TestItemActions):
"action and by $(plv_<plot_name>) for last-values output."),
)
ACTIONS = {
"open": TestItemPlotActionOpen,
"close": TestItemPlotActionClose,
"periodic": TestItemPlotActionPeriodic,
"add": TestItemPlotActionAdd,
"last_value": TestItemPlotActionLastValues,
"export": TestItemPlotActionExport,
}
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
super().__init__(
cst.TYPE_GRAPH, dict_item, parent, status_queue, filename=filename
)
self.register_actions(
open=TestItemPlotActionOpen,
close=TestItemPlotActionClose,
periodic=TestItemPlotActionPeriodic,
add=TestItemPlotActionAdd,
last_value=TestItemPlotActionLastValues,
export=TestItemPlotActionExport,
)
self.actions_token = self._prms.getParam("plot_name", required=True)

View File

@@ -24,41 +24,17 @@ from interpreter.utils.test_init import _constants_init
# Action class -> parent cmd (the action's parent in the YAML). Action classes
# aren't first-class TestItemType entries (TYPE_*_ACTION is one generic bucket),
# so we resolve their YAML key by looking at how each parent registers them.
# so we resolve their YAML key from the parent's declarative ``ACTIONS`` map.
def _collect_action_classes(parent_class):
"""Return {action_yaml_key: action_class} for a TestItemActions parent.
Each parent's ``__init__`` calls ``self.register_actions(name=class, ...)``
*during* construction, so we can't read the registry without instantiating
one. We work around it by parsing the source for the registration call —
cheap, no side effects, and the schema export is a CLI command anyway.
Each parent declares its actions as a class-level ``ACTIONS = {key: class}``
attribute (see ``item_actions/TestItemActions``). We read it directly — no
instantiation, no source inspection — so this works identically whether the
package runs from source, a wheel, or a frozen (PyInstaller) build where the
``.py`` source isn't on disk.
"""
import ast
import inspect
try:
src = inspect.getsource(parent_class)
except (OSError, TypeError):
return {}
actions = {}
tree = ast.parse(src)
for node in ast.walk(tree):
if not isinstance(node, ast.Call):
continue
fn = node.func
if not (isinstance(fn, ast.Attribute) and fn.attr == "register_actions"):
continue
for kw in node.keywords:
if kw.arg is None or not isinstance(kw.value, ast.Name):
continue
# ast.Name gives us only the bare identifier; resolve it through
# the parent class's defining module.
mod = __import__(parent_class.__module__, fromlist=[kw.value.id])
cls = getattr(mod, kw.value.id, None)
if cls is not None:
actions[kw.arg] = cls
return actions
return dict(getattr(parent_class, "ACTIONS", None) or {})
def _params_to_schema(item_class, common_params):

View File

@@ -0,0 +1,102 @@
#!/usr/bin/env python3
"""Per-channel check of the testium language server.
Given the channel's testium invocation as argv (e.g. ``flatpak run
--command=testium org.testium.Testium``, a PyInstaller binary path, or
``python -m testium``), verify two things end-to-end against that exact build:
1. ``<cmd> schema`` produces valid JSON whose item registry still includes the
nested action sets (console/plot/json_rpc). This catches a frozen build
that lost the actions — the failure mode the declarative ``ACTIONS``
refactor fixed (no more ``inspect.getsource`` at runtime).
2. ``<cmd> lsp`` starts a real language server: it must answer an LSP
``initialize`` request with a capabilities result and must NOT report the
pygls dependency as missing. This catches a channel that forgot to bundle
the ``[lsp]`` extra.
Exits non-zero (with a diagnostic) on the first failure so the validation run
fails loudly. Used by ``run.sh`` before launching the main suite.
"""
import json
import subprocess
import sys
EXPECTED_ACTION_PARENTS = ("console", "plot", "json_rpc")
def fail(msg):
print(f"LSP CHECK: FAIL — {msg}", file=sys.stderr)
sys.exit(1)
def _extract_json(raw):
"""Parse JSON from ``raw`` bytes, tolerating leading non-JSON noise.
The source-mode launcher (run.sh) may print env-setup lines before the
schema JSON, so we fall back to parsing from the first ``{``.
"""
try:
return json.loads(raw)
except json.JSONDecodeError:
start = raw.find(b"{")
if start < 0:
raise
return json.loads(raw[start:])
def check_schema(cmd):
try:
out = subprocess.run(cmd + ["schema"], capture_output=True, timeout=120)
except Exception as e: # noqa: BLE001
fail(f"`{' '.join(cmd)} schema` could not run: {e}")
if out.returncode != 0:
fail(f"`schema` exited {out.returncode}: {out.stderr.decode()[:300]}")
try:
data = _extract_json(out.stdout)
except json.JSONDecodeError as e:
fail(f"`schema` output is not valid JSON: {e}")
items = data.get("items", {})
for parent in EXPECTED_ACTION_PARENTS:
actions = (items.get(parent) or {}).get("actions") or {}
if not actions:
fail(f"schema item '{parent}' has no actions — a frozen build lost "
f"the declarative ACTIONS (item keys: {sorted(items)[:8]}…)")
print(f"LSP CHECK: schema OK ({len(items)} items; actions present for "
f"{', '.join(EXPECTED_ACTION_PARENTS)})")
def check_lsp(cmd):
body = json.dumps({
"jsonrpc": "2.0", "id": 1, "method": "initialize",
"params": {"processId": None, "rootUri": None, "capabilities": {}},
}).encode()
msg = b"Content-Length: %d\r\n\r\n%s" % (len(body), body)
try:
out = subprocess.run(cmd + ["lsp"], input=msg,
capture_output=True, timeout=30)
stdout, stderr = out.stdout, out.stderr
except subprocess.TimeoutExpired as e:
# A server that stays alive past initialize is fine — it just never saw
# a shutdown. Use whatever it wrote so far as the response.
stdout, stderr = e.stdout or b"", e.stderr or b""
blob = stdout + stderr
if b"dependencies missing" in blob:
fail("`lsp` reports the pygls dependency missing — this channel did "
"not bundle the [lsp] extra.")
if b'"capabilities"' not in stdout:
fail("`lsp` did not return an initialize result. "
f"stdout[:200]={stdout[:200]!r} stderr[:200]={stderr[:200]!r}")
print("LSP CHECK: lsp initialize OK (server answered with capabilities)")
def main():
cmd = sys.argv[1:]
if not cmd:
fail("usage: lsp_check.py <testium-invocation...>")
check_schema(cmd)
check_lsp(cmd)
print("LSP CHECK: PASS")
if __name__ == "__main__":
main()

View File

@@ -137,6 +137,13 @@ esac
echo "-- validation mode: $MODE"
echo "-- launch: ${CMD[*]}"
# ---------- LSP check (this exact channel) ------------------------------------
# Verify `testium lsp` / `testium schema` work in the build under test before
# running the suite: schema must keep its nested actions (declarative ACTIONS,
# survives frozen builds) and the language server must start (pygls bundled).
echo "-- LSP check ($MODE)"
"$VENV_PYTHON" "$SCRIPT_DIR/lsp_check.py" "${CMD[@]}"
exec "${CMD[@]}" -b \
-d "python_bin=$VENV_PYTHON" \
-d "validation_report_file=validation-$MODE" \