flatpak: run host interpreters via flatpak-spawn; validation --mode flag
py_func, lua_func and the run item now reach host binaries through `flatpak-spawn --host` instead of trying to load them under the sandbox runtime (which fails with a glibc ABI mismatch). Adds `--talk-name=org.freedesktop.Flatpak` to the manifest, stages the /app/lib/testium tree under /tmp so the host can read it, and drops the dead `_FLATPAK_HOST_DIRS` / lib-injection code paths that the new approach makes obsolete. Validation suite gains a `--mode source|wheel|pyinstaller|flatpak| appimage` flag so the same item set can run against every packaging channel; per-mode report file names avoid clobbering. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -1,34 +1,67 @@
|
||||
# Validation
|
||||
|
||||
This directory contains the testium validation suite.
|
||||
This directory contains the testium validation suite. A single set of
|
||||
items (`items/`), fixtures and post-processing (`post_execution.py`) is
|
||||
re-used across every packaging channel.
|
||||
|
||||
## Running the suite
|
||||
|
||||
```sh
|
||||
./test/validation/run.sh # Linux
|
||||
test\validation\run.bat # Windows
|
||||
./test/validation/run.sh # default mode = source
|
||||
./test/validation/run.sh --mode wheel
|
||||
./test/validation/run.sh --mode pyinstaller
|
||||
./test/validation/run.sh --mode flatpak
|
||||
./test/validation/run.sh --mode appimage
|
||||
```
|
||||
|
||||
The wrapper creates a dedicated Python venv in the system temp dir
|
||||
(`${TMPDIR:-/tmp}/testium-validation-venv` on Linux, `%TEMP%\testium-validation-venv`
|
||||
on Windows), using `--system-site-packages` so existing system packages
|
||||
stay visible. The validation suite is then run with that venv pinned as
|
||||
`python_bin`. Every test-execution subprocess (inline `<| ... |>`
|
||||
evaluation, `py_func`, `cycle`, `post_execution`, ...) runs inside the
|
||||
venv, while testium itself keeps running in the project's own
|
||||
environment.
|
||||
On Windows (only `source`, `wheel`, `pyinstaller` are supported):
|
||||
|
||||
Pass `clean` as the first argument to recreate the venv from scratch
|
||||
(useful after a system Python upgrade):
|
||||
```bat
|
||||
test\validation\run.bat --mode pyinstaller
|
||||
```
|
||||
|
||||
Pass `clean` as the **first** argument to recreate the validation venv
|
||||
from scratch (useful after a system Python upgrade):
|
||||
|
||||
```sh
|
||||
./test/validation/run.sh clean
|
||||
./test/validation/run.sh clean --mode flatpak
|
||||
```
|
||||
|
||||
Any extra arguments after the mode flag are forwarded to testium.
|
||||
|
||||
## Modes
|
||||
|
||||
| Mode | What it launches | Prerequisite |
|
||||
|---------------|-------------------------------------------------------------|------------------------------------------------------------------|
|
||||
| `source` | `python3 src/testium` via the project's `run.sh` | none — works straight out of the repo |
|
||||
| `wheel` | `python -m testium` inside a dedicated wheel venv | `./build_all.sh` produced `dist/testium-<v>-py3-none-any.whl` |
|
||||
| `pyinstaller` | `dist/testium-<v>` (frozen binary) | `./build_all.sh` produced the PyInstaller binary |
|
||||
| `flatpak` | `flatpak run --command=testium org.testium.Testium` | the Flatpak bundle is installed (`flatpak install --user dist/testium-<v>.flatpak`) |
|
||||
| `appimage` | `dist/Testium-<v>-x86_64.AppImage` | `./build_all.sh` produced the AppImage |
|
||||
|
||||
Each mode writes its results to a distinct report file
|
||||
(`validation-<mode>.sqlite` / `validation-<mode>-<item>.xml`), so you
|
||||
can run several modes in a row without clobbering previous reports.
|
||||
|
||||
## How `python_bin` is pinned
|
||||
|
||||
Every test-execution subprocess (inline `<| ... |>` evaluation,
|
||||
`py_func`, `cycle`, `post_execution`, …) is routed through a dedicated
|
||||
venv at `${TMPDIR:-/tmp}/testium-validation-venv`. The venv is created
|
||||
with `--system-site-packages` so existing system packages stay visible,
|
||||
then `junit-xml` is pip-installed for `post_execution.py`.
|
||||
|
||||
This is a **host** venv. In every mode (including Flatpak) the
|
||||
test-execution subprocesses end up running on the host — directly for
|
||||
source/wheel/pyinstaller/appimage, and via `flatpak-spawn --host` for
|
||||
Flatpak — so the same venv works across modes. The wheel mode
|
||||
additionally creates a separate `testium-wheel-venv-<v>` to hold the
|
||||
installed wheel; that one is only used to launch testium itself.
|
||||
|
||||
## What is checked
|
||||
|
||||
The `venv` item under `items/venv/` asserts that the venv is actually
|
||||
being used:
|
||||
The `venv` item under `items/venv/` asserts that the validation venv is
|
||||
actually being used:
|
||||
|
||||
* `python_bin` is set in the global dict.
|
||||
* The eval subprocess (used for `<| ... |>` expressions) has
|
||||
|
||||
Reference in New Issue
Block a user