François Dausseur
9dae210f7f
Make the suite run cleanly on Windows.
Product code:
- __init__.py: force UTF-8 on stdout/stderr. The Windows console code
page (cp1252) cannot encode the box-drawing/accented characters the
runner prints, which crashed the parent capture_stdout thread. Only
the stream encoders are reconfigured; the locale default used to read
cp1252 config files is left untouched.
- report_export_junit/html: open the report file with encoding="utf-8"
(XML/HTML are UTF-8) instead of the platform default, matching the
txt/json exporters.
Validation:
- run.bat: source mode now sets up its own venv and runs testium from
src\ directly instead of delegating to the project run.bat (which
launches the GUI and drops its arguments). Installs the fake_exporter
entry-point plugin (report_plugin) and the [lsp] extra, and runs the
same lsp_check.py pre-flight as run.sh.
- jsonrpc/test.tum: launch the echo server via "$(python_bin)" instead
of "python3" (the Microsoft Store stub on Windows).
- post_execution.py: write the JUnit XML with encoding="utf-8".
- restore items/run/sub_pass.tum and sub_fail.tum, deleted by mistake in
d97d00c "removed test logs".
Validation
This directory contains the testium validation suite. A single set of
items (items/), fixtures and post-processing (post_execution.py) is
re-used across every packaging channel.
Running the suite
./test/validation/run.sh # default mode = source
./test/validation/run.sh --mode wheel
./test/validation/run.sh --mode pyinstaller
./test/validation/run.sh --mode flatpak
./test/validation/run.sh --mode appimage
On Windows (only source, wheel, pyinstaller are supported):
test\validation\run.bat --mode pyinstaller
Pass clean as the first argument to recreate the validation venv
from scratch (useful after a system Python upgrade):
./test/validation/run.sh clean --mode flatpak
Any extra arguments after the mode flag are forwarded to testium.
Modes
| Mode | What it launches | Prerequisite |
|---|---|---|
source |
python3 src/testium via the project's run.sh |
none — works straight out of the repo |
wheel |
python -m testium inside a dedicated wheel venv |
./build_all.sh produced dist/testium-<v>-py3-none-any.whl |
pyinstaller |
dist/testium-<v> (frozen binary) |
./build_all.sh produced the PyInstaller binary |
flatpak |
flatpak run --command=testium org.testium.Testium |
the Flatpak bundle is installed (flatpak install --user dist/testium-<v>.flatpak) |
appimage |
dist/Testium-<v>-x86_64.AppImage |
./build_all.sh produced the AppImage |
Each mode writes its results to a distinct report file
(validation-<mode>.sqlite / validation-<mode>-<item>.xml), so you
can run several modes in a row without clobbering previous reports.
How python_bin is pinned
Every test-execution subprocess (inline <| ... |> evaluation,
py_func, cycle, post_execution, …) is routed through a dedicated
venv at ${TMPDIR:-/tmp}/testium-validation-venv. The venv is created
with --system-site-packages so existing system packages stay visible,
then junit-xml is pip-installed for post_execution.py.
This is a host venv. In every mode (including Flatpak) the
test-execution subprocesses end up running on the host — directly for
source/wheel/pyinstaller/appimage, and via flatpak-spawn --host for
Flatpak — so the same venv works across modes. The wheel mode
additionally creates a separate testium-wheel-venv-<v> to hold the
installed wheel; that one is only used to launch testium itself.
What is checked
The venv item under items/venv/ asserts that the validation venv is
actually being used:
python_binis set in the global dict.- The eval subprocess (used for
<| ... |>expressions) hassys.executable == python_bin,sys.prefix == dirname(dirname(python_bin)), andsys.prefix != sys.base_prefix(i.e. is actually inside a venv). - A
py_funcsubprocess passes the same three checks.
These checks use abspath/normpath rather than realpath on
purpose: the venv's bin/python3 is a symlink to the host interpreter,
so realpath would map both venv and non-venv interpreters to the same
target. sys.prefix != sys.base_prefix is the venv-specific marker
that distinguishes the two cases.