Make the suite run cleanly on Windows.
Product code:
- __init__.py: force UTF-8 on stdout/stderr. The Windows console code
page (cp1252) cannot encode the box-drawing/accented characters the
runner prints, which crashed the parent capture_stdout thread. Only
the stream encoders are reconfigured; the locale default used to read
cp1252 config files is left untouched.
- report_export_junit/html: open the report file with encoding="utf-8"
(XML/HTML are UTF-8) instead of the platform default, matching the
txt/json exporters.
Validation:
- run.bat: source mode now sets up its own venv and runs testium from
src\ directly instead of delegating to the project run.bat (which
launches the GUI and drops its arguments). Installs the fake_exporter
entry-point plugin (report_plugin) and the [lsp] extra, and runs the
same lsp_check.py pre-flight as run.sh.
- jsonrpc/test.tum: launch the echo server via "$(python_bin)" instead
of "python3" (the Microsoft Store stub on Windows).
- post_execution.py: write the JUnit XML with encoding="utf-8".
- restore items/run/sub_pass.tum and sub_fail.tum, deleted by mistake in
d97d00c "removed test logs".
- term console via flatpak-spawn --host so host venvs resolve (bins.host_console_command)
- QSettings sync() before subprocess kill in choices/tested-refs dialogs
- console regression test: fails on the in-sandbox 0.2.1 console
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
eval_proc was started before -d/GUI defines reached gd, so
``-d python_bin=...`` and the GUI ``python_bin`` preference were
silently ignored by the very subprocess that runs ``<| ... |>`` evals
(and only took effect for later items once the discovery cache had
already been seeded with the system interpreter). apply_overrides() is
now applied before eval_process_init(), and bins._resolve()'s cache is
keyed by (name, override) so a later param.yaml change re-resolves on
the next lookup.
The validation suite now ships a wrapper (run.sh / run.bat) that
creates a dedicated venv in the system temp dir and pins it via
``-d python_bin=...``. A new ``venv`` item asserts the override took
effect for both eval_proc and py_func paths, with a
``sys.prefix != sys.base_prefix`` marker to catch the case where the
override happens to be a system interpreter (path-equality alone would
miss it, the venv's ``bin/python3`` being a symlink to the host).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Two new steps per language: function returning nothing and function
returning explicit nil/None. Both tagged $(test)_PASS — they would
have failed before the lua nil fix (Lua side reported nil result as
error). Python side already worked but is covered for parity.
Added some api accessible from python and lua sub_processes. Now the tests only access to py_func.tm instead of direct api.testium module access.
Corrected some f"xxx" to allow working with old python (bookworm).
Changed param.yaml of the test to allow lua to work in all situations.
Various other small fixes for frozen app, wheel.
Tested in all situations, and OK. Ready for tag !
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Move src/lib/ → src/testium/runtime/ (internal plumbing)
Move src/testium/libs/ → src/testium/api/ (public SDK for test scripts)
Move src/py_func/ → src/testium/py_func/ (Python subprocess)
Move src/lua_func/ → src/testium/lua_func/ (Lua subprocess data)
The package now ships as a single coherent unit instead of four sibling
top-level packages (testium, lib, py_func, lua_func) — pip install
gives a clean site-packages/testium/ with no namespace pollution; .lua
files travel with the wheel via package_data; the wheel installs
cleanly and `testium -b` runs end-to-end including py_func subprocesses
and entry-point exporter plugins.
Naming:
- runtime/ (internal, no API guarantees) clearer than lib/
- api/ (public SDK consumed as `import api.testium as tm`) clearer than libs/
Imports updated en masse: from lib. → from runtime. and from libs. →
from api., plus the importlib.import_module("libs.*") strings in
test_item_console.py and test_item_runtime_plot.py. Test/example
scripts (helper_lib.py, parallel.py, post_execution.py) and the
fake_exporter test suite migrated too.
paths.py: subproc_path() now returns testium_path() — both point at
the testium package directory since the subprocesses live inside.
pyproject.toml: removed exclude=["lua_func", "py_func"] (no longer
needed), added package-data for testium.lua_func/*.lua, removed the
license classifier (PEP 639 conflict with license expression).
Subprocess isolation contract: py_func/ and lua_func/ may only import
runtime/ and their own modules — never interpreter/, main_win/, api/,
or testium/. Enforced by test/validation/items/isolation/ which runs a
py_func that statically scans subprocess source files for forbidden
imports. The contract holds today; the test prevents future drift.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Replace the hardcoded if/elif in Export.exec() with a dict registry.
Built-in formats (text, json, junit, html) are registered as lazy
loaders; missing optional deps (junit_xml, lxml) print a clear message
with a pip install hint instead of raising. Entry-points
(group "testium.exporters") are discovered at import time — installed
plugins are auto-detected with no extra config.
An unknown or unavailable format prints an info line and skips the
export; the test run is not interrupted.
Validation:
- New testium-fake-exporter package under test/validation/fake_exporter/
installed automatically by scripts/build_env.sh on venv creation.
It registers fake_format via entry-points and exports the tests
table to CSV — a real, useful exporter that exercises the plugin
contract end-to-end (entry-point discovery, dispatch, SQLite query).
- New dedicated items/report_plugin/ test exercises both the
unknown-format skip path and the fake_format plugin path, with a
py_func check (file_check.py) on the produced CSV. Runs once per
validation suite.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The parallel item runs branches concurrently with sync:all or sync:any
policy and optional per-branch wait_for synchronization. Each branch
runs in its own daemon thread and produces a clean per-item entry in
the SQLite report; the live output is prefixed [<branch_name>] so
concurrent branches stay readable.
Supporting changes:
- StdoutProxy (lib/stdout_redirect.py): thread-aware sys.stdout/stderr
with per-thread capture buffers and per-branch live-output prefix.
Adds writeln() for Python 3.14 unittest compatibility.
- TestItemContainer: shared base extracted from Group/Cycle for the
sequential children execution pattern.
- TestItemSleep: interruptible loop polling _is_stopped so sync:any
can cancel slow branches quickly.
- TestReport: thread-safe SQLite (check_same_thread=False + lock).
Also drops the unused -m/--terminal mode and its module.
Validation: 11 scenarios in test/validation/items/parallel covering
sync:all/any, no_fail, wait_for + timeout, conditions, multi-branch,
nested parallel, parallel inside loop, real branch failure.
Documentation: new parallel_test_item.rst added to the manual.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- run item: rename tum_fime→tum, remove stdout=PIPE (deadlock with
spawn), support batch mode (-b), SUCCESS on any completed subprocess
regardless of sub-test result
- batch.py: fix control("loaded") deadlock via daemon thread + Event +
is_alive() polling; fix premature finish on gd_update messages;
propagate success flag from finished message; guard control("close")
- process.py: include success flag in send_finished message
- py_process/lua_process: add stdout/stderr=DEVNULL to Popen
- test_run.py: fix finished detection ("id" in m and m["id"] is None)
- testium_win.py: track run_exit_code, SIGABRT handler, clean exit
- __init__.py: sys.exit with batch success flag
- Add run item validation tests and CLAUDE.md documentation
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Each dialog test item now accepts an optional auto_result parameter
(ok/cancel/yes/no) and auto_value for text dialogs. When set, the dialog
window opens, stays visible 2 seconds, then closes automatically with the
specified result — allowing the validation suite to run without manual
interaction.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Allows any test item to store its result (or PASS/FAIL status when result
is None) into a named global variable, available to subsequent items via
$(variable_name). store_result runs after expected_result but before
no_fail so the real outcome is always captured.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Without expected_result, a False process_result value does not fail the
test. Adding expected_result: True makes the comparison fail as intended.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replaces the external jrpces binary dependency with a self-contained
Python script. The server supports TCP (newline-delimited JSON, port 4321)
and UDP (port 4323), handles JSON-RPC 1.0 and 2.0, and implements:
- echo(*args) -> [args, {}]
- unknown methods -> error {code: -32000, message: "function not found"}
test.tum is updated to launch jrpc_echo_server.py via python3 and wait
for the "ready" readiness message before running tests.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- py_func and lua_func items accept a context_id parameter; items sharing
the same id reuse the same subprocess for the duration of the test run
- Subprocess-side tm.setgd/tm.gd use a local fallback dict for non-JSON-
serializable values (py_func only); serializable values reach the main
process global dict and are accessible from any test item or subprocess
- Shared subprocess engines are cleaned up in process.py finally block
- LuaProcessBase gains is_alive() (was missing, broke all lua_func items)
- Validation tests cover serializable sharing across different context ids,
non-serializable sharing within the same context_id, and cross-item access
- RST documentation updated for both py_func and lua_func items
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
