francois
4d8cafb5a0
eval_proc was started before -d/GUI defines reached gd, so ``-d python_bin=...`` and the GUI ``python_bin`` preference were silently ignored by the very subprocess that runs ``<| ... |>`` evals (and only took effect for later items once the discovery cache had already been seeded with the system interpreter). apply_overrides() is now applied before eval_process_init(), and bins._resolve()'s cache is keyed by (name, override) so a later param.yaml change re-resolves on the next lookup. The validation suite now ships a wrapper (run.sh / run.bat) that creates a dedicated venv in the system temp dir and pins it via ``-d python_bin=...``. A new ``venv`` item asserts the override took effect for both eval_proc and py_func paths, with a ``sys.prefix != sys.base_prefix`` marker to catch the case where the override happens to be a system interpreter (path-equality alone would miss it, the venv's ``bin/python3`` being a symlink to the host). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Validation
This directory contains the testium validation suite.
Running the suite
./test/validation/run.sh # Linux
test\validation\run.bat # Windows
The wrapper creates a dedicated Python venv in the system temp dir
(${TMPDIR:-/tmp}/testium-validation-venv on Linux, %TEMP%\testium-validation-venv
on Windows), using --system-site-packages so existing system packages
stay visible. The validation suite is then run with that venv pinned as
python_bin. Every test-execution subprocess (inline <| ... |>
evaluation, py_func, cycle, post_execution, ...) runs inside the
venv, while testium itself keeps running in the project's own
environment.
Pass clean as the first argument to recreate the venv from scratch
(useful after a system Python upgrade):
./test/validation/run.sh clean
What is checked
The venv item under items/venv/ asserts that the venv is actually
being used:
python_binis set in the global dict.- The eval subprocess (used for
<| ... |>expressions) hassys.executable == python_bin,sys.prefix == dirname(dirname(python_bin)), andsys.prefix != sys.base_prefix(i.e. is actually inside a venv). - A
py_funcsubprocess passes the same three checks.
These checks use abspath/normpath rather than realpath on
purpose: the venv's bin/python3 is a symlink to the host interpreter,
so realpath would map both venv and non-venv interpreters to the same
target. sys.prefix != sys.base_prefix is the venv-specific marker
that distinguishes the two cases.