Compare commits
3 Commits
refactor/i
...
91341b74e2
| Author | SHA1 | Date | |
|---|---|---|---|
| 91341b74e2 | |||
| 2a5e0bebd4 | |||
| 8791a2e3f3 |
21
DESIGN.md
21
DESIGN.md
@@ -114,20 +114,11 @@ To add a new API call usable from subprocesses:
|
|||||||
### External interpreter resolution (`bins.py`)
|
### External interpreter resolution (`bins.py`)
|
||||||
`src/testium/interpreter/utils/bins.py` — single source of truth for the paths to the external Python and Lua interpreters used by subprocesses.
|
`src/testium/interpreter/utils/bins.py` — single source of truth for the paths to the external Python and Lua interpreters used by subprocesses.
|
||||||
|
|
||||||
- `python_bin()` / `lua_bin()` : resolve and cache. The cache is keyed by `(name, override)` so that a later change to `gd[python_bin]` (typically when a `param.yaml` sets the key) triggers a re-resolution on the next lookup instead of returning the stale auto-discovered path. Falls back to discovery on PATH (candidates: `python3`/`python` and `lua`/`lua5.5`/`lua5.4`/`lua5.3`/`lua5.2`/`lua5.1`).
|
- `python_bin()` / `lua_bin()` : resolve once, cache in memory. User can override via the `python_bin` / `lua_bin` global dict keys (typically populated from the YAML config). Falls back to discovery on PATH (candidates: `python3`/`python` and `lua`/`lua5.5`/`lua5.4`/`lua5.3`/`lua5.2`/`lua5.1`).
|
||||||
- `ensure(*names)` : called by `TestSet._validate_runtime_deps()` at test load. Always requires `python` (the eval engine always runs); requires `lua` only if a `lua_func` item is in the tree. Fails fast with a clear error citing tried candidates and override key.
|
- `ensure(*names)` : called by `TestSet._validate_runtime_deps()` at test load. Always requires `python` (the eval engine always runs); requires `lua` only if a `lua_func` item is in the tree. Fails fast with a clear error citing tried candidates and override key.
|
||||||
|
|
||||||
Engines (`PyProcessBase`, `LuaProcessBase`, `EvalExecEngine`) call `bins.python_bin()`/`bins.lua_bin()` themselves — call sites never pass an explicit binary path.
|
Engines (`PyProcessBase`, `LuaProcessBase`, `EvalExecEngine`) call `bins.python_bin()`/`bins.lua_bin()` themselves — call sites never pass an explicit binary path.
|
||||||
|
|
||||||
#### Override-timing contract (`apply_overrides`)
|
|
||||||
`bins.python_bin()` is called for the **first** time inside `eval_process_init()` (the long-lived inline-`<| … |>` subprocess), which happens **before** the YAML param files are loaded. To make `-d python_bin=…` and the GUI `python_bin` preference take effect for `eval_proc` itself, `process.py:run()` applies them to gd **before** `eval_process_init()` via the `apply_overrides()` helper extracted from `update_global()`. The post-load `update_global()` call then re-applies the same overrides (after `prepare_global()` clears gd), keeping the gd value in sync with the cached resolution.
|
|
||||||
|
|
||||||
| Override source | `eval_proc` | `py_func` / `cycle` / `post_exec` |
|
|
||||||
|---|---|---|
|
|
||||||
| `-d python_bin=…` (CLI) | ✅ | ✅ |
|
|
||||||
| GUI `python_bin` preference | ✅ | ✅ |
|
|
||||||
| `python_bin: …` in `param.yaml` | ❌ (eval_proc already started) | ✅ (cache re-resolves on key change) |
|
|
||||||
|
|
||||||
## Key files
|
## Key files
|
||||||
|
|
||||||
| Path | Role |
|
| Path | Role |
|
||||||
@@ -270,18 +261,12 @@ Both Flatpak and AppImage export `TESTIUM_VERSION` from a launcher (Flatpak: lau
|
|||||||
- `unittest` item: renamed from `unittest_file`.
|
- `unittest` item: renamed from `unittest_file`.
|
||||||
- GUI test tree: check and fold state preserved across same-file reloads.
|
- GUI test tree: check and fold state preserved across same-file reloads.
|
||||||
- Licence: EUPL-1.2.
|
- Licence: EUPL-1.2.
|
||||||
- Interpreter override timing: `apply_overrides()` extracted from `update_global()` and called by `process.py:run()` before `eval_process_init()`, so `-d python_bin=…` / GUI prefs reach `bins.python_bin()` on its first lookup. `bins._resolve()` cache is now keyed by `(name, override)` so later `param.yaml` changes are picked up by subsequently constructed engines.
|
|
||||||
|
|
||||||
## Validation tests
|
## Validation tests
|
||||||
Located in `test/validation/`. Two entry points:
|
Located in `test/validation/`. Run with `-b` flag:
|
||||||
```
|
```
|
||||||
./test/validation/run.sh # wrapper — uses a dedicated venv (see below)
|
./run.sh -b -- test/validation/main.tum
|
||||||
./run.sh -b -- test/validation/main.tum # direct — testium's own python is used for test execution
|
|
||||||
```
|
```
|
||||||
The `run.sh` / `run.bat` wrappers create a dedicated Python venv at `${TMPDIR:-/tmp}/testium-validation-venv` (Linux) or `%TEMP%\testium-validation-venv` (Windows), with `--system-site-packages` + `pip install junit-xml`, and run the suite with `-d python_bin=…` so every test-execution subprocess (eval_proc, py_func, cycle, post_exec) runs inside the venv. testium itself keeps running in the project's own environment. `clean` as the first argument recreates the venv.
|
|
||||||
|
|
||||||
The `venv` item (`test/validation/items/venv/`) asserts that the override actually took effect: `python_bin` is set, `sys.executable` matches it, `sys.prefix == dirname(dirname(python_bin))`, and `sys.prefix != sys.base_prefix` (the last marker catches the case where `python_bin` happens to be a system interpreter, which path-equality alone would miss because the venv's `bin/python3` is a symlink to the host). Both `eval_proc` (inline `<| … |>`) and `py_func` paths are exercised.
|
|
||||||
|
|
||||||
Parallel item tests: `test/validation/items/parallel/test.tum`
|
Parallel item tests: `test/validation/items/parallel/test.tum`
|
||||||
|
|
||||||
## Dependencies
|
## Dependencies
|
||||||
|
|||||||
117
build_all.sh
117
build_all.sh
@@ -8,9 +8,6 @@
|
|||||||
# release_note.txt is copied to dist/ up front (with a warning if it has no
|
# release_note.txt is copied to dist/ up front (with a warning if it has no
|
||||||
# entry for the current version).
|
# entry for the current version).
|
||||||
#
|
#
|
||||||
# By default, a step is skipped if its artifact already exists in dist/.
|
|
||||||
# Pass --clean to remove existing dist/ artifacts and rebuild everything.
|
|
||||||
#
|
|
||||||
# All artifacts are collected (copied) under <repo>/dist/. Original outputs in
|
# All artifacts are collected (copied) under <repo>/dist/. Original outputs in
|
||||||
# src/dist/, package/*/dist/, doc/manual/ are left in place. Wheel and AppImage
|
# src/dist/, package/*/dist/, doc/manual/ are left in place. Wheel and AppImage
|
||||||
# keep their original names (which already contain the version); manual,
|
# keep their original names (which already contain the version); manual,
|
||||||
@@ -18,35 +15,17 @@
|
|||||||
#
|
#
|
||||||
# Re-uses scripts/build_env.sh and scripts/set_env.sh — the same pair invoked
|
# Re-uses scripts/build_env.sh and scripts/set_env.sh — the same pair invoked
|
||||||
# by run.sh — so the venv at test/tmp/.venv stays the single source of Python
|
# by run.sh — so the venv at test/tmp/.venv stays the single source of Python
|
||||||
# dependencies. `build`, `pyinstaller`, `sphinx` and `linuxdoc` are installed
|
# dependencies. `build` and `pyinstaller` are installed into that venv on
|
||||||
# into that venv on demand if not already there. Flatpak and AppImage build in
|
# demand if not already there. Flatpak and AppImage build in their own
|
||||||
# their own container/sandbox; their build.sh scripts have their own toolchain
|
# container/sandbox; their build.sh scripts have their own toolchain checks.
|
||||||
# checks.
|
|
||||||
|
|
||||||
set -e
|
set -e
|
||||||
|
|
||||||
CLEAN=0
|
|
||||||
for arg in "$@"; do
|
|
||||||
case "$arg" in
|
|
||||||
--clean|-c) CLEAN=1 ;;
|
|
||||||
*) echo "Unknown option: $arg" >&2; exit 1 ;;
|
|
||||||
esac
|
|
||||||
done
|
|
||||||
|
|
||||||
SCRIPT_DIR=$(realpath "$(dirname "$0")")
|
SCRIPT_DIR=$(realpath "$(dirname "$0")")
|
||||||
VERSION=$(cat "$SCRIPT_DIR/src/VERSION")
|
VERSION=$(cat "$SCRIPT_DIR/src/VERSION")
|
||||||
DIST_DIR="$SCRIPT_DIR/dist"
|
DIST_DIR="$SCRIPT_DIR/dist"
|
||||||
mkdir -p "$DIST_DIR"
|
mkdir -p "$DIST_DIR"
|
||||||
|
|
||||||
if [ "$CLEAN" -eq 1 ]; then
|
|
||||||
echo "-- clean: removing existing dist artifacts for version $VERSION"
|
|
||||||
rm -f "$DIST_DIR/testium-manual-${VERSION}.pdf"
|
|
||||||
rm -f "$DIST_DIR"/testium-${VERSION}-*.whl
|
|
||||||
rm -f "$DIST_DIR/testium-${VERSION}"
|
|
||||||
rm -f "$DIST_DIR/testium-${VERSION}.flatpak"
|
|
||||||
rm -f "$DIST_DIR"/Testium-${VERSION}-*.AppImage
|
|
||||||
fi
|
|
||||||
|
|
||||||
# Release note: copy it to dist/ and warn (but don't fail) if it has no entry
|
# Release note: copy it to dist/ and warn (but don't fail) if it has no entry
|
||||||
# for the current version.
|
# for the current version.
|
||||||
RELEASE_NOTE_SRC="$SCRIPT_DIR/release_note.txt"
|
RELEASE_NOTE_SRC="$SCRIPT_DIR/release_note.txt"
|
||||||
@@ -63,6 +42,9 @@ export REQ_PATH="$SCRIPT_DIR/src/requirements.txt"
|
|||||||
bash "$SCRIPT_DIR/scripts/build_env.sh"
|
bash "$SCRIPT_DIR/scripts/build_env.sh"
|
||||||
source "$SCRIPT_DIR/scripts/set_env.sh"
|
source "$SCRIPT_DIR/scripts/set_env.sh"
|
||||||
|
|
||||||
|
# Ensure wheel/PyInstaller toolchains are present in the venv.
|
||||||
|
python -m pip install --quiet --upgrade build pyinstaller
|
||||||
|
|
||||||
step() {
|
step() {
|
||||||
echo
|
echo
|
||||||
echo "================================================================"
|
echo "================================================================"
|
||||||
@@ -70,90 +52,51 @@ step() {
|
|||||||
echo "================================================================"
|
echo "================================================================"
|
||||||
}
|
}
|
||||||
|
|
||||||
skip() { echo " (already built — skipping)"; }
|
|
||||||
|
|
||||||
# 1. Manual PDF
|
# 1. Manual PDF
|
||||||
MANUAL="$DIST_DIR/testium-manual-${VERSION}.pdf"
|
|
||||||
step "1/5 Manual PDF (version $VERSION)"
|
step "1/5 Manual PDF (version $VERSION)"
|
||||||
if [ ! -f "$MANUAL" ]; then
|
bash "$SCRIPT_DIR/doc/manual/sphinx/build_doc.sh"
|
||||||
python -m pip install --quiet --upgrade sphinx linuxdoc
|
MANUAL_SRC="$SCRIPT_DIR/doc/manual/testium_manual.pdf"
|
||||||
bash "$SCRIPT_DIR/doc/manual/sphinx/build_doc.sh"
|
MANUAL="$DIST_DIR/testium-manual-${VERSION}.pdf"
|
||||||
cp -f "$SCRIPT_DIR/doc/manual/testium_manual.pdf" "$MANUAL"
|
cp -f "$MANUAL_SRC" "$MANUAL"
|
||||||
else
|
|
||||||
skip
|
|
||||||
fi
|
|
||||||
|
|
||||||
# 2. Wheel — PEP 427 name kept (already contains version)
|
# 2. Wheel — PEP 427 name kept (already contains version)
|
||||||
step "2/5 Wheel (version $VERSION)"
|
step "2/5 Wheel (version $VERSION)"
|
||||||
WHEEL=$(ls -1t "$DIST_DIR"/testium-${VERSION}-*.whl 2>/dev/null | head -1)
|
(
|
||||||
if [ -z "$WHEEL" ]; then
|
|
||||||
python -m pip install --quiet --upgrade build
|
|
||||||
(
|
|
||||||
cd "$SCRIPT_DIR/src"
|
cd "$SCRIPT_DIR/src"
|
||||||
rm -rf dist build *.egg-info
|
rm -rf dist build *.egg-info
|
||||||
python -m build --wheel
|
python -m build --wheel
|
||||||
)
|
)
|
||||||
WHEEL_SRC=$(ls -1t "$SCRIPT_DIR/src/dist"/*.whl | head -1)
|
WHEEL_SRC=$(ls -1t "$SCRIPT_DIR/src/dist"/*.whl | head -1)
|
||||||
WHEEL="$DIST_DIR/$(basename "$WHEEL_SRC")"
|
WHEEL="$DIST_DIR/$(basename "$WHEEL_SRC")"
|
||||||
cp -f "$WHEEL_SRC" "$WHEEL"
|
cp -f "$WHEEL_SRC" "$WHEEL"
|
||||||
else
|
|
||||||
skip
|
|
||||||
fi
|
|
||||||
|
|
||||||
# 3. PyInstaller binary
|
# 3. PyInstaller binary
|
||||||
PYI_BIN="$DIST_DIR/testium-${VERSION}"
|
|
||||||
step "3/5 PyInstaller binary (version $VERSION)"
|
step "3/5 PyInstaller binary (version $VERSION)"
|
||||||
if [ ! -f "$PYI_BIN" ]; then
|
bash "$SCRIPT_DIR/package/pyinstaller/build.sh"
|
||||||
python -m pip install --quiet --upgrade pyinstaller
|
PYI_SRC="$SCRIPT_DIR/package/pyinstaller/dist/testium"
|
||||||
bash "$SCRIPT_DIR/package/pyinstaller/build.sh"
|
PYI_BIN="$DIST_DIR/testium-${VERSION}"
|
||||||
cp -f "$SCRIPT_DIR/package/pyinstaller/dist/testium" "$PYI_BIN"
|
cp -f "$PYI_SRC" "$PYI_BIN"
|
||||||
else
|
|
||||||
skip
|
|
||||||
fi
|
|
||||||
|
|
||||||
# 4. Flatpak bundle
|
# 4. Flatpak bundle
|
||||||
FLATPAK_BUNDLE="$DIST_DIR/testium-${VERSION}.flatpak"
|
|
||||||
step "4/5 Flatpak bundle (version $VERSION)"
|
step "4/5 Flatpak bundle (version $VERSION)"
|
||||||
if [ ! -f "$FLATPAK_BUNDLE" ]; then
|
(
|
||||||
FLATPAK_DEPS=(
|
|
||||||
"org.kde.Platform//6.10"
|
|
||||||
"org.kde.Sdk//6.10"
|
|
||||||
"io.qt.PySide.BaseApp//6.10"
|
|
||||||
)
|
|
||||||
if ! flatpak remotes --user | grep -q "^flathub"; then
|
|
||||||
echo " Adding Flathub remote"
|
|
||||||
flatpak remote-add --user --if-not-exists flathub https://dl.flathub.org/repo/flathub.flatpakrepo
|
|
||||||
fi
|
|
||||||
for dep in "${FLATPAK_DEPS[@]}"; do
|
|
||||||
if ! flatpak info --user "$dep" &>/dev/null && ! flatpak info --system "$dep" &>/dev/null; then
|
|
||||||
echo " Installing Flatpak dependency: $dep"
|
|
||||||
flatpak install --user --noninteractive flathub "$dep"
|
|
||||||
fi
|
|
||||||
done
|
|
||||||
(
|
|
||||||
cd "$SCRIPT_DIR/package/flatpak"
|
cd "$SCRIPT_DIR/package/flatpak"
|
||||||
bash build.sh
|
bash build.sh
|
||||||
)
|
)
|
||||||
cp -f "$SCRIPT_DIR/package/flatpak/testium.flatpak" "$FLATPAK_BUNDLE"
|
FLATPAK_SRC="$SCRIPT_DIR/package/flatpak/testium.flatpak"
|
||||||
else
|
FLATPAK_BUNDLE="$DIST_DIR/testium-${VERSION}.flatpak"
|
||||||
skip
|
cp -f "$FLATPAK_SRC" "$FLATPAK_BUNDLE"
|
||||||
fi
|
|
||||||
|
|
||||||
# 5. AppImage
|
# 5. AppImage
|
||||||
step "5/5 AppImage (version $VERSION)"
|
step "5/5 AppImage (version $VERSION)"
|
||||||
APPIMAGE=$(ls -1t "$DIST_DIR"/Testium-${VERSION}-*.AppImage 2>/dev/null | head -1)
|
(
|
||||||
if [ -z "$APPIMAGE" ]; then
|
|
||||||
(
|
|
||||||
cd "$SCRIPT_DIR/package/appimage"
|
cd "$SCRIPT_DIR/package/appimage"
|
||||||
bash build.sh
|
bash build.sh
|
||||||
)
|
)
|
||||||
APPIMAGE_SRC=$(ls -1t "$SCRIPT_DIR/package/appimage"/*.AppImage 2>/dev/null | head -1)
|
APPIMAGE_SRC=$(ls -1t "$SCRIPT_DIR/package/appimage"/*.AppImage 2>/dev/null | head -1)
|
||||||
APPIMAGE="$DIST_DIR/$(basename "$APPIMAGE_SRC")"
|
APPIMAGE="$DIST_DIR/$(basename "$APPIMAGE_SRC")"
|
||||||
cp -f "$APPIMAGE_SRC" "$APPIMAGE"
|
cp -f "$APPIMAGE_SRC" "$APPIMAGE"
|
||||||
chmod +x "$APPIMAGE"
|
chmod +x "$APPIMAGE"
|
||||||
else
|
|
||||||
skip
|
|
||||||
fi
|
|
||||||
|
|
||||||
step "All packages built"
|
step "All packages built"
|
||||||
printf " manual : %s\n" "$MANUAL"
|
printf " manual : %s\n" "$MANUAL"
|
||||||
|
|||||||
Binary file not shown.
@@ -16,11 +16,6 @@ finish-args:
|
|||||||
- --filesystem=home
|
- --filesystem=home
|
||||||
- --filesystem=/tmp
|
- --filesystem=/tmp
|
||||||
- --filesystem=host-os
|
- --filesystem=host-os
|
||||||
# Allow flatpak-spawn --host to launch host binaries (Python, Lua, git…)
|
|
||||||
# outside the sandbox. Required because the sandbox glibc/ABI is
|
|
||||||
# incompatible with arbitrary host shared libraries — we route py_func and
|
|
||||||
# lua_func through the host instead.
|
|
||||||
- --talk-name=org.freedesktop.Flatpak
|
|
||||||
|
|
||||||
build-options:
|
build-options:
|
||||||
build-args:
|
build-args:
|
||||||
|
|||||||
@@ -1,38 +1,3 @@
|
|||||||
version 0.2
|
|
||||||
==============
|
|
||||||
- Test items: each item type now declares its accepted parameters
|
|
||||||
(``PARAMS = ParamSet(...)``). Typos in a ``.tum`` are surfaced as a
|
|
||||||
WARN listing the accepted names instead of being silently ignored;
|
|
||||||
missing required parameters error out at load time with the source
|
|
||||||
``.tum`` file as context. No change to valid existing tests.
|
|
||||||
|
|
||||||
version 0.1.3
|
|
||||||
==============
|
|
||||||
- Stop interrupts engaged blocking steps (console, py_func, lua_func,
|
|
||||||
json_rpc, sleep) within ~200 ms instead of waiting for the step
|
|
||||||
to finish.
|
|
||||||
- GUI Start / Stop / Pause flow simplified.
|
|
||||||
- lua_func: a function returning nil is no longer reported as a failure.
|
|
||||||
- ``-d python_bin=...`` and the GUI ``python_bin`` preference now reach
|
|
||||||
the eval subprocess (used to be silently ignored). ``param.yaml`` can
|
|
||||||
also override ``python_bin`` for py_func / cycle / post_exec.
|
|
||||||
- Validation suite: ``test/validation/run.sh`` (and ``run.bat``)
|
|
||||||
runs the suite inside a dedicated venv in the system temp dir.
|
|
||||||
- build_all.sh: ``release_note.txt`` and the user manual copied into
|
|
||||||
``dist/``; warning if the file has no entry for the version being built.
|
|
||||||
- Flatpak: every GUI file/directory dialog (open test, save report, log
|
|
||||||
path, default report/log dirs, python/lua interpreter pickers) now
|
|
||||||
bypasses the XDG document portal — the v0.1.2 fix was only on the
|
|
||||||
"open test" dialog.
|
|
||||||
- Flatpak: py_func / lua_func / run sub-instance now execute on the host
|
|
||||||
via flatpak-spawn, lifting the previous glibc/ABI incompatibility that
|
|
||||||
prevented user-configured host Python or Lua interpreters from being
|
|
||||||
reached from the sandbox.
|
|
||||||
- Validation suite: single entry point with ``--mode source|wheel|
|
|
||||||
pyinstaller|flatpak|appimage`` to validate any packaging channel
|
|
||||||
against the same item set; reports are stamped per mode.
|
|
||||||
- GUI: the "Run tum" test item now uses the testium logo.
|
|
||||||
|
|
||||||
version 0.1.2
|
version 0.1.2
|
||||||
==============
|
==============
|
||||||
- Flatpak: opening a test from the GUI now correctly finds its companion
|
- Flatpak: opening a test from the GUI now correctly finds its companion
|
||||||
|
|||||||
73
schema/test.tum
Normal file
73
schema/test.tum
Normal file
@@ -0,0 +1,73 @@
|
|||||||
|
config_file:
|
||||||
|
- premier
|
||||||
|
- saluT
|
||||||
|
|
||||||
|
main:
|
||||||
|
name: Main file
|
||||||
|
steps:
|
||||||
|
- group:
|
||||||
|
name: Test
|
||||||
|
doc: Une peitite documentation
|
||||||
|
no_fail: true
|
||||||
|
steps:
|
||||||
|
- let:
|
||||||
|
values:
|
||||||
|
- my_var: <| ${salut} |>
|
||||||
|
- check:
|
||||||
|
values:
|
||||||
|
- <| ${salut} |>
|
||||||
|
|
||||||
|
- dialog_message:
|
||||||
|
question: c'est quoi?
|
||||||
|
|
||||||
|
- lua_func:
|
||||||
|
file: c'est quoi?
|
||||||
|
func_name: c'est quoi?
|
||||||
|
|
||||||
|
- console:
|
||||||
|
console_name: cons_1
|
||||||
|
steps:
|
||||||
|
- open:
|
||||||
|
protocol: telnet
|
||||||
|
terminal_path: ijfeifj
|
||||||
|
- read_until : {expected: "tutu", timeout: -4.5, mute: true}
|
||||||
|
- write: something
|
||||||
|
- writeln: tutu
|
||||||
|
- close :
|
||||||
|
|
||||||
|
- json_rpc:
|
||||||
|
name: JSONRPC console Query
|
||||||
|
doc: JSONRPC console Query not waiting (only send)
|
||||||
|
console:
|
||||||
|
name : jsonrpc_server
|
||||||
|
prompt: "@@>"
|
||||||
|
timeout: 1
|
||||||
|
version: "2.0"
|
||||||
|
steps:
|
||||||
|
- query:
|
||||||
|
method: echo
|
||||||
|
params:
|
||||||
|
- Hello world
|
||||||
|
- [0, 1, 2, 3]
|
||||||
|
id: 3095372
|
||||||
|
no_wait: true
|
||||||
|
- json_rpc:
|
||||||
|
name: JSONRPC console Reception
|
||||||
|
doc: JSONRPC console reception of the previous request
|
||||||
|
console: {name : jsonrpc_server}
|
||||||
|
timeout: 1
|
||||||
|
steps:
|
||||||
|
- receive:
|
||||||
|
id: 3095372
|
||||||
|
timeout: 0.5
|
||||||
|
|
||||||
|
report:
|
||||||
|
enabled: true
|
||||||
|
log_stored: true
|
||||||
|
export:
|
||||||
|
junit:
|
||||||
|
path: $(validation_report_path)
|
||||||
|
file_name: $(validation_report_file).junit
|
||||||
|
sqlite:
|
||||||
|
file_name: $(validation_report_file).sqlite
|
||||||
|
path: $(validation_report_path)
|
||||||
96
schema/test_schema/check.tum
Normal file
96
schema/test_schema/check.tum
Normal file
@@ -0,0 +1,96 @@
|
|||||||
|
config_file:
|
||||||
|
- param.yaml
|
||||||
|
- items/check/param.yaml
|
||||||
|
main:
|
||||||
|
name: Testium validation suite
|
||||||
|
steps:
|
||||||
|
- group:
|
||||||
|
name: Test preparation
|
||||||
|
steps:
|
||||||
|
- let:
|
||||||
|
condition: <| "$(os)" == "Linux" |>
|
||||||
|
name: Set test variables for Linux
|
||||||
|
values:
|
||||||
|
- terminal_prompt: $(linux_prompt)
|
||||||
|
- psep: /
|
||||||
|
- let:
|
||||||
|
condition: <| "$(os)" == "Windows" |>
|
||||||
|
name: Set test variables for Windows
|
||||||
|
values:
|
||||||
|
- terminal_prompt: $(windows_prompt)
|
||||||
|
- psep: \
|
||||||
|
- group:
|
||||||
|
name: Group of tests
|
||||||
|
steps:
|
||||||
|
- let:
|
||||||
|
name: check test constants
|
||||||
|
values:
|
||||||
|
- test: check
|
||||||
|
- test_path: items/$(test)
|
||||||
|
- group:
|
||||||
|
name: check test
|
||||||
|
steps:
|
||||||
|
- sequence:
|
||||||
|
steps:
|
||||||
|
- py_func:
|
||||||
|
file: $(test_path)$(psep)check.py
|
||||||
|
func_name: echo
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Dummy_int
|
||||||
|
param:
|
||||||
|
- 2
|
||||||
|
- py_func:
|
||||||
|
file: $(test_path)$(psep)check.py
|
||||||
|
func_name: echo
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Dummy_str
|
||||||
|
param:
|
||||||
|
- my taylor is rich
|
||||||
|
- check:
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Check condition on existing variable (PASS)
|
||||||
|
values:
|
||||||
|
- <| $(pfn_Dummy_int) > 1 |>
|
||||||
|
- check:
|
||||||
|
key: $(test)_FAIL
|
||||||
|
name: Check condition on existing variable (FAIL)
|
||||||
|
values:
|
||||||
|
- <| "tailor" in "$(pfn_Dummy_str)" |>
|
||||||
|
filename: /home/renish/workspace/testium/code/test/validation/items/check/test.tum
|
||||||
|
- sequence:
|
||||||
|
steps:
|
||||||
|
- report:
|
||||||
|
export:
|
||||||
|
- text:
|
||||||
|
key: $(test)_PASS
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_PASS.txt
|
||||||
|
- html:
|
||||||
|
key: $(test)_PASS
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_PASS.html
|
||||||
|
- junit:
|
||||||
|
key: $(test)_PASS
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_PASS.junit
|
||||||
|
name: Expected PASS $(test) test
|
||||||
|
- report:
|
||||||
|
export:
|
||||||
|
- text:
|
||||||
|
key: $(test)_FAIL
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_FAIL.txt
|
||||||
|
- html:
|
||||||
|
key: $(test)_FAIL
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_FAIL.html
|
||||||
|
- junit:
|
||||||
|
key: $(test)_FAIL
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_FAIL.junit
|
||||||
|
name: Expected FAIL $(test) test
|
||||||
|
filename: /home/renish/workspace/testium/code/test/validation/items/report.tum
|
||||||
|
report:
|
||||||
|
enabled: true
|
||||||
|
export:
|
||||||
|
junit:
|
||||||
|
file_name: $(validation_report_file).junit
|
||||||
|
path: $(validation_report_path)
|
||||||
|
sqlite:
|
||||||
|
file_name: $(validation_report_file).sqlite
|
||||||
|
path: $(validation_report_path)
|
||||||
|
log_stored: true
|
||||||
453
schema/test_schema/common.tum
Normal file
453
schema/test_schema/common.tum
Normal file
@@ -0,0 +1,453 @@
|
|||||||
|
config_file:
|
||||||
|
- param.yaml
|
||||||
|
- items/common/param.yaml
|
||||||
|
main:
|
||||||
|
name: Testium validation suite
|
||||||
|
steps:
|
||||||
|
- group:
|
||||||
|
name: Test preparation
|
||||||
|
steps:
|
||||||
|
- let:
|
||||||
|
condition: <| "$(os)" == "Linux" |>
|
||||||
|
name: Set test variables for Linux
|
||||||
|
values:
|
||||||
|
- terminal_prompt: $(linux_prompt)
|
||||||
|
- psep: /
|
||||||
|
- let:
|
||||||
|
condition: <| "$(os)" == "Windows" |>
|
||||||
|
name: Set test variables for Windows
|
||||||
|
values:
|
||||||
|
- terminal_prompt: $(windows_prompt)
|
||||||
|
- psep: \
|
||||||
|
- group:
|
||||||
|
name: Group of tests
|
||||||
|
steps:
|
||||||
|
- let:
|
||||||
|
name: common test constants
|
||||||
|
values:
|
||||||
|
test: common
|
||||||
|
test_path: items/$(test)
|
||||||
|
- group:
|
||||||
|
name: common test
|
||||||
|
steps:
|
||||||
|
- sequence:
|
||||||
|
steps:
|
||||||
|
- group:
|
||||||
|
name: Results
|
||||||
|
steps:
|
||||||
|
- sequence:
|
||||||
|
steps:
|
||||||
|
- group:
|
||||||
|
name: Expected Result
|
||||||
|
steps:
|
||||||
|
- py_func:
|
||||||
|
expected_result: true
|
||||||
|
file: $(test_path)$(psep)results$(psep)results.py
|
||||||
|
func_name: echo
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Return True expect True
|
||||||
|
param:
|
||||||
|
- true
|
||||||
|
- py_func:
|
||||||
|
expected_result: false
|
||||||
|
file: $(test_path)$(psep)results$(psep)results.py
|
||||||
|
func_name: echo
|
||||||
|
key: $(test)_FAIL
|
||||||
|
name: Return True expect False (must
|
||||||
|
fail)
|
||||||
|
param:
|
||||||
|
- true
|
||||||
|
- py_func:
|
||||||
|
expected_result: None
|
||||||
|
file: $(test_path)$(psep)results$(psep)results.py
|
||||||
|
func_name: return_none
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Return None expect None
|
||||||
|
- py_func:
|
||||||
|
expected_result: PASS
|
||||||
|
file: $(test_path)$(psep)results$(psep)results.py
|
||||||
|
func_name: return_none
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Return None expect PASS
|
||||||
|
- py_func:
|
||||||
|
expected_result: 14
|
||||||
|
file: $(test_path)$(psep)results$(psep)results.py
|
||||||
|
func_name: return_none
|
||||||
|
key: $(test)_FAIL
|
||||||
|
name: Return None expect 14 (must fail)
|
||||||
|
- group:
|
||||||
|
name: Expected Result Last test result
|
||||||
|
steps:
|
||||||
|
- py_func:
|
||||||
|
file: $(test_path)$(psep)results$(psep)results.py
|
||||||
|
func_name: echo
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: result is 28
|
||||||
|
param:
|
||||||
|
- 28
|
||||||
|
- py_func:
|
||||||
|
expected_result: $(last_step_result)
|
||||||
|
file: $(test_path)$(psep)results$(psep)results.py
|
||||||
|
func_name: echo
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: check that the last test result
|
||||||
|
is 28
|
||||||
|
param:
|
||||||
|
- 28
|
||||||
|
- group:
|
||||||
|
name: Expected result Failure raised issue
|
||||||
|
steps:
|
||||||
|
- py_func:
|
||||||
|
file: $(test_path)$(psep)results$(psep)results.py
|
||||||
|
func_name: raise_issue
|
||||||
|
key: $(test)_FAIL
|
||||||
|
name: Raise an issue (must fail)
|
||||||
|
param:
|
||||||
|
- $(str_example)
|
||||||
|
- py_func:
|
||||||
|
expected_result: FAIL
|
||||||
|
file: $(test_path)$(psep)results$(psep)results.py
|
||||||
|
func_name: raise_issue
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Raise an issue and expected the
|
||||||
|
test to be FAIL
|
||||||
|
param:
|
||||||
|
- $(str_example)
|
||||||
|
- py_func:
|
||||||
|
expected_result: FAIL
|
||||||
|
file: $(test_path)$(psep)results$(psep)results.py
|
||||||
|
func_name: echo
|
||||||
|
key: $(test)_FAIL
|
||||||
|
name: Return a String expect a FAILURE
|
||||||
|
(must fail)
|
||||||
|
param:
|
||||||
|
- $(str_example)
|
||||||
|
- group:
|
||||||
|
name: process result
|
||||||
|
steps:
|
||||||
|
- py_func:
|
||||||
|
file: $(test_path)$(psep)results$(psep)results.py
|
||||||
|
func_name: echo
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Process result equal String
|
||||||
|
param:
|
||||||
|
- $(str_example)
|
||||||
|
process_result: '''$(str_example)'' ==
|
||||||
|
''$(result)'''
|
||||||
|
- py_func:
|
||||||
|
expected_result: true
|
||||||
|
file: $(test_path)$(psep)results$(psep)results.py
|
||||||
|
func_name: echo
|
||||||
|
key: $(test)_FAIL
|
||||||
|
name: Process result string in the result
|
||||||
|
(must fail)
|
||||||
|
param:
|
||||||
|
- $(str_example)
|
||||||
|
process_result: '''44'' in ''$(result)'''
|
||||||
|
- py_func:
|
||||||
|
file: $(test_path)$(psep)results$(psep)results.py
|
||||||
|
func_name: echo
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Save the result in a global variable
|
||||||
|
param:
|
||||||
|
- 44
|
||||||
|
store_result: process_result_value
|
||||||
|
- py_func:
|
||||||
|
expected_result: $(process_result_value)
|
||||||
|
file: $(test_path)$(psep)results$(psep)results.py
|
||||||
|
func_name: echo
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Check the saved global variable
|
||||||
|
param:
|
||||||
|
- 44
|
||||||
|
- py_func:
|
||||||
|
file: $(test_path)$(psep)results$(psep)results.py
|
||||||
|
func_name: echo
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: store_result with process_result
|
||||||
|
param:
|
||||||
|
- $(str_example)
|
||||||
|
process_result: '''$(result)''.upper()'
|
||||||
|
store_result: upper_str_example
|
||||||
|
- py_func:
|
||||||
|
expected_result: $(upper_str_example)
|
||||||
|
file: $(test_path)$(psep)results$(psep)results.py
|
||||||
|
func_name: echo
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Check store_result with process_result
|
||||||
|
param:
|
||||||
|
- $(str_example)
|
||||||
|
process_result: '''$(result)''.upper()'
|
||||||
|
- let:
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: "store_result on let item (None\
|
||||||
|
\ value \u2192 stores PASS)"
|
||||||
|
store_result: let_store_result
|
||||||
|
values:
|
||||||
|
- dummy: 0
|
||||||
|
- py_func:
|
||||||
|
expected_result: $(let_store_result)
|
||||||
|
file: $(test_path)$(psep)results$(psep)results.py
|
||||||
|
func_name: echo
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Check store_result on let stores
|
||||||
|
PASS
|
||||||
|
param:
|
||||||
|
- PASS
|
||||||
|
- py_func:
|
||||||
|
expected_result: FAIL
|
||||||
|
file: $(test_path)$(psep)results$(psep)results.py
|
||||||
|
func_name: return_none
|
||||||
|
key: $(test)_FAIL
|
||||||
|
name: "store_result on failing test (None\
|
||||||
|
\ value \u2192 stores FAIL)"
|
||||||
|
store_result: none_fail_store_result
|
||||||
|
- py_func:
|
||||||
|
expected_result: $(none_fail_store_result)
|
||||||
|
file: $(test_path)$(psep)results$(psep)results.py
|
||||||
|
func_name: echo
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Check store_result on failing test
|
||||||
|
stores FAIL
|
||||||
|
param:
|
||||||
|
- FAIL
|
||||||
|
- py_func:
|
||||||
|
expected_result: FAIL
|
||||||
|
file: $(test_path)$(psep)results$(psep)results.py
|
||||||
|
func_name: return_none
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: "store_result with no_fail (None\
|
||||||
|
\ value \u2192 stores real FAIL, not\
|
||||||
|
\ forced PASS)"
|
||||||
|
no_fail: true
|
||||||
|
store_result: none_nofail_store_result
|
||||||
|
- py_func:
|
||||||
|
expected_result: $(none_nofail_store_result)
|
||||||
|
file: $(test_path)$(psep)results$(psep)results.py
|
||||||
|
func_name: echo
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Check store_result with no_fail
|
||||||
|
stores real FAIL
|
||||||
|
param:
|
||||||
|
- FAIL
|
||||||
|
- py_func:
|
||||||
|
file: $(test_path)$(psep)results$(psep)results.py
|
||||||
|
func_name: return_none
|
||||||
|
key: $(test)_FAIL
|
||||||
|
name: Process result when result is None
|
||||||
|
(must fail)
|
||||||
|
process_result: $(result) is None
|
||||||
|
- group:
|
||||||
|
name: no_fail result
|
||||||
|
steps:
|
||||||
|
- py_func:
|
||||||
|
expected_result: false
|
||||||
|
file: $(test_path)$(psep)results$(psep)results.py
|
||||||
|
func_name: echo
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Return True expect False but no_fail=True
|
||||||
|
no_fail: true
|
||||||
|
param:
|
||||||
|
- true
|
||||||
|
- py_func:
|
||||||
|
expected_result: false
|
||||||
|
file: $(test_path)$(psep)results$(psep)results.py
|
||||||
|
func_name: echo
|
||||||
|
key: $(test)_FAIL
|
||||||
|
name: Return True expect False but no_fail=False
|
||||||
|
(must fail)
|
||||||
|
no_fail: false
|
||||||
|
param:
|
||||||
|
- true
|
||||||
|
- py_func:
|
||||||
|
expected_result: false
|
||||||
|
file: $(test_path)$(psep)results$(psep)results.py
|
||||||
|
func_name: echo
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Return True expect False but no_fail
|
||||||
|
expansed
|
||||||
|
no_fail: <| bool(0) == False |>
|
||||||
|
param:
|
||||||
|
- true
|
||||||
|
- py_func:
|
||||||
|
expected_result: false
|
||||||
|
file: $(test_path)$(psep)results$(psep)results.py
|
||||||
|
func_name: echo
|
||||||
|
key: $(test)_FAIL
|
||||||
|
name: Return True expect False but no_fail
|
||||||
|
expansed (must fail)
|
||||||
|
no_fail: <| bool(1) == False |>
|
||||||
|
param:
|
||||||
|
- true
|
||||||
|
filename: /home/renish/workspace/testium/code/test/validation/items/common/results/test.tum
|
||||||
|
- group:
|
||||||
|
name: Conditional
|
||||||
|
steps:
|
||||||
|
- sequence:
|
||||||
|
steps:
|
||||||
|
- loop:
|
||||||
|
doc: This loop illustrate the way to exit on
|
||||||
|
a condition.
|
||||||
|
exit_condition:
|
||||||
|
value: <| $(pfn_Echo function) > 3 |>
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Infine loop with conditional exit
|
||||||
|
steps:
|
||||||
|
- sleep:
|
||||||
|
name: small wait
|
||||||
|
timeout: 0.2
|
||||||
|
- py_func:
|
||||||
|
file: $(test_path)$(psep)conditional$(psep)conditional.py
|
||||||
|
func_name: echo
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Echo function
|
||||||
|
param:
|
||||||
|
- $(loop_param)
|
||||||
|
stop_on_failure: false
|
||||||
|
- let:
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: let
|
||||||
|
values:
|
||||||
|
- conditional_exec: <| random.randint(1, 2) |>
|
||||||
|
- console:
|
||||||
|
condition: <| $(conditional_exec) == 1 |>
|
||||||
|
console_name: consname
|
||||||
|
doc: Opening the console
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Console creation
|
||||||
|
steps:
|
||||||
|
- open:
|
||||||
|
protocol: terminal
|
||||||
|
terminal_path: $(test_directory)
|
||||||
|
- writeln: echo "terminal loaded"
|
||||||
|
- console:
|
||||||
|
condition: <| $(conditional_exec) == 1 |>
|
||||||
|
console_name: consname
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Console read_until with timeout
|
||||||
|
steps:
|
||||||
|
- read_until:
|
||||||
|
expected: terminal loaded
|
||||||
|
timeout: 5
|
||||||
|
- console:
|
||||||
|
condition: <| $(conditional_exec) == 1 |>
|
||||||
|
console_name: consname
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Console write
|
||||||
|
steps:
|
||||||
|
- writeln: echo 0
|
||||||
|
- sleep:
|
||||||
|
condition: <| $(conditional_exec) == 1 |>
|
||||||
|
name: sleep item
|
||||||
|
timeout: 1
|
||||||
|
- console:
|
||||||
|
condition: <| $(conditional_exec) == 1 |>
|
||||||
|
console_name: consname
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Console read_until immediate
|
||||||
|
steps:
|
||||||
|
- read_until:
|
||||||
|
expected: '0'
|
||||||
|
timeout: 0
|
||||||
|
- console:
|
||||||
|
condition: <| $(conditional_exec) == 1 |>
|
||||||
|
console_name: consname
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Console read_until immediate (2)
|
||||||
|
steps:
|
||||||
|
- read_until:
|
||||||
|
expected: $(terminal_prompt)
|
||||||
|
timeout: 0
|
||||||
|
- console:
|
||||||
|
condition: <| $(conditional_exec) == 1 |>
|
||||||
|
console_name: consname
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Console closure
|
||||||
|
steps:
|
||||||
|
- close: consname
|
||||||
|
- sleep:
|
||||||
|
condition: <| $(conditional_exec) == 2 |>
|
||||||
|
name: sleep item
|
||||||
|
timeout: 1
|
||||||
|
filename: /home/renish/workspace/testium/code/test/validation/items/common/conditional/test.tum
|
||||||
|
- group:
|
||||||
|
name: Various syntax robustness
|
||||||
|
steps:
|
||||||
|
- sequence:
|
||||||
|
steps:
|
||||||
|
- sleep:
|
||||||
|
key: $(test)_PASS
|
||||||
|
timeout: 0.2
|
||||||
|
- sleep:
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: null
|
||||||
|
timeout: 0.2
|
||||||
|
- sleep:
|
||||||
|
doc: null
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Empty "doc:" declared (must PASS)
|
||||||
|
timeout: 0.2
|
||||||
|
filename: /home/renish/workspace/testium/code/test/validation/items/common/syntax_robustness/test.tum
|
||||||
|
- group:
|
||||||
|
name: Helper lib functions
|
||||||
|
steps:
|
||||||
|
- py_func:
|
||||||
|
file: $(test_path)$(psep)helper_lib.py
|
||||||
|
func_name: check_os
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: OS
|
||||||
|
param:
|
||||||
|
- $(os)
|
||||||
|
- py_func:
|
||||||
|
file: $(test_path)$(psep)helper_lib.py
|
||||||
|
func_name: check_get_main_dir
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: get_main_dir
|
||||||
|
- py_func:
|
||||||
|
file: $(test_path)$(psep)helper_lib.py
|
||||||
|
func_name: check_timestamp_as_sec_conversion
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: timestamp_as_sec conversion
|
||||||
|
- py_func:
|
||||||
|
file: $(test_path)$(psep)helper_lib.py
|
||||||
|
func_name: check_timestamp
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: timestamp and timestamp_as_sec
|
||||||
|
filename: /home/renish/workspace/testium/code/test/validation/items/common/test.tum
|
||||||
|
- sequence:
|
||||||
|
steps:
|
||||||
|
- report:
|
||||||
|
export:
|
||||||
|
- text:
|
||||||
|
key: $(test)_PASS
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_PASS.txt
|
||||||
|
- html:
|
||||||
|
key: $(test)_PASS
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_PASS.html
|
||||||
|
- junit:
|
||||||
|
key: $(test)_PASS
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_PASS.junit
|
||||||
|
name: Expected PASS $(test) test
|
||||||
|
- report:
|
||||||
|
export:
|
||||||
|
- text:
|
||||||
|
key: $(test)_FAIL
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_FAIL.txt
|
||||||
|
- html:
|
||||||
|
key: $(test)_FAIL
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_FAIL.html
|
||||||
|
- junit:
|
||||||
|
key: $(test)_FAIL
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_FAIL.junit
|
||||||
|
name: Expected FAIL $(test) test
|
||||||
|
filename: /home/renish/workspace/testium/code/test/validation/items/report.tum
|
||||||
|
report:
|
||||||
|
enabled: true
|
||||||
|
export:
|
||||||
|
junit:
|
||||||
|
file_name: $(validation_report_file).junit
|
||||||
|
path: $(validation_report_path)
|
||||||
|
sqlite:
|
||||||
|
file_name: $(validation_report_file).sqlite
|
||||||
|
path: $(validation_report_path)
|
||||||
|
log_stored: true
|
||||||
164
schema/test_schema/console.tum
Normal file
164
schema/test_schema/console.tum
Normal file
@@ -0,0 +1,164 @@
|
|||||||
|
config_file:
|
||||||
|
- param.yaml
|
||||||
|
- items/console/param.yaml
|
||||||
|
main:
|
||||||
|
name: Testium validation suite
|
||||||
|
steps:
|
||||||
|
- group:
|
||||||
|
name: Test preparation
|
||||||
|
steps:
|
||||||
|
- let:
|
||||||
|
condition: <| "$(os)" == "Linux" |>
|
||||||
|
name: Set test variables for Linux
|
||||||
|
values:
|
||||||
|
- terminal_prompt: $(linux_prompt)
|
||||||
|
- psep: /
|
||||||
|
- let:
|
||||||
|
condition: <| "$(os)" == "Windows" |>
|
||||||
|
name: Set test variables for Windows
|
||||||
|
values:
|
||||||
|
- terminal_prompt: $(windows_prompt)
|
||||||
|
- psep: \
|
||||||
|
- group:
|
||||||
|
name: Group of tests
|
||||||
|
steps:
|
||||||
|
- let:
|
||||||
|
name: console test constants
|
||||||
|
values:
|
||||||
|
- test: console
|
||||||
|
- test_path: items/$(test)
|
||||||
|
- group:
|
||||||
|
name: console test
|
||||||
|
steps:
|
||||||
|
- sequence:
|
||||||
|
steps:
|
||||||
|
- console:
|
||||||
|
console_name: term
|
||||||
|
doc: Opening the console
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Console creation
|
||||||
|
steps:
|
||||||
|
- open:
|
||||||
|
protocol: terminal
|
||||||
|
terminal_path: $(test_directory)
|
||||||
|
- writeln: echo "endOfOpen"
|
||||||
|
- console:
|
||||||
|
console_name: term
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Console read_until with timeout
|
||||||
|
steps:
|
||||||
|
- read_until:
|
||||||
|
expected: endOfOpen
|
||||||
|
timeout: 5
|
||||||
|
- console:
|
||||||
|
console_name: term
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Console write
|
||||||
|
steps:
|
||||||
|
- writeln: echo 0
|
||||||
|
- sleep:
|
||||||
|
name: sleep item
|
||||||
|
timeout: 1
|
||||||
|
- console:
|
||||||
|
console_name: term
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Console read_until immediate
|
||||||
|
steps:
|
||||||
|
- read_until:
|
||||||
|
expected: '0'
|
||||||
|
timeout: 0
|
||||||
|
- console:
|
||||||
|
console_name: term
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Console write
|
||||||
|
steps:
|
||||||
|
- writeln: echo "HelloConsole"
|
||||||
|
- console:
|
||||||
|
console_name: term
|
||||||
|
key: $(test)_FAIL
|
||||||
|
name: Console read_until fail
|
||||||
|
steps:
|
||||||
|
- read_until:
|
||||||
|
expected: Something never prints
|
||||||
|
timeout: 1
|
||||||
|
- console:
|
||||||
|
console_name: term
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Console write
|
||||||
|
steps:
|
||||||
|
- writeln: echo "HelloConsole"
|
||||||
|
- console:
|
||||||
|
console_name: term
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Console read_until no_fail
|
||||||
|
steps:
|
||||||
|
- read_until:
|
||||||
|
expected: Something never prints
|
||||||
|
no_fail: true
|
||||||
|
timeout: 1
|
||||||
|
- console:
|
||||||
|
console_name: term
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Console read_until muted
|
||||||
|
steps:
|
||||||
|
- writeln: echo "HelloConsole"
|
||||||
|
- read_until:
|
||||||
|
expected: HelloConsole
|
||||||
|
mute: true
|
||||||
|
timeout: 1
|
||||||
|
- console:
|
||||||
|
console_name: term
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Console read_until muted
|
||||||
|
steps:
|
||||||
|
- writeln: echo "HelloConsole is PASS" && echo "endOfCmd"
|
||||||
|
- read_until:
|
||||||
|
expected: endOfCmd
|
||||||
|
process_result: '''Hello'' in r''''''$(result)''''''
|
||||||
|
and ''PASS'' in r''''''$(result)'''''' '
|
||||||
|
timeout: 1
|
||||||
|
- console:
|
||||||
|
console_name: term
|
||||||
|
execute_on_stop: true
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Console closure
|
||||||
|
steps:
|
||||||
|
- close: term
|
||||||
|
filename: /home/renish/workspace/testium/code/test/validation/items/console/test.tum
|
||||||
|
- sequence:
|
||||||
|
steps:
|
||||||
|
- report:
|
||||||
|
export:
|
||||||
|
- text:
|
||||||
|
key: $(test)_PASS
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_PASS.txt
|
||||||
|
- html:
|
||||||
|
key: $(test)_PASS
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_PASS.html
|
||||||
|
- junit:
|
||||||
|
key: $(test)_PASS
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_PASS.junit
|
||||||
|
name: Expected PASS $(test) test
|
||||||
|
- report:
|
||||||
|
export:
|
||||||
|
- text:
|
||||||
|
key: $(test)_FAIL
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_FAIL.txt
|
||||||
|
- html:
|
||||||
|
key: $(test)_FAIL
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_FAIL.html
|
||||||
|
- junit:
|
||||||
|
key: $(test)_FAIL
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_FAIL.junit
|
||||||
|
name: Expected FAIL $(test) test
|
||||||
|
filename: /home/renish/workspace/testium/code/test/validation/items/report.tum
|
||||||
|
report:
|
||||||
|
enabled: true
|
||||||
|
export:
|
||||||
|
junit:
|
||||||
|
file_name: $(validation_report_file).junit
|
||||||
|
path: $(validation_report_path)
|
||||||
|
sqlite:
|
||||||
|
file_name: $(validation_report_file).sqlite
|
||||||
|
path: $(validation_report_path)
|
||||||
|
log_stored: true
|
||||||
106
schema/test_schema/cycle.tum
Normal file
106
schema/test_schema/cycle.tum
Normal file
@@ -0,0 +1,106 @@
|
|||||||
|
config_file:
|
||||||
|
- param.yaml
|
||||||
|
- items/cycle/param.yaml
|
||||||
|
main:
|
||||||
|
name: Testium validation suite
|
||||||
|
steps:
|
||||||
|
- group:
|
||||||
|
name: Test preparation
|
||||||
|
steps:
|
||||||
|
- let:
|
||||||
|
condition: <| "$(os)" == "Linux" |>
|
||||||
|
name: Set test variables for Linux
|
||||||
|
values:
|
||||||
|
terminal_prompt: $(linux_prompt)
|
||||||
|
psep: /
|
||||||
|
- let:
|
||||||
|
condition: <| "$(os)" == "Windows" |>
|
||||||
|
name: Set test variables for Windows
|
||||||
|
values:
|
||||||
|
- terminal_prompt: $(windows_prompt)
|
||||||
|
- psep: \
|
||||||
|
- group:
|
||||||
|
name: Group of tests
|
||||||
|
steps:
|
||||||
|
- let:
|
||||||
|
name: cycle test constants
|
||||||
|
values:
|
||||||
|
- test: cycle
|
||||||
|
- test_path: items/$(test)
|
||||||
|
- group:
|
||||||
|
name: cycle test
|
||||||
|
steps:
|
||||||
|
- sequence:
|
||||||
|
steps:
|
||||||
|
- loop:
|
||||||
|
iterator: 10
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Cycle number of loops
|
||||||
|
steps:
|
||||||
|
- py_func:
|
||||||
|
file: $(test_path)$(psep)cycle.py
|
||||||
|
func_name: donothing
|
||||||
|
name: do nothing
|
||||||
|
- loop:
|
||||||
|
iterator:
|
||||||
|
- 12
|
||||||
|
- 20
|
||||||
|
- 30
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Cycle iterating on list
|
||||||
|
steps:
|
||||||
|
- py_func:
|
||||||
|
file: $(test_path)$(psep)cycle.py
|
||||||
|
func_name: checkloopparam
|
||||||
|
name: check loop param
|
||||||
|
param:
|
||||||
|
- $(loop_param)
|
||||||
|
- loop:
|
||||||
|
exit_condition:
|
||||||
|
file: $(test_path)$(psep)cycle.py
|
||||||
|
func_name: exitcondition
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Infinite loop with exit condition
|
||||||
|
steps:
|
||||||
|
- py_func:
|
||||||
|
file: $(test_path)$(psep)cycle.py
|
||||||
|
func_name: donothing
|
||||||
|
name: do nothing
|
||||||
|
filename: /home/renish/workspace/testium/code/test/validation/items/cycle/test.tum
|
||||||
|
- sequence:
|
||||||
|
steps:
|
||||||
|
- report:
|
||||||
|
export:
|
||||||
|
- text:
|
||||||
|
key: $(test)_PASS
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_PASS.txt
|
||||||
|
- html:
|
||||||
|
key: $(test)_PASS
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_PASS.html
|
||||||
|
- junit:
|
||||||
|
key: $(test)_PASS
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_PASS.junit
|
||||||
|
name: Expected PASS $(test) test
|
||||||
|
- report:
|
||||||
|
export:
|
||||||
|
- text:
|
||||||
|
key: $(test)_FAIL
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_FAIL.txt
|
||||||
|
- html:
|
||||||
|
key: $(test)_FAIL
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_FAIL.html
|
||||||
|
- junit:
|
||||||
|
key: $(test)_FAIL
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_FAIL.junit
|
||||||
|
name: Expected FAIL $(test) test
|
||||||
|
filename: /home/renish/workspace/testium/code/test/validation/items/report.tum
|
||||||
|
report:
|
||||||
|
enabled: true
|
||||||
|
export:
|
||||||
|
junit:
|
||||||
|
file_name: $(validation_report_file).junit
|
||||||
|
path: $(validation_report_path)
|
||||||
|
sqlite:
|
||||||
|
file_name: $(validation_report_file).sqlite
|
||||||
|
path: $(validation_report_path)
|
||||||
|
log_stored: true
|
||||||
135
schema/test_schema/dialogs.tum
Normal file
135
schema/test_schema/dialogs.tum
Normal file
@@ -0,0 +1,135 @@
|
|||||||
|
config_file:
|
||||||
|
- param.yaml
|
||||||
|
- items/dialogs/param.yaml
|
||||||
|
main:
|
||||||
|
name: Testium validation suite
|
||||||
|
steps:
|
||||||
|
- group:
|
||||||
|
name: Test preparation
|
||||||
|
steps:
|
||||||
|
- let:
|
||||||
|
condition: <| "$(os)" == "Linux" |>
|
||||||
|
name: Set test variables for Linux
|
||||||
|
values:
|
||||||
|
- terminal_prompt: $(linux_prompt)
|
||||||
|
- psep: /
|
||||||
|
- let:
|
||||||
|
condition: <| "$(os)" == "Windows" |>
|
||||||
|
name: Set test variables for Windows
|
||||||
|
values:
|
||||||
|
- terminal_prompt: $(windows_prompt)
|
||||||
|
- psep: \
|
||||||
|
- group:
|
||||||
|
name: Group of tests
|
||||||
|
steps:
|
||||||
|
- let:
|
||||||
|
name: dialogs test constants
|
||||||
|
values:
|
||||||
|
test: dialogs
|
||||||
|
test_path: items/$(test)
|
||||||
|
- group:
|
||||||
|
name: dialogs test
|
||||||
|
steps:
|
||||||
|
- sequence:
|
||||||
|
steps:
|
||||||
|
- dialog_image:
|
||||||
|
auto_result: ok
|
||||||
|
condition: $(validation_dialogs)
|
||||||
|
filename: $(test_path)$(psep)IMG_20140213_171455.jpg
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: dialog image PASS
|
||||||
|
question: click ok if you see the image
|
||||||
|
- dialog_image:
|
||||||
|
auto_result: cancel
|
||||||
|
condition: $(validation_dialogs)
|
||||||
|
filename: $(test_path)$(psep)IMG_20140213_171455.jpg
|
||||||
|
key: $(test)_FAIL
|
||||||
|
name: dialog image FAIL
|
||||||
|
question: click cancel
|
||||||
|
- dialog_references:
|
||||||
|
auto_result: ok
|
||||||
|
condition: $(validation_dialogs)
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: dialog_reference PASS
|
||||||
|
question: click ok
|
||||||
|
- dialog_references:
|
||||||
|
auto_result: cancel
|
||||||
|
condition: $(validation_dialogs)
|
||||||
|
key: $(test)_FAIL
|
||||||
|
name: dialog_reference FAIL
|
||||||
|
question: click cancel
|
||||||
|
- dialog_value:
|
||||||
|
auto_result: ok
|
||||||
|
auto_value: '123'
|
||||||
|
condition: $(validation_dialogs)
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: dialog_value PASS
|
||||||
|
question: enter 123 and click ok
|
||||||
|
- dialog_value:
|
||||||
|
auto_result: ok
|
||||||
|
condition: $(validation_dialogs)
|
||||||
|
key: $(test)_FAIL
|
||||||
|
name: dialog_value empty FAIL
|
||||||
|
question: enter nothing and click ok
|
||||||
|
- dialog_value:
|
||||||
|
auto_result: cancel
|
||||||
|
condition: $(validation_dialogs)
|
||||||
|
key: $(test)_FAIL
|
||||||
|
name: dialog_value canceled FAIL
|
||||||
|
question: enter nothing and click cancel
|
||||||
|
- dialog_message:
|
||||||
|
auto_result: ok
|
||||||
|
condition: $(validation_dialogs)
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: dialog_message PASS
|
||||||
|
question: click ok
|
||||||
|
- dialog_question:
|
||||||
|
auto_result: 'yes'
|
||||||
|
condition: $(validation_dialogs)
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: dialog_question PASS
|
||||||
|
question: click yes
|
||||||
|
- dialog_question:
|
||||||
|
auto_result: 'no'
|
||||||
|
condition: $(validation_dialogs)
|
||||||
|
key: $(test)_FAIL
|
||||||
|
name: dialog_question FAIL
|
||||||
|
question: click no
|
||||||
|
filename: /home/renish/workspace/testium/code/test/validation/items/dialogs/test.tum
|
||||||
|
- sequence:
|
||||||
|
steps:
|
||||||
|
- report:
|
||||||
|
export:
|
||||||
|
- text:
|
||||||
|
key: $(test)_PASS
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_PASS.txt
|
||||||
|
- html:
|
||||||
|
key: $(test)_PASS
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_PASS.html
|
||||||
|
- junit:
|
||||||
|
key: $(test)_PASS
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_PASS.junit
|
||||||
|
name: Expected PASS $(test) test
|
||||||
|
- report:
|
||||||
|
export:
|
||||||
|
- text:
|
||||||
|
key: $(test)_FAIL
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_FAIL.txt
|
||||||
|
- html:
|
||||||
|
key: $(test)_FAIL
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_FAIL.html
|
||||||
|
- junit:
|
||||||
|
key: $(test)_FAIL
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_FAIL.junit
|
||||||
|
name: Expected FAIL $(test) test
|
||||||
|
filename: /home/renish/workspace/testium/code/test/validation/items/report.tum
|
||||||
|
report:
|
||||||
|
enabled: true
|
||||||
|
export:
|
||||||
|
junit:
|
||||||
|
file_name: $(validation_report_file).junit
|
||||||
|
path: $(validation_report_path)
|
||||||
|
sqlite:
|
||||||
|
file_name: $(validation_report_file).sqlite
|
||||||
|
path: $(validation_report_path)
|
||||||
|
log_stored: true
|
||||||
130
schema/test_schema/expanse.tum
Normal file
130
schema/test_schema/expanse.tum
Normal file
@@ -0,0 +1,130 @@
|
|||||||
|
config_file:
|
||||||
|
- param.yaml
|
||||||
|
- items/expanse/param.yaml
|
||||||
|
main:
|
||||||
|
name: Testium validation suite
|
||||||
|
steps:
|
||||||
|
- group:
|
||||||
|
name: Test preparation
|
||||||
|
steps:
|
||||||
|
- let:
|
||||||
|
condition: <| "$(os)" == "Linux" |>
|
||||||
|
name: Set test variables for Linux
|
||||||
|
values:
|
||||||
|
- terminal_prompt: $(linux_prompt)
|
||||||
|
- psep: /
|
||||||
|
- let:
|
||||||
|
condition: <| "$(os)" == "Windows" |>
|
||||||
|
name: Set test variables for Windows
|
||||||
|
values:
|
||||||
|
- terminal_prompt: $(windows_prompt)
|
||||||
|
- psep: \
|
||||||
|
- group:
|
||||||
|
name: Group of tests
|
||||||
|
steps:
|
||||||
|
- let:
|
||||||
|
name: expanse test constants
|
||||||
|
values:
|
||||||
|
test: expanse
|
||||||
|
test_path: items/$(test)
|
||||||
|
- group:
|
||||||
|
name: expanse test
|
||||||
|
steps:
|
||||||
|
- sequence:
|
||||||
|
steps:
|
||||||
|
- check:
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Check variables expansion is correct (PASS)
|
||||||
|
values:
|
||||||
|
- <| $(expanse_index) == 1 |>
|
||||||
|
- <| $(expanse_table)[$(expanse_index)] == 9012 |>
|
||||||
|
- <| $(expanse_eval) == True |>
|
||||||
|
- let:
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Dynamic variables expansion
|
||||||
|
values:
|
||||||
|
- expanse_select: <|"$(expanse_select)".replace("o", "a")|>
|
||||||
|
- expanse_index: $(expanse_index_$(expanse_select))
|
||||||
|
- expanse_table: $(expanse_table_$(expanse_select))
|
||||||
|
- expanse_eval: <|$(expanse_index) == 1|>
|
||||||
|
- check:
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Check variables expansion is correct (PASS)
|
||||||
|
values:
|
||||||
|
- <| $(expanse_index) == 0 |>
|
||||||
|
- <| $(expanse_table)[$(expanse_index)] == "abcd" |>
|
||||||
|
- <| $(expanse_eval) == False |>
|
||||||
|
- let:
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Complex variables expansion
|
||||||
|
values:
|
||||||
|
- var1: expanse
|
||||||
|
- var2: var
|
||||||
|
- var3: bla
|
||||||
|
- var4: blo
|
||||||
|
- expanse_var_bla: 3
|
||||||
|
- expanse_blo_var: 5
|
||||||
|
- expanse_complex: <|<|$(expanse_$(var2)_$(var3))*6|>
|
||||||
|
+ <|4*$($(var1)_$(var4)_$(var2))|>|>
|
||||||
|
- check:
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Check complex variables expansion is correct (PASS)
|
||||||
|
values:
|
||||||
|
- <| $(expanse_complex) == 38 |>
|
||||||
|
- let:
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Variables expansion in object
|
||||||
|
values:
|
||||||
|
- expanse_key: b
|
||||||
|
- expanse_var: 3
|
||||||
|
- expanse_var_2: 6
|
||||||
|
- expanse_object:
|
||||||
|
- $(expanse_key): <|2**3|>
|
||||||
|
a: $(expanse_var_2)
|
||||||
|
- <|"bla".replace("a", "o")|>:
|
||||||
|
- <|$(expanse_var)*$(expanse_var_2)|>
|
||||||
|
- 25
|
||||||
|
- check:
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Check complex variables expansion is correct (PASS)
|
||||||
|
values:
|
||||||
|
- '<| $(expanse_object) == [{"a": 6, "b": 8}, {"blo": [18,
|
||||||
|
25]}] |>'
|
||||||
|
filename: /home/renish/workspace/testium/code/test/validation/items/expanse/test.tum
|
||||||
|
- sequence:
|
||||||
|
steps:
|
||||||
|
- report:
|
||||||
|
export:
|
||||||
|
- text:
|
||||||
|
key: $(test)_PASS
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_PASS.txt
|
||||||
|
- html:
|
||||||
|
key: $(test)_PASS
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_PASS.html
|
||||||
|
- junit:
|
||||||
|
key: $(test)_PASS
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_PASS.junit
|
||||||
|
name: Expected PASS $(test) test
|
||||||
|
- report:
|
||||||
|
export:
|
||||||
|
- text:
|
||||||
|
key: $(test)_FAIL
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_FAIL.txt
|
||||||
|
- html:
|
||||||
|
key: $(test)_FAIL
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_FAIL.html
|
||||||
|
- junit:
|
||||||
|
key: $(test)_FAIL
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_FAIL.junit
|
||||||
|
name: Expected FAIL $(test) test
|
||||||
|
filename: /home/renish/workspace/testium/code/test/validation/items/report.tum
|
||||||
|
report:
|
||||||
|
enabled: true
|
||||||
|
export:
|
||||||
|
junit:
|
||||||
|
file_name: $(validation_report_file).junit
|
||||||
|
path: $(validation_report_path)
|
||||||
|
sqlite:
|
||||||
|
file_name: $(validation_report_file).sqlite
|
||||||
|
path: $(validation_report_path)
|
||||||
|
log_stored: true
|
||||||
82
schema/test_schema/git.tum
Normal file
82
schema/test_schema/git.tum
Normal file
@@ -0,0 +1,82 @@
|
|||||||
|
config_file:
|
||||||
|
- param.yaml
|
||||||
|
- items/git/param.yaml
|
||||||
|
main:
|
||||||
|
name: Testium validation suite
|
||||||
|
steps:
|
||||||
|
- group:
|
||||||
|
name: Test preparation
|
||||||
|
steps:
|
||||||
|
- let:
|
||||||
|
condition: <| "$(os)" == "Linux" |>
|
||||||
|
name: Set test variables for Linux
|
||||||
|
values:
|
||||||
|
- terminal_prompt: $(linux_prompt)
|
||||||
|
- psep: /
|
||||||
|
- let:
|
||||||
|
condition: <| "$(os)" == "Windows" |>
|
||||||
|
name: Set test variables for Windows
|
||||||
|
values:
|
||||||
|
- terminal_prompt: $(windows_prompt)
|
||||||
|
- psep: \
|
||||||
|
- group:
|
||||||
|
name: Group of tests
|
||||||
|
steps:
|
||||||
|
- let:
|
||||||
|
name: git test constants
|
||||||
|
values:
|
||||||
|
test: git
|
||||||
|
test_path: items/$(test)
|
||||||
|
- group:
|
||||||
|
name: git test
|
||||||
|
steps:
|
||||||
|
- sequence:
|
||||||
|
steps:
|
||||||
|
- git:
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Testium repo
|
||||||
|
repo: $(test_directory)
|
||||||
|
- git:
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Testium repo
|
||||||
|
repo:
|
||||||
|
- $(test_directory)
|
||||||
|
- $(test_directory)
|
||||||
|
filename: /home/renish/workspace/testium/code/test/validation/items/git/test.tum
|
||||||
|
- sequence:
|
||||||
|
steps:
|
||||||
|
- report:
|
||||||
|
export:
|
||||||
|
- text:
|
||||||
|
key: $(test)_PASS
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_PASS.txt
|
||||||
|
- html:
|
||||||
|
key: $(test)_PASS
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_PASS.html
|
||||||
|
- junit:
|
||||||
|
key: $(test)_PASS
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_PASS.junit
|
||||||
|
name: Expected PASS $(test) test
|
||||||
|
- report:
|
||||||
|
export:
|
||||||
|
- text:
|
||||||
|
key: $(test)_FAIL
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_FAIL.txt
|
||||||
|
- html:
|
||||||
|
key: $(test)_FAIL
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_FAIL.html
|
||||||
|
- junit:
|
||||||
|
key: $(test)_FAIL
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_FAIL.junit
|
||||||
|
name: Expected FAIL $(test) test
|
||||||
|
filename: /home/renish/workspace/testium/code/test/validation/items/report.tum
|
||||||
|
report:
|
||||||
|
enabled: true
|
||||||
|
export:
|
||||||
|
junit:
|
||||||
|
file_name: $(validation_report_file).junit
|
||||||
|
path: $(validation_report_path)
|
||||||
|
sqlite:
|
||||||
|
file_name: $(validation_report_file).sqlite
|
||||||
|
path: $(validation_report_path)
|
||||||
|
log_stored: true
|
||||||
129
schema/test_schema/include.tum
Normal file
129
schema/test_schema/include.tum
Normal file
@@ -0,0 +1,129 @@
|
|||||||
|
config_file:
|
||||||
|
- param.yaml
|
||||||
|
- items/include/param.yaml
|
||||||
|
main:
|
||||||
|
name: Testium validation suite
|
||||||
|
steps:
|
||||||
|
- group:
|
||||||
|
name: Test preparation
|
||||||
|
steps:
|
||||||
|
- let:
|
||||||
|
condition: <| "$(os)" == "Linux" |>
|
||||||
|
name: Set test variables for Linux
|
||||||
|
values:
|
||||||
|
- terminal_prompt: $(linux_prompt)
|
||||||
|
- psep: /
|
||||||
|
- let:
|
||||||
|
condition: <| "$(os)" == "Windows" |>
|
||||||
|
name: Set test variables for Windows
|
||||||
|
values:
|
||||||
|
- terminal_prompt: $(windows_prompt)
|
||||||
|
- psep: \
|
||||||
|
- group:
|
||||||
|
name: Group of tests
|
||||||
|
steps:
|
||||||
|
- let:
|
||||||
|
name: include test constants
|
||||||
|
values:
|
||||||
|
test: include
|
||||||
|
test_path: items/$(test)
|
||||||
|
- group:
|
||||||
|
name: include test
|
||||||
|
steps:
|
||||||
|
- sequence:
|
||||||
|
steps:
|
||||||
|
- sequence:
|
||||||
|
steps:
|
||||||
|
- py_func:
|
||||||
|
file: $(test_path)$(psep)include.py
|
||||||
|
func_name: ValidationTest
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: My first include test
|
||||||
|
param:
|
||||||
|
- $(test parameter)
|
||||||
|
filename: /home/renish/workspace/testium/code/test/validation/items/include/inc
|
||||||
|
no template/my first include.tum
|
||||||
|
- sequence:
|
||||||
|
steps:
|
||||||
|
- py_func:
|
||||||
|
file: $(test_path)$(psep)include.py
|
||||||
|
func_name: ValidationTest
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: My first include test
|
||||||
|
param:
|
||||||
|
- $(test parameter)
|
||||||
|
filename: /home/renish/workspace/testium/code/test/validation/items/include/inc
|
||||||
|
no template/my first include.tum
|
||||||
|
- sequence:
|
||||||
|
steps:
|
||||||
|
- py_func:
|
||||||
|
file: $(test_path)$(psep)include.py
|
||||||
|
func_name: ValidationTest
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: My second include test
|
||||||
|
param:
|
||||||
|
- My second include test parameter
|
||||||
|
filename: /home/renish/workspace/testium/code/test/validation/items/include/inc
|
||||||
|
with template/my second include.tum
|
||||||
|
- sequence:
|
||||||
|
steps:
|
||||||
|
- py_func:
|
||||||
|
file: $(test_path)$(psep)include.py
|
||||||
|
func_name: ValidationTest
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: My second include test
|
||||||
|
param:
|
||||||
|
- My second include test parameter
|
||||||
|
filename: /home/renish/workspace/testium/code/test/validation/items/include/inc
|
||||||
|
with template/my second include.tum
|
||||||
|
- let:
|
||||||
|
name: Declare param for inclusion
|
||||||
|
values:
|
||||||
|
- inc: Dali
|
||||||
|
- Dali_inc: Dalida
|
||||||
|
- sequence:
|
||||||
|
steps:
|
||||||
|
- let:
|
||||||
|
name: Test param inclusion 1
|
||||||
|
values:
|
||||||
|
- inclusion: $($(inc)_inc)
|
||||||
|
filename: /home/renish/workspace/testium/code/test/validation/items/include/inc
|
||||||
|
with template/my_3d_include.tum
|
||||||
|
filename: /home/renish/workspace/testium/code/test/validation/items/include/test.tum
|
||||||
|
- sequence:
|
||||||
|
steps:
|
||||||
|
- report:
|
||||||
|
export:
|
||||||
|
- text:
|
||||||
|
key: $(test)_PASS
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_PASS.txt
|
||||||
|
- html:
|
||||||
|
key: $(test)_PASS
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_PASS.html
|
||||||
|
- junit:
|
||||||
|
key: $(test)_PASS
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_PASS.junit
|
||||||
|
name: Expected PASS $(test) test
|
||||||
|
- report:
|
||||||
|
export:
|
||||||
|
- text:
|
||||||
|
key: $(test)_FAIL
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_FAIL.txt
|
||||||
|
- html:
|
||||||
|
key: $(test)_FAIL
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_FAIL.html
|
||||||
|
- junit:
|
||||||
|
key: $(test)_FAIL
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_FAIL.junit
|
||||||
|
name: Expected FAIL $(test) test
|
||||||
|
filename: /home/renish/workspace/testium/code/test/validation/items/report.tum
|
||||||
|
report:
|
||||||
|
enabled: true
|
||||||
|
export:
|
||||||
|
junit:
|
||||||
|
file_name: $(validation_report_file).junit
|
||||||
|
path: $(validation_report_path)
|
||||||
|
sqlite:
|
||||||
|
file_name: $(validation_report_file).sqlite
|
||||||
|
path: $(validation_report_path)
|
||||||
|
log_stored: true
|
||||||
80
schema/test_schema/isolation.tum
Normal file
80
schema/test_schema/isolation.tum
Normal file
@@ -0,0 +1,80 @@
|
|||||||
|
config_file:
|
||||||
|
- param.yaml
|
||||||
|
- items/isolation/param.yaml
|
||||||
|
main:
|
||||||
|
name: Testium validation suite
|
||||||
|
steps:
|
||||||
|
- group:
|
||||||
|
name: Test preparation
|
||||||
|
steps:
|
||||||
|
- let:
|
||||||
|
condition: <| "$(os)" == "Linux" |>
|
||||||
|
name: Set test variables for Linux
|
||||||
|
values:
|
||||||
|
- terminal_prompt: $(linux_prompt)
|
||||||
|
- psep: /
|
||||||
|
- let:
|
||||||
|
condition: <| "$(os)" == "Windows" |>
|
||||||
|
name: Set test variables for Windows
|
||||||
|
values:
|
||||||
|
- terminal_prompt: $(windows_prompt)
|
||||||
|
- psep: \
|
||||||
|
- group:
|
||||||
|
name: Group of tests
|
||||||
|
steps:
|
||||||
|
- let:
|
||||||
|
name: isolation test constants
|
||||||
|
values:
|
||||||
|
test: isolation
|
||||||
|
test_path: items/$(test)
|
||||||
|
- group:
|
||||||
|
name: isolation test
|
||||||
|
steps:
|
||||||
|
- sequence:
|
||||||
|
steps:
|
||||||
|
- py_func:
|
||||||
|
expected_result: true
|
||||||
|
file: $(test_path)$(psep)check_isolation.py
|
||||||
|
func_name: check_isolation
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: py_func/lua_func do not depend on testium internals
|
||||||
|
param:
|
||||||
|
- $(testium_path)
|
||||||
|
filename: /home/renish/workspace/testium/code/test/validation/items/isolation/test.tum
|
||||||
|
- sequence:
|
||||||
|
steps:
|
||||||
|
- report:
|
||||||
|
export:
|
||||||
|
- text:
|
||||||
|
key: $(test)_PASS
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_PASS.txt
|
||||||
|
- html:
|
||||||
|
key: $(test)_PASS
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_PASS.html
|
||||||
|
- junit:
|
||||||
|
key: $(test)_PASS
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_PASS.junit
|
||||||
|
name: Expected PASS $(test) test
|
||||||
|
- report:
|
||||||
|
export:
|
||||||
|
- text:
|
||||||
|
key: $(test)_FAIL
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_FAIL.txt
|
||||||
|
- html:
|
||||||
|
key: $(test)_FAIL
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_FAIL.html
|
||||||
|
- junit:
|
||||||
|
key: $(test)_FAIL
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_FAIL.junit
|
||||||
|
name: Expected FAIL $(test) test
|
||||||
|
filename: /home/renish/workspace/testium/code/test/validation/items/report.tum
|
||||||
|
report:
|
||||||
|
enabled: true
|
||||||
|
export:
|
||||||
|
junit:
|
||||||
|
file_name: $(validation_report_file).junit
|
||||||
|
path: $(validation_report_path)
|
||||||
|
sqlite:
|
||||||
|
file_name: $(validation_report_file).sqlite
|
||||||
|
path: $(validation_report_path)
|
||||||
|
log_stored: true
|
||||||
490
schema/test_schema/jsonrpc.tum
Normal file
490
schema/test_schema/jsonrpc.tum
Normal file
@@ -0,0 +1,490 @@
|
|||||||
|
config_file:
|
||||||
|
- param.yaml
|
||||||
|
- items/jsonrpc/param.yaml
|
||||||
|
main:
|
||||||
|
name: Testium validation suite
|
||||||
|
steps:
|
||||||
|
- group:
|
||||||
|
name: Test preparation
|
||||||
|
steps:
|
||||||
|
- let:
|
||||||
|
condition: <| "$(os)" == "Linux" |>
|
||||||
|
name: Set test variables for Linux
|
||||||
|
values:
|
||||||
|
- terminal_prompt: $(linux_prompt)
|
||||||
|
- psep: /
|
||||||
|
- let:
|
||||||
|
condition: <| "$(os)" == "Windows" |>
|
||||||
|
name: Set test variables for Windows
|
||||||
|
values:
|
||||||
|
- terminal_prompt: $(windows_prompt)
|
||||||
|
- psep: \
|
||||||
|
- group:
|
||||||
|
name: Group of tests
|
||||||
|
steps:
|
||||||
|
- let:
|
||||||
|
name: jsonrpc test constants
|
||||||
|
values:
|
||||||
|
test: jsonrpc
|
||||||
|
test_path: items/$(test)
|
||||||
|
- group:
|
||||||
|
name: jsonrpc test
|
||||||
|
steps:
|
||||||
|
- sequence:
|
||||||
|
steps:
|
||||||
|
- console:
|
||||||
|
console_name: jrpces
|
||||||
|
doc: check if jrpc_echo_server.py is available
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: json rpc echo server
|
||||||
|
steps:
|
||||||
|
- open:
|
||||||
|
protocol: terminal
|
||||||
|
- read_until:
|
||||||
|
expected: $(terminal_prompt)
|
||||||
|
no_fail: true
|
||||||
|
timeout: 1
|
||||||
|
- writeln: test -f /home/renish/workspace/testium/code/test/validation/items/jsonrpc/jrpc_echo_server.py
|
||||||
|
&& echo JRPC_OK
|
||||||
|
- read_until:
|
||||||
|
expected: JRPC_OK
|
||||||
|
no_fail: true
|
||||||
|
timeout: 2
|
||||||
|
- group:
|
||||||
|
condition: <| 'JRPC_OK' in r'''$(cn_json rpc echo server)''' |>
|
||||||
|
name: jsonrpc tests
|
||||||
|
steps:
|
||||||
|
- console:
|
||||||
|
console_name: jrpces
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Start the json rpc echo server
|
||||||
|
steps:
|
||||||
|
- writeln: python3 /home/renish/workspace/testium/code/test/validation/items/jsonrpc/jrpc_echo_server.py
|
||||||
|
-c /home/renish/workspace/testium/code/test/validation/items/jsonrpc/jrpces.ini
|
||||||
|
- read_until:
|
||||||
|
expected: ready
|
||||||
|
timeout: 5
|
||||||
|
- console:
|
||||||
|
console_name: jsonrpc_server
|
||||||
|
doc: Opening the RAW TCP console
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Open the raw tcp Console
|
||||||
|
skipped: $(skip_tcp)
|
||||||
|
steps:
|
||||||
|
- open:
|
||||||
|
protocol: rawtcp
|
||||||
|
tcp_host: localhost
|
||||||
|
tcp_port: 4321
|
||||||
|
- json_rpc:
|
||||||
|
console:
|
||||||
|
name: jsonrpc_server
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: JSONRPC console Query waiting for reception
|
||||||
|
skipped: $(skip_tcp)
|
||||||
|
steps:
|
||||||
|
- query:
|
||||||
|
method: echo
|
||||||
|
params:
|
||||||
|
- Hello World
|
||||||
|
- a: 1
|
||||||
|
b: hello
|
||||||
|
timeout: 1
|
||||||
|
- json_rpc:
|
||||||
|
console:
|
||||||
|
name: jsonrpc_server
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: JSONRPC console Query not waiting (only send)
|
||||||
|
skipped: $(skip_tcp)
|
||||||
|
steps:
|
||||||
|
- query:
|
||||||
|
id: 3095372
|
||||||
|
method: echo
|
||||||
|
no_wait: true
|
||||||
|
params:
|
||||||
|
- a: -1
|
||||||
|
b: olleh
|
||||||
|
- World Hello
|
||||||
|
timeout: 1
|
||||||
|
- sleep:
|
||||||
|
name: Small delay for the test
|
||||||
|
skipped: $(skip_tcp)
|
||||||
|
timeout: 1
|
||||||
|
- json_rpc:
|
||||||
|
console:
|
||||||
|
name: jsonrpc_server
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: JSONRPC console Reception
|
||||||
|
skipped: $(skip_tcp)
|
||||||
|
steps:
|
||||||
|
- receive:
|
||||||
|
id: 3095372
|
||||||
|
timeout: 1
|
||||||
|
- console:
|
||||||
|
console_name: jsonrpc_server
|
||||||
|
doc: Opening the RAW TCP console
|
||||||
|
execute_on_stop: true
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Close the raw tcp console
|
||||||
|
skipped: $(skip_tcp)
|
||||||
|
steps:
|
||||||
|
- close: null
|
||||||
|
- json_rpc:
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: JSONRPC UDP query waiting for reception
|
||||||
|
steps:
|
||||||
|
- open: null
|
||||||
|
- query:
|
||||||
|
method: echo
|
||||||
|
name: echo
|
||||||
|
params:
|
||||||
|
- Hello World
|
||||||
|
- a: 1
|
||||||
|
b: hello
|
||||||
|
timeout: 1
|
||||||
|
- close: null
|
||||||
|
timeout: 1
|
||||||
|
udp:
|
||||||
|
rcv_port: 8765
|
||||||
|
server: localhost
|
||||||
|
snd_port: 4323
|
||||||
|
- json_rpc:
|
||||||
|
key: $(test)_FAIL
|
||||||
|
name: Failing JSONRPC UDP query waiting for reception
|
||||||
|
(returning an error)
|
||||||
|
steps:
|
||||||
|
- open: null
|
||||||
|
- query:
|
||||||
|
method: echo2
|
||||||
|
params:
|
||||||
|
- Hello World
|
||||||
|
- a: 1
|
||||||
|
b: hello
|
||||||
|
timeout: 1
|
||||||
|
- close: null
|
||||||
|
timeout: 1
|
||||||
|
udp:
|
||||||
|
rcv_port: 8765
|
||||||
|
server: localhost
|
||||||
|
snd_port: 4323
|
||||||
|
- json_rpc:
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: JSONRPC UDP query waiting for reception of
|
||||||
|
an expected error
|
||||||
|
steps:
|
||||||
|
- open: null
|
||||||
|
- query:
|
||||||
|
expected_result:
|
||||||
|
code: -32000
|
||||||
|
message: function not found
|
||||||
|
method: echo2
|
||||||
|
params:
|
||||||
|
- Hello World
|
||||||
|
- a: 1
|
||||||
|
b: hello
|
||||||
|
- close: null
|
||||||
|
timeout: 1
|
||||||
|
udp:
|
||||||
|
rcv_port: 8765
|
||||||
|
server: localhost
|
||||||
|
snd_port: 4323
|
||||||
|
- json_rpc:
|
||||||
|
doc: 'Failing JSONRPC UDP query waiting for reception
|
||||||
|
and checking result
|
||||||
|
|
||||||
|
and timeout elapses (wrong udp port)
|
||||||
|
|
||||||
|
'
|
||||||
|
key: $(test)_FAIL
|
||||||
|
name: Failing UDP JSONRPC query timeout elapses (wrong
|
||||||
|
udp port)
|
||||||
|
steps:
|
||||||
|
- open: null
|
||||||
|
- query:
|
||||||
|
method: echo
|
||||||
|
params:
|
||||||
|
- Hello World
|
||||||
|
- a: 1
|
||||||
|
b: hello
|
||||||
|
timeout: 0.5
|
||||||
|
- close: null
|
||||||
|
timeout: 1
|
||||||
|
udp:
|
||||||
|
rcv_port: 48393
|
||||||
|
server: localhost
|
||||||
|
snd_port: 4326
|
||||||
|
- json_rpc:
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: JSONRPC UDP query not waiting (only send)
|
||||||
|
steps:
|
||||||
|
- open: null
|
||||||
|
- query:
|
||||||
|
id: 3095372
|
||||||
|
method: echo
|
||||||
|
no_wait: true
|
||||||
|
params:
|
||||||
|
- a: -1
|
||||||
|
b: olleh
|
||||||
|
- World Hello
|
||||||
|
timeout: 1
|
||||||
|
udp:
|
||||||
|
rcv_port: 8765
|
||||||
|
server: localhost
|
||||||
|
snd_port: 4323
|
||||||
|
- sleep:
|
||||||
|
name: Small delay for the test
|
||||||
|
timeout: 1
|
||||||
|
- json_rpc:
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: JSONRPC UDP Reception
|
||||||
|
steps:
|
||||||
|
- receive:
|
||||||
|
id: 3095372
|
||||||
|
- close: null
|
||||||
|
timeout: 1
|
||||||
|
udp:
|
||||||
|
rcv_port: 8765
|
||||||
|
server: localhost
|
||||||
|
snd_port: 4323
|
||||||
|
- json_rpc:
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: JSONRPC UDP query not waiting (only send)
|
||||||
|
steps:
|
||||||
|
- open: null
|
||||||
|
- query:
|
||||||
|
id: 3095372
|
||||||
|
method: echo2
|
||||||
|
no_wait: true
|
||||||
|
params:
|
||||||
|
- a: -1
|
||||||
|
b: olleh
|
||||||
|
- World Hello
|
||||||
|
timeout: 1
|
||||||
|
udp:
|
||||||
|
rcv_port: 8765
|
||||||
|
server: localhost
|
||||||
|
snd_port: 4323
|
||||||
|
- sleep:
|
||||||
|
name: Small delay for the test
|
||||||
|
timeout: 1
|
||||||
|
- json_rpc:
|
||||||
|
key: $(test)_FAIL
|
||||||
|
name: Failing JSONRPC UDP Reception (returning an
|
||||||
|
error)
|
||||||
|
steps:
|
||||||
|
- receive:
|
||||||
|
id: 3095372
|
||||||
|
timeout: 1
|
||||||
|
- close: null
|
||||||
|
timeout: 1
|
||||||
|
udp:
|
||||||
|
rcv_port: 8765
|
||||||
|
server: localhost
|
||||||
|
snd_port: 4323
|
||||||
|
- json_rpc:
|
||||||
|
doc: JSONRPC UDP query waiting for reception and
|
||||||
|
checking result
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: UDP JSONRPC query waiting and checking
|
||||||
|
steps:
|
||||||
|
- open: null
|
||||||
|
- query:
|
||||||
|
expected_result:
|
||||||
|
- - Hello World
|
||||||
|
- a: 1
|
||||||
|
b: hello
|
||||||
|
- {}
|
||||||
|
method: echo
|
||||||
|
params:
|
||||||
|
- Hello World
|
||||||
|
- a: 1
|
||||||
|
b: hello
|
||||||
|
timeout: 1
|
||||||
|
- close: null
|
||||||
|
timeout: 1
|
||||||
|
udp:
|
||||||
|
rcv_port: 48393
|
||||||
|
server: localhost
|
||||||
|
snd_port: 4323
|
||||||
|
- json_rpc:
|
||||||
|
doc: JSONRPC UDP query waiting for reception and
|
||||||
|
checking result
|
||||||
|
key: $(test)_FAIL
|
||||||
|
name: Failing UDP JSONRPC query waiting and checking
|
||||||
|
steps:
|
||||||
|
- open: null
|
||||||
|
- query:
|
||||||
|
expected_result:
|
||||||
|
- []
|
||||||
|
- {}
|
||||||
|
method: echo
|
||||||
|
params:
|
||||||
|
- Hello World
|
||||||
|
- a: 1
|
||||||
|
b: hello
|
||||||
|
timeout: 1
|
||||||
|
- close: null
|
||||||
|
timeout: 1
|
||||||
|
udp:
|
||||||
|
rcv_port: 48393
|
||||||
|
server: localhost
|
||||||
|
snd_port: 4323
|
||||||
|
- json_rpc:
|
||||||
|
doc: JSONRPC UDP query not waiting, with the purpose
|
||||||
|
to check the result at reception
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: UDP JSONRPC query not waiting (for checking)
|
||||||
|
steps:
|
||||||
|
- open: null
|
||||||
|
- query:
|
||||||
|
id: 3095372
|
||||||
|
method: echo
|
||||||
|
no_wait: true
|
||||||
|
params:
|
||||||
|
- a: -1
|
||||||
|
b: olleh
|
||||||
|
- World Hello
|
||||||
|
timeout: 1
|
||||||
|
udp:
|
||||||
|
rcv_port: 9876
|
||||||
|
server: localhost
|
||||||
|
snd_port: 4323
|
||||||
|
- sleep:
|
||||||
|
name: Small delay for the test
|
||||||
|
timeout: 1
|
||||||
|
- json_rpc:
|
||||||
|
doc: JSONRPC UDP Reception and checking result
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: UDP JSONRPC reception checking
|
||||||
|
steps:
|
||||||
|
- receive:
|
||||||
|
expected_result:
|
||||||
|
- - a: -1
|
||||||
|
b: olleh
|
||||||
|
- World Hello
|
||||||
|
- {}
|
||||||
|
id: 3095372
|
||||||
|
timeout: 1
|
||||||
|
- close: null
|
||||||
|
timeout: 1
|
||||||
|
udp:
|
||||||
|
rcv_port: 9876
|
||||||
|
server: localhost
|
||||||
|
snd_port: 4323
|
||||||
|
- json_rpc:
|
||||||
|
doc: JSONRPC UDP Reception and checking result
|
||||||
|
key: $(test)_FAIL
|
||||||
|
name: Failing UDP JSONRPC reception checking
|
||||||
|
steps:
|
||||||
|
- receive:
|
||||||
|
expected_result:
|
||||||
|
- - a: -1
|
||||||
|
b: ollhe
|
||||||
|
- World Hello
|
||||||
|
- {}
|
||||||
|
id: 3095372
|
||||||
|
timeout: 1
|
||||||
|
- close: null
|
||||||
|
timeout: 1
|
||||||
|
udp:
|
||||||
|
rcv_port: 9876
|
||||||
|
server: localhost
|
||||||
|
snd_port: 4323
|
||||||
|
- json_rpc:
|
||||||
|
doc: 'JSONRPC UDP query waiting for reception and
|
||||||
|
checking result with
|
||||||
|
|
||||||
|
replacing $(result) and evaluating string.
|
||||||
|
|
||||||
|
'
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: UDP JSONRPC query waiting and evaluating result
|
||||||
|
steps:
|
||||||
|
- open: null
|
||||||
|
- query:
|
||||||
|
expected_result: 1
|
||||||
|
method: echo
|
||||||
|
params:
|
||||||
|
- Hello World
|
||||||
|
- a: 1
|
||||||
|
b: hello
|
||||||
|
process_result: $(result)[0][1]['a']
|
||||||
|
timeout: 1
|
||||||
|
- close: null
|
||||||
|
timeout: 1
|
||||||
|
udp:
|
||||||
|
rcv_port: 48393
|
||||||
|
server: localhost
|
||||||
|
snd_port: 4323
|
||||||
|
- json_rpc:
|
||||||
|
doc: 'JSONRPC UDP query waiting for reception and
|
||||||
|
checking result with
|
||||||
|
|
||||||
|
replacing $(result) and evaluating string.
|
||||||
|
|
||||||
|
'
|
||||||
|
key: $(test)_FAIL
|
||||||
|
name: Failing UDP JSONRPC query waiting and evaluating
|
||||||
|
result
|
||||||
|
steps:
|
||||||
|
- open: null
|
||||||
|
- query:
|
||||||
|
expected_result: $(result)[0][1]['a'] == 0
|
||||||
|
method: echo
|
||||||
|
params:
|
||||||
|
- Hello World
|
||||||
|
- a: 1
|
||||||
|
b: hello
|
||||||
|
timeout: 1
|
||||||
|
- close: null
|
||||||
|
timeout: 1
|
||||||
|
udp:
|
||||||
|
rcv_port: 48393
|
||||||
|
server: localhost
|
||||||
|
snd_port: 4323
|
||||||
|
- console:
|
||||||
|
console_name: jrpces
|
||||||
|
doc: check if the jsonrpc echo server is installed
|
||||||
|
execute_on_stop: true
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Stop json rpc echo server
|
||||||
|
steps:
|
||||||
|
- close:
|
||||||
|
protocol: terminal
|
||||||
|
filename: /home/renish/workspace/testium/code/test/validation/items/jsonrpc/test.tum
|
||||||
|
- sequence:
|
||||||
|
steps:
|
||||||
|
- report:
|
||||||
|
export:
|
||||||
|
- text:
|
||||||
|
key: $(test)_PASS
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_PASS.txt
|
||||||
|
- html:
|
||||||
|
key: $(test)_PASS
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_PASS.html
|
||||||
|
- junit:
|
||||||
|
key: $(test)_PASS
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_PASS.junit
|
||||||
|
name: Expected PASS $(test) test
|
||||||
|
- report:
|
||||||
|
export:
|
||||||
|
- text:
|
||||||
|
key: $(test)_FAIL
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_FAIL.txt
|
||||||
|
- html:
|
||||||
|
key: $(test)_FAIL
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_FAIL.html
|
||||||
|
- junit:
|
||||||
|
key: $(test)_FAIL
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_FAIL.junit
|
||||||
|
name: Expected FAIL $(test) test
|
||||||
|
filename: /home/renish/workspace/testium/code/test/validation/items/report.tum
|
||||||
|
report:
|
||||||
|
enabled: true
|
||||||
|
export:
|
||||||
|
junit:
|
||||||
|
file_name: $(validation_report_file).junit
|
||||||
|
path: $(validation_report_path)
|
||||||
|
sqlite:
|
||||||
|
file_name: $(validation_report_file).sqlite
|
||||||
|
path: $(validation_report_path)
|
||||||
|
log_stored: true
|
||||||
157
schema/test_schema/let.tum
Normal file
157
schema/test_schema/let.tum
Normal file
@@ -0,0 +1,157 @@
|
|||||||
|
config_file:
|
||||||
|
- param.yaml
|
||||||
|
- items/let/param.yaml
|
||||||
|
main:
|
||||||
|
name: Testium validation suite
|
||||||
|
steps:
|
||||||
|
- group:
|
||||||
|
name: Test preparation
|
||||||
|
steps:
|
||||||
|
- let:
|
||||||
|
condition: <| "$(os)" == "Linux" |>
|
||||||
|
name: Set test variables for Linux
|
||||||
|
values:
|
||||||
|
- terminal_prompt: $(linux_prompt)
|
||||||
|
- psep: /
|
||||||
|
- let:
|
||||||
|
condition: <| "$(os)" == "Windows" |>
|
||||||
|
name: Set test variables for Windows
|
||||||
|
values:
|
||||||
|
- terminal_prompt: $(windows_prompt)
|
||||||
|
- psep: \
|
||||||
|
- group:
|
||||||
|
name: Group of tests
|
||||||
|
steps:
|
||||||
|
- let:
|
||||||
|
name: let test constants
|
||||||
|
values:
|
||||||
|
- test: let
|
||||||
|
- test_path: items/$(test)
|
||||||
|
- group:
|
||||||
|
name: let test
|
||||||
|
steps:
|
||||||
|
- sequence:
|
||||||
|
steps:
|
||||||
|
- loop:
|
||||||
|
iterator: 10
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Cycle number of loops
|
||||||
|
steps:
|
||||||
|
- py_func:
|
||||||
|
file: $(test_path)$(psep)let.py
|
||||||
|
func_name: donothing
|
||||||
|
name: do nothing
|
||||||
|
- let:
|
||||||
|
name: Let it be
|
||||||
|
values:
|
||||||
|
- be: <| $(loop_param) == $(it) |>
|
||||||
|
- it: $(loop_param)
|
||||||
|
- loop:
|
||||||
|
iterator:
|
||||||
|
- 12
|
||||||
|
- 20
|
||||||
|
- 30
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Cycle iterating on list
|
||||||
|
steps:
|
||||||
|
- py_func:
|
||||||
|
file: $(test_path)$(psep)let.py
|
||||||
|
func_name: checkloopparam
|
||||||
|
name: check loop param
|
||||||
|
param:
|
||||||
|
- $(loop_param)
|
||||||
|
- let:
|
||||||
|
name: Let it be
|
||||||
|
values:
|
||||||
|
- it: $(loop_param)
|
||||||
|
- be: <| $(loop_param) == $(it) |>
|
||||||
|
- let:
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Get time
|
||||||
|
values:
|
||||||
|
- loop_t0: $(ts_start_Cycle iterating on list)
|
||||||
|
- loop_t1: $(ts_end_Cycle iterating on list)
|
||||||
|
- loop_duration: $(ts_duration_Cycle iterating on list)
|
||||||
|
- let:
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Get parameter file value
|
||||||
|
values:
|
||||||
|
- test_overwrite_me: <| $(overwrite_me) == True |>
|
||||||
|
- py_func:
|
||||||
|
file: $(test_path)$(psep)let.py
|
||||||
|
func_name: checkGlobalDic
|
||||||
|
name: Check global dic pass
|
||||||
|
param:
|
||||||
|
- test_overwrite_me
|
||||||
|
- true
|
||||||
|
- let:
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Overwrite parameter file value
|
||||||
|
values:
|
||||||
|
- overwrite_me: false
|
||||||
|
- py_func:
|
||||||
|
expected_result: $(overwrite_me) == False
|
||||||
|
file: $(test_path)$(psep)let.py
|
||||||
|
func_name: checkGlobalDic
|
||||||
|
key: $(test)_FAIL
|
||||||
|
name: Check global dic fail
|
||||||
|
param:
|
||||||
|
- overwrite_me
|
||||||
|
- true
|
||||||
|
- py_func:
|
||||||
|
expected_result: fail
|
||||||
|
file: $(test_path)$(psep)let.py
|
||||||
|
func_name: checkGlobalDic
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Check global dic fail
|
||||||
|
param:
|
||||||
|
- overwrite_me
|
||||||
|
- true
|
||||||
|
- let:
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Evaluate Overwriting parameter file value
|
||||||
|
values:
|
||||||
|
- test_overwrite_me: <| "$(overwrite_me)" == True |>
|
||||||
|
- check:
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Check Overwriting parameter file value
|
||||||
|
values:
|
||||||
|
- <| $(test_overwrite_me) == False |>
|
||||||
|
filename: /home/renish/workspace/testium/code/test/validation/items/let/test.tum
|
||||||
|
- sequence:
|
||||||
|
steps:
|
||||||
|
- report:
|
||||||
|
export:
|
||||||
|
- text:
|
||||||
|
key: $(test)_PASS
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_PASS.txt
|
||||||
|
- html:
|
||||||
|
key: $(test)_PASS
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_PASS.html
|
||||||
|
- junit:
|
||||||
|
key: $(test)_PASS
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_PASS.junit
|
||||||
|
name: Expected PASS $(test) test
|
||||||
|
- report:
|
||||||
|
export:
|
||||||
|
- text:
|
||||||
|
key: $(test)_FAIL
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_FAIL.txt
|
||||||
|
- html:
|
||||||
|
key: $(test)_FAIL
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_FAIL.html
|
||||||
|
- junit:
|
||||||
|
key: $(test)_FAIL
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_FAIL.junit
|
||||||
|
name: Expected FAIL $(test) test
|
||||||
|
filename: /home/renish/workspace/testium/code/test/validation/items/report.tum
|
||||||
|
report:
|
||||||
|
enabled: true
|
||||||
|
export:
|
||||||
|
junit:
|
||||||
|
file_name: $(validation_report_file).junit
|
||||||
|
path: $(validation_report_path)
|
||||||
|
sqlite:
|
||||||
|
file_name: $(validation_report_file).sqlite
|
||||||
|
path: $(validation_report_path)
|
||||||
|
log_stored: true
|
||||||
297
schema/test_schema/lua_func.tum
Normal file
297
schema/test_schema/lua_func.tum
Normal file
@@ -0,0 +1,297 @@
|
|||||||
|
config_file:
|
||||||
|
- param.yaml
|
||||||
|
- items/lua_func/param.yaml
|
||||||
|
main:
|
||||||
|
name: Testium validation suite
|
||||||
|
steps:
|
||||||
|
- group:
|
||||||
|
name: Test preparation
|
||||||
|
steps:
|
||||||
|
- let:
|
||||||
|
condition: <| "$(os)" == "Linux" |>
|
||||||
|
name: Set test variables for Linux
|
||||||
|
values:
|
||||||
|
- terminal_prompt: $(linux_prompt)
|
||||||
|
- psep: /
|
||||||
|
- let:
|
||||||
|
condition: <| "$(os)" == "Windows" |>
|
||||||
|
name: Set test variables for Windows
|
||||||
|
values:
|
||||||
|
- terminal_prompt: $(windows_prompt)
|
||||||
|
- psep: \
|
||||||
|
- group:
|
||||||
|
name: Group of tests
|
||||||
|
steps:
|
||||||
|
- let:
|
||||||
|
name: lua_func test constants
|
||||||
|
values:
|
||||||
|
test: lua_func
|
||||||
|
test_path: items/$(test)
|
||||||
|
- group:
|
||||||
|
name: lua_func test
|
||||||
|
steps:
|
||||||
|
- sequence:
|
||||||
|
steps:
|
||||||
|
- let:
|
||||||
|
name: lua_func test constants,
|
||||||
|
values:
|
||||||
|
lua_func test parameter: test parameter lua_func
|
||||||
|
- lua_func:
|
||||||
|
file: $(test_path)$(psep)lua_func.lua
|
||||||
|
func_name: assertparam
|
||||||
|
key: $(test)_FAIL
|
||||||
|
name: fail lua_func
|
||||||
|
param:
|
||||||
|
- false
|
||||||
|
- lua_func:
|
||||||
|
expected_result: FAIL
|
||||||
|
file: $(test_path)$(psep)lua_func.lua
|
||||||
|
func_name: assertparam
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: fail lua_func with expected result FAIL
|
||||||
|
param:
|
||||||
|
- false
|
||||||
|
- lua_func:
|
||||||
|
expected_result: FAIL
|
||||||
|
file: $(test_path)$(psep)lua_func.lua
|
||||||
|
func_name: assertparam
|
||||||
|
key: $(test)_FAIL
|
||||||
|
name: pass lua_func with expected result FAIL
|
||||||
|
param:
|
||||||
|
- true
|
||||||
|
- lua_func:
|
||||||
|
expected_result: -1
|
||||||
|
file: $(test_path)$(psep)lua_func.lua
|
||||||
|
func_name: echo
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: expected -1
|
||||||
|
param:
|
||||||
|
- -1
|
||||||
|
- lua_func:
|
||||||
|
expected_result: 354848436 - 354848437
|
||||||
|
file: $(test_path)$(psep)lua_func.lua
|
||||||
|
func_name: echo
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: expected eval
|
||||||
|
param:
|
||||||
|
- -1
|
||||||
|
- lua_func:
|
||||||
|
expected_result: '[-1, ''a'', {''toto'': ''tata''}]'
|
||||||
|
file: $(test_path)$(psep)lua_func.lua
|
||||||
|
func_name: echo
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: expected table
|
||||||
|
param:
|
||||||
|
- - -1
|
||||||
|
- a
|
||||||
|
- toto: tata
|
||||||
|
- lua_func:
|
||||||
|
expected_result: $(lua_func test parameter)
|
||||||
|
file: $(test_path)$(psep)lua_func.lua
|
||||||
|
func_name: checkglobal
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: global param lua_func
|
||||||
|
param:
|
||||||
|
- lua_func test parameter
|
||||||
|
- lua_func:
|
||||||
|
expected_result: ($(lua_data_to_be_returned))[0]
|
||||||
|
file: $(test_path)$(psep)lua_func.lua
|
||||||
|
func_name: checkglobal2
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: global param lua_func 1
|
||||||
|
param:
|
||||||
|
- 1
|
||||||
|
- lua_func:
|
||||||
|
expected_result: ($(lua_data_to_be_returned))[1]
|
||||||
|
file: $(test_path)$(psep)lua_func.lua
|
||||||
|
func_name: checkglobal2
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: global param lua_func 2
|
||||||
|
param:
|
||||||
|
- 2
|
||||||
|
- lua_func:
|
||||||
|
expected_result: ($(lua_data_to_be_returned))[2]
|
||||||
|
file: $(test_path)$(psep)lua_func.lua
|
||||||
|
func_name: checkglobal2
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: global param lua_func 3
|
||||||
|
param:
|
||||||
|
- 3
|
||||||
|
- let:
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: python2func
|
||||||
|
values:
|
||||||
|
- py: $(test_path)$(psep)lua_func.lua
|
||||||
|
- lua_func:
|
||||||
|
file: $(test_path)$(psep)lua_func.lua
|
||||||
|
func_name: should_not_be_called
|
||||||
|
name: skipped_checkglobal
|
||||||
|
param:
|
||||||
|
- $(test parameter)
|
||||||
|
- lua_func:
|
||||||
|
file: $(test_path)$(psep)lua_func.lua
|
||||||
|
func_name: checkglobal
|
||||||
|
name: skipped true
|
||||||
|
param:
|
||||||
|
- $(test parameter)
|
||||||
|
skipped: true
|
||||||
|
- lua_func:
|
||||||
|
file: $(test_path)$(psep)lua_func.lua
|
||||||
|
func_name: checkglobal
|
||||||
|
name: skipped 1
|
||||||
|
param:
|
||||||
|
- $(test parameter)
|
||||||
|
skipped: true
|
||||||
|
- group:
|
||||||
|
name: Function results check
|
||||||
|
steps:
|
||||||
|
- group:
|
||||||
|
name: Function result failure
|
||||||
|
steps:
|
||||||
|
- lua_func:
|
||||||
|
file: $(test_path)$(psep)lua_func.lua
|
||||||
|
func_name: echo
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: int failure
|
||||||
|
param:
|
||||||
|
- -1
|
||||||
|
- lua_func:
|
||||||
|
file: $(test_path)$(psep)lua_func.lua
|
||||||
|
func_name: echo
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: float failure
|
||||||
|
param:
|
||||||
|
- -1.3
|
||||||
|
- lua_func:
|
||||||
|
file: $(test_path)$(psep)lua_func.lua
|
||||||
|
func_name: echo
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: String failure
|
||||||
|
param:
|
||||||
|
- FAIL
|
||||||
|
- lua_func:
|
||||||
|
file: $(test_path)$(psep)lua_func.lua
|
||||||
|
func_name: tuple_return
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Tuple int,str failure
|
||||||
|
param:
|
||||||
|
- -1
|
||||||
|
- Got a failure
|
||||||
|
- group:
|
||||||
|
name: Functions result success
|
||||||
|
steps:
|
||||||
|
- lua_func:
|
||||||
|
file: $(test_path)$(psep)lua_func.lua
|
||||||
|
func_name: echo
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: int success
|
||||||
|
param:
|
||||||
|
- 0
|
||||||
|
- lua_func:
|
||||||
|
file: $(test_path)$(psep)lua_func.lua
|
||||||
|
func_name: echo
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: float success
|
||||||
|
param:
|
||||||
|
- 0.3
|
||||||
|
- lua_func:
|
||||||
|
file: $(test_path)$(psep)lua_func.lua
|
||||||
|
func_name: echo
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: String success
|
||||||
|
param:
|
||||||
|
- Something that is not only strictly FAIL
|
||||||
|
- lua_func:
|
||||||
|
file: $(test_path)$(psep)lua_func.lua
|
||||||
|
func_name: tuple_return
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Tuple int,str success
|
||||||
|
param:
|
||||||
|
- 0
|
||||||
|
- OK
|
||||||
|
- lua_func:
|
||||||
|
file: $(test_path)$(psep)lua_func.lua
|
||||||
|
func_name: test_delgd
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: delgd test
|
||||||
|
- lua_func:
|
||||||
|
file: $(test_path)$(psep)lua_func.lua
|
||||||
|
func_name: return_nothing
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: function returning nothing should succeed
|
||||||
|
- lua_func:
|
||||||
|
file: $(test_path)$(psep)lua_func.lua
|
||||||
|
func_name: return_explicit_nil
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: function returning explicit nil should succeed
|
||||||
|
- group:
|
||||||
|
name: context_id tests
|
||||||
|
steps:
|
||||||
|
- lua_func:
|
||||||
|
context_id: lua_ctx_test
|
||||||
|
expected_result: hello lua
|
||||||
|
file: $(test_path)$(psep)lua_func.lua
|
||||||
|
func_name: set_context_value
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: set context value
|
||||||
|
param:
|
||||||
|
- hello lua
|
||||||
|
- lua_func:
|
||||||
|
context_id: lua_ctx_test
|
||||||
|
expected_result: hello lua
|
||||||
|
file: $(test_path)$(psep)lua_func.lua
|
||||||
|
func_name: get_context_value
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: get context value (same context_id)
|
||||||
|
- lua_func:
|
||||||
|
expected_result: hello lua
|
||||||
|
file: $(test_path)$(psep)lua_func.lua
|
||||||
|
func_name: get_context_value
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: get context value (no context_id, from main
|
||||||
|
gd)
|
||||||
|
- lua_func:
|
||||||
|
context_id: lua_ctx_other
|
||||||
|
expected_result: hello lua
|
||||||
|
file: $(test_path)$(psep)lua_func.lua
|
||||||
|
func_name: get_context_value
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: get context value (different context_id)
|
||||||
|
filename: /home/renish/workspace/testium/code/test/validation/items/lua_func/test.tum
|
||||||
|
- sequence:
|
||||||
|
steps:
|
||||||
|
- report:
|
||||||
|
export:
|
||||||
|
- text:
|
||||||
|
key: $(test)_PASS
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_PASS.txt
|
||||||
|
- html:
|
||||||
|
key: $(test)_PASS
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_PASS.html
|
||||||
|
- junit:
|
||||||
|
key: $(test)_PASS
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_PASS.junit
|
||||||
|
name: Expected PASS $(test) test
|
||||||
|
- report:
|
||||||
|
export:
|
||||||
|
- text:
|
||||||
|
key: $(test)_FAIL
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_FAIL.txt
|
||||||
|
- html:
|
||||||
|
key: $(test)_FAIL
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_FAIL.html
|
||||||
|
- junit:
|
||||||
|
key: $(test)_FAIL
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_FAIL.junit
|
||||||
|
name: Expected FAIL $(test) test
|
||||||
|
filename: /home/renish/workspace/testium/code/test/validation/items/report.tum
|
||||||
|
report:
|
||||||
|
enabled: true
|
||||||
|
export:
|
||||||
|
junit:
|
||||||
|
file_name: $(validation_report_file).junit
|
||||||
|
path: $(validation_report_path)
|
||||||
|
sqlite:
|
||||||
|
file_name: $(validation_report_file).sqlite
|
||||||
|
path: $(validation_report_path)
|
||||||
|
log_stored: true
|
||||||
414
schema/test_schema/parallel.tum
Normal file
414
schema/test_schema/parallel.tum
Normal file
@@ -0,0 +1,414 @@
|
|||||||
|
config_file:
|
||||||
|
- param.yaml
|
||||||
|
- items/parallel/param.yaml
|
||||||
|
main:
|
||||||
|
name: Testium validation suite
|
||||||
|
steps:
|
||||||
|
- group:
|
||||||
|
name: Test preparation
|
||||||
|
steps:
|
||||||
|
- let:
|
||||||
|
condition: <| "$(os)" == "Linux" |>
|
||||||
|
name: Set test variables for Linux
|
||||||
|
values:
|
||||||
|
- terminal_prompt: $(linux_prompt)
|
||||||
|
- psep: /
|
||||||
|
- let:
|
||||||
|
condition: <| "$(os)" == "Windows" |>
|
||||||
|
name: Set test variables for Windows
|
||||||
|
values:
|
||||||
|
- terminal_prompt: $(windows_prompt)
|
||||||
|
- psep: \
|
||||||
|
- group:
|
||||||
|
name: Group of tests
|
||||||
|
steps:
|
||||||
|
- let:
|
||||||
|
name: parallel test constants
|
||||||
|
values:
|
||||||
|
test: parallel
|
||||||
|
test_path: items/$(test)
|
||||||
|
- group:
|
||||||
|
name: parallel test
|
||||||
|
steps:
|
||||||
|
- sequence:
|
||||||
|
steps:
|
||||||
|
- parallel:
|
||||||
|
branches:
|
||||||
|
- name: Branch A
|
||||||
|
steps:
|
||||||
|
- let:
|
||||||
|
name: Set A done
|
||||||
|
values:
|
||||||
|
- branch_a_done: true
|
||||||
|
- name: Branch B
|
||||||
|
steps:
|
||||||
|
- let:
|
||||||
|
name: Set B done
|
||||||
|
values:
|
||||||
|
- branch_b_done: true
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Both branches pass
|
||||||
|
sync: all
|
||||||
|
- check:
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Both branches ran
|
||||||
|
values:
|
||||||
|
- <| $(branch_a_done) == True |>
|
||||||
|
- <| $(branch_b_done) == True |>
|
||||||
|
- parallel:
|
||||||
|
branches:
|
||||||
|
- name: Pass branch
|
||||||
|
steps:
|
||||||
|
- let:
|
||||||
|
name: Set pass flag
|
||||||
|
values:
|
||||||
|
- pass_branch_ran: true
|
||||||
|
- name: Fail branch
|
||||||
|
steps:
|
||||||
|
- py_func:
|
||||||
|
expected_result: fail
|
||||||
|
file: $(test_path)$(psep)parallel.py
|
||||||
|
func_name: sleep_func
|
||||||
|
name: Raise exception
|
||||||
|
param:
|
||||||
|
- 0
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: One branch fails
|
||||||
|
no_fail: true
|
||||||
|
sync: all
|
||||||
|
- check:
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Pass branch still ran
|
||||||
|
values:
|
||||||
|
- <| $(pass_branch_ran) == True |>
|
||||||
|
- let:
|
||||||
|
name: Reset slow flag
|
||||||
|
values:
|
||||||
|
- slow_done: false
|
||||||
|
- parallel:
|
||||||
|
branches:
|
||||||
|
- name: Fast branch
|
||||||
|
steps:
|
||||||
|
- let:
|
||||||
|
name: Fast done
|
||||||
|
values:
|
||||||
|
- fast_done: true
|
||||||
|
- name: Slow branch
|
||||||
|
steps:
|
||||||
|
- py_func:
|
||||||
|
file: $(test_path)$(psep)parallel.py
|
||||||
|
func_name: sleep_func
|
||||||
|
name: Sleep 2s
|
||||||
|
param:
|
||||||
|
- 2
|
||||||
|
- let:
|
||||||
|
name: Slow done
|
||||||
|
values:
|
||||||
|
- slow_done: true
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: sync any - first wins
|
||||||
|
sync: any
|
||||||
|
- check:
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Fast branch ran, slow branch was stopped
|
||||||
|
values:
|
||||||
|
- <| $(fast_done) == True |>
|
||||||
|
- <| $(slow_done) == False |>
|
||||||
|
- let:
|
||||||
|
name: Reset sync flag
|
||||||
|
values:
|
||||||
|
- sync_flag: ''
|
||||||
|
- waiter_ran: false
|
||||||
|
- parallel:
|
||||||
|
branches:
|
||||||
|
- name: Setter branch
|
||||||
|
steps:
|
||||||
|
- py_func:
|
||||||
|
file: $(test_path)$(psep)parallel.py
|
||||||
|
func_name: sleep_func
|
||||||
|
name: Sleep 0.3s then set flag
|
||||||
|
param:
|
||||||
|
- 0.3
|
||||||
|
- let:
|
||||||
|
name: Set sync flag
|
||||||
|
values:
|
||||||
|
- sync_flag: ready
|
||||||
|
- name: Waiter branch
|
||||||
|
steps:
|
||||||
|
- let:
|
||||||
|
name: Got flag
|
||||||
|
values:
|
||||||
|
- waiter_ran: true
|
||||||
|
wait_for:
|
||||||
|
condition: <| "$(sync_flag)" == "ready" |>
|
||||||
|
timeout: 10
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: wait_for synchronization
|
||||||
|
sync: all
|
||||||
|
- check:
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Waiter branch ran after flag was set
|
||||||
|
values:
|
||||||
|
- <| $(waiter_ran) == True |>
|
||||||
|
- parallel:
|
||||||
|
branches:
|
||||||
|
- name: Sleep A
|
||||||
|
steps:
|
||||||
|
- sleep:
|
||||||
|
name: Sleep 1s A
|
||||||
|
timeout: 1
|
||||||
|
- name: Sleep B
|
||||||
|
steps:
|
||||||
|
- sleep:
|
||||||
|
name: Sleep 1s B
|
||||||
|
timeout: 1
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Timing test
|
||||||
|
sync: all
|
||||||
|
- let:
|
||||||
|
name: Capture parallel duration
|
||||||
|
values:
|
||||||
|
- parallel_duration: $(ts_duration_Timing test)
|
||||||
|
- check:
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Duration < 1.8s (would be 2s if sequential)
|
||||||
|
values:
|
||||||
|
- <| float("$(parallel_duration)") < 1.8 |>
|
||||||
|
- let:
|
||||||
|
name: Reset N flags
|
||||||
|
values:
|
||||||
|
- n_a: false
|
||||||
|
- n_b: false
|
||||||
|
- n_c: false
|
||||||
|
- n_d: false
|
||||||
|
- parallel:
|
||||||
|
branches:
|
||||||
|
- name: NA
|
||||||
|
steps:
|
||||||
|
- let:
|
||||||
|
name: set n_a
|
||||||
|
values:
|
||||||
|
- n_a: true
|
||||||
|
- name: NB
|
||||||
|
steps:
|
||||||
|
- let:
|
||||||
|
name: set n_b
|
||||||
|
values:
|
||||||
|
- n_b: true
|
||||||
|
- name: NC
|
||||||
|
steps:
|
||||||
|
- let:
|
||||||
|
name: set n_c
|
||||||
|
values:
|
||||||
|
- n_c: true
|
||||||
|
- name: ND
|
||||||
|
steps:
|
||||||
|
- let:
|
||||||
|
name: set n_d
|
||||||
|
values:
|
||||||
|
- n_d: true
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Four branches
|
||||||
|
sync: all
|
||||||
|
- check:
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Four branches all set their flag
|
||||||
|
values:
|
||||||
|
- <| $(n_a) == True |>
|
||||||
|
- <| $(n_b) == True |>
|
||||||
|
- <| $(n_c) == True |>
|
||||||
|
- <| $(n_d) == True |>
|
||||||
|
- let:
|
||||||
|
name: Reset nested flags
|
||||||
|
values:
|
||||||
|
- outer_x: false
|
||||||
|
- inner_x_1: false
|
||||||
|
- inner_x_2: false
|
||||||
|
- parallel:
|
||||||
|
branches:
|
||||||
|
- name: Outer X
|
||||||
|
steps:
|
||||||
|
- let:
|
||||||
|
name: set outer_x
|
||||||
|
values:
|
||||||
|
- outer_x: true
|
||||||
|
- parallel:
|
||||||
|
branches:
|
||||||
|
- name: Inner X1
|
||||||
|
steps:
|
||||||
|
- let:
|
||||||
|
name: set inner_x_1
|
||||||
|
values:
|
||||||
|
- inner_x_1: true
|
||||||
|
- name: Inner X2
|
||||||
|
steps:
|
||||||
|
- let:
|
||||||
|
name: set inner_x_2
|
||||||
|
values:
|
||||||
|
- inner_x_2: true
|
||||||
|
name: Inner parallel
|
||||||
|
sync: all
|
||||||
|
- name: Outer Y
|
||||||
|
steps:
|
||||||
|
- sleep:
|
||||||
|
name: brief sleep
|
||||||
|
timeout: 0
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Outer parallel
|
||||||
|
sync: all
|
||||||
|
- check:
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Nested parallel set all flags
|
||||||
|
values:
|
||||||
|
- <| $(outer_x) == True |>
|
||||||
|
- <| $(inner_x_1) == True |>
|
||||||
|
- <| $(inner_x_2) == True |>
|
||||||
|
- let:
|
||||||
|
name: Reset waiter timeout flag
|
||||||
|
values:
|
||||||
|
- waiter_timeout_ran: false
|
||||||
|
- parallel:
|
||||||
|
branches:
|
||||||
|
- name: Quick branch
|
||||||
|
steps:
|
||||||
|
- sleep:
|
||||||
|
name: brief sleep
|
||||||
|
timeout: 0
|
||||||
|
- name: Doomed waiter
|
||||||
|
steps:
|
||||||
|
- let:
|
||||||
|
name: should not run
|
||||||
|
values:
|
||||||
|
- waiter_timeout_ran: true
|
||||||
|
wait_for:
|
||||||
|
condition: <| "never" == "ready" |>
|
||||||
|
timeout: 1
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: wait_for timeout
|
||||||
|
no_fail: true
|
||||||
|
sync: all
|
||||||
|
- check:
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Doomed waiter never ran its steps
|
||||||
|
values:
|
||||||
|
- <| $(waiter_timeout_ran) == False |>
|
||||||
|
- parallel:
|
||||||
|
branches:
|
||||||
|
- name: ok branch
|
||||||
|
steps:
|
||||||
|
- let:
|
||||||
|
name: noop
|
||||||
|
values:
|
||||||
|
- noop_var: 1
|
||||||
|
- name: broken branch
|
||||||
|
steps:
|
||||||
|
- py_func:
|
||||||
|
expected_result: fail
|
||||||
|
file: $(test_path)$(psep)parallel.py
|
||||||
|
func_name: sleep_func
|
||||||
|
name: Forced fail
|
||||||
|
param:
|
||||||
|
- 0
|
||||||
|
key: $(test)_FAIL
|
||||||
|
name: One branch really fails
|
||||||
|
sync: all
|
||||||
|
- let:
|
||||||
|
name: Reset branch condition flag
|
||||||
|
values:
|
||||||
|
- cond_branch_ran: false
|
||||||
|
- other_branch_ran: false
|
||||||
|
- parallel:
|
||||||
|
branches:
|
||||||
|
- condition: <| "always" == "false" |>
|
||||||
|
name: Skipped branch
|
||||||
|
steps:
|
||||||
|
- let:
|
||||||
|
name: should not run
|
||||||
|
values:
|
||||||
|
- cond_branch_ran: true
|
||||||
|
- name: Other branch
|
||||||
|
steps:
|
||||||
|
- let:
|
||||||
|
name: ran
|
||||||
|
values:
|
||||||
|
- other_branch_ran: true
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Condition-skipped branch
|
||||||
|
sync: all
|
||||||
|
- check:
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Skipped condition branch did not run
|
||||||
|
values:
|
||||||
|
- <| $(cond_branch_ran) == False |>
|
||||||
|
- <| $(other_branch_ran) == True |>
|
||||||
|
- let:
|
||||||
|
name: Reset loop counters
|
||||||
|
values:
|
||||||
|
- loop_count_a: 0
|
||||||
|
- loop_count_b: 0
|
||||||
|
- loop:
|
||||||
|
iterator: 3
|
||||||
|
name: Loop wrapping parallel
|
||||||
|
steps:
|
||||||
|
- parallel:
|
||||||
|
branches:
|
||||||
|
- name: LA
|
||||||
|
steps:
|
||||||
|
- let:
|
||||||
|
name: bump A
|
||||||
|
values:
|
||||||
|
- loop_count_a: <| int("$(loop_count_a)")
|
||||||
|
+ 1 |>
|
||||||
|
- name: LB
|
||||||
|
steps:
|
||||||
|
- let:
|
||||||
|
name: bump B
|
||||||
|
values:
|
||||||
|
- loop_count_b: <| int("$(loop_count_b)")
|
||||||
|
+ 1 |>
|
||||||
|
name: Per-iteration parallel
|
||||||
|
sync: all
|
||||||
|
- check:
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Both branches ran 3 times
|
||||||
|
values:
|
||||||
|
- <| int("$(loop_count_a)") == 3 |>
|
||||||
|
- <| int("$(loop_count_b)") == 3 |>
|
||||||
|
filename: /home/renish/workspace/testium/code/test/validation/items/parallel/test.tum
|
||||||
|
- sequence:
|
||||||
|
steps:
|
||||||
|
- report:
|
||||||
|
export:
|
||||||
|
- text:
|
||||||
|
key: $(test)_PASS
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_PASS.txt
|
||||||
|
- html:
|
||||||
|
key: $(test)_PASS
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_PASS.html
|
||||||
|
- junit:
|
||||||
|
key: $(test)_PASS
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_PASS.junit
|
||||||
|
name: Expected PASS $(test) test
|
||||||
|
- report:
|
||||||
|
export:
|
||||||
|
- text:
|
||||||
|
key: $(test)_FAIL
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_FAIL.txt
|
||||||
|
- html:
|
||||||
|
key: $(test)_FAIL
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_FAIL.html
|
||||||
|
- junit:
|
||||||
|
key: $(test)_FAIL
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_FAIL.junit
|
||||||
|
name: Expected FAIL $(test) test
|
||||||
|
filename: /home/renish/workspace/testium/code/test/validation/items/report.tum
|
||||||
|
report:
|
||||||
|
enabled: true
|
||||||
|
export:
|
||||||
|
junit:
|
||||||
|
file_name: $(validation_report_file).junit
|
||||||
|
path: $(validation_report_path)
|
||||||
|
sqlite:
|
||||||
|
file_name: $(validation_report_file).sqlite
|
||||||
|
path: $(validation_report_path)
|
||||||
|
log_stored: true
|
||||||
137
schema/test_schema/plot.tum
Normal file
137
schema/test_schema/plot.tum
Normal file
@@ -0,0 +1,137 @@
|
|||||||
|
config_file:
|
||||||
|
- param.yaml
|
||||||
|
- items/plot/param.yaml
|
||||||
|
main:
|
||||||
|
name: Testium validation suite
|
||||||
|
steps:
|
||||||
|
- group:
|
||||||
|
name: Test preparation
|
||||||
|
steps:
|
||||||
|
- let:
|
||||||
|
condition: <| "$(os)" == "Linux" |>
|
||||||
|
name: Set test variables for Linux
|
||||||
|
values:
|
||||||
|
- terminal_prompt: $(linux_prompt)
|
||||||
|
- psep: /
|
||||||
|
- let:
|
||||||
|
condition: <| "$(os)" == "Windows" |>
|
||||||
|
name: Set test variables for Windows
|
||||||
|
values:
|
||||||
|
- terminal_prompt: $(windows_prompt)
|
||||||
|
- psep: \
|
||||||
|
- group:
|
||||||
|
name: Group of tests
|
||||||
|
steps:
|
||||||
|
- let:
|
||||||
|
name: plot test constants
|
||||||
|
values:
|
||||||
|
test: plot
|
||||||
|
test_path: items/$(test)
|
||||||
|
- group:
|
||||||
|
name: plot test
|
||||||
|
steps:
|
||||||
|
- sequence:
|
||||||
|
steps:
|
||||||
|
- group:
|
||||||
|
condition: <| $(validation_dialogs) and not tm.text_mode()
|
||||||
|
|>
|
||||||
|
name: Plot test
|
||||||
|
steps:
|
||||||
|
- plot:
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Open the plot
|
||||||
|
plot_name: Mon Plot
|
||||||
|
steps:
|
||||||
|
- open:
|
||||||
|
log_path: $(validation_report_path)
|
||||||
|
- plot:
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Add periodic to the plot
|
||||||
|
plot_name: Mon Plot
|
||||||
|
steps:
|
||||||
|
- periodic:
|
||||||
|
eval: '{"periodic": $(result)}'
|
||||||
|
file: $(test_path)$(psep)plot.py
|
||||||
|
func_name: random_value
|
||||||
|
period: 1
|
||||||
|
- sleep:
|
||||||
|
dialog: true
|
||||||
|
name: sleep
|
||||||
|
timeout: 3
|
||||||
|
- loop:
|
||||||
|
iterator: 10
|
||||||
|
name: Add of other data in the plot
|
||||||
|
steps:
|
||||||
|
- plot:
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Add to the plot
|
||||||
|
plot_name: Mon Plot
|
||||||
|
steps:
|
||||||
|
- add:
|
||||||
|
value1: $(loop_index)
|
||||||
|
value2: $(loop_index)+2
|
||||||
|
- sleep:
|
||||||
|
name: sleep between values
|
||||||
|
timeout: 1
|
||||||
|
- py_func:
|
||||||
|
file: $(test_path)$(psep)plot.py
|
||||||
|
func_name: LastValues
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: last plot values
|
||||||
|
param:
|
||||||
|
- Mon Plot
|
||||||
|
- plot:
|
||||||
|
execute_on_stop: true
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Export
|
||||||
|
plot_name: Mon Plot
|
||||||
|
steps:
|
||||||
|
- export: $(validation_report_path)/plot_export.pdf
|
||||||
|
- export: $(validation_report_path)/plot_export.csv
|
||||||
|
- plot:
|
||||||
|
execute_on_stop: true
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Close the plot
|
||||||
|
plot_name: Mon Plot
|
||||||
|
steps:
|
||||||
|
- close:
|
||||||
|
timeout: 2
|
||||||
|
wait_dialog_exit: true
|
||||||
|
filename: /home/renish/workspace/testium/code/test/validation/items/plot/test.tum
|
||||||
|
- sequence:
|
||||||
|
steps:
|
||||||
|
- report:
|
||||||
|
export:
|
||||||
|
- text:
|
||||||
|
key: $(test)_PASS
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_PASS.txt
|
||||||
|
- html:
|
||||||
|
key: $(test)_PASS
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_PASS.html
|
||||||
|
- junit:
|
||||||
|
key: $(test)_PASS
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_PASS.junit
|
||||||
|
name: Expected PASS $(test) test
|
||||||
|
- report:
|
||||||
|
export:
|
||||||
|
- text:
|
||||||
|
key: $(test)_FAIL
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_FAIL.txt
|
||||||
|
- html:
|
||||||
|
key: $(test)_FAIL
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_FAIL.html
|
||||||
|
- junit:
|
||||||
|
key: $(test)_FAIL
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_FAIL.junit
|
||||||
|
name: Expected FAIL $(test) test
|
||||||
|
filename: /home/renish/workspace/testium/code/test/validation/items/report.tum
|
||||||
|
report:
|
||||||
|
enabled: true
|
||||||
|
export:
|
||||||
|
junit:
|
||||||
|
file_name: $(validation_report_file).junit
|
||||||
|
path: $(validation_report_path)
|
||||||
|
sqlite:
|
||||||
|
file_name: $(validation_report_file).sqlite
|
||||||
|
path: $(validation_report_path)
|
||||||
|
log_stored: true
|
||||||
326
schema/test_schema/py_func.tum
Normal file
326
schema/test_schema/py_func.tum
Normal file
@@ -0,0 +1,326 @@
|
|||||||
|
config_file:
|
||||||
|
- param.yaml
|
||||||
|
- items/py_func/param.yaml
|
||||||
|
main:
|
||||||
|
name: Testium validation suite
|
||||||
|
steps:
|
||||||
|
- group:
|
||||||
|
name: Test preparation
|
||||||
|
steps:
|
||||||
|
- let:
|
||||||
|
condition: <| "$(os)" == "Linux" |>
|
||||||
|
name: Set test variables for Linux
|
||||||
|
values:
|
||||||
|
- terminal_prompt: $(linux_prompt)
|
||||||
|
- psep: /
|
||||||
|
- let:
|
||||||
|
condition: <| "$(os)" == "Windows" |>
|
||||||
|
name: Set test variables for Windows
|
||||||
|
values:
|
||||||
|
- terminal_prompt: $(windows_prompt)
|
||||||
|
- psep: \
|
||||||
|
- group:
|
||||||
|
name: Group of tests
|
||||||
|
steps:
|
||||||
|
- let:
|
||||||
|
name: py_func test constants
|
||||||
|
values:
|
||||||
|
test: py_func
|
||||||
|
test_path: items/$(test)
|
||||||
|
- group:
|
||||||
|
name: py_func test
|
||||||
|
steps:
|
||||||
|
- sequence:
|
||||||
|
steps:
|
||||||
|
- let:
|
||||||
|
name: py_func test constants,
|
||||||
|
values:
|
||||||
|
py_func test parameter: test parameter
|
||||||
|
- py_func:
|
||||||
|
file: $(test_path)$(psep)py_func.py
|
||||||
|
func_name: assertparam
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: pass py_func
|
||||||
|
param:
|
||||||
|
- true
|
||||||
|
- py_func:
|
||||||
|
file: $(test_path)$(psep)py_func.py
|
||||||
|
func_name: assertparam
|
||||||
|
key: $(test)_FAIL
|
||||||
|
name: fail py_func
|
||||||
|
param:
|
||||||
|
- false
|
||||||
|
- py_func:
|
||||||
|
expected_result: FAIL
|
||||||
|
file: $(test_path)$(psep)py_func.py
|
||||||
|
func_name: assertparam
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: fail py_func with expected result "FAIL"
|
||||||
|
param:
|
||||||
|
- false
|
||||||
|
- py_func:
|
||||||
|
expected_result: FAIL
|
||||||
|
file: $(test_path)$(psep)py_func.py
|
||||||
|
func_name: assertparam
|
||||||
|
key: $(test)_FAIL
|
||||||
|
name: pass py_func with expected result FAIL
|
||||||
|
param:
|
||||||
|
- true
|
||||||
|
- py_func:
|
||||||
|
expected_result: -1
|
||||||
|
file: $(test_path)$(psep)py_func.py
|
||||||
|
func_name: echo
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: expected -1
|
||||||
|
param:
|
||||||
|
- -1
|
||||||
|
- py_func:
|
||||||
|
expected_result: 354848436 - 354848437
|
||||||
|
file: $(test_path)$(psep)py_func.py
|
||||||
|
func_name: echo
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: expected eval
|
||||||
|
param:
|
||||||
|
- -1
|
||||||
|
- py_func:
|
||||||
|
expected_result: '[-1, ''a'', {''toto'': ''tata''}]'
|
||||||
|
file: $(test_path)$(psep)py_func.py
|
||||||
|
func_name: echo
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: expected table
|
||||||
|
param:
|
||||||
|
- - -1
|
||||||
|
- a
|
||||||
|
- toto: tata
|
||||||
|
- py_func:
|
||||||
|
file: $(test_path)$(psep)py_func.py
|
||||||
|
func_name: checkglobal
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: global param py_func
|
||||||
|
param:
|
||||||
|
- $(py_func test parameter)
|
||||||
|
- let:
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: python2func
|
||||||
|
values:
|
||||||
|
- py: $(test_path)$(psep)py_func.py
|
||||||
|
- py_func:
|
||||||
|
expected_result: $(py_func test parameter)
|
||||||
|
file: $(py)
|
||||||
|
func_name: checkglobal2
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: global param py_func 2
|
||||||
|
- py_func:
|
||||||
|
file: $(py)
|
||||||
|
func_name: checkglobal
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: global param py_func
|
||||||
|
param:
|
||||||
|
- $(py_func test parameter)
|
||||||
|
- py_func:
|
||||||
|
file: $(test_path)$(psep)py_func.py
|
||||||
|
func_name: should_not_be_called
|
||||||
|
name: skipped_checkglobal
|
||||||
|
param:
|
||||||
|
- $(py_func test parameter)
|
||||||
|
- py_func:
|
||||||
|
file: $(test_path)$(psep)py_func.py
|
||||||
|
func_name: checkglobal
|
||||||
|
name: skipped true
|
||||||
|
param:
|
||||||
|
- $(py_func test parameter)
|
||||||
|
skipped: true
|
||||||
|
- py_func:
|
||||||
|
file: $(test_path)$(psep)py_func.py
|
||||||
|
func_name: checkglobal
|
||||||
|
name: skipped 1
|
||||||
|
param:
|
||||||
|
- $(py_func test parameter)
|
||||||
|
skipped: 1
|
||||||
|
- py_func:
|
||||||
|
file: $(test_path)$(psep)py_func.py
|
||||||
|
func_name: ValidationTest
|
||||||
|
name: FunctionItem test
|
||||||
|
param:
|
||||||
|
- $(py_func test parameter)
|
||||||
|
- group:
|
||||||
|
name: Function results check
|
||||||
|
steps:
|
||||||
|
- group:
|
||||||
|
name: Function result 1
|
||||||
|
steps:
|
||||||
|
- py_func:
|
||||||
|
expected_result: -1
|
||||||
|
file: $(test_path)$(psep)py_func.py
|
||||||
|
func_name: echo
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: int failure
|
||||||
|
param:
|
||||||
|
- -1
|
||||||
|
- py_func:
|
||||||
|
expected_result: -1.3
|
||||||
|
file: $(test_path)$(psep)py_func.py
|
||||||
|
func_name: echo
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: float failure
|
||||||
|
param:
|
||||||
|
- -1.3
|
||||||
|
- py_func:
|
||||||
|
expected_result: FAIL
|
||||||
|
file: $(test_path)$(psep)py_func.py
|
||||||
|
func_name: echo
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: String failure
|
||||||
|
param:
|
||||||
|
- FAIL
|
||||||
|
- py_func:
|
||||||
|
expected_result:
|
||||||
|
- -1
|
||||||
|
- Got a failure
|
||||||
|
file: $(test_path)$(psep)py_func.py
|
||||||
|
func_name: tuple_return
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Tuple int,str failure
|
||||||
|
param:
|
||||||
|
- -1
|
||||||
|
- Got a failure
|
||||||
|
- group:
|
||||||
|
name: Functions result 2
|
||||||
|
steps:
|
||||||
|
- py_func:
|
||||||
|
expected_result: 0
|
||||||
|
file: $(test_path)$(psep)py_func.py
|
||||||
|
func_name: echo
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: int success
|
||||||
|
param:
|
||||||
|
- 0
|
||||||
|
- py_func:
|
||||||
|
expected_result: 0.3
|
||||||
|
file: $(test_path)$(psep)py_func.py
|
||||||
|
func_name: echo
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: float success
|
||||||
|
param:
|
||||||
|
- 0.3
|
||||||
|
- py_func:
|
||||||
|
expected_result: Something that is not only
|
||||||
|
strictly FAIL
|
||||||
|
file: $(test_path)$(psep)py_func.py
|
||||||
|
func_name: echo
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: String success
|
||||||
|
param:
|
||||||
|
- Something that is not only strictly FAIL
|
||||||
|
- py_func:
|
||||||
|
expected_result:
|
||||||
|
- 0
|
||||||
|
- OK
|
||||||
|
file: $(test_path)$(psep)py_func.py
|
||||||
|
func_name: tuple_return
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Tuple int,str success
|
||||||
|
param:
|
||||||
|
- 0
|
||||||
|
- OK
|
||||||
|
- py_func:
|
||||||
|
file: $(test_path)$(psep)py_func.py
|
||||||
|
func_name: test_delgd
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: delgd test
|
||||||
|
- py_func:
|
||||||
|
file: $(test_path)$(psep)py_func.py
|
||||||
|
func_name: return_nothing
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: function returning nothing should succeed
|
||||||
|
- py_func:
|
||||||
|
file: $(test_path)$(psep)py_func.py
|
||||||
|
func_name: return_explicit_none
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: function returning explicit None should succeed
|
||||||
|
- group:
|
||||||
|
name: context_id tests
|
||||||
|
steps:
|
||||||
|
- py_func:
|
||||||
|
expected_result: hello context
|
||||||
|
file: $(test_path)$(psep)py_func.py
|
||||||
|
func_name: set_context_value
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: set serializable value
|
||||||
|
param:
|
||||||
|
- hello context
|
||||||
|
- py_func:
|
||||||
|
context_id: ctx_test
|
||||||
|
expected_result: hello context
|
||||||
|
file: $(test_path)$(psep)py_func.py
|
||||||
|
func_name: get_context_value
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: get serializable value (same context_id)
|
||||||
|
- py_func:
|
||||||
|
expected_result: hello context
|
||||||
|
file: $(test_path)$(psep)py_func.py
|
||||||
|
func_name: get_context_value
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: get serializable value (no context_id, from
|
||||||
|
main gd)
|
||||||
|
- py_func:
|
||||||
|
context_id: ctx_other
|
||||||
|
expected_result: hello context
|
||||||
|
file: $(test_path)$(psep)py_func.py
|
||||||
|
func_name: get_context_value
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: get serializable value (different context_id)
|
||||||
|
- py_func:
|
||||||
|
context_id: ctx_ns_test
|
||||||
|
expected_result: hello ns
|
||||||
|
file: $(test_path)$(psep)py_func.py
|
||||||
|
func_name: set_ns_value
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: set non-serializable value
|
||||||
|
param:
|
||||||
|
- hello ns
|
||||||
|
- py_func:
|
||||||
|
context_id: ctx_ns_test
|
||||||
|
expected_result: hello ns
|
||||||
|
file: $(test_path)$(psep)py_func.py
|
||||||
|
func_name: get_ns_value
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: get non-serializable value (same context_id)
|
||||||
|
filename: /home/renish/workspace/testium/code/test/validation/items/py_func/test.tum
|
||||||
|
- sequence:
|
||||||
|
steps:
|
||||||
|
- report:
|
||||||
|
export:
|
||||||
|
- text:
|
||||||
|
key: $(test)_PASS
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_PASS.txt
|
||||||
|
- html:
|
||||||
|
key: $(test)_PASS
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_PASS.html
|
||||||
|
- junit:
|
||||||
|
key: $(test)_PASS
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_PASS.junit
|
||||||
|
name: Expected PASS $(test) test
|
||||||
|
- report:
|
||||||
|
export:
|
||||||
|
- text:
|
||||||
|
key: $(test)_FAIL
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_FAIL.txt
|
||||||
|
- html:
|
||||||
|
key: $(test)_FAIL
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_FAIL.html
|
||||||
|
- junit:
|
||||||
|
key: $(test)_FAIL
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_FAIL.junit
|
||||||
|
name: Expected FAIL $(test) test
|
||||||
|
filename: /home/renish/workspace/testium/code/test/validation/items/report.tum
|
||||||
|
report:
|
||||||
|
enabled: true
|
||||||
|
export:
|
||||||
|
junit:
|
||||||
|
file_name: $(validation_report_file).junit
|
||||||
|
path: $(validation_report_path)
|
||||||
|
sqlite:
|
||||||
|
file_name: $(validation_report_file).sqlite
|
||||||
|
path: $(validation_report_path)
|
||||||
|
log_stored: true
|
||||||
87
schema/test_schema/sleep.tum
Normal file
87
schema/test_schema/sleep.tum
Normal file
@@ -0,0 +1,87 @@
|
|||||||
|
config_file:
|
||||||
|
- param.yaml
|
||||||
|
- items/sleep/param.yaml
|
||||||
|
main:
|
||||||
|
name: Testium validation suite
|
||||||
|
steps:
|
||||||
|
- group:
|
||||||
|
name: Test preparation
|
||||||
|
steps:
|
||||||
|
- let:
|
||||||
|
condition: <| "$(os)" == "Linux" |>
|
||||||
|
name: Set test variables for Linux
|
||||||
|
values:
|
||||||
|
- terminal_prompt: $(linux_prompt)
|
||||||
|
- psep: /
|
||||||
|
- let:
|
||||||
|
condition: <| "$(os)" == "Windows" |>
|
||||||
|
name: Set test variables for Windows
|
||||||
|
values:
|
||||||
|
- terminal_prompt: $(windows_prompt)
|
||||||
|
- psep: \
|
||||||
|
- group:
|
||||||
|
name: Group of tests
|
||||||
|
steps:
|
||||||
|
- let:
|
||||||
|
name: sleep test constants
|
||||||
|
values:
|
||||||
|
test: sleep
|
||||||
|
test_path: items/$(test)
|
||||||
|
- group:
|
||||||
|
name: sleep test
|
||||||
|
steps:
|
||||||
|
- sequence:
|
||||||
|
steps:
|
||||||
|
- sleep:
|
||||||
|
condition: $(validation_dialogs)
|
||||||
|
dialog: true
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Sleep timeout with dialogs
|
||||||
|
timeout: 3
|
||||||
|
- sleep:
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Sleep timeout without dialog
|
||||||
|
timeout: 3.0
|
||||||
|
- sleep:
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Sleep timeout in textual format
|
||||||
|
skipped: true
|
||||||
|
timeout: 1h 3m 2s
|
||||||
|
filename: /home/renish/workspace/testium/code/test/validation/items/sleep/test.tum
|
||||||
|
- sequence:
|
||||||
|
steps:
|
||||||
|
- report:
|
||||||
|
export:
|
||||||
|
- text:
|
||||||
|
key: $(test)_PASS
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_PASS.txt
|
||||||
|
- html:
|
||||||
|
key: $(test)_PASS
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_PASS.html
|
||||||
|
- junit:
|
||||||
|
key: $(test)_PASS
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_PASS.junit
|
||||||
|
name: Expected PASS $(test) test
|
||||||
|
- report:
|
||||||
|
export:
|
||||||
|
- text:
|
||||||
|
key: $(test)_FAIL
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_FAIL.txt
|
||||||
|
- html:
|
||||||
|
key: $(test)_FAIL
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_FAIL.html
|
||||||
|
- junit:
|
||||||
|
key: $(test)_FAIL
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_FAIL.junit
|
||||||
|
name: Expected FAIL $(test) test
|
||||||
|
filename: /home/renish/workspace/testium/code/test/validation/items/report.tum
|
||||||
|
report:
|
||||||
|
enabled: true
|
||||||
|
export:
|
||||||
|
junit:
|
||||||
|
file_name: $(validation_report_file).junit
|
||||||
|
path: $(validation_report_path)
|
||||||
|
sqlite:
|
||||||
|
file_name: $(validation_report_file).sqlite
|
||||||
|
path: $(validation_report_path)
|
||||||
|
log_stored: true
|
||||||
84
schema/test_schema/unittest.tum
Normal file
84
schema/test_schema/unittest.tum
Normal file
@@ -0,0 +1,84 @@
|
|||||||
|
config_file:
|
||||||
|
- param.yaml
|
||||||
|
- items/unittest/param.yaml
|
||||||
|
main:
|
||||||
|
name: Testium validation suite
|
||||||
|
steps:
|
||||||
|
- group:
|
||||||
|
name: Test preparation
|
||||||
|
steps:
|
||||||
|
- let:
|
||||||
|
condition: <| "$(os)" == "Linux" |>
|
||||||
|
name: Set test variables for Linux
|
||||||
|
values:
|
||||||
|
- terminal_prompt: $(linux_prompt)
|
||||||
|
- psep: /
|
||||||
|
- let:
|
||||||
|
condition: <| "$(os)" == "Windows" |>
|
||||||
|
name: Set test variables for Windows
|
||||||
|
values:
|
||||||
|
- terminal_prompt: $(windows_prompt)
|
||||||
|
- psep: \
|
||||||
|
- group:
|
||||||
|
name: Group of tests
|
||||||
|
steps:
|
||||||
|
- let:
|
||||||
|
name: unittest test constants
|
||||||
|
values:
|
||||||
|
test: unittest
|
||||||
|
test_path: items/$(test)
|
||||||
|
- group:
|
||||||
|
name: unittest test
|
||||||
|
steps:
|
||||||
|
- sequence:
|
||||||
|
steps:
|
||||||
|
- unittest:
|
||||||
|
key: $(test)_PASS
|
||||||
|
name: Unittest item
|
||||||
|
test_file: /home/renish/workspace/testium/code/test/validation/items/unittest/unittest.py
|
||||||
|
test_method: test_01_pass
|
||||||
|
- unittest:
|
||||||
|
key: $(test)_FAIL
|
||||||
|
name: Unittest item
|
||||||
|
test_file: /home/renish/workspace/testium/code/test/validation/items/unittest/unittest.py
|
||||||
|
test_method:
|
||||||
|
- test_04_disabled
|
||||||
|
- test_03_fail
|
||||||
|
filename: /home/renish/workspace/testium/code/test/validation/items/unittest/test.tum
|
||||||
|
- sequence:
|
||||||
|
steps:
|
||||||
|
- report:
|
||||||
|
export:
|
||||||
|
- text:
|
||||||
|
key: $(test)_PASS
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_PASS.txt
|
||||||
|
- html:
|
||||||
|
key: $(test)_PASS
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_PASS.html
|
||||||
|
- junit:
|
||||||
|
key: $(test)_PASS
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_PASS.junit
|
||||||
|
name: Expected PASS $(test) test
|
||||||
|
- report:
|
||||||
|
export:
|
||||||
|
- text:
|
||||||
|
key: $(test)_FAIL
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_FAIL.txt
|
||||||
|
- html:
|
||||||
|
key: $(test)_FAIL
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_FAIL.html
|
||||||
|
- junit:
|
||||||
|
key: $(test)_FAIL
|
||||||
|
path: $(validation_report_path)$(psep)$(test)_FAIL.junit
|
||||||
|
name: Expected FAIL $(test) test
|
||||||
|
filename: /home/renish/workspace/testium/code/test/validation/items/report.tum
|
||||||
|
report:
|
||||||
|
enabled: true
|
||||||
|
export:
|
||||||
|
junit:
|
||||||
|
file_name: $(validation_report_file).junit
|
||||||
|
path: $(validation_report_path)
|
||||||
|
sqlite:
|
||||||
|
file_name: $(validation_report_file).sqlite
|
||||||
|
path: $(validation_report_path)
|
||||||
|
log_stored: true
|
||||||
532
schema/tum.json
Normal file
532
schema/tum.json
Normal file
@@ -0,0 +1,532 @@
|
|||||||
|
{
|
||||||
|
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||||
|
|
||||||
|
"$defs": {
|
||||||
|
"config_file":{
|
||||||
|
"desciption": "The list of the configuration files",
|
||||||
|
"type": "array",
|
||||||
|
"items": {
|
||||||
|
"type": "string"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
|
||||||
|
"items":{
|
||||||
|
"properties": {
|
||||||
|
"name": { "type": "string" },
|
||||||
|
"stop_on_failure":{ "type": "boolean" },
|
||||||
|
"execute_on_stop":{ "type": "boolean" },
|
||||||
|
"skipped":{ "type": "boolean" },
|
||||||
|
"no_fail":{ "type": "boolean" },
|
||||||
|
"doc":{ "type": "string" },
|
||||||
|
"key":{ "type": "string" },
|
||||||
|
"report":{ "$ref": "#/$defs/report_export" },
|
||||||
|
"condition":{ "type": "string" },
|
||||||
|
"process_result":{ "type": "string" },
|
||||||
|
"expected_result":{ },
|
||||||
|
"store_result":{ "type": "string" }
|
||||||
|
}
|
||||||
|
},
|
||||||
|
|
||||||
|
"steps" : {
|
||||||
|
"required": ["steps"],
|
||||||
|
"properties": {
|
||||||
|
"version": { "type": "string" },
|
||||||
|
"steps": {
|
||||||
|
"type": "array",
|
||||||
|
"items": {
|
||||||
|
"type": "object",
|
||||||
|
"additionalProperties": false,
|
||||||
|
"properties":{
|
||||||
|
"let": { "$ref": "#/$defs/let" },
|
||||||
|
"check": { "$ref": "#/$defs/check" },
|
||||||
|
"dialog_question": { "$ref": "#/$defs/dialog" },
|
||||||
|
"dialog_message": { "$ref": "#/$defs/dialog" },
|
||||||
|
"dialog_note": { "$ref": "#/$defs/dialog" },
|
||||||
|
"dialog_value": { "$ref": "#/$defs/dialog" },
|
||||||
|
"dialog_image": { "$ref": "#/$defs/dialog" },
|
||||||
|
"lua_func": { "$ref": "#/$defs/func" },
|
||||||
|
"py_func": { "$ref": "#/$defs/func" },
|
||||||
|
"console": { "$ref": "#/$defs/console" },
|
||||||
|
"sleep": { "$ref": "#/$defs/sleep" },
|
||||||
|
"json_rpc": { "$ref": "#/$defs/json_rpc" },
|
||||||
|
"group": { "$ref": "#/$defs/group" },
|
||||||
|
"sequence": { "$ref": "#/$defs/sequence" },
|
||||||
|
"report": { "$ref": "#/$defs/report" },
|
||||||
|
"loop": { "$ref": "#/$defs/loop" },
|
||||||
|
"git": { "$ref": "#/$defs/git" },
|
||||||
|
"unittest": { "$ref": "#/$defs/unittest" }
|
||||||
|
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
|
||||||
|
"cons_open" : {
|
||||||
|
"type":"object",
|
||||||
|
"allOf":
|
||||||
|
[
|
||||||
|
{ "$ref": "#/$defs/items" },
|
||||||
|
{
|
||||||
|
"properties": {
|
||||||
|
"protocol" : {
|
||||||
|
"enum": ["telnet", "ssh", "serial", "rawtcp", "terminal"]
|
||||||
|
},
|
||||||
|
"telnet_host" : {"type": "string" },
|
||||||
|
"telnet_port" : {"type": "number" },
|
||||||
|
"ssh_host": {"type": "string" },
|
||||||
|
"ssh_user": {"type": "string" },
|
||||||
|
"ssh_pwd": {"type": "string" },
|
||||||
|
"serial_port": {"type": "string" },
|
||||||
|
"serial_baudrate": {"type": "integer" },
|
||||||
|
"buffered": {"type": "boolean" },
|
||||||
|
"tcp_host" : {"type": "string" },
|
||||||
|
"tcp_port" : {"type": "number" },
|
||||||
|
"terminal_path": {"type":"string" },
|
||||||
|
"shell": {"type":"string" }
|
||||||
|
},
|
||||||
|
"required": ["protocol"]
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
|
||||||
|
"cons_read": {
|
||||||
|
"type":"object",
|
||||||
|
"unevaluatedProperties": false,
|
||||||
|
"allOf": [
|
||||||
|
{ "$ref": "#/$defs/items" },
|
||||||
|
{
|
||||||
|
"properties": {
|
||||||
|
"protocol" : {
|
||||||
|
"enum": ["telnet", "ssh", "serial", "rawtcp", "terminal"]
|
||||||
|
},
|
||||||
|
"expected" : {"type": "string" },
|
||||||
|
"timeout": {"type": "number" },
|
||||||
|
"mute": {"type": "boolean" }
|
||||||
|
},
|
||||||
|
"required":["expected"]
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
|
||||||
|
"console" : {
|
||||||
|
"description": "The let items",
|
||||||
|
"type": "object",
|
||||||
|
"unevaluatedProperties": false,
|
||||||
|
"allOf": [
|
||||||
|
{ "$ref": "#/$defs/items" },
|
||||||
|
{
|
||||||
|
"required": ["steps", "console_name"],
|
||||||
|
"properties":{
|
||||||
|
"console_name": {"type":"string" },
|
||||||
|
"steps": {
|
||||||
|
"type": "array",
|
||||||
|
"items": {
|
||||||
|
"type": "object",
|
||||||
|
"properties":{
|
||||||
|
"open": {
|
||||||
|
"$ref": "#/$defs/cons_open"
|
||||||
|
},
|
||||||
|
"write": { "type": "string" },
|
||||||
|
"writeln": { "type": "string" },
|
||||||
|
"read_until": {
|
||||||
|
"$ref": "#/$defs/cons_read"
|
||||||
|
},
|
||||||
|
"close": {
|
||||||
|
"anyOf": [
|
||||||
|
{"type": "null"},
|
||||||
|
{"type": "string"}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"additionalProperties": false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
|
||||||
|
"json_rpc_console": {
|
||||||
|
"type":"object",
|
||||||
|
"properties": {
|
||||||
|
"name" : {"type": "string" },
|
||||||
|
"prompt" : {"type": "string" }
|
||||||
|
},
|
||||||
|
"required":["name"],
|
||||||
|
"additionalProperties": false
|
||||||
|
},
|
||||||
|
|
||||||
|
"json_rpc_udp": {
|
||||||
|
"type":"object",
|
||||||
|
"properties": {
|
||||||
|
"server" : {"type": "string" },
|
||||||
|
"udp_snd_port" : {"type": "integer" },
|
||||||
|
"udp_rcv_port" : {"type": "integer" },
|
||||||
|
"bufsize" : {"type": "integer" }
|
||||||
|
},
|
||||||
|
"required":["name"],
|
||||||
|
"additionalProperties": false
|
||||||
|
},
|
||||||
|
|
||||||
|
"json_rpc_query": {
|
||||||
|
"type":"object",
|
||||||
|
"properties": {
|
||||||
|
"method" : { "type": "string" },
|
||||||
|
"params" : { "type": "array" },
|
||||||
|
"id" : {"type": "integer" },
|
||||||
|
"no_wait" : {"type": "boolean" }
|
||||||
|
},
|
||||||
|
"required":["method"],
|
||||||
|
"additionalProperties": false
|
||||||
|
},
|
||||||
|
|
||||||
|
"json_rpc_receive": {
|
||||||
|
"type":"object",
|
||||||
|
"properties": {
|
||||||
|
"id" : {"type": "integer" },
|
||||||
|
"timeout" : {"type": "number" }
|
||||||
|
},
|
||||||
|
"required":["id"],
|
||||||
|
"additionalProperties": false
|
||||||
|
},
|
||||||
|
|
||||||
|
"json_rpc" : {
|
||||||
|
"description": "The json_rpc items",
|
||||||
|
"type": "object",
|
||||||
|
"unevaluatedProperties": false,
|
||||||
|
"allOf": [
|
||||||
|
{ "$ref": "#/$defs/items" },
|
||||||
|
{
|
||||||
|
"required": ["steps"],
|
||||||
|
"properties":{
|
||||||
|
|
||||||
|
"udp" : { "$ref": "#/$defs/json_rpc_udp" },
|
||||||
|
"console" : { "$ref": "#/$defs/json_rpc_console" },
|
||||||
|
"timeout": {"type": "number" },
|
||||||
|
"version": {"enum": ["1.0", "2.0"]},
|
||||||
|
"steps": {
|
||||||
|
"type": "array",
|
||||||
|
"items": {
|
||||||
|
"type": "object",
|
||||||
|
"additionalProperties": false,
|
||||||
|
"properties":{
|
||||||
|
"open": { "type": "null" },
|
||||||
|
"query": { "$ref": "#/$defs/json_rpc_query" },
|
||||||
|
"receive": { "$ref": "#/$defs/json_rpc_receive" },
|
||||||
|
"writeln": { "type": "string" },
|
||||||
|
"read_until": { "$ref": "#/$defs/cons_read"
|
||||||
|
},
|
||||||
|
"close": { "type": "null" }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
|
||||||
|
"group" : {
|
||||||
|
"description": "The group items",
|
||||||
|
"type": "object",
|
||||||
|
"unevaluatedProperties": false,
|
||||||
|
"allOf": [
|
||||||
|
{ "$ref": "#/$defs/items" },
|
||||||
|
{ "$ref": "#/$defs/steps" }
|
||||||
|
]
|
||||||
|
},
|
||||||
|
|
||||||
|
"sequence" : {
|
||||||
|
"description": "The sequence items",
|
||||||
|
"type": "object",
|
||||||
|
"unevaluatedProperties": false,
|
||||||
|
"allOf": [
|
||||||
|
{ "$ref": "#/$defs/items" },
|
||||||
|
{ "$ref": "#/$defs/steps" },
|
||||||
|
{
|
||||||
|
"properties": {
|
||||||
|
"filename": { "type": "string" }
|
||||||
|
},
|
||||||
|
"required": ["filename"]
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
|
||||||
|
"loop" : {
|
||||||
|
"description": "The loop items",
|
||||||
|
"type": "object",
|
||||||
|
"unevaluatedProperties": false,
|
||||||
|
"allOf": [
|
||||||
|
{ "$ref": "#/$defs/items" },
|
||||||
|
{ "$ref": "#/$defs/steps" },
|
||||||
|
{
|
||||||
|
"properties": {
|
||||||
|
"iterator": { },
|
||||||
|
"exit_condition": {
|
||||||
|
"description": "the posibility to stop the loop",
|
||||||
|
"type": "object",
|
||||||
|
"additionalProperties": false,
|
||||||
|
"properties": {
|
||||||
|
"time" : {"type":"number"},
|
||||||
|
"value" : {"type":"number"},
|
||||||
|
"file" : {"type":"string"},
|
||||||
|
"func_name" : {"type":"string"},
|
||||||
|
"eval" : {"type":"string"}
|
||||||
|
},
|
||||||
|
"dependentRequired": {
|
||||||
|
"file" : ["func_name"]
|
||||||
|
}
|
||||||
|
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
|
||||||
|
"main" : {
|
||||||
|
"description": "The Main items",
|
||||||
|
"type": "object",
|
||||||
|
"unevaluatedProperties": false,
|
||||||
|
"allOf": [
|
||||||
|
{ "$ref": "#/$defs/items" },
|
||||||
|
{ "$ref": "#/$defs/steps" },
|
||||||
|
{
|
||||||
|
"properties": {
|
||||||
|
"version": { "type": "string" }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
|
||||||
|
"sleep" : {
|
||||||
|
"description": "Sleep for X time [secondes",
|
||||||
|
"type": "object",
|
||||||
|
"unevaluatedProperties": false,
|
||||||
|
"allOf": [
|
||||||
|
{ "$ref": "#/$defs/items" },
|
||||||
|
{
|
||||||
|
"properties":{
|
||||||
|
"dialog": { "type": "boolean" },
|
||||||
|
"timeout": { "type": "number" }
|
||||||
|
},
|
||||||
|
"required": ["timeout"]
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
|
||||||
|
"let" : {
|
||||||
|
"description": "The let items",
|
||||||
|
"type": "object",
|
||||||
|
"unevaluatedProperties": false,
|
||||||
|
"allOf": [
|
||||||
|
{ "$ref": "#/$defs/items" },
|
||||||
|
{
|
||||||
|
"properties":{
|
||||||
|
"values": {
|
||||||
|
"anyOf": [
|
||||||
|
{
|
||||||
|
"type": "array",
|
||||||
|
"items": {"type": "object" }
|
||||||
|
},
|
||||||
|
{ "type":"object" }
|
||||||
|
]
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"required": ["values"]
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
|
||||||
|
"check" : {
|
||||||
|
"description": "The let items",
|
||||||
|
"type": "object",
|
||||||
|
"unevaluatedProperties": false,
|
||||||
|
"allOf": [
|
||||||
|
{ "$ref": "#/$defs/items" },
|
||||||
|
{
|
||||||
|
"properties":{
|
||||||
|
"values": {
|
||||||
|
"type": "array",
|
||||||
|
"items": {"type": "string" }
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"required": ["values"]
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
|
||||||
|
"git" : {
|
||||||
|
"description": "The git items",
|
||||||
|
"type": "object",
|
||||||
|
"unevaluatedProperties": false,
|
||||||
|
"allOf": [
|
||||||
|
{ "$ref": "#/$defs/items" },
|
||||||
|
{
|
||||||
|
"properties":{
|
||||||
|
"repo": { "type": "string" }
|
||||||
|
},
|
||||||
|
"required": ["repo"]
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
|
||||||
|
"unittest" : {
|
||||||
|
"description": "The unittest items",
|
||||||
|
"type": "object",
|
||||||
|
"unevaluatedProperties": false,
|
||||||
|
"allOf": [
|
||||||
|
{ "$ref": "#/$defs/items" },
|
||||||
|
{
|
||||||
|
"properties":{
|
||||||
|
"test_file": { "type": "string" },
|
||||||
|
"test_method": {
|
||||||
|
"anyOf": [
|
||||||
|
{"type":"string"},
|
||||||
|
{
|
||||||
|
"type":"array",
|
||||||
|
"items": {"type":"string"}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"required": ["test_file"]
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
|
||||||
|
"dialog" : {
|
||||||
|
"description": "The let items",
|
||||||
|
"type": "object",
|
||||||
|
"unevaluatedProperties": false,
|
||||||
|
"allOf": [
|
||||||
|
{ "$ref": "#/$defs/items" },
|
||||||
|
{
|
||||||
|
"properties":{
|
||||||
|
"question": { "type": "string" },
|
||||||
|
"auto_result": { "type": "string" },
|
||||||
|
"auto_value": { "type": ["number", "string"] },
|
||||||
|
"filename" : {"type": "string"},
|
||||||
|
"reference" : {
|
||||||
|
"type":"array",
|
||||||
|
"items": { "type":"string"}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"required": ["question"]
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
|
||||||
|
"dialog_choice" : {
|
||||||
|
"description": "The one choi items",
|
||||||
|
"type": "object",
|
||||||
|
"additionalProperties": false,
|
||||||
|
"properties":{
|
||||||
|
"name": { "type": "string" },
|
||||||
|
"description": { "type": "string" },
|
||||||
|
"icon": { "type": "string"},
|
||||||
|
"choices": {
|
||||||
|
"type":"array",
|
||||||
|
"items": { "$ref": "#/$defs/dialog_choice" }
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"required": ["name", "description"]
|
||||||
|
},
|
||||||
|
"dialog_choices" : {
|
||||||
|
"description": "The dialog choices items",
|
||||||
|
"type": "object",
|
||||||
|
"unevaluatedProperties": false,
|
||||||
|
"allOf": [
|
||||||
|
{ "$ref": "#/$defs/items" },
|
||||||
|
{
|
||||||
|
"properties":{
|
||||||
|
"name": { "type": "string" },
|
||||||
|
"question": { "type": "string" },
|
||||||
|
"icon": { "type": "string"},
|
||||||
|
"choices": {
|
||||||
|
"type":"array",
|
||||||
|
"items": { "$ref": "#/$defs/dialog_choice" }
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"required": ["question"]
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"func" : {
|
||||||
|
"description": "The py_fun and lua_func items",
|
||||||
|
"type": "object",
|
||||||
|
"unevaluatedProperties": false,
|
||||||
|
"allOf": [
|
||||||
|
{ "$ref": "#/$defs/items" },
|
||||||
|
{
|
||||||
|
"properties":{
|
||||||
|
"file": { "type": "string" },
|
||||||
|
"func_name": { "type": "string" },
|
||||||
|
"context_id": { "type": "string" },
|
||||||
|
"param" : {
|
||||||
|
"anyOf": [
|
||||||
|
{"type": "array"},
|
||||||
|
{"type": "object"}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"required": ["file", "func_name"]
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
|
||||||
|
"report_export": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"path": {"type":"string" },
|
||||||
|
"file_name": {"type":"string" },
|
||||||
|
"pattern": {"type":"string" },
|
||||||
|
"key": {"type":"string" }
|
||||||
|
}
|
||||||
|
},
|
||||||
|
|
||||||
|
"report":{
|
||||||
|
"desciption": "The list of the configuration files",
|
||||||
|
"type": "object",
|
||||||
|
"additional_properties":false,
|
||||||
|
"properties": {
|
||||||
|
"enabled": {"type":"boolean" },
|
||||||
|
"log_stored": {"type":"boolean" },
|
||||||
|
"export": {
|
||||||
|
"anyOf": [
|
||||||
|
{
|
||||||
|
"type": "object",
|
||||||
|
"properties":{
|
||||||
|
"html": { "$ref": "#/$defs/report_export" },
|
||||||
|
"sqlite": { "$ref": "#/$defs/report_export" },
|
||||||
|
"junit": { "$ref": "#/$defs/report_export" }
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "array",
|
||||||
|
"items": {
|
||||||
|
"type":"object",
|
||||||
|
"properties":{
|
||||||
|
"html": { "$ref": "#/$defs/report_export" },
|
||||||
|
"sqlite": { "$ref": "#/$defs/report_export" },
|
||||||
|
"junit": { "$ref": "#/$defs/report_export" }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
},
|
||||||
|
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"config_file" : { "$ref": "#/$defs/config_file" },
|
||||||
|
"main": { "$ref": "#/$defs/main" },
|
||||||
|
"report" : { "$ref": "#/$defs/report" }
|
||||||
|
|
||||||
|
},
|
||||||
|
"required" : ["main"],
|
||||||
|
"additionalProperties": false
|
||||||
|
|
||||||
|
}
|
||||||
@@ -1 +1 @@
|
|||||||
0.2
|
0.1.2
|
||||||
@@ -16,7 +16,6 @@ from interpreter.utils.test_init import (
|
|||||||
env_init,
|
env_init,
|
||||||
prepare_global,
|
prepare_global,
|
||||||
update_global,
|
update_global,
|
||||||
apply_overrides,
|
|
||||||
set_standard_gd_keys,
|
set_standard_gd_keys,
|
||||||
test_run_init,
|
test_run_init,
|
||||||
test_run_header,
|
test_run_header,
|
||||||
@@ -211,19 +210,6 @@ class TestProcess(Process):
|
|||||||
|
|
||||||
env_init()
|
env_init()
|
||||||
|
|
||||||
# Apply GUI defaults and CLI defines to the global dict
|
|
||||||
# *before* eval_proc starts: bins.python_bin() reads
|
|
||||||
# ``python_bin`` from gd on its very first call (during
|
|
||||||
# eval_process_init) and caches the result. Without this,
|
|
||||||
# ``-d python_bin=...`` and the GUI ``python_bin`` preference
|
|
||||||
# would only take effect for items spawned *after* the cache
|
|
||||||
# was already populated with the auto-discovered interpreter,
|
|
||||||
# i.e. they would silently be ignored for eval_proc itself.
|
|
||||||
# _load_initial_params re-applies the same overrides after
|
|
||||||
# ``prepare_global()`` clears gd, so the gd value stays in
|
|
||||||
# sync with the cached path.
|
|
||||||
apply_overrides(self.__defs, self.__gui_defaults)
|
|
||||||
|
|
||||||
# Creation of the python evaluation process for loading of the complete test
|
# Creation of the python evaluation process for loading of the complete test
|
||||||
eval_proc = eval_process_init(api_request, 10, test_dir)
|
eval_proc = eval_process_init(api_request, 10, test_dir)
|
||||||
eval_proc.start()
|
eval_proc.start()
|
||||||
|
|||||||
@@ -5,9 +5,6 @@ from copy import deepcopy
|
|||||||
from interpreter.test_items.test_result import TestResult, TestValue
|
from interpreter.test_items.test_result import TestResult, TestValue
|
||||||
import api.testium as tm
|
import api.testium as tm
|
||||||
from interpreter.utils.params import TestItemParams
|
from interpreter.utils.params import TestItemParams
|
||||||
from interpreter.utils.param_decl import (
|
|
||||||
Param, ParamSet, LIST, BLOCK, unknown_keys, missing_required,
|
|
||||||
)
|
|
||||||
from interpreter.utils.constants import TestItemType as cst_type
|
from interpreter.utils.constants import TestItemType as cst_type
|
||||||
from interpreter.utils.eval import eval_to_boolean, evaluate, post_evaluate
|
from interpreter.utils.eval import eval_to_boolean, evaluate, post_evaluate
|
||||||
from runtime.tum_except import ETUMSyntaxError, item_load_context
|
from runtime.tum_except import ETUMSyntaxError, item_load_context
|
||||||
@@ -16,32 +13,6 @@ LOG_TEST_STOP = '<----- step "{}" finished'
|
|||||||
LOG_TEST_START = '-----> step "{}" started'
|
LOG_TEST_START = '-----> step "{}" started'
|
||||||
|
|
||||||
|
|
||||||
# Parameters accepted by every test item, regardless of its type. Subclasses
|
|
||||||
# concatenate their own ``PARAMS`` to this set; the merged result drives
|
|
||||||
# unknown-param warnings and (later) the LSP schema export.
|
|
||||||
COMMON_PARAMS = ParamSet(
|
|
||||||
Param("name", doc="Display name shown in the GUI tree and reports."),
|
|
||||||
Param("doc", doc="Free-form documentation; surfaced in tooltips."),
|
|
||||||
Param("skipped", doc="If truthy, the step is skipped (static expression, "
|
|
||||||
"evaluated at load time)."),
|
|
||||||
Param("key", doc="Report key used to classify the result "
|
|
||||||
"(typically <test>_PASS or <test>_FAIL)."),
|
|
||||||
Param("stop_on_failure", doc="If true, abort the surrounding container on failure."),
|
|
||||||
Param("execute_on_stop", doc="If true, run this step even when its container "
|
|
||||||
"is being stopped (cleanup)."),
|
|
||||||
Param("process_result", doc="Post-evaluation expression applied to the test result."),
|
|
||||||
Param("store_result", doc="Global-dict key in which to store the test result."),
|
|
||||||
Param("expected_result", doc="Expected outcome; the step is failed if it doesn't match."),
|
|
||||||
Param("no_fail", doc="If truthy, never report a FAILURE for this step."),
|
|
||||||
Param("report", doc="Per-step reporting override."),
|
|
||||||
Param("condition", doc="Optional gating expression evaluated before each "
|
|
||||||
"run; false ⇒ the step is skipped."),
|
|
||||||
Param("steps", kind=LIST, doc="Children (for container items)."),
|
|
||||||
Param("seq_filename", doc="(internal) source .tum file of this step; injected "
|
|
||||||
"by the loader."),
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
class TestItem:
|
class TestItem:
|
||||||
pass
|
pass
|
||||||
|
|
||||||
@@ -126,11 +97,6 @@ def test_data(item: TestItem, child: dict) -> dict:
|
|||||||
|
|
||||||
|
|
||||||
class TestItem:
|
class TestItem:
|
||||||
# Subclasses override with their own ParamSet to opt into the declarative
|
|
||||||
# validation. While ``PARAMS`` is empty / unset, the base class skips the
|
|
||||||
# unknown-param check for this item type — keeps the migration incremental.
|
|
||||||
PARAMS = None
|
|
||||||
|
|
||||||
def __init__(
|
def __init__(
|
||||||
self, dict_item: dict = None, parent: TestItem = None, status_queue=None, filename = ""
|
self, dict_item: dict = None, parent: TestItem = None, status_queue=None, filename = ""
|
||||||
):
|
):
|
||||||
@@ -168,13 +134,6 @@ class TestItem:
|
|||||||
# creation of the params object
|
# creation of the params object
|
||||||
self._prms = TestItemParams(dict_item, parent)
|
self._prms = TestItemParams(dict_item, parent)
|
||||||
|
|
||||||
# Declarative-params validation. Only kicks in when the concrete
|
|
||||||
# subclass declares ``PARAMS`` — items not yet migrated stay
|
|
||||||
# silent. Warnings (not errors) during the migration window so
|
|
||||||
# existing .tum files don't break suddenly; will be flipped to
|
|
||||||
# errors once every item has migrated.
|
|
||||||
self._validate_declared_params(dict_item)
|
|
||||||
|
|
||||||
# getting parameters for the test item
|
# getting parameters for the test item
|
||||||
try:
|
try:
|
||||||
self._name = self._prms.getParam("name", default="", processed=True)
|
self._name = self._prms.getParam("name", default="", processed=True)
|
||||||
@@ -231,36 +190,6 @@ class TestItem:
|
|||||||
|
|
||||||
self.result = TestResult(self, TestValue.FAILURE, "Failure by default")
|
self.result = TestResult(self, TestValue.FAILURE, "Failure by default")
|
||||||
|
|
||||||
def _validate_declared_params(self, dict_item):
|
|
||||||
"""Warn on unknown / missing-required params, if PARAMS is declared.
|
|
||||||
|
|
||||||
The check is opt-in per subclass: it only runs when the concrete
|
|
||||||
class sets a non-empty ``PARAMS`` attribute. Items not yet migrated
|
|
||||||
produce no diagnostics — preserving the historical "silently accept
|
|
||||||
anything" behavior until they get their declaration.
|
|
||||||
"""
|
|
||||||
if not self.PARAMS:
|
|
||||||
return
|
|
||||||
# ``self._type`` is the parent root type at this point (subclasses set
|
|
||||||
# it after super().__init__), so use the class name as a stable label
|
|
||||||
# in diagnostics. ``self._name`` was preset to the type name by every
|
|
||||||
# subclass before super() ran, which gives a useful prefix.
|
|
||||||
label = f"{type(self).__name__} '{self._name}'"
|
|
||||||
declared = COMMON_PARAMS + self.PARAMS
|
|
||||||
unknown = unknown_keys(declared, dict_item)
|
|
||||||
if unknown:
|
|
||||||
accepted = ", ".join(sorted(declared.names()))
|
|
||||||
for k in unknown:
|
|
||||||
tm.print_warn(
|
|
||||||
f"{label}: unknown parameter '{k}'. Accepted: {accepted}."
|
|
||||||
)
|
|
||||||
missing = missing_required(declared, dict_item)
|
|
||||||
for k in missing:
|
|
||||||
raise ETUMSyntaxError(
|
|
||||||
f"{label}: required parameter '{k}' is missing.",
|
|
||||||
self._seq_filename,
|
|
||||||
)
|
|
||||||
|
|
||||||
def _filter_dict_item(self, dict_item):
|
def _filter_dict_item(self, dict_item):
|
||||||
# Stores the content of the step to be displayed
|
# Stores the content of the step to be displayed
|
||||||
# in the GUI
|
# in the GUI
|
||||||
|
|||||||
@@ -5,22 +5,11 @@ from runtime.tum_except import ETUMSyntaxError, item_load_context
|
|||||||
import api.testium as tm
|
import api.testium as tm
|
||||||
from interpreter.utils.constants import TestItemType as cst
|
from interpreter.utils.constants import TestItemType as cst
|
||||||
from interpreter.utils.eval import evaluate
|
from interpreter.utils.eval import evaluate
|
||||||
from interpreter.utils.param_decl import Param, ParamSet, LIST
|
|
||||||
|
|
||||||
class TestItemCheckValue(TestItem):
|
class TestItemCheckValue(TestItem):
|
||||||
"""check item usage.
|
"""check item usage.
|
||||||
check usage:{check: {name: check my func output, steps: ['$(pfn_echo) < 5']}}
|
check usage:{check: {name: check my func output, steps: ['$(pfn_echo) < 5']}}
|
||||||
"""
|
"""
|
||||||
|
|
||||||
PARAMS = ParamSet(
|
|
||||||
Param("values", kind=LIST, required=True,
|
|
||||||
doc="List of expressions to evaluate. Each is expanded then "
|
|
||||||
"evaluated; non-truthy results fail the check."),
|
|
||||||
# 'steps' is intentionally not redeclared here — it's the deprecated
|
|
||||||
# alias of 'values' and is already accepted by COMMON_PARAMS for
|
|
||||||
# container items. A runtime warning is emitted when 'steps' is used.
|
|
||||||
)
|
|
||||||
|
|
||||||
def __init__(self, dict_item, parent = None, status_queue=None, filename=""):
|
def __init__(self, dict_item, parent = None, status_queue=None, filename=""):
|
||||||
self._name = cst.TYPE_CHECK.item_name
|
self._name = cst.TYPE_CHECK.item_name
|
||||||
super().__init__(dict_item, parent, status_queue, filename=filename)
|
super().__init__(dict_item, parent, status_queue, filename=filename)
|
||||||
|
|||||||
@@ -2,25 +2,11 @@ from interpreter.test_items.test_item import test_run
|
|||||||
from interpreter.test_items.test_result import TestValue
|
from interpreter.test_items.test_result import TestValue
|
||||||
from interpreter.test_items.test_item_dialog_base import TestItemDialogBase, _is_text_mode, _is_interactive
|
from interpreter.test_items.test_item_dialog_base import TestItemDialogBase, _is_text_mode, _is_interactive
|
||||||
from interpreter.utils.constants import TestItemType as cst
|
from interpreter.utils.constants import TestItemType as cst
|
||||||
from interpreter.utils.param_decl import Param, ParamSet, BLOCK
|
|
||||||
from runtime.tum_except import item_load_context
|
from runtime.tum_except import item_load_context
|
||||||
import api.testium as tm
|
import api.testium as tm
|
||||||
|
|
||||||
|
|
||||||
class TestItemChoicesDialog(TestItemDialogBase):
|
class TestItemChoicesDialog(TestItemDialogBase):
|
||||||
|
|
||||||
PARAMS = ParamSet(
|
|
||||||
Param("question", required=True,
|
|
||||||
doc="Prompt shown above the list of choices."),
|
|
||||||
Param("choices", kind=BLOCK, required=True,
|
|
||||||
doc="Tree of choices: either a list of strings, or a nested "
|
|
||||||
"mapping {label: subchoices, ...} to build a multi-level menu."),
|
|
||||||
Param("icon", default=None,
|
|
||||||
doc="Default icon name shown next to each choice."),
|
|
||||||
Param("auto_result", default=None,
|
|
||||||
doc="Batch-mode selection (path or label). None ⇒ FAILURE."),
|
|
||||||
)
|
|
||||||
|
|
||||||
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
|
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
|
||||||
self._name = cst.TYPE_CHOICES_DLG.item_name
|
self._name = cst.TYPE_CHOICES_DLG.item_name
|
||||||
super().__init__(dict_item, parent, status_queue, filename=filename)
|
super().__init__(dict_item, parent, status_queue, filename=filename)
|
||||||
|
|||||||
@@ -10,7 +10,6 @@ from interpreter.test_items.test_item import test_run
|
|||||||
from interpreter.test_items.item_actions import TestItemActions
|
from interpreter.test_items.item_actions import TestItemActions
|
||||||
from interpreter.test_items.item_actions.action import TestItemAction
|
from interpreter.test_items.item_actions.action import TestItemAction
|
||||||
from interpreter.utils.constants import TestItemType as cst
|
from interpreter.utils.constants import TestItemType as cst
|
||||||
from interpreter.utils.param_decl import Param, ParamSet
|
|
||||||
from interpreter.test_items.test_result import TestResult, TestValue
|
from interpreter.test_items.test_result import TestResult, TestValue
|
||||||
|
|
||||||
|
|
||||||
@@ -22,38 +21,6 @@ class TestItemConsoleAction(TestItemAction):
|
|||||||
|
|
||||||
|
|
||||||
class TestItemConsoleOpen(TestItemConsoleAction):
|
class TestItemConsoleOpen(TestItemConsoleAction):
|
||||||
|
|
||||||
PARAMS = ParamSet(
|
|
||||||
Param("protocol", required=True,
|
|
||||||
doc="Transport: 'telnet', 'ssh', 'rawtcp', 'serial' or 'terminal'."),
|
|
||||||
Param("write_delay", default=0,
|
|
||||||
doc="Inter-character write delay in ms (slow devices)."),
|
|
||||||
Param("log", doc="Path to a log file capturing the console traffic."),
|
|
||||||
Param("overwrite_log", default=True,
|
|
||||||
doc="If true, truncate the log file at open; else append."),
|
|
||||||
# telnet
|
|
||||||
Param("telnet_host", doc="Hostname/IP for the telnet target."),
|
|
||||||
Param("telnet_port", default=69, doc="TCP port for telnet."),
|
|
||||||
# ssh
|
|
||||||
Param("ssh_host", doc="Hostname/IP for the SSH target."),
|
|
||||||
Param("ssh_user", doc="SSH login user."),
|
|
||||||
Param("ssh_pwd", doc="SSH password (if key-based auth is not used)."),
|
|
||||||
# rawtcp
|
|
||||||
Param("tcp_host", doc="Hostname/IP for a raw-TCP connection."),
|
|
||||||
Param("tcp_port", doc="TCP port for a raw-TCP connection."),
|
|
||||||
# serial
|
|
||||||
Param("serial_port", doc="Serial device path (e.g. /dev/ttyUSB0 or COM3)."),
|
|
||||||
Param("serial_baudrate", doc="Serial baudrate."),
|
|
||||||
Param("buffered", default=True,
|
|
||||||
doc="If true, the serial console buffers received bytes between reads."),
|
|
||||||
# terminal
|
|
||||||
Param("terminal_path",
|
|
||||||
doc="Working directory for the local terminal protocol."),
|
|
||||||
Param("shell",
|
|
||||||
doc="Shell command used for the local terminal protocol "
|
|
||||||
"(default: 'cmd.exe' on Windows, '/usr/bin/env bash' elsewhere)."),
|
|
||||||
)
|
|
||||||
|
|
||||||
def __init__(
|
def __init__(
|
||||||
self, action_name, dict_item, parent=None, status_queue=None, filename=""
|
self, action_name, dict_item, parent=None, status_queue=None, filename=""
|
||||||
):
|
):
|
||||||
@@ -316,17 +283,6 @@ class TestItemConsoleWriteLn(TestItemConsoleAction):
|
|||||||
|
|
||||||
|
|
||||||
class TestItemConsoleReadUntil(TestItemConsoleAction):
|
class TestItemConsoleReadUntil(TestItemConsoleAction):
|
||||||
|
|
||||||
PARAMS = ParamSet(
|
|
||||||
Param("expected", required=True,
|
|
||||||
doc="Regex matched against incoming console output until found "
|
|
||||||
"or until timeout."),
|
|
||||||
Param("timeout", default=-1,
|
|
||||||
doc="Seconds before giving up. Negative means infinite."),
|
|
||||||
Param("mute", default=False,
|
|
||||||
doc="If true, don't echo received bytes to testium's stdout/log."),
|
|
||||||
)
|
|
||||||
|
|
||||||
def __init__(
|
def __init__(
|
||||||
self, action_name, dict_item, parent=None, status_queue=None, filename=""
|
self, action_name, dict_item, parent=None, status_queue=None, filename=""
|
||||||
):
|
):
|
||||||
@@ -380,14 +336,6 @@ class TestItemConsoleReadUntil(TestItemConsoleAction):
|
|||||||
|
|
||||||
|
|
||||||
class TestItemConsole(TestItemActions):
|
class TestItemConsole(TestItemActions):
|
||||||
|
|
||||||
PARAMS = ParamSet(
|
|
||||||
Param("console_name", required=True,
|
|
||||||
doc="Identifier of the console — used by every nested action to "
|
|
||||||
"reach back the same transport. Multiple consoles can coexist "
|
|
||||||
"as long as their names differ."),
|
|
||||||
)
|
|
||||||
|
|
||||||
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
|
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
|
||||||
super().__init__(
|
super().__init__(
|
||||||
cst.TYPE_CONSOLE, dict_item, parent, status_queue, filename=filename
|
cst.TYPE_CONSOLE, dict_item, parent, status_queue, filename=filename
|
||||||
|
|||||||
@@ -8,36 +8,9 @@ from interpreter.test_items.test_result import TestResult, TestValue
|
|||||||
import api.testium as tm
|
import api.testium as tm
|
||||||
from interpreter.utils.params import TestItemParams
|
from interpreter.utils.params import TestItemParams
|
||||||
from interpreter.utils.constants import TestItemType as cst
|
from interpreter.utils.constants import TestItemType as cst
|
||||||
from interpreter.utils.param_decl import Param, ParamSet, BLOCK
|
|
||||||
|
|
||||||
|
|
||||||
# Sub-block validation: 'cycle' accepts an 'exit_condition:' mapping whose
|
|
||||||
# own params are reported here so unknown keys inside it can be flagged
|
|
||||||
# during a future Block-aware diagnostic pass. For now the parent only
|
|
||||||
# declares that 'exit_condition' is an accepted top-level key.
|
|
||||||
EXIT_CONDITION_PARAMS = ParamSet(
|
|
||||||
Param("time", doc="HH:MM time of day after which the loop exits."),
|
|
||||||
Param("value", doc="Expression; when truthy the loop exits."),
|
|
||||||
Param("file", doc="Python file containing the exit-condition function."),
|
|
||||||
Param("func_name", doc="Function name in 'file' returning the exit value."),
|
|
||||||
Param("param", doc="Arguments passed to the exit function."),
|
|
||||||
Param("eval", default="",
|
|
||||||
doc="Post-evaluation expression applied to the function's return."),
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
class TestItemCycle(TestItem):
|
class TestItemCycle(TestItem):
|
||||||
|
|
||||||
PARAMS = ParamSet(
|
|
||||||
Param("iterator",
|
|
||||||
doc="Iterable (or string expanding to one) driving the loop. "
|
|
||||||
"The current value is exposed as $(loop_param)."),
|
|
||||||
Param("exit_condition", kind=BLOCK,
|
|
||||||
doc="Optional block stopping the loop early: combine 'time', "
|
|
||||||
"'value', or a 'file'+'func_name' pair (with optional "
|
|
||||||
"'param' and 'eval')."),
|
|
||||||
)
|
|
||||||
|
|
||||||
def __init__(self, dict_cycle, parent=None, status_queue=None, filename=""):
|
def __init__(self, dict_cycle, parent=None, status_queue=None, filename=""):
|
||||||
self._name = cst.TYPE_CYCLE.item_name
|
self._name = cst.TYPE_CYCLE.item_name
|
||||||
super().__init__(dict_cycle, parent, status_queue, filename=filename)
|
super().__init__(dict_cycle, parent, status_queue, filename=filename)
|
||||||
|
|||||||
@@ -1,7 +1,6 @@
|
|||||||
from interpreter.test_items.test_item import (TestItem, test_run)
|
from interpreter.test_items.test_item import (TestItem, test_run)
|
||||||
from interpreter.test_items.test_result import (TestValue)
|
from interpreter.test_items.test_result import (TestValue)
|
||||||
from interpreter.utils.constants import TestItemType as cst
|
from interpreter.utils.constants import TestItemType as cst
|
||||||
from interpreter.utils.param_decl import Param, ParamSet, LIST
|
|
||||||
from runtime.tum_except import ETUMParamError, ETUMSyntaxError
|
from runtime.tum_except import ETUMParamError, ETUMSyntaxError
|
||||||
import interpreter.utils.version as git
|
import interpreter.utils.version as git
|
||||||
|
|
||||||
@@ -9,13 +8,6 @@ class TestItemGit(TestItem):
|
|||||||
"""
|
"""
|
||||||
This item expect only one parameter which is a string or list of string being the path to the git folder
|
This item expect only one parameter which is a string or list of string being the path to the git folder
|
||||||
"""
|
"""
|
||||||
|
|
||||||
PARAMS = ParamSet(
|
|
||||||
Param("repo", kind=LIST, required=True,
|
|
||||||
doc="Path to a git checkout, or list of such paths. Each is "
|
|
||||||
"reported with its current version (tag + dirty state)."),
|
|
||||||
)
|
|
||||||
|
|
||||||
def __init__(self, dict_item, parent = None, status_queue=None, filename=""):
|
def __init__(self, dict_item, parent = None, status_queue=None, filename=""):
|
||||||
self._name = cst.TYPE_GIT.item_name
|
self._name = cst.TYPE_GIT.item_name
|
||||||
super().__init__(dict_item, parent, status_queue, filename=filename)
|
super().__init__(dict_item, parent, status_queue, filename=filename)
|
||||||
|
|||||||
@@ -1,17 +1,10 @@
|
|||||||
from interpreter.test_items.test_item import (TestItem, test_run)
|
from interpreter.test_items.test_item import (TestItem, test_run)
|
||||||
from interpreter.test_items.test_result import (TestResult, TestValue)
|
from interpreter.test_items.test_result import (TestResult, TestValue)
|
||||||
from interpreter.utils.constants import TestItemType as cst
|
from interpreter.utils.constants import TestItemType as cst
|
||||||
from interpreter.utils.param_decl import ParamSet
|
|
||||||
from runtime.tum_except import ETUMSyntaxError
|
from runtime.tum_except import ETUMSyntaxError
|
||||||
import api.testium as tm
|
import api.testium as tm
|
||||||
|
|
||||||
class TestItemGroup(TestItem):
|
class TestItemGroup(TestItem):
|
||||||
|
|
||||||
# 'group' has no item-specific parameters; 'steps' is handled by COMMON_PARAMS.
|
|
||||||
# Declaring an empty ParamSet still opts in to unknown-param validation
|
|
||||||
# (e.g. typo 'stop_on_failures').
|
|
||||||
PARAMS = ParamSet()
|
|
||||||
|
|
||||||
def __init__(self, dict_cycle, parent = None, status_queue=None, filename=""):
|
def __init__(self, dict_cycle, parent = None, status_queue=None, filename=""):
|
||||||
self._name = cst.TYPE_GROUP.item_name
|
self._name = cst.TYPE_GROUP.item_name
|
||||||
super().__init__(dict_cycle, parent, status_queue, filename=filename)
|
super().__init__(dict_cycle, parent, status_queue, filename=filename)
|
||||||
|
|||||||
@@ -4,7 +4,6 @@ from interpreter.test_items.test_item import test_run
|
|||||||
from interpreter.test_items.test_result import TestValue
|
from interpreter.test_items.test_result import TestValue
|
||||||
from interpreter.test_items.test_item_dialog_base import TestItemDialogBase, _is_text_mode, _is_interactive
|
from interpreter.test_items.test_item_dialog_base import TestItemDialogBase, _is_text_mode, _is_interactive
|
||||||
from interpreter.utils.constants import TestItemType as cst
|
from interpreter.utils.constants import TestItemType as cst
|
||||||
from interpreter.utils.param_decl import Param, ParamSet
|
|
||||||
from runtime.tum_except import item_load_context
|
from runtime.tum_except import item_load_context
|
||||||
import api.testium as tm
|
import api.testium as tm
|
||||||
|
|
||||||
@@ -13,17 +12,6 @@ class TestItemImageDialog(TestItemDialogBase):
|
|||||||
"""dialog_image item usage.
|
"""dialog_image item usage.
|
||||||
dialog_image name: Nice image, question: could you press the red button, filename: img.jpg
|
dialog_image name: Nice image, question: could you press the red button, filename: img.jpg
|
||||||
"""
|
"""
|
||||||
|
|
||||||
PARAMS = ParamSet(
|
|
||||||
Param("question", required=True,
|
|
||||||
doc="Prompt shown above the image."),
|
|
||||||
Param("filename", required=True,
|
|
||||||
doc="Path to the image file (relative to the test directory or absolute)."),
|
|
||||||
Param("auto_result", default=None,
|
|
||||||
doc="Outcome used in batch/non-interactive mode. Truthy ⇒ SUCCESS, "
|
|
||||||
"None ⇒ FAILURE."),
|
|
||||||
)
|
|
||||||
|
|
||||||
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
|
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
|
||||||
self._name = cst.TYPE_IMAGE_DLG.item_name
|
self._name = cst.TYPE_IMAGE_DLG.item_name
|
||||||
super().__init__(dict_item, parent, status_queue, filename=filename)
|
super().__init__(dict_item, parent, status_queue, filename=filename)
|
||||||
|
|||||||
@@ -11,7 +11,6 @@ from interpreter.test_items.item_actions.action import TestItemAction
|
|||||||
|
|
||||||
from interpreter.utils.constants import TestItemType as cst
|
from interpreter.utils.constants import TestItemType as cst
|
||||||
from interpreter.utils.eval import evaluate
|
from interpreter.utils.eval import evaluate
|
||||||
from interpreter.utils.param_decl import Param, ParamSet, BLOCK
|
|
||||||
|
|
||||||
from interpreter.test_items.test_item_json_rpc.jsonrpc_adapters import (
|
from interpreter.test_items.test_item_json_rpc.jsonrpc_adapters import (
|
||||||
JrpcAdapter,
|
JrpcAdapter,
|
||||||
@@ -77,20 +76,6 @@ class TestItemJSRPCActionClose(TestItemAction):
|
|||||||
|
|
||||||
class TestItemJSRPCActionQuery(TestItemAction):
|
class TestItemJSRPCActionQuery(TestItemAction):
|
||||||
|
|
||||||
PARAMS = ParamSet(
|
|
||||||
Param("method", required=True,
|
|
||||||
doc="JSON-RPC method name to call."),
|
|
||||||
Param("params",
|
|
||||||
doc="Parameters payload (list, dict or scalar) sent to the method."),
|
|
||||||
Param("id", default="rand",
|
|
||||||
doc="JSON-RPC request id. 'rand' (default) ⇒ a random integer is used."),
|
|
||||||
Param("no_wait", default=False,
|
|
||||||
doc="If true, send the request without waiting for a response."),
|
|
||||||
Param("timeout", default=None,
|
|
||||||
doc="Seconds to wait for a response. None ⇒ inherits the transport "
|
|
||||||
"default."),
|
|
||||||
)
|
|
||||||
|
|
||||||
def __init__(
|
def __init__(
|
||||||
self, action_name, dict_item, parent=None, status_queue=None, filename=""
|
self, action_name, dict_item, parent=None, status_queue=None, filename=""
|
||||||
):
|
):
|
||||||
@@ -144,13 +129,6 @@ class TestItemJSRPCActionQuery(TestItemAction):
|
|||||||
|
|
||||||
class TestItemJSRPCActionReceive(TestItemAction):
|
class TestItemJSRPCActionReceive(TestItemAction):
|
||||||
|
|
||||||
PARAMS = ParamSet(
|
|
||||||
Param("id", required=True,
|
|
||||||
doc="JSON-RPC request id whose response we expect."),
|
|
||||||
Param("timeout", default=None,
|
|
||||||
doc="Seconds to wait for the response. None ⇒ transport default."),
|
|
||||||
)
|
|
||||||
|
|
||||||
def __init__(
|
def __init__(
|
||||||
self, action_name, dict_item, parent=None, status_queue=None, filename=""
|
self, action_name, dict_item, parent=None, status_queue=None, filename=""
|
||||||
):
|
):
|
||||||
@@ -194,22 +172,6 @@ class TestItemJSON_RPC(TestItemActions):
|
|||||||
This item TBD
|
This item TBD
|
||||||
"""
|
"""
|
||||||
|
|
||||||
PARAMS = ParamSet(
|
|
||||||
Param("console", kind=BLOCK,
|
|
||||||
doc="Console-transport block: {console_name, …}. Either 'console' "
|
|
||||||
"or 'udp' must be set."),
|
|
||||||
Param("udp", kind=BLOCK,
|
|
||||||
doc="UDP-transport block: {host, port, …}. Either 'console' or "
|
|
||||||
"'udp' must be set."),
|
|
||||||
Param("version", default="1.0",
|
|
||||||
doc="JSON-RPC protocol version ('1.0' or '2.0')."),
|
|
||||||
Param("timeout", required=True,
|
|
||||||
doc="Default seconds to wait for a JSON-RPC response across all "
|
|
||||||
"child query/receive actions."),
|
|
||||||
Param("mute", default=False,
|
|
||||||
doc="If true, don't echo wire traffic to the log."),
|
|
||||||
)
|
|
||||||
|
|
||||||
def __init__(
|
def __init__(
|
||||||
self, dict_item: dict, parent: TestItem = None, status_queue=None, filename=""
|
self, dict_item: dict, parent: TestItem = None, status_queue=None, filename=""
|
||||||
):
|
):
|
||||||
|
|||||||
@@ -8,20 +8,12 @@ from interpreter.test_items.test_result import (TestResult, TestValue)
|
|||||||
from runtime.tum_except import ETUMSyntaxError, item_load_context
|
from runtime.tum_except import ETUMSyntaxError, item_load_context
|
||||||
import api.testium as tm
|
import api.testium as tm
|
||||||
from interpreter.utils.constants import TestItemType as cst
|
from interpreter.utils.constants import TestItemType as cst
|
||||||
from interpreter.utils.param_decl import Param, ParamSet, LIST
|
|
||||||
|
|
||||||
class TestItemLet(TestItem):
|
class TestItemLet(TestItem):
|
||||||
"""let item usage.
|
"""let item usage.
|
||||||
let values: {variable1: a, variable2: /dev/ttyUSB0, variable3: 115200}
|
let values: {variable1: a, variable2: /dev/ttyUSB0, variable3: 115200}
|
||||||
let eval: {conditional_exec: "random.randint(1, 4)"}
|
let eval: {conditional_exec: "random.randint(1, 4)"}
|
||||||
"""
|
"""
|
||||||
|
|
||||||
PARAMS = ParamSet(
|
|
||||||
Param("values", kind=LIST, required=True,
|
|
||||||
doc="Mapping (or list of single-pair mappings) of global-dict "
|
|
||||||
"key → value to set. Values are expanded at execution time."),
|
|
||||||
)
|
|
||||||
|
|
||||||
def __init__(self, dict_item, parent = None, status_queue=None, filename=""):
|
def __init__(self, dict_item, parent = None, status_queue=None, filename=""):
|
||||||
self._name = cst.TYPE_LET.item_name
|
self._name = cst.TYPE_LET.item_name
|
||||||
super().__init__(dict_item, parent, status_queue, filename=filename)
|
super().__init__(dict_item, parent, status_queue, filename=filename)
|
||||||
|
|||||||
@@ -11,7 +11,6 @@ import api.testium as tm
|
|||||||
from interpreter.utils.lua_func_exec import LuaFuncExecEngine
|
from interpreter.utils.lua_func_exec import LuaFuncExecEngine
|
||||||
from interpreter.utils.api_srv import api_request
|
from interpreter.utils.api_srv import api_request
|
||||||
from interpreter.utils.constants import TestItemType as cst
|
from interpreter.utils.constants import TestItemType as cst
|
||||||
from interpreter.utils.param_decl import Param, ParamSet, LIST
|
|
||||||
|
|
||||||
_LUA_FUNC_CONTEXTS_KEY = "_lua_func_contexts"
|
_LUA_FUNC_CONTEXTS_KEY = "_lua_func_contexts"
|
||||||
|
|
||||||
@@ -22,21 +21,6 @@ class TestItemLuaFunc(TestItem):
|
|||||||
Optional: context_id: <id> — share a persistent process with other lua_func items using the same id.
|
Optional: context_id: <id> — share a persistent process with other lua_func items using the same id.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
PARAMS = ParamSet(
|
|
||||||
Param("file", required=True,
|
|
||||||
doc="Path to the .lua file containing the function."),
|
|
||||||
Param("func_name", required=True,
|
|
||||||
doc="Name of the function to call in the file."),
|
|
||||||
Param("param", kind=LIST,
|
|
||||||
doc="Arguments passed to the function. Each entry is expanded "
|
|
||||||
"before the call. Special tokens $(loop_param) / $(loop_index) "
|
|
||||||
"resolve from the surrounding cycle."),
|
|
||||||
Param("context_id", default=None,
|
|
||||||
doc="If set, the lua_func subprocess is kept alive and reused by "
|
|
||||||
"every other lua_func item with the same context_id — enables "
|
|
||||||
"shared in-memory state between successive calls."),
|
|
||||||
)
|
|
||||||
|
|
||||||
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
|
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
|
||||||
self._name = cst.TYPE_LUA_FUNCTION.item_name
|
self._name = cst.TYPE_LUA_FUNCTION.item_name
|
||||||
super().__init__(dict_item, parent, status_queue, filename=filename)
|
super().__init__(dict_item, parent, status_queue, filename=filename)
|
||||||
|
|||||||
@@ -5,7 +5,6 @@ from interpreter.test_items.test_item import test_run
|
|||||||
from interpreter.test_items.test_result import TestValue
|
from interpreter.test_items.test_result import TestValue
|
||||||
from interpreter.test_items.test_item_dialog_base import TestItemDialogBase, _is_text_mode, _is_interactive
|
from interpreter.test_items.test_item_dialog_base import TestItemDialogBase, _is_text_mode, _is_interactive
|
||||||
from interpreter.utils.constants import TestItemType as cst
|
from interpreter.utils.constants import TestItemType as cst
|
||||||
from interpreter.utils.param_decl import Param, ParamSet
|
|
||||||
from runtime.tum_except import item_load_context
|
from runtime.tum_except import item_load_context
|
||||||
|
|
||||||
|
|
||||||
@@ -13,15 +12,6 @@ class TestItemMsgDialog(TestItemDialogBase):
|
|||||||
"""dialog_message item usage.
|
"""dialog_message item usage.
|
||||||
dialog_message name: Nice message, question: Open the door and press OK
|
dialog_message name: Nice message, question: Open the door and press OK
|
||||||
"""
|
"""
|
||||||
|
|
||||||
PARAMS = ParamSet(
|
|
||||||
Param("question", required=True,
|
|
||||||
doc="Message body shown to the user. Multi-line strings are supported."),
|
|
||||||
Param("auto_result", default=None,
|
|
||||||
doc="Outcome used in batch/non-interactive mode instead of waiting "
|
|
||||||
"for the user. Truthy ⇒ SUCCESS, None ⇒ FAILURE."),
|
|
||||||
)
|
|
||||||
|
|
||||||
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
|
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
|
||||||
self._name = cst.TYPE_MESSAGE_DLG.item_name
|
self._name = cst.TYPE_MESSAGE_DLG.item_name
|
||||||
super().__init__(dict_item, parent, status_queue, filename=filename)
|
super().__init__(dict_item, parent, status_queue, filename=filename)
|
||||||
|
|||||||
@@ -2,23 +2,11 @@ from interpreter.test_items.test_item import test_run
|
|||||||
from interpreter.test_items.test_result import TestValue
|
from interpreter.test_items.test_result import TestValue
|
||||||
from interpreter.test_items.test_item_dialog_base import TestItemDialogBase, _is_text_mode, _is_interactive
|
from interpreter.test_items.test_item_dialog_base import TestItemDialogBase, _is_text_mode, _is_interactive
|
||||||
from interpreter.utils.constants import TestItemType as cst
|
from interpreter.utils.constants import TestItemType as cst
|
||||||
from interpreter.utils.param_decl import Param, ParamSet
|
|
||||||
from runtime.tum_except import item_load_context
|
from runtime.tum_except import item_load_context
|
||||||
import api.testium as tm
|
import api.testium as tm
|
||||||
|
|
||||||
|
|
||||||
class TestItemNoteDialog(TestItemDialogBase):
|
class TestItemNoteDialog(TestItemDialogBase):
|
||||||
|
|
||||||
PARAMS = ParamSet(
|
|
||||||
Param("question", required=True,
|
|
||||||
doc="Prompt shown above the note input field."),
|
|
||||||
Param("auto_result", default=None,
|
|
||||||
doc="Batch-mode outcome: None ⇒ FAILURE, 'cancel' ⇒ cancelled, "
|
|
||||||
"any other truthy ⇒ SUCCESS with auto_value."),
|
|
||||||
Param("auto_value", default=None,
|
|
||||||
doc="Note text used in batch mode when auto_result is set."),
|
|
||||||
)
|
|
||||||
|
|
||||||
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
|
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
|
||||||
self._name = cst.TYPE_NOTE_DLG.item_name
|
self._name = cst.TYPE_NOTE_DLG.item_name
|
||||||
super().__init__(dict_item, parent, status_queue, filename=filename)
|
super().__init__(dict_item, parent, status_queue, filename=filename)
|
||||||
|
|||||||
@@ -6,7 +6,6 @@ from interpreter.test_items.test_item import test_run
|
|||||||
from interpreter.test_items.test_result import TestResult, TestValue
|
from interpreter.test_items.test_result import TestResult, TestValue
|
||||||
from interpreter.utils.constants import TestItemType as cst
|
from interpreter.utils.constants import TestItemType as cst
|
||||||
from interpreter.utils.eval import eval_to_boolean
|
from interpreter.utils.eval import eval_to_boolean
|
||||||
from interpreter.utils.param_decl import Param, ParamSet, LIST, BLOCK, Enum
|
|
||||||
from runtime.tum_except import ETUMSyntaxError
|
from runtime.tum_except import ETUMSyntaxError
|
||||||
from runtime.string_queue import StringQueue
|
from runtime.string_queue import StringQueue
|
||||||
from runtime.stdout_redirect import stdio_redir
|
from runtime.stdout_redirect import stdio_redir
|
||||||
@@ -16,12 +15,6 @@ class TestItemParallelBranch(TestItemContainer):
|
|||||||
"""One branch of a parallel item. Runs its children sequentially,
|
"""One branch of a parallel item. Runs its children sequentially,
|
||||||
optionally waiting for a condition before starting."""
|
optionally waiting for a condition before starting."""
|
||||||
|
|
||||||
PARAMS = ParamSet(
|
|
||||||
Param("wait_for", kind=BLOCK,
|
|
||||||
doc="Optional block {condition, timeout} that defers the branch "
|
|
||||||
"start until the condition is truthy (or the timeout elapses)."),
|
|
||||||
)
|
|
||||||
|
|
||||||
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
|
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
|
||||||
super().__init__(cst.TYPE_PARALLEL_BRANCH, dict_item, parent, status_queue, filename=filename)
|
super().__init__(cst.TYPE_PARALLEL_BRANCH, dict_item, parent, status_queue, filename=filename)
|
||||||
self._wait_condition = None
|
self._wait_condition = None
|
||||||
@@ -94,15 +87,6 @@ class TestItemParallel(TestItemContainer):
|
|||||||
- ...
|
- ...
|
||||||
"""
|
"""
|
||||||
|
|
||||||
PARAMS = ParamSet(
|
|
||||||
Param("branches", kind=LIST, required=True,
|
|
||||||
doc="List of branch blocks (each branch holds its own 'steps' "
|
|
||||||
"and optional 'wait_for')."),
|
|
||||||
Param("sync", kind=Enum("all", "any"), default="all",
|
|
||||||
doc="'all' (default) waits for every branch; 'any' returns as "
|
|
||||||
"soon as the first branch completes."),
|
|
||||||
)
|
|
||||||
|
|
||||||
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
|
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
|
||||||
branches = dict_item.get("branches", [])
|
branches = dict_item.get("branches", [])
|
||||||
if not branches:
|
if not branches:
|
||||||
|
|||||||
@@ -11,7 +11,6 @@ import api.testium as tm
|
|||||||
from interpreter.utils.py_func_exec import PyFuncExecEngine
|
from interpreter.utils.py_func_exec import PyFuncExecEngine
|
||||||
from interpreter.utils.api_srv import api_request
|
from interpreter.utils.api_srv import api_request
|
||||||
from interpreter.utils.constants import TestItemType as cst
|
from interpreter.utils.constants import TestItemType as cst
|
||||||
from interpreter.utils.param_decl import Param, ParamSet, LIST
|
|
||||||
|
|
||||||
_PY_FUNC_CONTEXTS_KEY = "_py_func_contexts"
|
_PY_FUNC_CONTEXTS_KEY = "_py_func_contexts"
|
||||||
|
|
||||||
@@ -22,21 +21,6 @@ class TestItemPyFunc(TestItem):
|
|||||||
Optional: context_id: <id> — share a persistent process with other py_func items using the same id.
|
Optional: context_id: <id> — share a persistent process with other py_func items using the same id.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
PARAMS = ParamSet(
|
|
||||||
Param("file", required=True,
|
|
||||||
doc="Path to the .py file containing the function."),
|
|
||||||
Param("func_name", required=True,
|
|
||||||
doc="Name of the function to call in the file."),
|
|
||||||
Param("param", kind=LIST,
|
|
||||||
doc="Arguments passed to the function. Each entry is expanded "
|
|
||||||
"before the call. Special tokens $(loop_param) / $(loop_index) "
|
|
||||||
"resolve from the surrounding cycle."),
|
|
||||||
Param("context_id", default=None,
|
|
||||||
doc="If set, the py_func subprocess is kept alive and reused by "
|
|
||||||
"every other py_func item with the same context_id — enables "
|
|
||||||
"shared in-memory state between successive calls."),
|
|
||||||
)
|
|
||||||
|
|
||||||
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
|
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
|
||||||
self._name = cst.TYPE_PY_FUNCTION.item_name
|
self._name = cst.TYPE_PY_FUNCTION.item_name
|
||||||
super().__init__(dict_item, parent, status_queue, filename=filename)
|
super().__init__(dict_item, parent, status_queue, filename=filename)
|
||||||
|
|||||||
@@ -2,7 +2,6 @@ from interpreter.test_items.test_item import test_run
|
|||||||
from interpreter.test_items.test_result import TestValue
|
from interpreter.test_items.test_result import TestValue
|
||||||
from interpreter.test_items.test_item_dialog_base import TestItemDialogBase, _is_text_mode, _is_interactive
|
from interpreter.test_items.test_item_dialog_base import TestItemDialogBase, _is_text_mode, _is_interactive
|
||||||
from interpreter.utils.constants import TestItemType as cst
|
from interpreter.utils.constants import TestItemType as cst
|
||||||
from interpreter.utils.param_decl import Param, ParamSet
|
|
||||||
from runtime.tum_except import item_load_context
|
from runtime.tum_except import item_load_context
|
||||||
|
|
||||||
|
|
||||||
@@ -10,14 +9,6 @@ class TestItemQuestionDialog(TestItemDialogBase):
|
|||||||
"""dialog_question item usage.
|
"""dialog_question item usage.
|
||||||
dialog_question name: Nice question, question: "If OK, press OK, If not, press cancel"
|
dialog_question name: Nice question, question: "If OK, press OK, If not, press cancel"
|
||||||
"""
|
"""
|
||||||
|
|
||||||
PARAMS = ParamSet(
|
|
||||||
Param("question", required=True,
|
|
||||||
doc="Yes/No prompt presented to the user."),
|
|
||||||
Param("auto_result", default=None,
|
|
||||||
doc="Batch-mode answer ('yes'/'no' or truthy/falsy). None ⇒ FAILURE."),
|
|
||||||
)
|
|
||||||
|
|
||||||
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
|
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
|
||||||
self._name = cst.TYPE_QUESTION_DLG.item_name
|
self._name = cst.TYPE_QUESTION_DLG.item_name
|
||||||
super().__init__(dict_item, parent, status_queue, filename=filename)
|
super().__init__(dict_item, parent, status_queue, filename=filename)
|
||||||
|
|||||||
@@ -3,17 +3,9 @@ from interpreter.test_items.test_item import (TestItem, test_run)
|
|||||||
from interpreter.test_items.test_result import (TestValue)
|
from interpreter.test_items.test_result import (TestValue)
|
||||||
from runtime.tum_except import ETUMSyntaxError
|
from runtime.tum_except import ETUMSyntaxError
|
||||||
from interpreter.utils.constants import TestItemType as cst
|
from interpreter.utils.constants import TestItemType as cst
|
||||||
from interpreter.utils.param_decl import Param, ParamSet, LIST
|
|
||||||
from interpreter.test_report.test_report import Export
|
from interpreter.test_report.test_report import Export
|
||||||
|
|
||||||
class TestItemReport(TestItem):
|
class TestItemReport(TestItem):
|
||||||
|
|
||||||
PARAMS = ParamSet(
|
|
||||||
Param("export", kind=LIST, required=True,
|
|
||||||
doc="List of exporters to run (junit, sqlite, …). Each entry is a "
|
|
||||||
"mapping describing the exporter type and its parameters."),
|
|
||||||
)
|
|
||||||
|
|
||||||
def __init__(self, dict_item, parent = None, status_queue=None, filename=""):
|
def __init__(self, dict_item, parent = None, status_queue=None, filename=""):
|
||||||
self._name = cst.TYPE_REPORT.item_name
|
self._name = cst.TYPE_REPORT.item_name
|
||||||
super().__init__(dict_item, parent, status_queue, filename=filename)
|
super().__init__(dict_item, parent, status_queue, filename=filename)
|
||||||
|
|||||||
@@ -10,7 +10,6 @@ from interpreter.test_items.test_item import (TestItem, test_run)
|
|||||||
from interpreter.test_items.test_result import (TestValue)
|
from interpreter.test_items.test_result import (TestValue)
|
||||||
import api.testium as tm
|
import api.testium as tm
|
||||||
from interpreter.utils.constants import TestItemType as cst
|
from interpreter.utils.constants import TestItemType as cst
|
||||||
from interpreter.utils.param_decl import Param, ParamSet
|
|
||||||
from runtime.tum_except import ETUMSyntaxError, ETUMRuntimeError, item_load_context
|
from runtime.tum_except import ETUMSyntaxError, ETUMRuntimeError, item_load_context
|
||||||
|
|
||||||
|
|
||||||
@@ -25,15 +24,9 @@ def _testium_launch_cmd():
|
|||||||
appimage = os.environ.get("APPIMAGE")
|
appimage = os.environ.get("APPIMAGE")
|
||||||
if appimage:
|
if appimage:
|
||||||
return [appimage]
|
return [appimage]
|
||||||
# Flatpak: re-launch via the Flatpak app id, but on the host side —
|
# Flatpak: re-launch via the Flatpak app id.
|
||||||
# the `flatpak` CLI cannot run inside our sandbox (no D-Bus access to the
|
|
||||||
# host Flatpak service, and the host binary would need host libs that are
|
|
||||||
# ABI-incompatible with the sandbox runtime). flatpak-spawn proxies the
|
|
||||||
# call to the host via org.freedesktop.Flatpak (allowed by --talk-name in
|
|
||||||
# the manifest).
|
|
||||||
if os.path.isfile("/.flatpak-info"):
|
if os.path.isfile("/.flatpak-info"):
|
||||||
return ["flatpak-spawn", "--host",
|
return ["flatpak", "run", "org.testium.Testium"]
|
||||||
"flatpak", "run", "org.testium.Testium"]
|
|
||||||
# PyInstaller frozen exe: sys.executable is the binary itself.
|
# PyInstaller frozen exe: sys.executable is the binary itself.
|
||||||
if getattr(sys, "frozen", False):
|
if getattr(sys, "frozen", False):
|
||||||
return [sys.executable]
|
return [sys.executable]
|
||||||
@@ -58,25 +51,6 @@ def nowInBetween(start, end):
|
|||||||
|
|
||||||
|
|
||||||
class TestItemRun(TestItem):
|
class TestItemRun(TestItem):
|
||||||
|
|
||||||
PARAMS = ParamSet(
|
|
||||||
Param("tum", required=True,
|
|
||||||
doc="Path to the .tum file launched in a fresh testium instance."),
|
|
||||||
Param("param_file", default="",
|
|
||||||
doc="Optional path to a param.yaml passed to the sub-instance."),
|
|
||||||
Param("log_file", default="",
|
|
||||||
doc="Path where the sub-instance writes its log."),
|
|
||||||
Param("report_file", default="",
|
|
||||||
doc="Path where the sub-instance writes its report."),
|
|
||||||
Param("start_time",
|
|
||||||
doc="HH:MM time of day after which the sub-instance may run."),
|
|
||||||
Param("end_time",
|
|
||||||
doc="HH:MM time of day after which the sub-instance no longer runs."),
|
|
||||||
Param("wait_for_exec",
|
|
||||||
doc="If true, block until the time window opens. Requires both "
|
|
||||||
"start_time and end_time."),
|
|
||||||
)
|
|
||||||
|
|
||||||
def __init__(self, dict_item, parent = None, status_queue=None, filename=""):
|
def __init__(self, dict_item, parent = None, status_queue=None, filename=""):
|
||||||
self._name = cst.TYPE_RUN.item_name
|
self._name = cst.TYPE_RUN.item_name
|
||||||
super().__init__(dict_item, parent, status_queue, filename=filename)
|
super().__init__(dict_item, parent, status_queue, filename=filename)
|
||||||
|
|||||||
@@ -11,7 +11,6 @@ from interpreter.test_items.item_actions import TestItemActions
|
|||||||
from interpreter.test_items.item_actions.action import TestItemAction
|
from interpreter.test_items.item_actions.action import TestItemAction
|
||||||
from interpreter.utils.constants import TestItemType as cst
|
from interpreter.utils.constants import TestItemType as cst
|
||||||
from interpreter.utils.eval import evaluate
|
from interpreter.utils.eval import evaluate
|
||||||
from interpreter.utils.param_decl import Param, ParamSet, LIST
|
|
||||||
|
|
||||||
|
|
||||||
class TestItemPlotAction(TestItemAction):
|
class TestItemPlotAction(TestItemAction):
|
||||||
@@ -22,12 +21,6 @@ class TestItemPlotAction(TestItemAction):
|
|||||||
|
|
||||||
|
|
||||||
class TestItemPlotActionOpen(TestItemPlotAction):
|
class TestItemPlotActionOpen(TestItemPlotAction):
|
||||||
|
|
||||||
PARAMS = ParamSet(
|
|
||||||
Param("log_path", default=None,
|
|
||||||
doc="Optional file to which the plot data are appended."),
|
|
||||||
)
|
|
||||||
|
|
||||||
def __init__(
|
def __init__(
|
||||||
self, action_name, dict_item, parent=None, status_queue=None, filename=""
|
self, action_name, dict_item, parent=None, status_queue=None, filename=""
|
||||||
):
|
):
|
||||||
@@ -64,15 +57,6 @@ class TestItemPlotActionOpen(TestItemPlotAction):
|
|||||||
|
|
||||||
|
|
||||||
class TestItemPlotActionClose(TestItemPlotAction):
|
class TestItemPlotActionClose(TestItemPlotAction):
|
||||||
|
|
||||||
PARAMS = ParamSet(
|
|
||||||
Param("wait_dialog_exit", default=False,
|
|
||||||
doc="If true, the close action blocks until the user closes the "
|
|
||||||
"plot window (or timeout)."),
|
|
||||||
Param("timeout", default=-1,
|
|
||||||
doc="Seconds to wait when wait_dialog_exit is true. Negative ⇒ infinite."),
|
|
||||||
)
|
|
||||||
|
|
||||||
def __init__(
|
def __init__(
|
||||||
self, action_name, dict_item, parent=None, status_queue=None, filename=""
|
self, action_name, dict_item, parent=None, status_queue=None, filename=""
|
||||||
):
|
):
|
||||||
@@ -112,20 +96,6 @@ class TestItemPlotActionClose(TestItemPlotAction):
|
|||||||
|
|
||||||
|
|
||||||
class TestItemPlotActionPeriodic(TestItemPlotAction):
|
class TestItemPlotActionPeriodic(TestItemPlotAction):
|
||||||
|
|
||||||
PARAMS = ParamSet(
|
|
||||||
Param("period", required=True,
|
|
||||||
doc="Seconds between two calls of the periodic function."),
|
|
||||||
Param("file", required=True,
|
|
||||||
doc="Path to the .py file holding the periodic function."),
|
|
||||||
Param("func_name", required=True,
|
|
||||||
doc="Name of the periodic function."),
|
|
||||||
Param("param", kind=LIST,
|
|
||||||
doc="Arguments passed to the periodic function on each call."),
|
|
||||||
Param("eval", default="",
|
|
||||||
doc="Post-evaluation applied to the function's return value."),
|
|
||||||
)
|
|
||||||
|
|
||||||
def __init__(
|
def __init__(
|
||||||
self, action_name, dict_item, parent=None, status_queue=None, filename=""
|
self, action_name, dict_item, parent=None, status_queue=None, filename=""
|
||||||
):
|
):
|
||||||
@@ -199,13 +169,6 @@ class TestItemPlotActionAdd(TestItemPlotAction):
|
|||||||
|
|
||||||
|
|
||||||
class TestItemPlotActionLastValues(TestItemPlotAction):
|
class TestItemPlotActionLastValues(TestItemPlotAction):
|
||||||
|
|
||||||
PARAMS = ParamSet(
|
|
||||||
Param("name", kind=LIST,
|
|
||||||
doc="List of plot variable names whose last sample is returned. "
|
|
||||||
"Result is stored in $(plv_<plot_name>) as a dict."),
|
|
||||||
)
|
|
||||||
|
|
||||||
def __init__(self, action_name, dict_item, parent=None, status_queue=None, filename=""):
|
def __init__(self, action_name, dict_item, parent=None, status_queue=None, filename=""):
|
||||||
super().__init__(
|
super().__init__(
|
||||||
action_name, cst.TYPE_GRAPH_ACTION, dict_item, parent, status_queue, filename=filename
|
action_name, cst.TYPE_GRAPH_ACTION, dict_item, parent, status_queue, filename=filename
|
||||||
@@ -256,13 +219,6 @@ class TestItemPlotActionExport(TestItemPlotAction):
|
|||||||
|
|
||||||
|
|
||||||
class TestItemPlot(TestItemActions):
|
class TestItemPlot(TestItemActions):
|
||||||
|
|
||||||
PARAMS = ParamSet(
|
|
||||||
Param("plot_name", required=True,
|
|
||||||
doc="Identifier of the plot window — referenced by every nested "
|
|
||||||
"action and by $(plv_<plot_name>) for last-values output."),
|
|
||||||
)
|
|
||||||
|
|
||||||
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
|
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
|
||||||
super().__init__(
|
super().__init__(
|
||||||
cst.TYPE_GRAPH, dict_item, parent, status_queue, filename=filename
|
cst.TYPE_GRAPH, dict_item, parent, status_queue, filename=filename
|
||||||
|
|||||||
@@ -7,7 +7,6 @@ import api.testium as tm
|
|||||||
from interpreter.test_items.test_item import (TestItem, test_run)
|
from interpreter.test_items.test_item import (TestItem, test_run)
|
||||||
from interpreter.test_items.test_result import (TestValue)
|
from interpreter.test_items.test_result import (TestValue)
|
||||||
from interpreter.utils.constants import TestItemType as cst
|
from interpreter.utils.constants import TestItemType as cst
|
||||||
from interpreter.utils.param_decl import Param, ParamSet
|
|
||||||
from runtime.tum_except import ETUMSyntaxError, ETUMRuntimeError, item_load_context
|
from runtime.tum_except import ETUMSyntaxError, ETUMRuntimeError, item_load_context
|
||||||
|
|
||||||
class TestItemSleep(TestItem):
|
class TestItemSleep(TestItem):
|
||||||
@@ -15,15 +14,6 @@ class TestItemSleep(TestItem):
|
|||||||
sleep timeout: 10
|
sleep timeout: 10
|
||||||
"""
|
"""
|
||||||
|
|
||||||
PARAMS = ParamSet(
|
|
||||||
Param("timeout", required=True,
|
|
||||||
doc="Duration to sleep. Number of seconds, or a string "
|
|
||||||
"like '1d 2h 30m 15s'."),
|
|
||||||
Param("dialog", default=False,
|
|
||||||
doc="If true, show a cancel dialog (GUI mode) or an interactive "
|
|
||||||
"Ctrl+C-able countdown (text mode)."),
|
|
||||||
)
|
|
||||||
|
|
||||||
def __init__(self, dict_item, parent = None, status_queue=None, filename=""):
|
def __init__(self, dict_item, parent = None, status_queue=None, filename=""):
|
||||||
self._name = cst.TYPE_SLEEP.item_name
|
self._name = cst.TYPE_SLEEP.item_name
|
||||||
super().__init__(dict_item, parent, status_queue, filename=filename)
|
super().__init__(dict_item, parent, status_queue, filename=filename)
|
||||||
|
|||||||
@@ -2,23 +2,11 @@ from interpreter.test_items.test_item import test_run
|
|||||||
from interpreter.test_items.test_result import TestValue
|
from interpreter.test_items.test_result import TestValue
|
||||||
from interpreter.test_items.test_item_dialog_base import TestItemDialogBase, _is_text_mode, _is_interactive
|
from interpreter.test_items.test_item_dialog_base import TestItemDialogBase, _is_text_mode, _is_interactive
|
||||||
from interpreter.utils.constants import TestItemType as cst
|
from interpreter.utils.constants import TestItemType as cst
|
||||||
from interpreter.utils.param_decl import Param, ParamSet, LIST
|
|
||||||
from runtime.tum_except import item_load_context
|
from runtime.tum_except import item_load_context
|
||||||
import api.testium as tm
|
import api.testium as tm
|
||||||
|
|
||||||
|
|
||||||
class TestItemTestedRefsDialog(TestItemDialogBase):
|
class TestItemTestedRefsDialog(TestItemDialogBase):
|
||||||
|
|
||||||
PARAMS = ParamSet(
|
|
||||||
Param("question", required=True,
|
|
||||||
doc="Prompt asking the operator to enter the tested references."),
|
|
||||||
Param("reference", kind=LIST,
|
|
||||||
doc="Pre-filled list of references shown in the dialog."),
|
|
||||||
Param("auto_result", default=None,
|
|
||||||
doc="Batch-mode outcome: None ⇒ FAILURE, truthy ⇒ SUCCESS with "
|
|
||||||
"the pre-filled references."),
|
|
||||||
)
|
|
||||||
|
|
||||||
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
|
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
|
||||||
self._name = cst.TYPE_REFERENCE_DLG.item_name
|
self._name = cst.TYPE_REFERENCE_DLG.item_name
|
||||||
super().__init__(dict_item, parent, status_queue, filename=filename)
|
super().__init__(dict_item, parent, status_queue, filename=filename)
|
||||||
|
|||||||
@@ -11,7 +11,6 @@ from interpreter.test_items.test_item import (TestItem, test_run, LOG_TEST_STOP,
|
|||||||
from interpreter.test_items.test_result import (TestResult, TestValue)
|
from interpreter.test_items.test_result import (TestResult, TestValue)
|
||||||
from interpreter.test_items.test_item import test_data
|
from interpreter.test_items.test_item import test_data
|
||||||
from interpreter.utils.constants import TestItemType as cst
|
from interpreter.utils.constants import TestItemType as cst
|
||||||
from interpreter.utils.param_decl import Param, ParamSet, LIST
|
|
||||||
from runtime.stdout_redirect import stdio_redir
|
from runtime.stdout_redirect import stdio_redir
|
||||||
|
|
||||||
class UnittestResult(TextTestResult):
|
class UnittestResult(TextTestResult):
|
||||||
@@ -96,15 +95,6 @@ class TestItemUnittestElement(TestItem):
|
|||||||
|
|
||||||
|
|
||||||
class TestItemUnittestFile(TestItem):
|
class TestItemUnittestFile(TestItem):
|
||||||
|
|
||||||
PARAMS = ParamSet(
|
|
||||||
Param("test_file", required=True,
|
|
||||||
doc="Path to the Python unittest file (TestCase subclass)."),
|
|
||||||
Param("test_method", kind=LIST,
|
|
||||||
doc="Optional list of method names to restrict the run to. "
|
|
||||||
"When empty, every test_* method in the file is run."),
|
|
||||||
)
|
|
||||||
|
|
||||||
def __init__(self, dict_item, parent = None, status_queue=None, filename=""):
|
def __init__(self, dict_item, parent = None, status_queue=None, filename=""):
|
||||||
self._name = cst.TYPE_UNITTEST.item_name
|
self._name = cst.TYPE_UNITTEST.item_name
|
||||||
super().__init__(dict_item, parent, status_queue, filename=filename)
|
super().__init__(dict_item, parent, status_queue, filename=filename)
|
||||||
|
|||||||
@@ -2,7 +2,6 @@ from interpreter.test_items.test_item import test_run
|
|||||||
from interpreter.test_items.test_result import TestValue
|
from interpreter.test_items.test_result import TestValue
|
||||||
from interpreter.test_items.test_item_dialog_base import TestItemDialogBase, _is_text_mode, _is_interactive
|
from interpreter.test_items.test_item_dialog_base import TestItemDialogBase, _is_text_mode, _is_interactive
|
||||||
from interpreter.utils.constants import TestItemType as cst
|
from interpreter.utils.constants import TestItemType as cst
|
||||||
from interpreter.utils.param_decl import Param, ParamSet
|
|
||||||
from runtime.tum_except import item_load_context
|
from runtime.tum_except import item_load_context
|
||||||
import api.testium as tm
|
import api.testium as tm
|
||||||
|
|
||||||
@@ -11,19 +10,6 @@ class TestItemValueDialog(TestItemDialogBase):
|
|||||||
"""dialog_value item usage.
|
"""dialog_value item usage.
|
||||||
dialog_value name: Enter value, question: "Which value did you measure?"
|
dialog_value name: Enter value, question: "Which value did you measure?"
|
||||||
"""
|
"""
|
||||||
|
|
||||||
PARAMS = ParamSet(
|
|
||||||
Param("question", required=True,
|
|
||||||
doc="Prompt shown above the value input field."),
|
|
||||||
Param("default", default="",
|
|
||||||
doc="Pre-filled value of the input field."),
|
|
||||||
Param("auto_result", default=None,
|
|
||||||
doc="Batch-mode outcome: None ⇒ FAILURE, 'cancel' ⇒ cancelled, "
|
|
||||||
"any other truthy ⇒ SUCCESS with auto_value."),
|
|
||||||
Param("auto_value", default=None,
|
|
||||||
doc="Value used in batch mode when auto_result is set."),
|
|
||||||
)
|
|
||||||
|
|
||||||
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
|
def __init__(self, dict_item, parent=None, status_queue=None, filename=""):
|
||||||
self._name = cst.TYPE_VALUE_DLG.item_name
|
self._name = cst.TYPE_VALUE_DLG.item_name
|
||||||
super().__init__(dict_item, parent, status_queue, filename=filename)
|
super().__init__(dict_item, parent, status_queue, filename=filename)
|
||||||
|
|||||||
@@ -17,11 +17,8 @@ Public API
|
|||||||
``reset()`` : clear the cache (mostly useful for tests)
|
``reset()`` : clear the cache (mostly useful for tests)
|
||||||
"""
|
"""
|
||||||
|
|
||||||
import atexit
|
|
||||||
import os
|
import os
|
||||||
import shutil
|
|
||||||
import subprocess
|
import subprocess
|
||||||
import tempfile
|
|
||||||
|
|
||||||
import api.testium as tm
|
import api.testium as tm
|
||||||
from interpreter.utils.paths import sys_app_path_lin, sys_app_path_win
|
from interpreter.utils.paths import sys_app_path_lin, sys_app_path_win
|
||||||
@@ -33,6 +30,20 @@ from runtime.tum_except import ETUMRuntimeError
|
|||||||
_PYTHON_CANDIDATES = ["python3", "python"]
|
_PYTHON_CANDIDATES = ["python3", "python"]
|
||||||
_LUA_CANDIDATES = ["lua", "lua5.5", "lua5.4", "lua5.3", "lua5.2", "lua5.1"]
|
_LUA_CANDIDATES = ["lua", "lua5.5", "lua5.4", "lua5.3", "lua5.2", "lua5.1"]
|
||||||
|
|
||||||
|
# When running inside a Flatpak, --filesystem=host-os mounts the host at
|
||||||
|
# /run/host (read-only). Binaries and libraries from the host are not on the
|
||||||
|
# sandbox PATH/LD_LIBRARY_PATH, so we probe and inject them explicitly.
|
||||||
|
_FLATPAK_HOST_DIRS = [
|
||||||
|
"/run/host/usr/local/bin",
|
||||||
|
"/run/host/usr/bin",
|
||||||
|
"/run/host/bin",
|
||||||
|
]
|
||||||
|
_FLATPAK_HOST_LIB_DIRS = [
|
||||||
|
"/run/host/usr/lib",
|
||||||
|
"/run/host/usr/lib64",
|
||||||
|
"/run/host/usr/local/lib",
|
||||||
|
]
|
||||||
|
|
||||||
# Inside an AppImage, AppRun prepends $APPDIR/usr/bin to PATH and exports a
|
# Inside an AppImage, AppRun prepends $APPDIR/usr/bin to PATH and exports a
|
||||||
# bundle-local PYTHONHOME / PYTHONPATH / LD_LIBRARY_PATH. We want py_func and
|
# bundle-local PYTHONHOME / PYTHONPATH / LD_LIBRARY_PATH. We want py_func and
|
||||||
# lua_func to run under the *host* interpreter (not the bundled one), so we
|
# lua_func to run under the *host* interpreter (not the bundled one), so we
|
||||||
@@ -53,17 +64,50 @@ def _in_appimage():
|
|||||||
return "APPIMAGE" in os.environ
|
return "APPIMAGE" in os.environ
|
||||||
|
|
||||||
|
|
||||||
def apply_host_libs(env):
|
def apply_host_lua_paths(env):
|
||||||
"""Strip bundle-local entries from *env* so a host binary can run cleanly.
|
"""Prepend host Lua module dirs to LUA_PATH / LUA_CPATH (Flatpak only).
|
||||||
|
|
||||||
Only meaningful for AppImage: removes $APPDIR-prefixed entries from
|
Must be called after user-defined lua_env overrides are applied, so host
|
||||||
LD_LIBRARY_PATH / PYTHONPATH / PATH and drops PYTHONHOME, so the host
|
paths are always first regardless of user config. User-defined paths remain
|
||||||
interpreter doesn't try to load the bundled (incompatible) Python
|
in the variable but after the host ones.
|
||||||
lib/site-packages. Flatpak is handled via flatpak-spawn --host instead
|
|
||||||
(see flatpak_host_spawn), so the sandbox env is irrelevant there.
|
|
||||||
"""
|
"""
|
||||||
if not _in_appimage():
|
if not _in_flatpak():
|
||||||
return
|
return
|
||||||
|
_LUA_VERSIONS = ["5.5", "5.4", "5.3", "5.2", "5.1"]
|
||||||
|
_HOST = "/run/host/usr"
|
||||||
|
cpath_dirs, lpath_dirs = [], []
|
||||||
|
for v in _LUA_VERSIONS:
|
||||||
|
for base in [f"{_HOST}/lib/lua/{v}",
|
||||||
|
f"{_HOST}/lib64/lua/{v}",
|
||||||
|
f"{_HOST}/lib/x86_64-linux-gnu/lua/{v}"]:
|
||||||
|
cpath_dirs.append(f"{base}/?.so")
|
||||||
|
lpath_dirs.append(f"{_HOST}/share/lua/{v}/?.lua")
|
||||||
|
lpath_dirs.append(f"{_HOST}/share/lua/{v}/?/init.lua")
|
||||||
|
sep = ";"
|
||||||
|
host_cpath = sep.join(cpath_dirs)
|
||||||
|
host_lpath = sep.join(lpath_dirs)
|
||||||
|
# ;; keeps Lua's compiled-in defaults at the end as last resort
|
||||||
|
env["LUA_CPATH"] = host_cpath + sep + env.get("LUA_CPATH", ";;")
|
||||||
|
env["LUA_PATH"] = host_lpath + sep + env.get("LUA_PATH", ";;")
|
||||||
|
|
||||||
|
|
||||||
|
def apply_host_libs(env):
|
||||||
|
"""Prepare *env* for launching a host binary from inside our bundle.
|
||||||
|
|
||||||
|
- Flatpak: prepend host library dirs to LD_LIBRARY_PATH so the dynamic
|
||||||
|
linker can find host .so files mounted under /run/host.
|
||||||
|
- AppImage: strip $APPDIR-prefixed entries from LD_LIBRARY_PATH and
|
||||||
|
PYTHONPATH and drop PYTHONHOME, so the host interpreter doesn't try
|
||||||
|
to load the bundled (incompatible) Python lib/site-packages.
|
||||||
|
- Otherwise: no-op.
|
||||||
|
"""
|
||||||
|
if _in_flatpak():
|
||||||
|
dirs = ":".join(d for d in _FLATPAK_HOST_LIB_DIRS if os.path.isdir(d))
|
||||||
|
if dirs:
|
||||||
|
existing = env.get("LD_LIBRARY_PATH", "")
|
||||||
|
env["LD_LIBRARY_PATH"] = dirs + (":" + existing if existing else "")
|
||||||
|
return
|
||||||
|
if _in_appimage():
|
||||||
appdir = os.environ.get("APPDIR", "")
|
appdir = os.environ.get("APPDIR", "")
|
||||||
if appdir:
|
if appdir:
|
||||||
for var, sep in (("LD_LIBRARY_PATH", ":"),
|
for var, sep in (("LD_LIBRARY_PATH", ":"),
|
||||||
@@ -83,132 +127,15 @@ def apply_host_libs(env):
|
|||||||
env.pop("PYTHONHOME", None)
|
env.pop("PYTHONHOME", None)
|
||||||
|
|
||||||
|
|
||||||
# ---------- Flatpak: spawn on host outside the sandbox -----------------------
|
|
||||||
#
|
|
||||||
# Inside a Flatpak the sandbox glibc is incompatible with host shared libraries,
|
|
||||||
# so we can't run host Python/Lua under the sandbox runtime — `LD_LIBRARY_PATH`
|
|
||||||
# tricks hit a `_dl_call_libc_early_init` assertion. The supported way out is
|
|
||||||
# `flatpak-spawn --host`, which talks to the session-bus Flatpak D-Bus service
|
|
||||||
# (org.freedesktop.Flatpak.Development) and asks it to spawn a process in the
|
|
||||||
# host execution environment instead of inside our sandbox. The manifest must
|
|
||||||
# grant `--talk-name=org.freedesktop.Flatpak` for the D-Bus call to be allowed.
|
|
||||||
#
|
|
||||||
# The host process can't see our /app/ contents (sandbox-only), so when we
|
|
||||||
# spawn host Python/Lua to run `py_func` / `lua_func`, the cwd must be a
|
|
||||||
# directory both sides can reach. /tmp is shared (--filesystem=/tmp), so we
|
|
||||||
# stage the testium package there once per process and reuse it for every
|
|
||||||
# spawn. In source mode (testium under $HOME) the host already sees the
|
|
||||||
# original path, so we skip the copy.
|
|
||||||
|
|
||||||
_staged_testium_path = None
|
|
||||||
|
|
||||||
|
|
||||||
def _get_host_testium_path():
|
|
||||||
"""Return a path to the testium package that the host can read.
|
|
||||||
|
|
||||||
- Source / wheel / PyInstaller install under $HOME → return testium_path()
|
|
||||||
as-is (host sees the same path via --filesystem=home).
|
|
||||||
- Flatpak bundle (testium under /app/) → stage a copy under /tmp on first
|
|
||||||
call and reuse it for the rest of the process.
|
|
||||||
"""
|
|
||||||
global _staged_testium_path
|
|
||||||
if _staged_testium_path is not None:
|
|
||||||
return _staged_testium_path
|
|
||||||
|
|
||||||
# Imported lazily to avoid a circular import (paths.py -> api.testium).
|
|
||||||
from interpreter.utils.paths import testium_path
|
|
||||||
tp = testium_path()
|
|
||||||
|
|
||||||
if not tp.startswith("/app/"):
|
|
||||||
_staged_testium_path = tp
|
|
||||||
return tp
|
|
||||||
|
|
||||||
staged = tempfile.mkdtemp(prefix="testium_host_", dir="/tmp")
|
|
||||||
# copytree refuses to write into an existing dir unless dirs_exist_ok=True.
|
|
||||||
# mkdtemp creates the dir, so we copy *into* it.
|
|
||||||
for entry in os.listdir(tp):
|
|
||||||
src = os.path.join(tp, entry)
|
|
||||||
dst = os.path.join(staged, entry)
|
|
||||||
if os.path.isdir(src):
|
|
||||||
shutil.copytree(src, dst, symlinks=True)
|
|
||||||
else:
|
|
||||||
shutil.copy2(src, dst, follow_symlinks=False)
|
|
||||||
_staged_testium_path = staged
|
|
||||||
atexit.register(shutil.rmtree, staged, ignore_errors=True)
|
|
||||||
return staged
|
|
||||||
|
|
||||||
|
|
||||||
_FORWARDED_ENV_KEYS = (
|
|
||||||
"HOME", "USER", "LOGNAME", "TMPDIR",
|
|
||||||
"XDG_RUNTIME_DIR", "XDG_DATA_HOME", "XDG_CONFIG_HOME", "XDG_CACHE_HOME",
|
|
||||||
"DBUS_SESSION_BUS_ADDRESS", "DISPLAY", "WAYLAND_DISPLAY",
|
|
||||||
"LANG", "LC_ALL",
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def flatpak_host_spawn(interp_bin, cmd_args, host_cwd, extra_env=None):
|
|
||||||
"""Build a flatpak-spawn --host command vector.
|
|
||||||
|
|
||||||
Args:
|
|
||||||
interp_bin: absolute path to the host interpreter (e.g. /usr/bin/python3).
|
|
||||||
cmd_args: list of arguments passed to the interpreter.
|
|
||||||
host_cwd: working directory on the host (must be reachable from host).
|
|
||||||
extra_env: optional {name: value} of env vars to set on the host side
|
|
||||||
in addition to the default forwarded set. Values of ""
|
|
||||||
unset the variable on the host.
|
|
||||||
|
|
||||||
Returns a list suitable for subprocess.Popen.
|
|
||||||
"""
|
|
||||||
spawn = ["flatpak-spawn", "--host", f"--directory={host_cwd}"]
|
|
||||||
forwarded = {}
|
|
||||||
for key in _FORWARDED_ENV_KEYS:
|
|
||||||
val = os.environ.get(key)
|
|
||||||
if val:
|
|
||||||
forwarded[key] = val
|
|
||||||
if extra_env:
|
|
||||||
forwarded.update(extra_env)
|
|
||||||
for k, v in forwarded.items():
|
|
||||||
if v == "":
|
|
||||||
spawn.append(f"--unset-env={k}")
|
|
||||||
else:
|
|
||||||
spawn.append(f"--env={k}={v}")
|
|
||||||
spawn.append(interp_bin)
|
|
||||||
spawn.extend(cmd_args)
|
|
||||||
return spawn
|
|
||||||
|
|
||||||
|
|
||||||
def _which_host_flatpak(name):
|
|
||||||
"""Resolve a binary name (or absolute path) on the host via flatpak-spawn.
|
|
||||||
|
|
||||||
We can't probe /run/host/... because (a) only host-os is mounted there,
|
|
||||||
not arbitrary paths like /scratch, and (b) returning a /run/host path
|
|
||||||
would be useless — the host-side spawn sees a different filesystem and
|
|
||||||
needs the host-native path anyway.
|
|
||||||
"""
|
|
||||||
if os.path.isabs(name):
|
|
||||||
cmd = flatpak_host_spawn("/bin/sh", ["-c", f'test -x "{name}"'],
|
|
||||||
host_cwd="/tmp")
|
|
||||||
try:
|
|
||||||
r = subprocess.run(cmd, capture_output=True, timeout=10)
|
|
||||||
except (FileNotFoundError, PermissionError, subprocess.TimeoutExpired):
|
|
||||||
return ""
|
|
||||||
return name if r.returncode == 0 else ""
|
|
||||||
cmd = flatpak_host_spawn("/bin/sh", ["-c", f'command -v "{name}"'],
|
|
||||||
host_cwd="/tmp")
|
|
||||||
try:
|
|
||||||
r = subprocess.run(cmd, capture_output=True, text=True, timeout=10)
|
|
||||||
except (FileNotFoundError, PermissionError, subprocess.TimeoutExpired):
|
|
||||||
return ""
|
|
||||||
if r.returncode != 0:
|
|
||||||
return ""
|
|
||||||
return r.stdout.strip()
|
|
||||||
|
|
||||||
|
|
||||||
def _which(name):
|
def _which(name):
|
||||||
if tm.OS() == "Windows":
|
if tm.OS() == "Windows":
|
||||||
return sys_app_path_win(name)
|
return sys_app_path_win(name)
|
||||||
if _in_flatpak():
|
if _in_flatpak():
|
||||||
return _which_host_flatpak(name)
|
for d in _FLATPAK_HOST_DIRS:
|
||||||
|
p = os.path.join(d, name)
|
||||||
|
if os.path.isfile(p) and os.access(p, os.X_OK):
|
||||||
|
return p
|
||||||
|
return ""
|
||||||
if _in_appimage():
|
if _in_appimage():
|
||||||
for d in _APPIMAGE_HOST_DIRS:
|
for d in _APPIMAGE_HOST_DIRS:
|
||||||
p = os.path.join(d, name)
|
p = os.path.join(d, name)
|
||||||
@@ -219,33 +146,14 @@ def _which(name):
|
|||||||
|
|
||||||
|
|
||||||
def _probe_env():
|
def _probe_env():
|
||||||
"""Subprocess env for probing host binaries.
|
"""Subprocess env for probing host binaries (adds host libs in Flatpak)."""
|
||||||
|
|
||||||
In AppImage we still need to scrub APPDIR-prefixed entries; in Flatpak we
|
|
||||||
delegate execution to the host via flatpak-spawn so the sandbox env doesn't
|
|
||||||
matter, but apply_host_libs is a no-op cost.
|
|
||||||
"""
|
|
||||||
env = os.environ.copy()
|
env = os.environ.copy()
|
||||||
apply_host_libs(env)
|
apply_host_libs(env)
|
||||||
return env
|
return env
|
||||||
|
|
||||||
|
|
||||||
def _run_probe(cmd):
|
def _python_version(path):
|
||||||
"""Run a probe command, dispatching through flatpak-spawn --host in Flatpak.
|
cmd = [path, "-c", "import sys; print(sys.version_info[:3])"]
|
||||||
|
|
||||||
Returns (stdout, stderr) as str, or None on failure.
|
|
||||||
"""
|
|
||||||
if _in_flatpak():
|
|
||||||
spawn = flatpak_host_spawn(cmd[0], cmd[1:], host_cwd="/tmp")
|
|
||||||
try:
|
|
||||||
r = subprocess.run(
|
|
||||||
spawn, capture_output=True, text=True, timeout=10,
|
|
||||||
)
|
|
||||||
except (FileNotFoundError, PermissionError, subprocess.TimeoutExpired):
|
|
||||||
return None
|
|
||||||
if r.returncode != 0:
|
|
||||||
return None
|
|
||||||
return r.stdout, r.stderr
|
|
||||||
try:
|
try:
|
||||||
r = subprocess.run(
|
r = subprocess.run(
|
||||||
cmd, capture_output=True, text=True,
|
cmd, capture_output=True, text=True,
|
||||||
@@ -253,15 +161,8 @@ def _run_probe(cmd):
|
|||||||
)
|
)
|
||||||
except (FileNotFoundError, PermissionError, subprocess.TimeoutExpired):
|
except (FileNotFoundError, PermissionError, subprocess.TimeoutExpired):
|
||||||
return None
|
return None
|
||||||
return r.stdout, r.stderr
|
|
||||||
|
|
||||||
|
|
||||||
def _python_version(path):
|
|
||||||
out = _run_probe([path, "-c", "import sys; print(sys.version_info[:3])"])
|
|
||||||
if out is None:
|
|
||||||
return None
|
|
||||||
try:
|
try:
|
||||||
return eval(out[0])
|
return eval(r.stdout)
|
||||||
except Exception:
|
except Exception:
|
||||||
return None
|
return None
|
||||||
|
|
||||||
@@ -272,11 +173,15 @@ def _is_python3(path):
|
|||||||
|
|
||||||
|
|
||||||
def _lua_version(path):
|
def _lua_version(path):
|
||||||
out = _run_probe([path, "-v"])
|
try:
|
||||||
if out is None:
|
r = subprocess.run(
|
||||||
|
[path, "-v"], capture_output=True, text=True, timeout=10,
|
||||||
|
env=_probe_env(),
|
||||||
|
)
|
||||||
|
except (FileNotFoundError, PermissionError, subprocess.TimeoutExpired):
|
||||||
return None
|
return None
|
||||||
# On Windows the version banner goes to stderr.
|
# On Windows the version banner goes to stderr.
|
||||||
line = out[0] or out[1]
|
line = r.stdout or r.stderr
|
||||||
try:
|
try:
|
||||||
major, minor, _patch = line.split(" ")[1].split(".")
|
major, minor, _patch = line.split(" ")[1].split(".")
|
||||||
return (int(major), int(minor))
|
return (int(major), int(minor))
|
||||||
@@ -297,33 +202,22 @@ _SPECS = {
|
|||||||
"lua": ("Lua 5.1+", "lua_bin", _LUA_CANDIDATES, _is_lua51),
|
"lua": ("Lua 5.1+", "lua_bin", _LUA_CANDIDATES, _is_lua51),
|
||||||
}
|
}
|
||||||
|
|
||||||
# Cached per (name, override) so that runtime changes to gd[gd_key] —
|
|
||||||
# e.g. ``python_bin`` set from a YAML config file loaded *after*
|
|
||||||
# eval_proc has already resolved its own interpreter — are picked up by
|
|
||||||
# the next lookup instead of returning the stale, auto-discovered path.
|
|
||||||
# Long-lived subprocesses (eval_proc) keep whatever they captured at
|
|
||||||
# construction time, but every new PyProcessBase / FuncExecEngine spawned
|
|
||||||
# afterwards sees the current override.
|
|
||||||
_resolved = {}
|
_resolved = {}
|
||||||
|
|
||||||
|
|
||||||
def _resolve(name):
|
def _resolve(name):
|
||||||
|
if name in _resolved:
|
||||||
|
return _resolved[name]
|
||||||
|
|
||||||
display, gd_key, candidates, validator = _SPECS[name]
|
display, gd_key, candidates, validator = _SPECS[name]
|
||||||
override = tm.gd(gd_key, "") or ""
|
override = tm.gd(gd_key, "") or ""
|
||||||
|
|
||||||
cached = _resolved.get(name)
|
|
||||||
if cached is not None and cached[0] == override:
|
|
||||||
return cached[1]
|
|
||||||
|
|
||||||
path = ""
|
path = ""
|
||||||
if override:
|
if override:
|
||||||
# Absolute path: accept as-is (user knows exactly what they want).
|
# Absolute path: accept as-is (user knows exactly what they want).
|
||||||
# Bare name: resolve via _which() so the override stays host-only in
|
# Bare name: resolve via _which() so the override stays host-only in
|
||||||
# Flatpak/AppImage instead of silently picking the bundled interpreter.
|
# Flatpak/AppImage instead of silently picking the bundled interpreter.
|
||||||
# In Flatpak we always defer to _which() so even absolute paths are
|
if os.path.isabs(override):
|
||||||
# checked from the host's perspective (the sandbox can't see e.g.
|
|
||||||
# /scratch/... paths that the user may have configured).
|
|
||||||
if os.path.isabs(override) and not _in_flatpak():
|
|
||||||
resolved = override if (os.path.isfile(override)
|
resolved = override if (os.path.isfile(override)
|
||||||
and os.access(override, os.X_OK)) else ""
|
and os.access(override, os.X_OK)) else ""
|
||||||
else:
|
else:
|
||||||
@@ -345,7 +239,7 @@ def _resolve(name):
|
|||||||
path = p
|
path = p
|
||||||
break
|
break
|
||||||
|
|
||||||
_resolved[name] = (override, path)
|
_resolved[name] = path
|
||||||
return path
|
return path
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -47,13 +47,6 @@ class LuaProcessBase:
|
|||||||
if self._process is not None:
|
if self._process is not None:
|
||||||
raise ETUMRuntimeError("The function subprocess has already been started.")
|
raise ETUMRuntimeError("The function subprocess has already been started.")
|
||||||
|
|
||||||
# In Flatpak the host can't see /app/lib/testium/lua_func, so use a
|
|
||||||
# staged copy under /tmp (shared between sandbox and host).
|
|
||||||
if bins._in_flatpak():
|
|
||||||
func_proc_path = os.path.join(
|
|
||||||
bins._get_host_testium_path(), "lua_func"
|
|
||||||
)
|
|
||||||
else:
|
|
||||||
func_proc_path = os.path.realpath(
|
func_proc_path = os.path.realpath(
|
||||||
os.path.join(subproc_path(), "lua_func")
|
os.path.join(subproc_path(), "lua_func")
|
||||||
)
|
)
|
||||||
@@ -78,6 +71,7 @@ class LuaProcessBase:
|
|||||||
env[k] = e
|
env[k] = e
|
||||||
else:
|
else:
|
||||||
env[k] = e + ";" + env.get(k, "")
|
env[k] = e + ";" + env.get(k, "")
|
||||||
|
bins.apply_host_lua_paths(env)
|
||||||
|
|
||||||
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
|
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
|
||||||
sock.bind(("localhost", 0))
|
sock.bind(("localhost", 0))
|
||||||
@@ -85,7 +79,8 @@ class LuaProcessBase:
|
|||||||
sock.close()
|
sock.close()
|
||||||
|
|
||||||
# POpen params
|
# POpen params
|
||||||
cmd_args = [
|
params = [
|
||||||
|
self._lbin,
|
||||||
"main.lua",
|
"main.lua",
|
||||||
"--timeout",
|
"--timeout",
|
||||||
f"{self._timeout}",
|
f"{self._timeout}",
|
||||||
@@ -96,31 +91,14 @@ class LuaProcessBase:
|
|||||||
]
|
]
|
||||||
|
|
||||||
if tm.debug_enabled() and tm.gd("debug_rpc", False):
|
if tm.debug_enabled() and tm.gd("debug_rpc", False):
|
||||||
cmd_args.append("--verbose")
|
params.append("--verbose")
|
||||||
|
|
||||||
if bins._in_flatpak():
|
|
||||||
# Run on the host outside the sandbox: avoids glibc ABI mismatches
|
|
||||||
# between the Flatpak runtime and host shared libraries.
|
|
||||||
host_env = {
|
|
||||||
k: env[k] for k in ("LUA_PATH", "LUA_CPATH", "PATH")
|
|
||||||
if k in env and env[k]
|
|
||||||
}
|
|
||||||
params = bins.flatpak_host_spawn(
|
|
||||||
self._lbin, cmd_args, host_cwd=func_proc_path,
|
|
||||||
extra_env=host_env,
|
|
||||||
)
|
|
||||||
popen_kwargs = {}
|
|
||||||
else:
|
|
||||||
params = [self._lbin, *cmd_args]
|
|
||||||
popen_kwargs = {"env": env, "cwd": func_proc_path}
|
|
||||||
|
|
||||||
self._process = subprocess.Popen(
|
self._process = subprocess.Popen(
|
||||||
params,
|
params, env=env, cwd=func_proc_path,
|
||||||
stdin=subprocess.DEVNULL,
|
stdin=subprocess.DEVNULL,
|
||||||
stdout=subprocess.PIPE,
|
stdout=subprocess.PIPE,
|
||||||
stderr=subprocess.PIPE,
|
stderr=subprocess.PIPE,
|
||||||
restore_signals=False,
|
restore_signals=False,
|
||||||
**popen_kwargs,
|
|
||||||
)
|
)
|
||||||
# Route subprocess stdout/stderr (lua require failures, syntax
|
# Route subprocess stdout/stderr (lua require failures, syntax
|
||||||
# errors, anything written to fd 1/2 before the in-script
|
# errors, anything written to fd 1/2 before the in-script
|
||||||
|
|||||||
@@ -1,175 +0,0 @@
|
|||||||
"""Declarative description of a test item's accepted parameters.
|
|
||||||
|
|
||||||
Each ``TestItem`` subclass declares its parameter surface as a class
|
|
||||||
attribute::
|
|
||||||
|
|
||||||
class TestItemFoo(TestItem):
|
|
||||||
PARAMS = ParamSet(
|
|
||||||
Param("bar", required=True, doc="The bar value."),
|
|
||||||
Param("baz", default=0, doc="Optional baz."),
|
|
||||||
Param("modes", kind=LIST, doc="Iterable of modes."),
|
|
||||||
Param("strategy", kind=ENUM("a", "b"), doc="..."),
|
|
||||||
Param("opts", kind=BLOCK, doc="Sub-block."),
|
|
||||||
)
|
|
||||||
|
|
||||||
The base ``TestItem.__init__`` consumes both ``COMMON_PARAMS`` (defined
|
|
||||||
in ``test_item.py``) and the subclass ``PARAMS`` to:
|
|
||||||
|
|
||||||
* warn on any key in the user's YAML that isn't declared anywhere
|
|
||||||
(catches typos like ``param_filee``);
|
|
||||||
* expose a machine-readable schema for documentation generation and,
|
|
||||||
eventually, an LSP server.
|
|
||||||
|
|
||||||
The descriptor is **purely about shape and naming**. Type coercion and
|
|
||||||
runtime checking of expanded values remain the responsibility of each
|
|
||||||
item's ``execute()`` method — most parameters are expressions
|
|
||||||
(``$(...)`` / ``<| ... |>``) whose effective type is only known after
|
|
||||||
expansion, so a static type would be misleading.
|
|
||||||
|
|
||||||
Validation of *values* (e.g. ``start_time`` must match HH:MM) can be
|
|
||||||
attached per-param via ``validate=lambda v: ...`` and is applied at
|
|
||||||
execution time on the expanded value, not at load time.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from dataclasses import dataclass, field
|
|
||||||
from typing import Any, Callable, Optional, Union
|
|
||||||
|
|
||||||
|
|
||||||
# ---------- Parameter "kinds" -------------------------------------------------
|
|
||||||
#
|
|
||||||
# These describe the YAML *shape* expected for a parameter, not its
|
|
||||||
# semantic type. They drive the LSP completion (do we suggest a single
|
|
||||||
# value, a list, a sub-block, an enum picker?) and the unknown-param
|
|
||||||
# diagnostic; nothing more.
|
|
||||||
|
|
||||||
SCALAR = "scalar" # single value (string, number, bool, expression, ...)
|
|
||||||
LIST = "list" # YAML list — the historical ``getParamAll`` case
|
|
||||||
BLOCK = "block" # nested dict — e.g. ``cycle.exit:``
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
|
||||||
class Enum:
|
|
||||||
"""Closed enumeration of acceptable scalar values."""
|
|
||||||
values: tuple
|
|
||||||
|
|
||||||
def __init__(self, *values):
|
|
||||||
# frozen=True forbids assignment; bypass via object.__setattr__.
|
|
||||||
object.__setattr__(self, "values", tuple(values))
|
|
||||||
|
|
||||||
def __repr__(self):
|
|
||||||
return f"Enum({', '.join(repr(v) for v in self.values)})"
|
|
||||||
|
|
||||||
|
|
||||||
Kind = Union[str, Enum]
|
|
||||||
|
|
||||||
|
|
||||||
# ---------- The descriptor ----------------------------------------------------
|
|
||||||
|
|
||||||
_MISSING = object()
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
|
||||||
class Param:
|
|
||||||
"""Declarative description of one accepted parameter.
|
|
||||||
|
|
||||||
Attributes
|
|
||||||
----------
|
|
||||||
name : str
|
|
||||||
The YAML key.
|
|
||||||
kind : ``SCALAR`` (default) | ``LIST`` | ``BLOCK`` | ``Enum(...)``
|
|
||||||
The YAML shape expected.
|
|
||||||
required : bool
|
|
||||||
If True, missing the parameter is a load-time error.
|
|
||||||
default : Any
|
|
||||||
Default value when the parameter is absent. ``_MISSING`` when no
|
|
||||||
default was set (used to distinguish "absent" from "None").
|
|
||||||
doc : str
|
|
||||||
Free-form description used for hover / generated documentation.
|
|
||||||
validate : Optional[Callable[[Any], bool]]
|
|
||||||
Optional post-expansion validator, evaluated at ``execute()``
|
|
||||||
time on the effective (expanded) value. Returning ``False``
|
|
||||||
raises a clear error pointing at the param.
|
|
||||||
"""
|
|
||||||
name: str
|
|
||||||
kind: Kind = SCALAR
|
|
||||||
required: bool = False
|
|
||||||
default: Any = _MISSING
|
|
||||||
doc: str = ""
|
|
||||||
validate: Optional[Callable[[Any], bool]] = None
|
|
||||||
|
|
||||||
def has_default(self):
|
|
||||||
return self.default is not _MISSING
|
|
||||||
|
|
||||||
def to_schema(self):
|
|
||||||
"""Return a dict suitable for JSON Schema generation."""
|
|
||||||
s = {"name": self.name, "required": self.required, "doc": self.doc}
|
|
||||||
if isinstance(self.kind, Enum):
|
|
||||||
s["kind"] = "enum"
|
|
||||||
s["enum"] = list(self.kind.values)
|
|
||||||
else:
|
|
||||||
s["kind"] = self.kind
|
|
||||||
if self.has_default():
|
|
||||||
s["default"] = self.default
|
|
||||||
return s
|
|
||||||
|
|
||||||
|
|
||||||
class ParamSet:
|
|
||||||
"""Ordered, name-indexed collection of ``Param`` descriptors.
|
|
||||||
|
|
||||||
Supports concatenation (``COMMON_PARAMS + SUBCLASS_PARAMS``) to
|
|
||||||
merge the common surface with each item's own params. Later
|
|
||||||
declarations override earlier ones (so a subclass can tighten a
|
|
||||||
common param's docstring without redeclaring everything).
|
|
||||||
"""
|
|
||||||
|
|
||||||
def __init__(self, *params):
|
|
||||||
self._params = {}
|
|
||||||
for p in params:
|
|
||||||
self.add(p)
|
|
||||||
|
|
||||||
def add(self, param):
|
|
||||||
if not isinstance(param, Param):
|
|
||||||
raise TypeError(f"ParamSet only accepts Param instances, got {type(param).__name__}")
|
|
||||||
self._params[param.name] = param
|
|
||||||
|
|
||||||
def __iter__(self):
|
|
||||||
return iter(self._params.values())
|
|
||||||
|
|
||||||
def __contains__(self, name):
|
|
||||||
return name in self._params
|
|
||||||
|
|
||||||
def __getitem__(self, name):
|
|
||||||
return self._params[name]
|
|
||||||
|
|
||||||
def names(self):
|
|
||||||
return tuple(self._params.keys())
|
|
||||||
|
|
||||||
def __add__(self, other):
|
|
||||||
if not isinstance(other, ParamSet):
|
|
||||||
return NotImplemented
|
|
||||||
merged = ParamSet()
|
|
||||||
merged._params = {**self._params, **other._params}
|
|
||||||
return merged
|
|
||||||
|
|
||||||
def to_schema(self):
|
|
||||||
return [p.to_schema() for p in self._params.values()]
|
|
||||||
|
|
||||||
|
|
||||||
# ---------- Validation primitives --------------------------------------------
|
|
||||||
|
|
||||||
def unknown_keys(declared, user_dict):
|
|
||||||
"""Return the user-provided keys that are not declared in *declared*.
|
|
||||||
|
|
||||||
*declared* is a ``ParamSet``; *user_dict* is the raw YAML mapping
|
|
||||||
for the item. Unknown keys catch typos and obsolete parameters.
|
|
||||||
"""
|
|
||||||
if not isinstance(user_dict, dict):
|
|
||||||
return ()
|
|
||||||
return tuple(k for k in user_dict.keys() if k not in declared)
|
|
||||||
|
|
||||||
|
|
||||||
def missing_required(declared, user_dict):
|
|
||||||
"""Return the names of declared required params absent from *user_dict*."""
|
|
||||||
if not isinstance(user_dict, dict):
|
|
||||||
return tuple(p.name for p in declared if p.required)
|
|
||||||
return tuple(p.name for p in declared if p.required and p.name not in user_dict)
|
|
||||||
@@ -61,18 +61,14 @@ class PyProcessBase:
|
|||||||
if sock is not None:
|
if sock is not None:
|
||||||
sock.close()
|
sock.close()
|
||||||
|
|
||||||
# In Flatpak the host can't see /app/lib/testium, so use a staged copy
|
# Add the path of the subprocess (root sources of testium)
|
||||||
# under /tmp (shared between sandbox and host) for both cwd and as the
|
|
||||||
# root in PYTHONPATH. Outside Flatpak the original paths are used.
|
|
||||||
if bins._in_flatpak():
|
|
||||||
tstium_path = bins._get_host_testium_path()
|
|
||||||
func_proc_path = tstium_path
|
|
||||||
else:
|
|
||||||
tstium_path = os.path.realpath(testium_path())
|
tstium_path = os.path.realpath(testium_path())
|
||||||
func_proc_path = os.path.realpath(subproc_path())
|
func_proc_path = os.path.realpath(subproc_path())
|
||||||
env["PYTHONPATH"] = tstium_path + os.pathsep + self._ppath + os.pathsep + env.get("PYTHONPATH", "")
|
env["PYTHONPATH"] = tstium_path + os.pathsep + self._ppath + os.pathsep + env.get("PYTHONPATH", "")
|
||||||
|
|
||||||
cmd_args = [
|
params = [
|
||||||
|
self._pbin,
|
||||||
|
# "-m",
|
||||||
"py_func",
|
"py_func",
|
||||||
"-p",
|
"-p",
|
||||||
f"{self._port}",
|
f"{self._port}",
|
||||||
@@ -81,31 +77,14 @@ class PyProcessBase:
|
|||||||
]
|
]
|
||||||
|
|
||||||
if tm.debug_enabled() and tm.gd("debug_rpc", False):
|
if tm.debug_enabled() and tm.gd("debug_rpc", False):
|
||||||
cmd_args.append("-v")
|
params.append("-v")
|
||||||
|
|
||||||
if bins._in_flatpak():
|
|
||||||
# Run on the host outside the sandbox: avoids glibc ABI mismatches
|
|
||||||
# between the Flatpak runtime and host shared libraries.
|
|
||||||
host_env = {
|
|
||||||
k: env[k] for k in ("PYTHONPATH", "PATH")
|
|
||||||
if k in env and env[k]
|
|
||||||
}
|
|
||||||
params = bins.flatpak_host_spawn(
|
|
||||||
self._pbin, cmd_args, host_cwd=func_proc_path,
|
|
||||||
extra_env=host_env,
|
|
||||||
)
|
|
||||||
popen_kwargs = {}
|
|
||||||
else:
|
|
||||||
params = [self._pbin, *cmd_args]
|
|
||||||
popen_kwargs = {"env": env, "cwd": func_proc_path}
|
|
||||||
|
|
||||||
self._process = subprocess.Popen(
|
self._process = subprocess.Popen(
|
||||||
params,
|
params, env=env, cwd=func_proc_path,
|
||||||
stdin=subprocess.DEVNULL,
|
stdin=subprocess.DEVNULL,
|
||||||
stdout=subprocess.PIPE,
|
stdout=subprocess.PIPE,
|
||||||
stderr=subprocess.PIPE,
|
stderr=subprocess.PIPE,
|
||||||
restore_signals=False,
|
restore_signals=False,
|
||||||
**popen_kwargs,
|
|
||||||
)
|
)
|
||||||
# Route subprocess stdout/stderr (early-startup errors,
|
# Route subprocess stdout/stderr (early-startup errors,
|
||||||
# unhandled exceptions, anything written to fd 1/2 before the
|
# unhandled exceptions, anything written to fd 1/2 before the
|
||||||
|
|||||||
@@ -165,14 +165,11 @@ def env_init():
|
|||||||
_constants_init()
|
_constants_init()
|
||||||
|
|
||||||
|
|
||||||
def apply_overrides(defines, gui_defaults):
|
def update_global(config_files, defines, gui_defaults, silent=False):
|
||||||
"""Push GUI defaults then CLI defines into the global dict.
|
"""Global dict updated with the content of the config file and a dict provided.
|
||||||
|
this function returns the resulting dict.
|
||||||
Extracted from update_global so it can be called *before* eval_proc
|
|
||||||
starts: interpreter overrides (python_bin, lua_bin) must be visible
|
|
||||||
to bins.python_bin() on its first lookup, which happens during
|
|
||||||
eval_process_init.
|
|
||||||
"""
|
"""
|
||||||
|
# GUI preferences applied first
|
||||||
for k, v in gui_defaults.items():
|
for k, v in gui_defaults.items():
|
||||||
try:
|
try:
|
||||||
val = ast.literal_eval(v)
|
val = ast.literal_eval(v)
|
||||||
@@ -180,6 +177,7 @@ def apply_overrides(defines, gui_defaults):
|
|||||||
val = v
|
val = v
|
||||||
tm.setgd(k, val)
|
tm.setgd(k, val)
|
||||||
|
|
||||||
|
# Then command line defines
|
||||||
for k, v in defines.items():
|
for k, v in defines.items():
|
||||||
try:
|
try:
|
||||||
val = ast.literal_eval(v)
|
val = ast.literal_eval(v)
|
||||||
@@ -187,14 +185,6 @@ def apply_overrides(defines, gui_defaults):
|
|||||||
val = v
|
val = v
|
||||||
tm.setgd(k, val)
|
tm.setgd(k, val)
|
||||||
|
|
||||||
|
|
||||||
def update_global(config_files, defines, gui_defaults, silent=False):
|
|
||||||
"""Global dict updated with the content of the config file and a dict provided.
|
|
||||||
this function returns the resulting dict.
|
|
||||||
"""
|
|
||||||
# GUI preferences applied first, then command line defines
|
|
||||||
apply_overrides(defines, gui_defaults)
|
|
||||||
|
|
||||||
# Then the configuration files
|
# Then the configuration files
|
||||||
# load global dic before test item
|
# load global dic before test item
|
||||||
_feed_gd_with_params(config_files, silent)
|
_feed_gd_with_params(config_files, silent)
|
||||||
|
|||||||
@@ -1,21 +0,0 @@
|
|||||||
"""Helpers for Qt file/directory dialogs.
|
|
||||||
|
|
||||||
In Flatpak the native QFileDialog goes through the XDG document portal,
|
|
||||||
which returns ``/run/user/UID/doc/.../<file>`` and only exposes the
|
|
||||||
selected file — sibling files (param.yaml, scripts, recent paths in
|
|
||||||
preferences, ...) are unreachable. Forcing Qt's own non-native dialog
|
|
||||||
makes it walk the real filesystem mounted via ``--filesystem=home``
|
|
||||||
and return a regular path.
|
|
||||||
"""
|
|
||||||
|
|
||||||
import os
|
|
||||||
|
|
||||||
from PySide6.QtWidgets import QFileDialog
|
|
||||||
|
|
||||||
|
|
||||||
def options():
|
|
||||||
"""Default ``QFileDialog`` options for the current runtime."""
|
|
||||||
opts = QFileDialog.Options()
|
|
||||||
if os.path.isfile("/.flatpak-info"):
|
|
||||||
opts |= QFileDialog.Option.DontUseNativeDialog
|
|
||||||
return opts
|
|
||||||
@@ -3,7 +3,6 @@ from PySide6.QtWidgets import QDialog, QFileDialog
|
|||||||
from PySide6.QtGui import QFont
|
from PySide6.QtGui import QFont
|
||||||
|
|
||||||
from main_win.preference_win.preference_core_win import Ui_preferenceWindow
|
from main_win.preference_win.preference_core_win import Ui_preferenceWindow
|
||||||
from main_win import file_dialog
|
|
||||||
|
|
||||||
import interpreter.utils.settings as prefs
|
import interpreter.utils.settings as prefs
|
||||||
|
|
||||||
@@ -194,7 +193,6 @@ class PrefWindow(QDialog):
|
|||||||
self,
|
self,
|
||||||
caption="Select the default report directory",
|
caption="Select the default report directory",
|
||||||
dir=self.ui.editDefaultReportPath.text(),
|
dir=self.ui.editDefaultReportPath.text(),
|
||||||
options=file_dialog.options(),
|
|
||||||
)
|
)
|
||||||
if path:
|
if path:
|
||||||
self.ui.editDefaultReportPath.setText(path)
|
self.ui.editDefaultReportPath.setText(path)
|
||||||
@@ -205,7 +203,6 @@ class PrefWindow(QDialog):
|
|||||||
self,
|
self,
|
||||||
caption="Select the default log directory",
|
caption="Select the default log directory",
|
||||||
dir=self.ui.editDefaultLogPath.text(),
|
dir=self.ui.editDefaultLogPath.text(),
|
||||||
options=file_dialog.options(),
|
|
||||||
)
|
)
|
||||||
if path:
|
if path:
|
||||||
self.ui.editDefaultLogPath.setText(path)
|
self.ui.editDefaultLogPath.setText(path)
|
||||||
@@ -216,7 +213,6 @@ class PrefWindow(QDialog):
|
|||||||
self,
|
self,
|
||||||
caption="Select the python interpreter",
|
caption="Select the python interpreter",
|
||||||
dir=self.ui.editPythonPath.text(),
|
dir=self.ui.editPythonPath.text(),
|
||||||
options=file_dialog.options(),
|
|
||||||
)
|
)
|
||||||
if path:
|
if path:
|
||||||
self.ui.editPythonPath.setText(path)
|
self.ui.editPythonPath.setText(path)
|
||||||
@@ -224,10 +220,7 @@ class PrefWindow(QDialog):
|
|||||||
@Slot()
|
@Slot()
|
||||||
def on_butLuaPath_pressed(self):
|
def on_butLuaPath_pressed(self):
|
||||||
path, _ = QFileDialog.getOpenFileName(
|
path, _ = QFileDialog.getOpenFileName(
|
||||||
self,
|
self, caption="Select the lua interpreter", dir=self.ui.editLuaPath.text()
|
||||||
caption="Select the lua interpreter",
|
|
||||||
dir=self.ui.editLuaPath.text(),
|
|
||||||
options=file_dialog.options(),
|
|
||||||
)
|
)
|
||||||
if path:
|
if path:
|
||||||
self.ui.editLuaPath.setText(path)
|
self.ui.editLuaPath.setText(path)
|
||||||
|
|||||||
@@ -9,7 +9,6 @@ from PySide6.QtWidgets import QApplication, QFileDialog, QProgressDialog
|
|||||||
from interpreter.process import TestProcess
|
from interpreter.process import TestProcess
|
||||||
from interpreter.utils.test_ctrl import TestSetController
|
from interpreter.utils.test_ctrl import TestSetController
|
||||||
from main_win.test_controller_service import TestControllerService
|
from main_win.test_controller_service import TestControllerService
|
||||||
from main_win import file_dialog
|
|
||||||
import interpreter.utils.settings as prefs
|
import interpreter.utils.settings as prefs
|
||||||
from runtime.tum_except import ETUMFileError, ETUMRuntimeError
|
from runtime.tum_except import ETUMFileError, ETUMRuntimeError
|
||||||
|
|
||||||
@@ -213,9 +212,17 @@ class TestFileManager:
|
|||||||
d = ""
|
d = ""
|
||||||
if w.testFile is not None:
|
if w.testFile is not None:
|
||||||
d = os.path.dirname(w.testFile)
|
d = os.path.dirname(w.testFile)
|
||||||
|
# In Flatpak the native dialog goes through the XDG document portal,
|
||||||
|
# which returns /run/user/UID/doc/.../test.tum and only exposes the
|
||||||
|
# selected file — sibling files (param.yaml, .py, etc.) are unreachable.
|
||||||
|
# Force Qt's own dialog, which walks the real filesystem mounted via
|
||||||
|
# --filesystem=home and returns a regular path with sibling access.
|
||||||
|
options = QFileDialog.Options()
|
||||||
|
if os.path.isfile("/.flatpak-info"):
|
||||||
|
options |= QFileDialog.Option.DontUseNativeDialog
|
||||||
file_name, _ = QFileDialog.getOpenFileName(
|
file_name, _ = QFileDialog.getOpenFileName(
|
||||||
w, "Open the test file", d,
|
w, "Open the test file", d,
|
||||||
"testium file (*.tum);;All Files (*)", options=file_dialog.options()
|
"testium file (*.tum);;All Files (*)", options=options
|
||||||
)
|
)
|
||||||
if file_name:
|
if file_name:
|
||||||
self.reload(file_name)
|
self.reload(file_name)
|
||||||
|
|||||||
@@ -37,7 +37,6 @@ from interpreter.utils.icons import icon_prefix
|
|||||||
|
|
||||||
from main_win.test_run.outlog import OutLog
|
from main_win.test_run.outlog import OutLog
|
||||||
from main_win.test_run.test_run import ThreadTestStatus
|
from main_win.test_run.test_run import ThreadTestStatus
|
||||||
from main_win import file_dialog
|
|
||||||
import interpreter.utils.settings as prefs
|
import interpreter.utils.settings as prefs
|
||||||
from runtime.stdout_redirect import stdio_redir
|
from runtime.stdout_redirect import stdio_redir
|
||||||
import api.testium as tm
|
import api.testium as tm
|
||||||
@@ -485,8 +484,7 @@ class MainWindow(QMainWindow, Ui_MainWindow):
|
|||||||
else:
|
else:
|
||||||
initialPath = None
|
initialPath = None
|
||||||
fileName, _ = QFileDialog.getSaveFileName(
|
fileName, _ = QFileDialog.getSaveFileName(
|
||||||
self, "Path to Log file", initialPath, "Log Files (*.log);;All Files (*)",
|
self, "Path to Log file", initialPath, "Log Files (*.log);;All Files (*)"
|
||||||
options=file_dialog.options(),
|
|
||||||
)
|
)
|
||||||
if fileName:
|
if fileName:
|
||||||
shutil.copy(self.logFileName, fileName)
|
shutil.copy(self.logFileName, fileName)
|
||||||
@@ -527,8 +525,7 @@ class MainWindow(QMainWindow, Ui_MainWindow):
|
|||||||
else:
|
else:
|
||||||
initialPath = None
|
initialPath = None
|
||||||
fileName, _ = QFileDialog.getSaveFileName(
|
fileName, _ = QFileDialog.getSaveFileName(
|
||||||
self, "Path to log file", initialPath, "Log Files (*.log);;All Files (*)",
|
self, "Path to log file", initialPath, "Log Files (*.log);;All Files (*)"
|
||||||
options=file_dialog.options(),
|
|
||||||
)
|
)
|
||||||
if fileName:
|
if fileName:
|
||||||
self.editLogFilePath.setText(fileName)
|
self.editLogFilePath.setText(fileName)
|
||||||
|
|||||||
@@ -1,76 +1,10 @@
|
|||||||
# Validation
|
# Validation
|
||||||
|
|
||||||
This directory contains the testium validation suite. A single set of
|
This directory contains the necessary material to run the testium validation.
|
||||||
items (`items/`), fixtures and post-processing (`post_execution.py`) is
|
|
||||||
re-used across every packaging channel.
|
|
||||||
|
|
||||||
## Running the suite
|
Here is the documentation on how to configure the validation, run it and check that the
|
||||||
|
results are correct.
|
||||||
|
|
||||||
```sh
|
# Tests
|
||||||
./test/validation/run.sh # default mode = source
|
|
||||||
./test/validation/run.sh --mode wheel
|
|
||||||
./test/validation/run.sh --mode pyinstaller
|
|
||||||
./test/validation/run.sh --mode flatpak
|
|
||||||
./test/validation/run.sh --mode appimage
|
|
||||||
```
|
|
||||||
|
|
||||||
On Windows (only `source`, `wheel`, `pyinstaller` are supported):
|
TBD
|
||||||
|
|
||||||
```bat
|
|
||||||
test\validation\run.bat --mode pyinstaller
|
|
||||||
```
|
|
||||||
|
|
||||||
Pass `clean` as the **first** argument to recreate the validation venv
|
|
||||||
from scratch (useful after a system Python upgrade):
|
|
||||||
|
|
||||||
```sh
|
|
||||||
./test/validation/run.sh clean --mode flatpak
|
|
||||||
```
|
|
||||||
|
|
||||||
Any extra arguments after the mode flag are forwarded to testium.
|
|
||||||
|
|
||||||
## Modes
|
|
||||||
|
|
||||||
| Mode | What it launches | Prerequisite |
|
|
||||||
|---------------|-------------------------------------------------------------|------------------------------------------------------------------|
|
|
||||||
| `source` | `python3 src/testium` via the project's `run.sh` | none — works straight out of the repo |
|
|
||||||
| `wheel` | `python -m testium` inside a dedicated wheel venv | `./build_all.sh` produced `dist/testium-<v>-py3-none-any.whl` |
|
|
||||||
| `pyinstaller` | `dist/testium-<v>` (frozen binary) | `./build_all.sh` produced the PyInstaller binary |
|
|
||||||
| `flatpak` | `flatpak run --command=testium org.testium.Testium` | the Flatpak bundle is installed (`flatpak install --user dist/testium-<v>.flatpak`) |
|
|
||||||
| `appimage` | `dist/Testium-<v>-x86_64.AppImage` | `./build_all.sh` produced the AppImage |
|
|
||||||
|
|
||||||
Each mode writes its results to a distinct report file
|
|
||||||
(`validation-<mode>.sqlite` / `validation-<mode>-<item>.xml`), so you
|
|
||||||
can run several modes in a row without clobbering previous reports.
|
|
||||||
|
|
||||||
## How `python_bin` is pinned
|
|
||||||
|
|
||||||
Every test-execution subprocess (inline `<| ... |>` evaluation,
|
|
||||||
`py_func`, `cycle`, `post_execution`, …) is routed through a dedicated
|
|
||||||
venv at `${TMPDIR:-/tmp}/testium-validation-venv`. The venv is created
|
|
||||||
with `--system-site-packages` so existing system packages stay visible,
|
|
||||||
then `junit-xml` is pip-installed for `post_execution.py`.
|
|
||||||
|
|
||||||
This is a **host** venv. In every mode (including Flatpak) the
|
|
||||||
test-execution subprocesses end up running on the host — directly for
|
|
||||||
source/wheel/pyinstaller/appimage, and via `flatpak-spawn --host` for
|
|
||||||
Flatpak — so the same venv works across modes. The wheel mode
|
|
||||||
additionally creates a separate `testium-wheel-venv-<v>` to hold the
|
|
||||||
installed wheel; that one is only used to launch testium itself.
|
|
||||||
|
|
||||||
## What is checked
|
|
||||||
|
|
||||||
The `venv` item under `items/venv/` asserts that the validation venv is
|
|
||||||
actually being used:
|
|
||||||
|
|
||||||
* `python_bin` is set in the global dict.
|
|
||||||
* The eval subprocess (used for `<| ... |>` expressions) has
|
|
||||||
`sys.executable == python_bin`, `sys.prefix == dirname(dirname(python_bin))`,
|
|
||||||
and `sys.prefix != sys.base_prefix` (i.e. is actually inside a venv).
|
|
||||||
* A `py_func` subprocess passes the same three checks.
|
|
||||||
|
|
||||||
These checks use `abspath`/`normpath` rather than `realpath` on
|
|
||||||
purpose: the venv's `bin/python3` is a symlink to the host interpreter,
|
|
||||||
so `realpath` would map both venv and non-venv interpreters to the same
|
|
||||||
target. `sys.prefix != sys.base_prefix` is the venv-specific marker
|
|
||||||
that distinguishes the two cases.
|
|
||||||
|
|||||||
@@ -1 +0,0 @@
|
|||||||
no_param: Null
|
|
||||||
@@ -1,53 +0,0 @@
|
|||||||
# venv test: assert that the dedicated validation venv is the python
|
|
||||||
# being used for every test-execution subprocess (eval_proc / py_func /
|
|
||||||
# cycle / ...). The venv path is pinned by ``-d python_bin=...`` in
|
|
||||||
# test/validation/run.sh (or run.bat).
|
|
||||||
#
|
|
||||||
# We use ``abspath``/``normpath`` rather than ``realpath`` on purpose:
|
|
||||||
# the venv's ``bin/python3`` is a symlink to the host python, so
|
|
||||||
# realpath would map every venv interpreter to the same system path and
|
|
||||||
# the comparison would silently pass even *without* a venv.
|
|
||||||
# ``sys.prefix != sys.base_prefix`` is the venv-specific marker that
|
|
||||||
# catches that case.
|
|
||||||
|
|
||||||
- check:
|
|
||||||
name: python_bin is set in the global dict
|
|
||||||
key: $(test)_PASS
|
|
||||||
values:
|
|
||||||
- <| bool(r"$(python_bin)") |>
|
|
||||||
|
|
||||||
- check:
|
|
||||||
name: eval_proc subprocess runs in the validation venv
|
|
||||||
key: $(test)_PASS
|
|
||||||
values:
|
|
||||||
- <| os.path.normpath(os.path.abspath(sys.executable)) == os.path.normpath(os.path.abspath(r"$(python_bin)")) |>
|
|
||||||
|
|
||||||
- check:
|
|
||||||
name: eval_proc sys.prefix matches python_bin venv root
|
|
||||||
key: $(test)_PASS
|
|
||||||
values:
|
|
||||||
- <| os.path.normpath(os.path.abspath(sys.prefix)) == os.path.dirname(os.path.dirname(os.path.normpath(os.path.abspath(r"$(python_bin)")))) |>
|
|
||||||
|
|
||||||
- check:
|
|
||||||
name: eval_proc is actually inside a venv (sys.prefix != sys.base_prefix)
|
|
||||||
key: $(test)_PASS
|
|
||||||
values:
|
|
||||||
- <| os.path.normpath(os.path.abspath(sys.prefix)) != os.path.normpath(os.path.abspath(sys.base_prefix)) |>
|
|
||||||
|
|
||||||
- py_func:
|
|
||||||
name: py_func subprocess runs in the validation venv
|
|
||||||
key: $(test)_PASS
|
|
||||||
file: $(test_path)$(psep)verify_venv.py
|
|
||||||
func_name: check_sys_executable
|
|
||||||
|
|
||||||
- py_func:
|
|
||||||
name: py_func sys.prefix matches python_bin venv root
|
|
||||||
key: $(test)_PASS
|
|
||||||
file: $(test_path)$(psep)verify_venv.py
|
|
||||||
func_name: check_sys_prefix_in_venv
|
|
||||||
|
|
||||||
- py_func:
|
|
||||||
name: py_func is actually inside a venv
|
|
||||||
key: $(test)_PASS
|
|
||||||
file: $(test_path)$(psep)verify_venv.py
|
|
||||||
func_name: check_is_venv
|
|
||||||
@@ -1,62 +0,0 @@
|
|||||||
import os
|
|
||||||
import sys
|
|
||||||
|
|
||||||
import py_func.tm as tm
|
|
||||||
|
|
||||||
|
|
||||||
def _norm(p):
|
|
||||||
# normpath + normcase, without resolving symlinks. realpath() would
|
|
||||||
# follow the venv's ``python3`` symlink to ``/usr/bin/python3.X`` and
|
|
||||||
# defeat the comparison.
|
|
||||||
return os.path.normcase(os.path.normpath(os.path.abspath(p)))
|
|
||||||
|
|
||||||
|
|
||||||
def _venv_dir():
|
|
||||||
# python_bin is at ``<venv>/(bin|Scripts)/python*`` so the venv root
|
|
||||||
# is two levels above the executable.
|
|
||||||
exe = tm.gd("python_bin", "")
|
|
||||||
if not exe:
|
|
||||||
return ""
|
|
||||||
return os.path.dirname(os.path.dirname(_norm(exe)))
|
|
||||||
|
|
||||||
|
|
||||||
def check_sys_executable():
|
|
||||||
"""py_func subprocess: sys.executable must match the configured python_bin."""
|
|
||||||
expected = _norm(tm.gd("python_bin", ""))
|
|
||||||
actual = _norm(sys.executable)
|
|
||||||
if expected and actual == expected:
|
|
||||||
return True
|
|
||||||
return (
|
|
||||||
-1,
|
|
||||||
f"sys.executable={actual!r} differs from python_bin={expected!r}",
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def check_sys_prefix_in_venv():
|
|
||||||
"""py_func subprocess: sys.prefix must match the venv root derived
|
|
||||||
from python_bin (two levels up from the executable)."""
|
|
||||||
venv = _venv_dir()
|
|
||||||
if not venv:
|
|
||||||
return (-1, "python_bin is not set in the global dict")
|
|
||||||
actual = _norm(sys.prefix)
|
|
||||||
if actual == venv:
|
|
||||||
return True
|
|
||||||
return (
|
|
||||||
-1,
|
|
||||||
f"sys.prefix={actual!r} is not the validation venv {venv!r}",
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def check_is_venv():
|
|
||||||
"""py_func subprocess: confirm we are inside a venv, i.e. sys.prefix
|
|
||||||
differs from sys.base_prefix. This catches the case where python_bin
|
|
||||||
happens to be a system interpreter and the path-based check would
|
|
||||||
pass trivially."""
|
|
||||||
actual = _norm(sys.prefix)
|
|
||||||
base = _norm(sys.base_prefix)
|
|
||||||
if actual != base:
|
|
||||||
return True
|
|
||||||
return (
|
|
||||||
-1,
|
|
||||||
f"sys.prefix == sys.base_prefix == {actual!r}: not running in a venv",
|
|
||||||
)
|
|
||||||
@@ -1,131 +0,0 @@
|
|||||||
@echo off
|
|
||||||
SETLOCAL EnableExtensions EnableDelayedExpansion
|
|
||||||
|
|
||||||
REM Runs the testium validation suite against any installable channel of
|
|
||||||
REM testium on Windows (source, wheel, pyinstaller).
|
|
||||||
REM
|
|
||||||
REM Usage:
|
|
||||||
REM test\validation\run.bat [clean] [--mode MODE] [extra testium args]
|
|
||||||
REM
|
|
||||||
REM clean remove the validation venv before recreating it
|
|
||||||
REM (must be the first argument; useful after a Python upgrade)
|
|
||||||
REM
|
|
||||||
REM --mode MODE which testium build to validate. One of:
|
|
||||||
REM source (default) project's run.bat (src\testium)
|
|
||||||
REM wheel dist\testium-<v>-py3-none-any.whl
|
|
||||||
REM pyinstaller dist\testium-<v>.exe (or dist\testium-<v>)
|
|
||||||
REM
|
|
||||||
REM Every test-execution subprocess runs in a dedicated host venv under
|
|
||||||
REM %TEMP%\testium-validation-venv (created with --system-site-packages,
|
|
||||||
REM then junit-xml is pip-installed for post_execution.py).
|
|
||||||
REM
|
|
||||||
REM The report file is suffixed with the mode so consecutive runs in
|
|
||||||
REM different modes don't overwrite each other.
|
|
||||||
|
|
||||||
SET "SCRIPT_DIR=%~dp0"
|
|
||||||
IF "%SCRIPT_DIR:~-1%"=="\" SET "SCRIPT_DIR=%SCRIPT_DIR:~0,-1%"
|
|
||||||
SET "PROJECT_DIR=%SCRIPT_DIR%\..\.."
|
|
||||||
SET /P VERSION=<"%PROJECT_DIR%\src\VERSION"
|
|
||||||
|
|
||||||
REM ---------- arg parsing ----------------------------------------------------
|
|
||||||
|
|
||||||
SET "MODE=source"
|
|
||||||
SET "CLEAN=0"
|
|
||||||
IF /I "%~1"=="clean" (
|
|
||||||
SET "CLEAN=1"
|
|
||||||
SHIFT
|
|
||||||
)
|
|
||||||
|
|
||||||
SET "EXTRA="
|
|
||||||
:PARSE_ARGS
|
|
||||||
IF "%~1"=="" GOTO ARGS_DONE
|
|
||||||
IF /I "%~1"=="--mode" (
|
|
||||||
SET "MODE=%~2"
|
|
||||||
SHIFT
|
|
||||||
SHIFT
|
|
||||||
GOTO PARSE_ARGS
|
|
||||||
)
|
|
||||||
SET "EXTRA=!EXTRA! "%~1""
|
|
||||||
SHIFT
|
|
||||||
GOTO PARSE_ARGS
|
|
||||||
:ARGS_DONE
|
|
||||||
|
|
||||||
REM ---------- locate host python ---------------------------------------------
|
|
||||||
|
|
||||||
SET "PYTHON_EXE="
|
|
||||||
py --version >nul 2>&1
|
|
||||||
IF %ERRORLEVEL% EQU 0 (
|
|
||||||
SET "PYTHON_EXE=py"
|
|
||||||
GOTO PYTHON_FOUND
|
|
||||||
)
|
|
||||||
python --version >nul 2>&1
|
|
||||||
IF %ERRORLEVEL% EQU 0 (
|
|
||||||
SET "PYTHON_EXE=python"
|
|
||||||
GOTO PYTHON_FOUND
|
|
||||||
)
|
|
||||||
echo ERROR: Python could not be found on this system.
|
|
||||||
exit /b 1
|
|
||||||
:PYTHON_FOUND
|
|
||||||
|
|
||||||
REM ---------- validation venv -------------------------------------------------
|
|
||||||
|
|
||||||
SET "VENV_DIR=%TEMP%\testium-validation-venv"
|
|
||||||
IF "%CLEAN%"=="1" IF EXIST "%VENV_DIR%" rmdir /s /q "%VENV_DIR%"
|
|
||||||
|
|
||||||
IF NOT EXIST "%VENV_DIR%" (
|
|
||||||
echo Creating validation venv at %VENV_DIR%
|
|
||||||
%PYTHON_EXE% -m venv --system-site-packages "%VENV_DIR%"
|
|
||||||
IF !ERRORLEVEL! NEQ 0 (
|
|
||||||
echo ERROR while creating the validation venv.
|
|
||||||
exit /b 1
|
|
||||||
)
|
|
||||||
call "%VENV_DIR%\Scripts\pip" install --quiet --upgrade pip
|
|
||||||
call "%VENV_DIR%\Scripts\pip" install --quiet junit-xml
|
|
||||||
)
|
|
||||||
SET "VENV_PYTHON=%VENV_DIR%\Scripts\python.exe"
|
|
||||||
|
|
||||||
REM ---------- shared "tail" forwarded to every launcher -----------------------
|
|
||||||
REM Reports are stamped with the mode so successive runs don't clobber each other.
|
|
||||||
|
|
||||||
SET "TAIL=-b -d "python_bin=%VENV_PYTHON%" -d "validation_report_file=validation-%MODE%" -- "%SCRIPT_DIR%\main.tum"%EXTRA%"
|
|
||||||
|
|
||||||
REM ---------- per-mode launcher ----------------------------------------------
|
|
||||||
|
|
||||||
echo -- validation mode: %MODE%
|
|
||||||
|
|
||||||
IF /I "%MODE%"=="source" GOTO MODE_SOURCE
|
|
||||||
IF /I "%MODE%"=="wheel" GOTO MODE_WHEEL
|
|
||||||
IF /I "%MODE%"=="pyinstaller" GOTO MODE_PYI
|
|
||||||
echo ERROR: unknown --mode '%MODE%'. Expected: source ^| wheel ^| pyinstaller.
|
|
||||||
exit /b 1
|
|
||||||
|
|
||||||
:MODE_SOURCE
|
|
||||||
call "%PROJECT_DIR%\run.bat" %TAIL%
|
|
||||||
exit /b %ERRORLEVEL%
|
|
||||||
|
|
||||||
:MODE_WHEEL
|
|
||||||
SET "WHEEL=%PROJECT_DIR%\dist\testium-%VERSION%-py3-none-any.whl"
|
|
||||||
IF NOT EXIST "%WHEEL%" (
|
|
||||||
echo ERROR: wheel not found at %WHEEL% -- run build_all.sh first.
|
|
||||||
exit /b 1
|
|
||||||
)
|
|
||||||
SET "WHEEL_VENV=%TEMP%\testium-wheel-venv-%VERSION%"
|
|
||||||
IF "%CLEAN%"=="1" IF EXIST "%WHEEL_VENV%" rmdir /s /q "%WHEEL_VENV%"
|
|
||||||
IF NOT EXIST "%WHEEL_VENV%" (
|
|
||||||
echo Creating wheel venv at %WHEEL_VENV%
|
|
||||||
%PYTHON_EXE% -m venv --system-site-packages "%WHEEL_VENV%"
|
|
||||||
call "%WHEEL_VENV%\Scripts\pip" install --quiet --upgrade pip
|
|
||||||
call "%WHEEL_VENV%\Scripts\pip" install --quiet "%WHEEL%"
|
|
||||||
)
|
|
||||||
"%WHEEL_VENV%\Scripts\python.exe" -m testium %TAIL%
|
|
||||||
exit /b %ERRORLEVEL%
|
|
||||||
|
|
||||||
:MODE_PYI
|
|
||||||
SET "PYI_BIN=%PROJECT_DIR%\dist\testium-%VERSION%.exe"
|
|
||||||
IF NOT EXIST "%PYI_BIN%" SET "PYI_BIN=%PROJECT_DIR%\dist\testium-%VERSION%"
|
|
||||||
IF NOT EXIST "%PYI_BIN%" (
|
|
||||||
echo ERROR: PyInstaller binary not found in %PROJECT_DIR%\dist -- run build_all.sh first.
|
|
||||||
exit /b 1
|
|
||||||
)
|
|
||||||
"%PYI_BIN%" %TAIL%
|
|
||||||
exit /b %ERRORLEVEL%
|
|
||||||
@@ -1,143 +0,0 @@
|
|||||||
#!/bin/bash
|
|
||||||
# Runs the testium validation suite against any installable channel of
|
|
||||||
# testium (source, wheel, pyinstaller, flatpak, appimage).
|
|
||||||
#
|
|
||||||
# Usage:
|
|
||||||
# ./test/validation/run.sh [clean] [--mode MODE] [extra testium args]
|
|
||||||
#
|
|
||||||
# clean remove the validation venv before recreating it
|
|
||||||
# (must be the first argument; useful after a Python upgrade)
|
|
||||||
#
|
|
||||||
# --mode MODE which testium build to validate. One of:
|
|
||||||
# source (default) src/testium via project run.sh
|
|
||||||
# wheel dist/testium-<v>-py3-none-any.whl
|
|
||||||
# pyinstaller dist/testium-<v>
|
|
||||||
# flatpak installed org.testium.Testium
|
|
||||||
# appimage dist/Testium-<v>-*.AppImage
|
|
||||||
#
|
|
||||||
# Every test-execution subprocess (inline <| ... |>, py_func, cycle,
|
|
||||||
# post_execution, ...) runs in a dedicated host venv under
|
|
||||||
# /tmp/testium-validation-venv. That venv is shared across modes —
|
|
||||||
# even Flatpak reaches it via flatpak-spawn --host. The validation venv
|
|
||||||
# is created with --system-site-packages so existing system packages
|
|
||||||
# (PySide6, lxml, ...) stay visible, then junit-xml is pip-installed
|
|
||||||
# for post_execution.py.
|
|
||||||
#
|
|
||||||
# The report file is suffixed with the mode (e.g. validation-flatpak.sqlite)
|
|
||||||
# so consecutive runs in different modes don't overwrite each other.
|
|
||||||
|
|
||||||
set -e
|
|
||||||
|
|
||||||
SCRIPT_PATH="$(readlink -f "$0")"
|
|
||||||
SCRIPT_DIR="$(realpath "$(dirname "$SCRIPT_PATH")")"
|
|
||||||
PROJECT_DIR="$(realpath "$SCRIPT_DIR/../..")"
|
|
||||||
VERSION="$(cat "$PROJECT_DIR/src/VERSION")"
|
|
||||||
|
|
||||||
# ---------- arg parsing -------------------------------------------------------
|
|
||||||
|
|
||||||
MODE=source
|
|
||||||
|
|
||||||
if [ "${1:-}" = "clean" ]; then
|
|
||||||
CLEAN=1
|
|
||||||
shift
|
|
||||||
else
|
|
||||||
CLEAN=0
|
|
||||||
fi
|
|
||||||
|
|
||||||
EXTRA=()
|
|
||||||
while [ $# -gt 0 ]; do
|
|
||||||
case "$1" in
|
|
||||||
--mode)
|
|
||||||
MODE="$2"
|
|
||||||
shift 2
|
|
||||||
;;
|
|
||||||
--mode=*)
|
|
||||||
MODE="${1#--mode=}"
|
|
||||||
shift
|
|
||||||
;;
|
|
||||||
*)
|
|
||||||
EXTRA+=("$1")
|
|
||||||
shift
|
|
||||||
;;
|
|
||||||
esac
|
|
||||||
done
|
|
||||||
|
|
||||||
# ---------- validation venv ---------------------------------------------------
|
|
||||||
|
|
||||||
VENV_DIR="${TMPDIR:-/tmp}/testium-validation-venv"
|
|
||||||
if [ "$CLEAN" -eq 1 ]; then
|
|
||||||
rm -rf "$VENV_DIR"
|
|
||||||
fi
|
|
||||||
|
|
||||||
if [ ! -d "$VENV_DIR" ]; then
|
|
||||||
echo "Creating validation venv at $VENV_DIR"
|
|
||||||
python3 -m venv --system-site-packages "$VENV_DIR"
|
|
||||||
"$VENV_DIR/bin/pip" install --quiet --upgrade pip
|
|
||||||
"$VENV_DIR/bin/pip" install --quiet junit-xml
|
|
||||||
fi
|
|
||||||
VENV_PYTHON="$VENV_DIR/bin/python3"
|
|
||||||
|
|
||||||
# ---------- per-mode launcher -------------------------------------------------
|
|
||||||
|
|
||||||
case "$MODE" in
|
|
||||||
source)
|
|
||||||
CMD=("$PROJECT_DIR/run.sh")
|
|
||||||
;;
|
|
||||||
wheel)
|
|
||||||
WHEEL="$PROJECT_DIR/dist/testium-${VERSION}-py3-none-any.whl"
|
|
||||||
if [ ! -f "$WHEEL" ]; then
|
|
||||||
echo "ERROR: wheel not found at $WHEEL — run ./build_all.sh first." >&2
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
WHEEL_VENV="${TMPDIR:-/tmp}/testium-wheel-venv-${VERSION}"
|
|
||||||
if [ "$CLEAN" -eq 1 ]; then
|
|
||||||
rm -rf "$WHEEL_VENV"
|
|
||||||
fi
|
|
||||||
if [ ! -d "$WHEEL_VENV" ]; then
|
|
||||||
echo "Creating wheel venv at $WHEEL_VENV"
|
|
||||||
python3 -m venv --system-site-packages "$WHEEL_VENV"
|
|
||||||
"$WHEEL_VENV/bin/pip" install --quiet --upgrade pip
|
|
||||||
"$WHEEL_VENV/bin/pip" install --quiet "$WHEEL"
|
|
||||||
fi
|
|
||||||
CMD=("$WHEEL_VENV/bin/python" -m testium)
|
|
||||||
;;
|
|
||||||
pyinstaller)
|
|
||||||
PYI_BIN="$PROJECT_DIR/dist/testium-${VERSION}"
|
|
||||||
if [ ! -x "$PYI_BIN" ]; then
|
|
||||||
echo "ERROR: PyInstaller binary not found at $PYI_BIN — run ./build_all.sh first." >&2
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
CMD=("$PYI_BIN")
|
|
||||||
;;
|
|
||||||
flatpak)
|
|
||||||
if ! flatpak info --user org.testium.Testium &>/dev/null \
|
|
||||||
&& ! flatpak info --system org.testium.Testium &>/dev/null; then
|
|
||||||
echo "ERROR: org.testium.Testium is not installed." >&2
|
|
||||||
echo " flatpak install --user $PROJECT_DIR/dist/testium-${VERSION}.flatpak" >&2
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
CMD=(flatpak run --command=testium org.testium.Testium)
|
|
||||||
;;
|
|
||||||
appimage)
|
|
||||||
APPIMAGE=$(ls -1t "$PROJECT_DIR/dist"/Testium-"${VERSION}"-*.AppImage 2>/dev/null | head -1)
|
|
||||||
if [ -z "$APPIMAGE" ] || [ ! -x "$APPIMAGE" ]; then
|
|
||||||
echo "ERROR: no AppImage for version $VERSION under $PROJECT_DIR/dist — run ./build_all.sh first." >&2
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
CMD=("$APPIMAGE")
|
|
||||||
;;
|
|
||||||
*)
|
|
||||||
echo "ERROR: unknown --mode '$MODE'. Expected: source|wheel|pyinstaller|flatpak|appimage." >&2
|
|
||||||
exit 1
|
|
||||||
;;
|
|
||||||
esac
|
|
||||||
|
|
||||||
# ---------- launch ------------------------------------------------------------
|
|
||||||
|
|
||||||
echo "-- validation mode: $MODE"
|
|
||||||
echo "-- launch: ${CMD[*]}"
|
|
||||||
|
|
||||||
exec "${CMD[@]}" -b \
|
|
||||||
-d "python_bin=$VENV_PYTHON" \
|
|
||||||
-d "validation_report_file=validation-$MODE" \
|
|
||||||
-- "$SCRIPT_DIR/main.tum" "${EXTRA[@]}"
|
|
||||||
Reference in New Issue
Block a user