Compare commits
4 Commits
fix/window
...
small_impr
| Author | SHA1 | Date | |
|---|---|---|---|
| 811e3d356e | |||
| 3fb982b057 | |||
| 53553dc1fa | |||
| 717727bf5a |
12
DESIGN.md
12
DESIGN.md
@@ -97,15 +97,6 @@ All dialog items (`dialog_image`, `dialog_question`, `dialog_references`, `dialo
|
||||
- For the live stream (terminal in batch / GUI panel), prefixes every line emitted from a branch's thread with `[<branch_name>] ` so concurrent branches stay readable.
|
||||
- Exposes `write` / `writeln` / `flush` (Python 3.14's `unittest` calls `stream.writeln()` directly without `_WritelnDecorator`).
|
||||
|
||||
### Subprocess RPC startup handshake (py_func / lua_func / eval_proc)
|
||||
|
||||
The parent ↔ subprocess JSON-RPC link runs over a localhost TCP socket. The **subprocess** owns the port: it binds `port 0` (OS-assigned), `listen()`s, then prints `__TESTIUM_RPC_PORT__=<port>` on stdout (constant `RPC_PORT_SENTINEL` in `runtime/jrpc.py`). The parent reads that line (`proc_drain.drain_and_read_port` + `wait_for_port`, deadline `gd("proc_start_timeout", 30)`) and only *then* connects — the server is guaranteed to be listening, so the connect succeeds on the first attempt.
|
||||
|
||||
This replaced the previous fragile scheme (parent reserved a port via `bind(0)`+close, child re-bound the same port, parent connected on a timing guess) which broke intermittently on Windows: cold-start/antivirus variance pushed the worker past the connect deadline, and `connect()` to a not-yet-listening localhost port *times out* (≈1 s) instead of refusing, exhausting the retry budget. Notes:
|
||||
- The server no longer sets `SO_REUSEADDR` (a fresh ephemeral port needs no TIME_WAIT override; on Windows it would enable port hijacking).
|
||||
- `JsonRpcBase.wait_ready()` always settles (event set on success **and** failure) and returns the actual connection outcome — a connect failure no longer hangs a `wait_ready()` caller.
|
||||
- Non-sentinel subprocess stdout/stderr is still forwarded to the parent log (early-startup errors stay visible).
|
||||
|
||||
### Subprocess API contract (py_func / lua_func)
|
||||
|
||||
User test scripts running inside a `py_func` or `lua_func` subprocess **must** use the JSON-RPC bridge to interact with testium state:
|
||||
@@ -124,7 +115,7 @@ To add a new API call usable from subprocesses:
|
||||
`src/testium/interpreter/utils/bins.py` — single source of truth for the paths to the external Python and Lua interpreters used by subprocesses.
|
||||
|
||||
- `python_bin()` / `lua_bin()` : resolve and cache. The cache is keyed by `(name, override)` so that a later change to `gd[python_bin]` (typically when a `param.yaml` sets the key) triggers a re-resolution on the next lookup instead of returning the stale auto-discovered path. Falls back to discovery on PATH (candidates: `python3`/`python` and `lua`/`lua5.5`/`lua5.4`/`lua5.3`/`lua5.2`/`lua5.1`).
|
||||
- `ensure(*names)` : called by `TestSet._validate_runtime_deps()` at test load. Always requires `python` (the eval engine always runs); requires `lua` only if a `lua_func` item is in the tree. Fails fast with a clear error citing tried candidates and override key. Also **publishes** each resolved path into gd (`python_bin` / `lua_bin`) when the key is unset, so test scripts can reference `$(python_bin)` / `$(lua_bin)` regardless of launch mode (e.g. GUI, where no `-d` override is passed). A user-provided value is never overwritten.
|
||||
- `ensure(*names)` : called by `TestSet._validate_runtime_deps()` at test load. Always requires `python` (the eval engine always runs); requires `lua` only if a `lua_func` item is in the tree. Fails fast with a clear error citing tried candidates and override key.
|
||||
|
||||
Engines (`PyProcessBase`, `LuaProcessBase`, `EvalExecEngine`) call `bins.python_bin()`/`bins.lua_bin()` themselves — call sites never pass an explicit binary path.
|
||||
|
||||
@@ -288,7 +279,6 @@ The `testium_assist` editor extension is a thin LSP client that spawns `testium
|
||||
Both Flatpak and AppImage export `TESTIUM_VERSION` from a launcher (Flatpak: launcher script in `org.testium.Testium.yaml`; AppImage: `runtime.env` in `AppImageBuilder.yml`). `get_testium_version()` checks `/.flatpak-info` / `APPIMAGE` and reads `TESTIUM_VERSION` rather than relying on package metadata or repo introspection.
|
||||
|
||||
## Recent fixes / notable changes
|
||||
- Subprocess RPC startup handshake: the `py_func`/`lua_func`/`eval_proc` worker now picks its own port (`bind 0`), announces it on stdout (`__TESTIUM_RPC_PORT__=`), and the parent connects only after reading it. Fixes intermittent Windows `failed to connect : timeout` and the matching `wait_ready()` hang; removes the reserve/close/rebind race and `SO_REUSEADDR`. See "Subprocess RPC startup handshake".
|
||||
- `build_all.sh`: builds the four heavy channels in parallel (serial prep for the shared venv + wheel), results in completion order, Ctrl+C kills the whole job tree; `--ram` puts the build scratch on tmpfs (`/dev/shm`) + skips UPX for fast builds on USB/SD storage (Flatpak excluded — rofiles-fuse can't mount tmpfs). See the "Building all channels" section.
|
||||
- LSP across packaging channels: `testium lsp` (and the `testium_assist` editor extension that spawns it) now works from source, wheel, PyInstaller, Flatpak and AppImage. Two enablers — (1) action items declare a class-level `ACTIONS = {key: class}` registry (like `PARAMS`), so `lsp/schema.py` builds the full schema from class attributes with no `inspect.getsource`/AST (which broke under frozen PyInstaller); (2) the `[lsp]` extra (pygls) is wired into every full-app channel. `test/validation/lsp_check.py`, run by `run.sh` before the suite, asserts per-channel that `schema` keeps its actions and `lsp` answers `initialize`. See the matching architecture sections.
|
||||
- Declarative test item parameters (v0.2): each `TestItem` subclass exposes a `PARAMS = ParamSet(...)` class attribute consumed by the base `__init__`. Catches unknown YAML keys (typo warnings listing the accepted names) and missing required params (load-time errors with `.tum` context). Lays the schema foundation for a future LSP server and auto-generated manual sections. See the matching architecture section.
|
||||
|
||||
@@ -9,9 +9,9 @@ This element is of the following form:
|
||||
- let:
|
||||
name: Let Item
|
||||
values:
|
||||
key1: value1
|
||||
key2: value2
|
||||
key3: <| $(variable)[$(loop_index)] |>
|
||||
- key1: value1
|
||||
- key2: value2
|
||||
- key3: <| $(variable)[$(loop_index)] |>
|
||||
|
||||
The ``let`` element is used to set values in the global directory.
|
||||
|
||||
|
||||
@@ -51,8 +51,8 @@ The parameter file can be specified in the `.tum` file root:
|
||||
:caption: configuration files definition in the main `.tum` test file
|
||||
|
||||
config_file:
|
||||
config1.yaml
|
||||
config2.yaml
|
||||
- config1.yaml
|
||||
- config2.yaml
|
||||
|
||||
main:
|
||||
name: Test example
|
||||
|
||||
@@ -1,8 +1,3 @@
|
||||
version 0.2.3
|
||||
=============
|
||||
- Windows version now working reliably. Fix of a problem of jrpc ports
|
||||
handshakes between the py and lua processes and testium
|
||||
|
||||
version 0.2.2
|
||||
==============
|
||||
- Flatpak sandbox issue fixed for term console. Now a term console is
|
||||
|
||||
@@ -1 +1 @@
|
||||
0.2.3
|
||||
0.2.2
|
||||
|
||||
@@ -11,16 +11,6 @@ sys.path.append(os.path.abspath(ourpath.parent))
|
||||
import interpreter.utils.constants as cst
|
||||
|
||||
def main():
|
||||
# Force UTF-8 on stdout/stderr so the runner's output survives a legacy
|
||||
# console code page (Windows cp1252 can't encode box-drawing/accented
|
||||
# chars). Only the stream encoders change; the locale default used for
|
||||
# config files is untouched.
|
||||
for _stream in (sys.stdout, sys.stderr):
|
||||
try:
|
||||
_stream.reconfigure(encoding="utf-8")
|
||||
except (AttributeError, ValueError):
|
||||
pass # no stdout (frozen GUI) or non-reconfigurable stream
|
||||
|
||||
# Subcommand dispatch (must run *before* argparse so neither 'schema' nor
|
||||
# 'lsp' has to share the GUI/batch flag surface). The subcommands also
|
||||
# skip the multiprocessing 'spawn' setup which is only meaningful for the
|
||||
|
||||
@@ -344,7 +344,7 @@ class TestItemConsoleReadUntil(TestItemConsoleAction):
|
||||
def execute(self):
|
||||
cons = self.get_console()
|
||||
ru = self._prms.expanse(self._read_until)
|
||||
read_timeout = int(self._prms.getParam("timeout", default=-1, processed=True))
|
||||
read_timeout = float(self._prms.getParam("timeout", default=-1, processed=True))
|
||||
mute = self._prms.getParam("mute", default=False, processed=True)
|
||||
if read_timeout < 0:
|
||||
read_timeout = None
|
||||
|
||||
@@ -90,7 +90,7 @@ class TestItemPyFunc(TestItem):
|
||||
|
||||
if not engine.is_alive():
|
||||
engine.start()
|
||||
if not engine.wait_ready(10):
|
||||
if not engine.wait_ready():
|
||||
raise ETUMRuntimeError(
|
||||
f"""Impossible to start the external python execution process.
|
||||
Is the python path correct ?
|
||||
|
||||
@@ -14,7 +14,7 @@ class ReportExportHTML(rpe.ReportExport):
|
||||
self.prepareFile()
|
||||
self.create_base()
|
||||
self.process_tests()
|
||||
with open(self._file_name, 'w', encoding="utf-8") as f:
|
||||
with open(self._file_name, 'w') as f:
|
||||
f.write(lxml.html.tostring(self.root, pretty_print=True).decode())
|
||||
|
||||
def testsIterate(self, row):
|
||||
|
||||
@@ -20,7 +20,7 @@ class ReportExportJUnit(rpe.ReportExport):
|
||||
|
||||
ts = TestSuite(repname, test_cases=self.test_cases,
|
||||
hostname=tm.gd('host_ip'))
|
||||
with open(self._file_name, 'w', encoding="utf-8") as f:
|
||||
with open(self._file_name, 'w') as f:
|
||||
TestSuite.to_file(f, [ts])
|
||||
|
||||
def testsIterate(self, row):
|
||||
|
||||
@@ -388,16 +388,12 @@ def ensure(*names):
|
||||
"""
|
||||
missing = []
|
||||
for n in names:
|
||||
path = _resolve(n)
|
||||
if not _resolve(n):
|
||||
display, gd_key, candidates, _ = _SPECS[n]
|
||||
if not path:
|
||||
missing.append(
|
||||
f" - {display}: tried {candidates} on PATH, none usable. "
|
||||
f"Set '{gd_key}' in the YAML config to override."
|
||||
)
|
||||
elif not tm.gd(gd_key):
|
||||
# Publish resolved path so test scripts can use $(python_bin)/$(lua_bin).
|
||||
tm.setgd(gd_key, path)
|
||||
if missing:
|
||||
raise ETUMRuntimeError(
|
||||
"Required external interpreter(s) not found:\n" + "\n".join(missing)
|
||||
|
||||
@@ -1,13 +1,14 @@
|
||||
import os
|
||||
import sys
|
||||
import subprocess
|
||||
import socket
|
||||
|
||||
import api.testium as tm
|
||||
from runtime.jrpc import JsonRpcClient
|
||||
from interpreter.utils.paths import subproc_path
|
||||
from runtime.tum_except import ETUMRuntimeError
|
||||
from interpreter.utils import bins
|
||||
from interpreter.utils.proc_drain import drain_and_read_port, wait_for_port
|
||||
from interpreter.utils.proc_drain import drain_to_log
|
||||
|
||||
|
||||
class LuaProcessBase:
|
||||
@@ -78,7 +79,12 @@ class LuaProcessBase:
|
||||
else:
|
||||
env[k] = e + ";" + env.get(k, "")
|
||||
|
||||
# POpen params (port 0 -> the Lua server picks a free port and reports it)
|
||||
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
|
||||
sock.bind(("localhost", 0))
|
||||
self._port = sock.getsockname()[1]
|
||||
sock.close()
|
||||
|
||||
# POpen params
|
||||
cmd_args = [
|
||||
"main.lua",
|
||||
"--timeout",
|
||||
@@ -86,7 +92,7 @@ class LuaProcessBase:
|
||||
"--host",
|
||||
"127.0.0.1",
|
||||
"--port",
|
||||
"0",
|
||||
f"{self._port}",
|
||||
]
|
||||
|
||||
if tm.debug_enabled() and tm.gd("debug_rpc", False):
|
||||
@@ -116,16 +122,10 @@ class LuaProcessBase:
|
||||
restore_signals=False,
|
||||
**popen_kwargs,
|
||||
)
|
||||
# Forward subprocess output to the log and read the startup port sentinel.
|
||||
holder = drain_and_read_port(self._process, prefix="[lua_func] ")
|
||||
self._port = wait_for_port(
|
||||
self._process, holder, tm.gd("proc_start_timeout", 30)
|
||||
)
|
||||
if self._port is None:
|
||||
# Worker died before announcing its port: reset so a later start() retries clean.
|
||||
self.stop()
|
||||
self.join()
|
||||
return
|
||||
# Route subprocess stdout/stderr (lua require failures, syntax
|
||||
# errors, anything written to fd 1/2 before the in-script
|
||||
# remote_print is set up) into the parent's log.
|
||||
drain_to_log(self._process, prefix="[lua_func] ")
|
||||
|
||||
self._rpc = JsonRpcClient(
|
||||
"localhost", self._port, req_handler=self._req_handler
|
||||
|
||||
@@ -8,9 +8,6 @@ exceptions before the in-process redirection kicks in, lua
|
||||
``require`` failures, anything written to fd 1/2 directly).
|
||||
"""
|
||||
import threading
|
||||
from time import monotonic
|
||||
|
||||
from runtime.jrpc import RPC_PORT_SENTINEL
|
||||
|
||||
|
||||
def _drain_pipe(pipe, prefix):
|
||||
@@ -49,60 +46,3 @@ def drain_to_log(process, prefix=""):
|
||||
t.start()
|
||||
threads.append(t)
|
||||
return threads
|
||||
|
||||
|
||||
def drain_and_read_port(process, prefix=""):
|
||||
"""Like :func:`drain_to_log`, but the stdout reader also watches for the
|
||||
startup port sentinel. Returns a ``holder`` dict (passed to
|
||||
:func:`wait_for_port`); all non-sentinel lines are still forwarded to the
|
||||
log. stderr is drained as usual.
|
||||
"""
|
||||
holder = {"port": None, "evt": threading.Event()}
|
||||
|
||||
def _read_stdout(pipe):
|
||||
try:
|
||||
for raw in iter(pipe.readline, b""):
|
||||
line = raw.decode("utf-8", errors="replace").rstrip("\r\n")
|
||||
if holder["port"] is None and line.startswith(RPC_PORT_SENTINEL):
|
||||
try:
|
||||
holder["port"] = int(line[len(RPC_PORT_SENTINEL):].strip())
|
||||
except ValueError:
|
||||
continue
|
||||
holder["evt"].set()
|
||||
continue
|
||||
if line:
|
||||
print(f"{prefix}{line}" if prefix else line)
|
||||
finally:
|
||||
try:
|
||||
pipe.close()
|
||||
except Exception:
|
||||
pass
|
||||
holder["evt"].set() # unblock waiter on EOF even without sentinel
|
||||
|
||||
if process.stdout is not None:
|
||||
threading.Thread(
|
||||
target=_read_stdout, args=(process.stdout,), daemon=True,
|
||||
).start()
|
||||
if process.stderr is not None:
|
||||
threading.Thread(
|
||||
target=_drain_pipe, args=(process.stderr, prefix), daemon=True,
|
||||
).start()
|
||||
return holder
|
||||
|
||||
|
||||
def wait_for_port(process, holder, deadline):
|
||||
"""Block until the port sentinel arrives, the process dies, or *deadline*
|
||||
seconds elapse. Returns the port int or ``None``.
|
||||
"""
|
||||
end = monotonic() + deadline
|
||||
while holder["port"] is None:
|
||||
remaining = end - monotonic()
|
||||
if remaining <= 0:
|
||||
break
|
||||
holder["evt"].wait(min(remaining, 0.2))
|
||||
if holder["port"] is not None:
|
||||
break
|
||||
if process.poll() is not None:
|
||||
holder["evt"].wait(0.2) # child exited; let the reader flush a trailing line
|
||||
break
|
||||
return holder["port"]
|
||||
|
||||
@@ -1,12 +1,13 @@
|
||||
import os
|
||||
import sys
|
||||
import subprocess
|
||||
import socket
|
||||
from runtime.jrpc import JsonRpcClient
|
||||
import api.testium as tm
|
||||
from runtime.tum_except import ETUMRuntimeError
|
||||
from interpreter.utils.paths import testium_path, subproc_path
|
||||
from interpreter.utils import bins
|
||||
from interpreter.utils.proc_drain import drain_and_read_port, wait_for_port
|
||||
from interpreter.utils.proc_drain import drain_to_log
|
||||
|
||||
|
||||
class PyProcessBase:
|
||||
@@ -53,6 +54,13 @@ class PyProcessBase:
|
||||
else:
|
||||
env[k] = e + os.pathsep + env.get(k, "")
|
||||
|
||||
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
|
||||
sock.bind(("localhost", 0))
|
||||
self._port = sock.getsockname()[1]
|
||||
# Port was reserved until the sub-process is started. Now released.
|
||||
if sock is not None:
|
||||
sock.close()
|
||||
|
||||
# In Flatpak the host can't see /app/lib/testium, so use a staged copy
|
||||
# under /tmp (shared between sandbox and host) for both cwd and as the
|
||||
# root in PYTHONPATH. Outside Flatpak the original paths are used.
|
||||
@@ -67,7 +75,7 @@ class PyProcessBase:
|
||||
cmd_args = [
|
||||
"py_func",
|
||||
"-p",
|
||||
"0",
|
||||
f"{self._port}",
|
||||
"-t",
|
||||
f"{self._timeout}",
|
||||
]
|
||||
@@ -99,16 +107,11 @@ class PyProcessBase:
|
||||
restore_signals=False,
|
||||
**popen_kwargs,
|
||||
)
|
||||
# Forward subprocess output to the log and read the startup port sentinel.
|
||||
holder = drain_and_read_port(self._process, prefix="[py_func] ")
|
||||
self._port = wait_for_port(
|
||||
self._process, holder, tm.gd("proc_start_timeout", 30)
|
||||
)
|
||||
if self._port is None:
|
||||
# Worker died before announcing its port: reset so a later start() retries clean.
|
||||
self.stop()
|
||||
self.join()
|
||||
return
|
||||
# Route subprocess stdout/stderr (early-startup errors,
|
||||
# unhandled exceptions, anything written to fd 1/2 before the
|
||||
# in-process JSON-RPC stdio_redir kicks in) into the parent's
|
||||
# log.
|
||||
drain_to_log(self._process, prefix="[py_func] ")
|
||||
|
||||
self._rpc = JsonRpcClient(
|
||||
"localhost", self._port, req_handler=self._req_handler
|
||||
|
||||
@@ -56,9 +56,17 @@ function handle.func_call(params)
|
||||
if err == nil then
|
||||
print(string.format("Function executed from '%s'", pfile))
|
||||
utils.log("func_call function found '%s', '%s'", file, fname)
|
||||
succ, ret = pcall(func, unpack(prms))
|
||||
err_res = {pcall(func, unpack(prms))}
|
||||
utils.log("func_call returned '%s', '%s'", tostring(succ), tostring(ret))
|
||||
|
||||
-- manage tuple ouput of a lua function
|
||||
succ = table.remove(err_res, 1)
|
||||
if #err_res > 1 then
|
||||
ret = err_res
|
||||
else
|
||||
ret = unpack(err_res)
|
||||
end
|
||||
|
||||
if succ then
|
||||
res = ret
|
||||
else
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
-- =========================
|
||||
local config = {
|
||||
host = "0.0.0.0",
|
||||
port = 0, -- 0 = OS-assigned; actual port is reported on stdout
|
||||
port = 9000,
|
||||
timeout = 60,
|
||||
verbose = false,
|
||||
}
|
||||
@@ -76,10 +76,6 @@ server_sock:listen(1)
|
||||
local ip, port = server_sock:getsockname()
|
||||
utils.log("listening on %s:%d for %.1f secs", ip, port, config.timeout)
|
||||
|
||||
-- Announce the actual bound port so the parent connects only once we listen.
|
||||
io.stdout:write("__TESTIUM_RPC_PORT__=" .. port .. "\n")
|
||||
io.stdout:flush()
|
||||
|
||||
server_sock:settimeout(config.timeout) -- Prevents hanging on dead connections
|
||||
|
||||
-- Main Server Loop
|
||||
|
||||
@@ -51,14 +51,18 @@ class TestFileManager:
|
||||
w.disconnect_signals()
|
||||
# Snapshot user-selected checkboxes and fold state so they survive a
|
||||
# reload of the same file (same logic as session-restore through prefs).
|
||||
# checkList works only if show_checkboxes is True
|
||||
previous_check_list = w.treeTests.getCheckList()
|
||||
previous_fold_list = w.treeTests.getFoldList()
|
||||
previous_count = w.treeTests.getItemCount()
|
||||
self.clear_process()
|
||||
if self.load(file_name) and w.test_service is not None:
|
||||
if w.treeTests.getItemCount() == previous_count:
|
||||
if self.load(file_name) and \
|
||||
w.test_service is not None and \
|
||||
w.treeTests.getItemCount() == previous_count:
|
||||
if prefs.settings.show_checkboxes :
|
||||
w.treeTests.restoreCheckList(previous_check_list, w.test_service)
|
||||
w.treeTests.restoreFoldList(previous_fold_list)
|
||||
|
||||
w.reconnect_signals()
|
||||
|
||||
def _make_progress(self, w):
|
||||
|
||||
@@ -1,9 +1,7 @@
|
||||
#!/usr/bin/env python
|
||||
import sys
|
||||
import multiprocessing
|
||||
from py_func.tm import _init_api, _remote_print
|
||||
from runtime.stdout_redirect import stdio_redir
|
||||
from runtime.jrpc import RPC_PORT_SENTINEL
|
||||
|
||||
|
||||
class TcpStdOut:
|
||||
@@ -26,29 +24,21 @@ def main():
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("-i", "--ip", type=str, help="Ip address or hostname to listen to",
|
||||
default="localhost")
|
||||
parser.add_argument("-p", "--port", type=int, help="port to listen to (0 = OS-assigned)",
|
||||
default=0)
|
||||
parser.add_argument("-p", "--port", type=int, help="port to listen to",
|
||||
default=9000)
|
||||
parser.add_argument("-t", "--timeout", type=float, help="Timeout waiting for connection",
|
||||
default=10)
|
||||
parser.add_argument("-v", "--verbose", action='store_true', help="port to listen to")
|
||||
args = parser.parse_args()
|
||||
|
||||
thrd_api = _init_api(args.ip, args.port, args.timeout)
|
||||
# redirect I/O
|
||||
outstream = TcpStdOut()
|
||||
stdio_redir.redirect(outstream)
|
||||
# debug the server
|
||||
if args.verbose:
|
||||
thrd_api.dbg_out = stdio_redir.ini_stdout
|
||||
thrd_api.start()
|
||||
|
||||
# Announce the bound port on real stdout (before redirection) so the parent connects.
|
||||
port = thrd_api.wait_bound(args.timeout)
|
||||
if port is None:
|
||||
print("py_func: failed to bind a listening port", file=sys.stderr, flush=True)
|
||||
return
|
||||
print(f"{RPC_PORT_SENTINEL}{port}", flush=True)
|
||||
|
||||
# redirect I/O
|
||||
outstream = TcpStdOut()
|
||||
stdio_redir.redirect(outstream)
|
||||
try:
|
||||
while thrd_api.is_alive():
|
||||
thrd_api.join(1)
|
||||
|
||||
@@ -12,9 +12,6 @@ except:
|
||||
|
||||
from runtime.tum_except import ETUMRuntimeError
|
||||
|
||||
# Startup handshake: subprocess prints this + its bound port on stdout once listening.
|
||||
RPC_PORT_SENTINEL = "__TESTIUM_RPC_PORT__="
|
||||
|
||||
"""Lightweight JSON-RPC 2.0 helpers over TCP sockets.
|
||||
|
||||
This module implements a minimal JSON-RPC 2.0 messaging layer using
|
||||
@@ -282,8 +279,6 @@ class JsonRpcBase(threading.Thread):
|
||||
self._req_handler = req_handler
|
||||
self._dbg_out = dbg_out
|
||||
self._event_ready = threading.Event()
|
||||
# Set on success AND failure so wait_ready() never hangs; outcome in _connected.
|
||||
self._connected = False
|
||||
|
||||
def handle_request(self, method, params):
|
||||
"""Override to implement server-side request handling.
|
||||
@@ -319,12 +314,10 @@ class JsonRpcBase(threading.Thread):
|
||||
self.name, sock, self.handle_request, dbg_out=self.dbg_out
|
||||
)
|
||||
self._rpc.wait_ready()
|
||||
self._connected = True
|
||||
self._event_ready.set()
|
||||
|
||||
def wait_ready(self, timeout=None):
|
||||
self._event_ready.wait(timeout)
|
||||
return self._connected
|
||||
return self._event_ready.wait(timeout)
|
||||
|
||||
@property
|
||||
def dbg_out(self):
|
||||
@@ -355,30 +348,20 @@ class JsonRpcSrv(JsonRpcBase):
|
||||
def __init__(self, host, port, req_handler=None, timeout=10):
|
||||
super().__init__(host, port, req_handler, timeout)
|
||||
self.name = f"JsonRpcSvr_{port}"
|
||||
self._bound_port = None
|
||||
self._bound_evt = threading.Event()
|
||||
|
||||
@property
|
||||
def bound_port(self):
|
||||
return self._bound_port
|
||||
|
||||
def wait_bound(self, timeout=None):
|
||||
self._bound_evt.wait(timeout)
|
||||
return self._bound_port
|
||||
|
||||
def run(self):
|
||||
# TCP/IP socket creation
|
||||
try:
|
||||
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
|
||||
|
||||
# No SO_REUSEADDR: fresh ephemeral port; on Windows it enables hijacking.
|
||||
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
|
||||
|
||||
# Link of the socket at the configured port
|
||||
sock.bind((self._host, self._port))
|
||||
|
||||
# Listens incoming connections
|
||||
sock.listen(1)
|
||||
self._bound_port = sock.getsockname()[1]
|
||||
self._bound_evt.set()
|
||||
self.print_info(f"listening on {self._host}:{self._bound_port}")
|
||||
self.print_info(f"listening on {self._host}:{self._port}")
|
||||
|
||||
self.print_info(f"awaiting connection for {self._timeout} secs")
|
||||
sock.settimeout(self._timeout)
|
||||
@@ -399,7 +382,6 @@ class JsonRpcSrv(JsonRpcBase):
|
||||
sleep(0.1)
|
||||
|
||||
finally:
|
||||
self._bound_evt.set() # unblock wait_bound() even on failure
|
||||
if self._rpc is not None:
|
||||
self._rpc.stop()
|
||||
self._rpc.join()
|
||||
@@ -425,34 +407,35 @@ class JsonRpcClient(JsonRpcBase):
|
||||
self.name = f"JsonRpcClt_{port}"
|
||||
|
||||
def run(self):
|
||||
try:
|
||||
if tm.OS() == "Windows":
|
||||
self.run_win()
|
||||
else:
|
||||
self.run_lin()
|
||||
except Exception as e:
|
||||
self.print_info(f"connection failed: {e}")
|
||||
finally:
|
||||
self._event_ready.set() # settle wait_ready() whatever the outcome
|
||||
|
||||
def run_win(self):
|
||||
# Server already listening (handshake); retry on refused/timeout until deadline.
|
||||
deadline = monotonic() + self._timeout
|
||||
# TCP/IP socket creation
|
||||
tslice = 1
|
||||
t = self._timeout
|
||||
sock = None
|
||||
try:
|
||||
while True:
|
||||
while t >= 0:
|
||||
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
|
||||
sock.settimeout(0.5)
|
||||
sock.settimeout(tslice)
|
||||
# Link of the socket at the configured port
|
||||
try:
|
||||
sock.connect((self._host, self._port))
|
||||
break
|
||||
except OSError as e:
|
||||
except socket.timeout:
|
||||
sock.close()
|
||||
if monotonic() >= deadline:
|
||||
t -= tslice
|
||||
if t < 0:
|
||||
raise ETUMRuntimeError(
|
||||
f"{self.name}: failed to connect : {e}"
|
||||
f"{self.name}: failed to connect : timeout"
|
||||
)
|
||||
sleep(0.1)
|
||||
else:
|
||||
sleep(tslice)
|
||||
except socket.error as e:
|
||||
raise ETUMRuntimeError(f"{self.name}: failed to connect : {e}")
|
||||
|
||||
self.print_info("Connected to server")
|
||||
self.connect(sock)
|
||||
|
||||
@@ -84,7 +84,18 @@
|
||||
- read_until: {expected: HelloConsole, timeout: 1, mute: true}
|
||||
|
||||
- console:
|
||||
name: Console read_until muted
|
||||
name: Console read_until float timeout
|
||||
console_name: term
|
||||
key: $(test)_PASS
|
||||
steps:
|
||||
- writeln: echo "HelloConsole"
|
||||
{% if os == "Windows" %}
|
||||
- read_until: {expected: echo "HelloConsole", timeout: 0.2}
|
||||
{% endif %}
|
||||
- read_until: {expected: HelloConsole, timeout: 0.2}
|
||||
|
||||
- console:
|
||||
name: Console read_until process result
|
||||
console_name: term
|
||||
key: $(test)_PASS
|
||||
steps:
|
||||
|
||||
@@ -20,7 +20,7 @@
|
||||
console_name: jrpces
|
||||
key: $(test)_PASS
|
||||
steps:
|
||||
- writeln: '"$(python_bin)" {{include_directory}}/jrpc_echo_server.py -c {{include_directory}}/jrpces.ini'
|
||||
- writeln: python3 {{include_directory}}/jrpc_echo_server.py -c {{include_directory}}/jrpces.ini
|
||||
- read_until: {expected: ready, timeout: 5}
|
||||
|
||||
- console:
|
||||
|
||||
@@ -11,8 +11,8 @@
|
||||
- let:
|
||||
name: Let it be
|
||||
values:
|
||||
it: $(loop_param)
|
||||
be: <| $(loop_param) == $(it) |>
|
||||
- it: $(loop_param)
|
||||
- be: <| $(loop_param) == $(it) |>
|
||||
|
||||
- loop:
|
||||
name: Cycle iterating on list
|
||||
|
||||
@@ -12,12 +12,12 @@ function module.assertparam(param)
|
||||
end
|
||||
|
||||
function module.checkglobal(param)
|
||||
local res = tm.gd(param)
|
||||
return res
|
||||
assert(param=='test parameter')
|
||||
return 0
|
||||
end
|
||||
|
||||
function module.checkglobal2(index)
|
||||
return tm.gd("lua_data_to_be_returned")[index]
|
||||
return tm.gd("data_to_be_returned")[index+1]
|
||||
end
|
||||
|
||||
function module.should_not_be_called(param)
|
||||
@@ -53,7 +53,7 @@ function module.return_nothing()
|
||||
-- Returns no value: ret is nil but no error.
|
||||
end
|
||||
|
||||
function module.return_explicit_nil()
|
||||
function module.return_explicit_none()
|
||||
return nil
|
||||
end
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
skipped_test_item: ['skipped_checkglobal']
|
||||
|
||||
lua_data_to_be_returned:
|
||||
data_to_be_returned:
|
||||
- 1
|
||||
- {a: 1, b: 2}
|
||||
- ["a", 1, 2.1, True]
|
||||
@@ -1,7 +1,15 @@
|
||||
- let:
|
||||
name: lua_func test constants,
|
||||
values:
|
||||
lua_func test parameter: test parameter lua_func
|
||||
- func_test_parameter: test parameter
|
||||
|
||||
- lua_func:
|
||||
name: pass lua_func
|
||||
key: $(test)_PASS
|
||||
file: $(test_path)$(psep)lua_func.lua
|
||||
func_name: assertparam
|
||||
param:
|
||||
- true
|
||||
|
||||
- lua_func:
|
||||
name: fail lua_func
|
||||
@@ -12,7 +20,7 @@
|
||||
- false
|
||||
|
||||
- lua_func:
|
||||
name: fail lua_func with expected result FAIL
|
||||
name: fail lua_func with expected result "FAIL"
|
||||
key: $(test)_PASS
|
||||
file: $(test_path)$(psep)lua_func.lua
|
||||
func_name: assertparam
|
||||
@@ -62,35 +70,7 @@
|
||||
file: $(test_path)$(psep)lua_func.lua
|
||||
func_name: checkglobal
|
||||
param:
|
||||
- lua_func test parameter
|
||||
expected_result: $(lua_func test parameter)
|
||||
|
||||
- lua_func:
|
||||
name: global param lua_func 1
|
||||
key: $(test)_PASS
|
||||
file: $(test_path)$(psep)lua_func.lua
|
||||
func_name: checkglobal2
|
||||
param:
|
||||
- 1
|
||||
expected_result: ($(lua_data_to_be_returned))[0]
|
||||
|
||||
- lua_func:
|
||||
name: global param lua_func 2
|
||||
key: $(test)_PASS
|
||||
file: $(test_path)$(psep)lua_func.lua
|
||||
func_name: checkglobal2
|
||||
param:
|
||||
- 2
|
||||
expected_result: ($(lua_data_to_be_returned))[1]
|
||||
|
||||
- lua_func:
|
||||
name: global param lua_func 3
|
||||
key: $(test)_PASS
|
||||
file: $(test_path)$(psep)lua_func.lua
|
||||
func_name: checkglobal2
|
||||
param:
|
||||
- 3
|
||||
expected_result: ($(lua_data_to_be_returned))[2]
|
||||
- $(func_test_parameter)
|
||||
|
||||
- let:
|
||||
name: python2func
|
||||
@@ -98,88 +78,189 @@
|
||||
values:
|
||||
- py: $(test_path)$(psep)lua_func.lua
|
||||
|
||||
- lua_func:
|
||||
name: global param int
|
||||
key: $(test)_PASS
|
||||
file: $(test_path)$(psep)lua_func.lua
|
||||
func_name: checkglobal2
|
||||
param:
|
||||
- 0
|
||||
expected_result: ($(data_to_be_returned))[0]
|
||||
|
||||
- lua_func:
|
||||
name: global param dict
|
||||
key: $(test)_PASS
|
||||
file: $(test_path)$(psep)lua_func.lua
|
||||
func_name: checkglobal2
|
||||
param:
|
||||
- 1
|
||||
expected_result: ($(data_to_be_returned))[1]
|
||||
|
||||
- lua_func:
|
||||
name: global param list
|
||||
key: $(test)_PASS
|
||||
file: $(test_path)$(psep)lua_func.lua
|
||||
func_name: checkglobal2
|
||||
param:
|
||||
- 2
|
||||
expected_result: ($(data_to_be_returned))[2]
|
||||
|
||||
|
||||
|
||||
- lua_func:
|
||||
name: global param lua_func
|
||||
key: $(test)_PASS
|
||||
file: $(py)
|
||||
func_name: checkglobal
|
||||
param:
|
||||
- $(func_test_parameter)
|
||||
|
||||
- lua_func:
|
||||
name: skipped_checkglobal
|
||||
key: $(test)_PASS
|
||||
file: $(test_path)$(psep)lua_func.lua
|
||||
func_name: should_not_be_called
|
||||
param:
|
||||
- $(test parameter)
|
||||
- $(func_test_parameter)
|
||||
|
||||
- lua_func:
|
||||
name: skipped true
|
||||
key: $(test)_FAIL
|
||||
file: $(test_path)$(psep)lua_func.lua
|
||||
func_name: checkglobal
|
||||
func_name: echo
|
||||
skipped: true
|
||||
param:
|
||||
- $(test parameter)
|
||||
- "skipped"
|
||||
|
||||
- lua_func:
|
||||
name: skipped 1
|
||||
key: $(test)_FAIL
|
||||
file: $(test_path)$(psep)lua_func.lua
|
||||
func_name: checkglobal
|
||||
func_name: echo
|
||||
skipped: 1
|
||||
param:
|
||||
- $(test parameter)
|
||||
- "skipped"
|
||||
|
||||
- group:
|
||||
name: Function results check
|
||||
steps:
|
||||
- group:
|
||||
name: Function result failure
|
||||
name: Functions result
|
||||
steps:
|
||||
- lua_func:
|
||||
name: int failure
|
||||
name: int
|
||||
key: $(test)_PASS
|
||||
file: $(test_path)$(psep)lua_func.lua
|
||||
func_name: echo
|
||||
param: [-1]
|
||||
- lua_func:
|
||||
name: float failure
|
||||
name: float
|
||||
key: $(test)_PASS
|
||||
file: $(test_path)$(psep)lua_func.lua
|
||||
func_name: echo
|
||||
param: [-1.3]
|
||||
param: [-20.3]
|
||||
- lua_func:
|
||||
name: String failure
|
||||
name: String
|
||||
key: $(test)_PASS
|
||||
file: $(test_path)$(psep)lua_func.lua
|
||||
func_name: echo
|
||||
param: [ "FAIL" ]
|
||||
- lua_func:
|
||||
name: Tuple int,str failure
|
||||
key: $(test)_PASS
|
||||
file: $(test_path)$(psep)lua_func.lua
|
||||
func_name: tuple_return
|
||||
param: [ -1, "Got a failure" ]
|
||||
- group:
|
||||
name: Functions result success
|
||||
steps:
|
||||
- lua_func:
|
||||
name: int success
|
||||
key: $(test)_PASS
|
||||
file: $(test_path)$(psep)lua_func.lua
|
||||
func_name: echo
|
||||
param: [0]
|
||||
- lua_func:
|
||||
name: float success
|
||||
key: $(test)_PASS
|
||||
file: $(test_path)$(psep)lua_func.lua
|
||||
func_name: echo
|
||||
param: [0.3]
|
||||
- lua_func:
|
||||
name: String success
|
||||
key: $(test)_PASS
|
||||
file: $(test_path)$(psep)lua_func.lua
|
||||
func_name: echo
|
||||
param: [ "Something that is not only strictly FAIL" ]
|
||||
- lua_func:
|
||||
name: Tuple int,str success
|
||||
name: Tuple int,str
|
||||
key: $(test)_PASS
|
||||
file: $(test_path)$(psep)lua_func.lua
|
||||
func_name: tuple_return
|
||||
param: [ 0, "OK" ]
|
||||
|
||||
- group:
|
||||
name: Functions result expected
|
||||
steps:
|
||||
- lua_func:
|
||||
name: int expected
|
||||
key: $(test)_PASS
|
||||
file: $(test_path)$(psep)lua_func.lua
|
||||
func_name: echo
|
||||
param: [18]
|
||||
expected_result: 18
|
||||
- lua_func:
|
||||
name: float expected
|
||||
key: $(test)_PASS
|
||||
file: $(test_path)$(psep)lua_func.lua
|
||||
func_name: echo
|
||||
param: [0.3]
|
||||
expected_result: 0.3
|
||||
- lua_func:
|
||||
name: String expected
|
||||
key: $(test)_PASS
|
||||
file: $(test_path)$(psep)lua_func.lua
|
||||
func_name: echo
|
||||
param: [ "Something" ]
|
||||
expected_result: Something
|
||||
- lua_func:
|
||||
name: Tuple int,str expected
|
||||
key: $(test)_PASS
|
||||
file: $(test_path)$(psep)lua_func.lua
|
||||
func_name: tuple_return
|
||||
param: [ 0, "OK" ]
|
||||
expected_result: [0, "OK"]
|
||||
- lua_func:
|
||||
name: small list expected
|
||||
key: $(test)_PASS
|
||||
file: $(test_path)$(psep)lua_func.lua
|
||||
func_name: echo
|
||||
param: [ [-23] ]
|
||||
expected_result: [-23]
|
||||
- lua_func:
|
||||
name: big list expected
|
||||
key: $(test)_PASS
|
||||
file: $(test_path)$(psep)lua_func.lua
|
||||
func_name: echo
|
||||
param: [ [-23, 17, 67] ]
|
||||
expected_result: [-23, 17, 67]
|
||||
- group:
|
||||
name: Function result not expected
|
||||
steps:
|
||||
- lua_func:
|
||||
name: int not expected
|
||||
key: $(test)_FAIL
|
||||
file: $(test_path)$(psep)lua_func.lua
|
||||
func_name: echo
|
||||
param: [18]
|
||||
expected_result: 17
|
||||
- lua_func:
|
||||
name: float not expected
|
||||
key: $(test)_FAIL
|
||||
file: $(test_path)$(psep)lua_func.lua
|
||||
func_name: echo
|
||||
param: [0.3]
|
||||
expected_result: 0.5
|
||||
- lua_func:
|
||||
name: String not expected
|
||||
key: $(test)_FAIL
|
||||
file: $(test_path)$(psep)lua_func.lua
|
||||
func_name: echo
|
||||
param: [ "Something" ]
|
||||
expected_result: Nothing
|
||||
- lua_func:
|
||||
name: Tuple int,str not expected
|
||||
key: $(test)_FAIL
|
||||
file: $(test_path)$(psep)lua_func.lua
|
||||
func_name: tuple_return
|
||||
param: [ 0, "OK" ]
|
||||
expected_result: [0, "OUPS"]
|
||||
- lua_func:
|
||||
name: small list not expected
|
||||
key: $(test)_FAIL
|
||||
file: $(test_path)$(psep)lua_func.lua
|
||||
func_name: echo
|
||||
param: [ [-23] ]
|
||||
expected_result: [-22]
|
||||
- lua_func:
|
||||
name: big list not expected
|
||||
key: $(test)_FAIL
|
||||
file: $(test_path)$(psep)lua_func.lua
|
||||
func_name: echo
|
||||
param: [ [-23, 17, 67] ]
|
||||
expected_result: [-23, 16, 67]
|
||||
- lua_func:
|
||||
name: delgd test
|
||||
key: $(test)_PASS
|
||||
@@ -193,40 +274,39 @@
|
||||
func_name: return_nothing
|
||||
|
||||
- lua_func:
|
||||
name: function returning explicit nil should succeed
|
||||
name: function returning explicit None should succeed
|
||||
key: $(test)_PASS
|
||||
file: $(test_path)$(psep)lua_func.lua
|
||||
func_name: return_explicit_nil
|
||||
func_name: return_explicit_none
|
||||
|
||||
- group:
|
||||
name: context_id tests
|
||||
steps:
|
||||
- lua_func:
|
||||
name: set context value
|
||||
name: set serializable value
|
||||
key: $(test)_PASS
|
||||
file: $(test_path)$(psep)lua_func.lua
|
||||
func_name: set_context_value
|
||||
context_id: lua_ctx_test
|
||||
param:
|
||||
- hello lua
|
||||
expected_result: hello lua
|
||||
- hello context
|
||||
expected_result: hello context
|
||||
- lua_func:
|
||||
name: get context value (same context_id)
|
||||
name: get serializable value (same context_id)
|
||||
key: $(test)_PASS
|
||||
file: $(test_path)$(psep)lua_func.lua
|
||||
func_name: get_context_value
|
||||
context_id: lua_ctx_test
|
||||
expected_result: hello lua
|
||||
context_id: ctx_test
|
||||
expected_result: hello context
|
||||
- lua_func:
|
||||
name: get context value (no context_id, from main gd)
|
||||
name: get serializable value (no context_id, from main gd)
|
||||
key: $(test)_PASS
|
||||
file: $(test_path)$(psep)lua_func.lua
|
||||
func_name: get_context_value
|
||||
expected_result: hello lua
|
||||
expected_result: hello context
|
||||
- lua_func:
|
||||
name: get context value (different context_id)
|
||||
name: get serializable value (different context_id)
|
||||
key: $(test)_PASS
|
||||
file: $(test_path)$(psep)lua_func.lua
|
||||
func_name: get_context_value
|
||||
context_id: lua_ctx_other
|
||||
expected_result: hello lua
|
||||
context_id: ctx_other
|
||||
expected_result: hello context
|
||||
|
||||
@@ -1 +1,6 @@
|
||||
skipped_test_item: ['skipped_checkglobal']
|
||||
|
||||
data_to_be_returned:
|
||||
- 1
|
||||
- {a: 1, b: 2}
|
||||
- ["a", 1, 2.1, True]
|
||||
@@ -16,8 +16,8 @@ def checkglobal(param):
|
||||
assert param=='test parameter'
|
||||
return 0
|
||||
|
||||
def checkglobal2():
|
||||
return tm.gd("py_func test parameter")
|
||||
def checkglobal2(index):
|
||||
return tm.gd("data_to_be_returned")[index]
|
||||
|
||||
def should_not_be_called(param):
|
||||
raise
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
- let:
|
||||
name: py_func test constants,
|
||||
values:
|
||||
py_func test parameter: test parameter
|
||||
- func_test_parameter: test parameter
|
||||
|
||||
- py_func:
|
||||
name: pass py_func
|
||||
@@ -70,7 +70,7 @@
|
||||
file: $(test_path)$(psep)py_func.py
|
||||
func_name: checkglobal
|
||||
param:
|
||||
- $(py_func test parameter)
|
||||
- $(func_test_parameter)
|
||||
|
||||
- let:
|
||||
name: python2func
|
||||
@@ -79,11 +79,32 @@
|
||||
- py: $(test_path)$(psep)py_func.py
|
||||
|
||||
- py_func:
|
||||
name: global param py_func 2
|
||||
name: global param int
|
||||
key: $(test)_PASS
|
||||
file: $(py)
|
||||
file: $(test_path)$(psep)py_func.py
|
||||
func_name: checkglobal2
|
||||
expected_result: $(py_func test parameter)
|
||||
param:
|
||||
- 0
|
||||
expected_result: ($(data_to_be_returned))[0]
|
||||
|
||||
- py_func:
|
||||
name: global param dict
|
||||
key: $(test)_PASS
|
||||
file: $(test_path)$(psep)py_func.py
|
||||
func_name: checkglobal2
|
||||
param:
|
||||
- 1
|
||||
expected_result: ($(data_to_be_returned))[1]
|
||||
|
||||
- py_func:
|
||||
name: global param list
|
||||
key: $(test)_PASS
|
||||
file: $(test_path)$(psep)py_func.py
|
||||
func_name: checkglobal2
|
||||
param:
|
||||
- 2
|
||||
expected_result: ($(data_to_be_returned))[2]
|
||||
|
||||
|
||||
|
||||
- py_func:
|
||||
@@ -92,104 +113,162 @@
|
||||
file: $(py)
|
||||
func_name: checkglobal
|
||||
param:
|
||||
- $(py_func test parameter)
|
||||
- $(func_test_parameter)
|
||||
|
||||
- py_func:
|
||||
name: skipped_checkglobal
|
||||
key: $(test)_PASS
|
||||
file: $(test_path)$(psep)py_func.py
|
||||
func_name: should_not_be_called
|
||||
param:
|
||||
- $(py_func test parameter)
|
||||
- $(func_test_parameter)
|
||||
|
||||
- py_func:
|
||||
name: skipped true
|
||||
key: $(test)_FAIL
|
||||
file: $(test_path)$(psep)py_func.py
|
||||
func_name: checkglobal
|
||||
func_name: echo
|
||||
skipped: true
|
||||
param:
|
||||
- $(py_func test parameter)
|
||||
- "skipped"
|
||||
|
||||
- py_func:
|
||||
name: skipped 1
|
||||
key: $(test)_FAIL
|
||||
file: $(test_path)$(psep)py_func.py
|
||||
func_name: checkglobal
|
||||
func_name: echo
|
||||
skipped: 1
|
||||
param:
|
||||
- $(py_func test parameter)
|
||||
- "skipped"
|
||||
|
||||
- py_func:
|
||||
name: FunctionItem test
|
||||
key: $(test)_PASS
|
||||
file: $(test_path)$(psep)py_func.py
|
||||
func_name: ValidationTest
|
||||
param:
|
||||
- $(py_func test parameter)
|
||||
- $(func_test_parameter)
|
||||
|
||||
- group:
|
||||
name: Function results check
|
||||
steps:
|
||||
- group:
|
||||
name: Function result 1
|
||||
name: Functions result
|
||||
steps:
|
||||
- py_func:
|
||||
name: int failure
|
||||
name: int
|
||||
key: $(test)_PASS
|
||||
file: $(test_path)$(psep)py_func.py
|
||||
func_name: echo
|
||||
param: [-1]
|
||||
expected_result: -1
|
||||
- py_func:
|
||||
name: float failure
|
||||
name: float
|
||||
key: $(test)_PASS
|
||||
file: $(test_path)$(psep)py_func.py
|
||||
func_name: echo
|
||||
param: [-1.3]
|
||||
expected_result: -1.3
|
||||
param: [-20.3]
|
||||
- py_func:
|
||||
name: String failure
|
||||
name: String
|
||||
key: $(test)_PASS
|
||||
file: $(test_path)$(psep)py_func.py
|
||||
func_name: echo
|
||||
param: [ "FAIL" ]
|
||||
expected_result: FAIL
|
||||
- py_func:
|
||||
name: Tuple int,str failure
|
||||
name: Tuple int,str
|
||||
key: $(test)_PASS
|
||||
file: $(test_path)$(psep)py_func.py
|
||||
func_name: tuple_return
|
||||
param: [ -1, "Got a failure" ]
|
||||
expected_result: [-1, "Got a failure"]
|
||||
param: [ 0, "OK" ]
|
||||
- group:
|
||||
name: Functions result 2
|
||||
name: Functions result expected
|
||||
steps:
|
||||
- py_func:
|
||||
name: int success
|
||||
name: int expected
|
||||
key: $(test)_PASS
|
||||
file: $(test_path)$(psep)py_func.py
|
||||
func_name: echo
|
||||
param: [0]
|
||||
expected_result: 0
|
||||
param: [18]
|
||||
expected_result: 18
|
||||
- py_func:
|
||||
name: float success
|
||||
name: float expected
|
||||
key: $(test)_PASS
|
||||
file: $(test_path)$(psep)py_func.py
|
||||
func_name: echo
|
||||
param: [0.3]
|
||||
expected_result: 0.3
|
||||
- py_func:
|
||||
name: String success
|
||||
name: String expected
|
||||
key: $(test)_PASS
|
||||
file: $(test_path)$(psep)py_func.py
|
||||
func_name: echo
|
||||
param: [ "Something that is not only strictly FAIL" ]
|
||||
expected_result: Something that is not only strictly FAIL
|
||||
param: [ "Something" ]
|
||||
expected_result: Something
|
||||
- py_func:
|
||||
name: Tuple int,str success
|
||||
name: Tuple int,str expected
|
||||
key: $(test)_PASS
|
||||
file: $(test_path)$(psep)py_func.py
|
||||
func_name: tuple_return
|
||||
param: [ 0, "OK" ]
|
||||
expected_result: [0, "OK"]
|
||||
|
||||
- py_func:
|
||||
name: small list expected
|
||||
key: $(test)_PASS
|
||||
file: $(test_path)$(psep)py_func.py
|
||||
func_name: echo
|
||||
param: [ [-23] ]
|
||||
expected_result: [-23]
|
||||
- py_func:
|
||||
name: big list expected
|
||||
key: $(test)_PASS
|
||||
file: $(test_path)$(psep)py_func.py
|
||||
func_name: echo
|
||||
param: [ [-23, 17, 67] ]
|
||||
expected_result: [-23, 17, 67]
|
||||
- group:
|
||||
name: Function result not expected
|
||||
steps:
|
||||
- py_func:
|
||||
name: int not expected
|
||||
key: $(test)_FAIL
|
||||
file: $(test_path)$(psep)py_func.py
|
||||
func_name: echo
|
||||
param: [18]
|
||||
expected_result: 17
|
||||
- py_func:
|
||||
name: float not expected
|
||||
key: $(test)_FAIL
|
||||
file: $(test_path)$(psep)py_func.py
|
||||
func_name: echo
|
||||
param: [0.3]
|
||||
expected_result: 0.5
|
||||
- py_func:
|
||||
name: String not expected
|
||||
key: $(test)_FAIL
|
||||
file: $(test_path)$(psep)py_func.py
|
||||
func_name: echo
|
||||
param: [ "Something" ]
|
||||
expected_result: Nothing
|
||||
- py_func:
|
||||
name: Tuple int,str not expected
|
||||
key: $(test)_FAIL
|
||||
file: $(test_path)$(psep)py_func.py
|
||||
func_name: tuple_return
|
||||
param: [ 0, "OK" ]
|
||||
expected_result: [0, "OUPS"]
|
||||
- py_func:
|
||||
name: small list not expected
|
||||
key: $(test)_FAIL
|
||||
file: $(test_path)$(psep)py_func.py
|
||||
func_name: echo
|
||||
param: [ [-23] ]
|
||||
expected_result: [-22]
|
||||
- py_func:
|
||||
name: big list not expected
|
||||
key: $(test)_FAIL
|
||||
file: $(test_path)$(psep)py_func.py
|
||||
func_name: echo
|
||||
param: [ [-23, 17, 67] ]
|
||||
expected_result: [-23, 16, 67]
|
||||
- py_func:
|
||||
name: delgd test
|
||||
key: $(test)_PASS
|
||||
|
||||
@@ -1,7 +0,0 @@
|
||||
main:
|
||||
name: run sub-test (always fail)
|
||||
steps:
|
||||
- check:
|
||||
name: fail
|
||||
values:
|
||||
- false
|
||||
@@ -1,7 +0,0 @@
|
||||
main:
|
||||
name: run sub-test (always pass)
|
||||
steps:
|
||||
- check:
|
||||
name: pass
|
||||
values:
|
||||
- true
|
||||
@@ -31,7 +31,11 @@ main:
|
||||
|
||||
{% for item in items %}
|
||||
# item test
|
||||
- let: {name: {{ item }} test constants, values: {test: {{ item }}, test_path: items/$(test)}}
|
||||
- let:
|
||||
name: {{ item }} test constants
|
||||
values:
|
||||
- test: {{ item }}
|
||||
- test_path: items/$(test)
|
||||
- group:
|
||||
name: {{ item }} test
|
||||
steps:
|
||||
|
||||
@@ -89,7 +89,7 @@ def exec():
|
||||
junit_report = report.replace(".sqlite", f"-{test}.xml")
|
||||
print(junit_report)
|
||||
_prepare_file_to_save(junit_report)
|
||||
with open(junit_report, "w", encoding="utf-8") as f:
|
||||
with open(junit_report, "w") as f:
|
||||
f.write(TestSuite.to_xml_string([ts]))
|
||||
|
||||
# cleanup
|
||||
|
||||
@@ -89,13 +89,6 @@ REM Reports are stamped with the mode so successive runs don't clobber each othe
|
||||
|
||||
SET "TAIL=-b -d "python_bin=%VENV_PYTHON%" -d "validation_report_file=validation-%MODE%" -- "%SCRIPT_DIR%\main.tum"%EXTRA%"
|
||||
|
||||
REM The report-exporter plugin (items\report_plugin) is a pip entry-point
|
||||
REM package. It must live in the *testium* environment, so it is installed into
|
||||
REM the source/wheel venvs below. A frozen PyInstaller binary cannot see
|
||||
REM externally-installed plugins, so report_plugin is expected to be skipped
|
||||
REM there (same as Linux pyinstaller mode).
|
||||
SET "FAKE_EXPORTER=%SCRIPT_DIR%\fake_exporter"
|
||||
|
||||
REM ---------- per-mode launcher ----------------------------------------------
|
||||
|
||||
echo -- validation mode: %MODE%
|
||||
@@ -107,25 +100,8 @@ echo ERROR: unknown --mode '%MODE%'. Expected: source ^| wheel ^| pyinstaller.
|
||||
exit /b 1
|
||||
|
||||
:MODE_SOURCE
|
||||
REM Run testium from src\ in a dedicated venv set up here. We do NOT delegate to
|
||||
REM the project's run.bat: that one launches the GUI and does not forward its
|
||||
REM arguments, so the suite would never run head-less.
|
||||
SET "TESTIUM_VENV=%PROJECT_DIR%\test\tmp\testium_venv"
|
||||
IF NOT EXIST "%TESTIUM_VENV%" (
|
||||
echo Creating testium venv at %TESTIUM_VENV%
|
||||
%PYTHON_EXE% -m venv "%TESTIUM_VENV%"
|
||||
IF !ERRORLEVEL! NEQ 0 (
|
||||
echo ERROR while creating the testium venv.
|
||||
exit /b 1
|
||||
)
|
||||
call "%TESTIUM_VENV%\Scripts\pip" install --quiet --upgrade pip
|
||||
call "%TESTIUM_VENV%\Scripts\pip" install --quiet -r "%PROJECT_DIR%\src\requirements.txt"
|
||||
REM language-server extra so `testium lsp` works from source (lsp_check.py)
|
||||
call "%TESTIUM_VENV%\Scripts\pip" install --quiet "pygls>=1.3"
|
||||
)
|
||||
call "%TESTIUM_VENV%\Scripts\pip" install --quiet -e "%FAKE_EXPORTER%"
|
||||
SET CMD="%TESTIUM_VENV%\Scripts\python.exe" "%PROJECT_DIR%\src\testium"
|
||||
GOTO LAUNCH
|
||||
call "%PROJECT_DIR%\run.bat" %TAIL%
|
||||
exit /b %ERRORLEVEL%
|
||||
|
||||
:MODE_WHEEL
|
||||
SET "WHEEL=%PROJECT_DIR%\dist\testium-%VERSION%-py3-none-any.whl"
|
||||
@@ -139,13 +115,10 @@ IF NOT EXIST "%WHEEL_VENV%" (
|
||||
echo Creating wheel venv at %WHEEL_VENV%
|
||||
%PYTHON_EXE% -m venv --system-site-packages "%WHEEL_VENV%"
|
||||
call "%WHEEL_VENV%\Scripts\pip" install --quiet --upgrade pip
|
||||
REM install with the [lsp] extra so the wheel channel is validated in its
|
||||
REM language-server-capable form (pulls pygls), matching `pip install testium[lsp]`.
|
||||
call "%WHEEL_VENV%\Scripts\pip" install --quiet "%WHEEL%[lsp]"
|
||||
call "%WHEEL_VENV%\Scripts\pip" install --quiet "%WHEEL%"
|
||||
)
|
||||
call "%WHEEL_VENV%\Scripts\pip" install --quiet -e "%FAKE_EXPORTER%"
|
||||
SET CMD="%WHEEL_VENV%\Scripts\python.exe" -m testium
|
||||
GOTO LAUNCH
|
||||
"%WHEEL_VENV%\Scripts\python.exe" -m testium %TAIL%
|
||||
exit /b %ERRORLEVEL%
|
||||
|
||||
:MODE_PYI
|
||||
SET "PYI_BIN=%PROJECT_DIR%\dist\testium-%VERSION%.exe"
|
||||
@@ -154,22 +127,5 @@ IF NOT EXIST "%PYI_BIN%" (
|
||||
echo ERROR: PyInstaller binary not found in %PROJECT_DIR%\dist -- run build_all.sh first.
|
||||
exit /b 1
|
||||
)
|
||||
SET CMD="%PYI_BIN%"
|
||||
GOTO LAUNCH
|
||||
|
||||
REM ---------- launch ----------------------------------------------------------
|
||||
|
||||
:LAUNCH
|
||||
echo -- launch: %CMD%
|
||||
|
||||
REM LSP check (this exact channel): `schema` must keep its nested actions and
|
||||
REM `lsp` must answer initialize. Mirrors run.sh; aborts the run on failure.
|
||||
echo -- LSP check (%MODE%)
|
||||
"%VENV_PYTHON%" "%SCRIPT_DIR%\lsp_check.py" %CMD%
|
||||
IF !ERRORLEVEL! NEQ 0 (
|
||||
echo ERROR: LSP check failed for mode %MODE%.
|
||||
exit /b 1
|
||||
)
|
||||
|
||||
%CMD% %TAIL%
|
||||
"%PYI_BIN%" %TAIL%
|
||||
exit /b %ERRORLEVEL%
|
||||
|
||||
Reference in New Issue
Block a user