diff --git a/doc/manual/sphinx/source/test_items/lua_func_test_item.rst b/doc/manual/sphinx/source/test_items/lua_func_test_item.rst index 2095975..6d8107e 100644 --- a/doc/manual/sphinx/source/test_items/lua_func_test_item.rst +++ b/doc/manual/sphinx/source/test_items/lua_func_test_item.rst @@ -39,11 +39,14 @@ The ``lua_func`` test item is of the form: Beside common test items attributes, lua_func item has specific attribute, some of which being mandatory. * ``file``: the script file name that contains the function to be executed. - Only python script format is supported. + Only Lua script format is supported. * ``func_name``: The function name to be executed. * ``param``: This is a list of parameters that are passed to the function in the order they are presented in the script. These parameters are not mandatory and are highly dependent of the function prototype. +* ``context_id``: Optional. When set, all ``lua_func`` items sharing the same + ``context_id`` value run inside the same persistent Lua subprocess for the + duration of the test. See :ref:`lua_func context` for details. .. code-block:: yaml :caption: ``lua_func`` test item example of usage @@ -56,16 +59,71 @@ Beside common test items attributes, lua_func item has specific attribute, some - $(my_param) The result of the function (after eventual post treatment) is stored in the global -variable named ``pfn_`` +variable named ``lfn_`` (See :ref:`global variables` for more detail on how to access to global variables from test items and scripts). In the example above, the global variable ``$(lfn_activity)`` would be created at the end of the item execution. It would contain the resulting -value of the funcToBeExecuted python function. +value of the methodName Lua function. The ``lua_func`` will always result ``PASS``, except if the called function raises -and exception or if the ``expected_result`` attribute is used. +an exception or if the ``expected_result`` attribute is used. + +.. _sec_lua_func_context: + +Sharing state between ``lua_func`` calls +------------------------------------------ + +Each ``lua_func`` item without a ``context_id`` runs in a dedicated subprocess that +is started and stopped around the call. Module-level variables are not preserved +between two such items. + +Inside a ``lua_func`` script, the ``tm`` module exposes ``tm.setgd`` and ``tm.gd`` +to read and write the testium global dictionary of the test process. Values stored +this way are accessible from any subsequent test item without requiring a shared +subprocess. + +.. code-block:: lua + :caption: sharing a value via the global dictionary + + local tm = require("tm") + local module = {} + + function module.produce(val) + tm.setgd("my_shared_value", val) + return val + end + + function module.consume() + return tm.gd("my_shared_value") + end + + return module + +When ``context_id`` is set, all ``lua_func`` items that share the same identifier +reuse the same persistent subprocess. This allows Lua-side state (upvalues, module +cache) to be retained across calls beyond what ``tm.setgd`` persists. + +.. code-block:: yaml + :caption: ``lua_func`` items sharing a persistent subprocess + + - lua_func: + name: produce value + file: my_script.lua + func_name: produce + context_id: my_context + param: + - hello + + - lua_func: + name: consume value + file: my_script.lua + func_name: consume + context_id: my_context + expected_result: hello + +The shared subprocess is automatically stopped at the end of the test run. **Lua Interpreter environment setup** diff --git a/doc/manual/sphinx/source/test_items/py_func_test_item.rst b/doc/manual/sphinx/source/test_items/py_func_test_item.rst index ae0c120..71e325e 100644 --- a/doc/manual/sphinx/source/test_items/py_func_test_item.rst +++ b/doc/manual/sphinx/source/test_items/py_func_test_item.rst @@ -89,6 +89,9 @@ some of which being mandatory. * ``param``: This is a list of parameters that are passed to the function in the order they are presented in the script. These parameters are not mandatory and are highly dependent of the function prototype. +* ``context_id``: Optional. When set, all ``py_func`` items sharing the same + ``context_id`` value run inside the same persistent Python subprocess for the + duration of the test. See :ref:`py_func context` for details. .. code-block:: yaml :caption: ``py_func`` test item example of usage @@ -111,6 +114,86 @@ value of the funcToBeExecuted python function. The ``py_func`` will always result ``PASS``, except if the called function raises and exception or if the ``expected_result`` attribute is used. +.. _sec_py_func_context: + +Sharing state between ``py_func`` calls +------------------------------------------ + +Each ``py_func`` item without a ``context_id`` runs in a dedicated subprocess that +is started and stopped around the call. State cannot be shared between two such +items using module-level variables. + +Two mechanisms are available to share data across calls: + +**Using the testium global dictionary** + +Inside a ``py_func`` script, the ``tm`` module exposes ``tm.setgd`` and ``tm.gd`` +to read and write the testium global dictionary of the test process. Values stored +this way are accessible from any subsequent test item (including other ``py_func`` +items) without requiring a shared subprocess. + +.. code-block:: python + :caption: sharing a serializable value via the global dictionary + + import py_func.tm as tm + + def produce(val): + tm.setgd("my_shared_value", val) + return val + + def consume(): + return tm.gd("my_shared_value", None) + +Values stored with ``tm.setgd`` must be JSON-serializable (str, int, float, list, +dict, bool, None). Non-serializable values (objects, connections, file handles…) +are handled transparently by the local fallback described below. + +**Using a shared persistent subprocess (``context_id``)** + +When ``context_id`` is set, all ``py_func`` items that share the same identifier +reuse the same subprocess. The subprocess is kept alive until the end of the test. + +This is required for non-JSON-serializable objects (e.g. a socket connection, a +device handle). Calling ``tm.setgd`` with such a value stores it inside the +subprocess local dictionary instead of sending it to the main process. It can then +be retrieved with ``tm.gd`` from any subsequent call that runs in the same subprocess. + +.. code-block:: python + :caption: sharing a non-serializable object via ``context_id`` + + import py_func.tm as tm + + class _Connection: # not JSON-serializable + def __init__(self): + self.value = "open" + + def open_connection(): + tm.setgd("conn", _Connection()) # stored locally in the subprocess + return "ok" + + def use_connection(): + conn = tm.gd("conn") # retrieved from the subprocess local dict + return conn.value + +.. code-block:: yaml + :caption: ``py_func`` items sharing a persistent subprocess + + - py_func: + name: open connection + file: my_script.py + func_name: open_connection + context_id: my_context + expected_result: ok + + - py_func: + name: use connection + file: my_script.py + func_name: use_connection + context_id: my_context + expected_result: open + +The shared subprocess is automatically stopped at the end of the test run. + **Python Interpreter environment setup** Some global variables have an impact on the ``py_func`` test item behavior: diff --git a/doc/manual/testium_manual.pdf b/doc/manual/testium_manual.pdf index fb2ace6..3042d8c 100644 Binary files a/doc/manual/testium_manual.pdf and b/doc/manual/testium_manual.pdf differ diff --git a/src/py_func/tm.py b/src/py_func/tm.py index 2a89096..d84566f 100644 --- a/src/py_func/tm.py +++ b/src/py_func/tm.py @@ -1,19 +1,26 @@ +import json import sys from py_func.handle import FuncHandler from lib.tum_except import ETUMRuntimeError from lib.api import SUPPORTED_API thismodule = sys.modules[__name__] -# Shared FuncHandler instance used to forward API calls. Remains None -# until `_init_api` is invoked. _func_call_thread = None +# Local storage for non-JSON-serializable values +_local_dict = {} + + +def _is_json_serializable(value): + try: + json.dumps(value) + return True + except (TypeError, ValueError): + return False + ############################################################################### -# Dynamically create module-level functions for each supported API name. -# Each generated function shares the implementation of `api_call` but -# has a distinct name used as the remote action identifier. def _make_api(name): def _wrapper(*params): if _func_call_thread is not None: @@ -31,21 +38,62 @@ def _make_api(name): return _wrapper for k in SUPPORTED_API: - setattr(thismodule, k, _make_api(k)) + if k not in ('gd', 'setgd', 'delgd'): + setattr(thismodule, k, _make_api(k)) + + +############################################################################### +# gd/setgd/delgd with local-dict fallback for non-serializable values + +def gd(*params): + key = params[0] if params else None + if key is not None and key in _local_dict: + return _local_dict[key] + if _func_call_thread is not None: + res = _func_call_thread.call("gd", params) + if "result" in res: + return res["result"] + elif "error" in res: + raise ETUMRuntimeError(f"api call to 'tm.gd' failed with error '{res['error']}'") + else: + raise ETUMRuntimeError("api call failure in jrpc client to be reported to testium support team.") + raise ETUMRuntimeError("api not initialized") + + +def setgd(*params): + key = params[0] if params else None + value = params[1] if len(params) > 1 else None + if key is not None and not _is_json_serializable(value): + _local_dict[key] = value + return None + if _func_call_thread is not None: + res = _func_call_thread.call("setgd", params) + if "result" in res: + return res["result"] + elif "error" in res: + raise ETUMRuntimeError(f"api call to 'tm.setgd' failed with error '{res['error']}'") + else: + raise ETUMRuntimeError("api call failure in jrpc client to be reported to testium support team.") + raise ETUMRuntimeError("api not initialized") + + +def delgd(*params): + key = params[0] if params else None + if key is not None and key in _local_dict: + del _local_dict[key] + return None + if _func_call_thread is not None: + res = _func_call_thread.call("delgd", params) + if "result" in res: + return res["result"] + elif "error" in res: + raise ETUMRuntimeError(f"api call to 'tm.delgd' failed with error '{res['error']}'") + else: + raise ETUMRuntimeError("api call failure in jrpc client to be reported to testium support team.") + raise ETUMRuntimeError("api not initialized") + def _init_api(host, port, timeout): - """Start and initialize the remote function handler. - - Starts a ``FuncHandler`` bound to ``port``, runs it and blocks until - it signals readiness. - - Args: - port: port number or identifier passed to ``FuncHandler``. - - Returns: - The initialized ``FuncHandler`` instance assigned to - ``_func_call_thread``. - """ global _func_call_thread _func_call_thread = FuncHandler(host, port, timeout=timeout) return _func_call_thread @@ -53,17 +101,10 @@ def _init_api(host, port, timeout): ############################################################################### def _remote_print(*values): - """Forward print-like output to the remote handler. - - If a ``_func_call_thread`` is available, this function calls the - handler with action name ``"print"`` and the provided values. Errors - during forwarding are ignored because printing is best-effort. - """ if _func_call_thread is not None: try: _func_call_thread.call("print", values) except: - # Best-effort: ignore forwarding failures pass diff --git a/src/testium/interpreter/process.py b/src/testium/interpreter/process.py index e858a4d..e17ace7 100644 --- a/src/testium/interpreter/process.py +++ b/src/testium/interpreter/process.py @@ -275,6 +275,13 @@ Is the python exec path correct ?""" # Stop python eval execution process eval_proc.stop() eval_proc.join() + # Stop shared func context engines (keep_context_id) + for engine in tm.gd("_py_func_contexts", {}).values(): + engine.stop() + engine.join() + for engine in tm.gd("_lua_func_contexts", {}).values(): + engine.stop() + engine.join() except Exception as e: print_exception(e) diff --git a/src/testium/interpreter/test_items/test_item_lua_func.py b/src/testium/interpreter/test_items/test_item_lua_func.py index a3da782..078362f 100644 --- a/src/testium/interpreter/test_items/test_item_lua_func.py +++ b/src/testium/interpreter/test_items/test_item_lua_func.py @@ -12,10 +12,13 @@ from interpreter.utils.lua_func_exec import LuaFuncExecEngine from interpreter.utils.api_srv import api_request from interpreter.utils.constants import TestItemType as cst +_LUA_FUNC_CONTEXTS_KEY = "_lua_func_contexts" + class TestItemLuaFunc(TestItem): """lua_func item usage. func file: func_file.lua, func_name: func, param: [$(variable1), [1, 2, 3], true] + Optional: context_id: — share a persistent process with other lua_func items using the same id. """ def __init__(self, dict_item, parent=None, status_queue=None, filename=""): @@ -27,14 +30,26 @@ class TestItemLuaFunc(TestItem): self.file_name = self._prms.getParam("file", required=True) self.func_name = self._prms.getParam("func_name", required=True) self.params = self._prms.getParamAll("param") + self._context_id = self._prms.getParam("context_id", default=None, processed=False) except: raise ETUMSyntaxError( f"The '{self.cmd()}' test item named '{self.name()}' (child of '{self.parent.name()}') has a missing or wrong parameter", self.seqFilename(), ) - # Lua functions call subprocess initialization self._lua_func_proc = LuaFuncExecEngine(tm.gd("lua_bin", ""), api_request, 10) + def _get_engine(self): + """Return (engine, persistent). If context_id is set, use a shared persistent engine.""" + if self._context_id is None: + return self._lua_func_proc, False + + ctx_id = self._prms.expanse(self._context_id) + contexts = tm.gd(_LUA_FUNC_CONTEXTS_KEY, {}) + if ctx_id not in contexts: + contexts[ctx_id] = LuaFuncExecEngine(tm.gd("lua_bin", ""), api_request, 10) + tm.setgd(_LUA_FUNC_CONTEXTS_KEY, contexts) + return contexts[ctx_id], True + @test_run def execute(self): self.result.set( @@ -48,22 +63,25 @@ class TestItemLuaFunc(TestItem): print("Parameters list:") print(textwrap.indent(pprint.pformat(pl), " |")) - self._lua_func_proc.start() - if not self._lua_func_proc.wait_ready(10): - raise ETUMRuntimeError( - f"""Impossible to start the external lua execution process. + engine, persistent = self._get_engine() + + if not engine.is_alive(): + engine.start() + if not engine.wait_ready(10): + raise ETUMRuntimeError( + f"""Impossible to start the external lua execution process. Is the lua path correct ? lua_bin = {tm.gd("lua_bin", "no lua path defined")} Are "lua-sockets" and "lua-cjson" installed ? Is the lua environnment well defined in the "LUA_PATH" and "LUA_CPATH" variables ?""" - ) + ) try: - success, ret = self._lua_func_proc.func_call(self.file_name, self.func_name, pl) + success, ret = engine.func_call(self.file_name, self.func_name, pl) finally: - # Stops lua function execution process - self._lua_func_proc.stop() - self._lua_func_proc.join() + if not persistent: + engine.stop() + engine.join() if success == TestValue.SUCCESS: self.result.set(TestValue.SUCCESS) @@ -73,7 +91,6 @@ Is the lua environnment well defined in the "LUA_PATH" and "LUA_CPATH" variables print("Returned value:") print(textwrap.indent(pprint.pformat(res), " |")) - # The result of the func test item is put in global dir and result tm.setgd("lfn_" + self._name, res) self.result.value = res @@ -88,5 +105,5 @@ Is the lua environnment well defined in the "LUA_PATH" and "LUA_CPATH" variables traceback.print_exception(*sys.exc_info()) self.result.set( TestValue.FAILURE, - 'Unrecoverable "py_func" item error from {}'.format(self.func_name), + 'Unrecoverable "lua_func" item error from {}'.format(self.func_name), ) diff --git a/src/testium/interpreter/test_items/test_item_py_func.py b/src/testium/interpreter/test_items/test_item_py_func.py index d4f1c2b..1d8ffee 100644 --- a/src/testium/interpreter/test_items/test_item_py_func.py +++ b/src/testium/interpreter/test_items/test_item_py_func.py @@ -12,10 +12,13 @@ from interpreter.utils.py_func_exec import PyFuncExecEngine from interpreter.utils.api_srv import api_request from interpreter.utils.constants import TestItemType as cst +_PY_FUNC_CONTEXTS_KEY = "_py_func_contexts" + class TestItemPyFunc(TestItem): """py_func item usage. func file: func_file.py, func_name: func, param: [$(variable1), [1, 2, 3], true] + Optional: context_id: — share a persistent process with other py_func items using the same id. """ def __init__(self, dict_item, parent=None, status_queue=None, filename=""): @@ -27,6 +30,7 @@ class TestItemPyFunc(TestItem): self.file_name = self._prms.getParam("file", required=True) self.func_name = self._prms.getParam("func_name", required=True) self.params = self._prms.getParamAll("param") + self._context_id = self._prms.getParam("context_id", default=None, processed=False) except: raise ETUMSyntaxError( f"The '{self.cmd()}' test item named '{self.name()}' (child of '{self.parent.name()}') has a missing or wrong parameter", @@ -34,6 +38,18 @@ class TestItemPyFunc(TestItem): ) self._py_func_proc = PyFuncExecEngine(tm.gd("python_bin", ""), api_request, 10) + def _get_engine(self): + """Return (engine, persistent). If context_id is set, use a shared persistent engine.""" + if self._context_id is None: + return self._py_func_proc, False + + ctx_id = self._prms.expanse(self._context_id) + contexts = tm.gd(_PY_FUNC_CONTEXTS_KEY, {}) + if ctx_id not in contexts: + contexts[ctx_id] = PyFuncExecEngine(tm.gd("python_bin", ""), api_request, 10) + tm.setgd(_PY_FUNC_CONTEXTS_KEY, contexts) + return contexts[ctx_id], True + @test_run def execute(self): self.result.set( @@ -47,20 +63,23 @@ class TestItemPyFunc(TestItem): print("Parameters list:") print(textwrap.indent(pprint.pformat(pl), " |")) - # start the process for executing external python - self._py_func_proc.start() - if not self._py_func_proc.wait_ready(): - raise ETUMRuntimeError( - f"""Impossible to start the external python execution process. + engine, persistent = self._get_engine() + + if not engine.is_alive(): + engine.start() + if not engine.wait_ready(): + raise ETUMRuntimeError( + f"""Impossible to start the external python execution process. Is the python path correct ? python_bin = {tm.gd("python_bin", "no python path defined")}""" - ) + ) + try: - success, ret = self._py_func_proc.func_call(self.file_name, self.func_name, pl) + success, ret = engine.func_call(self.file_name, self.func_name, pl) finally: - # Stops python function execution process - self._py_func_proc.stop() - self._py_func_proc.join() + if not persistent: + engine.stop() + engine.join() if success == TestValue.SUCCESS: self.result.set(TestValue.SUCCESS) @@ -70,7 +89,6 @@ python_bin = {tm.gd("python_bin", "no python path defined")}""" print("Returned value:") print(textwrap.indent(pprint.pformat(res), " |")) - # The result of the func test item is put in global dir and result tm.setgd("pfn_" + self._name, res) self.result.value = res diff --git a/src/testium/interpreter/utils/lua_process.py b/src/testium/interpreter/utils/lua_process.py index 516a210..2ccc771 100644 --- a/src/testium/interpreter/utils/lua_process.py +++ b/src/testium/interpreter/utils/lua_process.py @@ -221,6 +221,11 @@ class LuaProcessBase: return self._rpc.wait_ready(timeout) return False + def is_alive(self): + if self._rpc is not None: + return self._rpc.is_alive() + return False + def stop(self): """ Stops the RPC client. diff --git a/test/validation/items/lua_func/lua_func.lua b/test/validation/items/lua_func/lua_func.lua index 6f043d7..d1577b7 100644 --- a/test/validation/items/lua_func/lua_func.lua +++ b/test/validation/items/lua_func/lua_func.lua @@ -32,5 +32,13 @@ function module.tuple_return(first, second) return first, second end +function module.set_context_value(val) + tm.setgd("_lua_ctx_test_value", val) + return val +end + +function module.get_context_value() + return tm.gd("_lua_ctx_test_value") +end return module \ No newline at end of file diff --git a/test/validation/items/lua_func/test.tum b/test/validation/items/lua_func/test.tum index a673402..d8e617b 100644 --- a/test/validation/items/lua_func/test.tum +++ b/test/validation/items/lua_func/test.tum @@ -179,3 +179,36 @@ file: $(test_path)$(psep)lua_func.lua func_name: tuple_return param: [ 0, "OK" ] + +- group: + name: context_id tests + steps: + - lua_func: + name: set context value + key: $(test)_PASS + file: $(test_path)$(psep)lua_func.lua + func_name: set_context_value + context_id: lua_ctx_test + param: + - hello lua + expected_result: hello lua + - lua_func: + name: get context value (same context_id) + key: $(test)_PASS + file: $(test_path)$(psep)lua_func.lua + func_name: get_context_value + context_id: lua_ctx_test + expected_result: hello lua + - lua_func: + name: get context value (no context_id, from main gd) + key: $(test)_PASS + file: $(test_path)$(psep)lua_func.lua + func_name: get_context_value + expected_result: hello lua + - lua_func: + name: get context value (different context_id) + key: $(test)_PASS + file: $(test_path)$(psep)lua_func.lua + func_name: get_context_value + context_id: lua_ctx_other + expected_result: hello lua diff --git a/test/validation/items/py_func/py_func.py b/test/validation/items/py_func/py_func.py index 67c78d6..d30232a 100644 --- a/test/validation/items/py_func/py_func.py +++ b/test/validation/items/py_func/py_func.py @@ -27,3 +27,23 @@ def echo(param): def tuple_return(first, second): return first, second + +def set_context_value(val): + tm.setgd("_py_ctx_test_value", val) + return val + +def get_context_value(): + return tm.gd("_py_ctx_test_value", None) + + +class _NotSerializable: + def __init__(self, val): + self.val = val + +def set_ns_value(val): + tm.setgd("_py_ctx_ns_value", _NotSerializable(val)) + return val + +def get_ns_value(): + obj = tm.gd("_py_ctx_ns_value", None) + return obj.val if obj is not None else None diff --git a/test/validation/items/py_func/test.tum b/test/validation/items/py_func/test.tum index 6790867..67cd828 100644 --- a/test/validation/items/py_func/test.tum +++ b/test/validation/items/py_func/test.tum @@ -189,3 +189,51 @@ func_name: tuple_return param: [ 0, "OK" ] expected_result: [0, "OK"] + +- group: + name: context_id tests + steps: + - py_func: + name: set serializable value + key: $(test)_PASS + file: $(test_path)$(psep)py_func.py + func_name: set_context_value + param: + - hello context + expected_result: hello context + - py_func: + name: get serializable value (same context_id) + key: $(test)_PASS + file: $(test_path)$(psep)py_func.py + func_name: get_context_value + context_id: ctx_test + expected_result: hello context + - py_func: + name: get serializable value (no context_id, from main gd) + key: $(test)_PASS + file: $(test_path)$(psep)py_func.py + func_name: get_context_value + expected_result: hello context + - py_func: + name: get serializable value (different context_id) + key: $(test)_PASS + file: $(test_path)$(psep)py_func.py + func_name: get_context_value + context_id: ctx_other + expected_result: hello context + - py_func: + name: set non-serializable value + key: $(test)_PASS + file: $(test_path)$(psep)py_func.py + func_name: set_ns_value + context_id: ctx_ns_test + param: + - hello ns + expected_result: hello ns + - py_func: + name: get non-serializable value (same context_id) + key: $(test)_PASS + file: $(test_path)$(psep)py_func.py + func_name: get_ns_value + context_id: ctx_ns_test + expected_result: hello ns