v-and-v/testium
2
0
Fork 0
Code Issues 5 Pull Requests 1 Actions Packages Projects Releases 7 Wiki Activity

lua and python bin detection rationalized: bins.py module created.

Browse Source 
Added some api accessible from python and lua sub_processes. Now the tests only access to py_func.tm instead of direct api.testium module access.

Corrected some f"xxx" to allow working with old python (bookworm).

Changed param.yaml of the test to allow lua to work in all situations.

Various other small fixes for frozen app, wheel.

Tested in all situations, and OK. Ready for tag !

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
Foué
2026-05-03 10:16:56 +02:00
parent 077e1a97c1
commit d3c5bd01e5
30 changed files with 585 additions and 312 deletions
Download Patch File Download Diff File Expand all files Collapse all files

53
doc/manual/sphinx/source/helper_lib.rst
View File

@@ -4,7 +4,12 @@ Python helper library
======================
A python library including helper function for python modules called from
testium.
testium ``py_func`` items.
User scripts run inside the ``py_func`` subprocess and interact with testium
through a JSON-RPC bridge — the ``py_func.tm`` module. They must **not**
import ``api.testium`` or ``interpreter.*`` directly: those are main-process
modules and may not even be reachable in a packaged build (PyInstaller, .deb).
To include the support of this library in a python script, the following
line must be included in the script header:
@@ -18,58 +23,38 @@ line must be included in the script header:
Global variables helper functions
----------------------------------
To manage values in the global variables dataset, the following testium library API
must be used:
To manage values in the global variables dataset:
.. automodule:: py_func.tm
:members: gd, setgd, delgd
:undoc-members:
:no-index:
Console helper functions
------------------------
Every opened console instance is added to a list with the
key ``console_instances`` of the global variables.
The instance is removed from the list on close step of the ``console`` test item.
To manage consoles from within ``py_func`` python functions,
the following testium library API can be used:
.. automodule:: libs.testium
:members: add_console, remove_console, console
:undoc-members:
:no-index:
Plot helper functions
------------------------
Every opened plot window instance is added to a list with the
key ``plot_instances`` of the global variables.
Add values to a running plot or read the last value from it:
The instance is removed from the list on close step of the ``plot`` test item.
To manage plots from within ``py_func`` python functions,
the following testium library API can be used:
.. automodule:: libs.testium
:members: add_plot, remove_plot, plot, add_plot_values, last_plot_value
.. automodule:: py_func.tm
:members: add_plot_values, last_plot_value
:undoc-members:
:no-index:
Console and plot **lifecycle** management (``add_console``, ``remove_console``,
``console``, ``add_plot``, ``remove_plot``, ``plot``) is performed by the
``console`` and ``plot`` test items themselves — not from user ``py_func``
scripts. Use those test items to open/close consoles and plots.
Other helper functions
------------------------
.. automodule:: libs.testium
:members: OS, get_main_dir, timestamp, timestamp_as_sec
.. automodule:: py_func.tm
:members: OS, get_main_dir, init_timestamp, timestamp, timestamp_as_sec, text_mode
:undoc-members:
:no-index:
Debug mode
------------------------
.. automodule:: libs.testium
:members: debug_enabled, enable_debug, print_debug, print_info, print_warn
:undoc-members:
:no-index:
The ``test_debug`` global variable controls debug-only output. Read or write
it via ``tm.gd("test_debug")`` / ``tm.setgd("test_debug", True)``.

119
doc/manual/sphinx/source/reports.rst
View File

@@ -6,18 +6,25 @@ Reports
If a report is required (in addition to the log), the ``report`` YAML element
must be added at the root of the TUM main test file.
The ``report`` YAML element has the following form:
The ``report`` element accepts a single export or a list of them under the
``export`` key. Each export entry uses the format name as its key:
.. code-block:: yaml
:caption: reports global settings
:caption: reports global settings — multiple exports
report:
enabled: True
file_name: $(test_name).rep
path: $(home)/reports
pattern: "Console%"
export: junit
log_stored: False
log_stored: True
export:
- sqlite:
path: $(home)/reports
file_name: $(test_name).db
- junit:
path: $(home)/reports
file_name: $(test_name).xml
- html:
path: $(home)/reports
file_name: $(test_name).html
.. table:: report attributes
:widths: 20, 30, 50
@@ -27,21 +34,93 @@ The ``report`` YAML element has the following form:
+-----------------+-----------------------+-------------------------------------------+
| ``enabled`` | ``True`` | Report activated |
+-----------------+-----------------------+-------------------------------------------+
| ``file_name`` | / | Report file name |
| ``log_stored`` | ``False`` | When ``True``, captures stdout per test |
| | | item so exports (html, json) can include |
| | | the log of each item. |
+-----------------+-----------------------+-------------------------------------------+
| ``path`` | ``$(report_path)`` | Report storage path By default, it uses |
| | | the default one set in the |
| | | preferences. |
| ``export`` | / | One export entry or a list of them. Each |
| | | entry's key is the format name (see |
| | | below). |
+-----------------+-----------------------+-------------------------------------------+
| ``pattern`` | / | The pattern in SQL wildachars syntax |
| | | to be applied on test names to |
| | | selected reported tests. |
Each export entry supports the following sub-attributes:
.. table:: export attributes
:widths: 20, 30, 50
+-----------------+-----------------------+-------------------------------------------+
| ``export`` | / | The type of export. For exemple junit. |
| | | By default, the sqlite format is |
| | | used to generate reports. |
| Attribute | default value | Description |
+-----------------+-----------------------+-------------------------------------------+
| ``log_stored`` | / | Defines if the output log of each |
| | | test is accessible to generate the |
| | | report export. |
| ``path`` | ``$(report_path)`` | Output directory. |
+-----------------+-----------------------+-------------------------------------------+
| ``file_name`` | / | Output file name. May include |
| | | ``$(...)`` global-dict expansions. |
+-----------------+-----------------------+-------------------------------------------+
| ``pattern`` | / | One or more SQL ``LIKE`` patterns |
| | | applied on the test ``name``. |
+-----------------+-----------------------+-------------------------------------------+
| ``key`` | / | One or more SQL ``LIKE`` patterns |
| | | applied on the test ``key`` |
| | | (the per-item ``key`` attribute). |
+-----------------+-----------------------+-------------------------------------------+
Built-in formats
^^^^^^^^^^^^^^^^
* ``sqlite`` — raw SQLite database (storage layer; selecting it persists the run).
* ``text`` — simple indented text dump of the test tree.
* ``json`` — full report as JSON: ``{"header": {...}, "tests": [...]}``.
* ``junit`` — JUnit XML (requires the ``junit_xml`` Python package).
* ``html`` — single HTML page with header, results table and per-item logs (requires ``lxml``).
If a format is unknown or its optional dependency is missing, the export is
skipped with an ``[report] Export skipped: ...`` info line on stdout — the
test run is **not** interrupted.
.. _sec_reports_plugins:
Custom export formats (plugins)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
A third-party Python package can register additional export formats via the
``testium.exporters`` setuptools entry point group. Once installed in the same
Python environment as testium, the format is auto-detected at startup and can
be referenced from the YAML by its declared name.
Plugin contract — a class with this constructor signature:
.. code-block:: python
:caption: minimal exporter contract
class MyExporter:
def __init__(self, name, con, path, pats, keys, no_header=False):
# name : str — report name
# con : sqlite3.Connection (read) — tables: header, tests
# path : str — output file path (already expansed)
# pats : list[str] — LIKE filters on test_name (may be empty)
# keys : list[str] — LIKE filters on report_key (may be empty)
# no_header : bool — skip header section (set by the inline
# `report` test item)
... # do the work in __init__ and write to `path`
Tables and columns of the SQLite report:
* ``header(key TEXT, value TEXT)`` — keys: ``report_version``, ``test_file``,
``test_name``, ``test_result``, ``test_revision``, ``testium_version``,
``testrun_date``, ``testrun_time``, ``test_duration``.
* ``tests`` — 12 columns: ``timestamp_start``, ``test_id``, ``parent_id``,
``level``, ``test_name``, ``test_type``, ``report_key``, ``result``
(``PASS``/``FAIL``/``SKIP``), ``message``, ``duration`` (ms),
``log`` (captured stdout when ``log_stored: True``), ``data`` (JSON of
values reported via ``self.reportValue(...)``).
Declaration in the plugin's ``pyproject.toml``:
.. code-block:: toml
:caption: registering an exporter via entry-points
[project.entry-points."testium.exporters"]
my_format = "my_pkg:MyExporter"
The plugin is then usable in any ``.tum`` report block as ``my_format:`` —
no testium configuration change required.

2
doc/manual/sphinx/source/test_items/py_func_test_item.rst
View File

@@ -13,7 +13,7 @@ class ``py_func`` item
This is the normal way of calling some custom python code.
A class must be defined and derived from ``FunctionItem`` from the ``libs.testium`` module.
A class must be defined and derived from ``FunctionItem`` from the ``py_func.tm`` module.
From this class it is possible to define some custom reported values with the following API

BIN
doc/manual/testium_manual.pdf
View File

Binary file not shown.