v-and-v/testium
2
0
Fork 0
Code Issues 5 Pull Requests 1 Actions Packages Projects Releases 7 Wiki Activity

docs: refocus README on users, add quick_start + tutorial, fill CONTRIBUTING

Browse Source 
- README.md: pruned developer-oriented sections (Sphinx setup, Qt
  Creator workflow, VSCode debugging, release procedure, AppImage
  Wayland note) and replaced them with a user-facing layout: pre-built
  releases pointer, quick start, manual install, troubleshooting,
  licence.
- CONTRIBUTING.md: absorbed the developer content (debugging in VSCode,
  Qt GUI regen, Sphinx build, validation suite — batch + GUI variants,
  cross-distrib check, release procedure).
- doc/quick_start.md: 5-minute path from install to a passing test,
  in batch mode and in the GUI.
- doc/tutorial.md: guided walk-through against a small calc.py
  module — check, py_func, expected_result, $(...) expansion, group,
  let, condition, report (with the mkdir reminder), context_id.
- CLAUDE.md: subprocess API contract, bins.py, report-exporter
  plugin section, packaging matrix (wheel / PyInstaller / Flatpak /
  .deb work-in-progress), refreshed recent-fixes list. README/CLAUDE
  validation command no longer carries the spurious "-l" flag (which
  is GUI-only and a no-op in batch).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
Foué
2026-05-05 09:18:59 +02:00
parent d3c5bd01e5
commit b9475c6e9b
5 changed files with 494 additions and 181 deletions
Download Patch File Download Diff File Expand all files Collapse all files

66
doc/quick_start.md Normal file
View File

@@ -0,0 +1,66 @@
# Quick start
Five minutes from zero to a passing test.
## Install
From a checkout of the repository:
```sh
./run.sh --version # Linux
run.bat # Windows cmd
```
The wrapper creates a Python virtual environment on first run and verifies
testium starts. If you prefer a manual install, see the README.
## Your first test
Create `hello.tum`:
```yaml
main:
name: hello world
steps:
- check:
name: 1 + 1 makes 2
values:
- <| 1 + 1 == 2 |>
```
Run it in batch mode:
```sh
./run.sh -b -- hello.tum
```
You should see something like:
```
-----> step "1 + 1 makes 2" started
Check passed
<----- step "1 + 1 makes 2" finished: PASS
Test run success.
```
Replace `==` with `!=` and re-run — the step now ends with **FAIL** and
the process exits with code 1.
## Open it in the GUI
```sh
./run.sh hello.tum
```
The test tree appears in the left panel; click *Run test* in the toolbar.
Each item turns green or red live as it executes. Use `F1` on a selected
item to open its detail panel.
## Where to go next
* [`doc/tutorial.md`](tutorial.md) — a guided walk-through of the most
common test items (`py_func`, `let`, `group`, `condition`, `report`).
* [`doc/examples/`](examples/) — runnable `.tum` snippets covering one
feature each.
* [`doc/manual/testium_manual.pdf`](manual/testium_manual.pdf) —
full reference manual.

223
doc/tutorial.md Normal file
View File

@@ -0,0 +1,223 @@
# Tutorial — testing a small Python utility
This walk-through builds, step by step, a testium campaign that exercises
a small Python module. Each section adds one feature; you can follow
along by editing a single `.tum` file and re-running it.
If you have not yet run testium, start with [`quick_start.md`](quick_start.md).
## The code under test
Create `calc.py` next to your `.tum` file:
```python
def add(a, b):
return a + b
def divide(a, b):
return a / b
```
## Step 1 — a static check
The simplest item is `check`: it evaluates an expression and the test
passes iff the expression is truthy. Create `tutorial.tum`:
```yaml
main:
name: calc.py campaign
steps:
- check:
name: addition is correct
values:
- <| 2 + 3 == 5 |>
```
The `<| ... |>` markers turn the body into a Python expression evaluated
at run time. Run it:
```sh
./run.sh -b -- tutorial.tum
```
## Step 2 — call your code with `py_func`
`check` only sees Python literals; to exercise `calc.py` we need a
`py_func` item. Replace the step:
```yaml
- py_func:
name: add 2 and 3
file: calc.py
func_name: add
param: [2, 3]
expected_result: 5
```
`expected_result` makes the item PASS only when the function returns
exactly that value.
The result is also stored in the global dict under `pfn_<name>`
(here `pfn_add 2 and 3`).
Anywhere in a `.tum`, `$(key)` is replaced at runtime by the value
stored in the global dict under `key`. A subsequent step can read the
result back with `$(pfn_<name>)`:
```yaml
- check:
name: result was 5
values:
- <| $(pfn_add 2 and 3) == 5 |>
```
## Step 3 — group several checks
Wrap the steps in a `group` to keep them visually together and let
testium report a per-group status:
```yaml
main:
name: calc.py campaign
steps:
- group:
name: add
steps:
- py_func:
name: 2 + 3
file: calc.py
func_name: add
param: [2, 3]
expected_result: 5
- py_func:
name: -1 + 1
file: calc.py
func_name: add
param: [-1, 1]
expected_result: 0
- group:
name: divide
steps:
- py_func:
name: 6 / 2
file: calc.py
func_name: divide
param: [6, 2]
expected_result: 3.0
```
A group fails as soon as one of its steps fails (set
`stop_on_failure: false` to keep going).
## Step 4 — define a variable with `let`
Avoid hard-coding the same number twice with a variable:
```yaml
- let:
name: define numerator
values:
- num: 6
- py_func:
name: divide num by 2
file: calc.py
func_name: divide
param:
- $(num)
- 2
expected_result: 3.0
```
`$(num)` expands to the global dict entry — when the stored value is a
number it is substituted as a number, no need to wrap it in `<| ... |>`.
## Step 5 — conditional execution
Skip a step when a condition is false:
```yaml
- py_func:
name: divide by zero only on linux
condition: <| "$(os)" == "Linux" |>
file: calc.py
func_name: divide
param: [1, 0]
```
Items skipped this way report `SKIP` and do not affect the overall
result.
## Step 6 — generate a report
Add a `report` block at the root of the file:
```yaml
main:
name: calc.py campaign
steps:
# ... your steps here ...
report:
enabled: true
log_stored: true
export:
- junit:
path: ./reports
file_name: calc.xml
- html:
path: ./reports
file_name: calc.html
```
The `path` directory must exist before the test runs — testium does not
create it. Create it once:
```sh
mkdir -p reports
```
Re-run the test — `./reports/calc.xml` (CI-friendly) and
`./reports/calc.html` (human-friendly) are produced. Set
`log_stored: true` to include each item's captured stdout.
## Step 7 — share state between calls
By default each `py_func` runs in its own short-lived subprocess.
To keep state across calls, use `context_id`:
```yaml
- py_func:
name: open
file: calc.py
func_name: open_resource
context_id: my_ctx
- py_func:
name: use
file: calc.py
func_name: use_resource
context_id: my_ctx
```
Both steps share the same persistent Python interpreter, so `calc.py`
can store any object in module-level globals or in `tm.setgd()`.
To share data without `context_id`, write it to the testium global dict
via the JSON-RPC bridge:
```python
import py_func.tm as tm
def producer():
tm.setgd("computed", 42)
def consumer():
return tm.gd("computed")
```
## Where to go next
* [`doc/examples/`](examples/) — one runnable `.tum` per feature
(cycles, dialogs, console, plots, parallel, run-of-tum, …).
* [`doc/manual/testium_manual.pdf`](manual/testium_manual.pdf) — full
reference manual covering every test item, every attribute and the
YAML syntax extensions.