Compare commits
3 Commits
refactor/i
...
a01268cd0e
| Author | SHA1 | Date | |
|---|---|---|---|
| a01268cd0e | |||
| e47d422655 | |||
| 2d44f52e96 |
47
DESIGN.md
47
DESIGN.md
@@ -226,25 +226,44 @@ The `.deb` work-in-progress lives in `package/deb/`:
|
||||
|
||||
### Host-only py_func / lua_func in sandboxed bundles (Flatpak, AppImage)
|
||||
|
||||
The bundled Python (Flatpak's runtime python, AppImage's `python3.11`) is reserved for the **main process only**. Subprocesses (`py_func`, `lua_func`, `git`) must use the host's interpreters and tools so user-installed modules (pyserial, junit_xml, …) are visible. This is enforced by `interpreter/utils/bins.py`:
|
||||
The bundled Python (Flatpak's runtime python, AppImage's `python3.11`) is reserved for the **main process only**. Subprocesses (`py_func`, `lua_func`, `git`, the `run` item's sub-instance) must use the host's interpreters and tools so user-installed modules (pyserial, junit_xml, …) are visible. This is enforced by `interpreter/utils/bins.py`:
|
||||
|
||||
- `_in_flatpak()` (checks `/.flatpak-info`) and `_in_appimage()` (checks `APPIMAGE` env var) detect the sandbox.
|
||||
- `_which(name)` probes only host bin dirs in those modes:
|
||||
- Flatpak: `/run/host/usr/{local/,}bin`, `/run/host/bin` (host mounted via `--filesystem=host-os`).
|
||||
- AppImage: `/usr/local/bin`, `/usr/bin`, `/bin` (we are directly on the host filesystem).
|
||||
- If the host has no python3/lua, `ensure()` raises `ETUMRuntimeError` at test load with the candidate list — no silent fallback to a bundled interpreter.
|
||||
- User overrides (`python_bin`/`lua_bin` in globdict): bare names are resolved through `_which()` (host-only), absolute paths are accepted as-is.
|
||||
- `apply_host_libs(env)` is called by `py_process.py` / `lua_process.py` on the env passed to Popen:
|
||||
- Flatpak: prepends host lib dirs to `LD_LIBRARY_PATH` so the dynamic linker finds host `.so`'s.
|
||||
- AppImage: strips `$APPDIR`-prefixed entries from `LD_LIBRARY_PATH` / `PYTHONPATH` / `PATH` and drops `PYTHONHOME`, so the host Python doesn't try to load the bundled stdlib/site-packages.
|
||||
- `apply_host_lua_paths(env)` (Flatpak only) prepends `/run/host/usr/{lib,share}/lua/X.Y` to `LUA_PATH` / `LUA_CPATH` so `cjson`, `socket`, etc. resolve. Must be called **after** user `lua_env` overrides so host paths win. AppImage relies on host Lua's compiled-in defaults.
|
||||
- **Flatpak**: the sandbox glibc/ABI is incompatible with arbitrary host shared libraries, so we **cannot** run host binaries inside the Flatpak runtime — `LD_LIBRARY_PATH` injection trips a `_dl_call_libc_early_init` assertion. The supported way out is `flatpak-spawn --host`, a stub on `$PATH` inside every Flatpak that proxies an `exec` over D-Bus to the host's `org.freedesktop.Flatpak` service. The manifest grants `--talk-name=org.freedesktop.Flatpak` so the call is allowed. Helpers:
|
||||
- `flatpak_host_spawn(interp, args, host_cwd, extra_env=…)` builds the spawn command vector with a curated set of forwarded env vars (`HOME`, `USER`, `DISPLAY`, `DBUS_SESSION_BUS_ADDRESS`, …) plus any explicit overrides.
|
||||
- `_get_host_testium_path()` returns a path to the testium package the host can read. In Flatpak the package lives under `/app/lib/testium` which the host cannot see, so the package is staged once per process under `/tmp/testium_host_*` (`/tmp` is shared) and reused. In source / wheel / PyInstaller installs under `$HOME` the original path is returned untouched.
|
||||
- `_which_host_flatpak(name)` resolves a binary by spawning `command -v` on the host (or `test -x` for absolute paths) — sandbox-visible probing under `/run/host/...` is unreliable (only `host-os` is mounted; user paths like `/scratch` aren't there).
|
||||
- `_python_version()` and `_lua_version()` go through `_run_probe()` which dispatches to `flatpak-spawn` in Flatpak so validation happens against the actual host interpreter.
|
||||
- `py_process.py` / `lua_process.py` `start()` use `flatpak_host_spawn` with `host_cwd = _get_host_testium_path()[+/lua_func]` and forward `PYTHONPATH` / `LUA_PATH` / `LUA_CPATH` / `PATH` as `--env=` arguments.
|
||||
- The `run` item's `_testium_launch_cmd()` prefixes `flatpak run org.testium.Testium` with `flatpak-spawn --host` so the sub-instance is launched by the host's `flatpak` CLI, not by an unworkable in-sandbox `flatpak` binary.
|
||||
- **AppImage**: we are directly on the host filesystem, so the regular discovery on `/usr/local/bin`, `/usr/bin`, `/bin` suffices. `apply_host_libs(env)` strips `$APPDIR`-prefixed entries from `LD_LIBRARY_PATH` / `PYTHONPATH` / `PATH` and drops `PYTHONHOME` so the host Python doesn't try to load the bundled stdlib/site-packages.
|
||||
- User overrides (`python_bin`/`lua_bin` in globdict): in Flatpak, both bare names and absolute paths go through `_which()` so they are validated on the host side (the sandbox can't see e.g. `/scratch/...`). Outside Flatpak, absolute paths are accepted as-is and bare names go through PATH discovery.
|
||||
- If the host has no python3/lua, `ensure()` raises `ETUMRuntimeError` at test load with the candidate list — no silent fallback to a bundled interpreter.
|
||||
- `py_process.py` additionally pops `PYTHONUSERBASE` (set to `/var/data/python` by the Flatpak runtime, which would hide `~/.local/lib/...`).
|
||||
|
||||
### Declarative test item parameters
|
||||
|
||||
Each `TestItem` subclass declares its accepted parameters as a class attribute `PARAMS = ParamSet(Param(...), ...)` (`interpreter/utils/param_decl.py`). The descriptor carries the parameter name, *kind* (`SCALAR` — the default and may be omitted; `LIST`; `BLOCK`; `Enum("a", "b", ...)`), `required` flag, `default`, and free-form `doc`. There is **no Python type** in the descriptor on purpose: most parameter values are expressions (`$(...)` / `<| ... |>`) whose effective type is only known after expansion, so a static type would be misleading. Post-expansion `validate=lambda v: ...` callbacks are available as an opt-in for the rare cases where a runtime check is warranted (e.g. a specific format).
|
||||
|
||||
`TestItem.COMMON_PARAMS` (in `test_item.py`) declares the 14 parameters accepted by every item: `name`, `doc`, `skipped`, `key`, `stop_on_failure`, `execute_on_stop`, `process_result`, `store_result`, `expected_result`, `no_fail`, `report`, `condition`, `steps`, and the internal `seq_filename` injected by the loader. The base class concatenates `COMMON_PARAMS + subclass.PARAMS` in `_validate_declared_params()` and:
|
||||
|
||||
- emits a `tm.print_warn(...)` listing the accepted names when an unknown key appears in the user YAML (catches typos like `param_filee`);
|
||||
- raises `ETUMSyntaxError` (with the `.tum` source as context) when a `required=True` param is missing.
|
||||
|
||||
Validation is **opt-in per subclass**: while a subclass keeps `PARAMS = None` (the base-class default), the check is skipped entirely. This kept the migration incremental — items can be visited one by one without forcing a big-bang change. All structured items have been migrated; only the "unstructured-body" classes (`TestItemConsoleWrite`/`WriteLn` which carry the message as the raw value, `TestItemPlotActionAdd`/`Export` which take arbitrary plot-data keys, `TestItemUnittestElement` which is internally instantiated with `dict_item=None`) intentionally remain unvalidated.
|
||||
|
||||
Diagnostics are currently **warnings** for unknown params so an out-of-tree `.tum` with a pre-existing typo doesn't suddenly fail. The flip to a hard error is a one-line change in `_validate_declared_params()` once the user is comfortable.
|
||||
|
||||
The schema is also designed to be the source of truth for a future LSP server and for auto-generated manual sections: `ParamSet.to_schema()` returns the JSON-Schema-shaped representation that those consumers will read.
|
||||
|
||||
### Version reporting (`interpreter/utils/version.py`)
|
||||
|
||||
Both Flatpak and AppImage export `TESTIUM_VERSION` from a launcher (Flatpak: launcher script in `org.testium.Testium.yaml`; AppImage: `runtime.env` in `AppImageBuilder.yml`). `get_testium_version()` checks `/.flatpak-info` / `APPIMAGE` and reads `TESTIUM_VERSION` rather than relying on package metadata or repo introspection.
|
||||
|
||||
## Recent fixes / notable changes
|
||||
- Declarative test item parameters (v0.2): each `TestItem` subclass exposes a `PARAMS = ParamSet(...)` class attribute consumed by the base `__init__`. Catches unknown YAML keys (typo warnings listing the accepted names) and missing required params (load-time errors with `.tum` context). Lays the schema foundation for a future LSP server and auto-generated manual sections. See the matching architecture section.
|
||||
- Flatpak: `py_func` / `lua_func` / `run` sub-instance now execute on the host via `flatpak-spawn --host`. The previous attempt to inject host lib dirs into the sandbox's `LD_LIBRARY_PATH` was abandoned — host shared libs are ABI-incompatible with the Flatpak runtime's glibc and would trip `_dl_call_libc_early_init`. The manifest gained `--talk-name=org.freedesktop.Flatpak` so the spawn proxy call is allowed. The testium package is staged once per process under `/tmp` (shared with the host) so the host interpreter can locate `py_func` / `lua_func`.
|
||||
- Validation suite: single entry point with `--mode source|wheel|pyinstaller|flatpak|appimage` to validate every packaging channel against the same items. Per-mode report filenames prevent clobbering.
|
||||
- Restructure: single `src/testium/` Python package (was 4 sibling top-levels: `testium`, `lib`, `py_func`, `lua_func`). `lib/` → `runtime/`, `libs/` → `api/`. `pip install` now produces a clean `site-packages/testium/` with no top-level pollution; `.lua` files travel via `package_data`.
|
||||
- `bins.py`: centralised resolution + cache of external `python3` / `lua` binaries. Replaces the scattered `tm.gd("python_bin")`/`tm.gd("lua_bin")` dance and the duplicated discovery logic in `py_process.py`/`lua_process.py`. Validates at test load via `TestSet._validate_runtime_deps()` so missing interpreters fail fast.
|
||||
- Subprocess API contract: user scripts in `py_func`/`lua_func` use the JSON-RPC bridge (`py_func.tm` / Lua `tm`) — never `api.testium` / `interpreter.*` directly. `SUPPORTED_API` extended with `OS`, `get_main_dir`, `init_timestamp`, `timestamp`, `timestamp_as_sec` so subprocess scripts have the same surface as main-process code.
|
||||
@@ -275,10 +294,12 @@ Both Flatpak and AppImage export `TESTIUM_VERSION` from a launcher (Flatpak: lau
|
||||
## Validation tests
|
||||
Located in `test/validation/`. Two entry points:
|
||||
```
|
||||
./test/validation/run.sh # wrapper — uses a dedicated venv (see below)
|
||||
./run.sh -b -- test/validation/main.tum # direct — testium's own python is used for test execution
|
||||
./test/validation/run.sh [clean] [--mode MODE] [extra args] # wrapper — uses a dedicated venv (see below)
|
||||
./run.sh -b -- test/validation/main.tum # direct — testium's own python is used for test execution
|
||||
```
|
||||
The `run.sh` / `run.bat` wrappers create a dedicated Python venv at `${TMPDIR:-/tmp}/testium-validation-venv` (Linux) or `%TEMP%\testium-validation-venv` (Windows), with `--system-site-packages` + `pip install junit-xml`, and run the suite with `-d python_bin=…` so every test-execution subprocess (eval_proc, py_func, cycle, post_exec) runs inside the venv. testium itself keeps running in the project's own environment. `clean` as the first argument recreates the venv.
|
||||
The same item set is reused across every packaging channel — `--mode source|wheel|pyinstaller|flatpak|appimage` selects which testium binary launches the suite (`source` is the default, invoking the project's `run.sh`). Each mode stamps its results into a distinct report file (`validation-<mode>.sqlite`, `validation-<mode>-<item>.xml`) so successive runs in different modes don't clobber each other. Prerequisites (PyInstaller binary built, Flatpak bundle installed, …) are checked before launch with a hint pointing at `build_all.sh`. On Windows only `source`, `wheel`, `pyinstaller` are supported.
|
||||
|
||||
The `run.sh` / `run.bat` wrappers create a dedicated **host** Python venv at `${TMPDIR:-/tmp}/testium-validation-venv` (Linux) or `%TEMP%\testium-validation-venv` (Windows), with `--system-site-packages` + `pip install junit-xml`, and run the suite with `-d python_bin=…` so every test-execution subprocess (eval_proc, py_func, cycle, post_exec) runs inside that venv. testium itself keeps running in its own environment for the chosen mode. The venv is shared across modes because every test-execution subprocess ends up on the host either directly (source/wheel/pyinstaller/appimage) or via `flatpak-spawn --host` (flatpak). `clean` as the first argument recreates the venv. `wheel` mode also creates a separate `testium-wheel-venv-<v>` to hold the installed package.
|
||||
|
||||
The `venv` item (`test/validation/items/venv/`) asserts that the override actually took effect: `python_bin` is set, `sys.executable` matches it, `sys.prefix == dirname(dirname(python_bin))`, and `sys.prefix != sys.base_prefix` (the last marker catches the case where `python_bin` happens to be a system interpreter, which path-equality alone would miss because the venv's `bin/python3` is a symlink to the host). Both `eval_proc` (inline `<| … |>`) and `py_func` paths are exercised.
|
||||
|
||||
|
||||
@@ -30,6 +30,12 @@ dependencies = [
|
||||
]
|
||||
dynamic = ["version"]
|
||||
|
||||
[project.optional-dependencies]
|
||||
# `pip install testium[lsp]` adds the language-server dependencies. The
|
||||
# stdio-only LSP server (`testium lsp`) reuses the schema export from the
|
||||
# core install; pygls is the only marginal cost.
|
||||
lsp = ["pygls>=1.3"]
|
||||
|
||||
[project.scripts]
|
||||
testium = "testium:main"
|
||||
|
||||
|
||||
@@ -11,6 +11,30 @@ sys.path.append(os.path.abspath(ourpath.parent))
|
||||
import interpreter.utils.constants as cst
|
||||
|
||||
def main():
|
||||
# Subcommand dispatch (must run *before* argparse so neither 'schema' nor
|
||||
# 'lsp' has to share the GUI/batch flag surface). The subcommands also
|
||||
# skip the multiprocessing 'spawn' setup which is only meaningful for the
|
||||
# main runtime — schema is a pure stdout dump and lsp speaks JSON-RPC
|
||||
# over stdio without ever forking a test process.
|
||||
if len(sys.argv) >= 2 and sys.argv[1] in ("schema", "lsp"):
|
||||
sub = sys.argv[1]
|
||||
if sub == "schema":
|
||||
from lsp.schema import dump_all_schemas_json
|
||||
print(dump_all_schemas_json())
|
||||
return
|
||||
# lsp
|
||||
try:
|
||||
from lsp.server import serve
|
||||
except ImportError as e:
|
||||
print(
|
||||
f"testium lsp: language server dependencies missing ({e.name}). "
|
||||
"Install with: pip install 'testium[lsp]'",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(2)
|
||||
serve()
|
||||
return
|
||||
|
||||
# This line sets the method for the "Process" function. It is required for Linux
|
||||
# support of the test dialogs.
|
||||
multiprocessing.set_start_method('spawn')
|
||||
|
||||
16
src/testium/lsp/__init__.py
Normal file
16
src/testium/lsp/__init__.py
Normal file
@@ -0,0 +1,16 @@
|
||||
"""testium language tooling.
|
||||
|
||||
Hosts the JSON-Schema-style schema export of every test item type, and a
|
||||
``pygls`` language server that consumes the same schema to provide
|
||||
completion / hover / diagnostics for ``.tum`` files in any LSP-capable
|
||||
editor (VSCode, neovim, Helix, Emacs, …).
|
||||
|
||||
Entry points (both surfaced through the ``testium`` CLI):
|
||||
|
||||
- ``testium schema`` — dump the schema of every item type as JSON on stdout.
|
||||
Zero runtime dependencies; can be used by editors that already speak the
|
||||
YAML JSON Schema extension to get static completion immediately.
|
||||
|
||||
- ``testium lsp`` — start the language server over stdio. Requires the
|
||||
``pygls`` optional dependency (``pip install testium[lsp]``).
|
||||
"""
|
||||
6
src/testium/lsp/__main__.py
Normal file
6
src/testium/lsp/__main__.py
Normal file
@@ -0,0 +1,6 @@
|
||||
"""Entry point for ``python -m testium.lsp`` (alternative to ``testium lsp``)."""
|
||||
|
||||
from lsp.server import serve
|
||||
|
||||
if __name__ == "__main__":
|
||||
serve()
|
||||
146
src/testium/lsp/schema.py
Normal file
146
src/testium/lsp/schema.py
Normal file
@@ -0,0 +1,146 @@
|
||||
"""Schema export of the test item registry.
|
||||
|
||||
Walks every ``TestItemType`` entry (``interpreter/utils/constants.py``),
|
||||
combines its declared ``PARAMS`` with the common ones, and returns a
|
||||
serialisable structure keyed by ``item_cmd`` — the YAML key the user
|
||||
writes (e.g. ``sleep``, ``py_func``, ``dialog_message``).
|
||||
|
||||
Items intentionally without ``PARAMS`` (the unstructured-body classes
|
||||
like console ``write``/``writeln`` or plot ``add``/``export``) are
|
||||
emitted as ``"params_declared": false`` so consumers know to suggest
|
||||
nothing for them rather than reporting a closed empty set.
|
||||
|
||||
Action items (children of ``parallel``, ``console``, ``json_rpc``,
|
||||
``plot``) are registered separately under each parent's ``actions``
|
||||
entry — they're not top-level YAML keys, they live nested inside a
|
||||
parent's ``steps:``.
|
||||
"""
|
||||
|
||||
import json
|
||||
|
||||
from interpreter.utils.constants import TestItemType
|
||||
from interpreter.utils.test_init import _constants_init
|
||||
|
||||
|
||||
# Action class -> parent cmd (the action's parent in the YAML). Action classes
|
||||
# aren't first-class TestItemType entries (TYPE_*_ACTION is one generic bucket),
|
||||
# so we resolve their YAML key by looking at how each parent registers them.
|
||||
def _collect_action_classes(parent_class):
|
||||
"""Return {action_yaml_key: action_class} for a TestItemActions parent.
|
||||
|
||||
Each parent's ``__init__`` calls ``self.register_actions(name=class, ...)``
|
||||
*during* construction, so we can't read the registry without instantiating
|
||||
one. We work around it by parsing the source for the registration call —
|
||||
cheap, no side effects, and the schema export is a CLI command anyway.
|
||||
"""
|
||||
import ast
|
||||
import inspect
|
||||
|
||||
try:
|
||||
src = inspect.getsource(parent_class)
|
||||
except (OSError, TypeError):
|
||||
return {}
|
||||
|
||||
actions = {}
|
||||
tree = ast.parse(src)
|
||||
for node in ast.walk(tree):
|
||||
if not isinstance(node, ast.Call):
|
||||
continue
|
||||
fn = node.func
|
||||
if not (isinstance(fn, ast.Attribute) and fn.attr == "register_actions"):
|
||||
continue
|
||||
for kw in node.keywords:
|
||||
if kw.arg is None or not isinstance(kw.value, ast.Name):
|
||||
continue
|
||||
# ast.Name gives us only the bare identifier; resolve it through
|
||||
# the parent class's defining module.
|
||||
mod = __import__(parent_class.__module__, fromlist=[kw.value.id])
|
||||
cls = getattr(mod, kw.value.id, None)
|
||||
if cls is not None:
|
||||
actions[kw.arg] = cls
|
||||
return actions
|
||||
|
||||
|
||||
def _params_to_schema(item_class, common_params):
|
||||
"""Return the params-portion of an item's schema entry.
|
||||
|
||||
Common params are flagged so consumers can render them differently
|
||||
(an editor might show "common" parameters in a separate group).
|
||||
"""
|
||||
own = getattr(item_class, "PARAMS", None)
|
||||
if own is None:
|
||||
return {"params_declared": False}
|
||||
common_names = set(common_params.names())
|
||||
params = []
|
||||
for p in common_params:
|
||||
d = p.to_schema()
|
||||
d["common"] = True
|
||||
params.append(d)
|
||||
for p in own:
|
||||
if p.name in common_names:
|
||||
# Subclass overrode a common param (e.g. tightened doc).
|
||||
for d in params:
|
||||
if d["name"] == p.name:
|
||||
d.update(p.to_schema())
|
||||
break
|
||||
continue
|
||||
d = p.to_schema()
|
||||
d["common"] = False
|
||||
params.append(d)
|
||||
return {"params_declared": True, "params": params}
|
||||
|
||||
|
||||
def dump_all_schemas():
|
||||
"""Return the full schema as a Python dict ready for json.dumps.
|
||||
|
||||
Shape:
|
||||
{
|
||||
"items": {
|
||||
"sleep": {
|
||||
"display_name": "Sleep",
|
||||
"params_declared": true,
|
||||
"params": [{name, kind, required, default?, doc, common}, ...],
|
||||
},
|
||||
"console": {
|
||||
...,
|
||||
"actions": {"open": {...}, "close": {...}, ...},
|
||||
},
|
||||
...
|
||||
}
|
||||
}
|
||||
"""
|
||||
_constants_init()
|
||||
# Imported lazily — pulls test_item.py which references constants.
|
||||
from interpreter.test_items.test_item import COMMON_PARAMS
|
||||
|
||||
out = {"items": {}}
|
||||
for tp in TestItemType:
|
||||
cls = getattr(tp, "item_class", None)
|
||||
if cls is None:
|
||||
continue
|
||||
# Action types (CONSOLE_ACTION, GRAPH_ACTION, JSON_RPC_ACTION) have no
|
||||
# standalone YAML representation — skip them here, they show up under
|
||||
# their parent's "actions" key.
|
||||
cmd = tp.item_cmd
|
||||
if cmd.endswith("_action"):
|
||||
continue
|
||||
entry = {"display_name": tp.item_name}
|
||||
entry.update(_params_to_schema(cls, COMMON_PARAMS))
|
||||
|
||||
actions = _collect_action_classes(cls)
|
||||
if actions:
|
||||
entry["actions"] = {
|
||||
name: _params_to_schema(acls, COMMON_PARAMS)
|
||||
for name, acls in actions.items()
|
||||
}
|
||||
for name in entry["actions"]:
|
||||
entry["actions"][name]["display_name"] = name
|
||||
|
||||
out["items"][cmd] = entry
|
||||
return out
|
||||
|
||||
|
||||
def dump_all_schemas_json(indent=2):
|
||||
"""Same as ``dump_all_schemas`` but serialised to a JSON string."""
|
||||
return json.dumps(dump_all_schemas(), indent=indent, sort_keys=False,
|
||||
default=str)
|
||||
313
src/testium/lsp/server.py
Normal file
313
src/testium/lsp/server.py
Normal file
@@ -0,0 +1,313 @@
|
||||
"""LSP server for ``.tum`` files.
|
||||
|
||||
Features available so far:
|
||||
|
||||
- **Completion** — when the user starts a new YAML step (``- <cursor>``),
|
||||
the server proposes the full list of known item types. The completion
|
||||
item carries a short hover-style description listing required and
|
||||
optional parameters.
|
||||
- **Hover** — over a known item-type word (``sleep``, ``py_func``, …)
|
||||
the server renders the same description in a popup.
|
||||
- **Document symbols (outline)** — every ``- <type>:`` line becomes an
|
||||
entry in the editor's outline view. Nesting follows YAML indentation,
|
||||
so containers (``group``, ``loop``, ``parallel``, ``console`` …)
|
||||
display their children as a subtree.
|
||||
|
||||
The server speaks LSP over stdio. Start it with::
|
||||
|
||||
testium lsp
|
||||
|
||||
Editors invoke it through their LSP client; the connection layer
|
||||
(``vscode-languageclient``, ``nvim-lspconfig``, ``lsp-mode``, …) takes
|
||||
care of the JSON-RPC framing.
|
||||
|
||||
Architecture notes
|
||||
------------------
|
||||
|
||||
The schema is built once at server start (``dump_all_schemas()``) and
|
||||
kept in memory; an editor restart picks up upstream changes. The schema
|
||||
is the **only** source of truth — when testium adds a new item type or
|
||||
parameter, the LSP automatically exposes it without any change here.
|
||||
|
||||
The current handlers stay deliberately heuristic on the parser side:
|
||||
completion uses a line-prefix regex, outline a per-line ``- <known>:``
|
||||
sweep with indentation tracking. A proper YAML+Jinja parsing pass is
|
||||
still pending and is the prerequisite for *parameter*-level completion
|
||||
and diagnostics.
|
||||
"""
|
||||
|
||||
import re
|
||||
|
||||
try:
|
||||
# pygls 2.x moved LanguageServer under pygls.lsp.server. We pin >=1.3 in
|
||||
# the optional dependency to stay open to either family, but the import
|
||||
# path differs — try the new one first, then the legacy one.
|
||||
try:
|
||||
from pygls.lsp.server import LanguageServer
|
||||
except ImportError:
|
||||
from pygls.server import LanguageServer # pygls < 2
|
||||
from lsprotocol.types import (
|
||||
TEXT_DOCUMENT_COMPLETION,
|
||||
TEXT_DOCUMENT_DOCUMENT_SYMBOL,
|
||||
TEXT_DOCUMENT_HOVER,
|
||||
CompletionItem,
|
||||
CompletionItemKind,
|
||||
CompletionList,
|
||||
CompletionOptions,
|
||||
CompletionParams,
|
||||
DocumentSymbol,
|
||||
DocumentSymbolParams,
|
||||
Hover,
|
||||
HoverParams,
|
||||
InsertTextFormat,
|
||||
MarkupContent,
|
||||
MarkupKind,
|
||||
Position,
|
||||
Range,
|
||||
SymbolKind,
|
||||
)
|
||||
except ImportError as exc:
|
||||
# Surfaced by the CLI dispatcher with a friendly install hint.
|
||||
raise
|
||||
|
||||
|
||||
from lsp.schema import dump_all_schemas
|
||||
|
||||
|
||||
_LINE_START_STEP = re.compile(r"^\s*-\s*([A-Za-z_][A-Za-z0-9_]*)?\s*:?\s*$")
|
||||
|
||||
# Matches "- <identifier>:" for outline / hover purposes. Captures the start
|
||||
# column of the identifier and the identifier itself. Trailing tokens after
|
||||
# the colon (inline-form params, comments) are tolerated.
|
||||
_STEP_LINE = re.compile(r"^(?P<lead>\s*-\s*)(?P<ident>[A-Za-z_][A-Za-z0-9_]*)\s*:")
|
||||
|
||||
# Matches a ``name: <value>`` line under an item — used by the outline pass
|
||||
# to surface the user's display name next to the item type.
|
||||
_NAME_FIELD = re.compile(r"^\s*name\s*:\s*(?P<value>.+?)\s*$")
|
||||
|
||||
# Word boundary used by hover to extract the identifier under the cursor.
|
||||
_IDENT_AT = re.compile(r"[A-Za-z_][A-Za-z0-9_]*")
|
||||
|
||||
|
||||
def _render_item_markdown(cmd, entry):
|
||||
"""Render an item-type's schema entry as a Markdown hover string.
|
||||
|
||||
Reused by both the completion-item documentation and the hover
|
||||
handler so the editor presents identical information regardless of
|
||||
how the user reached it.
|
||||
"""
|
||||
detail = entry.get("display_name", cmd)
|
||||
lines = [f"**{cmd}** — {detail}", ""]
|
||||
if entry.get("params_declared"):
|
||||
non_common = [p for p in entry["params"] if not p["common"]]
|
||||
required = [p for p in non_common if p["required"]]
|
||||
optional = [p for p in non_common if not p["required"]]
|
||||
if required:
|
||||
lines.append("Required parameters:")
|
||||
for p in required:
|
||||
lines.append(f"- `{p['name']}` — {p['doc']}")
|
||||
lines.append("")
|
||||
if optional:
|
||||
lines.append("Optional parameters:")
|
||||
for p in optional:
|
||||
lines.append(f"- `{p['name']}` — {p['doc']}")
|
||||
else:
|
||||
lines.append("(Parameter list is not described — this item's body is the "
|
||||
"raw user value.)")
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def _build_item_completions(schema):
|
||||
"""Return a list of CompletionItem covering every top-level item type.
|
||||
|
||||
Each completion inserts ``<name>:`` with the cursor positioned after
|
||||
the colon so the user can immediately start typing parameters.
|
||||
"""
|
||||
items = []
|
||||
for cmd, entry in schema["items"].items():
|
||||
if cmd == "default":
|
||||
# Root sentinel; never appears as a YAML key.
|
||||
continue
|
||||
items.append(
|
||||
CompletionItem(
|
||||
label=cmd,
|
||||
kind=CompletionItemKind.Class,
|
||||
detail=entry.get("display_name", cmd),
|
||||
documentation=MarkupContent(
|
||||
kind=MarkupKind.Markdown,
|
||||
value=_render_item_markdown(cmd, entry),
|
||||
),
|
||||
insert_text=f"{cmd}:",
|
||||
insert_text_format=InsertTextFormat.PlainText,
|
||||
)
|
||||
)
|
||||
items.sort(key=lambda it: it.label)
|
||||
return items
|
||||
|
||||
|
||||
def _word_at(line, character):
|
||||
"""Return ``(start, end, text)`` of the identifier under ``character``.
|
||||
|
||||
Returns ``None`` when the cursor isn't on a word. Used by hover.
|
||||
"""
|
||||
for m in _IDENT_AT.finditer(line):
|
||||
if m.start() <= character <= m.end():
|
||||
return m.start(), m.end(), m.group(0)
|
||||
return None
|
||||
|
||||
|
||||
def _build_document_symbols(lines, item_cmds):
|
||||
"""Walk ``lines`` and produce a nested ``DocumentSymbol`` tree.
|
||||
|
||||
Heuristics (no YAML parsing yet):
|
||||
- Each ``- <known_cmd>:`` line becomes a symbol.
|
||||
- Nesting follows the indentation of the leading ``-``: a deeper-
|
||||
indented step is treated as a child of the most recent shallower
|
||||
step.
|
||||
- The symbol's ``detail`` is the ``name: <value>`` field if found
|
||||
within a small window after the step header (no YAML parsing —
|
||||
we just look at indented lines that aren't another ``- …`` step).
|
||||
|
||||
The result is suitable for the LSP outline panel even when the
|
||||
surrounding YAML is mid-edit and structurally invalid.
|
||||
"""
|
||||
root_children = []
|
||||
# Each stack entry: (indent_col, children_list_to_append_to,
|
||||
# pending_parent_symbol or None).
|
||||
stack = [(-1, root_children, None)]
|
||||
|
||||
def _attach_name(parent_symbol, start_line):
|
||||
"""Look for the nearest ``name:`` field in the children of ``parent``."""
|
||||
if parent_symbol is None or start_line + 1 >= len(lines):
|
||||
return
|
||||
base_indent = len(lines[start_line]) - len(lines[start_line].lstrip(" "))
|
||||
for j in range(start_line + 1, min(start_line + 10, len(lines))):
|
||||
l = lines[j]
|
||||
stripped = l.lstrip(" ")
|
||||
indent = len(l) - len(stripped)
|
||||
if indent <= base_indent and stripped.strip() != "":
|
||||
break
|
||||
m = _NAME_FIELD.match(l)
|
||||
if m:
|
||||
value = m.group("value").strip("\"' ")
|
||||
parent_symbol.detail = value
|
||||
return
|
||||
|
||||
for i, raw_line in enumerate(lines):
|
||||
m = _STEP_LINE.match(raw_line)
|
||||
if not m:
|
||||
continue
|
||||
cmd = m.group("ident")
|
||||
if cmd not in item_cmds:
|
||||
continue
|
||||
indent = len(m.group("lead")) - len(m.group("lead").lstrip(" "))
|
||||
# Pop the stack until we find a parent with strictly smaller indent.
|
||||
while stack and stack[-1][0] >= indent:
|
||||
stack.pop()
|
||||
if not stack:
|
||||
stack.append((-1, root_children, None))
|
||||
parent_children = stack[-1][1]
|
||||
|
||||
ident_start = m.start("ident")
|
||||
ident_end = m.end("ident")
|
||||
symbol = DocumentSymbol(
|
||||
name=cmd,
|
||||
detail=None,
|
||||
kind=SymbolKind.Function,
|
||||
range=Range(
|
||||
start=Position(line=i, character=0),
|
||||
end=Position(line=i, character=len(raw_line.rstrip("\n"))),
|
||||
),
|
||||
selection_range=Range(
|
||||
start=Position(line=i, character=ident_start),
|
||||
end=Position(line=i, character=ident_end),
|
||||
),
|
||||
children=[],
|
||||
)
|
||||
parent_children.append(symbol)
|
||||
stack.append((indent, symbol.children, symbol))
|
||||
_attach_name(symbol, i)
|
||||
return root_children
|
||||
|
||||
|
||||
def _make_server():
|
||||
server = LanguageServer("testium-lsp", "0.1.0")
|
||||
schema = dump_all_schemas()
|
||||
item_completions = _build_item_completions(schema)
|
||||
# Set of cmd names accepted by the outline / hover passes. We include
|
||||
# action names (console open/close/…, plot open/close/…, …) too so they
|
||||
# appear in the outline tree and respond to hover.
|
||||
item_cmds = set()
|
||||
for cmd, entry in schema["items"].items():
|
||||
if cmd == "default":
|
||||
continue
|
||||
item_cmds.add(cmd)
|
||||
item_cmds.update(entry.get("actions", {}).keys())
|
||||
|
||||
@server.feature(
|
||||
TEXT_DOCUMENT_COMPLETION,
|
||||
CompletionOptions(trigger_characters=["-", " "]),
|
||||
)
|
||||
def completion(params: CompletionParams):
|
||||
doc = server.workspace.get_text_document(params.text_document.uri)
|
||||
line_idx = params.position.line
|
||||
if line_idx >= len(doc.lines):
|
||||
return CompletionList(is_incomplete=False, items=[])
|
||||
line = doc.lines[line_idx]
|
||||
# Only look at what's left of the cursor.
|
||||
prefix = line[: params.position.character]
|
||||
if not _LINE_START_STEP.match(prefix):
|
||||
return CompletionList(is_incomplete=False, items=[])
|
||||
return CompletionList(is_incomplete=False, items=item_completions)
|
||||
|
||||
@server.feature(TEXT_DOCUMENT_HOVER)
|
||||
def hover(params: HoverParams):
|
||||
doc = server.workspace.get_text_document(params.text_document.uri)
|
||||
line_idx = params.position.line
|
||||
if line_idx >= len(doc.lines):
|
||||
return None
|
||||
line = doc.lines[line_idx]
|
||||
# Only respond when the cursor is on the type part of a step line
|
||||
# ("- sleep:") — never for arbitrary words in a string.
|
||||
step_match = _STEP_LINE.match(line)
|
||||
if not step_match:
|
||||
return None
|
||||
word = _word_at(line, params.position.character)
|
||||
if word is None:
|
||||
return None
|
||||
start, end, text = word
|
||||
if text != step_match.group("ident") or text not in item_cmds:
|
||||
return None
|
||||
# Resolve the entry: top-level item, or action of any parent.
|
||||
entry = schema["items"].get(text)
|
||||
if entry is None:
|
||||
for parent_entry in schema["items"].values():
|
||||
actions = parent_entry.get("actions") or {}
|
||||
if text in actions:
|
||||
entry = actions[text]
|
||||
break
|
||||
if entry is None:
|
||||
return None
|
||||
return Hover(
|
||||
contents=MarkupContent(
|
||||
kind=MarkupKind.Markdown,
|
||||
value=_render_item_markdown(text, entry),
|
||||
),
|
||||
range=Range(
|
||||
start=Position(line=line_idx, character=start),
|
||||
end=Position(line=line_idx, character=end),
|
||||
),
|
||||
)
|
||||
|
||||
@server.feature(TEXT_DOCUMENT_DOCUMENT_SYMBOL)
|
||||
def document_symbols(params: DocumentSymbolParams):
|
||||
doc = server.workspace.get_text_document(params.text_document.uri)
|
||||
return _build_document_symbols(doc.lines, item_cmds)
|
||||
|
||||
return server
|
||||
|
||||
|
||||
def serve():
|
||||
"""Start the LSP server on stdio. Blocks until the client disconnects."""
|
||||
server = _make_server()
|
||||
server.start_io()
|
||||
Reference in New Issue
Block a user