bs_explorer/doc/tutorial.md

# Tutorial — from probe to programmed part

This walks through the full `bs_explorer` flow: detect a probe, scan the
chain, identify the device, and program it. There are two programming
paths and the tutorial covers both:

- **Xilinx external SPI configuration flash**, via the BSCAN proxy —
  shown on a Kintex UltraScale+ KU15P / UltraScale KU040;
- **any other part** (Lattice, Microsemi, …), by playing a
  vendor-exported **SVF** — shown on a Microsemi IGLOO2 M2GL010T.

The early steps are identical for any device in `data/targets.yaml`;
only the IDCODE and BSDL filename change.

## Prerequisites

- A JTAG probe physically wired to the target's TCK/TDI/TDO/TMS/TRST.
- `libftdi1` + `libusb-1.0` installed (Arch: `libftdi`; Debian:
  `libftdi1-dev`) — open source, drives FTDI/Olimex probes.
- The target's BSDL in `data/bsdl_files/` (KU15P: `xcku15p_ffve1517.bsd` is
  bundled).
- An entry for the target in `data/targets.yaml` (KU15P is bundled).
  See [Adding a new FPGA](#6-add-a-new-fpga-target) below.
- For SPI flashing, eventually: a BSCAN proxy bitstream — see the
  [Phase 2.5 caveat](#phase-25-spi-through-the-bscan-proxy-bridge-bitstream) at the end.

If your board uses a Digilent JTAG-SMT2 / SMT2-NC module (KCU105,
ZCU102, …), the Digilent backend is built in by default on Linux — just
install the Adept Runtime system-wide so `libdjtg.so`/`libdmgr.so` are
present at runtime. Plain MPSSE does not work on those modules — see the
README and the `Digilent SMT2` block in `CLAUDE.md` for the why.

## Build & launch

```sh
mkdir build && cd build && cmake .. && make && cd ..
./build/bs        # run from the repo root: data/probes.yaml, data/targets.yaml,
                     # data/bsdl_files/ and data/bscan_proxies/ are looked up in the CWD
```

You should see:

```
  Boundary Scan Explorer  v2.6.7.1
  Based on Viveris jtag-boundary-scanner

  3 probe driver(s) available.
  Type 'help' or '?' to list commands, 'exit' or Ctrl-D to quit.
  <Tab> completes commands.

bs_explorer>
```

`<Tab>` completes commands; `help <cmd>` shows per-command help;
Ctrl-D or `exit` quits.

## 1. Detect and open the probe

```
bs_explorer> jtag_probes
  [0]  0x00000000  Digilent USB Device 210308AB06A6
  [1]  0x00000300  Digilent: JtagSmt2NC
```

Open a probe by the index in brackets:

```
bs_explorer> jtag_open 1
```

The `0x…` value next to each index is the raw probe id and is also
accepted (`jtag_open 0x300`) — handy in scripts where you'd rather pin
the exact backend than rely on enumeration order.

If `jtag_open` fails: check `lsusb` for the probe VID:PID, make
sure the user has access to the USB device (udev rule or group), and
confirm no other process holds the probe (e.g. `openocd`).

### Probe profiles

Some probes need pin tweaks the built-in defaults don't cover — e.g. the
embedded **FlashPro** on Microsemi eval kits (an FT4232H whose JTAG sits
on channel A = index 0 and needs `ADBUS4` left high-Z, or the chain stays
silent). `data/probes.yaml` captures these as named profiles; apply one when
opening:

```
bs_explorer> jtag_profiles
2 probe profile(s) in data/probes.yaml:
  flashpro
  ft2232h
bs_explorer> jtag_open 0 flashpro
Applied probe profile 'flashpro'.
Probe Ok !
```

`jtag_close` releases the current probe (frees its USB handle) — use it
to hand the probe to another tool, or to program two boards on different
probes in turn: `jtag_close`, then `jtag_open` the next one and
`jtag_autoinit` to rescan.

## 2. Scan the JTAG chain

The fastest path is `jtag_autoinit`: it scans the chain *and*
auto-loads every BSDL in `data/bsdl_files/` whose IDCODE matches a device.

```
bs_explorer> jtag_autoinit
```

Expected output for a single-FPGA board:

```
1 device(s) found
Device 0 (04A56093 - XCKU15P_FFVE1517) - BSDL Loaded : ...xcku15p_ffve1517.bsd
```

If the device count is 0 or the IDCODE is `0xFFFFFFFF`/`0x00000000`:
TDI/TDO are likely swapped, TRST not released, or the probe is
mis-wired. Power-cycle and re-check the harness before going further.

## 3. Identify the FPGA against the registry

`target_info` walks the chain and matches each IDCODE against the
registry in `data/targets.yaml`:

```
bs_explorer> target_info
Device 0  IDCODE 0x04A56093  -> Xilinx Kintex UltraScale+ XCKU15P [Xilinx UltraScale+]
           prog: proxy_spi
           caveat: CCLK routed via STARTUP primitive (not drivable in EXTEST)
```

`prog:` is the programming backend the registry assigns the part —
`proxy_spi` (Xilinx external flash, [§Phase 2.5](#phase-25-spi-through-the-bscan-proxy-bridge-bitstream))
or `svf` ([§Programming via SVF](#programming-via-svf-lattice-microsemi-)).

If you get `not in registry`, add an entry — see
[Adding a new FPGA](#6-add-a-new-fpga-target).

`target_list` prints the whole registry without needing a probe.

## JTAG clock (optional)

Each driver has its own default TCK (FTDI 1 MHz, Digilent 4 MHz). To set
a single driver-neutral clock, before `jtag_open`:

```
bs_explorer> set JTAG_TCK_FREQ_KHZ 1000
```

It's applied at open (mapped to the FTDI driver's variable, or read
directly by the Digilent one). A registry entry may declare a
`max_tck_khz`; if your requested clock exceeds it, `jtag_autoinit` clamps
it and re-opens the probe at the safe rate:

```
WARNING : JTAG clock 2000 kHz exceeds the device max 1000 kHz; clamping.
Re-opening at 1000 kHz and re-scanning.
```

## 4. (Optional) Sanity-check the low-level JTAG primitives

Before doing anything fancy, you can verify that `bscan_set_ir` and
`bscan_shift_dr` actually move bits correctly on your hardware. Drop
the BSDL state (so `jtag_core` doesn't fight us on IR caching) and
shift IDCODE manually:

```
bs_explorer> jtag_open 0             # index from jtag_probes
bs_explorer> jtag_scan               # detects devices, does NOT load BSDL
bs_explorer> bscan_set_ir 9 6        # IDCODE opcode (KU15P: 0x09, IR=6 bits)
bs_explorer> bscan_shift_dr 32
DR = 04 A5 60 93                     # bytes printed MSB-first
```

Match → primitives are healthy. Mismatch → wiring or clock issue, fix
that before moving on. The opcode and IR length come from the
`fpga_target` (and ultimately the BSDL `INSTRUCTION_OPCODE` block).

## 5. Read the SPI flash JEDEC ID via EXTEST (validation only)

This is the *slow* path — useful to confirm the SPI pins are wired
correctly to the flash, **not** a viable way to flash megabytes. See
[Phase 2.5](#phase-25-spi-through-the-bscan-proxy-bridge-bitstream) for the production path.

Put the FPGA in EXTEST, then map the four SPI signals onto the FPGA's
BSDL pin names:

```
bs_explorer> jtag_autoinit
bs_explorer> jtag_mode 0 EXTEST
bs_explorer> jtag_spi_cs   0 <PIN_CS>   0
bs_explorer> jtag_spi_clk  0 <PIN_CLK>  0
bs_explorer> jtag_spi_mosi 0 <PIN_MOSI> 0
bs_explorer> jtag_spi_miso 0 <PIN_MISO> 0
```

Pin names depend on the board: dump `jtag_pins 0` to discover
them. On Xilinx FPGAs, the SPI flash is typically wired to the
configuration bank (e.g. `D00_MOSI_0`, `D01_DIN_0`, `FCS_B_0`) —
**except** `CCLK`, which goes through the `STARTUPE3` primitive and is
not drivable in EXTEST (the `CCLK_VIA_STARTUP` caveat on the target).

Send the JEDEC ID command (`0x9F` + 3 dummy bytes):

```
bs_explorer> jtag_spi_xfer 9F000000
SPI TX: 9F 00 00 00
SPI RX: FF XX YY ZZ                  # XX YY ZZ identifies the flash vendor/part
```

This takes several seconds even for 4 bytes — that's the
~30 bytes/sec EXTEST ceiling. Don't try to read the whole flash this
way; you'd be there for weeks.

## 6. Add a new FPGA target

The registry — `data/targets.yaml` at the repo root — holds the
per-part facts that can't be derived from the BSDL alone (or are tedious
to). It's parsed at runtime, so adding a part is **editing YAML, no
rebuild**. The XCKU040 entry already there was added exactly with the
steps below — use it as your template.

### a. Drop the BSDL

Put the part's `.bsd` in `data/bsdl_files/`. Source: Xilinx/AMD device page
under "Design Files / BSDL", Intel in the Quartus install, Lattice per
part, Microsemi/Microchip via Libero. `jtag_autoinit` will then
auto-load it by IDCODE.

### b. Pull the facts out of the BSDL

Everything you need is in the file:

```sh
grep -iE "INSTRUCTION_LENGTH|IDCODE_REGISTER|\b(USER1|CFG_IN|JPROGRAM|JSTART|JSHUTDOWN|ISC_DISABLE)\b" \
     data/bsdl_files/xcku040_ffva1156.bsd
```

For the XCKU040 this yields IR length 6, and the private opcodes
`USER1=000010` (0x02), `CFG_IN=000101` (0x05), `JPROGRAM=001011`
(0x0B), `JSTART=001100` (0x0C), etc. The `IDCODE_REGISTER` string is
`XXXX...0010 0000 1001 0011` — the top four bits are the silicon
**revision** and read as `X` (don't-care), which is why the registry
masks them off.

### c. Add a YAML entry

Append a list item under `fpgas:` in `data/targets.yaml`:

| Key | What it is | XCKU040 |
|-----|-----------|---------|
| `name` | human-readable label (quoted) | "Xilinx Kintex UltraScale XCKU040" |
| `idcode` | IDCODE pattern (version nibble as 0) | `0x03822093` |
| `idcode_mask` | bits that must match; `0x0FFFFFFF` ignores the Xilinx revision nibble (default `0xFFFFFFFF`) | `0x0FFFFFFF` |
| `family` | `xilinx_7/us/usp`, `microsemi_igloo2/smartfusion2`, `lattice_machxo2/3` | `xilinx_us` |
| `bsdl` | basename in `data/bsdl_files/` | `xcku040_ffva1156.bsd` |
| `ir_length` | IR width in bits | `6` |
| `ir_cfg_in` / `ir_user1` / `ir_jprogram` / `ir_jstart` / `ir_jshutdown` / `ir_isc_disable` | private IR opcodes (omit = 0/N/A) | from the BSDL |
| `proxy_bitstream` | BSCAN proxy `.bit` in `data/bscan_proxies/` (omit if none) | `bscan_spi_xcku040.bit` |
| `caveats` | space/comma-separated flag names (omit if none) | `cclk_via_startup` |
| `max_tck_khz` | max safe JTAG TCK in kHz; `jtag_autoinit` clamps + re-opens if exceeded (omit = unspecified) | — |
| `prog` | programming backend `proxy_spi`/`svf`/`none` (omit → inferred: a proxy ⇒ `proxy_spi`, Microsemi/Lattice ⇒ `svf`) | `proxy_spi` |

The resulting entry (verbatim from `data/targets.yaml`):

```yaml
  - name:            "Xilinx Kintex UltraScale XCKU040"
    idcode:          0x03822093
    idcode_mask:     0x0FFFFFFF
    family:          xilinx_us
    bsdl:            xcku040_ffva1156.bsd
    ir_length:       6
    ir_cfg_in:       0x05
    ir_user1:        0x02
    ir_jprogram:     0x0B
    ir_jstart:       0x0C
    ir_jshutdown:    0x0D
    ir_isc_disable:  0x16
    proxy_bitstream: bscan_spi_xcku040.bit
    caveats:         cclk_via_startup
    prog:            proxy_spi
```

Omit any field that doesn't apply: a missing `proxy_bitstream` means
"none yet", a missing `caveats` means none, a missing `idcode_mask`
defaults to exact match (`0xFFFFFFFF`).

### What `caveats` means

`caveats` is a space/comma-separated list of flag names, each backed by
an `FPGA_CAVEAT_*` bit in `fpga.h`, marking **known hardware gotchas
that change how the tool must drive the part**. It is *not* a free-text
note — each flag is something the code (or you) can branch on. Omit the
field when the part has none. `target_info` prints any flag that is set,
as a human-readable line.

Currently one flag exists:

- `cclk_via_startup` (`FPGA_CAVEAT_CCLK_VIA_STARTUP`) — on Xilinx
  7-Series / UltraScale / UltraScale+, the SPI configuration clock
  **CCLK is not a normal I/O pin**: it is routed through the
  `STARTUP`/`STARTUPE3` primitive and therefore **cannot be toggled in
  EXTEST** boundary scan. Practical effect: the slow EXTEST SPI bit-bang
  can't clock the flash on these parts — you must use the BSCAN proxy
  (Phase 2.5), where CCLK is driven by the fabric internally and the
  problem disappears. Set this for any 7-Series/US/US+ part.

To introduce a *new* caveat: add a `#define FPGA_CAVEAT_xxx (1u << n)`
in `fpga.h`, teach `parse_caveats()` in `fpga.c` its YAML name, use that
name in the YAML, and (if it should be visible) print it in
`cmd_target_info` in `script.c`.

### d. Verify — no rebuild

The registry is loaded at runtime, so just (re)start bs_explorer from
the repo root and check:

```sh
./build/bs
bs_explorer> target_list          # your part should appear, with its source file
bs_explorer> jtag_autoinit
bs_explorer> target_info          # should show your part, family, and any caveats
```

(`target_list` reads the registry without needing a probe.)

## Phase 2.5: SPI through the BSCAN proxy (bridge bitstream)

Talking to the SPI flash via EXTEST is fine for a JEDEC ID but useless
for real flashing (~30 B/s, days to weeks for a config part). The
production path loads a tiny **BSCAN proxy** bitstream into the FPGA
fabric, then runs SPI through the `USER1` instruction at fabric speed
(~50–200 KB/s). The proxy uses a `BSCANE2` primitive to bridge the
`USER1` DR shift to the flash pins, and drives `CCLK` from the fabric
internally — so the `STARTUPE3`/CCLK problem of EXTEST disappears.

### Get the bridge bitstream

Pre-built proxies live in `quartiq/bscan_spi_bitstreams` (MIT). Drop
the one for your part in `data/bscan_proxies/`:

```sh
curl -L -o data/bscan_proxies/bscan_spi_xcku040.bit \
  https://raw.githubusercontent.com/quartiq/bscan_spi_bitstreams/master/bscan_spi_xcku040.bit
```

The registry entry for the part points at this file via its
`proxy_bitstream` field (e.g. the XCKU040 entry → `bscan_spi_xcku040.bit`).

#### When your part isn't pre-built (e.g. the KU15P)

quartiq ships `.bit` only for the parts its generator knows — it has
**no UltraScale+** proxy (its single UltraScale entry is the KU040), so
the KU15P has to be built from source. You need (o)Migen + Vivado
(2022.2; ISE 14.7 for older parts). From a clone of the quartiq repo,
per its README:

```sh
python3 -m venv --system-site-packages .venv
./.venv/bin/pip install -r requirements.txt
PATH=$PATH:/opt/Xilinx/Vivado/2022.2/bin \
  ./.venv/bin/python3 xilinx_bscan_spi.py ...
```

The XCKU15P first has to be **added to the generator's device table**
(a Migen platform entry) — it's not just a command-line part flag.

Once built, drop `bscan_spi_xcku15p.bit` into `data/bscan_proxies/` (it's
MIT, like the KU040 — keep `data/bscan_proxies/LICENSE.quartiq`) and set the
`proxy_bitstream` field on the KU15P entry in `data/targets.yaml`
(currently omitted).

### Load the bridge and talk SPI

```
bs_explorer> jtag_open 1
bs_explorer> jtag_autoinit
bs_explorer> bscan_load_bitstream 0 data/bscan_proxies/bscan_spi_xcku040.bit
bs_explorer> bscan_jedec 0
JEDEC ID: 20 BB 19  (manufacturer 0x20, device 0xBB19)
```

(Validated on a KCU105: `0x20` = Micron, `0xBB19` = MT25QU256, the
board's 1.8 V 256 Mbit config flash. Other boards will show different
device bytes.)

`bscan_load_bitstream` runs JPROGRAM → CFG_IN → shift → JSTART, which
**reconfigures the FPGA fabric**: the design currently running on the
part is wiped and replaced by the proxy. This is undone by a
power-cycle (the configuration flash reloads the original design at the
next boot), but be aware the board stops doing whatever it was doing.

`bscan_jedec` issues the SPI **Read Identification** command (`0x9F`,
also called RDID) and reads back 3 bytes — the chip's *JEDEC ID*. JEDEC
is the standards body (JESD216 / JEP106) that defines this identifier;
every serial flash answers `0x9F` the same way:

- **byte 0** — manufacturer, a code assigned by JEDEC (`0x20` Micron,
  `0xEF` Winbond, `0xC2` Macronix, `0x01` Cypress/Infineon, `0x9D`
  ISSI, …);
- **bytes 1–2** — device ID (memory type + capacity), vendor-specific.

So the JEDEC ID is how you discover *which* flash is wired up without
prior knowledge — Phase 3 will use it to look the part up in a chip
database and pull its page/sector sizes and command set.

A sane answer here (manufacturer `0x20` = Micron, the KCU105's config
flash) also confirms the whole proxy path end to end: the
`bscan_spi_xfer()` framing, the MSB-first bit order, and the TDO
read-latency skew.

### The transfer primitive

`bscan_spi_xfer(jc, t, tx, txlen, rx, rxlen)` in
`src/modules/bscan/bscan.c` performs one CS-framed transaction:
clock out `txlen` MOSI bytes, then read `rxlen` MISO bytes. It builds
the quartiq/OpenOCD jtagspi DR frame
(`marker | bit-count | MOSI | latency-skip | MISO`) and matches
OpenOCD's `src/flash/nor/jtagspi.c` so the same bitstreams work. Generic
flash `read`/`erase`/`program`/`verify` (Phase 3) will be built on top
of this primitive.

## Programming via SVF (Lattice, Microsemi, …)

The BSCAN-proxy path above is Xilinx-specific (external SPI config flash).
For everything else, the universal path is to play an **SVF** file
exported by the vendor tool — Libero / FlashPro Express (Microsemi),
Diamond / Radiant (Lattice), Vivado (Xilinx fabric), Quartus, … The
vendor bakes the programming *algorithm* into the SVF; `bs_explorer` just
replays its `SIR`/`SDR`/`RUNTEST` vectors and checks the masked `TDO`
compares that flag a failed erase/program.

```
bs_explorer> jtag_open 0 flashpro     # the probe profile for your kit
bs_explorer> jtag_autoinit            # target_info should show 'prog: svf'
bs_explorer> svf_play design.svf
...
SVF done: 1342 commands, 1338 scans, 71 compares
```

A `TDO` mismatch stops play and points at the failing vector:

```
ERROR : line 842: SDR TDO mismatch at bit 17 (len 696)
```

— usually a wrong device, a too-fast clock, or a part that isn't
erased/unlocked.

You can sanity-check the player without a programming file using a tiny
hand-written IDCODE check (after `STATE RESET` the TAP auto-loads IDCODE
into DR):

```
! idcode.svf — masked IDCODE check (top nibble = revision)
STATE RESET;
SDR 32 TDO (0F8031CF) MASK (0FFFFFFF);
```

```
bs_explorer> svf_play idcode.svf
SVF done: 2 commands, 1 scans, 1 compares
```

**Supported subset (single-device chain):** `SIR`/`SDR` with
`TDI`/`TDO`/`MASK`/`SMASK` and the masked compare; `RUNTEST` (TCK/SCK
counts and SEC delays); `STATE` (RESET/IDLE); `ENDIR`/`ENDDR` (IDLE only);
`HIR`/`HDR`/`TIR`/`TDR` (length 0 only); `TRST`; `FREQUENCY`. SMASK is
parsed but not applied. Multi-device headers/trailers (non-zero) and
non-IDLE end states are rejected with a clear error.

**Generating the SVF is the closed step.** There is no open-source
bitstream/SVF generator for Microsemi (or most vendors) — you need the
vendor tool to *produce* the SVF (Libero has a free tier covering the
small IGLOO2 parts). `bs_explorer` only *plays* it, which is fully open.

## One command: `program`

Rather than remember which backend a part uses, `program <dev> <file>`
dispatches on the device's registry `prog` method:

```
bs_explorer> jtag_autoinit
bs_explorer> program 0 design.svf     # prog=svf  -> plays the SVF
```

`svf` plays the file; `proxy_spi` points you at the flash workflow
(`bscan_load_bitstream` + `flash_write`/`flash_verify`); `arm_flash`
routes to the ARM backend.

### CPU targets (ARM7/9) — structure only

The registry also describes **CPUs** (`kind: cpu`): an ARM debug
transport (`debug: embeddedice`), work-RAM and an on-chip flash region.
`target_list` shows them and `program` routes `prog: arm_flash` to the
ARM backend — but that backend (halt the core over JTAG, load a RAM
flasher, program internal flash) is **not implemented yet**. An Olimex
ARM-USB-OCD is an FT2232, so it opens with the existing FTDI driver via
the `arm-usb-ocd` probe profile. See the ARM-debug design note in
`CLAUDE.md`.

## Troubleshooting cheat sheet

| Symptom | Likely cause |
|---------|--------------|
| `jtag_probes` returns nothing | FTDI not enumerated. Check `lsusb`, udev permissions, conflicting process. |
| `jtag_autoinit` finds 0 devices | TDI/TDO swap, TRST held low, voltage mismatch, or chain broken. |
| All IDCODEs read `0xFFFFFFFF` | TDO floats high — broken TDO link, wrong voltage reference, or a Digilent SMT2 module being driven via raw FTDI MPSSE (use the Digilent backend instead). |
| All IDCODEs read `0x00000000` | TDO tied low or no clock reaching the target. |
| `target_info` says "not in registry" | Add an entry to `data/targets.yaml`. |
| `bscan_shift_dr 32` doesn't return the expected IDCODE | Wrong IR opcode/length, wrong device index, or a multi-device chain (current primitives assume single device). |
| `jtag_spi_xfer` is hopelessly slow | That's expected via EXTEST — switch to BSCAN proxy (Phase 2.5). |
| Detected fine, then reads turn to garbage / `0x00000000` mid-session | Target board lost power — JTAG floats (the USB probe stays enumerated regardless). Re-power the board. |
| FT4232H FlashPro: `jtag_scan` finds 0 devices | JTAG is on channel A (index 0) and needs `ADBUS4` high-Z — open with the profile: `jtag_open 0 flashpro`. |
| `svf_play` mismatches only on the very first compare | FTDI link warm-up; `svf_play` handles it, but a bare `bscan_shift_dr` straight after `jtag_open` may need a `jtag_scan` first. |

## Where to go from here

- `help` in the REPL lists all commands; `help <cmd>` gives details.
- `CLAUDE.md` at the repo root captures the architecture, roadmap and
  technical decisions (machine-independent).
- `README.md` is the user-facing summary.
- The original Viveris library and its docs live untouched under
  `src/modules/jtag_core/`, `src/modules/bsdl_parser/`, `src/modules/bus_over_jtag/`.