bs_explorer/CLAUDE.md

Project guide for Claude Code

This file is loaded automatically when working on bs_explorer in any Claude Code session, on any machine where the repo is cloned. Keep machine-specific or user-specific facts out of here — they belong in ~/.claude/ memory, not in the repo.

Project status, decisions, and roadmap below are durable: update them when reality changes, not for every transient task.

What this project is

bs_explorer is an application layer on top of Viveris's jtag-boundary-scanner library (LGPL). End goal: program FPGAs/CPLDs over JTAG from a CLI tool on a host with an FTDI/Digilent/J-Link probe — Xilinx external SPI configuration flash via a BSCAN proxy (started with the KU15P), and other families (Lattice, Microsemi, …) by playing a vendor-exported SVF.

The Viveris library itself lives unchanged in src/modules/. Everything new is in src/bs/ (the REPL) and the project modules (target/, bscan/, spi_flash/, svf/, probes/, program/, arm_debug/) sitting alongside the Viveris ones.

Architecture

src/
├── bs/                    Application (readline REPL, no business logic)
└── modules/
    — Viveris's library (LGPL, unchanged) —
    ├── jtag_core/         TAP state machine, IR/DR shifts
    ├── bsdl_parser/       .bsd loader
    ├── bus_over_jtag/     SPI/I²C/MDIO/parallel mem bit-bang over EXTEST
    ├── drivers/           FTDI, J-Link, Linux GPIO, LPT, Digilent (optional, dlopen)
    ├── script/            Script engine (the real UI)
    ├── config/            Built-in config.script
    ├── os_interface/      Portable fs/network wrappers
    └── natsort/           Natural pin-name sorting
    — new (this project) —
    ├── target/            Target registry: FPGAs + CPUs (parses data/targets.yaml)
    ├── bscan/             JTAG TAP primitives (set_ir/shift_ir/shift_dr/tap_reset/
    │                      idle_cycles) + BSCAN proxy (bitstream load, SPI-over-USER1)
    ├── spi_flash/         SPI NOR chip DB + read/erase/program/verify over a callback
    ├── svf/               SVF player (svf_play): SIR/SDR/RUNTEST/STATE, masked compare
    ├── probes/            Probe-config profiles loader (parses data/probes.yaml, libyaml)
    ├── program/           `program` dispatch: routes a target to its backend by `prog`
    └── arm_debug/         ARM (EmbeddedICE) debug + flash backend (not implemented yet)
data/                      — runtime resources, looked up CWD-relative —
├── targets.yaml           Target registry (FPGAs + CPUs: IDCODE, BSDL/proxy, debug, flash, prog)
├── probes.yaml            Probe-config profiles (defaults + per-probe overrides)
├── bsdl_files/            BSDL files for target FPGAs
├── bscan_proxies/         BSCAN proxy bitstreams (MIT, from quartiq)
└── scripts/               Example scripts
doc/                       Tutorial and longer-form docs (doc/tutorial.md is the end-to-end walkthrough)
CMakeLists.txt             Top-level build (binary lands at build/bs)

User interacts with the project through the script engine commands (jtag_*, set, if, …), exposed via the REPL or piped script files. Adding a feature usually means adding a new script command in src/modules/script/script.c and the supporting C in a new module.

Roadmap

Phase	Module	Status	Summary
1	bs/ cleanup, REPL polish, README	done (commit 7cb3627)	Fix format-strings, delete dead code, tab-completion, banner
2	fpga/	done (commit 545fe09)	Per-target descriptor (IDCODE, BSDL, IR codes, proxy path, caveats). Now a runtime YAML registry (data/targets.yaml, libyaml), later gaining prog method + max_tck_khz.
2.5	bscan/	done (commit dec0d14)	Load BSCAN proxy bitstream via CFG_IN, expose fast bscan_spi_xfer() via USER1. Required for realistic flashing speeds.
3	spi_flash/	done (commit c4afe87)	Chip DB (JEDEC ID → page/sector/cmd set) + generic read/erase/program/verify over an xfer callback. detect+read validated on KCU105; erase/program implemented but not yet hardware-tested.
4	script commands	done (commit d6f843e)	flash_detect, flash_read (+file), flash_erase, flash_write, flash_verify. Full set validated on KCU105 (save/erase/write-random/verify/restore round-trip). ~100 KB/s write once the proxy is loaded.
5	probes/ + JTAG-link	done	data/probes.yaml probe-config profiles (jtag_open <idx> <profile>, jtag_profiles, jtag_close); driver-neutral JTAG_TCK_FREQ_KHZ/JTAG_RTCK; device max_tck_khz clock cap resolved at jtag_autoinit; prog method tag. See the config-strategy design note. Validated on the IGLOO2 (FlashPro).
6	svf/	done (subset, commit c77d86e)	SVF player + svf_play: SIR/SDR with masked TDO compare, RUNTEST, STATE — single-device. Validated on the IGLOO2 IDCODE.
7	target/ + program/ + arm_debug/	structure done; ARM impl TODO	Generalized fpga/ into a kind-aware target/ registry (FPGA | CPU). program <dev> <file> dispatches by prog (svf wired; proxy_spi points at the flash workflow). arm_debug/ (EmbeddedICE) + arm_flash backend are declared but not implemented; arm-usb-ocd probe profile added. FPGA path re-validated on the IGLOO2. See the ARM-debug design note.

| 8 | FTDI driver → libftdi1 | done | Replaced the proprietary libftd2xx with open-source libftdi1 (libusb): any VID:PID + auto kernel-detach. Detected an NXP LPC2103 (ARM7TDMI-S, IDCODE 0x4F1F0F0F) over an Olimex ARM-USB-OCD — the probe the old lib couldn't enumerate. Vendored src/libs/libftd2xx removed. |

Move forward phase by phase: validate one with the user before starting the next. Don't break the validated path jtag_open → jtag_autoinit → jtag_mode 0 EXTEST while refactoring.

Key technical decisions

Flashing path: BSCAN proxy, not EXTEST

The Viveris library has bus_over_jtag/spi_over_jtag.c which bit-bangs SPI on FPGA pins placed in EXTEST. This is not usable for actual flashing. Each SPI bit costs 2 full boundary-scan-register shifts through JTAG, plus USB latency on the FTDI side. Order of magnitude: ~30 bytes/s effective, i.e. weeks to flash a 128 MB part. Useful only for one-shot operations (read JEDEC ID, check wiring).

Real flashing path: load a small "BSCAN proxy" bitstream into the FPGA fabric via standard JTAG configuration (CFG_IN). The proxy uses a BSCANE2 primitive to bridge the JTAG USER1 instruction's DR shift to the physical SPI pins, running at fabric speed (CCLK driven internally — the STARTUPE3 problem disappears). Realistic throughput 50–200 KB/s, so ~10–40 min for 128 MB. This is what Vivado, OpenOCD's jtagspi driver, and xc3sprog all do.

Per-target descriptor

The jtag_target struct (target/target.h) holds the per-target facts that can't be derived from the IDCODE/BSDL alone. A kind (fpga|cpu) selects a sub-struct; the rest is shared:

• common — idcode+idcode_mask (chain match), family, ir_length, prog (programming backend), max_tck_khz
• fpga sub-struct — bsdl_filename, the Xilinx config IR opcodes (cfg_in/user1/jprogram/…), proxy_bitstream, caveats
• cpu sub-struct — debug transport (EmbeddedICE), work-RAM (ram_base/size) and on-chip flash region (flash_base/size)

Registry is a runtime YAML file (data/targets.yaml), parsed via libyaml — looked up CWD-relative (like data/bsdl_files/), overridable with $BS_TARGETS, loaded lazily. Adding a target = one flat YAML entry (+ a .bsd/proxy for FPGAs) — no rebuild. family accepts xilinx_*, microsemi_*, lattice_*, arm7/arm9; prog is proxy_spi/svf/arm_flash/none (inferred when omitted). Enums in target.h.

FTDI driver on libftdi1, not the proprietary libftd2xx

The FTDI MPSSE driver (drivers/ftdi_jtag/) runs on open-source libftdi1 (libusb), not FTDI's vendored libftd2xx.a. Why: libftdi1 opens any USB VID:PID (so probes with custom ids — Olimex ARM-USB-OCD 0x15ba:0x0003, etc. — enumerate with no FT_SetVIDPID tricks) and auto-detaches the kernel ftdi_sio driver. The MPSSE command construction, pin map and clocking are unchanged; only the transport (enumerate/open/read/write/bitmode) moved to ftdi_* calls. No proprietary binary in the repo. Build dep: pkg-config libftdi1 + libusb-1.0. Validated by detecting an LPC2103 over an ARM-USB-OCD — the probe the old lib couldn't even enumerate.

Digilent SMT2 modules need libdjtg, not raw MPSSE

Several Xilinx dev boards (KCU105, ZCU102, …) embed a Digilent JTAG-SMT2 / SMT2-NC for USB-JTAG. Even though it presents a stock FTDI FT232H over USB (VID:PID 0403:6014), it runs a proprietary firmware that does not respond to plain MPSSE commands — TCK toggles but the level-shifters/buffers stay disabled, so TDO floats high ("all ones" symptom). Standard FTDI driver path is dead on these boards.

Workaround: src/modules/drivers/digilent_jtag/ wraps libdjtg/libdmgr (Digilent Adept Runtime), loaded via dlopen at runtime — no Digilent binary or header in the repo. Because it's dlopen'd (degrades to "no probe" if the libs are absent), it costs nothing to build in: BS_ENABLE_DIGILENT defaults ON on UNIX (<dlfcn.h> required), disable with -DBS_ENABLE_DIGILENT=OFF. Adept Runtime is only needed at runtime to actually drive such a probe.

Probe config profiles (data/probes.yaml)

Probe wiring/electrical settings live in data/probes.yaml (parsed by src/modules/probes/, libyaml), layered on top of the built-in config.script defaults: a defaults: map applied on every jtag_open (so opening without a profile is deterministic) plus named profiles: selected with jtag_open <idx> <profile> (jtag_profiles lists them). Each key/value is pushed into the script envvar store the driver reads at open time. The mechanism is driver-agnostic (any set-able probe var), but today only the FTDI driver (and minimally the Linux-GPIO one) reads config envvars, so profiles mostly tune FTDI. Motivating case: the embedded FlashPro on Microsemi kits (FT4232H ch.A) needs ADBUS4 high-Z — the flashpro profile sets PROBE_FTDI_SET_PIN_DIR_ADBUS4: 0. See the config-strategy design note below for where this is headed.

Xilinx caveats

On 7-Series / UltraScale / UltraScale+, CCLK is not a regular I/O pin — it goes through the STARTUPE3 primitive. Cannot be driven directly in EXTEST. Non-issue once we use the BSCAN proxy (CCLK is driven by the fabric clock inside the proxy).

The FPGA must be in a configurable state before loading the proxy. Issue JPROGRAM first to reset, then CFG_IN + shift the bitstream, then JSTART and check DONE.

Probe / JTAG-link / device config strategy (design note)

Partly built, partly planned. Guiding idea: separate three concerns that are conflated today, and express the shared JTAG-link settings in driver-neutral terms that get resolved per session.

The three concerns

Layer	What it owns	Where it lives	Applied
Probe (sonde)	driver + interface, pin map, buffer-enable, TRST/SRST pins, level-shift, max TCK the adapter supports	data/probes.yaml (done)	at jtag_open
JTAG link	TCK freq, RTCK, reset behaviour, chain layout — driver-neutral names, resolved to effective values	missing today	open, then refined after detect
Device	IDCODE/BSDL/IR/proxy/caveats, programming method, max TCK the part/board tolerates	data/targets.yaml (done)	after IDCODE match

The smell that motivated this

Frequency has no single home. Only the FTDI driver reads PROBE_FTDI_TCK_FREQ_KHZ; our Digilent driver hardcodes 4 MHz (digilent_jtag_drv.c); J-Link/LPT read nothing. And the PROBE_FTDI_* namespace mixes probe wiring (pin map, ADBUS4) with link properties (freq, RTCK, TRST timing). Frequency isn't an FTDI fact — it's a link fact bounded by both the probe and the board/device.

Target model

• Driver-neutral link vars (tck_khz, rtck, reset, …) set once (by user / probe defaults / device). Each driver consumes them: our Digilent driver reads them directly; the Viveris FTDI driver is fed PROBE_FTDI_TCK_FREQ_KHZ via a thin translation shim at open (no Viveris edit). J-Link/LPT can stay on their defaults.
• Resolution: effective tck = min(user request, probe max, device/ board max). A small session step computes it at open, and re-applies after jtag_autoinit once the device (hence its cap / programming method) is known — using the jtag_close→reopen seam if a re-init is needed. This also dissolves the chicken-and-egg: a conservative link default gets you to detection, then the device refines it.
• data/probes.yaml gains an optional max_tck_khz; data/targets.yaml gains optional max_tck_khz + a prog method tag (proxy_spi/svf).

Phasing

• A (done) — one canonical JTAG_TCK_FREQ_KHZ (kHz): mirrored to PROBE_FTDI_TCK_FREQ_KHZ at jtag_open for the Viveris FTDI driver, read directly by our Digilent driver; unset → each driver's own default (FTDI 1000, Digilent 4000). Set it via set, a data/probes.yaml profile, or defaults:. (FTDI path hardware-validated; Digilent path untested.)
• B (done) — registry max_tck_khz caps the clock: jtag_autoinit, after identifying the chain, clamps JTAG_TCK_FREQ_KHZ to the smallest device max and re-opens the probe once (via the stored probe id) to apply, then re-scans. Within-cap / unset just report the cap. Effective tck = min(request, device max); probe-max (min(..., probe max)) is still TODO.
• C (done) — prog method tag (proxy_spi/svf/none) on each registry entry, inferred when omitted (proxy → proxy_spi; Microsemi/ Lattice → svf); shown by target_info/target_list and available via fpga_prog_method_name() for backend dispatch. RTCK generalised as a neutral JTAG_RTCK (mirrored to PROBE_FTDI_JTAG_ENABLE_RTCK, FTDI-only). Reset abstraction deferred — it's a bundle of probe- specific pin/polarity/timing vars with no clean neutral form yet. The actual program dispatch command lands with the SVF player.

What exists already: the probe layer (data/probes.yaml) and the device layer (data/targets.yaml). The new work is the JTAG-link layer in the middle.

Programming backends: beyond Xilinx external flash (design note)

Not yet implemented — captured so the design is ready. Guiding vision: one generic SVF player as a near-universal backend, with native backends only where streaming / speed / control justify them.

Why two layers

Practically every JTAG-programmable device (Xilinx fabric config, Lattice MachXO2/3, Microsemi IGLOO2/SmartFusion2, Altera, CPLDs, …) can be programmed from an SVF file exported by its vendor tool. SVF is a flat list of SIR/SDR/RUNTEST ops with the vendor's algorithm already baked in — so a single SVF player, built on our existing bscan_set_ir / bscan_shift_dr primitives, programs almost anything with no per-vendor algorithm code.

Native backends are the exception, justified only when you want fine control, speed, or no vendor-export step. Our Xilinx external SPI flash path is the prime example and stays native: load the BSCAN proxy once, then stream raw binary with separate read/erase/program/verify, progress and partial ops. The equivalent SVF ("indirect flash") is huge (tens of MB of ASCII) and inflexible.

Where each path applies

Target	Recommended path
Xilinx external SPI config flash	native proxy (bscan/+spi_flash/, done). SVF works but is bloated.
Xilinx fabric config (volatile)	SVF, or our bscan_load_bitstream (equivalent)
Lattice MachXO2/3 (internal flash)	SVF (Diamond/Radiant export). Native IEEE-1532 ISC optional, only for a self-contained .jed flow.
Microsemi IGLOO2 / SmartFusion2	SVF/STAPL (Libero/FlashPro export). Native algorithm too complex/proprietary.
Altera, CPLDs, misc JTAG	SVF

The SVF player (the real work)

Status: initial implementation in src/modules/svf/ + the svf_play <file> command. Supports the single-device subset: SIR/SDR with TDI/TDO/MASK/SMASK and the masked TDO compare, RUNTEST (TCK/SCK + SEC), STATE (RESET/IDLE), ENDIR/ENDDR (IDLE only), HIR/HDR/TIR/TDR (length 0 only), TRST, FREQUENCY. Built on the bscan_* primitives (shift_ir/shift_dr/tap_reset/idle_cycles). Validated on the live IGLOO2 with a hand-written IDCODE-check SVF (pass and a deliberate mismatch). svf_play warms up the FTDI link first (its first data read after open returns stale FIFO content). Not yet wired into a generic program dispatch off the prog tag; no multi-device headers, no non-IDLE end states. Remaining design below.

A player is more than shifting bits. It must handle:

• SIR/SDR with TDI/TDO/MASK — the masked TDO compare is what detects prog/erase failures; that's the main addition over today's primitives;
• RUNTEST delays (erase/program waits, in TCK or time) → reuse bscan_idle_cycles + a sleep;
• HIR/HDR/TIR/TDR headers/trailers and ENDIR/ENDDR/STATE (multi- device chains, TAP state) — a standard subset covers ~all files.

It reuses probe drivers, jtag_core, bscan_set_ir/bscan_shift_dr and IDCODE/BSDL detection unchanged. STAPL (.stp/.jam) is the richer native format (a full language → needs an interpreter); more work, defer — do SVF first.

Registry / dispatch

fpga_target is Xilinx-flavoured today (ir_cfg_in, ir_user1, …). Add a "programming method" tag (proxy_spi / svf / optional lattice_isc) selected off the family enum (add FPGA_FAMILY_LATTICE_*, FPGA_FAMILY_MICROSEMI_*). An SVF player is family-agnostic and needs no per-part opcodes — it can even run without a registry entry.

Scope caveat. iCE40 is usually programmed over SPI directly (or one-time NVCM) with minimal JTAG — out of scope for this JTAG-centric tool.

Programming CPUs over JTAG: ARM7/9 via EmbeddedICE (design note)

Structure in place (target/ kind=cpu, program/ dispatch, arm_debug/

• arm_flash declared); the debug/flash code is the next real work.

Why CPUs are a different shape

A CPU isn't configured like an FPGA — there's no bitstream or BSDL pin map to drive. You halt the core through its debug unit, write a small flash loader into on-chip RAM, run it to program the internal flash, and verify — the OpenOCD approach. So the target carries different facts (captured in the cpu sub-struct): the debug transport, a work-RAM region for the loader, and the flash region.

The debug seam (arm_debug/)

ARM7/ARM9 expose EmbeddedICE on a JTAG TAP: a scan chain (DSCR, DCC, breakpoint regs) reached via the standard IR/DR shifts. The seam is four primitives — halt, resume, mem_read, mem_write — built on the existing bscan_* IR/DR functions; everything else (the RAM loader, the per-MCU flash algorithm) sits on top. Cortex-M is the same idea behind a different transport (ADIv5/DAP, and SWD which isn't JTAG); it would be a second debug value behind the same seam, not a rewrite.

Backend dispatch (program/)

program <dev> <file> looks the device up in the registry and routes by its prog tag: svf→SVF player, proxy_spi→the Xilinx flash workflow, arm_flash→arm_debug's RAM-loader flash. Adding a backend = one prog value + one case. This is the seam that makes new target classes drop in without touching the REPL.

Probe

ARM-USB-OCD is an Olimex FT2232 (OpenOCD-class) adapter — the existing FTDI driver drives it; it just needs a probes.yaml profile (arm-usb-ocd) with the Olimex control-pin map (TRST/SRST, buffer enable). The profile slot exists; the exact pin numbers still need filling from the Olimex schematic / OpenOCD's interface config.

What's left (the implementation)

EmbeddedICE scan-chain access + halt/resume + memory R/W, then a per-MCU RAM flash loader (LPC2xxx, AT91SAM7, …) and the arm_flash backend. The registry, dispatch, probe-profile and config layers are ready for it.

Embedded port (design note)

Not yet implemented — captured for a possible standalone programmer. No hard blocker: standalone JTAG programmers are all MCU-based. The architecture has the right seam (the driver function-pointer table).

Reference target: Arduino GIGA R1 WiFi — STM32H747XI (Cortex-M7 @ 480 MHz + M4 @ 240 MHz, 1 MB RAM, 2 MB flash), WiFi/BT, Arduino/Mbed OS. The comfortable high end. ESP32 is a lighter alternative (WiFi on-die, ~520 KB RAM); avoid bare AVR (RAM too small).

The seam — the MCU is the JTAG master. Replace src/modules/drivers/ with a GPIO bit-bang (or SPI-assisted: MOSI=TDI, MISO=TDO, SCK=TCK, TMS bit-banged between scans → multi-MHz) driver behind drv_TX_TMS / drv_TXRX_DATA. Everything above that — jtag_core, the bscan_set_ir/bscan_shift_dr primitives, the fpga_target registry, the SVF player and the flash logic — is portable C, reused as-is.

Dropped / replaced on the MCU:

• readline REPL → command interface over WiFi (HTTP/telnet) or USB-CDC;
• os_interface (host fs/net) → Mbed OS filesystem + network (real implementations, little rework);
• runtime BSDL parsing → pre-baked tables (the registry already does this); bsdl_parser can be left out;
• libftdi1 / Digilent → gone (the MCU is the probe).

The one real rework: stream, don't malloc. bscan_shift_dr currently allocates the whole shift buffer (malloc(nbits) — ~19 MB for a Xilinx proxy). On an MCU the payload (proxy .bit, flash image, SVF) lives on SD or a USB stick and must be shifted in chunks; WiFi fetches the file / triggers the job. With an SD/USB source this is natural (read a block, shift, repeat). 1 MB RAM relaxes buffer sizes but does not remove the need (a 32 MB image never fits).

Ideal fit: an SVF player streamed from SD — a classic embedded programmer design, and it matches the small-Lattice/IGLOO2-in-a-PSU use case (small config, modest SDR vectors). Big Xilinx external-flash images (huge SVF vectors) stay on the host or a large MCU.

Hardware notes:

• the GIGA R1 has no onboard microSD — add it via the GIGA Display Shield, an SPI SD module, or its USB-A host port (USB mass storage);
• 3.3 V GPIO → level-shift for 1.8 V JTAG targets (e.g. KCU105), same as on the host side.

External references

• BSCAN proxy bitstreams: quartiq/bscan_spi_bitstreams (MIT). Pre-built .bit for many Xilinx parts; Migen sources to rebuild any part that's missing (needs Vivado). Building a proxy that isn't pre-built (e.g. the KU15P) is covered in doc/tutorial.md, Phase 2.5.
• Reference host-side implementation: openocd/src/flash/nor/jtagspi.c. Defines the proxy protocol (header with bit count + CS state, then payload). Don't reinvent — match what OpenOCD does so we share the same bitstreams.
• Viveris library docs: src/modules/jtag_core/ headers are well commented. src/modules/config/config.script documents every runtime variable.

Build & test

mkdir build && cd build && cmake .. && make
./bs/bs           # interactive REPL

Build needs libyaml (pkg-config yaml-0.1; Arch libyaml, Debian libyaml-dev) for the registry, and libftdi1 + libusb-1.0 (pkg-config libftdi1; Arch libftdi, Debian libftdi1-dev) for the FTDI/Olimex driver. Run bs from the repo root so it finds data/targets.yaml (and data/bsdl_files/, data/bscan_proxies/), or point $BS_TARGETS at it.

The Digilent SMT2 backend is built by default on UNIX (disable with -DBS_ENABLE_DIGILENT=OFF). To actually use such a probe, install the Adept Runtime system-wide (provides libdjtg.so + libdmgr.so).

No automated tests yet. Smoke test = banner appears, exit works. After changes touching jtag_core, drivers/ftdi_jtag, or the autoinit flow, manual hardware test required: probe + KU15P board should scan and load xcku15p_ffve1517.bsd.

The FTDI driver compiles with int-to-pointer-cast warnings (Viveris's code on 64-bit Linux). Don't fix in this repo — that's upstream.

Commit conventions

• Messages in English, lowercase first word, imperative or short descriptive (match existing style: "phase 1: cleanup, REPL polish, README", "translate README to English").
• Title ≤ ~70 chars. Body wrapped ~80 chars, with bullets for what changed. Mention the why for non-obvious choices.
• Sign-off block:

  Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
  

• One logical change per commit. Don't bundle README + bugfix + refactor.

What does NOT belong here

• User preferences (tone, language used in chat, "always do X first") → ~/.claude/
• Machine-local facts (which probes are physically attached, paths outside the repo, validated-on-this-machine notes) → ~/.claude/
• Transient task state, in-progress decisions → conversation, not file
• Generated artifacts (build/, downloaded proxy .bit if we choose not to vendor) → .gitignore