bs_explorer/CLAUDE.md

Project guide for Claude Code

This file is loaded automatically when working on bs_explorer in any Claude Code session, on any machine where the repo is cloned. Keep machine-specific or user-specific facts out of here — they belong in ~/.claude/ memory, not in the repo.

Project status, decisions, and roadmap below are durable: update them when reality changes, not for every transient task.

What this project is

bs_explorer is an application layer on top of Viveris's jtag-boundary-scanner library (LGPL). End goal: program SPI configuration memories attached to FPGAs (Xilinx KU15P first, then others) over JTAG, from a CLI tool running on a host with an FTDI probe.

The Viveris library itself lives unchanged in modules/. Everything new is in bs/ (the REPL) and future modules (fpga/, bscan_spi/, spi_flash/) sitting alongside the Viveris ones.

Architecture

bs/                  Application (readline REPL, no business logic)
modules/             — Viveris's library (LGPL, unchanged) —
├── jtag_core/       TAP state machine, IR/DR shifts
├── bsdl_parser/     .bsd loader
├── bus_over_jtag/   SPI/I²C/MDIO/parallel mem bit-bang over EXTEST
├── drivers/         FTDI, J-Link, Linux GPIO, LPT, Digilent (optional)
├── script/          Script engine (40+ commands, the real UI)
├── config/          Built-in config.script
├── os_interface/    Portable fs/network wrappers
└── natsort/         Natural pin-name sorting
bsdl_files/          BSDL files for target FPGAs
scripts/             Example scripts
doc/                 Tutorial and longer-form docs (doc/tutorial.md is the end-to-end walkthrough)
libs/libftd2xx/      Vendored FTDI SDK

User interacts with the project through the script engine commands (jtag_*, set, if, …), exposed via the REPL or piped script files. Adding a feature usually means adding a new script command in modules/script/script.c and the supporting C in a new module.

Roadmap

Phase	Module	Status	Summary
1	bs/ cleanup, REPL polish, README	done (commit 7cb3627)	Fix format-strings, delete dead code, tab-completion, banner
2	fpga/	done (commit 545fe09)	Per-target descriptor (IDCODE, BSDL, IR codes, proxy path, caveats). Compile-time registry.
2.5	bscan_spi/	done (commit dec0d14)	Load BSCAN proxy bitstream via CFG_IN, expose fast bscan_spi_xfer() via USER1. Required for realistic flashing speeds.
3	spi_flash/	done (commit c4afe87)	Chip DB (JEDEC ID → page/sector/cmd set) + generic read/erase/program/verify over an xfer callback. detect+read validated on KCU105; erase/program implemented but not yet hardware-tested.
4	script commands	done (commit d6f843e)	flash_detect, flash_read (+file), flash_erase, flash_write, flash_verify. Full set validated on KCU105 (save/erase/write-random/verify/restore round-trip). ~100 KB/s write once the proxy is loaded.

Move forward phase by phase: validate one with the user before starting the next. Don't break the validated path jtag_open → jtag_autoinit → jtag_mode 0 EXTEST while refactoring.

Key technical decisions

Flashing path: BSCAN proxy, not EXTEST

The Viveris library has bus_over_jtag/spi_over_jtag.c which bit-bangs SPI on FPGA pins placed in EXTEST. This is not usable for actual flashing. Each SPI bit costs 2 full boundary-scan-register shifts through JTAG, plus USB latency on the FTDI side. Order of magnitude: ~30 bytes/s effective, i.e. weeks to flash a 128 MB part. Useful only for one-shot operations (read JEDEC ID, check wiring).

Real flashing path: load a small "BSCAN proxy" bitstream into the FPGA fabric via standard JTAG configuration (CFG_IN). The proxy uses a BSCANE2 primitive to bridge the JTAG USER1 instruction's DR shift to the physical SPI pins, running at fabric speed (CCLK driven internally — the STARTUPE3 problem disappears). Realistic throughput 50–200 KB/s, so ~10–40 min for 128 MB. This is what Vivado, OpenOCD's jtagspi driver, and xc3sprog all do.

Per-FPGA descriptor

fpga_target struct (Phase 2) holds the per-target facts that can't be derived from the BSDL alone:

• idcode + idcode_mask — match the device on the chain
• bsdl_filename — BSDL to auto-load
• cfg_in_ir_code, user1_ir_code, jprogram_ir_code — Xilinx-specific private IR opcodes (read from BSDL when available)
• proxy_bitstream_path — path to the BSCAN proxy .bit for this part
• caveats — flags for known hardware gotchas (e.g. CCLK via STARTUPE3)

Registry is a runtime YAML file (fpga_registry.yaml at the repo root), parsed via libyaml — no longer a compile-time array. It is looked up CWD-relative (like bsdl_files/), overridable with $BS_FPGA_REGISTRY, and loaded lazily on first access. Adding a part = one YAML entry + its .bsd in bsdl_files/ + (optionally) its proxy .bit in bscan_proxies/ — no rebuild. The family field accepts xilinx_*, microsemi_igloo2/smartfusion2, lattice_machxo2/3 (enum in fpga.h).

Digilent SMT2 modules need libdjtg, not raw MPSSE

Several Xilinx dev boards (KCU105, ZCU102, …) embed a Digilent JTAG-SMT2 / SMT2-NC for USB-JTAG. Even though it presents a stock FTDI FT232H over USB (VID:PID 0403:6014), it runs a proprietary firmware that does not respond to plain MPSSE commands — TCK toggles but the level-shifters/buffers stay disabled, so TDO floats high ("all ones" symptom). Standard FTDI driver path is dead on these boards.

Workaround: modules/drivers/digilent_jtag/ wraps libdjtg/libdmgr (Digilent Adept Runtime), loaded via dlopen at runtime — no Digilent binary or header in the repo. Because it's dlopen'd (degrades to "no probe" if the libs are absent), it costs nothing to build in: BS_ENABLE_DIGILENT defaults ON on UNIX (<dlfcn.h> required), disable with -DBS_ENABLE_DIGILENT=OFF. Adept Runtime is only needed at runtime to actually drive such a probe.

Probe config profiles (probes.yaml)

Probe wiring/electrical settings live in probes.yaml (parsed by modules/probes/, libyaml), layered on top of the built-in config.script defaults: a defaults: map applied on every jtag_open (so opening without a profile is deterministic) plus named profiles: selected with jtag_open <idx> <profile> (jtag_profiles lists them). Each key/value is pushed into the script envvar store the driver reads at open time. The mechanism is driver-agnostic (any set-able probe var), but today only the FTDI driver (and minimally the Linux-GPIO one) reads config envvars, so profiles mostly tune FTDI. Motivating case: the embedded FlashPro on Microsemi kits (FT4232H ch.A) needs ADBUS4 high-Z — the flashpro profile sets PROBE_FTDI_SET_PIN_DIR_ADBUS4: 0. See the config-strategy design note below for where this is headed.

Xilinx caveats

On 7-Series / UltraScale / UltraScale+, CCLK is not a regular I/O pin — it goes through the STARTUPE3 primitive. Cannot be driven directly in EXTEST. Non-issue once we use the BSCAN proxy (CCLK is driven by the fabric clock inside the proxy).

The FPGA must be in a configurable state before loading the proxy. Issue JPROGRAM first to reset, then CFG_IN + shift the bitstream, then JSTART and check DONE.

Probe / JTAG-link / device config strategy (design note)

Partly built, partly planned. Guiding idea: separate three concerns that are conflated today, and express the shared JTAG-link settings in driver-neutral terms that get resolved per session.

The three concerns

Layer	What it owns	Where it lives	Applied
Probe (sonde)	driver + interface, pin map, buffer-enable, TRST/SRST pins, level-shift, max TCK the adapter supports	probes.yaml (done)	at jtag_open
JTAG link	TCK freq, RTCK, reset behaviour, chain layout — driver-neutral names, resolved to effective values	missing today	open, then refined after detect
Device	IDCODE/BSDL/IR/proxy/caveats, programming method, max TCK the part/board tolerates	fpga_registry.yaml (done)	after IDCODE match

The smell that motivated this

Frequency has no single home. Only the FTDI driver reads PROBE_FTDI_TCK_FREQ_KHZ; our Digilent driver hardcodes 4 MHz (digilent_jtag_drv.c); J-Link/LPT read nothing. And the PROBE_FTDI_* namespace mixes probe wiring (pin map, ADBUS4) with link properties (freq, RTCK, TRST timing). Frequency isn't an FTDI fact — it's a link fact bounded by both the probe and the board/device.

Target model

• Driver-neutral link vars (tck_khz, rtck, reset, …) set once (by user / probe defaults / device). Each driver consumes them: our Digilent driver reads them directly; the Viveris FTDI driver is fed PROBE_FTDI_TCK_FREQ_KHZ via a thin translation shim at open (no Viveris edit). J-Link/LPT can stay on their defaults.
• Resolution: effective tck = min(user request, probe max, device/ board max). A small session step computes it at open, and re-applies after jtag_autoinit once the device (hence its cap / programming method) is known — using the jtag_close→reopen seam if a re-init is needed. This also dissolves the chicken-and-egg: a conservative link default gets you to detection, then the device refines it.
• probes.yaml gains an optional max_tck_khz; fpga_registry.yaml gains optional max_tck_khz + a prog method tag (proxy_spi/svf).

Phasing

• A (done) — one canonical JTAG_TCK_FREQ_KHZ (kHz): mirrored to PROBE_FTDI_TCK_FREQ_KHZ at jtag_open for the Viveris FTDI driver, read directly by our Digilent driver; unset → each driver's own default (FTDI 1000, Digilent 4000). Set it via set, a probes.yaml profile, or defaults:. (FTDI path hardware-validated; Digilent path untested.)
• B (done) — registry max_tck_khz caps the clock: jtag_autoinit, after identifying the chain, clamps JTAG_TCK_FREQ_KHZ to the smallest device max and re-opens the probe once (via the stored probe id) to apply, then re-scans. Within-cap / unset just report the cap. Effective tck = min(request, device max); probe-max (min(..., probe max)) is still TODO.
• C — generalise the other link settings (reset/RTCK) and wire the prog method tag into backend dispatch (ties into the SVF player).

What exists already: the probe layer (probes.yaml) and the device layer (fpga_registry.yaml). The new work is the JTAG-link layer in the middle.

Programming backends: beyond Xilinx external flash (design note)

Not yet implemented — captured so the design is ready. Guiding vision: one generic SVF player as a near-universal backend, with native backends only where streaming / speed / control justify them.

Why two layers

Practically every JTAG-programmable device (Xilinx fabric config, Lattice MachXO2/3, Microsemi IGLOO2/SmartFusion2, Altera, CPLDs, …) can be programmed from an SVF file exported by its vendor tool. SVF is a flat list of SIR/SDR/RUNTEST ops with the vendor's algorithm already baked in — so a single SVF player, built on our existing bscan_set_ir / bscan_shift_dr primitives, programs almost anything with no per-vendor algorithm code.

Native backends are the exception, justified only when you want fine control, speed, or no vendor-export step. Our Xilinx external SPI flash path is the prime example and stays native: load the BSCAN proxy once, then stream raw binary with separate read/erase/program/verify, progress and partial ops. The equivalent SVF ("indirect flash") is huge (tens of MB of ASCII) and inflexible.

Where each path applies

Target	Recommended path
Xilinx external SPI config flash	native proxy (bscan_spi/+spi_flash/, done). SVF works but is bloated.
Xilinx fabric config (volatile)	SVF, or our bscan_load_bitstream (equivalent)
Lattice MachXO2/3 (internal flash)	SVF (Diamond/Radiant export). Native IEEE-1532 ISC optional, only for a self-contained .jed flow.
Microsemi IGLOO2 / SmartFusion2	SVF/STAPL (Libero/FlashPro export). Native algorithm too complex/proprietary.
Altera, CPLDs, misc JTAG	SVF

The SVF player (the real work)

A player is more than shifting bits. It must handle:

• SIR/SDR with TDI/TDO/MASK — the masked TDO compare is what detects prog/erase failures; that's the main addition over today's primitives;
• RUNTEST delays (erase/program waits, in TCK or time) → reuse bscan_idle_cycles + a sleep;
• HIR/HDR/TIR/TDR headers/trailers and ENDIR/ENDDR/STATE (multi- device chains, TAP state) — a standard subset covers ~all files.

It reuses probe drivers, jtag_core, bscan_set_ir/bscan_shift_dr and IDCODE/BSDL detection unchanged. STAPL (.stp/.jam) is the richer native format (a full language → needs an interpreter); more work, defer — do SVF first.

Registry / dispatch

fpga_target is Xilinx-flavoured today (ir_cfg_in, ir_user1, …). Add a "programming method" tag (proxy_spi / svf / optional lattice_isc) selected off the family enum (add FPGA_FAMILY_LATTICE_*, FPGA_FAMILY_MICROSEMI_*). An SVF player is family-agnostic and needs no per-part opcodes — it can even run without a registry entry.

Scope caveat. iCE40 is usually programmed over SPI directly (or one-time NVCM) with minimal JTAG — out of scope for this JTAG-centric tool.

Embedded port (design note)

Not yet implemented — captured for a possible standalone programmer. No hard blocker: standalone JTAG programmers are all MCU-based. The architecture has the right seam (the driver function-pointer table).

Reference target: Arduino GIGA R1 WiFi — STM32H747XI (Cortex-M7 @ 480 MHz + M4 @ 240 MHz, 1 MB RAM, 2 MB flash), WiFi/BT, Arduino/Mbed OS. The comfortable high end. ESP32 is a lighter alternative (WiFi on-die, ~520 KB RAM); avoid bare AVR (RAM too small).

The seam — the MCU is the JTAG master. Replace modules/drivers/ with a GPIO bit-bang (or SPI-assisted: MOSI=TDI, MISO=TDO, SCK=TCK, TMS bit-banged between scans → multi-MHz) driver behind drv_TX_TMS / drv_TXRX_DATA. Everything above that — jtag_core, the bscan_set_ir/bscan_shift_dr primitives, the fpga_target registry, the SVF player and the flash logic — is portable C, reused as-is.

Dropped / replaced on the MCU:

• readline REPL → command interface over WiFi (HTTP/telnet) or USB-CDC;
• os_interface (host fs/net) → Mbed OS filesystem + network (real implementations, little rework);
• runtime BSDL parsing → pre-baked tables (the registry already does this); bsdl_parser can be left out;
• libftd2xx / Digilent dlopen → gone (the MCU is the probe).

The one real rework: stream, don't malloc. bscan_shift_dr currently allocates the whole shift buffer (malloc(nbits) — ~19 MB for a Xilinx proxy). On an MCU the payload (proxy .bit, flash image, SVF) lives on SD or a USB stick and must be shifted in chunks; WiFi fetches the file / triggers the job. With an SD/USB source this is natural (read a block, shift, repeat). 1 MB RAM relaxes buffer sizes but does not remove the need (a 32 MB image never fits).

Ideal fit: an SVF player streamed from SD — a classic embedded programmer design, and it matches the small-Lattice/IGLOO2-in-a-PSU use case (small config, modest SDR vectors). Big Xilinx external-flash images (huge SVF vectors) stay on the host or a large MCU.

Hardware notes:

• the GIGA R1 has no onboard microSD — add it via the GIGA Display Shield, an SPI SD module, or its USB-A host port (USB mass storage);
• 3.3 V GPIO → level-shift for 1.8 V JTAG targets (e.g. KCU105), same as on the host side.

External references

• BSCAN proxy bitstreams: quartiq/bscan_spi_bitstreams (MIT). Pre-built .bit for many Xilinx parts; Migen sources to rebuild any part that's missing (needs Vivado). Building a proxy that isn't pre-built (e.g. the KU15P) is covered in doc/tutorial.md, Phase 2.5.
• Reference host-side implementation: openocd/src/flash/nor/jtagspi.c. Defines the proxy protocol (header with bit count + CS state, then payload). Don't reinvent — match what OpenOCD does so we share the same bitstreams.
• Viveris library docs: modules/jtag_core/ headers are well commented. modules/config/config.script documents every runtime variable.

Build & test

mkdir build && cd build && cmake .. && make
./bs/bs           # interactive REPL

Build needs libyaml (pkg-config yaml-0.1; Arch libyaml, Debian libyaml-dev) — the FPGA registry is parsed from fpga_registry.yaml at runtime. Run bs from the repo root so it finds that file (and bsdl_files/, bscan_proxies/), or point $BS_FPGA_REGISTRY at it.

The Digilent SMT2 backend is built by default on UNIX (disable with -DBS_ENABLE_DIGILENT=OFF). To actually use such a probe, install the Adept Runtime system-wide (provides libdjtg.so + libdmgr.so).

No automated tests yet. Smoke test = banner appears, exit works. After changes touching jtag_core, drivers/ftdi_jtag, or the autoinit flow, manual hardware test required: probe + KU15P board should scan and load xcku15p_ffve1517.bsd.

The FTDI driver compiles with int-to-pointer-cast warnings (Viveris's code on 64-bit Linux). Don't fix in this repo — that's upstream.

Commit conventions

• Messages in English, lowercase first word, imperative or short descriptive (match existing style: "phase 1: cleanup, REPL polish, README", "translate README to English").
• Title ≤ ~70 chars. Body wrapped ~80 chars, with bullets for what changed. Mention the why for non-obvious choices.
• Sign-off block:

  Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
  

• One logical change per commit. Don't bundle README + bugfix + refactor.

What does NOT belong here

• User preferences (tone, language used in chat, "always do X first") → ~/.claude/
• Machine-local facts (which probes are physically attached, paths outside the repo, validated-on-this-machine notes) → ~/.claude/
• Transient task state, in-progress decisions → conversation, not file
• Generated artifacts (build/, downloaded proxy .bit if we choose not to vendor) → .gitignore