Booting Linux on a Custom Processor on ZCU104

Background

The ZCU104 board is a combination of FPGA and an ARM-based MPSoC. The FPGA part of ZCU104 is tightly bonded to the MPSoC making many of the peripherals on ZCU104 usable only via ARM core. This feature gives an advantage when we are using both MPSoC and FPGA parts, but it turns into a disadvantage when we want to use just the PL (stand-alone PL logic) part of ZCU104 without any intervention of MPSoC. Because of this reason, most of the important peripherals on ZCU104 become inaccessible to the stand-alone PL logic. Booting a custom processor on the PL part is one such application.

Our Use case and application

For our application, we needed a RISC-V based custom SoC in the PL part, that too stand- alone, with no usage of ARM-based MPSoC.

The SoC contains several other modules like the DLA, I2C, UART, SPI peripherals as required for the application.

1st Step

The first step for this task is to boot the bitstream directly to the PL part. This was easily achieved by setting ZCU104 in JTAG mode by manually altering its SW6 switch pins on the board to 0000. Now we can boot the bitstream directly to the PL part with the help of the Vivado Hardware Manager.

2nd Step and Challenges

The second important task was to boot the Linux image onto our programmed stand-alone SoC and have a debug connection over JTAG.

This is where we faced all challenges as we can’t use general peripherals like onboard SPI flash, FT4232H, and SD Card as all of them are connected to the MPSoC.

In our application, we want to boot up Linux on the RISC-V processor with JTAG. Since the usage of onboard JTAG was out of the question we had to use external JTAG adapters connected to the RISC-V processor via PMOD.

We chose to use the Digilent JTAG HS2 programming module for this purpose. Our processor has an inbuilt JTAG TAP controller and exposes 5 pins (TCK, TDO, TMS, TDI, and TRSTN) for connecting an external JTAG.
We made these pins external and connected them to PMOD, however without the required constraints we had a hard time successfully establishing a connection between SoC JTAG and Digilent HS2 module.

The reason is that, during implementation, Vivado treats these pins optimistically and tries to meet the timing requirements without considering any of the JTAG properties. Not only that, sometimes Vivado even throws errors during implementation saying that the TCK does not follow the clock placement rules.

Resolution: Step 1:

Therefore, It is better to add the below timing constraints to the XDC file along with the general PMOD constraints for proper implementation.

1. create clock -period 100.000 -name tck -waveform {0.000 50.000} [get_ports tck_clk_0]

As TCK comes under a virtual clock, we need to inform Vivado the time-period of TCK, its duty cycle, and its phase shift.
For our application, we defined the time period as 100ns (f = 10Mhz).

2. set_clock_groups -asynchronous -group [get_clocks -of_objects [get_pins ]] -group [get_clocks tck]

When the design is more congested then defining the SoC clock and TCK as asynchronous helps us in better timing closure.

3. set_input_jitter tck 1.000

It tells Vivado the difference between successive clock edges of TCK with respect to variation from the ideal clock arrival times

4. set_property CLOCK_DEDICATED_ROUTE FALSE [get_nets tck_clk_0_IBUF_inst/O] –

This property tells Vivado not to follow general clock placement rules during the placement of TCK

5. set_input_delay -clock tck -clock_fall 5.000 [get_ports tdi_in_0]

6. set_input_delay -clock tck -clock_fall 5.000 [get_ports tms_in_0]

It specifies the input path delay of TMS relative to a falling clock edge at the interface of the design.

7. set_output_delay -clock tck 5.000 [get_ports tdo_out_0]

It specifies the output path delay of TDO relative to a clock edge at the interface of the design.

8. set_max_delay -to [get_ports tdo_out_0] 20.000

9. set_max_delay -from [get_ports tms_in_0] 20.000

10. set_max_delay -from [get_ports tdi_in_0] 20.000

It sets the minimum and maximum path delay values. This overrides the default setup and hold constraints with user-specified maximum and minimum delay values

After adding them to XDC files we can continue the implementation. If everything is correct then Vivado will be able to complete the implementation without any timing failure.

Note: In our case, as our design is more resource intensive and hence it became very congested, resulting in timing failure even after adding the required constraints. In order to avoid this, we used the Vivado implementation strategy Congestion_SpreadLogic_high. Under this strategy, Vivado spreads the logic throughout the device to avoid creating congested regions (high setting is the highest degree of spreading among other congestion spread logic strategies).

Resolution : Step 2:

OpenOCD
We used OpenOCD for connecting the host to the processor via GDB over JTAG. As our SoC is based on RISC-V architecture we used RISC-V GDB.

For OpenOCD to establish a successful connection with the target we need to configure the properties of the JTAG adaptor with help of a config file. The config file which was used is listed below.

Result

With the above resolutions, we were successful in connecting the GDB over JTAG to the RISC-V processor programmed in the PL part of the ZCU104 board.

References and Clarifications:

1. Xilinx UG904 document for Vivado implementation strategies

2. Xilinx UG903 document for timing constraints related information

3. Call with Xilinx support team which clarified the following:

Confirmed with the Xilinx team that we can’t use onboard FT4232H Jtag for our application
These kinds of applications (booting processors in the PL part) are not supported by Xilinx.

4. Call with Shakti Vajra team which confirmed the following:

Shakti RISC-V SoC (Shakti vajra C-Class SoC) supports external JTAG
Shakti Vajra team used JLINK brand of adaptor

5. Default OpenOCD config file given by Shakti Config file for Shakti.
(https://gitlab.com/shaktiproject/sp2020/-/blob/master/c64-a100/shakti-arty.cfg)