Booting Linux on a Custom Processor on ZCU104

Background

The ZCU104 board is a combination of FPGA and an ARM-based MPSoC. The FPGA part of ZCU104 is tightly bonded to the MPSoC making many of the peripherals on ZCU104 usable only via ARM core. This feature gives an advantage when we are using both MPSoC and FPGA parts, but it turns into a disadvantage when we want to use just the PL (stand-alone PL logic) part of ZCU104 without any intervention of MPSoC. Because of this reason, most of the important peripherals on ZCU104 become inaccessible to the stand-alone PL logic. Booting a custom processor on the PL part is one such application.

Our Use case and application

For our application, we needed a RISC-V based custom SoC in the PL part, that too stand- alone, with no usage of ARM-based MPSoC.

The SoC contains several other modules like the DLA, I2C, UART, SPI peripherals as required for the application.

1st Step

The first step for this task is to boot the bitstream directly to the PL part. This was easily achieved by setting ZCU104 in JTAG mode by manually altering its SW6 switch pins on the board to 0000. Now we can boot the bitstream directly to the PL part with the help of the Vivado Hardware Manager.

2nd Step and Challenges

The second important task was to boot the Linux image onto our programmed stand-alone SoC and have a debug connection over JTAG.

This is where we faced all challenges as we can’t use general peripherals like onboard SPI flash, FT4232H, and SD Card as all of them are connected to the MPSoC.

In our application, we want to boot up Linux on the RISC-V processor with JTAG. Since the usage of onboard JTAG was out of the question we had to use external JTAG adapters connected to the RISC-V processor via PMOD.

We chose to use the Digilent JTAG HS2 programming module for this purpose. Our processor has an inbuilt JTAG TAP controller and exposes 5 pins (TCK, TDO, TMS, TDI, and TRSTN) for connecting an external JTAG.
We made these pins external and connected them to PMOD, however without the required constraints we had a hard time successfully establishing a connection between SoC JTAG and Digilent HS2 module.

The reason is that, during implementation, Vivado treats these pins optimistically and tries to meet the timing requirements without considering any of the JTAG properties. Not only that, sometimes Vivado even throws errors during implementation saying that the TCK does not follow the clock placement rules.

Resolution: Step 1:

Therefore, It is better to add the below timing constraints to the XDC file along with the general PMOD constraints for proper implementation.

1. create clock -period 100.000 -name tck -waveform {0.000 50.000} [get_ports tck_clk_0]

As TCK comes under a virtual clock, we need to inform Vivado the time-period of TCK, its duty cycle, and its phase shift.
For our application, we defined the time period as 100ns (f = 10Mhz).

2. set_clock_groups -asynchronous -group [get_clocks -of_objects [get_pins ]] -group [get_clocks tck]

When the design is more congested then defining the SoC clock and TCK as asynchronous helps us in better timing closure.

3. set_input_jitter tck 1.000

It tells Vivado the difference between successive clock edges of TCK with respect to variation from the ideal clock arrival times

4. set_property CLOCK_DEDICATED_ROUTE FALSE [get_nets tck_clk_0_IBUF_inst/O] –

This property tells Vivado not to follow general clock placement rules during the placement of TCK

5. set_input_delay -clock tck -clock_fall 5.000 [get_ports tdi_in_0]

6. set_input_delay -clock tck -clock_fall 5.000 [get_ports tms_in_0]

It specifies the input path delay of TMS relative to a falling clock edge at the interface of the design.

7. set_output_delay -clock tck 5.000 [get_ports tdo_out_0]

It specifies the output path delay of TDO relative to a clock edge at the interface of the design.

8. set_max_delay -to [get_ports tdo_out_0] 20.000

9. set_max_delay -from [get_ports tms_in_0] 20.000

10. set_max_delay -from [get_ports tdi_in_0] 20.000

It sets the minimum and maximum path delay values. This overrides the default setup and hold constraints with user-specified maximum and minimum delay values

After adding them to XDC files we can continue the implementation. If everything is correct then Vivado will be able to complete the implementation without any timing failure.

Note: In our case, as our design is more resource intensive and hence it became very congested, resulting in timing failure even after adding the required constraints. In order to avoid this, we used the Vivado implementation strategy Congestion_SpreadLogic_high. Under this strategy, Vivado spreads the logic throughout the device to avoid creating congested regions (high setting is the highest degree of spreading among other congestion spread logic strategies).

Resolution : Step 2:

OpenOCD
We used OpenOCD for connecting the host to the processor via GDB over JTAG. As our SoC is based on RISC-V architecture we used RISC-V GDB.

For OpenOCD to establish a successful connection with the target we need to configure the properties of the JTAG adaptor with help of a config file. The config file which was used is listed below.

				
					adapter_khz     10000
#It defines the frequency of TCK, better to use the frequency less than or equal
#to the one we defined in Vivado constraints

# Digilent JTAG-HS2 specific information interface ftdi
ftdi_vid_pid 0x0403 0x6014
ftdi_channel 0
ftdi_layout_init 0x00e8 0x60eb

set _CHIPNAME riscv

jtag newtap $_CHIPNAME unknown0 -irlen 5 -expected-id 0x10102001
jtag newtap $_CHIPNAME cpu -irlen 5 -expected-id 0x100039D3
# Even though we are using only one tap we need to define multiple taps and specify #one as unknown to tell the OpenOCD to neglect any other jtag tap connection
# Care should be taken while defining the expected ID as it can vary from processor #to processor

set _TARGETNAME $_CHIPNAME.cpu
target create $_TARGETNAME riscv -chain-position $_TARGETNAME
$_TARGETNAME configure -work-area-phys 0x80000000 -work-area-size 10000 -work-area- backup 1
#This tells OpenOCD to what address it needs to push the target file.

gdb_report_data_abort enable
#Specifies whether data aborts cause an error to be reported by GDB memory read #packets.
gdb_report_register_access_error enable
#Specifies whether register accesses requested by GDB register read/write packets
#report errors or no

riscv set_reset_timeout_sec 120
#Set the maximum time to wait for a hart to come out of reset after reset is
#deasserted

riscv set_command_timeout_sec 120
#Set the wall-clock timeout (in seconds) for individual commands. The default #should work fine for all but the slowest targets

# Prefer to use sba for system bus access

riscv set_prefer_sba on

# dump jtag chain
scan_chain

init
halt

Result

With the above resolutions, we were successful in connecting the GDB over JTAG to the RISC-V processor programmed in the PL part of the ZCU104 board.

References and Clarifications:

1. Xilinx UG904 document for Vivado implementation strategies

2. Xilinx UG903 document for timing constraints related information

3. Call with Xilinx support team which clarified the following:

Confirmed with the Xilinx team that we can’t use onboard FT4232H Jtag for our application
These kinds of applications (booting processors in the PL part) are not supported by Xilinx.

4. Call with Shakti Vajra team which confirmed the following:

Shakti RISC-V SoC (Shakti vajra C-Class SoC) supports external JTAG
Shakti Vajra team used JLINK brand of adaptor

5. Default OpenOCD config file given by Shakti Config file for Shakti.
(https://gitlab.com/shaktiproject/sp2020/-/blob/master/c64-a100/shakti-arty.cfg)