XRT C++ driver generation

[neilkimn]

Hi all,

I am looking into linking the FINN-generated RTL kernels as .xos with other kernels generated from C++.
However, much of the guidance material (1, 2, 3) on this matter is either too simplified or uses OpenCL on the host side as opposed to XRT.

I stumbled upon this idea on the project ideas page and was wondering if anyone had given this any thought yet? I am especially interested in hearing about how the RTL kernels are handled on the host-side in terms of how to set up the device buffers through xrt::bo and how the dataflow nodes are executed from the host code.

If anyone has made any considerations about this, I am happy to delve into further details about this project idea.

[maltanar]

Hey @neilkimn ,

Great question, we don't have a lot of ready-to-go material about this but I can provide a few more details on how it could work with a few examples.

• The simplest case is when the FINN-generated IP is completely surrounded by other kernels (either RTL or HLS) and all communication with other kernels is via on-chip streams; something like (video source IP) -> (FIFO) -> (FINN-generated IP) -> (FIFO) -> (render on screen IP) . Let's also assume that it does not use any weights in either URAM or external memory. In this case, the generated IP block is always-on (also called free-running in Vitis), everything is baked into the bitfile in terms of weights, and it does not need any start/stop signal. All backpressure is handled by handshakes on the input and output AXI streams. In this case the host code doesn't need to do anything special for the FINN part as it "just runs" as it receives the data.

• If the communication between FINN-generated IP and other kernels is happening through external memory, and you want to use the FINN-generated HLS data movers (called IODMAs), this will need some driver code because those IODMAs are not free-running kernels. With VitisBuild, the data movers and the core streaming part will be implemented as separate Vitis kernels, like (IODMA for input) -> (streaming NN) -> (IODMA for output). The streaming NN part is free-running and needs no driver. The IODMAs will have some AXI master interfaces to read/write external memory, and some AXI Lite slave interface. This AXI lite interface is what the driver needs to interact with in order to tell the input/output IODMAs three things: where is the buffer to be read/written (physical address), how many images/samples are to be read/written, and a control/status register for start/stop/status. The driver also needs to take care of buffer object (xrt::bo) allocation and initial bitfile flashing too. So the driver needs to:

  1. Flash the bitfile/xclbin initially
  2. Set up buffers for the accelerator to read/write
  3. Set up the registers for IODMAs to indicate buffer locations, amount of samples to read/write, give the start signal and wait until output DMA is finished

On the Vitis tutorials side the closest example I can find is this one:
https://github.com/Xilinx/Vitis_Accel_Examples/blob/master/host_xrt/streaming_free_running_k2k_xrt/src/host.cpp

There's also this one that shows how to directly read/write registers by offset:
https://github.com/Xilinx/Vitis_Accel_Examples/blob/master/host_xrt/register_access_xrt/src/host.cpp

On the FINN side, if you look closely at how execute_on_buffers works for Zynq platforms, you can see the individual registers being set up for the input/output/external weight IODMAs :
https://github.com/Xilinx/finn-examples/blob/main/finn_examples/driver.py#L342

Hope this helps shed some light onto the topic. Do you have a concrete project around this in mind?

[neilkimn]

Hi Yaman,

Thanks for the thorough reply. I am currently looking at a project aimed at prepending some 'advanced' (pre)processing to the FINN-generated RTL kernel. Specifically, this would be aimed at some processing/transformations that cannot be baked into the accelerator and would otherwise go through CPU processing.

The specific case I'm looking at involves passing text data through an XRT C++ generated kernel-flow in which the goal is to compare what the increase in throughput is over CPU preproessing. The text is to be passed through a hash table containing embeddings that would then be passed to the FINN IP.

Having had some time to go over the resources in the Vitis_Accel_Examples repo, it has occured to me that there are many ways to achieve higher throughput for the C++ kernels such as being able to leverage different memory banks and various HLS pragmas. I assume this sort of control is already covered by FINN to a certain degree, and therefore I wonder what the long-term thoughts behind this idea may be? Is it to streamline any work related to combining the FINN-generated IPs with other kernels for future work?

I am happy to hear more about what the ideas in this space may entail for FINN in general.
Also, thanks for picking up on #338 after it had gotten stale over the summer :)