So far on this journey of exploring how we can use Alveo cards outside of the traditional flow we have examined creating the reference design, installing the XDMA drivers and of course looked at OpenNIC and PYNQ on Alveo.
One of the things we often want to be able to do with FPGA based designs is leverage the interfacing capabilities which are provided by the flexibility of programmable logic. In the case of the U55C Alveo card this is provided by Dual QSFP28 Ports.
If you are not familiar with QSFP it stands for Quad Small Form-Factor Pluggable, the 28 indicates the line rate at which each gigabit serial link used in the connector is capable of running at. In this instance as it is a Quad SFP there are four TX and RX lines for each QSFP port from the FPGA on the Alveo card each capable of running at up to 20 Gbps.
This means a single QSFP is capable of supporting a range of high bandwidth applications such as 100G Ethernet as we saw when we looked at the OpenNIC example a few weeks ago.
In this blog I wanted to show how we can test the Bit Error Rate of the QSFP connector this in important as when we are connected to downstream equipment being able to check quality is critical.
To get started with this I am going to create a new project within Vivado targeting the U55C as we did previously. Once the project is opened we are going to open the IP catalog and select the IBERT UltraScale GTY.
This will open a configuration dialog which allows us to define the GTY Quads we wish to work with and the line rates at which we will be doing so. For this first example I set the line rate to be 20.625 Gbps this works nicely with the provided GTY reference clock for the custom protocol and set the quad count to two. This means we can use this protocol on two quads in the FPGA.
We apply the protocol to the quads on the protocol selection tab however, first we need to determine which quads are used for the QSFP. To determine this and the reference clocks we need to open the XDC file which is provided here.
From the XDC file we can see the clocking architecture is defined very clearly in the comments, as such we are able to see the GTY reference clocks for the GTY associated with the QSFP Interfaces. These are 161.1328125 MHz and are provided to Quad 130 and Quad 131. Further down in the XDC file we will see the locations of the GTY used for the QSFP interfaces, these are Quad 130 and Quad 131.
The final setting is to define the system clock again from the XDC file this is 100MHz and located on pin F24 and is LVDS.
With these modifications complete we can close the IP core and generate the IP core and its outputs. Once this has completed, under the project manager, right click on the ibert ultrascale gty and select open IP example design.
This will create a new project, which uses the configurations of the Ibert and creates an example project which we can implement and test on hardware. Feel free to explore the design and check the constraints etc. once you are satisfied generate the bit stream to program the device.
Once the bit stream is available the next step is to open the hardware manager and connect to the Alveo card using JTAG. Program the card with the image and the ltx file which contains the debug information.
With the device programmed we will be asked to create links for the loop back – connect each TX with the associated RX.
Following the establishment of the links we will be able to view and configure the links.
Using the serial I/O links loop back mode, we can set the loop back to near or far end or none. If we do not have a QSFP loop back adapter or a cabling etc to enable the far end loop back you can perform initial testing using near end loop back in in the PCS or the PMA.
For this project will use a QSFP loop back and a the Near End PMA loop back, starting with the near end PMA loop back.
In the serial IO links set the loop back mode to Near End PMA and reset the TX, RX and the BER. You will observe the status of the link showing the line rate and the BER the longer you leave it running the more this will improve.
With a QSFP loopback connected in one of the two QSFP ports we can change the loopback method to none. Not the QSFP which does not have a loopback inserted reports no link.
We can also perform a eye scan if desired using the Serial IO scans tab.
To check the status of the link when no loop back is enabled and of course these the routing on the PCB etc. I placed a QSFP loop back connector which loops back Tx to Rx at the QSFP port. To perform this test set the Loopback Mode to none and ensure the link is established and monitor the BER, again the longer this run the better it will be.
If you wish to you can inject an error using the inject error button or you could change the RX pattern to make it different to the TX pattern to show the link when it breaks. Changing the RX pattern back to match the TX patter should recover the link and show the correct status again.
Having examine the IBERT we are now familiar with both of the main interfaces on the Alveo U55C card the PCIe Interface and the QSFP Interfaces.
Workshops and Webinars
If you enjoyed the blog why not take a look at the free webinars, workshops and training courses we have created over the years. Highlights include
Professional PYNQ Learn how to use PYNQ in your developments
Introduction to Vivado learn how to use AMD Vivado
Ultra96, MiniZed & ZU1 three day course looking at HW, SW and Petalinux
Arty Z7-20 Class looking at HW, SW and Petalinux
Mastering MicroBlaze learn how to create MicroBlaze solutions
HLS Hero Workshop learn how to create High Level Synthesis based solutions
Perfecting Petalinux learn how to create and work with PetaLinux OS
Embedded System Book
Do you want to know more about designing embedded systems from scratch? Check out our book on creating embedded systems. This book will walk you through all the stages of requirements, architecture, component selection, schematics, layout, and FPGA / software design. We designed and manufactured the board at the heart of the book! The schematics and layout are available in Altium here Learn more about the board (see previous blogs on Bring up, DDR validation, USB, Sensors) and view the schematics here.
Sponsored by AMD