Over several weeks we have examined how we can use Alveo cards using a Vivado based design flow coupled with examining the XMDA drivers. We can use this approach to implement a wide variety of applications from AI acceleration to video transcoding and many applications in between.
One of the very interesting use cases for Alveo cards due to the close coupling with an x86 processor and high performance PCIe Gen4 interconnectivity is a Smart Network Interface Card (Smart NIC). Smart NICs enabled the offloading of tasks which increase overall system performance, this can be critical when high network speeds are crucial for example in data centres. Examples of offloading can be relatively simple such as processing elements of the TCP/IP Stack, SSL or TLS cryptography and packet filtering. Smart NICs also enable developers to implement programmability to adapt to application specific needs or evolving protocols. While also being able to support advanced features such as Quality or Service, and network virtualisation.
Creating a Smart NIC from scratch is not a insignificant task, which is where the OpenNIC project comes in. Created by AMD the OpenNIC project is an open source project which aims to enable networking researchers and developers a way to accelerate their design.
The OpenNIC architecture is based around a Connection Manager Access Control based on the UltraScale+ Integrated 100G Ethernet IP core and the QMDA Subsystem. This provides developers with the ability to connect the PCIe interface and the QSFP cages, while providing user configurable partitions in the design for user development. To align with Microsoft research project catapult, the OpenNIC design is split into two elements the “shell” and “role”. The shell provides developers with the base design which provides the CMAC, QDMA, device infrastructure and interconnections along with the appropriate Linux driver. Within the shell the OpenNIC provides up to four PCIe physical functions (PFs) and two 100G Ethernet ports.
While the role are locations within the design where the developers can implement their desired functionality. Within the OpenNIC design there are two specific role areas provided the first sits directly on the data path between the QMDA and the CMAC while the second is intended to enable customisation of the CMAC.
To get started with OpenNIC we are provided with three git repositories which cover
OpenNIC Shell
OpenNIC Driver
OpenNIC DPDK
While I am not a networking researcher I was curious as to how easy the OpenNIC shell was to create and how the design looks for implementation and custom additions to the role. With that in mind I cloned the repository and was glad to see it supported the U55C card we have been examining in this series.
Using Vivado 2022.2 the latest supported version for OpenNIC, I ran the command below to build the shell
vivado -mode gui -source build.tcl -tclargs -board au55c -synth_ip 0
This creates the shell with two user boxes for the role, the first is clocked at 250MHz while the second is clocked at 322MHz. The resources utilisation of the shell is very small as the Alveo card implements the CMAC and PCIe as hard IP.
Once the design is created within Vivado we can explore the design, within the source view we can clearly see the two role areas for our custom addition and the logic inserted to complete the base design.
If you want to get started developing your own SmartNIC application OpenNic is a very interesting concept to explore and proves a great starting point. Also provided as part of the OpenNIC Shell is a Cocotb test bench which provides enables you to get start with verification of your new custom features!
Workshops and Webinars
If you enjoyed the blog why not take a look at the free webinars, workshops and training courses we have created over the years. Highlights include
Professional PYNQ Learn how to use PYNQ in your developments
Introduction to Vivado learn how to use AMD Vivado
Ultra96, MiniZed & ZU1 three day course looking at HW, SW and Petalinux
Arty Z7-20 Class looking at HW, SW and Petalinux
Mastering MicroBlaze learn how to create MicroBlaze solutions
HLS Hero Workshop learn how to create High Level Synthesis based solutions
Perfecting Petalinux learn how to create and work with PetaLinux OS
Embedded System Book
Do you want to know more about designing embedded systems from scratch? Check out our book on creating embedded systems. This book will walk you through all the stages of requirements, architecture, component selection, schematics, layout, and FPGA / software design. We designed and manufactured the board at the heart of the book! The schematics and layout are available in Altium here Learn more about the board (see previous blogs on Bring up, DDR validation, USB, Sensors) and view the schematics here.
Sponsored by AMD