Announcing the Hardware Acceleration WG, meeting #1

vmayoral · 2021-06-08T15:12:13.036Z

erlingrj:

Will you record the meeting?

We will @erlingrj, or at least I’ll do my best on that end . Minutes will also be available afterwards.

erlingrj:

I think easy integration with ROS could mean that more projects would consider using SoC FPGAs. I am very curios to hear what you guys over at Xilinx are thinking

We are of the same opinion. Moreover, as discussed at https://discourse.ros.org/t/challenges-of-gpu-acceleration-in-ros/19297/12:

ROS 2 users have additional priorities computational graph-wise which should be considered. E.g. it appears to be quite a hot topic to optimize interactions betwen nodes (inter-process, intra-process and even over the network). Accelerators on this end will add significant value to the overall ROS 2 computational graphs (and even to the underlying data layer graphs, which at the end of the day, matter as most in real deployments). In other words, while it’d be great to have OpenCL kernels to accelerate specific computations, real impact won’t be achieved unless a holistic ROS view is applied (bottlenecks identified, etc.)

Real acceleration would require optimizations not just at the application layer itself, but also at the ROS underlayers so that they don’t impose bottlenecks on the computational graphs. That’s why we’ve set this as a key objective first.

As we mentioned in the past, our effort is also to fight against the Not Invented Here syndrome, which is a big thing in the acceleration/semiconductors world. Big silicon vendors are too self-centric and rather than contributing to expert communities, prefer to reinvent the wheel with their yet-not-new-feature-in-acceleration-language-A or a new GUI-for-robot-simulation-that-adds-nothing-but-looks-B . Our team at Xilinx wants to tackle this and contribute to the ROS 2 community directly, connecting to real ROS users.

I’d encourage you to participate in the future and send to us what you’re working on, so that we consider your needs.

Pedro · 2021-06-08T16:51:53.726Z

Hi,
I would like to add the Ultra96-v2 board to the list of community supported boards, because is popular (so more people could contribute), not so expensive, and is readily available so we can start working with it. What do you think?

regards,
Pedro

vmayoral · 2021-06-09T08:49:01.487Z

Hola @Pedro !

I think this is an awesome idea. I made a section for this at the community repo of the WG and will add your request to the Agenda above. We’ll allocate a few minutes and discuss this during the meeting.

I don’t have an Ultra96-v2 with me so I won’t be able to help much debugging, but I purposely architected things so that extending it to a new board is all about creating another custom acceleration_firmware branch with the firmware artifacts for that specific board (accel_fw in the following diagram):

               HAWG @ ROS 2 stack

           +--------------------+
           |  xilinx_examples   |
+-------------------+-----------+-------+--------------+
|       Drivers     |     Libraries     |    Cloud     |
+---------------+---+--------+-------------------------+
|   ament_vitis | ament_rocm |          |  accel_fw    |
+---------------+----------+-+----------+-+------------+
|     ament_acceleration   | colcon_accel |  accel_fw  |
+------------------------------------------------------+
|      build system        |   meta build |  firmware  |
+--------------------------+--------------+------------+

All of this will be detailed in the repo. If done properly, then ament_acceleration derived packages (e.g. ament_vitis) should pick all the artifacts automatically and connect it with the ROS build system and tools so that you’d only need to invoke colcon build with a mixin, that’ll be it. We sent these software components to legal for open sourcing them and they’ll be out as soon as we get the approvals. I’ll be demonstrating this in the first WG meeting. Also, I’ll make sure to detail the process of porting things to another board.

For new developments, I’d highly encourage to stick to the official reference hardware platform (K26) since that’s what we’ll be testing and where all the features will be confirmed to work and available.

If there’s anyone else interested in doing something similar, please speak out!

jopequ · 2021-06-09T09:47:29.766Z

I will try to get a KV260 as soon as they are available but I would also like to have support for the Ultra96-v2. We have several of those that we use for teaching in a couple master courses.
In general this looks really interesting! Thanks for the effort in pushing this forward @vmayoral

vmayoral · 2021-06-09T10:11:21.927Z

Noted, thanks @jopequ!

Joe_Dinius · 2021-06-21T18:33:32.928Z

I second @jopequ’s recommendation to support the Ultra96-v2 SoM as well. The board uses the UltraScale+ architecture (same as the KV260, I believe) and has a form-factor that can more easily be integrated with lower SWAP platforms (such as quadrotors or smaller-scale fixed wing UAVs).

victor_sillerico · 2021-06-21T23:39:50.100Z

Great news!. It is great to know all the possibilities that Hardware Accelerations platforms will bring to the ROS 2 community.

uayucar · 2021-06-25T13:23:07.098Z

Really interesting! Accelerated hardware towards robotic system optimization will lead to a wide variety of complex computational implementation. Concepts, such as cybersecurity could clearly be benefited by this kind of technology! Looking forward to the WG.

mamerlan · 2021-06-30T06:33:48.385Z

Hi @vmayoral,

Great news! We are very interested in knowing more about the Hardware Acceleration WG initiatives.
We are hosting the Embedded WG focused on micro-ROS features and use cases, and for sure there will be simliar nature interests within the ecosystem.

Happy to read that the meeting will be recorded!

vmayoral · 2021-06-30T07:26:46.635Z

Thanks @mamerlan, meeting is happening today. I know it’s a bit late in Spain but provided I don’t mess up out of being tired, recordings should be out soon after the meeting. We’ll keep minutes nevertheless .

As for alignment with Embedded WG, I’m sure you know I’m quite tuned to micro-ROS. I look forward to cooperate. Here’re some early thoughts for alignment between the HAWG and the EWG:

it’d be interesting to run micro-ROS Client in either the R5s or into a soft-core (e.g. in a MicroBlaze)
it’d be interesting to offload the agent into Programmable Logic (the FPGA)
- which obviously leads to consider pushing part of FastDDS also to the PL

We’ve got a few more ideas involving other architectures for mixed-criticality and link layers that allow for real-time distributed communications (e.g. TSN).

vmayoral · 2021-07-02T04:49:00.137Z

Thanks everyone for a fantastic first meeting!

We had a successful kick-off of the ROS 2 Hardware Acceleration Working Group where we discussed how we can make ROS 2 faster through FPGAs and GPUs with open technologies, initially focusing on C++ and OpenCL. We registered 27 participants initially and reached more than 30 (I’m being told) during the course of the meeting. This was a first for me in a WG kick-off session. We are very excited and thankful for the interest and the support messages received. Looking forward to continue contributing!

Here’s a summary of the resources discussed:

Minutes
ros-acceleration Github organization
KV260 kit as reference hardware platform
source code contributed:
- ament_vitis, CMake macros and utilities to include Vitis platform into the ROS 2 build system (ament) and its development flows.
- Xilinx Runtime (XRT), an open-source standardized software interface that facilitates communication between the application code and the accelerated-kernels
and finally, the video recording the session:

For those of you that don’t have the time to watch the whole piece, see below for a chapter-ed split:

0:00 Introduction
12:25 Objectives, rationale and hardware reference platforms
16:40 Initial hardware acceleration architecture
22:10 Short demonstration
26:44 Community hardware platforms
29:48 Q&A
43:00 Final remarks

We’ll be following up shortly on a number of the topics discussed over the call. Stay tuned for the next meeting and enjoy summer time !

vmayoral · 2021-07-05T20:28:18.617Z

As a quick update, following the community ask, I ordered today an Ultra96-v2 board from Farnell only to find that the lead time is 16 weeks !

@Pedro, @jopequ, @Joe_Dinius, do any of you have any advice for me on how to get my hands into one faster?

Joe_Dinius · 2021-07-05T22:39:03.463Z

Looks like it is in stock at Avnet - Avnet: Quality Electronic Components & Services

vmayoral · 2021-07-06T06:39:19.700Z

Thanks @Joe_Dinius, I was hoping to avoid that since I’m in Europe … but I found no better option so that’ll do I guess. Will cancell my order with Farnell and go with Avnet US.

In case someone bumps on it as well, there seem to be also some delays reported. Hopefully it’s just that:

Captura de pantalla 2021-07-06 a las 8.35.39996×216 26.2 KB

jopequ · 2021-07-06T09:43:22.147Z

We got the last batch to Finland from Farnell in December last year so no experience with other sources, sorry

YangZ · 2021-07-06T10:32:12.537Z

Ultra96-v2 is a flagship 96Boards - let me know if you need help so I can forward you to Xilinx and Avnet folks directly.

For full disclosure, I am responsible for 96Boards specifications which Ultra96-v2 is built to be compliant.

Yang

vmayoral · 2021-07-06T13:05:09.651Z

@YangZ having someone from Avnet look at this would be awesome. Specially, I think they’ll benefit a lot if they could commit some engineering resources . We at Xilinx are already doing so.

PM me if you need further details.

YangZ · 2021-07-06T14:01:44.079Z

Didn’t realise Xilinx fellow here - I know you guys are in

I have forwarded this thread to Avent folks just now.

vmayoral · 2021-09-01T16:50:20.491Z

@Pedro, @Joe_Dinius and @jopequ, meeting your previous ask, have a look at acceleration_firmware_kv260. Creating a similar acceleration_firmware_ultra96v2 would be the way to go to support it.

This repo, together with the REP PR should give you the insights needed to start with it. I’ll make sure to go though the firmware quickly in the next meeting to address possible questions, but feel free to reach out as well.

Shawn_Schaerer · 2021-09-01T19:00:50.086Z

@vmayoral, any safety ratings for that SOM? I realize this is probably the wrong place to ask.