Announcing the Hardware Acceleration WG, meeting #1

vmayoral · 2021-06-09T08:49:01.487Z

I think this is an awesome idea. I made a section for this at the community repo of the WG and will add your request to the Agenda above. We’ll allocate a few minutes and discuss this during the meeting.

I don’t have an Ultra96-v2 with me so I won’t be able to help much debugging, but I purposely architected things so that extending it to a new board is all about creating another custom acceleration_firmware branch with the firmware artifacts for that specific board (accel_fw in the following diagram):

               HAWG @ ROS 2 stack

           +--------------------+
           |  xilinx_examples   |
+-------------------+-----------+-------+--------------+
|       Drivers     |     Libraries     |    Cloud     |
+---------------+---+--------+-------------------------+
|   ament_vitis | ament_rocm |          |  accel_fw    |
+---------------+----------+-+----------+-+------------+
|     ament_acceleration   | colcon_accel |  accel_fw  |
+------------------------------------------------------+
|      build system        |   meta build |  firmware  |
+--------------------------+--------------+------------+

All of this will be detailed in the repo. If done properly, then ament_acceleration derived packages (e.g. ament_vitis) should pick all the artifacts automatically and connect it with the ROS build system and tools so that you’d only need to invoke colcon build with a mixin, that’ll be it. We sent these software components to legal for open sourcing them and they’ll be out as soon as we get the approvals. I’ll be demonstrating this in the first WG meeting. Also, I’ll make sure to detail the process of porting things to another board.

For new developments, I’d highly encourage to stick to the official reference hardware platform (K26) since that’s what we’ll be testing and where all the features will be confirmed to work and available.

If there’s anyone else interested in doing something similar, please speak out!

jopequ · 2021-06-09T09:47:29.766Z

I will try to get a KV260 as soon as they are available but I would also like to have support for the Ultra96-v2. We have several of those that we use for teaching in a couple master courses.
In general this looks really interesting! Thanks for the effort in pushing this forward @vmayoral

vmayoral · 2021-06-09T10:11:21.927Z

Noted, thanks @jopequ!

Joe_Dinius · 2021-06-21T18:33:32.928Z

I second @jopequ’s recommendation to support the Ultra96-v2 SoM as well. The board uses the UltraScale+ architecture (same as the KV260, I believe) and has a form-factor that can more easily be integrated with lower SWAP platforms (such as quadrotors or smaller-scale fixed wing UAVs).

victor_sillerico · 2021-06-21T23:39:50.100Z

Great news!. It is great to know all the possibilities that Hardware Accelerations platforms will bring to the ROS 2 community.

uayucar · 2021-06-25T13:23:07.098Z

Really interesting! Accelerated hardware towards robotic system optimization will lead to a wide variety of complex computational implementation. Concepts, such as cybersecurity could clearly be benefited by this kind of technology! Looking forward to the WG.

mamerlan · 2021-06-30T06:33:48.385Z

Hi @vmayoral,

Great news! We are very interested in knowing more about the Hardware Acceleration WG initiatives.
We are hosting the Embedded WG focused on micro-ROS features and use cases, and for sure there will be simliar nature interests within the ecosystem.

Happy to read that the meeting will be recorded!

vmayoral · 2021-06-30T07:26:46.635Z

Thanks @mamerlan, meeting is happening today. I know it’s a bit late in Spain but provided I don’t mess up out of being tired, recordings should be out soon after the meeting. We’ll keep minutes nevertheless .

As for alignment with Embedded WG, I’m sure you know I’m quite tuned to micro-ROS. I look forward to cooperate. Here’re some early thoughts for alignment between the HAWG and the EWG:

it’d be interesting to run micro-ROS Client in either the R5s or into a soft-core (e.g. in a MicroBlaze)
it’d be interesting to offload the agent into Programmable Logic (the FPGA)
- which obviously leads to consider pushing part of FastDDS also to the PL

We’ve got a few more ideas involving other architectures for mixed-criticality and link layers that allow for real-time distributed communications (e.g. TSN).

vmayoral · 2021-07-02T04:49:00.137Z

Thanks everyone for a fantastic first meeting!

We had a successful kick-off of the ROS 2 Hardware Acceleration Working Group where we discussed how we can make ROS 2 faster through FPGAs and GPUs with open technologies, initially focusing on C++ and OpenCL. We registered 27 participants initially and reached more than 30 (I’m being told) during the course of the meeting. This was a first for me in a WG kick-off session. We are very excited and thankful for the interest and the support messages received. Looking forward to continue contributing!

Here’s a summary of the resources discussed:

Minutes
ros-acceleration Github organization
KV260 kit as reference hardware platform
source code contributed:
- ament_vitis, CMake macros and utilities to include Vitis platform into the ROS 2 build system (ament) and its development flows.
- Xilinx Runtime (XRT), an open-source standardized software interface that facilitates communication between the application code and the accelerated-kernels
and finally, the video recording the session:

For those of you that don’t have the time to watch the whole piece, see below for a chapter-ed split:

0:00 Introduction
12:25 Objectives, rationale and hardware reference platforms
16:40 Initial hardware acceleration architecture
22:10 Short demonstration
26:44 Community hardware platforms
29:48 Q&A
43:00 Final remarks

We’ll be following up shortly on a number of the topics discussed over the call. Stay tuned for the next meeting and enjoy summer time !

vmayoral · 2021-07-05T20:28:18.617Z

As a quick update, following the community ask, I ordered today an Ultra96-v2 board from Farnell only to find that the lead time is 16 weeks !

@Pedro, @jopequ, @Joe_Dinius, do any of you have any advice for me on how to get my hands into one faster?

Joe_Dinius · 2021-07-05T22:39:03.463Z

Looks like it is in stock at Avnet - Avnet: Quality Electronic Components & Services

vmayoral · 2021-07-06T06:39:19.700Z

Thanks @Joe_Dinius, I was hoping to avoid that since I’m in Europe … but I found no better option so that’ll do I guess. Will cancell my order with Farnell and go with Avnet US.

In case someone bumps on it as well, there seem to be also some delays reported. Hopefully it’s just that:

Captura de pantalla 2021-07-06 a las 8.35.39996×216 26.2 KB

jopequ · 2021-07-06T09:43:22.147Z

We got the last batch to Finland from Farnell in December last year so no experience with other sources, sorry

YangZ · 2021-07-06T10:32:12.537Z

Ultra96-v2 is a flagship 96Boards - let me know if you need help so I can forward you to Xilinx and Avnet folks directly.

For full disclosure, I am responsible for 96Boards specifications which Ultra96-v2 is built to be compliant.

Yang

vmayoral · 2021-07-06T13:05:09.651Z

@YangZ having someone from Avnet look at this would be awesome. Specially, I think they’ll benefit a lot if they could commit some engineering resources . We at Xilinx are already doing so.

PM me if you need further details.

YangZ · 2021-07-06T14:01:44.079Z

Didn’t realise Xilinx fellow here - I know you guys are in

I have forwarded this thread to Avent folks just now.

vmayoral · 2021-09-01T16:50:20.491Z

@Pedro, @Joe_Dinius and @jopequ, meeting your previous ask, have a look at acceleration_firmware_kv260. Creating a similar acceleration_firmware_ultra96v2 would be the way to go to support it.

This repo, together with the REP PR should give you the insights needed to start with it. I’ll make sure to go though the firmware quickly in the next meeting to address possible questions, but feel free to reach out as well.

Shawn_Schaerer · 2021-09-01T19:00:50.086Z

@vmayoral, any safety ratings for that SOM? I realize this is probably the wrong place to ask.

vmayoral · 2021-09-03T15:14:00.888Z

hi @Shawn_Schaerer, I’m assuming we’re speaking about IEC 61508. If so, none that I know for K26 today and that come out of the box, sorry .

If you’re pursuing this and would like to consider the K26 SOM, I’m interested and happy to help. K26 may or may not be the best way forward, depending on your criticality level (i.e. SIL-level).

If you’re looking for an immediate solution, you may want to check out the modular safety approach followed with Xilinx’s Zynq 7000 SoCs, which showed a high degree of flexibility across use cases.

Shawn_Schaerer · 2021-09-03T17:01:11.175Z

Hi @vmayoral, yes I mean IEC61508. I know about the Znyq already. Is that processor being considered for the work you are doing here? We should continue this conversation on a separate thread.