ROS acceleration using Xilinx AI Edge computing

Future Robots are smarter and capable of making decisions in real-time. Decision making requires the use of AI for performing classification and semantic segmentation. Such AI algorithms require the use of dedicated hardware for making decisions at the edge. This group is meant for discussing subjects related to the integration of the:

  1. ROS
  2. ROS2
  3. OpenCV4
  4. Librealsense2
  5. Xilinx DPU TRM

The following slack workspace (https://rosxilinxedgeai.slack.com/home) is available for online discussing.
Xilinx Edge AI framework,

Xilinx AI Edge Platform

The Xilinx Edge AI Platform provides comprehensive tools and models which utilize unique deep compression and hardware-accelerated Deep Learning technology.

The platform provides efficient, convenient and economical inference deployments for embedded-CPU-based FPGAs.

The Xilinx AI team consists of renowned researchers and experienced professionals known for their pioneering work in the field of deep learning.

4 Likes

I would tempted to encourage people start by looking at Ultra96.

1 Like

Thanks @YangZ. I have experience using the following boards:

  1. VCS-1 system
  2. ZCU104
  3. ZCU102

I would like to encourage you to share your experience using the Ultra96
Kind Regards
Pedro

1 Like





https://www.element14.com/community/community/designcenter/zedboardcommunity/ultra96

One of the best Edge AI platforms out there today following 96Boards specification.

For full disclosure - I am the author of 96Boards specification.

2 Likes

@YangZ,
Have you come across any Ultra96 project where ROS is used? If so can you share it here?

… I am also totally biased, as I created Sundance’s VCS platform.

We wanted to use the EV version of the Zynq MPSoC to allow the “Edge-AI” to benefit from the integrated codecs, etc. + give more fabric and more memory. The VCS platform is also in Open Source, but has a much higher cost, intended for industrial application.

Even Xilinx like Sundance’s ‘story’ and efforts - see https://forums.xilinx.com/t5/AI-and-Machine-Learning-Blog/Xilinx-Zynq-Devices-Take-Charge-of-Robots/ba-p/1015700 - and

The Ultra96 is a ‘gift’ from Farnell/Avnet to help R&D and I can only recommend buying one to learn about Xilinx Vitis and Edge-AI.

2 Likes

i got some experience on zcu102, including ROS1 to publish the sensor data.

Hi @tomoyafujita,
Can I kindly ask you to share your experience here? Perhaps a simple tutorial.

In my experience the xilinx workflow works really well, IF the model you are trying to deploy was built with the same version of caffe or tensorflow as your DECENT and DNNC. Trying to deploy pre-trained models you find online can be tricky and time-consuming (unless they come from the AI model zoo of course :stuck_out_tongue: then you are one step removed from a DPU kernel). Deploying your own models should be fine if you align your environments.

I’m also interested to see if and when pytorch support arrives. I’m currently working with DNNDK on the ZCU102, maybe the newer VITIS AI (built on top of DNNDK) has support for more frameworks and easier integration of models built with/against different versions of tensorflow and caffe?

1 Like

Hi @MartinCornelis,
Please join our growing community on slack https://join.slack.com/t/rosxilinxedgeai/shared_invite/enQtODc4MzgxNDgwMzg5LWU0NWJmN2FmYjAzYzZhZDlhOWYxMDMyYmY2NWRlYmMzYWRiZjI4ZTUxODZlNzU1OTUwMDk2MGM4Y2Y1YTJiNWE.
Lets make our life easier.
Bw,
Pedro

@pedrombmachado

we did internally put some login in PL for doing pre-process for imaging (not so much i can share), and the CPU receives that pre-processed data and publishes as ROS topic. this is long time ago, but if i remember correctly, i did use ROS kinetic.

hope that helps you.

Hi @MartinCornelis,
Can you please join our slack group https://join.slack.com/t/rosxilinxedgeai/shared_invite/enQtODc4MzgxNDgwMzg5LWU0NWJmN2FmYjAzYzZhZDlhOWYxMDMyYmY2NWRlYmMzYWRiZjI4ZTUxODZlNzU1OTUwMDk2MGM4Y2Y1YTJiNWE

Hi @pedrombmachado, thank you for the invite, however we got a ZCU102 loaner to do some testing for a project. We will be handing that back over soon. So I am not sure if Ill be handling any FPGAs for the foreseeable future. I am interested in the subject in general, but I dont think I’ll be contributing much.

@pedrombmachado please invite me to slack

@pedrombmachado could you please invite me to the slack channel

Hi everyone!
I have an Ultra96 v2 with me.
@pedrombmachado can you please share the slack channel link with me?

Just dropping this link here:


Could be an interesting read for some of you.

I am working with Ultra96v2 and I am researching about ROS and FPGA acceleration. Can you invite me to your slack? Many thanks.

Hi @pedrombmachado,

Can you invite me to this slack group?

Same in here @pedrombmachado, can I please get invited as well with victor at aliasrobotics.com?
Thanks!