Hello everyone, on behalf of Xilinx, I’m happy to bring up to the community the proposal of a new working group, the
Hardware Acceleration WG. Find below a short description of the objectives and intentions.
Why should I care about hardware acceleration?
There’s a critical relationship between the hardware and the software capabilities in a robot. Robotic systems usually have limited on-board resources, including memory, I/O, disk or compute capabilities, which difficults the system integration process (e.g. when extending, adapting or fixing them), making it hard to meet the real-time requirements and limiting robots reaction, speed or capabilities. It is thereby essential to choose a proper compute platform for the robotic system. One that simplifies system integration, meets power restrictions and adapts to the changing demands of robotic applications.
CPUs are widely used commercial compute platforms in robotics due to their availability and generalized use. The general purpose nature of CPUs makes them specially interesting for roboticists to kickstart projects, however this comes at a cost when translating into real applications:
- their fixed architectures difficult adaptability to new robotic scenarios. Additional demands often require additional hardware which usually imply additional system integration.
- their general-purpose nature leads them to time inefficiencies, which impacts determinism (hard to meet hard real-time deadlines)
- their power consumption is generally one or two orders of magnitude above specialized deeply pipelined compute architectures realized with FPGAs and ASICs.
Acceleration with dedicated compute architectures (in either FPGAs, GPUs and ASICs) is presented as an alternative to CPUs. One that allows to adaptively generate custom computing architectures to meet the robotic demands delivering a mixed-criticality solution that can comply with real-time requirements, increasing reliability and lowering power consumption. Particularly, we at Xilinx know that FPGAs are heavily used by popular industrial manufacturers as well as in automotive, medical and space robotic applications.
If this proposal were to receive enough attention, we plan on moving forward into a formal WG. We’re seeking here to drive the discussion on acceleration and engage with hardware, software and embedded engineers, as well as other silicon vendors and groups to jointly get more involved, coordinate and contribute in the open with accelerators that impact ROS 2 and Gazebo flows.
Here’s the proposal:
Drive creation, maintenance and testing of acceleration kernels on top of open standards (C++ and OpenCL) for optimized ROS 2 and Gazebo interactions over different compute substrates (including FPGAs, GPUs and ASICs).
Hardware Acceleration WG will focus on reducing the time (real fast, as opposed to real-time) to compute Gazebo and ROS 2-based robotic flows by producing specialized compute architectures that rely on FPGAs, GPUs or ASICs. Alignment with various other WGs is expected on specific acceleration topics (e.g. with
Real-time WG to come up with dual real-time (deterministic) and real-fast (low latency) accelerators, with
Embedded WG to leverage MCU-based ROS 2 implementations in soft cores, with
Navigation to ensure AMCL accelerators meet their needs, etc.).
Initially, accelerators will target ROS 2 underlayers to optimize interactions betwen nodes within the ROS computational graph. After this, we’ll pivot into compute architectures at the application level with initial consideration for a) perception, b) actuation, c) navigation and d) manipulation (in no specific order for now). The acceleration kernels will initially target hardware embedded devices but future efforts will attempt to extend the kernels to workstations, data centers or cloud as applicable.
The work of this WG will leverage open standards for parallel computing and acceleration, namely OpenCL and libraries built on top. While maintaining a ROS-centric experience, we expect ROS users to be able to adapt and reuse the resulting kernels for their applications. For that, out of our work we expect to produce tools and conventions that facilitate including hardware acceleration seamlessly into the existing ROS build system (
ament) and meta build tools (
colcon). To facilitate initiation and engagement, reference acceleration architectures, designs and examples will be produced. Ultimately, testing and benchmarks will be proposed to evaluate accelerators across the power-consumption and time domains.
The WG seeks to do the following for ROS 2:
- Design tools and conventions to seamlessly integrate acceleration kernels and related embedded binaries into the ROS 2 computational graphs leveraging its existing build system (
ament) and meta build tools (
colcon). In a nutshell, we want to build awesome tools and conventions for running ROS components while leveraging hardware acceleration in a simplified manner. ()
- Provide reference examples and blueprints for acceleration architectures used in ROS 2 and Gazebo. First targeting ROS 2 underlayers. Second, stacks on top of ROS 2 and Gazebo. ()
- Facilitate testing environments that allow to benchmark accelerators with special focus on power consumption and time. ()
- Survey the community interests on acceleration for ROS 2 and Gazebo. ()
- Produce demonstrators with robot components, real robots and fleets that include acceleration to meet their targets. Disseminate results and show how ROS 2 fulfills industry needs. ()
Selected acceleration-related past threads:
- ROS acceleration using Xilinx AI Edge computing
- Challenges of GPU acceleration in ROS
- 10x acceleration for point cloud processing with CUDA PCL on Jetson
- Hardware Accelerated Depth Camera Simulation
- Huawei is cooperating with ADLink to accelerate Edge AI computation
- Unified ML Inference in Autoware - A proposal
Pinging a subset of the folks which I’m aware showed past interest in acceleration and hardware architectures: @rgov, @ak-nv, @pedrombmachado, @YangZ, @Flemming_Sundance, @tomoyafujita, @MartinCornelis, @arthur.berkowitz, @jopequ, @kscharan