Challenges of GPU acceleration in ROS

[vmayoral]

@smac

I had no idea that was true, that puts a set of breaks on some work I had planned on supporting GPUs in some of the Nav2 algorithms. I’ll probably scale back that project to just multi-threading support now.

Myself and my team in the past bumped into the same situation with Jetson. We ended up moving to different compute substrates for that specific reason.

@smac

• Rviz
• PCL, a very important dependency
• Gazebo / ignition rendering
• BFL / filtering libraries / Ceres / g2o / etc

While this is true from a tooling perspective, I’d also mention ROS 2 users have additional priorities computational graph-wise which should be considered. E.g. it appears to be quite a hot topic to optimize interactions betwen nodes (inter-process, intra-process and even over the network). Accelerators on this end will add significant value to the overall ROS 2 computational graphs (and even to the underlying data layer graphs, which at the end of the day, matter as most in real deployments). In other words, while it’d be great to have OpenCL kernels to accelerate specific computations, real impact won’t be achieved unless a holistic ROS view is applied (bottlenecks identified, etc.)

Determinism and real-time are also important topics to consider. While GPUs have historically performed poorly on RT aspects, most robotic behaviors have some sort of deadlines. It’d be great to hear @ak-nv and team how pipelined accelerators in GPUs could help in here as well.

@ak-nv

Please let us know if anything specific Packages if you are looking for and we will certainly add it to our priority list.

Emphasizing one of @smac’s points above, I’d heavily encourage you and team @ak-nv to consider GPU accelerators for simulation. It’s a long dream of many to get such capabilities and this somehow aligns well with your current trend of AI-related demos. This can heavily impact ROS developers but also AI researchers (e.g. doing RL setups like this could heavily benefit from GPU simulation accelerations).

As a point of criticism @ak-nv, NVIDIA is somehow disengaged from the ROS community (with their Gems, its own physics engine not integrated in Gazebo/ignition, it’s own UIs, etc). IMHO there’re lots of wasted resources in here. ROS/Gazebo communities already provide much of that and there’s a decent level of “reinventing the wheel”. Why don’t you consider integrating PhysX in Gazebo and/or helping developers accelerate their simulations and integrate it in their DevOps pipelines (that’s a fantastic business case for selling more GPUs to roboticists )?