ROS2 speed

General
,
21

I’m so embarrassed. I mis-read the README in your repo, and thought the goal was 5,000 elements per message, but then I re-read and saw 50,000 :rofl: so a few more games are required:

  • replace the array of builtin_interfaces/Time with primitive arrays of secs and nsecs. Otherwise it has to call the serialization/deserialization functions for that type.
  • set the UDP socket buffers much larger. I set them all to 64 MB just for fun.
  • pre-allocate the message outside the main loop, rather than allocating a new MsgType every time.
  • for Cyclone, set MaxMessageSize and FragmentSize to very large values, along with increasing WchHigh, in a cyclone XML file. Documented here: GitHub - eclipse-cyclonedds/cyclonedds: Eclipse Cyclone DDS project

With those steps, it sends fine (no dropped messages) at 50,000 elements on FastDDS and Cyclone, and on Cyclone it goes up through 150,000 elements per message (at 1 KHz) without any drops. That is just about maxing a core for each (publisher and subscriber) on my machine, so if you actually do anything to the data, it would start having dropped messages unless you’re careful with multi-threading. I didn’t spend time tuning FastDDS but I’m sure it could also go well beyond 50k elements with some tuning effort.

In general though, I think this is probably close to the limit of what DDS (UDP) can do. I’d suggest to use the composition feature of ROS 2 as well as using the new shared-memory features of both FastDDS and Cyclone (iceoryx), some of which may need fixed-size messages in order to work nicely. It’s a lot of data to throw around.

3 Likes