ROS Resources: Documentation | Support | Discussion Forum | Service Status | Q&A answers.ros.org

Maximum number of nodes in ROS2

Next Generation ROS
#1

I’ve been playing around with launching lots of nodes in the ROS2 alpha7 infrastructure.
I’m facing a hard limit of 10 maximum nodes.

After snooping around in the opensplice docs and google searching the issue, I think I’m running out of Participant IDs.

After reading up on the Alpha8 release notes, the Opensplice portion of this issue isn’t as relevant, but maybe FastRTPS will suffer from the same issue?

Is there a design methodology for supporting more than 10 nodes in a subsystem that skirts the Participant Id issue? Or is there a way to increase the available Participant Ids to allow more than 10?

Has anyone else encountered this issue, or have recommendations?

Thanks.

ROS graph information tools implementation discussion
#2

It should be possible to have many more than 10 nodes. I’d assume this is a bug in our code. I’ll try to reproduce it when I get a chance.

#3

The error I get is on the DDS layer. It is:

rtps_init: failed to find a free participant index for domain 0

For your reference, I looked at this Vortex documentation:
http://download.prismtech.com/docs/Vortex/html/ospl/DeploymentGuide/guide.html#configuration

specifically these sections appeared to me to be the cause of the failure:

12.7.3.6. MaxAutoParticipantIndex
This element specifies the maximum DDSI participant index selected by this instance of the DDSI service if the Discovery/ParticipantIndex is “auto”.
Full path: //OpenSplice/DDSI2Service/Discovery/MaxAutoParticipantIndex
Format: integer
Default value: 9
Occurrences min-max: 0-1
12.7.3.7. ParticipantIndex
This element specifies the DDSI participant index used by this instance of the DDSI service for discovery purposes. Only one such participant id is used, independent of the number of actual DomainParticipants on the node. It is either:
auto: which will attempt to automatically determine an available participant index (see also Discovery/MaxAutoParticipantIndex), or
a non-negative integer less than 120, or
none:, which causes it to use arbitrary port numbers for unicast sockets which entirely removes the constraints on the participant index but makes unicast discovery impossible.
The default is auto. The participant index is part of the port number calculation and if predictable port numbers are needed and fixing the participant index has no adverse effects, it is recommended that the second be option be used.
Full path: //OpenSplice/DDSI2Service/Discovery/ParticipantIndex
Format: string
Default value: auto
Occurrences min-max: 0-1

The crazy thing is that if I try to add these fields into the ros_ospl.xml file, DDS can’t recognize them and fails.

#4

I was aware of the 120 participants per machine per domain limit, I’m not sure why it’s starting out at 9. Again I’ll try to reproduce it and try changing the configs when I have time.

#5

After playing with this some more, changing the ParticipantIndex tag in the ros_ospl.xml file actually works, and setting it to ‘none’ allows 10+ nodes to launch successfully. Of course, you loose unicast discovery, but this is at least a temporary workaround.

#6

Cool, sounds like a workaround for now. I probably won’t pursue it further since we’re going to be moving away from OpenSplice, but if you think it makes sense to change this open a pr. Or you could add a wiki page which describes how to change this setting.