ROS2 in large architectures

Quality Assurance
,
1

Good afternoon!

We are a team of developers currently discussing a possible adaptation of our current framework into ROS2. As a bit of context, we have a decent sized framework for robotics (>100k lines) and we are looking into exporting parts of it into ROS2 packages. Most of the work would be switching our “in the house” communications into ROS2 comms such as nodes, services, and actions.

We want to make sure that the initial structure of the system is correct and high quality before committing to the change. After some research, we have reached a point where we don’t exactly know what would be the best way to proceed, and we have several points where we couldn’t find a clear answer:

In our current framework, the equivalent of ROS2 packages are modules, so I might refer to them as synonyms.

In our framework we have some modules that have basically no dependencies (math modules, control, etc…). We would like to keep these modules “dependency free” for anyone to use inside and outside ROS2. As far as we’ve seen, creating a pure CMake package is not feasible as ament_CMake is used in ROS2, or we haven’t seen any examples at least. Ideally, we would like to maintain them in CMake and then use them in several of our packages without having to introduce them in the sources of all the other packages that use them.

MoveIt2 has a module that they claim to be non-dependent on ROS2 in their README (moveit_core: moveit2/moveit_core at main · ros-planning/moveit2 · GitHub), however, in their files they use the ROS2 logger with the rclcpp dependency which makes us doubt this claim. Are we missing something or is this module still ROS dependent even if they claim it’s not?

Overall, it seems that using pure CMake to create a package to use easily in other packages without copying it everywhere is not easily obtainable. Is this the case? Is there a way to achieve this?

We might have a situation in the future where we have a CMake module where some implementations are done with ROS and others are not (such as an implementation using a ROS2 algorithm). Then we would like to have this module be able to compile standalone and inside a ROS2 workspace. Hopefully, an example will make this clearer:

Imagine a case where we have a module or package called slam_module. Inside this module, we have two implementations. The first one is our personal SLAM algorithm (with no ROS2 dependencies), the second implementation could be a wrapper of a ROS2 package. We would like to use this module inside of a ROS2 workspace but also outside of it (not compiling the ROS2 dependent implementation). This would mean that the main CMake in the module cannot be ROS2 dependent.

Making our modules or packages open-source as of this time is not possible (WIP!), which means that publishing the packages to manage them with rosdep is also not possible yet. These packages are needed by other packages to compile. As far as we understand, we’ll need to manage these dependencies ourselves to ensure all the packages are available before compilation.

It’s not clear to us if using a monolithic repository to store all of our packages is the best practice, or if having several divided repos is preferred. It seems like one repo per package would be ideal, but large packages such as MoveIt2 and Navigation2 have one repo with all the packages inside. Having one repository per package also messes up the pipelines since a multitude of small packages are needed to compile some large packages and dependencies would have to be manually taken care of. Is there some recommended best practice regarding these different structures?

The last option would be to not make our framework fully integrated into ROS2 or just make it compatible with it by changing the comms and generating some bigger packages instead of all the small ones.

We were wondering if any of you encountered these issues and had a clear perspective on how to tackle them.

Thanks for any help or advice!

1 Like
2

You can use plain CMake packages in ROS 2. colcon will happily build a plain CMake package. What ament_cmake is for is making certain things we do in ROS easier, such as generating messages. If you do not do things like generate messages, then making a plain CMake package will be fine. This includes using standard CMake calls to depend on ROS packages.

There are two options here: One is to manage your own buildfarm and package repository infrastructure, which would allow you to have your packages available to you and you alone as binaries. There are many companies that do this. The second option is to rely on having all the packages available in the workspace as source; this can be easily done by creating a .repos file (see for an example the .repos file for ROS 2) that lists the repositories that should be in a workspace.

This is really a question of your preferred workflow. The main difference between one repository and many repositories is that when you create releases from those repositories, the ROS tooling for creating releases can only function at the repository level. What this means is that all packages in a single repository must be released together. If you will always be versioning and releasing your packages together then a monolithic repository will be fine. If you want to make separate releases of individual or smaller groups of packages, you should consider splitting up your repository into smaller chunks.

I think this would be a significant amount of work, due to the need to match all the data types.

1 Like
3

Our company is in a very similar boat - looking to slowly migrate parts of our in-house framework (beginning with comms) to use ROS 2.

It is important for us to maintain our current approach as we have large projects mid-development, while allowing for “experimentation” with ROS. We have found the following:

  • Packages that are pure CMake work fine
  • Packages that are fully ROS (built with ament) work fine
  • Packages that we want to be able to build both without ROS and with ROS (e.g. to include constructors/casts to ROS types, which are macro’d out for non-ROS builds) are a real mess since we need to toggle between using ament functions and the regular CMake functions (e.g. target_link_libraries and similar).

We ended up writing a CMake macro to handle it all as well as a few other things that we were doing for every package which cleaned it up a bit.

Interested to see how others have tackled similar issues.

1 Like
4

Thanks for the answers!

We took a look at the .repos file and it sounds like a neat solution! This helps us with our main problem of having separate repositories. Since most of our modules have dependencies on others, we believe having separate repositories for each module it’s probably the best way forward. This way we can version each package separately and then include them with the .repos file as you commented.

That’s exactly our current situation, it’s super helpful to see that your company is also going through the same process and to hear your experiences!

That seems to be the general opinion, for now our approach will probably be to depend fully on ROS if we have at least one implementation that depends on it. If we don’t have any, then we’ll do a “pure” CMake module for that package.

Thanks a lot for the answers! We’ll probably come back with more questions, so we welcome any other views or experiences regarding these topics!

1 Like
5

Reading these lines, I have the feeling that maybe these libraries would best be split into two - one core library without ROS dependencies, and ROS bindings library. I know it might be difficult sometimes, but looks to me like a much cleaner development pattern. Something like GitHub - norlab-ulaval/libpointmatcher: An Iterative Closest Point (ICP) library for 2D and 3D mapping in Robotics and GitHub - norlab-ulaval/libpointmatcher_ros: A bridge between libpointmatcher and ROS.

1 Like
6

For what it’s worth, regarding the monolithic repo thing, our approach (which we do NOT necessarily think is a great approach, it’s just a hangover from how we have been doing things for a while) is to have repos with one package, or a few related packages, and then we have “workspace/project” repos which store each of the packages used in that project as submodules.

We don’t have a thorough semantic versioning system for packages at the moment, instead we rely on those top-level repos to provide an indication of a set of compatible commits of the package repos. Although this approach served us well in the early days it is long overdue for an overhaul as we have more (and varied) projects in the field.

One complicating (though not unique) factor for us is that we are working on very large industrial equipment, where deploying anything but the most trivial updates would incur lengthy, costly recommissioning periods. As such, if a client requires (for example) a new feature to be added to their existing system, rather than upgrading to the “latest-and-greatest” version, the feature will be backported to whatever version their system was commissioned with, resulting in a somewhat fractured version history (but the total number of deployments is sufficiently low for this to be manageable).

7

You may be right. It’s pretty early days in this “experiment” for us, and one of my colleagues (who deals with our lower-level packages) took the “integrated” approach - I didn’t really mind as long as my top-level code could interact with it easily!

It may be that upon further development we find a way to break it off more neatly as you’ve described. We do like being able to have constructors and cast operators (which as I understand it cannot be trivially implemented separate to the class in C++), not to mention it makes prototyping a bit easier (e.g. as we give our sensor drivers the ability to communicate with either legacy or ROS comms, we can put the ROS code right next to the legacy code but inside an #if).

8

I understand the constructor problem. You’d just have to create a wrapper class and construct all ROS-enabled instances via the ROS-enabled class… I don’t say it’s the most comfortable approach. Cast operators should be definable as free functions, so this could work.

Another option would be to implement a ROS “mock” library, that just provides the headers and empty function bodies, and compile your library either against raw ROS, or against this mock. I use it with one of our libraries that is difficult to install, most users don’t need it, but its headers appear all over the code. CMake autodetects whether it is installed or not, and depending on that uses either a mock or the raw library.

2 Likes
9

Thanks! Some good food for thought :slight_smile: