ROS2 Build Tools & Systems

[Satco]

I’m quite happy with the build system This thread looks like a fun Christmas activity Anyway I’ve had way more quarrels with CMake than with colcon. But yeah building custom messages is tricky

[tylerweaver]

I’ll just point out that we could have infinite nice things if people built and open-source them. No one here is trying to make a flawed system; there are just way more things we could work on than we have time. If you want a better, more ergonomic, more debugable, simpler-to-use build system… you are free to build it. If you open-source it in a way we can all use, that would also be wonderful.

If you look at nice things in software, you can use for free… say python. These things took mountains of work to get to where they are now. You can be part of climbing that mountain if the build tools we use are what you want to fix.

I hate CMake as much as everyone else for how much time it seems to steal from me… but I still haven’t managed to escape it in any meaningful way. If you do, please try to make it easier for someone else who might try to follow you.

[mjcarroll]

@Satco

This thread looks like a fun Christmas activity

Take all that pent-up frustration from spending time with the in-laws and use it towards something productive like bikeshedding buildsystems.

It may be healthier to just take a walk around the block, but certainly not as entertaining.

[doganulus]

@tylerweaver

If you look at nice things in software, you can use for free… say python. These things took mountains of work to get to where they are now. You can be part of climbing that mountain if the build tools we use are what you want to fix.

This is why we want to use regular C++ or Python or Rust tooling. Custom tooling is a huge effort. Apparently the ROS community does not have resources and ambition to keep up with their own tooling (based on regular comments here). It is outdated, no one has intention to improve it.

Please prove me wrong. Show me your intention to improve… Then I will give concrete suggestions.

[tylerweaver]

@doganulus

Please prove me wrong. Show me your intention to improve… Then I will give concrete suggestions.

There is a saying about what everyone has… something, something, suggestions. Build something real and helpful, and then people will take you seriously.

Also, C++ tooling… get out of here with that nonsense. C++ has no concept of tooling, it doesn’t even consider computer architectures to be a real thing.

[gavanderhoorn]

(I do take this seriously though. It’s just the nth thread about this, with the usual suspects (look, even I commented again ), the usual critiques and the usual expected result …)

[mjcarroll]

@doganulus

This is why we want to use regular C++ or Python or Rust tooling

I’m confused on this point because we use CMake (with some additional functions/macros), which I think is a generally accepted “regular” tool for C++. And on the Python front, we use setup.py and setuptools, which are generally accepted standards. I’m not sure what candidates there would be for doing “less” while also maintaining the convenience of the build/packaging/distribution that users currently expect.

The “custom tooling” in this case are things like ament/bloom, which handle the packaging and distribution of ROS packages, and colcon, which lets you build cmake packages in dependency order. I would say that these are more of convenience tools, as you could do without (but certainly not be as productive).

[doganulus]

@mjcarroll

The “custom tooling” in this case are things like ament/bloom, which handle the packaging and distribution of ROS packages, and colcon, which lets you build cmake packages in dependency order. I would say that these are more of convenience tools, as you could do without (but certainly not be as productive).

By regular tooling, I mean CMake, Meson, or plain Makefiles. I do not suggest any particular tool but you know CMake is the most popular. Similar to Boost libraries do not dictate a particular build tool when using, I expect core ROS libraries should not dictate any build tool when using. You can keep your existing ROS tooling in place but enable a path for those who do not want to use them.

I wouldn’t oppose ament packages if it is distributed as regular cmake scripts. For example, distribute them at /opt/ros/humble/cmake or /usr/local/share/cmake/ros/humble but not as a package dependency. But still do not dictate their usage nor recommend. Keep things as vanilla cmake as much as possible.

As I said above, Colcon is harder to avoid because of local workspaces but still avoidable by moving away from them… For example, if I want to use Boost libraries from source, I clone it and run its build script and install it a regular location. Third-party ROS packages can be installable similar. Clone it, run its build script and install it a conventional place.

Bloom is a background tool. The Foundation can use any tool there.

I appreciate comments addressing these technical points. Then we can finally discuss the core.

[clalancette]

@cst0

The reason I joked about ‘‘this conversation again’’ is because I’ve seen versions of this thread a handful of times before, and we didn’t come up with a concrete solution that does it better than what we currently have. We can debate pain points all day, but I’ve yet to see a concrete technical contribution that resolves them.

Yes, I have to agree completely with this. I certainly don’t think that what we have is the best of all possible worlds. But I have yet to see other workable proposals that solve the unique problems that we have. At the very least, any solution would have to:

1. Deal with multiple different build types. Despite the initial assertion in this thread, it is extremely common for robotics projects to use multiple languages, and hence multiple build systems. Python and C++ are the two major ones today, but I expect Rust will join that group in the next year or so.
2. Deal with a distributed ecosystem of packages.
3. Allow overlays to work properly. This allows you to use a “base” of packages, and then add/replace ones that you need to.

Finally, I will note (as others have) that colcon, in particular, is not required in any way. We know this for a fact because we don’t use colcon to build the binary packages on the buildfarm. However, we do believe that colcon makes things far more practical for many developers, which is why we have it and use it. (ament is somewhat less optional, since it knows how to place and find things on the filesystem. But it actually is possible to write a ROS package without ament at all. Pull requests to make this easier are, of course, accepted)

[Phil_Brown]

Before one can discuss solutions, one must first define the goals.
Where is there a clear, but high level description of the goals the ROS build system needs to achieve?

[doganulus]

@clalancette

Deal with multiple different build types. Despite the initial assertion in this thread, it is extremely common for robotics projects to use multiple languages, and hence multiple build systems. Python and C++ are the two major ones today, but I expect Rust will join that group in the next year or so.

Yes, but use native tooling of those programming languages. Treat them as regular C++, Python, and Rust projects. Treat ROS as a library in those languages. Even consider distributing rclpy or rclrs using their package managers.

@clalancette

Allow overlays to work properly. This allows you to use a “base” of packages, and then add/replace ones that you need to.

I would say that this requirement must be reconsidered. If we need to modify a base (third-party) package, a repository fork must be needed. Then install it using regular tooling as usual (explained in the previous post). Move ROS packages towards repository-first approach rather than local-first. Declare dependencies by remote repository links (like Go) rather than rosdep keys.

@clalancette

Deal with a distributed ecosystem of packages.

Consider package collections as the fundamental unit of distribution to reduce the burden on build systems. Consider collection.yml instead of package.xml. Declare dependencies by collections, etc.

[tylerweaver]

How do you propose a Rust project depends on a C++ project? Are we going back to manually creating debians for every library and having to look up seemingly random conventions for every dependency. In the wild west of C++ code there is no standard for what a C++ library even is, tools just try to guess by looking at your filesystem.

[cst0]

Where is there a clear, but high level description of the goals the ROS build system needs to achieve?

Take a look at the ROS 2 design documents. The relevant one for this conversation is linked at the very top of this thread by gbiggs.

[doganulus]

@tylerweaver

How do you propose a Rust project depends on a C++ project? Are we going back to manually creating debians for every library and having to look up seemingly random conventions for every dependency. In the wild west of C++ code there is no standard for what a C++ library even is, tools just try to guess by looking at your filesystem.

If we use different languages, we must deal with different conventions. Trying to wrap them under the same interface is futile. ROS repositories must be able to declare dependency to other ROS repositories whether they are C++ or Python or Rust projects. And they must be self-installable from the repository. We can discuss the mechanism for this. This is why we are here. I am not sure about the recursive dependency management though.

[tylerweaver]

So you know, the current ROS thing is much better than the standard in the Rust ecosystem for declaring dependencies on non-crates.io dependencies. It is the same bad wild west as C libraries declaring their dependencies on other C libraries (readme’s that say to “apt install things” which are mostly useless to anyone not on the specific version of debian as the author of the package and not automatable). pixi seems like it might help in that area… but we’ve yet to figure out how to meet the other needs of the ROS project with it.

[mjcarroll]

@doganulus

I wouldn’t oppose ament packages if it is distributed as regular cmake scripts. For example, distribute them at /opt/ros/humble/cmake or /usr/local/share/cmake/ros/humble but not as a package dependency. But still do not dictate their usage nor recommend. Keep things as vanilla cmake as much as possible.

So this isn’t terribly far from what we currently do, ament just adds a little convenience. Packages should install their cmake into <CMAKE_INSTALL_PREFIX>/share/<PACKAGE_NAME>/cmake which can be used downstream without ament.

For existing ROS packages that are installed correctly, you should be able to find_package() and target_link_libraries() to build a library/executable that depends on that package.

For example, this should be sufficient to build a node with dependencies on rclcpp and std_msgs:

cmake_minimum_required(VERSION 3.12)

project(demo_node)

find_package(rclcpp REQUIRED)
find_package(std_msgs REQUIRED)

add_executable(my_node main.cpp)
target_link_libraries(my_node PRIVATE rclcpp::rclcpp ${std_msgs_TARGETS})

@doganulus

As I said above, Colcon is harder to avoid because of local workspaces but still avoidable by moving away from them… For example, if I want to use Boost libraries from source, I clone it and run its build script and install it a regular location. Third-party ROS packages can be installable similar. Clone it, run its build script and install it a conventional place.

colcon is actually easier to avoid. It merely automates building and installing the packages in the correct order. It actually goes unused in the buildfarm, and is a non-essential tool other than developer convenience.

Edit: I meant to say here that you can git clone && mkdir build && cd build && cmake .. && make && make install ROS packages if you do them in the correct order. There isn’t anything particularly magical there.

@doganulus

ROS repositories must be able to declare dependency to other ROS repositories whether they are C++ or Python or Rust projects.

While it may not handle every case, this is precisely package.xml encodes. You can then fetch the corresponding dependencies based on the rosdep keys for your platform/configuration.

[smac]

I think an example project of non-trivial complexity using only raw cmake would be rather illustrative and provide something to reference whenever this conversation comes up.

I could make a cute little lidar detector lib package and a ROS wrapped node package that pulls in TF2, some messages, PCL, and whatever else I need. If someone more familiar with ament’s internals could open a PR to rip out the ament stuff in favor of cmake & provides the build command to invoke to use as a demo to show “hey, for real, you don’t need this”, that could do some of the heavy lifting to have a practical starting point showing the basics (e.g. pulling in both ROS and non-ROS deps; including some in the same repository/workspace).

I see two major problems here (among other sub-problems like Bazel support or whatnot) related to “you don’t know the depth of the unknown”. Shedding some light could help to at least remove the “depth” problem.

(1) The docs explain the “what” but not the “how”. What exactly is ament exporting when it does so, and what is being found and set when we perform actions later to use the library. It seems to me we’re missing a REP which describes what ament is doing, what cmake variables it creates, and what each represent in context.

(2) An example of someone actually having a package that can be ROS 2 and involve neither ament nor colcon in its CMakeLists & build instructions.

[tylerweaver]

We’ve been trying to go ament-free with a tiny package that MoveIt and most of our code depends on called RSL: GitHub - PickNikRobotics/RSL: ROS Support Library

Credit goes to @ChrisThrasher for taking on this effort. His latest change is getting compatibility with ament_target_dependencies when users choose to use that over target_link_libraries. I’m working on testing that change right now as I type this.

Currently, the only thing that doesn’t work is underlay source builds. We hope to figure that out soon, or maybe one of you knows the magic sauce?

Note that there is much more complexity in wanting to publish a library that others use that doesn’t have an ament dependency. If you are building some leaf thing that you never plan on packaging, it is way easier.

[mjcarroll]

@smac

I could make a cute little lidar detector lib package and a ROS wrapped node package that pulls in TF2, some messages, PCL, and whatever else I need. If someone more familiar with ament’s internals could open a PR to rip out the ament stuff in favor of cmake & provides the build command to invoke to use as a demo to show “hey, for real, you don’t need this”, that could do some of the heavy lifting to have a practical starting point showing the basics (e.g. pulling in both ROS and non-ROS deps; including some in the same repository/workspace).

I would be willing to attempt this. Like @tylerweaver said, there is a bunch of boilerplate that would be required if you actually wanted to package said library, but maybe that is out of the scope of the initial conversation here.

[mjcarroll]

I think another good canonical example of this is PlotJuggler like this (PlotJuggler/CMakeLists.txt at 54bd6836ca571bfadb68acaece19b94603e03f99 · facontidavide/PlotJuggler · GitHub)

Basically, if catkin is found, do some catkin-specific things, if ament_cmake is found, do some ament_cmake-specific things, but otherwise just be a CMake package until someone tells you otherwise.