ROS Resources: Documentation | Support | Discussion Forum | Service Status | Q&A answers.ros.org

ROS 2 Conan Integration

Hi everyone,

with this post I would like to start a discussion around one of the challenges that people face when they look into transitioning to ROS 2.

As we all know, ROS 2 is an highly-federated ecosystem of packages and comes with its own build tools and package managers (e.g. colcon).
In my opinion, these tools work great when you are developing a new project from scratch and you want to completely embrace the ROS ecosystem.
On the other hand, if you are in a situation where you already have a large project and you only would like to be able to add ROS 2 to it, the aforementioned tools may not look like the best solution.

To address these issues, in the past few months Eric Riff and I, from iRobot, have developed a set of tools that allow to integrate ROS 2 into Conan, one of the most popular C++ package managers.
If you are not familiar with Conan, I invite you to have a look at the extremely well-written documentation that can be found on their website.

As a quick summary, the Conan package manager is based on the concept of “recipe”, i.e. a Python script that provide instructions for building and packaging a given library. It specifies its name, the version/s, sources and dependencies. It also provides a list of options which can be used to enable/disable functionalities or change certain aspects of the library to be compiled
Some of the great features of this approach are:

  • Recipes allow to easily fetch pre-built packages and use them in your application
  • There is a lot of support for cross-compilation
  • You can pin exact versions of packages
  • Conan is open-source and decentralized, so you can have your own repository of recipes and server of packages

We created a Github repository where you can find some Conan recipes that can be used to build ROS 2 packages, together with an example application that consumes them.

The core of the contribution consists in some base-class recipes that allow to easily write generic ROS 2 recipes.
Then it’s possible to choose any type of granularity: you may have a single recipe that contains all the ROS 2 libraries that you want to use or you may want to separate ROS 2 build tools (ament) from the ROS 2 core libraries (which need to be cross-compiled) and from the developer tools (rviz, rqt, etc).

The approach has still some areas where it can be improved: for example conan recipes require to manually list all the dependencies, which will become very tedious and error-prone the more recipes you have (especially in the extreme case where you may have one recipe for each ROS 2 library)!

We would like to hear feedbacks from the community to be able to improve this infrastructure and, hopefully, to have officially released ROS 2 Conan recipes one day!

Thank you,
Alberto

8 Likes

:+1: for Conan in general.

I’m wondering though:

How does using Conan support the use-case (requirement? constraint?) you mention specifically (ie: being able to add ROS 2 to it, instead of completely embrace[ing] the ROS ecosystem)?

(I expect you know / have figured that out, but your post seems not to make it explicit)

Another +1 for Conan in general.

2 Likes

This is the problem that tools like bloom (bloom - ROS Wiki) and superflore (GitHub - ros-infrastructure/superflore: An extended platform release manager for Robot Operating System) were designed to solve. We have all this information via the package.xml and rosdep, and we use it to create deb and rpm artifacts.

I experimented with generating Homebrew formulae for macOS but it stalled due to the moving target of macOS+homebrew.

If you don’t use a tool like this, then you have to maintain the recipes manually and as you suggested this would be very time consuming.

It seems to me that generating these Conan recipes from bloom and then having aggregating packages to combine them might make sense.

At the very least you could use the same api and logic bloom uses to automate the creation of your recipes, even if you end up grouping things.

In general I’m in favor of using newer tools like Conan, but I do think automating the integration is the most sustainable thing to do, even if it requires a larger initial investment.

6 Likes

Just for the sake of providing backlings to anyone exploring this topic, in Packaging Ignition with Conan - Projects - Gazebo Community there was similar discussion on packaging Ignition libraries for conan-center-index, that is the sort of main repo for conan recipes that I guess can be used as a source for non-ROS dependencies (the one usually provided by Debian/Ubuntu repos in apt distributions).

The approach has still some areas where it can be improved: for example conan recipes require to manually list all the dependencies, which will become very tedious and error-prone the more recipes you have (especially in the extreme case where you may have one recipe for each ROS 2 library)!

In this area, for RoboStack (the packaging of ROS for conda, another user-level package manager as Conan) @Tobias_Fischer worked on rosdep integration, you can read about his work in patch rosdep to work with conda-forge/robostack packages · Issue #41 · RoboStack/ros-noetic · GitHub , perhaps it can be interesting for your use case.

By the way, with this announcement and considering existing approaches in packaging ROS on conda, nix and vcpkg I think that the main user-level package managers for which no ROS packaging as been attempted so far are Spack and Julia’s BinaryBuilder/Yggdrasil, I wonder if interest for having ROS there will emerge as well (for BinaryBuilder I was just able to find a GitHub issue: Artifacts? · Issue #90 · jdlangs/RobotOS.jl · GitHub ).

For people reading mail notifications: the previous message was sent incomplete, and it is not has been fixed.

It’s great to see a lot of Conan support here!
Sorry for the late replies, but I just returned from vacations.

There are multiple reasons why we looked into doing this work.

Despite colcon being in theory a generic package manager, it is in practice used almost exclusively by the ros community. This implies that 1) there are way less libraries that already support it (e.g. that have a package.xml,etc) and 2) its features and design choices are quite specific to a ros-best-practice ecosystem.

A standard approach when starting a ROS project consists of creating one or more ROS workspaces containing all your required packages, build and source them and then build your project in a new workspace.
This may work ok with the ROS-model, where there are a lot of different repositories/packages, so the number of dependencies of each of them is not that big.
But it will quickly become problematic for example when dealing with complex packages with hundreds of dependencies (most of them non-ROS related) or with single-repositories-systems.

There are a wide variety of issues, that I think would require the creation and maintenance of complex infrastructure on top of colcon itself.
For example: being able to download pre-compiled packages with a specific version or for a specific architecture or having some dependencies be available only for certain architectures.

Conan solves most of these issues.

I don’t think colcon was ever intended to be a “package manager”, but is considered to be purely a build tool. It seems like you may be expecting colcon to also be a dependency manager, which seems out of scope for most build tools, let alone most build systems.

Most plane Cmake and python packages I’ve encountered seem to build fine with colcon, and I don’t think many folks expect cmake or setup.py scripts to automatically install dependencies onto your system, in fact I prefer that they don’t. Rust & cargo perhaps being one notable exception.

That is rather cool. How does conan currently automatically infer the build order of workspace dependencies; remaining OS and architecture agnostic? Is conan specific to only pure C/C++ projects with recursively pure C/C++ dependencies, or is it also applicable for building libraries and dependencies written in other programing languages? E.g. if I needed to build multiple C projects that have Fortran or Rust dependencies for numerical computing or for FFI with sensor SDKs?

1 Like

I sort-of agree with @ruffsl in that it looks like comparing Colcon to a package manager + build tool + dependency fetcher seems to conflate a few things. It’s expressly only 1 of those.

Separating responsibilities makes sense to me, but that doesn’t necessarily change your argument of course.

I would suppose it’s more appropriate to compare conan with rosdep. I do think colcon is trying to do too much (i.e traversing directories within your workspace looking for packages, running msg generation scripts, single build tool for c++ and python, etc.) I would hands down prefer to use industry (outside of ros) standard practices and tools like conan and straight cmake. Trying to switch our cmake project to using ROS2 as the middleware has been quite painful.

It’s been said elsewhere, but colcon doesn’t require you to use any kind of special cmake. You can build plain cmake stuff with colcon.

If you have more than one cmake project colcon will try to figure out the order, but if it is too complicated you might need a package.xml.

If you have just one cmake package then you don’t need colcon at all. Just source the setup file so that the cmake prefix path is set and build your project as normal.

It’s a common misconception that you have to use the ROS infrastructure to build on top of ROS but it isn’t really the case.

Colcon must find packages in your workspace, but it doesn’t run anything for message generation, that happens through cmake and would happen if you just ran cmake and make on the package with messages.

There’s no industry standard that mixes cmake and python in the way that we need, as far as I’m aware. Maybe Conan does that.

It sucks that you’ve had trouble migrating but I’m curious what the issues were because it’s likely something that can be worked around or avoided entirely if you prefer.

3 Likes

I think colcon fills exactly the gap it was designed to fill. And while I think there’s a slight learning curve to using colcon, it has been an asset. Being able to do colcon build --packages-up-to XXX and have it compile all the deps and the package itself is magical.

I have had less success compiling external ‘non-ament’ packages with slightly awkward build processes with colcon. I was never able to figure out how to tell colcon the exact steps it should take to compile it.

Back to the topic at hand:
Perhaps conan support should be introduced into rosdep as well?

Sorry for the very late replies, but I completely missed the last comments.

Unfortunately, there are some limitations in relying on the setup shell scripts.

  • They are not really relocatable, as you can only specify one path in the COLCON_CURRENT_something env variable, so this does not work if you built your ROS dependencies in multiple workspaces.
  • Using environment variables may cause problems as you may mix things up (e.g. the ROS package that you are sourcing and your host environment).

On a quick note, using a single ROS workspace is not a solution:

  • some packages don’t need to be cross-compiled
  • the ament CMake logic automatically searches for things to determine what ROS interfaces to generate: if both fast-dds and cyclone are found, both will be built, even if you may only care about one (small problem) and similarly if rosidl_python is found, python interfaces will be built (this is a major problem because if you are cross-compiling for a platform that does not have python, then the build will fail).

@wjwood how do these tools handle the versions of dependencies? these are not specified in the package xml