Happy new year! As you may know, a new year means a new debate about ROS tooling… /j
If your reaction to this title is, “not another beginner coming to this forum to voice their displeasure with bespoke build tools within the ROS ecosystem,” I’d encourage you to read it before responding, as I believe the ideas expressed here are substantially different from previous posts.
I have read through essentially endless arguments on this forum regarding bespoke build system and package distribution tools within the ROS ecosystem, as well the Universal Build Tool design document.
Based on the doc, and personal experience, the technical requirements of a build and packaging system for ROS are very (uniquely) challenging because:
- Users must be able to develop multiple packages simultaneously
- Packages must be able to be written in any language and built with any build tool
- Packages must be published for specific ROS releases, which need to work on multiple operating systems
- I might be missing something
This is indeed a very complex problem, and it’s one that “just using cmake” does not come close to solving. Usually, the anti-bespoke-tooling-crowd is told to put in the work to think of a better solution or replace colcon
/ament
, which is understandable, since there aren’t very obvious better solutions (foreshadowing).
The current approach taken to solve these problems in the ROS ecosystem is through colcon to build arbitrary numbers of C++ and Python packages with ament
, rosdep
to wrap operating-system specific package managers, and vcstool
to wrap version control systems. In theory, these provide necessary abstractions for ROS users to effectively manage their packages and dependencies.
As someone who has been working on software infrastructure for an autonomous vehicle, which depends on the full extent of ROS2 tooling, along with podman
, to handle everything ranging from nicely packaged device drivers on ros index to git repositories with outdated academic paper implementations… I have found that these tools do little to mitigate dependency hell.
“Oh God This Guy Is Gonna Start Preaching Nix”
I recently started building a project with Nix for unrelated reasons, and as I was learning, I couldn’t help but think of just how useful this would be in ROS projects. Someone much more talented than me has put in the work already to make this possible, with nix-ros-overlay. This project automatically rebuilds and distributes every/almost every package in supported ros distros’ package indexes as nix packages. I am writing this post because I couldn’t find any discussion on here about the project.
To very briefly describe what Nix is: It is a package manager based around representing “derivations” (build steps) as pure functions, meaning all package builds are fully reproducible because they are not impacted by system state. This is accomplished via a lazily evaluated functional programming language/DSL with the same name.
Some of the benefits from building and distributing ROS packages through Nix (flakes) are:
- All packages are reproducible, and can be built from source
- This is very beneficial in the context of the ROS ecosystem, because engineers are often utilizing tools published by researchers based on old tech stacks and not actively maintained.
- Significantly less maintenance burden
- Any bespoke ROS build tooling is either optional or redundant. For example,
nix-ros-overlay
can wrapament-cmake
, but projects written with nix can get away with using standardcmake
for building their C++. Projects likevcstool
androsdep
are replaced by a more programmatic solution, since Nix derivations deterministically properly handle dependencies from any source, rather than rely on a long file mapping ros index names to os-specific package index names.- Note that Nix packages can’t currently be published directly to
nix-ros-overlay
, so they need to be built and released to the ros package index with standard tooling.
- Note that Nix packages can’t currently be published directly to
- Significantly less effort is needed to spend ensuring each ROS release corresponds to specific versions of Ubuntu with specific packages versions that can be depended on. This is because ROS packages built with Nix can directly specify commits of
nixpkgs
to depend on, with their desired package versions.
- Any bespoke ROS build tooling is either optional or redundant. For example,
- More standardized tooling
- While Nix documentation is very far from “good,” it has more comprehensive documentation than a lot of ROS tooling, and is improving.
- No need for containerization
- Since nix packages can be installed and built reproducibly on any system that runs Nix (Linux and MacOS natively, Windows with WSL), there’s no reason to stuff all your ROS tooling and projects into a docker container, which seems to be the approach favored by a lot of ROS projects.
In the interest of fair discussion, there are some notable drawbacks I can think of:
- Nix files, unlike
package.xml
, are written in a (relatively simple) turing complete DSL. Although this can be managed by templates, it definitely spooks developers in the same way someone who only learned python might be horrified by cmake. - Nix flakes, a standard that is pretty essential to using and distributing reproducible packages, along with a better CLI, have been “experimental” since 2021, despite a large portion of the Nix ecosystem depending on their functionality. There seems to be a lot of this kind of weird drama within the nix community.
- The ROS ecosystem does not currently use Nix
- This is pretty straightforward; no one likes change or being forced to learn something new.
I believe the benefits of using Nix for building and packaging projects within the ROS ecosystem outweigh the drawbacks, and that it should be a topic of discussion, but am curious what others think.