In case others haven’t seen this: Better ROS Builds with Nix - ROSCon 2022
7 Likes
Hi,
I seem to be a bit late to the party. I’m helping maintaining nix-ros-overlay and I use Nix and ROS heavily for an autonomous driving project at a university. I’ve also developed ros2nix project, which should make is almost trivial to turn any existing ROS project to Nix and which might answer some questions raised above.
I believe that Nix is very beneficial for distribution of ROS as well as for any other complex software. And I’m (slowly) trying to make use of ROS with Nix easier. For that, I don’t think that ROS needs big changes (like replacing package.xml), but what’s needed is more coordinated effort and I’m happy that from this thread it look like there might be some interest in that.
I’m suggesting people interested in ROS and Nix to come and discuss things at nix-ros-overlay. However, not everybody relevant is there (cc @nim65s).
If you have any question, suggestion or whatever, I’ll be happy to answer or help. Also, I have a long list of thing which could be improved in nix-ros-overlay in my mind, but don’t have time for all of it.
Below I answer some questions raised above, which, I think, were not answered.
What value is a reproducible build when its inputs are not available or its outputs cannot be run?
For the main nixpkgs repository, all sources are backed up at cache.nixos.org and as far as I know, Nix can fall back to download them from Software Heritage.
Do why do you think nix would make this any better. Researchers want to write code, not makefiles.
That’s what Nix makes easy. Once you compile your code with Nix, everybody can do it. No need to figure out how to find various libraries on different systems, which is the point of makefiles and similar.
Last, I’d actually wonder if it were easy to convert all of ROS to nix.
That’s what we’re trying to do in nix-ros-overlay. For Jazzy, approximately 90% of all packages builds. Not everything works well, but you can already do a lot.
However, I don’t believe a Nix based build system (with templates) would be more difficult to use than the existing ros2 pkg create … + adding a few package names to CMakeLists.txt and package.xml and getting to work. They’d likely work very similarly, only the template would include a flake.nix instead of package.xml.
I’ve also developed ros2nix, which automatically converts package.xml to a Nix expression. If works quite well so there is no need to change anything. In many cases, package.xml misses dependencies, but a simple PR can easily fix it.
I’ll have to make a few changes to nix-ros-overlay, since it currently doesn’t export any building functions, only packages (this may take a bit because I’m learning as I go).
This should not be necessary. See how ros2nix does that.
So if I collect 10 random research packages built with the hypothetical NixROS and put them in a single workspace to combine their capabilities, I’ll end up with 10 different versions of GCC (which I’ll need to build from source), 10 different versions of Boost, 10 different versions of rclcpp etc.?
It’s up to you. If you don’t care and your project works (e.g. no multiple versions of the same library linked together), you can do that. But you can also use the flake “inputs.*.follows” feature to compile everything against the same version of all packages. However, with “research” packages, this usual requires some patching.
7 Likes
Well, it needs… Because package.xml and simplistic ROS dependency management (rosdep keys) offers very little compared to full power of Nix, Bazel, and even Cmake. Hiding their power behind package.xml is like fighting with one hand tied in the back.
You need to get rid of every little ROS tools and be integrated into the larger software world. This includes ament, rosdep, colcon, and the rest. There is nothing special about ROS. This is the thing you need to accept first. And this is the point where the ROS leadership and community would struggle. (I honestly think, you gonna fail under this leadership).
However, you are right that this requires a coordinated effort more than individual demos and proposals. So @gbiggs may take notes from this discussion and bring them to the next TGC meeting to discuss with the corporate members.
This is a Foundation level task. They must take initiative if they want to be relevant.
Can you just…stop with the snide comments and veiled insults and try to be a bit more professional? Your signal to noise ratio makes it really hard to consume the technical insights you do have.
I can guarantee that the board and management are very aware of your personal feelings and opinions.
8 Likes
The bigger problem for you is that my comments tell the truth. These comments will continue to be true even if you silence me. Therefore, I would advise against your suggestion. Here, I perform an important function for open-source robotics. Therefore, your board and management should support me wholeheartedly instead of inviting me to stop.
Remember that I am just a single academic who does not have as many resources as you have. But I will continue to call out your wrong and outdated software engineering practices as a part of my community duty. If you take my comments personally, it means you are too attached to your precious software. This is always a path to failure.
I am very reasonable when I speak with sincere people, even if they talk to me harshly. But I am not your cup of tea if you are an expediency politician for sure.
I am not going to get drawn into a debate about how your experience in community governance or software engineering does or does not stack up to that of others in the community, no matter their background or affiliation.
But, no matter the strength of your arguments, the unprofessional, insulting, and off topic ways in which you express them are not only violations of the community code of conduct, they are also taking attention from collectively making much needed improvements to ROS. This thread alone is proof of that.
8 Likes
You give me too much credit for not making or preventing much-needed improvements to ROS. I do not have such an immense power or resources to prevent a foundation with the world’s largest corporations and best robotics talent. If your statement is true, if a random academic could stop you from making such important improvements, then again, that’s quite a big problem in your organization where you are responsible as a board member. Yet, if my comments seem powerful, that’s only because they tell the truth.
Specifically, in this thread, I am concerned about the path for such a Nix proposal to be accepted as we more-or-less know the technical part is feasible. I assume this proposal requires at least rclcpp nixification to have an impact and therefore it needs to pass core ROS committees and TGC. In any of these steps, it is very easy to kill the proposal for various technical reasons ranging from not respecting package.xml to that oh, we don’t manage our dependencies until it is broken. TGC is especially susceptible to block-voting from the ex-OSRC folk as you design it. Even that, again, by your design, the TGC is just an advising mechanism to the Board, and you have the final say about the fate of such proposals. Yet board actions are unchecked because you didn’t place any check and balance mechanism over the Board—unlike other public organizations. All technical efforts and proposals go into an opaque, unchecked decision process guided by personal relations. Not surprising that you didn’t get any third-party proposals in the past two years. Then, you, as a board member of a public Foundation (not his private company), complain about why we are questioning his choices in a public forum and feel insulted. Dear @rgariepy and the OSRF staff, this is the standard procedure everywhere in the civilized and professional world; please get used to answering such questions. For example, your Constitution protects the right to criticize government officials as a pillar of democracy. We have come a long way since the trial of unwanted citizens in Athens.
Still, I suggested @gbiggs bring this Nix issue to the TGC, which had a similar proposal/discussion on Bazel in November. We do not know any details about that proposal either. But at least its scope could be helpful here for Nix friends. But we never saw such a willingness to change from the Foundation, never saw an initiative nor an invitation to member companies. This is not the behavior you would show if you think ROS needs a collective effort for improvements.
Therefore, I conclude that any change must start from the Foundation itself. Unfortunately, I do not have any hope nor any power as I have stated.
All, I apologize for letting this go on so much in this thread vs. taking it to DMs, but @doganulus has made many past allegations (including in this thread) about opaque decision making processes, so I felt it best to have everything out in the open.
His public posting permissions have been restricted for the rest of 2025. This is not a complete ban - he is free to message people directly if he’d like to continue sharing his opinions.
I hope we can return to an otherwise excellent discussion, and I would like to thank @AngleSideAngle and others (including @doganulus) for all of the technical ideas - the build system could indeed be more approachable.
P.S. Fun Sunday fact! ‘My Constitution’ does nothing of the sort, the correct reference would have been ‘Charter of Rights and Freedoms’, which, by the way, has a carve out for defamatory libel.
12 Likes
Nix against other Package Managers
Nix isn’t designed to replace existing package managers entirely. Its primary focus is on simplifying and enhancing how packages are described and built. Notably, Nix’s capabilities allow building packages for other package managers like RPM and DEB.
Nix itself boasts wide platform compatibility, supporting Linux, macOS, and even Windows through WSL. In theory, if a system solely relies on Nix-built packages, it wouldn’t necessitate another package manager, potentially rendering building packages for various platforms unnecessary. However, the current software landscape isn’t there yet, and widespread adoption of such a scenario seems unlikely in the foreseeable future.
Ros against the system
Modern problems require modern solutions. Docker initially seemed like a solution for me, enabling consistent development environments across different machines. However, installing and running ROS 2 on platforms other than Ubuntu can be challenging, often requiring workarounds.
Docker itself presents limitations. For example, running graphical applications within a Docker container necessitates complex configurations, often involving exposing the host’s X server socket, which significantly compromises security. To run ROS 2 effectively within a Docker container, security measures often need to be relaxed, potentially increasing the system’s overall vulnerability.
In contrast, Nix flakes offer a more integrated and potentially more secure approach. By leveraging a Nix environment, you can achieve reproducible builds and installations without the inherent security risks associated with Docker containers. Nix packages are installed within the Nix store, providing a degree of isolation and preventing conflicts with the host system’s packages.
Nix against Standards
Nix is a powerful system that encompasses a declarative language and a set of tools. At its core, Nix is designed to describe and build software packages, primarily for Linux-based systems. While it can leverage other package managers like PyPI, Conda, and npm (for Node.js) for certain dependencies, Nix primarily focuses on building packages from source, often directly from Git repositories.
Crucially, Nix itself is a fully functional programming language. This allows for complex logic and automation within the package definitions and system configurations. This flexibility makes Nix a versatile solution for various tasks beyond simple package management, such as:
- Reproducible builds: Ensure that software builds are identical across different environments.
- System configuration management: Define and manage system configurations in a declarative and reproducible manner.
- Development environment management: Create and share isolated and reproducible development environments.
- Infrastructure as code: Define and manage infrastructure resources (like servers and networks) using code.
This combination of package management, functional programming, and declarative system configuration makes Nix a powerful tool for developers, system administrators, and anyone seeking to improve the reliability and reproducibility of their software and systems.
In fact, since nix is used as the programming and configuration system within the nix ecosystem, these capabilities could be applied to other concepts within the ROS world, such as launch files, urdfs or package.xml. So, if I may exaggerate a bit. 
3 Likes
This is true, although wrapping launch files or urdfs with nix doesn’t have a very obvious advantage. One really cool application of nix as a configuration system is deploying projects on top of NixOS (a linux distribution built around nix as a package and configuration manager). This would let you define the exact specification for your robot’s entire operating system in a text file, and result in a minimal, reproducible linux system with only your package’s dependencies installed.
Did not know this!
This section doesn’t address @peci1’s concern properly. My prior argument was that reproducibility is fundamentally important because academic research should be exactly reproducible by other researchers and engineers without wasting time fiddling with build systems and package managers. I believe docker is unrelated to reproducibility because docker images are not reproducible, nor do they manage packages.
Some earlier stuff re. packaging that I forgot to reply to
This is definitely a challenge. Pure nix (eg. using nix to manage ament, setuptools, dependencies, etc, rather than wrapping colcon with nix) packages would be difficult to integrate into non nix packages without building them using nix.
@wentasah’s ros2nix solution (which is very cool, thank you for your work) solves this by converting existing workspaces into nix projects by inserting package.nix files alongside package.xml files, and having an overarching default.nix in the parent directory to manage the sub packages. I think this is less elegant than building a package using purely nix because you have duplicated packaging files everywhere, but it completely preserves compatibility with existing tools.
I think an improvement that could be made to this (not experienced enough to know if it is practical*), while continuing to preserve compatibility, would be writing a nix function that parses a package with package.xml and returns the result of buildRosPackage for it, without generating a new nix file in that directory. That way, building existing ros projects inside nix would be more straightforward and less invasive (just a flake.nix with a few function calls in the root directory).
Doing free labor for the most valuable company on the planet
One major hurdle I noticed while experimenting with packaging ros projects in nix is nvidia cuda support. Cuda is theoretically packaged in nix without issue, but because of nvidia’s absurd licensing, it’s not legal to publicly distribute compiled binaries of cuda. This means (unless I’m seriously missing something) it can’t legally be part of a nixpkgs cache, so it must be built from source for internal use. This is very, very annoying.
I am not a lawyer, and it seems there are community nix projects that publicly cache cuda libraries, so it is possible nixpkgs is overlay cautious. I haven’t had success getting them to work. Either way, it’s probably possible to distribute a nixpkgs overlay that replaces the cuda build process with a binary downloaded directly from nvidia.
[insert gif of linus torvalds making a hand gesture that probably violates some community code of conduct]
1 Like
You’re correct that a Docker image itself is essentially a snapshot of a file system. However, Dockerfiles, which describe the process of building these images, offer a degree of reproducibility.
Dockerfiles leverage existing Linux distributions as base images and then define a series of instructions to build the desired environment. This approach offers several advantages:
- Leveraging existing images: Docker Hub provides access to a vast repository of pre-built images, allowing developers to build upon existing work and accelerate development.
- Familiarity with existing tools: Dockerfiles primarily utilize common Linux commands, making them potentially more accessible to developers already familiar with those tools.
However, Dockerfiles also have some limitations:
- Chronological and imperative: Dockerfiles are inherently sequential. Each instruction is executed in the order it appears, which can lead to complex and difficult-to-maintain build processes.
- Limited declarative capabilities: Compared to the declarative nature of Nix flakes or shells, Dockerfiles offer more limited declarative capabilities for defining dependencies and environment configurations.
In contrast, Nix emphasizes a declarative approach where dependencies and configurations are explicitly defined, making it easier to understand, reproduce, and maintain complex environments.
One of the challenges I’ve encountered when using Nix with Python and tools like pip is the inherent isolation of the Nix store.
Nix packages are typically designed to be self-contained and operate within a specific, isolated environment. While you can expose Nix packages to the system, it’s generally not recommended or straightforward to allow packages outside the Nix store to influence Nix packages.
This isolation can create friction when working with tools like pip, which often rely on system-level installations. For example:
- Scenario: You install a Jupyter Notebook environment using Nix (via
shell.nixorflake.nix), including some necessary Python packages. - Issue: You discover a missing Python package (‘x’) that’s not readily available in Nixpkgs.
- Attempt: You try to install package ‘x’ using pip.
- Problem: The Jupyter Notebook environment, managed by Nix, likely won’t automatically recognize or utilize the Python packages installed with pip outside of the Nix store.
This limitation requires users to either:
- Find an existing Nix package: Search Nixpkgs or community repositories for a pre-built Nix package for the missing dependency.
- Create a new Nix package: Manually create a Nix expression to build the missing dependency as a Nix package, which can be a significant overhead.
This added complexity can be a barrier for some users, especially those accustomed to the more flexible installation methods provided by pip.
First of all, thank you for raising this discussion here. I’ve been using NixOS for nearly 5 years now, on the other hand I’ve only recently been introduced to ROS. The experience of trying to build smaller source-only packages has not been a fun one, even when I’m using the recommended Ubuntu setup.
Technically, Nix packages can be bundled for non-Nix systems using the experimental bundle command or the nix-bundle project. Unfortunately though I don’t think this would be appropriate for bundling ROS packages as it has some significant draw backs and is more akin to making a AppImage or FlatPak.
Ultimately though, it’s not like packages that can be built for Nix can’t be built for other systems and it’s arguably easier to do than the reverse. Nix only describes the dependencies and build process but it doesn’t replace the build tooling like gcc and cmake, so you should still have all the configuration files present to build it on any other system. Additionally as a side effect of Nix being so extremely picky and building in isolation, I usually find these files are better defined than they’d normally be. Nix forces you to properly define your dependencies because if you don’t you simply won’t get them. That’s not to say you can’t do Nix wrong, but that’s normally quite a bit more work and even more poorly documented than the rest of Nix.
All of that said, I sadly don’t really see Nix as the mainstream solution for ROS unless there’s significant work done to make it as simple and bullet proof as possible. While I love it to bits I don’t usually recommend it to others because the learning curve is absolutely massive. While ros-nix-overlay and ros2nix are already a big step in that direction, they’re more in the vein of “ROS for Nix users” rather than “Nix for ROS users”.
5 Likes