Process for third-party client libraries to be incorporated into the core ROS 2 distribution

[nnmm]

@nuclearsandwich

@tylerweaver

Does this imply that there is a chance that with rclrs released into crates.io you might be able to write pure rust code that does not need an installed version of ROS on the system and that the crates.io dependency will contain everything you need to develop a rust application that uses ROS?

At least reading the current setup, the ros2-rust project requires a working ROS 2 installation on the local system. So this is at least not the case currently.

That’s right – I imagine vendoring its ROS 2 dependencies into the rclrs package to be really complicated and probably not worth it. But that’s just my first guess, I haven’t looked into it at all.

[nnmm]

@nuclearsandwich

Hrm, that definitely makes it harder for us to map into rosdep in any kind of consistent way since as far as I can tell that means there is no way to request the installation of a Rust library outside the context of a cargo build.

Okay, so I’d like to amend my earlier statement that you need to use path = or the .cargo/config.toml hack to use a package from the local machine. After reading through the Source Replacement - The Cargo Book doc, I think a directory source could theoretically be a third option, which would allow installing a crate into a central directory, where it can be found by other crates. And like you mentioned, the Debian Rust packages in /usr/share/cargo/registry seem to be intended to work with this mechanism¹.

But, in practice, I couldn’t get the Debian Rust packages to work, and the restrictions of the source replacement approach may make it a non-starter:

Cargo has a core assumption about source replacement that the source code is exactly the same from both sources. Note that this also means that a replacement source is not allowed to have crates which are not present in the original source.

So apparently we cannot use it for anything, or any version of something, that wasn’t published to crates.io before.

There is a fourth option, private registries, which is somewhat similar to a directory source except that it also involves git and a local web server for Cargo to communicate with. It would require that users write some_dependency = { version = "1.0", registry = "colcon_registry" } for every dependency … I’m not sure what exactly that approach entails.

¹ One difference is that I don’t see the “associated metadata file” mentioned in the doc.

@nuclearsandwich

Based on what you describe, it seems like there isn’t really a way for an installed Rust library that’s already been “installed” to declare “find me here” without touching a solitary file: .cargo/config.toml. And while it is possible for colcon to modify the process environment for each individual package it is probably harder to create per-package cargo configuration files and keep those up-to-date.
It looks like cargo config files are hierarchically merged but it doesn’t appear that we can control that hierarchy in any way in order to support multiple workspaces without having to manually unify all underlay configuration into the current workspace’s .cargo/config.toml.

Let me recap what’s currently implemented just in case: The .cargo/config.toml file doesn’t exist once per package, it exists once per workspace, in the workspace root. If you have any underlay install directories sourced, it will contain the packages from those too: colcon-ros-cargo searches all ament indices in AMENT_PREFIX_PATH for Rust packages and adds a [patch] entry for each one to .cargo/config.toml. So you could say that we have solved the “unification” at the colcon level.

@nuclearsandwich

Here’s an extremely underbaked idea: If we were using out-of-tree builds, a CMake recommended practice and something colcon does with CMake packages, albeit for different reasons than we’d do them for Rust packages then the Cargo.toml of the build directory could be ephemeral and thus modifiable by colcon-cargo without worrying about interfering or confusing the source Cargo.toml and you could use whatever hard-coded paths you want there to route in-workspace dependencies where they need to go. Since you’re only wiring the dependencies needed by the current package you would eliminate the unused entry warnings.

That’s not a bad idea at all – we considered that option a while back and were on the fence between that one and the .cargo/config.toml one. If I remember correctly, if you’re only building with colcon, that option is indeed preferable. But what .cargo/config.toml allows you to do is to build with colcon once to create .cargo/config.toml, and then after that do all your builds directly with cargo. Being able to use cargo directly is important for faster builds and for using tools like cargo clippy, cargo fmt, and currently cargo test (because the test verb for colcon-cargo isn’t implemented yet). With the build-dir Cargo.toml, you’d need to navigate to the build dir to be able to run these tools, which would be a really confusing workflow.

Another idea we had was checking in only a Cargo.toml.in template, which would be expanded by colcon-ros-cargo to a Cargo.toml with paths. But that would modify the source directory, which was considered inappropriate for a build tool.

@nuclearsandwich

I can take my own turn at asking a naive question: How is it currently possible to use ros2-rust purely with cargo? Looking at these instructions it looks like a ROS 2 installation is required and while the interface may be “pure Cargo” you’re still working with the system package manager or a manually installed build of ROS 2 behind the scenes. What system modifications do you think they’d want to avoid?

Right, it’s necessary to install ROS 2. That was badly worded. But you would not need to install colcon-cargo, colcon-ros-cargo, and optionally cargo-ament-build.

The “pure cargo” ties into the previous section: With crates.io, you don’t even need the initial colcon build, you can right away start working with cargo.

---

What do you think about including the Rust message generation library into core ROS 2 soon, before we have figured out how to build Rust in (or for) ROS?

Theoretically, like Esteve said, it’s just a Python package that produces source code files. That source code is not supposed to be built during the build of the message package anyway.
Note that the code does have dependencies – and currently those would be satisfied by crates.io when the message crate is built, later on. But would that be a problem?

That step would already help users a lot, I think it’s currently the most-requested feature.

[nuclearsandwich]

@nnmm

So apparently we cannot use it for anything, or any version of something, that wasn’t published to crates.io before.

I think that this would predominantly affect Rust packages that are meant to be ROS specific. I am not sure that it would make sense to publish them to crates.io just so we can access them via source replacement but if they’re being published to crates.io already as part of a pre-existing developer workflow then that may still work.

@nnmm

There is a fourth option, private registries, which is somewhat similar to a directory source except that it also involves git and a local web server for Cargo to communicate with. It would require that users write some_dependency = { version = "1.0", registry = "colcon_registry" } for every dependency … I’m not sure what exactly that approach entails.

Is it not possible to change the “default registry” using a .cargo/config.toml so that we don’t have to mess with the individual package cargo.toml files? Rewriting the build cargo.toml files to point to the local registry could work though if we stay on that tactic. It isn’t outside the realm of possibility to have a colcon extension run a basic cargo registry during build operations but it doesn’t sound like a simple option so I think it’s best to treat that as an option of last resort for now.

Backing up to to the previous conversation about source replacement and colcon, would it make sense for the Rust crates that are exclusive to ROS to occupy their own (pseudo) registry and use source replacement for them but allow other crates to work directly with crates.io?
Then we would use colcon-cargo to do source replacement for “ROS crates” but allow other crates to come from upstream during development. When releasing packages on the buildfarm there would be some checks to make sure that the crates.io crates are made available in the apt repositories either via upstream or additional debcargo-packaged crates in the ROS repositories.

I don’t think that would be as tidy as assuming a common registry for all packages but it pushes the problem of relying on packaged crates down to packages that are trying to build on the build farm. The major drawback of that approach is that you don’t get feedback about your release’s dependencies until you try and create it and it fails but that is also on par with the current experience for packages and it enables packages that are not intended for build farm release to operate using unofficial dependency sources without irritation. From a workflow perspective I’d prefer if dependency hygiene warnings were available and on by default from the first build but suppressible for developers or packages that don’t intend to do build farm releases.

@nnmm

That’s not a bad idea at all – we considered that option a while back and were on the fence between that one and the .cargo/config.toml one. If I remember correctly, if you’re only building with colcon, that option is indeed preferable. But what .cargo/config.toml allows you to do is to build with colcon once to create .cargo/config.toml, and then after that do all your builds directly with cargo. Being able to use cargo directly is important for faster builds and for using tools like cargo clippy, cargo fmt, and currently cargo test (because the test verb for colcon-cargo isn’t implemented yet). With the build-dir Cargo.toml, you’d need to navigate to the build dir to be able to run these tools, which would be a really confusing workflow.

I’m going to put on a colcon maintainer hat for a second and make a very hedged claim: I’m not sure how much support colcon can provide for building packages in the workspace outside of colcon’s purview. On the surface, it ought to work just fine since colcon is just setting up an environment and running the individual package build steps itself. But that environment is complex and I don’t think that running build steps by hand is part of any existing colcon workflow, but I am perhaps being uncreative here as my work often requires building entire workspaces rather than iterating on individual packages. All this is to say that I’m not sure that supporting builds that circumvent colcon needs to be a requirement but I recognize that the main motivation would be to preserve the developer workflow when working on an individual Rust package.

@nnmm

What do you think about including the Rust message generation library into core ROS 2 soon, before we have figured out how to build Rust in (or for) ROS?

I think that getting the infrastructure which doesn’t require extending the build tools to support new package build types in early will be good for any members of the ros2-rust team who aren’t yet familiar with the ROS 2 release processes and get everyone into the rhythm of making bloom releases for packages that are on the buildfarm.
My only reservation is if we have to radically change things after they’ve already been released onto the buildfarm but I think that if we target ROS 2 Rolling to start out we don’t have to worry about stability guarantees because there are none.

But that is just about getting it onto the ROS 2 build farm. I don’t think it’s likely that we would integrate a rosidl generator into the core without at least one client library that utilizes it in the core as well. A middle ground would be adding rosidl_generator_rs to the list of packages injected as build dependencies for interface packages which would trigger the rust codegen. However I think there’s a problem with that approach: even though the Rust sources that would be generated and included in the interface package wouldn’t be required to build with the absent rust dependencies, if the interface package now distributes rust sources which depend on additional rust runtime we either have to leave those dependencies undeclared and trust crates.io to fill in the blanks (and figure out what that means when we restrict network access and use source overrides) or we actually must wait until we can release at least rosidl_runtime_rs before actually enabling the generator.

One reservation about adding additional generators to the default stack has to do with the effect on interface package build times, this is something that we’ll just have to measure before and after.

Talking with @cottsay, one of the other members of the ROS 2 infrastructure team, creating packages that include Rust sources but don’t depend on the Rust runtime necessary to use them directly may be something only I feel strongly against. This is a situation where more robust dependency expression would help us but anything that we implement in ROS must eventually be rendered in system package managers and that usually means targeting a lowest common denominator of features.

As a first order of business, I think that we should try and just release rosidl_generator_rs as a standalone package in Rolling without hooking it in to the default set of generators injected as interface package build dependencies. This won’t improve the situation for most ros2-rust users but it will make it easier to incorporate the generator and in the meantime we can keep visiting the various options and their tradeoffs for enabling Rust generation on the buildfarm for all interface packages.

[nnmm]

Sorry for the late reply. I want to comment on a few of your points eventually, but for now I’ll just address one thing.

@nuclearsandwich

One reservation about adding additional generators to the default stack has to do with the effect on interface package build times, this is something that we’ll just have to measure before and after.

Here are my results (6 builds per configuration):

The x axis is in seconds. So the build time impact seems pretty low, which makes sense to me since nothing is being compiled.