It is quite common to handle ROS 2 package dependencies through the use of .repos files and vcs. After several years of using vcs, I have encountered several issues and quirk behaviors with it that motivated me to create an alternative CLI tool. Here I am introducing ripvcs (following the naming scheme of ripgrep). It is a CLI tool written in Go that offers improvement both in speed as well a set of new features.
Highlights
Faster operations for commands shared with vcs
Recursive import: Automatically import other .repos files within the cloned repositories
Repositories sync: Stash and pull any changes in any repositories found in given path.
Nested repository pull: Pull changes even within nested repositories.
I invite you to try ripvcs out and provide feedback on any issues you encounter to improve the performance and reliability of it.
Wow, this looks good! We use our custom wrapper around wstool that added the recursive option. Your tool looks great and I’d give it a try. But there are a few features I’m missing:
Support for .rosinstall files in addition to .repos
Possibility to specify a different folder for the recursively imported repositories (so that you can have one workspace with just the top-level repos and another workspace with their dependencies).
Support for proxy remotes (this is basically a temporary remote that is tried instead of e.g. github.com)
We use this in case we go to a field experiment and we expect that Internet will be inaccessible in that place. Before we go, we pull all repos on a central server we take with us and other computers should temporarily use this remote instead of github/gitlab.
This task cannot be easily done by setting a HTTP proxy, as git pulls are over HTTPS or SSH and we can’t (unfortunately) fake github’s host keys.
Alternatively, instead of this quite nonstandard feature, ripvcs could just support specifying the name of a remote used for pull/sync. We could then manually add the temporary remote using git remote add and just tell ripvcs to pull from this one.
Yes, as @gavanderhoorn said, they offer nothing in addition to .repos. But sometimes we don’t even control the source of the files as they’re a part of a 3rd party package.
Thanks for sharing. As I understand from the code, it does not have a solution for cases that different packages depend on different versions of the same package.
Is there a solution for that?
Are you still using that wrapper? Have you tried the introduced ripvcs tool?
What do you think about the current best solution?
Yes, different versions of the same package are a problem. Currently, we keep the first version that is encountered. If it’s our direct dependency, then our version wins. If it’s a transitive dependency, then the first package that defines it wins.
There is no good solution to this except ripping the workspace apart into two, or possibly dockerizing its incompatible parts. This can, however, not be done in an automated way.
We haven’t tried ripvcs yet. But I wanted to give it a try.
As to the state-of-the-art solution to this issue: I think the best is not needing recursive wstool at all. Ideally, you depend not on a branch of a package, but on a fixed and tested version, and your release manager’s role is to periodically test newer versions etc
This release manager can also record all the transitive dependencies and decide their versions. But as a small reaearch team, we don’t have the luxury to have a release manager in our team. So we just use the recursive wstool until it breaks, and when it breaks, the least lucky guy in the lab has to fix it =)
Currently the docs say that this is “Fast CLI tool for managing multiple Git repositories.” Do you have plans to support other version control systems? Although git is dominant, there’s definitely still other systems in usage.
I suspect that a bunch of the quirks and limitations that you’re hitting are because vcs is limited to the lowest common denominator for the different implementations.