It’s agnostic to ROS. Isn’t it fair, however, to treat the tool as ROS defacto standard for some vcs operations (the tool is mentioned in various places in the official ROS2 doc (e.g. humble). There’s also this post from the original author Dirk discourse.ros.org#7334)?
There’s vcstool#242 where concerned people share thoughts, offer helping maintenance.
Can we move forward?
Last release was 2 years ago, last merge of PR was 1.5 years ago. There may not be a lot of PRs, but some promising feature additions incl. vcstool#219 were closed as the PR authors walked away due to lack of response from maintenance team. I’m sure many users use their own fork (and btw while a pypi package is released from the upstream repo, there’s also vcstool2 made from a fork vcstool#242-comment).
In its current state, the ROS tooling is a blocking stone in front of the productivity and well-being of the ROS developer, especially compared to contemporary alternatives. We believe the solution to this problem is not more ROS-specific custom tools but the opposite: More general-purpose tooling must be used to provide the functionality in ROS projects.
We have an experimental project called ansible-collection-ros (GitHub - bounverif/ansible-collection-ros: Ansible roles to help the dependency management of ROS projects.) using the Ansible automation tool to handle the basic tasks of dependency management and automation. In particular, the functionality provided by vcstool —that is, cloning external repositories— can be largely replaced by ansible.builtin.git. We have written a small ros.repositories role allowing developers to declare their repos in Ansible format and handle the vcstool format for legacy.
We do not have the resources to promise any reliability and continuity for this project, as it is experimental, but we would love to help the ROS community move forward…
Wouldn’t this take us from an, apparently, unmaintained tool to another, potentially/likely, unmaintained tool? Where it would also be yet-another-tool, with unknown (to current vcstools users) limitations, gotchas and workflows.
Your extension/Ansible role would seem to be the tool in this case. It would at least fulfil that role for users if it would be a replacement for vcstool.
(edit: to clarify: I know what Ansible is. I’m not suggesting Ansible == vcstool. What you suggest however is to replace an existing stand-alone tool with a role for Ansible. At that point Ansible + your extension == vcstool (not exactly, but close enough))
It is acceptable for a professor and his two students not to give maintenance guarantees for their code.
It is unacceptable for multi-billion dollar corporations and an umbrella foundation not to maintain a tool they recommend everywhere.
Why do you comment here specifically for (1), and not (2)?
More importantly, what are your plans for vcstool? Would you continue to recommend the tool in an unmaintained state? This is the original question. I would be happy to hear more voices.
I opened this thread to raise awareness of vcstool’s maintenance status and issues. I personally propose to continue the discussion for the improvement in vcstool#242.
I like the idea of using general purpose tool (ansible in this case) as well! Thanks for sharing your tool @doganulus !
For vcstool particularly, it’s the tool that has been mentioned in many places in the tutorial suite for years and, as a natural consequence, many community members including private companies are likely using it. This means the way to execute the tool (e.g. bash CLI vcs %VERB%, or Python API if you delved into the tool’s API) and configs (i.e. X.repos files) are already carved in the repositories. Of course switching to better alternative is always an option though, that’s up to each team.
As for un-maintenance status (, which is not particular to vcstool nor to your tool), I guess the maintenance status of a software (as a dependency of your software) has become more and more impactful criteria when you (esp. in a team) choose the direction of software solution. So it’s a reasonable thing to bring up IMO.
I’d like to make sure that in this discussion that we differentiate between a maintained stable product with few changes and an unmaintained abandoned platform.
The software is being used as intended and meeting the design requirements without any major bugs and is available on all expected platforms is not “unmaintained”. You’ll find that this software has found its way onto all the more recent platforms that are supported since the last “release”.
Yes there are feature requests. And some of them have come with pull requests such as #219 that have not been serviced. It would be nice to have more feedback but there’s a level that development of these features has not been prioritized by the maintenance team. The tool is working well following the design. And if there are new features that want maintainers attention it’s important to present them and make the value case as to why it’s worth the maintainers time to work on them.
This one in particular is a specific feature that is only supported on some platforms (git >= 2.25) so is only supported on some of our target platforms. It also is only implementing a command for a specific vcs type of 4. It also is changing a moderately stable file format. And the workaround is to just checkout the whole tree.
The sparse checkout doesn’t completely fit with the general git model of work. Consequently the git spare-checkout is explicitly an Experimental feature with the warning in the docs
“THIS COMMAND IS EXPERIMENTAL. ITS BEHAVIOR, AND THE BEHAVIOR OF OTHER COMMANDS IN THE PRESENCE OF SPARSE-CHECKOUTS, WILL LIKELY CHANGE IN THE FUTURE.”
I understand that the maintainers could have taken time to do more analysis of this and explain more. But not engaging in this feature enhancement PR does not mean that the package is unmaintained. In general there are a lot of things that maintainers need to pay attention to. And when opening feature requests or PRs, communicating the value provided to the users is important so that maintainers can prioritize their efforts in many areas.
As far as I know the tool vcs continues to operate very effectively as it’s intended and works well as the defacto way vcs tool of the ROS ecosystem. And I would like to give credit to the maintainers who have ensured that it stays that way.
I think we can also differentiate between a maintained stable product with few changes and an unmaintained abandoned platform. The product is far from maintained as stable as you claim. Several members of the community brought up these issues without satisfactory replies.
Yes, the software may continue to work; this is indeed a very special property of software artifacts but professional software and robotics development requires more. We work on increasing the level of quality of software and system design. This reliability cannot be maintained by marketing phrases.
Finally, I will put this quote from Grady Booch, one of the inventors of the CI concept:
Although I’m a big git fan, I like the freedom of choice given to developers by vcstool. It covers all of the used VCS systems out there (maybe except Perforce, but I don’t really know anyone using it). Although last officially released non-Git ROS1 packages were in Melodic (they used hg), there may still be teams around that do not publish their packages and use non-git VCS. And I feel it is important that this openness is there.
And if you look around, there aren’t many tools that are VCS-agnostic in such a way as vcstool. I know of uses of gitman, but that’s git-only (and maybe SVN via git-svn). Ansible seems to support all VCS types vcstool does, but some are only community-managed. Not sure about the level of actual support you get there. However, one big problem of Ansible I see is that it is very verbose in configuration (you wouldn’t do with a single file) and too non-verbose in execution. I.e., by default, it hides all CLI output from the user and only shows the first failed thing. Also, I’m not sure how good Ansible is in parallelization of checkouts, but for our projects with ~100 repos, it is a game-changer (until we reached our uni gitlab DDOS limits ).
So, I’d pose the question in a different way: Why isn’t vcstool much more adopted outside ROS?
It’s a design problem. Not specific to vcstool. This is prevalent in the whole ROS ecosystem, and unfortunately, the ROS community is used to bad tooling very much. There is a well-known maxim in designing languages, tools, user interfaces, etc. as follows:
[It should] make easy things easy, complex things possible.
I think the ROS tooling fails in that. Yes, it makes easy things easier, but also complex things much harder. Looking at threads by external senior engineers in this forum, you will see the common pattern of such complaints.
Sounds like vcstool is in maintained condition. So I’m convincend. Thank you @tfoote. I will post my followup in vcstool/issues/242. Rest is just a response.
I’d like to make sure that in this discussion that we differentiate between a maintained stable product with few changes and an unmaintained abandoned platform.
The software is being used as intended and meeting the design requirements without any major bugs and is available on all expected platforms is not “unmaintained”.
Understood. Indeed, with dictionary definition (e.g. thefreedictionary.com/unmaintained), what you described seems to align better with.
1. not kept in proper or good condition
2. formal not continued, retained, or kept in existence
The need for generic tools to support more platforms/OSes and various versions of each is something that is so easy to be missed in community contributors like me who only uses specific OSes. So thanks for the reminder.
As for vcstool#219 and the issue it aims to address vcstool#218, I was a part of the team that provided !219 but AFAIR we did not try to reach out maintainer team. So I learned / remind myself a good’ol lesson – contact maintainer team.
As the person that created that ticket, I guess I should respond
My concern was that vcstool is under @dirk-thomas 's account, so I thought that given that it’s such a widely-used tool in the ROS community, it might be better off if it were under an organization account, for example ros-tooling. Is the PyPI page also owned by Dirk?
This is so cool! Thanks for sharing, perhaps you could present it at the next OpenAD kit WG meeting? It may replace vcstool in Autoware, especially because we already use Ansible.
@doganulus as for the rest of your comments, I sometimes get the impression that they come off as a bit antagonistic. Which is unfortunate, because you’re clearly an intelligent person, and when you show your work, it’s of good quality. I’m sure you’re familiar with the concept of “soft power” in international diplomacy, which applies to so many aspects of our lives, including this community. Many members of this community can get people rally behind them because they are not only really good roboticists, but also because they know that in the end, software is written by people, not machines.
Anyway, here’s a quote by Joseph S. Nye Jr., who coined the term “soft power”:
When you can get others to admire your ideals and to want what you want, you do not have to spend as much on sticks and carrots to move them in your direction.
Let’s call them ROS heroes… They made great contributions to the community in the past. I truly appreciate their efforts and wouldn’t be here if I didn’t see the value they had created in the past. This includes Dirk Thomas, perhaps you, and other great contributors. But I also think too much attachment and emotional value to the good old days negatively affects the future of ROS. This is one recurring theme in my discussions here and there.
System building requires a different set of qualities other than charismatic leadership. This stage must follow the Age of Heroes for any long-lasting organization. The lack of system building is the most apparent for the tooling issues I mostly comment on. Therefore, here, I always say it is time to build a system requiring no heroic efforts. And no, I am not a hero as well. This is perhaps why I seem like an adversary to most. Apparently to the old Heroes the most, as they immediately reply to my posts in an adverse way.
But the next big thing Heroes can do for the community is to use their soft power to persuade the community to this paradigm shift instead of advocating the status quo. I don’t have that power.
Gonna skip all the tangent conversations, this one really is the heart of what we’re trying to figure out.
Like @tfoote said, the tool is not broken which in my opinion is a pretty good state. Sure I’d like a few more features, but that’s life. Based on the conversation on the issue in question, sounds like @nuclearsandwich is not specifically opposed to vcstool’s official home moving to some organization.
@esteve mentioned ros-tooling, which as has been noted in the github conversation doesn’t have a super clear focus (you can partly blame me for that). It started out a desire for a community-owned organization with OSRF “blessing” to hold some common tooling, especially around rosbag2 development. It’s got some github actions, which are “developer tooling”, but also topic_tools, keyboard_handler, libstatistics_collector which are application-side ROS utility packages. So, I don’t know if it’s necessarily the perfect place, but as one of the maintainers of that org I wouldn’t be opposed to it moving there – of course being careful to note that this doesn’t magically making new maintainers appear but maybe allows for a more transparent permissions and ownership story.
My take on other possible places:
A new organization for it seems a bit overkill as it’s just one repo, but also I guess why not
colcon · GitHub is a technically separate project but is in practice closely related in workflow, so maybe could also be an appropriate home.
ros-infrastructure · GitHub seems probably not a good fit, it’s really tied to the maintenance and release of the ros project - but you know rosdep and bloom and catkin_pkg kinda fit under a similar umbrella of “things you do on the commandline when developing ros”
Open Robotics · GitHub also contains just about anything under the sun, but is home of one of my more favorite semi-generic developer doodads, rocker, so it fits the bill as well as anything in that regard
But hey maybe just the open contributions haven’t been meeting the bar and nothing needs to happen here at all, that’s not out of the question. Maybe I’d feel differently if I was the one attempting contributions though - I’ve only got my feature branches that are not fleshed out enough for contribution, just for personal use.
to trying to improve the situation, but:
).
