ROS Resources: Documentation | Support | Discussion Forum | Service Status | Q&A answers.ros.org

Releasing repositories form "other" people

Packaging and Release Management
#1

We have currently multiple PRs pending which release packages from “other” people (new versions and/or into more ROS distros). Either because the previous / current maintainer doesn’t respond or doesn’t want to do the work. In general it is a great thing that someone takes over the effort of releasing new versions.

But I have one concern with that approach. The person making the rosdistro PR is commonly not listed as a maintainer in the upstream package. As a consequence the person doesn’t get notified if the packages they released fails now or in the future on the buildfarm.

While the new releases clearly have a benefit I think we should clarify what we expect in this case:

  • is it ok, that essentially nobody will act on such failures (the maintainer is already not acting now so it is unexpected they will on future email)?
  • should the person release the packages add himself to the manifest in the release repo to ensure that he gets notified?
  • (and other approach you can imagine)?
1 Like
Baxter release permissions
Lack of a velodyne release
ROS orphaned package maintainer info. REP 152 for review
Maintaining and releasing orphaned packages
#2

… and even if they were notified, there wouldn’t be much they could do about it because they don’t have push access to the upstream package.

OTOH, when a build for an abandoned package fails in one of the distros it’s already released for, we already have the same problem.

I don’t really have an opinion one way or the other. Abandoned packages suck, but drive-by releases without willing to commit to maintaining a package are hardly going to fix the problem.

2 Likes
#3

I don’t think a drive-by-release with no plans for maintenance makes much sense.

Some of those pending rosdistro PRs have a lot of open issues and PRs on the upstream repo. Just doing a bloom release with no way to fix those issues only half solves the issue of package abandonment.

A releaser should at least fork the repo and make themselves a maintainer in the package.xml.

Thats just my 2¢.

3 Likes
#4

From the Debian Policy Manual:

An orphaned package is one with no current maintainer. Orphaned packages should have their Maintainer control field set to Debian QA Group packages@qa.debian.org. These packages are considered maintained by the Debian project as a whole until someone else volunteers to take over maintenance.

The above is official policy. It references the Debian Developer’s Reference, which is considered non-normative but best practice. The relevant sections are 5.9.4, Orphaning a package and 5.9.5, Adopting a package (I note that the situation with usb_cam may have been considered by Debian to be package hijacking). Also relevant is 7.4, Dealing with inactive and/or unreachable maintainers.

I think the Debian policy and procedure sounds good. It requires a bit of infrastructure work, but nothing too burdensome. It relies on having a known “QA group” but I feel that the OSRF could fill this role.

2 Likes
#5

That can only work if there is a team of people (possibly in OSRF) willing to take on that task. I can imagine most people have enough work on their own without picking up abandoned packages. Of course, if there are people willing and capable of doing it, awesome.

#6

I also think the Debian procedure sounds good. The job of the “QA group” here wouldn’t have to be that they automatically become maintainers for all abandoned packages (OSRF doesn’t have the resources to do that). But it would be nice to have a procedure for orphaning and adopting packages. I imagine it could go like this:

  • maintainer decides to orphan package, gives github “admin” access to QA group
  • package is added to a list of orphaned packages
  • someone decides to adopt package, QA group transfers admin access to new maintainer

Most of this could be automated. To further reduce the burden on OSRF, the QA group could be a set of volunteers that’s only managed by OSRF.

Of course this is only going to solve the cases where a maintainer makes a conscious decision to orphan a package instead of just gradually moving on to other things.

#7

I can’t speak for whether the OSRF has the resources to be a QA group, but I think that collecting volunteers from outside the OSRF to be members is a fantastic idea (Debian is a volunteer organisation, after all; I actually quite like Debian’s web-of-trust organisation in general). I would happily volunteer to be a part of such a group, and would also be happy to help with managing it if the OSRF does not have the resources to do so.

If there is interest, I can draft a REP that lays out a set of policies and procedures for dealing with orphaned and abandoned packages in ROS (including how those terms are defined) so we can get some more formal discussion going.

1 Like
#8

I don’t like the idea of a separate group to track orphaned packages. It seems like a lot of overhead, plus few people are likely to be aware of and care about it.

I think “hijacking” packages should be encouraged if… (rough guidelines)

  • The license allows for it (Let the creator of the original package enforce this, if they care to.)
  • A package is at least one LTS release behind.
  • An effort was made to contact the previous maintainer(s).
  • The new releaser adds his name & email to the package manifest, aka rosdistro.

Maybe bloom could check for a change to the maintainers or source url and display this checklist when it detects a change.

1 Like
#9

I also think a QA group is high overhead. Bloom already has the specification of unmaintained for a package’s status (REP 137 - Release File). I think it would make more practical sense to advertise use of that and perhaps get the build farm status pages (e.g. kinetic) to highlight and add a quick filter for them.

As for Dirk’s original question - I don’t like the bit-rotted maintainer tag problem we currently have but I concur with Martin’s point - often the maintainer can’t push to ensure that flag is correct. A maintainer could/often will be releasing from a fork though, however that is still tedious repeatedly updating the maintainer tag everytime upstream is merged. I tentatively thought it might make more sense to have this information in bloom (just as the status is), but I do find it useful having maintainer flags around even for non-bloomed packages. I could live with that loss though.

#10

We have talked about the topic in todays meeting and we came to the following conclusion. Doing “drive-by” releases have the advantage of making the package easier accessible for more users. But the downside of not being possible of notifying the person who did the release outweights the benefit in our opinion.

Therefore we think one of the following steps should be taken (clearly preferring the first):

  • Contact the current maintainer, propose to help, request access to upstream / gbp repos in order to make releases.

    • When being granted write access to the gbp to do new releases please still add yourself as a maintainer to the package manifest files in order to receive notifications.

  • If the current maintainer is not responsive (a single day isn’t enough to conclude that) and you still want to release the package there are two different paths forward depending on the “location” of the upstream and gbp repo:

    • If the repos are hosted on GitHub within an ros- org unit the admins of the org unit will usually grant the necessary access to these repo (just post in this category and reference previous attempts to contact the maintainer).

    • If nobody else beside the original author (which doesn’t respond) has access you should fork the repo, add yourself as a maintainer in the package manifest files, and create a new release from the fork. Please also reference the communication attempt in the rosdistro PR so that it is obvious why you should be allowed to “claim” the existing package / repo name.

This is pretty much in line with what is already documented in the wiki (see http://wiki.ros.org/MaintenanceGuide#Claiming_Maintainership). With the details of this thread it might just be more clear that “drive-by” releases are not desired.

2 Likes
Perception SiG Maintainers Still Maintaining?
#11

I’m very sorry to try to continue discussion which seems already come to conclusion.

I’m considering “abandoned/orphaned” packages, and if I understand correctly @dirk-thomas conclusions is basically “normal” package maintenance procedure. But “abandoned/orphaned” packages becomes abandoned or orphaned because this “normal” procedure did not work well.

And I understand that there is an opinion like “if no one willing to become maintainer, that packages is not have enough value, so it’s ok to drop it”, But I do not agree on this idea. I believe these packages are still useful, everyone want to use it. but no one can take a responsibility for being maintainer for that. I think this is the problem.

For example I have sent so many hijacking PR last week (sorry for doing that …), mostly for usb camera driver for kinetic. I did released, because ,for me, usb camera is so essential and important, and I believe it worth to include kinetic ros package list, but because it is so general and important that I feel I’m not right person to maintain such a great packages. I guess, these packages did not have been released after 1 years since kinetic release, because everyone feel same way,

So, I like @gbiggs 's idea of having team of maintainer who take care about maintained packages, and from my experience on hijacking release, https://github.com/ros/rosdistro/pull/14890#issue-226190621 , there might be several type of maintenance and if we restrict the role of that team to bloom release, package release, (package migration), then it will not cost to the team members and everyone can continue using existing package to future ros distros.

In conclusion, my proposal for releasing un-maintained packages are;

  • contact current maintainer, ask for release
  • if we can get answer but find original maintainer lose attention. ask for package transfer to ex. ros-unmaintained organization
  • if we can not get response, fork that package and gbp to ros-unmaintaned organization
  • put ros-unmaintaned-maintaner mailing list to package.xml
  • someone in the team will release it

Basically, this is as same as @Martin_Guenther 's procedure and the difference between @dirk-thomas “normal” procedures is instead of add yourself as a maintainer in the package manifest files, we will add teams as a maintaner. This might be reduce responsibility for people within the mailing list, but that is the aim of making support team for orphaned packages. So if adding mailing list into maintaner tags is allowed, I think this procedure will not violate http://wiki.ros.org/MaintenanceGuide#Claiming_Maintainership, and it seems we had enough people who volunteer to do that (me, @gbiggs and @Martin_Guenther ?)

Minar things :

  • @Daniel_Stonier 's using unmaintaned tag and build farm status pages will be nice
  • If we create orphaned supporting team group in one organization, do they can get “admin” access to others package repository?
  • to reduce team work flow, I think it’s nice to have put all orphaned packages to one organization, so if we add volunteers to existing team, one can working on maintenance work, for same reason we do not create ros-unmaintanerd-gbp. team member will not want to maintain user management.
  • moving to all orphaned packages to ros-unmaintaned even if admin can add list may overfitting, but once the packages are orphaned, I’m afraid that it is very hard to come back.
1 Like
#12

I think this depends on what a QA group has to do. If it’s simply maintaining an infrastructure that tracks orphaned and abandoned packages and recruiting new maintainers, then that’s a small workload, If, as @k-okada suggests, the work also includes making a release for each new ROS release, then the workload obviously grows, especially if people get lazy and start dumping packages onto the group instead of maintaining their packages themselves, but clear procedures and good management can help tame this. I think that expecting such a group to actually maintain the package by fixing bugs would be going too far.

I agree with @k-okada’s experience, and I find it surprising that there are many packages that do not get released into a ROS release despite being apparently commonly used. I’ve run into the lack of a usb_cam release myself. MoveIt was a shocker for me; the idea that such a commonly-used set of packages could be absent from a ROS release for so long told me that not enough people really care about or use non-LTS ROS releases. Now, I accept that ROS is volunteer-driven, does not have the resources of a project like Debian, and is possibly not aiming to have the same sort of release as a Debian release. But I think that the problems with getting packages released and making a ROS release equivalent to the previous release when it’s released (and not 6 or 9 months later) is going to be an increasing problem and reflects poorly on the impression of ROS as quality software. I think that one of the things we can do to improve the situation is to assist with orphaned and abandoned packages so that they remain released and visible until a new maintainer is found. Another is to separate the burden of developing software in a package from the burden of releasing that software into a ROS release - not many Linux distributions ask software developers to build packages for them.

I fully agree with anyone who says that this is difficult because everyone is volunteers, but I don’t think that’s a good excuse to not try to improve the situation.

Another idea is possibly slowing down releases even more, to once every two years, with minor updates allowed except perhaps for core packages like roscpp (similar to how Debian settles on a kernel version and sticks with it until the next release two years later), but that’s a separate discussion.

1 Like
#13

I fully agree with @dirk-thomas’s conclusion. A package shouldn’t be released without somebody actually willing to maintain it.

BTW, I’m strictly against forking/copying repos if it can be avoided. If the repos have to change place (for example, to be moved into the ros-drivers organization), the move functionality of GitHub should be used by the current owner. That way, all issues etc. are preserved, people don’t open issues on the wrong repo, and even the old URLs still work because GitHub creates a redirect.

For reference, these are the packages that sparked this discussion:

All of the maintainers still seem to be reachable, so if somebody steps forward and volunteers to maintain one of them, the current maintainer could give them write (or better “admin”) access.

#14

I think there is value in continuing to release a package that is known to be working. I agree that if the package is not working with the other software in a ROS release, is uncompilable, is full of show-stopping bugs, or is just plain broken, then not releasing it is the right approach. But if the package is known to work and only has some minor problems, there is no reason not to produce a binary release. If the package’s developer is not responsive, then I see value in having a team of volunteers who can do the release instead and making it known that the package is in an unmaintained state.

Maybe a voting system to vote for unmaintained/unreleased packages that people do want released could be useful? That would give such a team an indication of where to concentrate their efforts and possibly reduce the workload.

#15

Please see the REPs 127 / 140 which already describe something like such a “QA group”.

You can use the following maintainer information in the package manifest:

  • name: Unmaintained see http://wiki.ros.org/MaintenanceGuide#Claiming_Maintainership
  • email: ros-orphaned-packages@googlegroups.com

If you release a package with this information you should subscribe to that Google group and act on future notification emails for the packages. Otherwise it will be the same problem that nobody gets notified and takes action on build failures in the future. I also want to clarify that OSRF does not have the resources to subscribe to that group, investigate incoming messages and act on them.

I see the risk that if multiple people are subscribing to the mailing list nobody might act on incoming notification emails. But I guess that is something the potential QA group needs to figure out.

Regarding the location of “orphaned” repositories: I don’t think moving them around when the maintainer status changes (which could happen multiple times over time) is beneficial. Consider users having cloned such a repository and the org unit is changing just for the sake of grouping them. Also the maintainer status in the distribution files already provides a good list to enumerate such packages. Adding a filter for that on the status pages would be great - patches adding this functionality would be highly appreciated.

1 Like
Baxter release permissions
#16

dirk-thomas https://discourse.ros.org/u/dirk-thomas
May 10

Oh didn’t know that, btw how can we subscribe list? Please confirm the
settings again or I may miss something

Adding a filter for that on the status pages would be great - patches

adding this functionality would be highly appreciated.

Can you point out the repository to send the patches?

Best

#17

The status pages are being generated by the ros_buildfarm. The relevant Python code is in the file status_page.py. And this folder contains the templates.

#18

When all else fails, it is important to be able to search within github and have forks point to the main repository for ongoing development, so as a last resort ROS forks of repos should be deleted and re-uploaded from a local clone.

As others outlined in this discussion, it is important to get in contact with the owners of repositories that no longer wish to maintain them and ask them to transfer their ownership within github for the reasons below.

Even if an owner adds notice to a repository regarding a new upstream fork, there is still an actual problem transferring repository ownership would fix. I found this conversation from my post in sniekum/ar_track_alvar/issues/124, so please look at the top of ros_perception/ar_track_alvar page. There is a message there:

forked from sniekum/ar_track_alvar

Every single fork points to the owner’s repository, which is inconvenient, but there is a second problem.

Next, click this to search “tag” in ros-perception/ar_track_alvar, you will see this error:

Sorry, forked repositories are not currently searchable.

As you can see, it is impossible to search in the other repository because github thinks this is the main repository, which is a problem that gets worse as the forked repository evolves.

Tansferring is the correct solution, then deleting and reuploading without a fork is a good backup, though this will still have the problem in all other existing forks. Therefore, a transfer is the only way github will correctly reflect the new primary development repository, such as https://github.com/ros-perception/ar_track_alvar/. These are the reasons I advise making every effort to see if owners will agree to and make an ownership transfer, and I’m sure many owners would agree once these problems are explained!

As a backup I suggest the ROS team delete and upload from their local clone, with a link/credit to the original authors in the README. This will enable search and allow other users who fork to point back to the new source of ROS development.

Time for reviewing ROS distro release cycle?
#19

Hello,

Is there any progress or anything that I can help on this issue?

This topic is really important for those who have robots that are no more supported by suppliers. (e.g. PR2)

Regards,
Yuki

#20

For future reference, here are recent developments on this: Maintaining and releasing orphaned packages

1 Like