Maintainer best practices handling changes through ROS releases

Packaging and Release Management
1

While I’ve been working in the different pull requests in gazebo_ros_pkgs I come up with doubts about how would be the best policy to handle changes in the different branches that we host there for every ROS release. As of now gazebo_ros_pkgs currently support changes in: Indigo, Jade and Kinetic. All of them officially released and supported. Contributors are submitting pull requests against any of the released branches and we usually cherry-pick the changes to the rest of the branches but the whole process is arbitrary and not documented in any way.

I can see that we have different kind of changes that will impact on users, at least:

  • Bugs (mainly things that are broken)
  • Changes to the API
  • Changes to the ABI (looking at catkin ws users)
  • Changes that affect rostopic/roslaunch/rosparams/… names or arguments.
  • Changes that affect behaviour (some of them are bug fixes)

My main point is: in which ROS release would be reasonable to introduce a change that forces consumers of the ROS packages to have to react to adapt to the change having in mind that some of them would be not trivial to detect?

Thanks

4 Likes
2

Some of my ideas: try to define “virtual types of distributions” that match stability vs new features. We could consider an “stable release” and “development release” and how these concepts fit into the ROS releases:

  • stable release: most of the users, any disruptive change should not be permitted only important bug fixes to be committed.
  • development release: brave users expecting fast new features and able to adapt quickly to changes.

At this moment I would say that Indigo would fit perfectly in what we consider a stable release. I’m not so sure about Jade. Probably Kinetic can fit well into what we can consider a development release. Obviously the upcoming Lunar falls into this category.

Reviewing the types of changes that I listed in my first post:

  • Bugs: if they fix things clearly broken, stable and development.

  • Changes to the API: clearly unacceptable for stable. If we consider Kinetic as a development release they probably are acceptable although they are have a large impact.

  • Changes to the ABI: given the position of ROS in this topic, they should not represent a problem except for people relying on mixing packages and catkin builds. I would probably exclude stable of this.

  • Changes that affect rostopic/roslaunch/rosparams/… names or arguments: in most the cases the impact could be really important, even worse than modifying the API since problems could not be directly visible.

  • Changes that affect behaviour (some of them are bug fixes): this category or group is difficult to manage. Take this PR as a good example of how a clear bug fix needs to be handle with care. I would say that have a case by case analysis and decision is a good way but by default anything touching a behaviour in an stable release (even if clearly fix it in the right way) should not be accepted.

There are other models if we define other “virtual types” of distributions (for example having Jade as a testing release before changes landed into Indigo) but I would like to keep the whole thing somehow simple.

2 Likes
3

@jrivero FYI I bumped this into the Release category as it’s best targeted at maintainers.

And there’s details per distro here: http://wiki.ros.org/Distributions/ReleasePolicy with LTS vs on-LTS. Note that Kinetic is also an LTS.

Our original versioning policy was here: http://wiki.ros.org/StackVersionPolicy but we’ve relaxed this as it was too burdensome.

As a general guideline I like to refer people to Semantic Versioning

I’ll note that our convention is a little bit different that Semantic Versioning in that we typically use patch releases for bug fixes as well as added functionality, as long as it is backwards compatible. And iterate the MINOR version between ROS distros. This is something we might want to consider changing, and suggesting that we follow the slightly more strict guidelines developed since then in semantic versioning. And then we could recommend that only major version changes go into new distros for stable packages.

One of the challenges is that we have a lot of packages in the greater ROS ecosystem and they are not all at the same level of maturity.

At the core we work very hard to maintain full ABI compatibility within any ROS distro for the life of the distro. We also make sure that any API changes have a full deprecation cycle of at least one ROS distro before an API is removed.

However, there are many users using ROS packages to collaborate fluidly on short term projects and need to have more flexibility. Binary builds are a very powerful tool for easy deployment and collaboration, and these faster moving projects cannot maintain the stability while still being productive. As such we do not mandate a specific policy for all packages. The important thing is to clearly document the level of stability for a given package. We do have a few states in the rosdistro status field defined in REP 141 but it’s not really adequate. Better communication of the maturity would be helpful. This was discussed in the package.xml format discussions but could not reach consensus so was left out of the specification.

We also try to manage expectations by only promoting packages to be inside the desktop and desktop-full if we consider them stable.

As a rule of thumb the more dependencies any package has the more stable it’s development process should be. As every incompatible change requires all dependent packages to update.

Our buildfarm makes no assumptions about ABI stability for any given release, and can successfully rebuild everything if there’s an ABI change. However, this may break a user who only does a partial system update. We have discussed explicitly versioning the ABI and in the future would like to be able to be more explicit about it. But ABI versioning has many different approaches and we could not find a consensus during the discussions so deferred the decision.

2 Likes
4

+1. But there should also be a point where Kinetic joins the stable-LTS family. I don’t know when and how that’ll be happen though. Maybe when the number of users of Kinetic surpasses that of Indigo?

Another concern in this subject I’d like to raise, which @jrivero did not include in the list of the change types, is about the new capabilities. While the addition could still cause regression even when AB/PI compatibility is carefully maintained, IMO it’s natural for the most of developers/users to think that new features will become available to the older supported distros. I’ve just seen some examples where new features added to kinetic branch were not backported to that of indigo. I guess the final decision may always be up to the package maintainers but it would be great if discussion/guidance at the core level (like on this forum) is available.

5

Formerly, the rule was simple: introduce new features in a new distro, so existing users are not impacted.

With Indigo LTS, I found myself wanting to support new hardware that was not available when Indigo was initially released. An example is the popular Velodyne VLP-16 LIDAR.

Since I believe I can add that model with minimal disruption to existing users, I’ve decided to make an exception to the old rule, which made more sense when there was a new ROS distro every six months.

6

This discussion is added to BestPractices on roswiki.

1 Like