On (git) branching strategies

[carlosjoserg]

Hi all,

Motivated by the discussion I started here, or similar threads like this one, I’m gonna fire this thread.

First of all, I reckon that a maintainer chooses the way that best fits his/her needs to maintain a repository. So I’d like to avoid pointing at particular cases where something’s happened or not, how often or not… I rather have a discussion regarding pros and cons of git branching strategies within the ROS community practices.

Currently, in most of ROS repositories, one sees this approach:

           C5 <- C6 <- C7 <- [melodic-devel]
          / 
  C3' <- C4 <- [lunar-devel]
 /
C1 <- C2 <- C3 <- [kinetic-devel]

Where a new development branch is created for every release candidate. That requires the maintainer to create this branch, and be aware of the latest branch created to accept PRs or continue developing.

The most critical thing to me is that, newer releases are not guaranteed to have bug fixes / new features. For instance, in the figure above, if C3 gets accepted on kinetic for any reason, then it needs to be manually forward ported to lunar, that is, from an older version to a newer version, which does not seem right. No matter how often this might happen, there is the possibility that it might.

Another minor issue is, I think in some repositories the branch is created only if there is a change to make and there is a new available release, or even because the maintainer just stopped creating branches. In that case, if a repository has not changed since, say hydro, and no one has PRed since, then usually the latest branch in the repository is that one, even when the same version of code exist for, say melodic, via apt-get, which doesn’t seem right either or might lead to confusion.

An alternative would be:

  C3' <- C6' <- [kinetic]    [lunar]     [melodic]
 /                        /           /
C1 <- C2 <- C3 <- C4 <- C5 <- C6 <- C7 <- [master]

Where there is a single development line, say master, devel, or unstable, something you readily identify as the branch to target PRs to by default for new stuff. Maintainers still need to tag / branch for a new release, but at least they don’t need to worry about which is the latest development branch and coordinate PRs.

A small big difference about this strategy w.r.t. the one before is that, if C3 is again a bug fix, newer releases will have it by construction, no need to be aware of any latest branch to accept PRs, or wait for a branch to be created because it’s better to target it to the a newer release. Development and releases are decoupled this way. Moreover, one can still back port to older releases, and if it happens that one needs more work to do that, one can still add stuff to older release branches. The concept of pre-release tags is clearer to use in this strategy too.

For the record, I know this is not a new idea, there are a bunch of repositories out there being maintained like this, hence the motivation of the post, it makes me wonder why most of ROS repositories choose the first strategy.

Any thoughts?

[gavanderhoorn]

Thanks for starting this. It’s an interesting topic, but also one I feel is almost impossible to reach some sort of consensus on, as it appears it comes down as much to personal preference as it does to some objective measure of ‘being better’ (do a search on Git branching strategies, there are hundreds if not thousands of results.

@carlosjoserg

The most critical thing to me is that, newer releases are not guaranteed to have bug fixes / new features. For instance, in the figure above, if C3 gets accepted on kinetic for any reason, then it needs to be manually forward ported to lunar , that is, from an older version to a newer version, which does not seem right. No matter how often this might happen, there is the possibility that it might.

You also described this example workflow in Proposed changes to the ROS releases - #10 by carlosjoserg and I feel it’s brought as describing a common practice, but I’m not convinced it is.

My experience is that PRs are expected to target the newest development branch and are then selected for backporting if it makes sense. PRs not targeting the latest branch will either be retargeted or rejected. At least, this is what I see in more ‘mature’ repositories. ROS being as distributed / federated as it is, this may be different for the repositories that you see often.

Managing things this way seems to already be rather close to what you are suggesting (single development line: latest branch, PRs target that by default, backporting, and “add stuff to older release branches if required”).

@carlosjoserg

For the record, I know this is not a new idea, there are a bunch of repositories out there being maintained like this, hence the motivation of the post, it makes me wonder why most of ROS repositories choose the first strategy.

“most repositories” is a rather vague statement, which makes it hard to verify. I know you didn’t want to point to any repositories in particular, but perhaps this topic would become more actionable if we had at least some numbers (“only 10 out of 100 repositories examined do this” ) or specific examples that clearly don’t or do things like this.

[de-vri-es]

@gavanderhoorn

My experience is that PRs are expected to target the newest development branch and are then selected for backporting if it makes sense. PRs not targeting the latest branch will either be retargeted or rejected. At least, this is what I see in more ‘mature’ repositories. ROS being as distributed / federated as it is, this may be different for the repositories that you see often.

This is indeed my experience as well, but many repositories still have indigo-devel, kinetic-devel and melodic-devel branches, even if only one of them is used to accept PRs. I found this confusing in the past and it’s still bugging me from time to time.

The correct devel branch changes all the time. With most non-ROS projects the workflow for making a PR is simple: update master, branch off, fix stuff, send PR. With most ROS projects you first have to check what the right devel branch is to split off from. Once you get used to it, it’s not so much of a problem. But I know it was confusing to me at the start.

A small tweak here could make a big difference I think: remove the -devel prefix from the release branches, and have one master/devel branch that targets the latest supported ROS release for a specific project. This is much closer to a regular git workflow, so it should lower the entrance barrier for new developers.

[simonschmeisser]

I agree with @de-vri-es, I was quite confused at the beginning about using/contributing to a hydro-devel when the current version is kinetic. After some time you realize that the branch name basically encodes the level of backwards compatibility. But my first reaction was to think this is orphaned/stale code.
But I think this is mainly a communication issue, not a work flow or organizational one.

[carlosjoserg]

Thanks for your inputs.

@gavanderhoorn

…almost impossible to reach some sort of consensus on, as it appears it comes down as much to personal preference as it does to some objective measure of ‘being better’ (do a search on Git branching strategies, there are hundreds if not thousands of results.

True, so it seems like it worth the discussion.

@gavanderhoorn

My experience is that PRs are expected to target the newest development branch and are then selected for backporting if it makes sense. PRs not targeting the latest branch will either be retargeted or rejected. At least, this is what I see in more ‘mature’ repositories. ROS being as distributed / federated as it is, this may be different for the repositories that you see often.

@gavanderhoorn

“most repositories” is a rather vague statement, which makes it hard to verify. I know you didn’t want to point to any repositories in particular, but perhaps this topic would become more actionable if we had at least some numbers (“only 10 out of 100 repositories examined do this” ) or specific examples that clearly don’t or do things like this.

I couldn’t query github by branch names, since only the default branch is indexed, though that wouldn’t explain the strategy either. And if you consider that other platforms like bitbucket or gitlab are also used, yes, I agree it is really hard to verify. However, I think we can agree that it is a very common practice in ROS repositories to use the first strategy, for either trunk or leaf projects, otherwise there wouldn’t be point to talk about, right?

I didn’t want to point at repositories because I wanted to be abstract about the discussion. But certainly examples help to illustrate the issue, and support the statement: “it can happen”. I already gave you one example on the ros_comm repository.

And, with all due respect to maintainers, just happened in the ros-controls project too, for instance: PR#335 just got merged on kinetic-devel with a new feature that would be good to have in further releases. And the melodic-devel branch is already there. So the melodic-devel branch does not have this commit, it will need to be added manually at some point, or a rebase of the melodic-devel branch too, which in this strategy is not a good idea either, I believe.

@de-vri-es

The correct devel branch changes all the time. With most non-ROS projects the workflow for making a PR is simple: update master, branch off, fix stuff, send PR. With most ROS projects you first have to check what the right devel branch is to split off from…

Precisely my question: Why don’t ROS projects choose the simple version? Which IMO is not only good to lower the barriers for new devs as you say later, but it is also safer.

@simonschmeisser

But I think this is mainly a communication issue, not a work flow or organizational one.

Also true, but what I’m interested in is on what are the benefits of using the first strategy above, because on the contrary, I think is not safe for the issue I illustrated before.

[Daniel_Stonier]

I’ve been maintaining repositories for quite a while in the ROS world and have had my perspective change considerably. A big part of that is just reducing scope to that which I can practically and reliably maintain / document. That’s kind of taken me full circle back to a very regular devel/release style:

1. Only ever one ‘devel’ branch, all other branches are releases
2. The README in the devel branch communicates what distro it is currently (unofficially) supported on
3. Bugfixes only on release branches, PATCH version bumps only, always tag every version
4. Reflect the distro it is used on in the name of the release branch
5. Reflect the MAJOR.MINOR version number in the name of the release branch as well

I find this communicates the purpose of each branch effectively with minimal fuss. People know then where and what they should PR to (features to devel, bugfixes to the release), what distro they need for any branch (no need to document). They can also quickly grokk when feature changes occur across distros - don’t bump your version just because the distro bumped. Another key point is that the root url to anything in your repo will never change (as it would if you changed your default branch to -devel for every ros release).

Example:

devel
release/0.6-kinetic
release/0.7-lunar-melodic

One downside with this is that you can’t support users who wish to contribute features for a distro that you’ve already moved on from (i.e. not supported by ‘devel’). At the end of the day though, supporting that is complicated, prone to error and there are good reasons in the software world why this is rarely done (be sure you have the cycles to do so).

[floris]

Just a comment about the popularity of the approach. I think that the alternative model @carlosjoserg describes in his post is what in general software engineering is called Trunk-Based Development. These days it is quite popular for companies which are trying to practice DevOps, in fact there is a website dedicated to it: https://trunkbaseddevelopment.com/

It is also referred to in books such as Accelerate as being “correlated with higher delivery performance” (Humble, Forsgren, Kim, 2018). Of course, that should be taken with a pinch of salt given that what works in IT might not necessarily work in robotics (nor do I completely trust their research approach).

[musamarcusso]

I have been looking through many repositories to try and learn how the workflow is done since I am trying to do it for my packages as well. I agree with @carlosjoserg, having multiple devel branches is very difficult to understand (at least that is my personal view).
Are there examples of repositories that use something closer to the trunk-based development workflow or something closer to how git flow provides? I would really appreciate to have more information on that.