Open Letter to MoveIt! Community

MoveIt
1

An Open Letter to the MoveIt! Community

Dave Coleman
8 Nov 2018

Dear MoveIt Community,

I’m writing to make some changes to the direction of MoveIt! - let’s shake things up, try new things, and be less conservative in making improvements. I’m doing this because I really care about the role of open source robotics in the years to come. I envision a future where everyone has access to the economic prosperity of robotic manipulators.

A few weeks ago I attended an open source conference where I led a discussion session I titled “Stability vs Growth: Breaking API”. I presented to the audience of diverse projects a problem I believe we’ve been struggling with in our ROS and MoveIt! community: stagnation and irrelevance. After speaking with many attendees, I came away with the belief that our now 7 year old project is stuck, making only minor incremental progress since Willow Garage shuttered its doors in 2013. Meanwhile, the robotics world has raced forward.

There are many reasons I believe MoveIt! is facing this innovator’s dilemma, and I want to outline some of them here with the public declaration that we at PickNik intend to change this, with your help. We see an exciting future for MoveIt!, and we believe we can transform it from a stagnant open source project, to a globally recognized platform that enables organizations of all sizes to leverage robotics for their applications. Before I outline the solution, I want to clearly state some major parts of the problem:

Lack of Major Versions

MoveIt! has not yet officially been declared 1.0, yet we maintainers typically take the stance that API should never be broken, even in new distro releases. API must be broken for us to make the progress necessary for major new features like constraint-based planning, underconstrained Cartesian planning, and tighter integration with low level controllers. A recent SwRI effort came to the same conclusion, stating that "Early on in the process several limitations were found in both MoveIt!’s Kinematics and Collision environment… requiring significant API changes across multiple repositories" [1]. By not incrementing major release versions, I believe we are severely limiting our ability to innovate.

Lack of Master Branch

The MoveIt! policy has always been to run development out of the latest ROS distro *-devel branch, which I usually create right after OSRF announces the ROS distro as "released". This creates the conflict that any user desiring to use the most recent release of ROS will be getting a non-stable version of MoveIt!, or conversely, that the developers are always handcuffed from making meaningful changes to MoveIt!. I believe we need to instead develop out of a proper "master" branch, which is occasionally forked for each official ROS distro.

Under-Optimizing for Contributors

In the past, I have maintained the assumption, based on general community consent, that requiring any migration effort by users of MoveIt! due to breaking changes between versions should be avoided at all costs. I’ve now come to the conclusion that reasons such as “our company doesn’t have time to deal with the hassles of upgrading” should not be used to discourage open source contributors from improving MoveIt!. Companies building a product have more resources (or promise of them) than volunteer contributors. In order to innovate, I ask we shift our priorities away from stability and more towards growth.

Moving Forward

I believe we can take MoveIt! to the next level of motion planning capabilities by creating a more frictionless experience for contributors to make necessary and important changes to MoveIt!. This can be accomplished through following well-established software development practices such as incrementing major versions of software and working from an actual master branch. The various pain points that will arise in this process can be mitigated via clear communication (i.e. announcements on Discourse) and clear documentation (i.e. keeping the migration guide up to date).

PickNik is committed to growing MoveIt!: for example we’ve put effort into new tutorials, a new website, and hosting 3 GSoC students this past summer [2]. We’ve set quarterly goals that everyone on our team review MoveIt! pull requests on a weekly basis, and that we find avenues of funding MoveIt! 2.0 (MoveIt! running on ROS 2). We will be working with the other MoveIt! maintainers and the community in addressing the other issues outlined above.

I ask the community to join us in reviewing pull requests, adding new capabilities, and in being understanding when APIs change in future major versions of MoveIt!. If you disagree with my conclusions in this letter please voice them here, and let’s see if we can have a healthy discussion to find the best outcome for everyone.

Let’s continue to grow this awesome open source robotic planning framework!

Sincerely,

Dave Coleman
PickNik Consulting

[1] https://rosindustrial.org/news/2018/7/5/optimization-motion-planning-with-tesseract-and-trajopt-for-industrial-applications

[2] http://moveit.ros.org/blog/

19 Likes
2

Overall, I think this is a smart move. A stable API is excellent, but more important than anything is the communication of breakage to your users (e.g. semantic versioning).

I assume you won’t upgrade major versions in the same ROS distro, right? So they won’t have to deal with the hassles of upgrading until they also need to upgrade ROS distros. Do you anticipate maintaining release branches so you can potentially backport bugfixes?

1 Like
3

Your questions are topics of ongoing discussions for the whole MoveIt! maintainer team, but I’ll provide my personal thoughts here.

I assume you won’t upgrade major versions in the same ROS distro, right?

This is getting trickier now that ROS1 releases are every 2 years. Software likes to develop faster than 2 year cycles, so I think we will need to come up with new release approaches outside of the ROS distro system. ROS originally was released every 6 months, which is fast enough to allow the packages to sync to the distros. I’m not sure what a good alternative is, here’s some ideas:

  1. Everyone builds newer releases from source, using perhaps git tags
  2. We setup our own debian repositories url like Gazebo does (packages.osrfoundation.org)
  3. We offer debians for direct download

I agree that we shouldn’t be breaking a ROS distro except perhaps the latest one.

So they won’t have to deal with the hassles of upgrading until they also need to upgrade ROS distros.

The unfortunate consequence of this philosophy is that it requires you to do a major upgrade of your OS to simply get a newer version of a motion planning software. This is a major hurdle to do all at once, but perhaps not worth fighting.

Do you anticipate maintaining release branches so you can potentially backport bugfixes?

We already have release branches for every version of ROS that we backport crtical fixes to, including indigo, jade, kinetic, and of course melodic.

4

I agree so intensely with your letter and proposal! Coming from other OSS communities I always found disconcerting the way this one in particular deals with backwards compatibility.

+1

5

I don’t think there is anything stopping you having multiple versions of moveit in a single ROS release. You just need to name the packages appropriately and ensure you prevent two versions being installed at once (or allow it but document how to choose which is compiled against).

1 Like
6

You just need to name the packages appropriately and ensure you prevent two versions being installed at once

This is another disadvantage of the ROS way of tiny packages… MoveIt! has at least 32 packages (based on the README.md) so renaming and re-releasing them all to e.g. moveit2_core would be cumbersome but certainly not impossible.

7

How independent are all those packages? If they are always used together, then maybe you could make the binary release a single deb. I’m not sure if the build farm or bloom support combining multiple packages into a single deb, though.

8

+1

I found this previous discussion about ROS-generic releases is relevant. I particularly agree with @tfoote there noting “The important thing is to clearly document the level of stability for a given package”.

Utilizing major version helps toward that goal. As noted at the bottom of this page on moveit.ros.org, MoveIt! uses minor version to differentiate platforms/distros, which limits the version number’s semantics (while the context of each release needs to be clearly communicated as Dave suggests, the clearer semantics of the version number by itself helps a lot IMO). Applying major version (and updating aforementioned version number policy) definitely helps toward better presentation of the software maturity of MoveIt!, which can bring about less confusion about API consistency.

(And personally as a release maintainer of MoveIt!, better semantics allows better handling of release cycles, which can result in quicker release cycles. Patch/bugfix release may require less caution than feature release does. I apologize for often holding releases unintentionally that may have contributed to stagnancy of MoveIt! (I hope that’s not the case…), but I can see this version policy change will positively affect release handling.)

As a (startup) company employee, I’m not sure if I can agree with that particular statement :confused:

Anyways, lack of guaranteed API stability definitely concerns corporate usage IMO. But there are ways to deal with it, and I’m sure people (not just in companies) have been handling it somehow,

My last point below is probably for maintainers only.

I assume the proposed plan is to consolidate all (active) *-devel branches into a single development branch and remove distro-specific devel branches. +1

Release maintenance still requires extra branching or any way to differentiate discrepancy per platform (I’m just not sure what the last sentence of cited paragraph points to), as some dependency new features require might not be available on older platform. We can continue in a recently started discussion among maintainers ros-planning/moveit#1168.

9

I assume the proposed plan is to consolidate all (active) *-devel branches into a single development branch and remove distro-specific devel branches. +1

What we have been discussing in the maintainer meetings is to create an additional master branch where most all new features go. The *-devel branches would remain but only be used for bug fixes after the initial release is made. This will give us the flexibility to improve the code dramatically in the master branch while keeping stability during a ROS release. The open question is how to make releases more often than the ROS distro schedule, which has slowed down to every 2 years.

1 Like