ROS Resources: Documentation | Support | Discussion Forum | Service Status | Q&A answers.ros.org

Roadmapping and Future Direction

Recently at the first annual MoveIt Workshop (more details will be posted soon), we had many discussions about the future roadmap for MoveIt, both for ROS 2 support as well as other features to be added. We had a group brainstorming session with sticky notes that I hope Mark Moll can share in this thread.

I’d like to continue that discussion here for the greater community to get involved with, even if you couldn’t make the event in China.

We’ve just created a new Roadmap page on the MoveIt website with our immedate ROS 2 plans, as well as some farther reaching ideas. However, these ideas are just that - hypotheticals that require resources to further develop and pursue.

Please provide your feedback and thoughts below, with the knowledge that an open source project depends on community volunteers, as well as possibly donors, to actually make these new features happen.

I’ll copy the current roadmap table here, also, for ease of reference:

MoveIt 2 Immediate Milestones

Future Possible Milestones

p.s. next week is World MoveIt Day (Nov 20th 2019)

6 Likes

Here are some notes from discussions at the roadmapping session of the MoveIt workshop (thanks in part to @mikeferguson and @marip8):

Core Planning

  • The main concerns related to integration between planning and control:
    • Planning with dynamics limits, torque limits, or end-effector payload
    • Real-time collision checking/avoidance & dealing with moving objects while respecting joint limits
    • Closed-loop plan execution (maybe synthesize a controller along with a path/trajectory?)
  • Better support for Cartesian planning and constrained planning more generally.
  • Determinism and predictable behavior are also high on the wish list.
    • Personally, I think multiple parallel planning pipelines (as described in the previous comment) is the way to go.
  • Planning as a service to allow planners to be written in any language (this probably belongs in the next group).
  • More flexibility is needed in how planning is performed:
    • Per-planner planner request adapters
    • Allow multiple planner plugins in the same group
    • Multi-robot cooperative planning. This can mean many different things:
      • Centralized planning for multiple robots
      • Decentralized planning and using scheduling to interleave motions as much as possible
      • Reactive planning
      • Etc.
  • It would be nice to get search-based planning working again

API / High-level interface

  • Concerns
    • How to provide appropriate Python support
    • Improve modularity with more plugin interfaces to core components/functionality
    • Improve the layering of abstraction from core components to higher level interfaces
  • Action items
    ​ - Look into PyBind 11 for Python bindings
    ​ - Look into replacing plugins with ROS2 component nodes
    • Generate high-level architecture diagram illustrating how the components of MoveIt are structured and interact with each other
    • Deprecate the moveit_commander and use a more “stand-alone” Python package with a high-level API to call move_group services
    • Provide Python function for directly planning/moving to a Cartesian position (possibly already exists?)

Applications / Industry / Specialties

  • Industry Priority 1 - Broadly described as better environment modeling/integration
    • Support for dynamically updated robot model (for instances, supporting tool changers, especially when the end effectors are actuated)
    • Support for multiple robots in a scene (and actually being able to plan with multiple robots at the same time)
    • Support for external IO integration (ties back to the tool changer aspect, first class support for integration with typical industrial IO)
  • Industry Priority 2 - Broadly described as “more moveit task constructor features”
    • Support for synthesizing a sequence of tasks (large portions of this probably already exist in MTC - but aren’t widely understood yet)
    • Exposing motion primitives through action interfaces (for instance, action interface to have MTC execute a particular task. At the far end of this, having MTC basically replace move_group so that we have truly dynamic/programmable/extensible ROS interface to MoveIt)
    • Process planning and simulation (Gijs can certainly expand on what this means – I didn’t capture all of that)
  • ROS2 Priority 1 - Environment Representation / API
    • Look into unifying environment representations between MoveIt, ROS2 Navigation and Autoware (e.g. we have Octomap+links, Navigation has cost maps, etc - they all do similar things, lots of code being duplicated)
    • More sensors supported in planning scene, instead of just pointclouds
    • Add scene graph support to represent the relationship between objects
  • Community Priority 1 - Add support for Convex Collision checking, to support things like TrajOpt
  • Visual Impact 1 - Add support for mobile manipulation
    • Add planning support for diff drive robots
    • Probably need to fix bugs related to assumptions of fixed frame
    • Consider other/expanded environment representations (e.g. base_link-centered 3d-costmap instead of world-frame-centered Octomap)

This is a follow-up to the rather misplaced initial listing in https://github.com/ros-planning/moveit.ros.org/pull/371

Thank you so much @mamoll, @mikeferguson, @marip8 for the additional notes!

As I sadly did not attend the workshop, here is some feedback from my side, too.

Milestone 2

MoveIt already preempts motions if a new collision is detected.
isRemainingPathValid does exactly that and is implicitly used for every single trajectory
we execute and for every change in the planning scene the system observes.

Lifecycle Management of MoveIt nodes

Supporting a simple STARTING_UP , RUNNING , SHUTDOWN transition for move_group is simple enough, but is this what the point is supposed to mean?
Usually industry argues in favor of not using standalone nodes for MoveIt because this intrudes on their internal design concepts.
If people are supposed to use classes/libraries in their own node, I don’t see what we can/should do to support lifecycle management of their nodes.

Replace pluginlib with components

In the last maintainer meeting where we discussed this point, it was very fuzzy what this could look like.
Also, there was no agreement at all among the maintainers whether this would be a good idea, because it would make tracing bugs much harder if no backtraces in clear call-hierarchies are possible anymore.
Were there more insights / some consensus on possible approaches discussed during the workshop?

Industry Priority 1

Support for dynamically updated robot model (for instances, supporting tool changers, especially when the end effectors are actuated)

This is not just a matter of MoveIt, but the general question on how this is should be implemented across ROS2. Still, MoveIt could pioneer a new “standard” approach for this (and extend other packages to support it).

Support for multiple robots in a scene (and actually being able to plan with multiple robots at the same time)

This is already possible right now by having a big urdf with all robots and many people did it in the past. What exactly is new/requested here?
If multiple separate URDF models in one scene are targeted, I would second this request, but this also leads to problems such as

  • how/where do you define the geometric relation between the models?
  • how/where are joint states for all models reported and merged?
  • Should MoveIt support any number of CurrentStateMonitors ? Should we generalize all interfaces to support any number of robots?

Industry Priority 2 - Broadly described as “more moveit task constructor features”

Great! I would appreciate more detailed feedback and further explanations to actually improve things in person.

More sensors supported in planning scene, instead of just pointclouds

What does this mean? Sensors are not at all supported in the PlanningScene, geometric measurements are. What alternative measurements are relevant to people?

Add scene graph support to represent the relationship between objects

This would make for a great “Milestone 2/3” entry!
It’s an often-discussed shortcoming of MoveIt as of today and it’s clearly achievable.

Add support for Convex Collision checking, to support things like TrajOpt

We noticed that this might mean two entirely different things and I would still like to know what was meant in the workshop discussions:

  • Support for declaring/handling meshes as convex, so as to exploit faster collision checking algorithms in this case
  • Support for heuristically checking for collisions of trajectories by creating the convex hull of start and end state.

Convex mesh support would make a wonderful MoveIt2 milestone entry.

The second interpretation could improve continuous collision checking, but might be more involved.

Visual Impact 1 - Add support for mobile manipulation

As I’m probably one of the very few people who used this in recent years, here’s my experience with the current state:

  • The virtual joint can be used to define a single holonomic joint for driving or flying.
  • The whole concept of “workspace” in MoveIt currently limits the positional range of these (otherwise unlimited) joints.
  • With a holonomic robot base, e.g., the PR2, planning arm&base motions together works. The results are trajectories that, for example, move the robot around while grasping something.
  • The trajectories can, however, not easily be executed as there are usually no base controllers that can receive a MultiDOF trajectory and actuate it. moveit_simple_controller_manager does not support any ROS interface to forward such a trajectory either and needs to be extended/replaced.
  • especially with such base motions, motion planning should typically be biased to prefer moving arms over the entire base. This is not trivially available in the pipeline at the moment, although OMPL probably has some support for it (e.g., optimizing for minimum effort)

Add planning support for diff drive robots

This boils down to implementing a new joint type DifferentialJoint and support it throughout the pipeline.
I would expect this to be quite challenging when working on the details.