GSoC: Creating a default grasping library

MoveIt
1

Hi all,
I am a Robotics Master student at TUM in Germany and currently in the process of applying to the GSoC 2019. My project of choice is "Creating a default grasping library" for MoveIt! which was proposed by the OSRF.

While exploring possibilities for the project, a few questions came up:
What is the future of the existing MoveIt! Grasps? From the contribution, it still seems more or less actively maintained and developed. However, for example a melodic branch does not exist yet. Is there some kind of future roadmap for the package?

I did some further research on alternative grasping libraries and compiled a list below and a more detailed spreadsheet. Some of them are not available as ROS packages. Therefore it would need collaboration with the respective authors to integrate them. Do you think this is possible to achieve in the timeframe of the GSoC? Are you aware of any other promising candidates?

I am open to any other feedback / input to the project!

Best,
Jens

2 Likes
2

@davetcoleman and @mlautman, I believe you guys are the right ones to advice.

3

@jpetit I wrote my thesis on a grasp technique as well . I can provide the full document if there’s enough interest.

1 Like
4

I am interested in what you did for your thesis if your going to share it

5

@kfriesth sure, I’ve placed the full doc and code in this github repo

2 Likes
6

We are about to officially release moveit_grasps, a package we’ve been developing on internally at PickNik for many years. The code is already available in ros-planning org.

Regarding GSoC, MoveIt experts @brycew and @felixvd are interesting in being mentors for MoveIt this summer. Though I do not believe for this project.

7

@davetcoleman, thanks for the info. In the OSRF wiki entry about GSoC 2019, you (and @v4hn) are mentioned as a potential mentors for the project idea “create a default grasping library”. But from your answer I am not so sure if you are available as a mentor? Can you clarify that?

8

From what I understand, PickNik won’t be mentoring GSoC students this year because they have their own interns coming in. Personally, I would applaud this effort, but I’m not sure I would be a helpful mentor on this topic, since I am not too deep in grasping.

If you are interested in manipulation however, there is a project I would be happy to mentor involving the planning scene and collision checking. There are a few directions the project can be extended in, so you could leave your mark on something that suits you. You can find the details below. The text will go on the OSRF wiki shortly, too.

Feel free to ping Bryce or me if it’s up your alley.

The following two GSoC projects focus on porting changes from Tesseract (Github), which has implemented some features that would be valuable additions to MoveIt. The projects are excellent opportunities for students to dive into and extend MoveIt, and making an important contribution without needing to do all the work from scratch.

1) Implement continuous collision checking and integrate Bullet library

  • List of prerequisities: ROS
  • Description of programming skills: C++
  • Difficulty level: Medium-high
  • List of potential mentors: Felix von Drigalski, Bryce Willey
  • Detailed Description: This project should add many useful and necessary features to MoveIt’s collision detection, including:
    • Continuous collision checking, to avoid collisions between discrete states
    • Optimizations to the collision checking process itself to improve performance (cache/scene reconstruction improvements, convex-convex collisions)
    • The ability to dynamically change large parts of the kinematics of the world (i.e. picking up a tray with many items should attach all of those items to the robot, not just the tray).
    • Integration with different collision checking libraries, such as Bullet
      Next to API changes, this includes writing tests and tutorials.

2) TrajOpt motion planner and optimization libraries

  • List of prerequisites: ROS, some motion planning background
  • Description of programming skills: C++
  • Difficulty level: Medium
  • List of potential mentors: Bryce Willey
  • Detailed Description: MoveIt was designed to be highly plugin-based and is able to support many different planing libraries, as seen with last year’s GSOC addition of STOMP and CHOMP. A next good candidate to be added to MoveIt is the TrajOpt planner. The changes made in the collision checking project (see above) will allow the TrajOpt planner to be added to MoveIt.
    • Assist with implementing features critical for TrajOpt, such as collision checking, including convex-convex collisions and continuous collision detection (see above)
    • Add TrajOpt as a library to be used in MoveIt
    • Add adapters that allow TrajOpt to post-process paths produced by other planners
    • Write Moveit tutorials and tests for TrajOpt motion planning
9

I would also be available as a secondary mentor for those two proposals (Improved collision checking and TrajOpt)

10

I would be very interested in working on one of the proposed topics @felixvd, preferably the first one. A few questions came up:

  • In general, the project is about taking features from Tesseract and porting them to MoveIt. Therefore, should the new features like continuous collision checking and convex-convex collisions be adapted from Tesseract and use Bullet?
  • In the Tesseract website it is stated that implementing TrajOpt ultimately resulted in not using MoveIt but instead developing the new lightweight motion planning framework Tesseract because the API changes to MoveIt would have been too big. How do we deal with this issue in the project?
11

Short answer: We will wrestle the dragon and change the API too :sunglasses:

Long answer: The necessary API changes are not so big that they will break MoveIt. If I remember @Levi-Armstrong’s explanation correctly, it was a combination of time constraints and external factors that made Tesseract end up as a separate fork. He can correct me if I’m wrong, but you can read that phrase as “too big to maintain reasonably sized PRs while developing the new functionality at full speed”. There was agreement in the maintainer meetings that the features will be welcome and possible to merge.

Yes, the idea is to adapt the feature from Tesseract to MoveIt. I believe there are some abstractions to be done rather than using Bullet directly though.

If you want to discuss the details, maybe we should take this to a Github issue? Anyone else who is interested can join us there. We imagined this as a 2-person project, if we can find enough hands.

12

Thanks for all the feedback so far, I started a follow up discussion as an Github issue.

13

It looks like @jpetit was convinced to look into one of Felix’s proposal, and I agree this is probably a better topic for GSOC than the one this thread is about. A request for convex shape representations and maybe a better abstraction for Collision checking plugins, that allows for using Bullet or FCL can be better defined beforehand. Though, I would advise to make continuous collision checking and Bullet-support fully independent items for now.
The whole proposal also requires a lot of insight into how things work behind the scene. :slight_smile:

I wanted to add a quick follow-up to the original question, in case anyone reads this later on.
There is basically two sides to “a grasping library”. Applicable research (agile grasps, gpd, pointcnn, gqcnn, …) usually focuses on detection of grasp points. Some of the usual problems in making them “default” are

  • they usually work best (or at all) with a specific sensor
  • they depend on unpackaged or hard-to-package machine learning libraries

On MoveIt’s end the next questions though is how to plan a trajectory through this grasp point that actually grasps the object - and how to make errors in this process transparent.
In the recent past there has been quite some work (including a GSOC project last year) on https://github.com/ros-planning/moveit_grasps, and also on the MoveIt Task Constructor to get robust tools for the latter part.

2 Likes
14

Hi @jpetit, adding to @v4hn’s answer about grasping:
The melodic branch of MoveIt Grasps has been added and the release version should be finalized the next days. Currently, the package supports sampling of top and side grasps for cuboid objects using finger or suction grippers. The grasp planner computes approach, grasp, retreat and lift motions that can be executed sequentially. I think this approach could be used as the default grasp planner, at least if we add support for other primitive shapes or even meshes.

However, I would vote for implementing some high level pick and place pipeline/capability.
Since grasping and placing often go hand in hand it would really make sense to setup a multi-stage-planner with MTC. This way we could use a default grasp generator (i.e. moveit_grasps) but still allow integration of other more sophisticated or experimental grasp libraries.

1 Like
15

Thanks for the info @Henning_Kayser, good to see new features for grasping in Moveit - for GSoC however, I decided to apply for the topic proposed by @felixvd.

1 Like
16

Hi everyone,

My name is Ayush Garg. I am a graduate student at Northeastern University with a major in Robotics. I am really excited to see MoveIt participating in GSoC this year again. I would like to contribute in the MoveIt Grasps project. Is it being considered this year?