ROS2 perception porting status

As a maintainer and/or contributor of some of these, a few thoughts:

  • calibration - not sure it is worth actually porting to ROS2 – while I spent a lot of time working on it at Willow Garage, it is still fairly PR2 specific. I think both the robot_calibration and industrial_calibration packages are in wider use, and more robot-generic. This calibration package is also mostly python-based, with hardly any tests, and so moving it to 3.0 will be quite a bit of work.

  • graft - was never really finished. I would consider it superseded by the robot_localization package, which has already been ported to ROS2 AFAIK.

  • sparse_bundle_adjustment / slam_karto - I would suggest that we not port sparse_bundle_adustment, and instead merge one of the several branches out there that implement better solvers for slam_karto – for instance, using ceres-solver.

-Fergs

There has been discussions in the past about stopping releasing (open)slam_gmapping in ROS distributions in favor of more performant SLAM approaches like karto or cartographer, so not sure if it should be ported to ROS 2.

Some packages have been already (partially?) ported:

I would agree with the decision to avoid poor-performing packages to ROS2. In general, the large number of ROS1 packages can be daunting for someone who is just starting out, especially when it’s not immediately clear which of those packages are considered “best practice” or are even actively maintained. The ROS QA group should help alleviate this but I think the easiest way to make it clear is to simply not make older outdated packages available in ROS 2.

On this topic, I notice that we have no object detector package available, either in this list or in ROS2. The “standard” answer to this problem, Object Recognition Kitchen, has not seen updates in quite some time, so I would argue that it is not the best candidate going forward. It would be great if the community could find an existing package that fills that gap for object detection.

@yechun openslam_gmapping and slam_gmapping are actually ported already.

@marguedas I ported these a while ago, just as an exercise. So maybe the solution is not to release them, but know they exist?

I share the same view. “Object Recognition Kitchen, has not seen updates in quite some time”. For object detection/recognition, I would expect state-of-the-art algorithms being introduced to ROS2 :slight_smile:.

I noticed these existing ROS2 packages missing from the ROS2 perception list, which I think they really could be part of it:


Real-time object detection, object tracking, and object localization.


People related detection


Mobilenet-SSD and Tiny-Yolo object detection
Alex-Net/GoogleNet/SqueezeNet/MobileNet/Inception object recognition

@sharronliu - object analytics does detection but not recognition, is that correct? What about the last one on your list

Has anyone created a ROS2 package for that? Do we have any object recognition solution in ROS2?

@mkhansen Right you’re. The “Object Analytics” takes the detection results, and provides additional tracking and localization information.

For object recognition, the ROS2 package “ros2_intel_movidus_ncs” provide that feature. See details for which network models supported:


The “classification” results include label (class or type) of the object, probability.

Probably it is not always just about performance but reliability in the architectural level as well: Functional Safety Design Patterns

I think the challenge here is that object recognition isn’t “solved”, as most would consider robot navigation or arm planning (I know “solved” is a loaded term, but I hope it conveys the idea here, the navigation stack works about 80% of the time for a large portion of the robots out there. ORK in contrast has a hard time keeping up with changes in consumer packaging, and only supported ~60 or so objects at the time it was released, if I recall correctly). There are numerous cutting-edge research packages out there, but many of them aren’t maintained because the authors move onto better algorithms and approaches.

-Fergs

I totally agree with your assessment. Perhaps it would be a useful exercise to try and figure out what an 80% solution is for computer vision, such as something that would not require object-specific detectors.

I’ve previously used https://github.com/mikeferguson/simple_grasping at a number of places – it does the basic "segment a table and the objects that are on it, and generates grasps for MoveIt. I imagine that I will port it to ROS2 at some point, but there are a number of dependencies not yet ported (actions, MoveIt, and I think some PCL related stuff).

-Fergs

@mikeferguson would you please list detail packages name as you mentioned about some PCL related stuff not yet ported, not all ROS perception packages are worth to port, but the packages which are required or depended by other ROS2 application should be considered to port or improve on ROS2.

I’d have to look into it in more detail – a quick look at the CMake shows it depends on “pcl_ros” – mainly for the transform stuff if I recall, although once the package is ported to TF2, all that changes anyways, so it might not be a problem. Actions/MoveIt are the big ones (no sense having a grasp planner if you’ve got no arm planning software, and even in the upcoming release of ROS2 it is my understanding that actions will not be able to cross the ROS1/ROS2 bridge)

@mikeferguson, regarding grasping, I’ve done some upstream works on “GPD”. It’s a state-of-the-art algorithm for grasp pose detection and very easy for using in ROS. I’ve enabled a ROS2 grasp library as MoveIt plug-in for this algorithm, now in alpha version, coming soon :slight_smile:

The GPD has been enabled with Intel OpenVINO technology for neural network computation deployment across various devices (CPU, GPU, VPU, etc.) and maximizes performance. Linkage here:

1 Like

@yechun

I have seen on Discourse and Github resources but found that no one has started to port pcl_msgs. Please let me know if anyone has not started pcl_msgs package. I am interested to port the same.

Thanks.

@sandiprakhasiya I have not found someone work on it too. You could port the package and also announce on ROS 2 Collaboration Bulletin Board. thanks.

ROS2 perception (Updated on 12/26/2018)

  1. laser_assembler - Done
  2. image_transport_plugins - Done
  3. depthimage_to_laserscan - Done
  4. vision_opencv - Done
  5. image_common - Done
  6. openslam_gmapping - Ported
  7. slam_gmapping - Ported
  8. image_pipeline - In progress
  9. perception_pcl - Started(by prasenjitdan)
  10. ar_track_alvar - Not start
  11. slam_karto - Not start
  12. opencv_apps - Not start
  13. sparse_bundle_adjustment - Not start
  14. open_karto - Not start
  15. laser_filters - In progress (by vandanamandik)
  16. laser_proc - started (by yechun)
  17. imu_pipeline - not start
  18. pointcloud_to_laserscan - not start
  19. laser_pipeline - no need
  20. pcl_msgs - Started (by sandiprakhasiya)
  21. camera_info_manager_py - not start
  22. megatree - not start
  23. camera_pose - not start
  24. calibration - not start
    • may not worth porting, see comments of mikeferguson.
  25. graft - not start

Other perception modules

ROS2 perception (Updated on 2/27/2019)

  1. laser_assembler - Done
  2. image_transport_plugins - Done
  3. depthimage_to_laserscan - Done (branch ros2/depthimage_to_laserscan.git should likely be moved to the upstream repository)
  4. vision_opencv - Done
  5. image_common - Done
  6. openslam_gmapping - Ported (allenh1/openslam_gmapping )
  7. slam_gmapping - Ported (allenh1/slam_gmapping)
  8. laser_proc - ported (allenh1/laser_proc)
  9. laser_filters - Done
  10. pcl_msgs - Done
  11. perception_pcl - Started(by prasenjitdan) (pcl_conversions - original port on ros2 branch of [ros2/pcl_conversions.git ](https://github.com/ros2/pcl_conversions.git is currently used)
  12. image_pipeline - In progress (Issue: https://github.com/ros-perception/image_pipeline/issues/360
  13. imu_pipeline - started (by yechun)
  14. pointcloud_to_laserscan - not start
  15. ar_track_alvar - Not start
  16. slam_karto - Not start
  17. opencv_apps - Not start
  18. sparse_bundle_adjustment - Not start
  19. open_karto - Not start
  20. camera_info_manager_py - not start
  21. megatree - not start
  22. camera_pose - not start
  23. calibration - not start (may not worth porting, see comments of mikeferguson.)
  24. graft - not start (should be superseded by robot_localization, see comments of mikeferguson.)

Other perception modules

@yechun what about the support to Lifecycle managed nodes in image_common (and eventually other stacks)?

For example, to use image_transport with my node based on Lifecycle I had to write another “normal” node subscribing to image topics published by the “main” node and republishing them using image_transport.

There is an open issue: https://github.com/ros-perception/image_common/issues/108

I do not know if there are other stacks (other than tf2_ros that is outside “perception”) that lack the support to LifeCyleNode

I don’t have a need yet for ar_track_alvar on ROS2, but I’m happy to chime in case someone is depending on it. The ROS fluff on top of the rest of the source code is relatively small.