Perception includes very important components on ROS, there are 30 useful packages exist on github.com/ros-percepiton but not available on ROS2. Some topics people discussed to ask about the porting status (ROS2 pcl_perceptions, Porting Image Pipeline to ROS2), some developers may like to contribute. Not found an overall plan for RO2 perception porting, so just did some search and summarized here to show which are available for ros2, and which are not start that people could contribute to. Hope this will be helpful. If the information is not up-to-date, please share comments, I’ll continuously update it.
As a maintainer and/or contributor of some of these, a few thoughts:
calibration - not sure it is worth actually porting to ROS2 – while I spent a lot of time working on it at Willow Garage, it is still fairly PR2 specific. I think both the robot_calibration and industrial_calibration packages are in wider use, and more robot-generic. This calibration package is also mostly python-based, with hardly any tests, and so moving it to 3.0 will be quite a bit of work.
graft - was never really finished. I would consider it superseded by the robot_localization package, which has already been ported to ROS2 AFAIK.
sparse_bundle_adjustment / slam_karto - I would suggest that we not port sparse_bundle_adustment, and instead merge one of the several branches out there that implement better solvers for slam_karto – for instance, using ceres-solver.
There has been discussions in the past about stopping releasing (open)slam_gmapping in ROS distributions in favor of more performant SLAM approaches like karto or cartographer, so not sure if it should be ported to ROS 2.
Some packages have been already (partially?) ported:
I would agree with the decision to avoid poor-performing packages to ROS2. In general, the large number of ROS1 packages can be daunting for someone who is just starting out, especially when it’s not immediately clear which of those packages are considered “best practice” or are even actively maintained. The ROS QA group should help alleviate this but I think the easiest way to make it clear is to simply not make older outdated packages available in ROS 2.
On this topic, I notice that we have no object detector package available, either in this list or in ROS2. The “standard” answer to this problem, Object Recognition Kitchen, has not seen updates in quite some time, so I would argue that it is not the best candidate going forward. It would be great if the community could find an existing package that fills that gap for object detection.
I share the same view. “Object Recognition Kitchen, has not seen updates in quite some time”. For object detection/recognition, I would expect state-of-the-art algorithms being introduced to ROS2 .
I noticed these existing ROS2 packages missing from the ROS2 perception list, which I think they really could be part of it:
Real-time object detection, object tracking, and object localization.
People related detection
Mobilenet-SSD and Tiny-Yolo object detection
Alex-Net/GoogleNet/SqueezeNet/MobileNet/Inception object recognition
I think the challenge here is that object recognition isn’t “solved”, as most would consider robot navigation or arm planning (I know “solved” is a loaded term, but I hope it conveys the idea here, the navigation stack works about 80% of the time for a large portion of the robots out there. ORK in contrast has a hard time keeping up with changes in consumer packaging, and only supported ~60 or so objects at the time it was released, if I recall correctly). There are numerous cutting-edge research packages out there, but many of them aren’t maintained because the authors move onto better algorithms and approaches.
I totally agree with your assessment. Perhaps it would be a useful exercise to try and figure out what an 80% solution is for computer vision, such as something that would not require object-specific detectors.
I’ve previously used https://github.com/mikeferguson/simple_grasping at a number of places – it does the basic "segment a table and the objects that are on it, and generates grasps for MoveIt. I imagine that I will port it to ROS2 at some point, but there are a number of dependencies not yet ported (actions, MoveIt, and I think some PCL related stuff).
@mikeferguson would you please list detail packages name as you mentioned about some PCL related stuff not yet ported, not all ROS perception packages are worth to port, but the packages which are required or depended by other ROS2 application should be considered to port or improve on ROS2.
I’d have to look into it in more detail – a quick look at the CMake shows it depends on “pcl_ros” – mainly for the transform stuff if I recall, although once the package is ported to TF2, all that changes anyways, so it might not be a problem. Actions/MoveIt are the big ones (no sense having a grasp planner if you’ve got no arm planning software, and even in the upcoming release of ROS2 it is my understanding that actions will not be able to cross the ROS1/ROS2 bridge)
@mikeferguson, regarding grasping, I’ve done some upstream works on “GPD”. It’s a state-of-the-art algorithm for grasp pose detection and very easy for using in ROS. I’ve enabled a ROS2 grasp library as MoveIt plug-in for this algorithm, now in alpha version, coming soon
The GPD has been enabled with Intel OpenVINO technology for neural network computation deployment across various devices (CPU, GPU, VPU, etc.) and maximizes performance. Linkage here:
I have seen on Discourse and Github resources but found that no one has started to port pcl_msgs. Please let me know if anyone has not started pcl_msgs package. I am interested to port the same.
@yechun what about the support to Lifecycle managed nodes in image_common (and eventually other stacks)?
For example, to use image_transport with my node based on Lifecycle I had to write another “normal” node subscribing to image topics published by the “main” node and republishing them using image_transport.
.