I have some questions about the labels for ROS Industrial code. Recently, I got a question from a company about the applicability of ROS Industrial for their production. For a typical solution with robot arms, I can use the industrial core, which is production ready. For path planning, we could select MoveIt, which is experimental, or Descartes, which is also experimental. I also need a vendor specific package and none of the vendor specific packages has the label “production ready”. They are all developmental. Does this mean that we can’t create a “production ready” solution at this moment?
Moreover, from the outside, it looks like all vendor specific packages have the same maturity or quality, because they are all developmental. However, to me it looks like there are differences in features and support between e.g. Yaskawa and Fanuc. Should there be a more detailed label for these vendor specific packages? With this information one should be able to know which robot / make / type to buy when one wants good ROS interoperability.
Personally I think the labels experimental for MoveIt! and Descartes are misleading - both have been deployed in many production systems, possibly with some cleanup.
Speaking for MoveIt! in particular, it has had stable Ubuntu debian releases since 2013, being re-released every few months for every version of ROS since 2013. Admittedly, MoveIt! has plenty of areas it could be hardened for production systems but has not been due to resource constraints. Yet calling it experimental seems unfair given the amount of work that goes into keeping it stable.
Relative to labels for packages, first a reference. We maintain on the wiki a software status page, it can be found: http://wiki.ros.org/Industrial/Software_Status. Where you can find the quote: “This status by design is not based on any quantitative evaluation, although these will come later. Rather, the status is merely the author/developer/maintainer’s informed opinion on whether the code is production ready.” This informed opinion could be based on actual experience running the code within an application and having performed means testing. However, at this time that is not guaranteed, and as indicated more quantitative measures are desired for future designation systems.
That being said we recently had a discussion on this topic among ROS-I stakeholders. And it is important to differentiate between code quality and application robustness. In one case we can have the highest quality code within a package but the implementation of all the components and the resultant application may not be production viable. I would argue that a number of traditional automation deployments in the manufacturing world, the software would not be rated “production ready”, however the solution provider has refined the solution to the point the return on the development mitigates the risk that they are willing to tolerate to support. This is the calculation the solution integrator makes each time for each deployment.
We are excited to see developments and improvements to the labeling and rating of packages for ROS driven by ROSin for instance, but understand that application verification and validation are still key steps in ensuring your application is ready for production. Obviously those verification and validation steps are less onerous the better the software components are.
Another area, such as OEM drivers for instance, while experimental, we are wanting to highlight those OEM packages that are OEM supported. At that point they can update the label based on their understanding of use in application, or via their own internal testing. Ideally we can get to the point where all developers use the same measures for those packages be they OEM supported or community supported.
I hope this adds some clarity and again, we look forward to further refining how we label and classify the packages moving forward.