New ROS ecosystem paper

sophiakolak · 2020-11-11T22:54:33.013Z

Just wanted to share our recently published paper on the ROS community. “It Takes a Village to Build a Robot: An Empirical Study of The ROS Ecosystem”, check it out: https://ieeexplore.ieee.org/abstract/document/9240632

I’ll be speaking on the Software Quality panel tomorrow at ROS World, hope to see many of you there!

chfritz · 2020-11-11T22:59:28.429Z

Sounds very interesting! Looking forward to hearing more at the panel tomorrow.

Katherine_Scott · 2020-11-11T23:10:08.465Z

Amazing how you can find papers by adding “PDF” to Duck Duck Go searches of the title.

ruffsl · 2020-11-12T00:34:11.502Z

Permalinking the DOI number is also helpful for tracking down the paper from other online or library sources; incase the IEEE link breaks, or the github repo changes over the decades.

It Takes a Village to Build a Robot: An Empirical Study of The ROS Ecosystem | IEEE Conference Publication | IEEE Xplore

ieeexplore.ieee.org

It Takes a Village to Build a Robot: An Empirical Study of The ROS Ecosystem

Over the past eleven years, the Robot Operating System (ROS), has grown from a small research project into the most popular framework for robotics development. Composed of packages released on the Rosdistro package manager, ROS aims to simplify...

gavanderhoorn · 2020-11-12T08:46:52.062Z

I believe this is the presentation at ROSCon19 where these results were also shown: It Takes a Village: Collaboration in ROS (video, slides).

skpro19 · 2020-11-13T18:43:24.160Z

This seems to be behind a paywall.

chfritz · 2020-11-13T20:27:05.114Z

@skpro19, see the link Kat posted: https://afsafzal.github.io/materials/KolakROS20.pdf.

33thou · 2020-11-14T19:51:49.288Z

This is a fascinating read, thank you. As a newcomer to the community it really resonates with my experience so far.

Packages developed by organizations or individuals other than those [provided by foundational working groups (FWGs)] are rarely successful in convincing others to rely on their packages. When these other packages do gain reverse dependencies, they are almost always from other packages within the same organization. Explaining the reasons behind such behavior is beyond the scope of this study, however, releasing packages with very specific functionality, low reliability, and limited documentation could be among the reasons.

For me, the biggest impediment to finding and using packages from the community has been the poor UX and slow performance of the ROS package index compared with some other ecosystems e.g. PyPi, Crates, NPM, VS Code, Sublime, etc…

For example, I tried to find whether there were any generalized implementations of extended kalman filters in ROS2:

image1583×1134 116 KB

Some immediate questions I’m left with:

What are the descriptions?
How do I sort by the number of dependents, number of contributors, frequency of updates, github stars/forks or other indicators of quality or popularity?
How do I filter by compatibility?
How do I show more than 10 results per page?

I quickly gave up on searching using the package index, and instead just googled, which didn’t turn up much either. This experience to me explains the discrepancy between these two charts:

The user flow which I imagine leads to this is:

An organization likes the ROS infrastructure and wants to use it
The FWG packages are easy to find and seem official, therefore trusted, and become a de facto standard library
Everything else gets built in-house because it’s hard to find community packages

Recently I left a related comment on the RFC for package documentation requirements. This also dovetails somewhat with a recent thread I saw about migrating Answers to Stack Exchange rather than maintaining an in-house platform.

My general sense is that the ROS ecosystem might benefit from relying more on Github and other third-party services for measuring popularity of packages, viewing documentation, tracking issues, etc.