New ROS ecosystem paper

General
1

Just wanted to share our recently published paper on the ROS community. “It Takes a Village to Build a Robot: An Empirical Study of The ROS Ecosystem”, check it out: https://ieeexplore.ieee.org/abstract/document/9240632

I’ll be speaking on the Software Quality panel tomorrow at ROS World, hope to see many of you there!

6 Likes
2

Sounds very interesting! Looking forward to hearing more at the panel tomorrow.

3

Amazing how you can find papers by adding “PDF” to Duck Duck Go searches of the title. :wink:

1 Like
4

Permalinking the DOI number is also helpful for tracking down the paper from other online or library sources; incase the IEEE link breaks, or the github repo changes over the decades.

It Takes a Village to Build a Robot: An Empirical Study of The ROS Ecosystem | IEEE Conference Publication | IEEE Xplore

1 Like
5

I believe this is the presentation at ROSCon19 where these results were also shown: It Takes a Village: Collaboration in ROS (video, slides).

1 Like
6

This seems to be behind a paywall. :frowning:

7

@skpro19, see the link Kat posted: https://afsafzal.github.io/materials/KolakROS20.pdf.

1 Like
8

This is a fascinating read, thank you. As a newcomer to the community it really resonates with my experience so far.

Packages developed by organizations or individuals other than those [provided by foundational working groups (FWGs)] are rarely successful in convincing others to rely on their packages. When these other packages do gain reverse dependencies, they are almost always from other packages within the same organization. Explaining the reasons behind such behavior is beyond the scope of this study, however, releasing packages with very specific functionality, low reliability, and limited documentation could be among the reasons.

For me, the biggest impediment to finding and using packages from the community has been the poor UX and slow performance of the ROS package index compared with some other ecosystems e.g. PyPi, Crates, NPM, VS Code, Sublime, etc…

For example, I tried to find whether there were any generalized implementations of extended kalman filters in ROS2:

image
image1583×1134 116 KB

Some immediate questions I’m left with:

  • What are the descriptions?
  • How do I sort by the number of dependents, number of contributors, frequency of updates, github stars/forks or other indicators of quality or popularity?
  • How do I filter by compatibility?
  • How do I show more than 10 results per page?

I quickly gave up on searching using the package index, and instead just googled, which didn’t turn up much either. This experience to me explains the discrepancy between these two charts:

image image

The user flow which I imagine leads to this is:

  1. An organization likes the ROS infrastructure and wants to use it
  2. The FWG packages are easy to find and seem official, therefore trusted, and become a de facto standard library
  3. Everything else gets built in-house because it’s hard to find community packages

Recently I left a related comment on the RFC for package documentation requirements. This also dovetails somewhat with a recent thread I saw about migrating Answers to Stack Exchange rather than maintaining an in-house platform.

My general sense is that the ROS ecosystem might benefit from relying more on Github and other third-party services for measuring popularity of packages, viewing documentation, tracking issues, etc.

2 Likes