I was just reading this nature article by Anna Nowogrodzki from yesterday and wondered if anyone else here in the community had any opinions or perspectives on this topic in relation to robotic software:
Releasing lab-built open-source software often involves a mountain of unforeseen work for the developers. doi: 10.1038/d41586-019-02046-0
A particular parts that resonated well with my experience in robotic research:
Scientists writing open-source software often lack formal training in software engineering, which means that they might never have learnt best practices for code documentation and testing. But poorly maintained software can waste time and effort, and hinder reproducibility. Biologists who use computational tools routinely spend “hours and hours” trying to get other researchers’ code to run …
“They try to find it and there’s no website, or the link is broken, or it no longer compiles, or crashes when they’ve tried to run it on their data.”
References related to article
Invisible Work, Incentives, and Burnout in Open Communities: A Qualitative and Quantitative Study
SciPy 2019, track: Open Source Communities, Friday, July 12, Room 204
Dorothy Howard, UC San Diego Department of Communication & The Design Lab
Stuart Geiger, University of California, Berkeley, Berkeley Institute for Data Science
Lilly Irani, UC San Diego Department of Communication & The Design Lab
Alexandra Paxton, University of Connecticut Department of Psychological Sciences
Chris Holdgraf, University of California, Berkeley
Nelle Varoquaux, University of California, Berkeley
Of course this has happened to me plenty of times. The packages in the ROS ecosystem differ a lot in quality, from well-maintained, tested and documented software to “worked once to produce the results for the paper”.
The part that didn’t resonate with me from your quote is the thing about formal training. In contrast to biologists, roboticists often do have formal training in software engineering. I think it’s more a problem of incentives. Most of the open-source ROS software comes from academia, but maintaining good open-source software is not something that is incentivized. The output that counts is high-quality papers, innovative project proposals etc., and while you need to write software for this (as a roboticist), it only needs to work once. Making it usable for other people is something you do in your spare time, or not at all. I believe that is why organizations like Willow Garage in the past and OSRF today are so critical: They focus on creating and maintaining critical infrastructure, not writing papers. If only the funding agencies would recognize this and fund them directly instead of funding the n-th research project whose software output is either never made open source or is unusable to begin with, we would all be better off.
Once thing I should add perhaps: This was not intended to be a rant about academia. Industry is even worse in this respect - they have even less of an incentive to provide free tech support, trouble-shoot user’s problems, review pull requests and provide and maintain well-documented and tested open source software.
Bit of a plug, and unfortunately we can’t support anyone once the project is over (that idea didn’t make it past the reviewers) but what you describe was a big motivation for writing the proposal.
Great read… and while within the ROS-ecosystem there are a fair number of properly trained software people, there are quite a few that are “learning on the job” so we find here that there is some of that difficulty in spin up/meaningful leverage and subsequent contribution.
I look forward to sharing this with some other programs here in the US where we are seeking to incentivize the continued support and growth in the utility and capability and less on the paper.
