Hi all,
in collaboration with researchers of the Carnegie Mellon University and the Vrije Universiteit Amsterdam, we defined a set of 49 guidelines about how to architect ROS-based systems. You can find the guidelines and the steps we performed for extracting them here.
Since we are just starting to do this kind of research on ROS, any possible feedback is very welcome!
What do you think about this kind of research? Do you find it useful? Is there some other aspect of ROS which you think is worth this kind of activities?
The full dataset we used for extracting the guidelines from GitHub repositories is available here.
Many thanks for sharing this! I read the whole paper and found lots of useful information! I will allow to share it on Weekly Robotics as Iām sure many people will find it useful.
Here are some things I think I would find useful mingled with some suggestions:
Iād be very interested to learn about the depth of launch files used in the analyzed projects. Quite often we have launchfiles including other launchfiles. Wonder what is the maximum depth that can be found in the wild
Iād also mention some guidelines on adding external packages. Ideally we want to list them as dependency and make rosdep handle it for us but sometimes packages are not released or we need to fork them in which case submodules are helpful (what I usually use but I know some people use some other repo managers)
What would be a killer for me is if someone took top 5-10 ātop repos of platformsā and recorded bag files with representative data. I think this would be very useful for āgood practicesā as you can quickly see how the whole stack works what kind of sensor data is provided and in what format etc.
Thanks Mateusz!
Itās a pleasure to be included in the Weekly Robotics mailing list.
About your mentioned points:
Launch files: there is a paper from 2015 about how ROS launch files are developed, see it here
External packages: I agree, this may also create maintainability issues since then you need to manually check if the added packages have been updated, right?
Bag files recording: this is a very good idea and it is technically doable. I am wondering about how to (i) choose the ātop platformsā in a replicable and unbiased manner and (ii) how to generate scenarios which are ārepresentativeā of the common usages of those platforms. I will think about it in the coming days.
This shouldnāt be too much of an issue actually:
worried about continuity (ie: external repositories disappearing): fork and point to your fork
worried about reproducability (ie: avoid tracking moving upstreams): specify versions (ie: commits/tags, not branch names) in scripts/.rosinstall/.repos files (avoid using scripts for seeding workspaces btw)
The second bullet is also less of a problem when using Git submodules, but I tend to not use those (for various reasons).
Comment/suggestion: this would be a great post for Quality Assurance category, instead of here in General. You could consider moving the post there.
@iivanoo: could you perhaps also say a bit more on the repository selection process? The paper doesnāt seem to explain how you selected the initial set of repositories (ie: before you started pruning).
Right, it all makes sense, thanks Gijs for the hints.
Yes, indeed, it started as an announcement and it is becoming a full discussion now
Sure. The initial set of target repositories has been defined by reusing two other research contributions:
Rosmap: it is an analysis tool for scanning dependencies among ROS packages in GitHub, GitLab, and Bitbucket. The first step of the analysis tool consists in performing queries in those three platform and look for repositories taggeg with the ROS topic. Also, rosmap checks the official ROS Index in the rosdistro-repository and considers all repositories mentioned in there as well. In my research I am using rosmap only for the search step.
GHTorrent: it is a database (either as MySQL or mongodb) containing a periodically-built mirror of all projectsā metadata in GitHub. I am querying GHTorrent by selecting all repositories which are either tagged with the āROSā topic or their name (or description) contain the āROSā substring. Here you can see the specific SQL queries I performed in GHTorrent.
Many thanks for the launch files paper! Will definitely have a look at it!
External packages: yes, if you use submodules then you need to manually sync them with upstream, especially if there are some interesting developments in there for you.
Bagfiles: I know that chossing ātop platformsā in unbiased manner is a challenge but maybe you can select them based on number of stars in the repo? Or you can be biased and say: these platforms do everything correctly in our view, check out examples of their raw data.
Iām now realizing that the bagfile data project Iām proposing might not necessarily be fit for a research project but having something similar to AOSAA book but for Robotics could be very interesting. @Katherine_Scott maybe OpenRobotics could lead such effort? Iād be up for writing a chapter on some platform if this was to become a thing.
You are welcome! And thanks for the clarification on the external packages.
In empirical software engineering there are many ways in which this bias may be controlled. For example, here is a paper about the doās and dontās related to the treatment of third-party GitHub repositories for scientific purposes.
This is extremely interesting for me (and for my research group) and we are definitely willing to help and put effort in this!
This is probably in the paper and I just missed it, but do you have any recommendations on the best way to document the system architecture? The paper says:
ā¦ documented architecture (i.e. the system is described in terms of nodes, services, topics, and their configuration).
Are these best described in a single README file or as ROS node graphs with text descriptions or do you have some other example of great architecture documentation that the rest of us can follow?
Is there one particularly excellent architecture that could be used as a starting point for most other robot architectures? Perhaps something like the Niryo One five layers architecture of external communication, command, motion planning, control, and hardware? Or are most robots unique enough to require independently designed architectures? Thanks!
Thanks Peter, here are my thoughts on your points.
My experience with this study is that people document the architecture of their systems in a very heterogeneous way, ranging from raw node communication graphs, to box-and-line diagrams done in Powerpoint, etc. Actually, in the dataset I collected a direct link to each Architecture documentation, you can refer to the āSA documentationā column here and explore.
About recommendations, I think that the best architectures (in terms of documentation) are the ones combining a graphical representation (of both nodes, topics, and pub/subs) and a textual description of the main responsibilities of each node. Personally, I also like those in which you have a high-level architecture description at the beginning and then details are provided in a top-down fashion, such as in MoveIt.
This is difficult to say. I think it is doable to have some kind of reference architecture for robots belonging to specific domains (e.g., delivery robots, or industrial arms), but it really depends on the requirements and regulations of each domain. If we want to be generic and cover all application domains, maybe the Niryo One layered architecture is a good candidate, but I doubt it will be able to cover all possible needs. Funnily enough, in these weeks I am working with a group of researchers between Italy and Sweden on a generic platform for (1) specifying reference architectures of arbitrary application domains (e.g., delivery robots) and (2) automatically checking the compliance of specific architectures against the reference architecture of their domain. I will keep you posted about it, if you are interested
The MoveIt documentation looks fantastic. I imagine it takes a lot of manual work to make it look that nice. Do you think it is possible to generate system architecture documentation automatically, even if it does not look quite that good, in a standardized way that makes it easy to compare architectures and subsystem compatibility? Maybe something like rqt_graph. I do not know how much could be determined statically and how much would need runtime information. Or does proper documentation require high level information that you cannot get from a set of packages alone? It seems like if you have enough information to check the compliance of an architecture then you would have enough information to automatically create documentation for it.
Maybe something like rqt_graph. I do not know how much could be determined statically and how much would need runtime information.
Here goes another shameless plug.
HAROS is able to extract the computation graph statically - to some extent.
An automatic documentation generator is something I have been meaning to do for a while, but never got the opportunity. I might have some time in a couple of months.
And yes, it could be fantastic (also for us, researchers) to have a standard notation for representing the architecture of robotics systems and to be able to compare architectures across systems, types of robots, and application domains. This would open up for large-scale studies where we could extract even more details information from the system, even more than architecting guidelines!
HAROS looks really interesting thanks! Do you have plans to extend it to ROS 2? Could it work with just vcstool yaml files or does it need more information in its configuration files?
Not right now, although a few community members have contributed in that regard.
In a few months I might get into another project which, likely, will require ROS 2 support at some point.
It does need its own YAML files with a little extra information, unfortunately. That is also something I want to address whenever I have the time to start working on v4.0.
Btw, thanks for sharing that PR with the Node IDL! I had no clue it was a thing, and it might be useful for HAROS, since I already have a similar mini language going on.
Please do review that PR, weāre developing the code behind it now. It sounds like we have similar interests, and Iād love to make sure your use-case is empowered.
