Compare the running architecture with the designed architecture in CI

Tobias · 2020-02-07T14:35:52.062Z

Hi ROS community,

we are building a quite complex ROS system at StreetScooter.
For our industrialization we want to be sure that the running architecture fits to the designed architecture.

One way to ensure this is to use code generators like this one:
https://www.eclipse.org/papyrus/components/robotics/
This systems engineering approach is used in many industries, but in my view it kills creativity and system understanding by to much abstraction.
Also this papyrus does not seem to work stable.

So I would like to discuss another approach:

Create a GraphML based template for a flowchart that displays nodes, topics and services.
Design your architecture based on this template flowchart.
Create a fingerprint for your robot with tools like: https://github.com/aliasrobotics/aztarna
Create a CI test, that reads data from your design documents and compares it with the fingerprint of the running ROS architecture.

In this way we can keep the consistency between designed and implemented architecture to a certain level.
What do you think about this? Do you see better or other ways to maintain consistency?

afsantos · 2020-02-07T16:19:17.234Z

Hi Tobias,

(Sort of a shameless plug)
In the past few years I have been working on HAROS, a framework for static analysis of ROS systems, as part of my PhD thesis. As of version 3.0, it is able to extract an architectural model of the system by parsing the source code (as if compiling it).

This extraction process is not perfect nor complete, but it should handle many common cases in C++, and I’ve had external help in implementing a Python equivalent (still under evaluation).

It should be possible to use HAROS for the kind of checks that you seek (i.e., “is my implementation compliant with the architecture I want?”), with the advantage that the feedback is returned without running the system.

Of course, coupling this with runtime tools, such as you mention, further boosts your confidence in the implemented product.

Katherine_Scott · 2020-02-07T18:13:58.759Z

@Tobias FYI your first link is currently a 404. I will say my general experience with eclipse isn’t positive, and piling on hard contract requirements is a surefire way to kill developer creativity.

At a high level it would appear that the architecture you propose should work. It will be a bit heavy handed as if you wish to change the system configuration you’ll need to update both your template and your code. An alternative is that you generate a fingerprint and fingerprint delta at each commit/release so you at least have visibility of how things changed, when, and why. This might get you most of what you want without being as heavy handed. I would also suggest that if you undertake this effort you make sure to report your progress / results to the quality working group.

Tobias · 2020-02-10T09:44:58.068Z

@afsantos:
I will dig into HAROS. It was already on my list but not in terms of architecture consistency.

“As of version 3.0, it is able to extract an architectural model of the system by parsing the source code (as if compiling it).”
I did not know HAROS can do that but I will try it out with our ROS1 Stack. I also see a merge for ROS2 support.

@Katherine_Scott:
What you are proposing has already been implemented in our development in some way. We have a tool called “systemstamp” that collects all kind of system data like network config, nodes, OS versions,etc… to create a fingerprint of our robot system. This helps to track changes between the different development teams on the systems. We also want to add this finger print on every issue that is created to identify the system status when a issue occurs. Our ring recorder for bagfiles is also connected to this. The combination of Issue, Issue Bagfiles and system fingerprint enables us to evolve the robot over time between multiple parties. I will join the architecture workshop to give and get feedback on this topics.

Maybe It would be interesting to create a common community driven GraphML template, parsing and system finger print creation. This would make it easier to work within the community on an consistent ROS architecture.

afsantos · 2020-02-10T10:30:51.353Z

@Tobias ROS2 support is still in its early stages, contributed by Max Krichenbauer. It is mostly to identify the new workspaces. I have not implemented the parsers for the ROS2 API yet.

Feel free to PM me if you cannot set it up for some reason (it needs more extensive documentation).

fkromer · 2020-02-10T12:43:03.465Z

There is ArchUnit for Java. As far as I know Java is the only language with a tool that allows to define rules for the architecture and to check if the architecture complies with the rules. Probably there are concepts one could adapt…

Katherine_Scott · 2020-02-10T19:39:03.686Z

Is that an open source tool, or an internal tool?

Tobias · 2020-02-11T14:25:27.501Z

It is just a python internal script interacting with a Gitlab repository representing your system states. It reads a fingerprint from the documentation of robot and describes its system state.
Depending on your system it could be different states you want to track starting with your deployment script, network configuration, os configuration, firmware, xacro, functions stacks and many more. All this can be tagged in a single repository using git submodules, git-repo or rosinstall.

An open source tool for reading such information into a standard way for ROS2 would definitely be a good idea in combination with an architecture design representation.

gbiggs · 2020-02-13T04:59:22.280Z

Tobias:

This systems engineering approach is used in many industries, but in my view it kills creativity and system understanding by to much abstraction.

For what it’s worth, a model-based approach is close to being the expected approach in many industries using software because it introduces the necessary engineering formality that you don’t get in going straight from mind to code. Whether the tools we currently have to do it are good enough is a separate topic that I don’t want to discuss here.

Tobias · 2020-02-13T06:47:38.089Z

I think the model-based approach is just one way to reach the formality you are talking about.
The question is how you can iterate through a development model like the v-model fast, consistent and still “agile”?
This discussion should definitely not go more into detail about this topic, but I thing extended continuous integration practices could help here a lot.

Let’s keep to the question “How to check the running against designed architecture by CI ?” that is also a formality for many developers but I thing it can help a lot to create a common understanding for everyone involved in the product.