Hello ROS Community,
I work at a place that has a bunch of different robotics projects with a variety of purposes developing in a quite agile strategy. Recently we’ve realized that we are having a lot of issues on robot testing (which are usually not connected to internet but only a local robot network), as multiple people are working on different problems on the same robot (occasionally using the same code base). A vast majority of problems that we had to solve was someone changing a number of settings to test their own code and when the next person tries to run the robot, there is a problem that arises from these settings causing the robot no to run as expected. That’s why we are going for a complete change in our QA and testing system. I am already aware of current QA solutions related with software, though I am particularly interested in testing and QA processes on ROS, real robots and simulation.
I’ve already gone through ROS, ROS 2 and ROS-Industrial QA documents but I would appreciate more input on how you lead your QA and testing efforts. I am dropping some specific questions below but I will be more than happy to hear general advices as well.
- How does your git branching look like? Do you have different branches or different repos for simulation and real robots? How does your git flow look like? We do write ros packages in different repositories and add them as submodules to the robot project in a different repository, where and how do you keep your robot code?
- How do you decide if a particular code or ros package is stable and ready to be released, and what kind of testing you do on robots to determine stability?
- How much simulation testing do you perform before robot testing?
- Is it a good idea to do all the testing on robot in dockers?
- How do you keep your robots free from unstable code?
- What kind of processes are running with CI? (unittest, auto docker builds, compiling in different environments, format checking etc.)
- In which platform and how do you write the documentation and how detailed is it? (We are usually writing a page of README and a few pages of Latex report on theory per ros package)
- How much unit testing are you doing?
- How much integration tests are you doing?
Thanks a lot!