Towards a ROSdoc integation testing tutorial

Quality Assurance
, , ,
1

Hi all!
As-is, the ROS doc site has tutorials on unit testing but none yet on integration testing, despite the latter being a key part of software development and ROS also having the tools already available. As discussed here on GitHub, I’d like to contribute a tutorial based on this recent blog article I wrote.

Before working the blog article out to a ROSdoc tutorial, I’d appreciate feedback on it, especially whether I’m missing anything important or am covering something incorrectly.

Other questions:

  • What is the current purpose of the ros_testing repo with the packages ros2test and ros_testing? Should it be covered in the ROSdoc integration tutorial? This repository is maintained, but not included in the desktop distributions and is close to inactive. It offers the add_ros_test CMake macro, does it offer any benefits over the four-line macro add_ros_isolated_launch_test defined here? @wjwwood @hidmic

  • The current launch_testing package is based on the Python unittest framework. There’s an effort (launch_pytest package) to support pytest also for integration testing. Should we cover both in the tutorial? @wjwwood @adityapande

  • How to visualize the test output? In my blog post, I recommended the tool xunit-viewer, which worked well and offers both terminal and web output. Are there other relevant tools to achieve similar results (i.e. a clear overview of which tests succeeded and which failed)?

  • Is there anything you’d really like to see in the tutorial?

All feedback welcome!

Related threads:

3 Likes
2

Update: wrote a draft of the tutorial and made a PR, feel free to share your feedback.

1 Like
3

Awesome of you to put this together. The ROS domain ID approach to conflicting tests is really great! Currently, we just run our tests with a sequential executor, but it’s SLOW. Do you know how the approach works when you have more than 255 tests?