Should Fuzzy testing be moved to a different block containing black-box fuzzing as a sub-component? (Check updated list)
Could you please correct me if I’m wrong about black-box fuzzy testing:
- It would be similar to using hypothesis (python package) to create messages and checking output
- It would have templates for different message-types, not for node-specific requirements. Eg: Range input for float message, 3 different ranges for acceleration, etc.
Would node-specific behavior such as accepting messages in strictly increasing time order be included in black-box testing? This obv uses a small but nonetheless inner knowledge of how a node should work.
IMO, white-box fuzzing might be helpful in checking corner cases like a typical test but more on the lines of model-based testing, ie. developed specially for the package or node, specially for non-infrastructure packages. On the other-hand, black-box fuzzers would be generic and their use would involve minimal (one or a few lines only) involvement by the maintainers.
As for development, I think ROS2 with node-lifetimes would make such testing both simple and a necessity. However, a generic fuzzer on the messages, services (and actions) without lifetimes would be similar for ROS1 and ROS2 unless my assumptions are wrong.
pyros-dev should make the testing more or less similar, right?