For 2020-August-18 Real-time Working Group
As I can not find the right place, let me report progress about “4.1 Performance testing (with an emphasis on determinism) in 2020-August-04” here.
In short words, we(ISP) are now surveying what is already done and what should be done.
We are surveying following:
- [g1] performance_test (Apex)
- [g2] iRobot ros2-performance (iRobot)
- [g3] buildfarm_perf_test (OpenRobotics)
[g3] is CI and uses [g1] internally.
They have some similarities and differences as below.
- Each tool measures topic trip-time(pub -> sub)
[g1] checks combination of DDS/RMW
[g2] checks complex topology - Executor:
- [g1] uses SingleThreadedExecutor.
std::this_thread::sleep_until
is used for publishers,
Executor::spin_some
is used for subscribers. - [g2] uses StaticSingleThreadedExecutor.
This usesExecutor::spin
but has an original thread model.
- [g1] uses SingleThreadedExecutor.
- Communication mechanism
- [g1] uses only topic
- [g2] uses topic/service but not action/pamameter.
- Memory usage/Data model
- [g1] allocates unique_ptr at first, and publishs it by reference
- [g2] uses
unique_ptr
orshared_ptr
according to option but make_unique/make_shared for each loop - Which are optimal for real-time? Or Should we use TLFSAllocator and MessageMemoryPoolStrategy?
- and so on
We now preparing a comparison table.
At the same time, we should define conditions and metrics.
Following are described in some literatures:
- conditions
- HW setting
- OS setting(kernel build, kernels threads, kernel parameters, …)
- ROS2 process setting(memory/scheduling as in pendulum, affinity, …)
- system stress
- metrics
- timer precision
- [g1] and [g2] do not measure this
- packet trip-time(1-way, 2-way)
In ROS2, topic/service/action/(parameter) may be the targets.- only topic is common target now
- time of procedures (may be callback in ros2)
- [g1] [g2] has rather simple callback(only takes statistics)
- each values should be worst execuion time (WCET)
- need to measure at least a few hours.
- timer precision