Dear All,
Due the rdiscussion in this topic, this article will benchmark the difference between Auto(Bot)ware spherical grid map clustering, and Autoware euclidean clustering. starting with the setup and configuration the two algorithms, KPIs, results and conclusions.
and sorry for taking sometime, I fine tuned both algorithms because I believe being honest is more important than being right.
Simulation test rig setup:
test rig is an array of cars setup in a 5x5 grid each grid with a distance of 10 meters between each cone, and the robot is in the middle of it
as shown in the below figure.
Sensor model:
we will use two sensors first is 16 layer velodyne sensor, second is 64 layer velodyne sensor, to measure the change of point cloud density on the detection algorithms.
Autobotware setup:
clone the repo and “catkin build”
will set two sets of configurations:
- “use_morphological_filter” = false.
- “ground_segmentation_threshold” = 0.1
Autoware setup:
Follow the setup guide and build the lidar_euclidean_cluster_detect package.
will set two sets of configurations:
- “remove_ground” = true.
- “keep_lanes” = false.
- “clip_min_height” = -5.
- “clip_max_height” = 5.
- “output_frame” = 3.
- “downsample_cloud” = true.
KPIs:
the KPI that i think fit the most is range of detection, as both are detection algorithms, so simply how far can each algorithm can detect, and how the frequency will be.
Experiment1:
-
velodyne16-SGM algorithm with ground segmentation threshold of 0.1
result:
effective detection range around 15.9m, detection frequency beween 137 and 150 hz
-
velodyne16-euclidean algorithm with downsample = true
result:
effective detection range 25.5m, detection frequency between 74 and 90 hz
Experiment2:
-
velodyne64-SGM algorithm with ground segmentation threshold of 0.1
result:
effective detection range around 60.4m, detection frequency beween 22 and 33 hz
-
velodyne64-euclidean algorithm with downsample = true
result:
effective detection range around 63.2m, detection frequency beween 12 and 15 hz
Conclusion:
- euclidean clustering has better detection range than SGM clustering by around 60% in velodyne16(less dense), and around 4% in velodyne64( denser point cloud).
- SGM is faster than euclidean clustering in velodyne16 by around 66%~84%, and in velodyne64 by around 83%~102%.
- SGM is faster, losless, less thresholds/parameters, euclidean clustering can deal better with less points, but it is a lossy data clustering.
- SGM requires V2 implementation where it can deal with far clusters, and we have added it to our action plan.
- SGM provides freespace euclidean clustering doesn’t, euclidean clustering provides pose estimation, SGM doesn’t.
which approach to choose:
choosing which approach to use based on your usecase:
- if you have a high dense LiDAR sensor, then 4% difference in the detection range by using SGM clustering is acceptable in exchange you will have more time to do mapping/planning/tracking/controlling within the same perception time stamp(56msec~69ms) on the other side euclidean clustering will only offer around 20msec~37msec, so it is a designer choice.
- if you will use the detection output in other process then lossy point cloud output will bound you either with post processing else if you are just using it for detection as an input for tracking, then it is good enough I think.
- if you are working with less dense LiDAR, then Euclidean clustering will be good enough for your case.
as in fact you may find someone thinking the same way you are thinking but in the other side of the world, here is another SGM clustering, the implementation is different but the same idea.
Euclidean clustering in autoware.
and finally, all opinions and feedback are welcomed, please follow the community rules.
Thanks,
Khalid elmadawi