I had a thought that I had on the back of my mind for a while and I would like to share it with the community.
We know that PointClouds messages can be quite large.
The image transport allow us to send compressed images. This has great benefits when we are connecting remotely to a robot over Wifi or when recording with a rosbag.
But we miss a similar functionality iwith even large messages, i.e. poit clouds.
I wonder if anyone has addressed this issue already or if we can propose to have an “official” solution to this problem.
In my mind, a potential simplistic solution would be to:
Add this message to drivers, such as the Velodyne or RealSense-ROS ones. Compression and publication can be skipped in no one subscribe to the compressed topic.
Create an official C++ and Python library to convert the compressed type to a PointCloud2 message or, better, a pcl::PointCloud.
Add this new compressed cloud to RViz
I would personally use a very fast compression algorithm such as LZ4 (we don’t want to add too much latency).
Do you have any data on the compression efficiency for point clouds?
Considering that they are usually highly unstructured data, I’m not sure if off the shelf compression algorithms perform well.
I haven’t tried it, though.
In general, I think compressing point clouds is a good idea.
Maybe even with a lossy option for tasks that do not need accurate data such as visualization.
Pointcloud compression is an active field of research, and there are quite a few algorithms available (at least in papers). PCL already supports some aspects: Point Cloud Compression.
Up till Groovy the object_manipulation stack even contained a compressed_pointcloud_transport package: compressed_pointcloud_transport.
The ROS Answers Q&A has a comment that links to paplhjak/point_cloud_transport which seems to be a package that implements a nr of the things you suggest @facontidavide. Might be worth a look.
I hacked a solution one year ago and my experience is that LZ4 is so fast that the time wasted to do compression and decompression is by FAR lower than the time wasted to transmit a large message over Wifi.
Rephrasing, not only we reduced the bandwidth used, but the latency was actually better.
We also made a solution for this last year.
Although it’s not entirely lossless, one can change voxel size to individual needs. In our lab, we run the solution on six jetson TX2 nodes that map a space of 10x10x4m in 4cm voxels @20Hz. We get 40.5 in compression ratio and use the point-count per voxel to calculate an intensity value.
You can read about it here: https://www.mdpi.com/1424-8220/19/3/636 (free) and try the ROS-package here: https://github.com/SFI-Mechatronics/wp3_compressor/tree/v1.0
Just to mention a lossy solution to this issue we came up with a few years ago.
While developing communication solutions for the NASA Space Robotics Challenge and its limited bandwidth (team Olympus), we investigated the possibility to convert point clouds to range images and back-- PCL <----> OpenCV. The point cloud transport would then take the shape of a usual image. If I recall correctly, this image could be further compressed as a sensor_msgs/CompressedImage.
Given that we were converting sparse point clouds from sensor reading -as opposed to dense object mesh- this solution worked really great in terms of compression ratio and reconstruction accuracy (not so great in term of CPU consumption ^^)
The package is not in good shape but one might be able to scavenge some code from it: pointcloud_to_rangeimage.
Maybe @v-lopez could provide some more info, possibly numbers?
I’d also suggest working on the base of paplhjak/point_cloud_transport . This package has been written by a student of Czech Technical University a few months ago and the goal was exactly to mimick the behavior of image_transport. It implements the very same plugin behavior.
What is already implemented is the lossy compression with draco library.
If you want some lossless solution, it should be pretty easy to implement it as a plugin into this framework.
As for support in RViz - there is none yet, but you can launch a node similar to image republisher (also implemented in the package) which converts the compressed pointclouds to raw PointCloud2.
I don’t have the exact numbers of the original pointcloud size. But by transporting a 26KB depth image and a 85KB rgb image we were able to reconstruct the original pointcloud.
As a matter of fact, I wrote something similar that can turn a PCL into a depth image. Its original purpose was to merge depth images from different perspectives into one as it would be seen from an arbitrary perspective around the point cloud. Worked pretty well, albeit rather slow and costly wrt. calculation time.