Introducing Foxglove integration with rosboard for real-time visualizations

kiwibot-ai · 2024-06-28T17:15:59.374Z

Hi everyone!

At Kiwibot, we develop sidewalk delivery robots. As we continue to deploy an increasing number of Kiwibots worldwide, we realized the necessity for a robust remote debugging tool. While rosbags can be great for this purpose, they tend to become huge files that are hard to distribute and serve only for offline analysis. Often, it is useful to be able to debug the system online and in real-time. Such a tool would allow us to stream topics over a limited bandwidth LTE network connection between the robots and our remote personal computers to visualize and diagnose any problems easily.

Previously, we solved this by introducing the rosboard client. Nonetheless, this required our local setup to have all ROS dependencies installed and involved a cumbersome setup. Fortunately, there is Foxglove, a great web-based alternative that allows excellent debugging and visualization of ROS systems, all by just needing a browser. They offer streaming of topics by running either rosbridge_suite or foxglove_websocket_protocol on the robots’ side. This was our first option when testing, but it consumed a lot of bandwidth and accumulated significant latency over time, which was unsuitable for our case.

As discussed in our previous post, although rosboard was not originally designed for streaming over the Internet, we have seen it overcome these bandwidth and latency problems by relying on a set of subsampling and compression techniques. It requires few dependencies and is easily installable.

The Kiwibot team wanted to have the excellent visualization capabilities of Foxglove while keeping the data transfer advantages of rosboard’s networking protocol. For that reason, we are releasing our Foxglove fork and rosboard fork that were modified to:

Add an additional connection type to Foxglove to allow connection to a rosboard server.
Make small modifications on rosboard’s side to communicate with the Foxglove client.
Add a button to rosboard’s original interface to redirect to Foxglove’s for instant connection.

You can play with the foxglove version using our docker image from dockerhub and our mentioned rosboard fork. Caveat: at the moment we only support ROS2 systems.

foxglove_rosboard-ezgif.com-optimize

Example of using ngrok to make the robot available to the internet and then connect to the robot using Foxglove to visualize cameras, costmaps, pointcloud and navigation information

Thanks to this new tool, we can simultaneously stream several costmaps, laserscans, RGB images, paths, and transforms in near real-time over a bandwidth-limited LTE cellular network and visualize them using Foxglove without any issues.

With ngrok, you can effortlessly make the server on your robot’s local network accessible over the internet as shown in the demo video.

Credits to @franyol and @charlielito who made this integration possible. Credits to @dheera for developing the rosboard server and all the foxglove team for the amazing visualization tool.

We are thrilled to receive feedback from the community!

Happy debugging!

vinnnyr · 2024-06-28T18:54:12.581Z

Hi @kiwibot-ai ,

This looks really cool! Can you briefly describe the compression techniques used to make this bandwidth efficient? Specifically, are you doing any special compression techniques on point clouds?

Thank you in advance!

chfritz · 2024-06-28T19:54:21.642Z

What you did there is extremely similar to what I did in the Foxglove-webrtc connector. Like yours, it is also a fork of Foxglove that adds a new connection type, but instead of using websocket (over TCP) it uses webrtc. The latter, of course, was specifically designed for teleconferencing, i.e., live video-streaming into the browser. It hence has, unsurprisingly, several advantages over websockets (e.g., UDP, h264 video compression, congestion control, packet loss mitigation, no latency build-up when network is unstable, ability to get past firewalls – no ngrok required). See image below for a comparison on bandwidth efficiency. More details in this blog post 5 Ways to Stream Video from Robots and Why You Should Use WebRTC.

image1920×1238 121 KB

transitiverobotics.com

Foxglove WebRTC | Transitive Robotics

Open your robots live in Foxglove, no matter where they are

edvart · 2024-06-28T20:29:43.860Z

I’m part of the cult of webRTC, all because of chfritz

kiwibot-ai · 2024-07-02T14:27:44.383Z

Sure! The compression techniques are primarily derived from the original repository at GitHub - dheera/rosboard: ROS node that turns your robot into a web server to visualize ROS topics, which applies different compression methods depending on the data type.

For image topics and occupancy grids:

Everything is converted to uint8.
If the image exceeds certain dimensions, it is subsampled.
For image topics, it is sent as a JPG-compressed image. For occupancy grid topics, it is sent as a PNG-compressed image to avoid losing information.

For laser scan and PointCloud2 topics:

The data is converted to uint16 to save space.
For PointCloud topics:
- If there are more than approximately 65k points, it is subsampled to 65k.
- The point clouds are trimmed to only send xyz fields.

kiwibot-ai · 2024-07-02T14:49:33.579Z

Thank you for your response and for sharing information about the Foxglove-webrtc connector!

Our solution is fully open source and free to use, making it accessible for a wider range of applications without the associated costs of paid solutions. We understand the advantages of WebRTC; however, we aimed to create a simpler and more cost-effective solution that can be easily set up and deployed, even in environments where complex WebRTC setups (e.g., TURN, relay servers) might pose additional challenges. By relying on existing open-source tools, we avoided the need to invest much time in implementing a complex WebRTC pipeline.

We appreciate the innovations and capabilities your solution brings to the table and look forward to the community’s feedback and collaboration to enhance the tools available for ROS-based systems.

chfritz · 2024-07-02T15:12:21.319Z

kiwibot-ai:

Our solution is fully open source and free to use

That’s fine, but please keep in mind that your cloud provider will charge you around $0.1/GB of traffic (egress) and your incremental LTE/5G data charges will likely be even higher. So sending 15 MB/s instead of 0.07 MB/s means that you’ll just be paying someone else and quite possibly more.

kiwibot-ai · 2024-07-02T17:11:34.473Z

That’s a valid concern. Nonetheless, that bandwidth you mentioned was for some stress testing we did earlier. In practice, we throttle image topics, and the bandwidth consumption is significantly less. Additionally, this tool is intended for debugging sessions that last just a few minutes, so data usage is not much of a concern.