Introducing Rosboard Client

[kiwibot-ai]

Hi everyone!

At Kiwibot we build sidewalk delivery robots. While deploying more and more Kiwibots worldwide, we came to need a robust remote debugging tool, which makes it possible to stream topics through a limited bandwidth LTE network connection between robots and our personal computers, with which we can use ROS’ amazing desktop viz tools. Today, we are introducing rosboard_client, a package that bridges two ROS systems and allows you to use rviz to visualize the data in real time coming from a robot located anywhere in the world in real time.

One of the most known tools for internet communication between two machines running ROS is rosbridge_suite. When testing, this was our first option , but it demonstrated consuming a lot of bandwidth and accumulating significant latency over time (some benchmarks showing this are presented in the repo). The library features some compression methods, but they did not provide too much of an improvement. Rosbridge aims to provide all the communication capabilities you would expect from two ROS systems running on the same network (topics, services, action servers), which go beyond what we needed.

On the other hand there is rosboard, which overcomes these bandwidth and latency problems in our use case, by relying on a set of subsampling and compression techniques. Thanks to that, it keeps good messaging rates and latency while less bandwidth is used. However rosboard has a web based UI that shows each message in a different widget, which makes it harder to understand complex navigation data like pointclouds and costmaps. Also, has a limited amount message types that support visualization, and does not allow to interact with 3D data or to change the tf frame in which it is shown.

Rosboard (left) vs rosbridge (right) transmitting a 640x360 RGB image on the same network conditions. visualized on the client side using rviz. Note how rosboard’s image looks more fluid and has a higher FPS count.

The Kiwibot team wanted to have the great visualization capabilities of the ROS desktop toolchain (rviz, mapviz, rqt, etc) while keeping the data transfer advantages of rosboard’s networking protocol. For that reason we developed and opensourced rosboard_client, a package for desktop environments that is comprised by the following:

1. A ROS-independent networking library that connects to a rosboard server. It is able to subscribe to topics published on the server side and to publish on the server topics that are available on the client side (the latter only using our custom fork of rosboard).

2. A ROS GUI similar to rosboard’s web GUI. It allows people to see all the available topics on the server and interactively subscribe/unsubscribe to them. When subscribed to a topic, the received data gets republished as ROS messages on the local machine, allowing people to use the desktop viz toolchain that everyone is already so familiar with.

The rosboard desktop gui (Left) and rviz (right). The GUI shows all the available topics. When a subscription to a costmap is created the data gets republished as a ROS message and can be viewed in rviz.

Thanks to this new tool, we have been able to stream several costmaps, a laserscan, an RGB image, paths and transforms at the same time without problems, at almost real time, on a bandwidth limited LTE cellular network and visualize them using rviz.

Using ngrok, you can easily make the server running on your robot’s local network available to the internet. More instructions are available on the repo.

We benchmarked our client against rosbridge using different compressions for transmitting a 640x360x15fps RGB image from an intel realsense and noticed an important improvement. More details on the test results and the code to run the benchmark by yourself are available on the repo.

Streaming Tool	Avg. Latency [ms]	Avg. Bandwidth Usage [KiB/s]
Rosboard (JPEG compression)	96.38	11010.14
Rosbridge (no compression)	22330.06	12420.32
Rosbridge (CBOR compression)	6035.99	8215.36
Rosbridge (PNG compression)	19096.12	12054.46

Compression for rosbridge is only available using our forks of rosbridge and roslibpy, since this feature was broken for ROS2 on the main repos at the time we did the tests (August 2022).

Credits to @charlielito and @Pepis who developed the networking library, and to @NicolasRochaPacheco who developed the GUI. Credits to @dheera for developing the rosboard server and to Osnabrück University for developing rospy message converter

We are more than happy to receive feedback from the community!

[peci1]

Looks really nice! I guess you don’t plan a ROS 1 version, do you?

[chfritz]

Very nice!

Question re. video streaming: are you sure your units in the streaming comparison table are correct? 11 MB/s is a lot! Using webrtc you can stream video of that resolution at 50KB/s – that’s 220 times less bandwidth! We have such a capability on Transitive if you are interested: https://transitiverobotics.com/caps/@transitive-robotics/webrtc-video.

The other benefit of using webrtc, of course, is that it works across firewalls, e.g., when your robot is on an LTE connection. How do you solve that in rosboard_client? Do you require the user to be on a VPN with the robot or is there a proxy one can deploy to help with firewalls?

[kiwibot-ai]

Unfortunately not at the time, however PRs are welcome. A nice integration could be done with @dheera 's rospy2 which offers a single API and uses the appropriate functions according to the ROS version, making the same code compatible both with ROS1 & ROS2.

[dheera]

Thanks @kiwibot-ai , great to see people building on rosboard! (I know I’ve been a bit MIA recently since I have been travelling and ramping up on a new position, but I haven’t dropped ROSboard completely I will continue development on it as I get more time in the future.)

@chfritz the rosboard server already supports both ROS1 and ROS2, so if rosboard_client is just a client for it that doesn’t interact with ROS directly, then it should work in either ROS version.

I haven’t tested this, but if it doesn’t work as expected we can try to make the appropriate fixes.

[dheera]

I have been pondering adding WebRTC streaming to ROSboard at some point. One option is to use this

GitHub - aiortc/aiortc: WebRTC and ORTC implementation for Python using asyncio

though it does require ffmpeg, opus, libvpx and other things installed on the robot that could end up being a mess if it’s a Jetson- or Pi-based robot. Quite possibly one way is to have jpeg streaming be a graceful fallback instead of outright complaining if the dependencies aren’t found.

Another thought I had for managing the bandwidth was perhaps the bandwidth/lag could actually be measured and resolution/framerate adjusted accordingly. If the user is playing back rosbags on the same computer, I’d even opt to not compress so as to preserve images to the highest quality, whereas if a lag is detected, drop framerate and resolution accordingly. This is a bit hard over TCP-based things like websockets but could be doable.

That said though the initial intention of ROSboard wasn’t to stream stuff over the internet, it was more intended to be an on-site debugging and dev tool. (Hence the support for pointclouds and other things, too)

[kiwibot-ai]

Thanks!

Yeah, those numbers are pretty high because even tough rosboard applies some compression to the images it is applied image-wise, which I guess is much less efficient than the more performant h264 or h265 encoding often performed by webrtc. Its worth noting than even though bandwidth numbers look similar between the different streaming tools, with rosboard we were able to get a stable 15fps video feed while with rosbridge this number dropped significantly (as the gifs show), making the per-frame data usage higher in the second case.

I think the use cases of rosboard are different from the ones of a webrtc system: rosboard is meant to be an easy to use / easy to deploy debugging tool to transmit all kind of topics (not only images) from a robot to your local machine for relatively short periods of time. We decided to use images for the benchmarking because they are relatively data intensive and are easy to generate at a fixed rate, however I think the true potential of rosboard lies on what I mentioned above. In contrast a webrtc system targets mainly images and is much more efficient and robust at transmitting them, however it come at the cost of a more complex (and often not free) setup and the fact that it would be harder to transmit different ROS data types.

The proxy we use to bridge websocket connections across different networks is called ngrok, which can be launched easily on the robot. To be honest I don’t exactly know the specifics on how ngrok handles firewalls in the backend but we have been able to use it without any problems at all.

[kiwibot-ai]

Thanks @dheera !

Your rosboard server proved really useful! There are some small changes we made to it though to better support our use case. We would be happy to submit a PR if you think it may be useful:

1. Changed the way in which qos profiles were obtained for ros2 subscriptions to ensure compatibility and get all the messages from transient_local topics like /tf_static, which is often important to use rviz in a meaningful way.
2. We changed the encoding of occupancy grids from jpeg to grayscale png. Since costmaps in rviz can show really different things if the cost of a tile changes even by a single unit we preferred to use lossless even if it means a little more bandwidth is necessary. This new encoding stills work well when visualizing the costmap on a web browser though.
3. Added a new message and the necessary code to stream topics from the client back to the server.

To reduce the bandwidth usage of data intensive topics, in the future we will likely include a “throttle rate” parameter to limit the frequency with which they can be streamed, most likely people don’t need an image at 15fps just to debug. This seems fairly easy to do and can quickly bring down bandwidth usage. However your idea of further encoding images sounds great, we would be happy to integrate the decoding in the client.

People using rosboard_client may also use our rosboard fork