Announcing DataTamer beta: a fearless logging library

PickNik and I are excited to introduce DataTamer, an open-source C++ library designed for ROS users to seamlessly record hundreds or even thousands of variables within their applications, even at frequencies as high as 1 KHz.

We all know that debugging robotic applications can be challenging and, sometimes, frustrating. Robotic software is typically distributed, and analyzing the entire system requires considering how different components interact.

Combining logging with data visualization is the most effective approach for identifying software issues; if you’re a ROS developer, you likely already use this method with tools like rosbags in conjunction with RViz, PlotJuggler, and Foxglove.

DataTamer’s primary goal is to significantly simplify the data recording process, especially within control loops. The library is engineered to add minimal CPU overhead and latency.

In our preliminary benchmarks, we observed that taking a ‘snapshot’ of 1000 variables adds a delay in the ‘hot loop’ as low as 10 microseconds. Importantly, this latency is deterministic, making DataTamer well-suited for real-time applications.

While it’s theoretically possible to implement a similar solution using a custom ROS message with multiple fields, this often introduces significant friction, leading developers to resort to the infamous ‘debug by printf’ method.

DataTamer features a versatile multi-sink architecture, enabling users to effortlessly integrate their custom ‘backend.’ A specialized sink may store the data in a timeseries database or publish it through alternative inter-process communication middleware, different from ROS/DDS.

To start visualizing the data recorded with DataTamer, download the latest PlotJuggler (version 3.8.3)

While DataTamer is still in beta, its API is mostly stable. Give it a try, and share your thoughts with us!

16 Likes

It was noted that this is not the best option for asynchronous events. Is this because typically you take snapshots periodically and you might miss asynchronous transitions if the transition happens between snapshots? Is there any thought of supporting capture of asynchronous events in the future as this might be useful?

Regardless, this looks like an awesome tool and I can’t wait to start using it.

Hi,

we don’t record events, we do sampling.

I.e. at a given time, we take a snapshot of the value of ALL the registered variables.

Those values may change 1 million times between snapshots, and we would never know :smile:
There is no plan to change that, for the time being, but time will tell.

A simple workaround would be to have an entire channel for a single event. It is not a big overhead after all :wink:

2 Likes

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.