Over the last 4 years, I’ve been working on a robotics project (autonomous racecars!) with quite a few people (roughly around 30 people). We used ROS 2 for all of our software systems and generally love what ROS 2 provides and all the tooling surrounding it (so thank you very much for that!).
One of the problems that we started having as we grew was data management and wondered how others were solving problems that we were encountering.
- How do people record data? Do you use
ros2 bag record, or some other mechanism? We had trouble with dropped frames, especially for camera and lidar messages. - How do you get the data off your robot? We went through a period of using USB sticks
- How do you search for and manage all the data you collect? Do you have a tool?
We were also curious about testing with collected data
4. How do you manage versioning of the stack and using bags to test different versions of your packages? For example, we didn’t have a good way of updating bags with more recent perception outputs for the planning team to take advantage of.
5. How do people use the data collected to create tests / do post-processing activities such using data to update ML models? For example, we wanted a system to automatically evaluate how well our robot did from the bag data that we collected.
I came across some platforms like Foxglove, Sensorsurf, Viam and MARV robotics. Does anyone have experience with any of these and what they liked or didn’t like?
Would love to hear your thoughts!