How do you annotate BAGs for machine learning?

Hello everybody,

I am currently working on my diploma thesis where my goal is to create a web based tool for replaying and annotating BAG files. Essentially something similar to Webviz but with annotation support and colaboration. To make the application usable in real world, I need to collect information from people that actually annotate data from ROS BAGs, to help my decision making in develepment of this tool.

So my questions to you are:
How do you currently annotate data from ROS BAGs? What software do you use?
What kind of data do you annotate and what type of annotation do you use?
What in what format do you export annotations?
What key features would the tool need to have for it to be useful to you?

Also if you have any suggestions regarding this tool, feel free to share them. I would love to hear them!

Any information is welcome.

Thank you

1 Like

You mention your tool is specifically to annotate bag files. I’m not sure how popular that actually is/how it works, but what I’ve seen is you normally export a bag file of a camera stream (rgb image) into jpg/png and then label the images using normal ML labelling software eg. labelme, labelbox, superannotate to then train a ML model.

I am not an expert, but specifically for the image processing pipeline, this is exactly my experience as well.

Q: How do you currently annotate data from ROS BAGs? What software do you use?
A: Foxglove

Annotating data from ROS (Robot Operating System) BAGs involves extracting relevant information from these files, often containing sensor data like images, lidar scans, IMU (Inertial Measurement Unit) data

This is an interesting use case which many of the annotation tools in the market doesn’t handle. There are several tools capable of annotating data from ROS BAGs. Some popular choices include labeling tools like LabelImg, Supervisely, Labellerr, or custom scripts developed using Python with libraries such as OpenCV or ROS libraries themselves.

Annotations could encompass various sensor data such as images (for object detection, segmentation), lidar point clouds (for object localization), IMU data (for trajectory or motion annotations), etc., depending on the robotic application

Annotations are typically exported in standardized formats compatible with other tools or frameworks. Common formats include XML, JSON, YAML, CSV, or specialized formats depending on the tool used and the application requirements.

key features the tool need to have:-

  • ROS BAG Compatibility: Seamless integration or direct support for reading ROS BAG files to visualize and annotate the sensor data within these files.
  • Multi-Sensor Annotation Support: Ability to annotate various types of sensor data present in the ROS BAG files, accommodating annotations for images, lidar scans, IMU data, etc.
  • Flexible Annotation Types: Support for multiple annotation types (bounding boxes, polygons, semantic segmentation masks) depending on the data and annotation requirements.
  • Export/Import Flexibility: Capabilities to export annotations in common formats for compatibility with other tools or systems and import existing annotations.
  • Ease of Use and Efficiency: Intuitive user interface, efficient annotation workflow, collaboration features for multiple annotators, and automation capabilities to speed up the annotation process.