Rosbag dataset hub

General
,
1

Hi all,

Is there a well-known repository of rosbag datasets that is publicly available? I’m curious if this is something people would be interested in contributing to, if it doesn’t exist today? Something like a “dataset hub for robots”. I’ve found some links (added below for completeness), but nothing large enough to call it a repository of diverse robot logs. I’d love to put together a set of logs across domains/applications to build an evaluation benchmark of sorts.

Some related topics/links I was able to dig-up:

5 Likes
2

I’ve talked about building a list previously. I might be able to put sometime on this later this year.

I did grep through weekly udpates and found these.

5 Likes
3

Thanks for the links @Katherine_Scott.

For the folks in industry: I’m curious how you manage your collection of rosbags (presumably on S3, GFS). I’ve heard of in-house solutions for this, are there open-source resources for managing and exploring rosbags in a DB?

2 Likes
4

SWRI has a nice tool for this:

5 Likes
5

Thanks for pointing out that tool, @Timple. We built it specifically to have an on-prem bag database. We just dump bags in a directory on an internal shared drive and let the database parse them all out for us. (If you love your hierarchies, it maintains those, too, so you can view the tree in the web interface if you want.)

1 Like
6

This looks like a very nice tool. Thanks for sharing

7

Thanks @Timple for the link. I noticed that link a while back; @jatowler how does this tool scale with more rosbags (>10K) / larger bags (>300GB / rosbag) in your internal shared drive? Also, was this tool meant more for web visualization or for general metadata cataloging and bag management?

8

@avatar042 currently we have roughly 12k bags indexed in our bag database instance without any adverse effects. Most of these bags are under 10 GB. For large bags over 20 GB, you will need have speedy storage for timely processing.

Our current setup stores about 20TB of bags on a (dated) Synology NAS, mounted as an NFS share over a 10 gig connection to the VM running the bag database. The limit on individual bag sizes right now is the NAS, which is on my wishlist of hardware upgrades.

Overall, this tool is focused more towards “general metadata cataloging and bag management” over “web visualization.”

1 Like