Tooling Proposal rosbag2 leveldb plugin

Next Generation ROS
, ,
1

we’ve been considering leveldb plugin support for rosbag2. any comments and feedback will be good :smile:

Problems

  • especially on edge devices, we need light-weight implementation.
  • the faster, the better. (most likely throughput for rosbag2 use cases)

Conclusion

  • leveldb is light-weight implementation.
  • leveldb can provide better performance than sqlite3 on read.
  • leveldb with writer buffer, it can have better write performance than sqlite3.

Design Comparison

Category leveldb sqlite Remark
license BSD-3-Clause License Public Domain
architecture in-process library in-process library
storage model persist (cache available) persist and in-memery support leveldb cache
sql Non-SQL SQL Full
model Key-Value Store Relational DBMS
multi-thread support support
ACID-compliance ACID-compliance with WriteBatch ACID-compliance Also leveldb Synchronous
compression snappy Not supported
utilities Not available CLI Available sqlite3 command is available, leveldbutil needs to compile
footprint 371KB(libleveldb.so.1.20) 1.1M(/usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6) Dynamic shared library code size

Benchmark

Public Performance Comparison

you can refer the the following result from Google. (it is old but worth to take a look)

Internal Performance Test Result

we also checked the performance by our own, compared to sqlite3.

  • Raspberry Pi 4 Model B Rev 1.1
    Broadcom BCM2711, Quad core Cortex-A72 (ARM v8) 64-bit SoC @ 1.5GHz, memory 3.7GiB
  • Ubuntu 20.04.1 and leveldb (commit: 1454924)
  • Performance Test Program
  • Performance Result Comparison

Sequential Reads

MB/s 100 10K 100K 1M 2M 4M 8M
leveldb 175.5 492.4 563.0 501.8 578.1 562.6 45.8
sqlite 115.9 361.3 441.1 328.8 345.2 135.9 41.8

Random Reads

ops/sec 100 10K 100K 1M 2M 4M 8M
leveldb 53333 13527 1643 176 129 91 5
sqlite 54528 3785 470 40 21 5 1

Sequential Writes

MB/s 100 10K 100K 1M 2M 4M 8M
leveldb 8.7 16.9 15.2 8.4 5.5 8.9 13.5
sqlite 2.3 26.7 24.1 3.5 4.3 6.8 9.4

Random Writes

MB/s 100 10K 100K 1M 2M 4M 8M
leveldb 5.4 4.0 2.8 1.4 1.7 2.9 4.3
sqlite 1.1 13.0 16.4 3.4 4.8 12.1 9.6

According to above result, we can see

The read performance (sequential/random read) of leveldb is much better than sqlite3.
For some write cases, sqlite3 has better performance than leveldb.

  • Performance Result Comparison with DB cache buffer

Sequential Writes

MB/s 10K 100K 1M
leveldb (writer buffer: 100M) 25.9 19.4 21.9
sqlite3 (cache: 100M) 22.3 11.7 13.3

Random Writes

MB/s 10K 100K 1M
leveldb (writer buffer: 100M) 28.1 24 10.2
sqlite3 (cache: 100M) 14 16.2 15.1

According to above result, we can find

leveldb improves performance with writer buffer better than sqlite3 obviously.

1 Like
2

@emersonknapp @dabonnie

I know that you are facilitating ROS2 Tooling WG, appreciate for the effort. Please let me know if it is worth to share this information in WG. we are willing to do that and would like more feedback from community. (hopefully different time slot for quick sync, it is really hard to participate in Tooling WG from Japan local time…:sleepy: or maybe i am gonna die trying :dizzy_face:)

3

Here is a performance improvement result with rosbag2_performance

Environment

  • Raspberry Pi 4 Model B Rev 1.1, Broadcom BCM2711, Quad core Cortex-A72 (ARM v8) 64-bit SoC @ 1.5GHz, memory 3.7GiB
  • Ubuntu 20.04
  • ROS2 rolling (build from source codes).
  • rosbag2 (commit: 67d193ed054e) with sqlite3 storage plugin
  • leveldb storage plugin (new developed)

Performance Comparison for Recorded Percentage

(*)…cache size is how many messages can be kept in rosbag2 memory. (This is not database write buffer or cache)

  • Instance :1, frequency: 100, size: 1000000

image

  • Instance :10, frequency: 100, size: 100000

image

  • Instance :100, frequency: 100, size: 10000

image

1 Like
4

following PR is open,

3 Likes
5

This is great. Thank you so much for the contribution!

6

I think it’s a great idea to have alternative storage options available for rosbag2!

I would be happy to schedule a breakout meeting to discuss this plugin, how about 2300 UTC? That should be the morning in Japan time, and at least possible to join for Europe. If that’s ok, we can select a day for it.

7

that works for us :+1: appreciate it!! what about the week after next week? 16th.Nov to 20th.Nov or later? I think that in next week everyone is getting busy for ROS world 2020.

8

That sounds perfect - I created a post about it here Tooling WG upcoming breakouts!, please choose the day that is best for you and we can go from there.

1 Like
9

feedback from ROS2 Tooling WG, it would be nice to check CPU and memory consumption during benchmark at runtime.