Depthai ROS2 driver

ROS Projects
1

Hey there, I’ve been working both in my free time and in work on a driver for OAK cameras from Luxonis both as a tool and as an exercise for developing a device driver from scratch, here are some of the newest features:

  • Much more parameters for tuning the device, for both RGB as well as Depth streams,
  • In it’s basic form it can act as a regular RGBD camera such as Kinect or Realsense,
  • You can set some of the parameters such as rgb focus and exposure in real time, for example by using rqt,
  • It’s written as a ROS2 component so in theory it should be more performant,
  • Multicamera support is pretty easy, I’ve tested it with 3 cameras running in parallel (though if you use max resolution and all streams the PC might choke a little bit),
  • Mobilenet and Segmentation examples are provided (in a very basic form)
  • Example visual slam launches are provided for rtabmap and stella_slam,
  • Some example Dockerfiles and build scripts are provided for building it on RPi and PC (Nvidia Jetson dockerfile probably coming soon)
  • Currently it supports ROS2 Galactic, although it should also run fine on Humble

It’s still in active development, so I’m happy to receive PRs to improve this project :slight_smile:

Link:

10 Likes
2

Hi!

Thank you for uploading your work, it seems very promising! I have tested the depthai-ros driver of the Humble release and works fine, but I didn’t find this interesting points to tf code.

Thanks again @Serafadam!

3

Nice job! I came across your driver a few days ago, after I had just finished recording my recent video that included using an OAK-D Lite with ROS (How to get your robot to see in 3D! (Depth Cameras in ROS) - YouTube).

I had a quick go of your one (in case it was worth re-filming) and from brief testing I found yours produced better results and was a little easier to work with than the official one, but also found it to run much slower, especially on the older PC I use for filming, so I left things as-is.

I’m hoping to do more projects with it in future so I’m keen to follow this, and potentially test/improve it based on my needs.

One quick question (and I haven’t made much effort to explore this for myself), are the topics published from your driver rectified? RGB, left, right, and depth?

It would be great to have a demo that performs RGB depth registration.

Thanks again!

4

Hey,
The performance issues might be due to set USB speed, currently it is set to max available speed, but I plan to add USB2 support parameter in near future. When it comes to rectification, I assume they are, I’ve had some good results with Rtabmap, currently the parameter to align depth is set to true (I’m also scaling up the depth image to better match the rgb one). I’ll also add a launch file for depth registration soon.

1 Like
5

Yeah ok, and cool, I haven’t got around to trying Rtabmap yet, it’s on my to-do list.

Thanks again - as I said, I’m looking forward to using this in in a couple of projects at some point!

1 Like
6

Hi, Ive been using the ros driver with the oak-d shortrange and it works very well, thank you. I was wondering, does the driver support the oak-ffc dev boards? OAK-FFC 3P – Luxonis
Im looking into pairing it up with a specific arducam product that is supposed to be compatible with the ffc dev board : https://www.arducam.com/product/arducam-20mp-imx283-camera-module-with-m12-mount-lens-and-adapter-board-for-depthai/
What configurations must be done to enable the usage of the ros-driver with the setup Im proposing?

7

Hi, IMX283 will be supported with new release, though if you want you can enable support now just by building driver from source & modifying this variable. Regarding FFC, it should work, but only on the 4-lane ports, on FFC-3P the center port (CAM_A). Resolutions supported are 4K, and if a different one is specified it defaults to 5312*3692 (almost full res, width slightly cropped because of ISP limitation).