Intel Cancelling its Realsense business: Alternatives?

General
1

Today it was let out that Intel is closing up shop in supporting robotics sensing with the Realsense camera.

Sources

While over the years, I’ve given alot of flack to the Realsense unit for not really supporting their drivers or SDK as much as I’d like. But it was still a great piece of hardware if you were willing to put in a bit of professional leg work and really the basis of so many ROS robotics companies. The pricepoint was awesome and for applications in direct sunlight (outside, in buildings with large windows) or when you needed to work with, lets say, 5 of them in a single computer, it was really the only decent option that I found to work well. While I’m a huge personal fan of the Orbecc depth cameras, they themselves suffer from some issues that made it less product-ready for the types of robotics products I was working on.

So I wanted to start this thread, as I’m sure this will be coming up alot in the next few weeks / months about potential alternatives to the Realsense cameras once they’re really just done and gone. If you’ve had experience or know of some new product-ready RGB-D cameras out there, this would be a great place to comment and let us know about your experiences.

My personal list of things to think about

  • Orbecc
  • pmd/IFM (albeit, no color)
  • Mynt
  • ZED (if you have a GPU and GPU power to spare)
32 Likes
2

The open.vision.computer is another open source alternative though (currently) there are no manufacturers / distributors. Anyone interested? :slight_smile:

This repo contains hardware, firmware, and software for the Open Vision Computer (OVC), with active development happening on OVC5

4 Likes
3

Such unfortunate news. I wonder if they tried to sell that part of the business to someone else before shelving it. I have to assume someone would of been interested in buying it if the price was right.

4

We make an attempt to maintain a camera survey: 3D Camera Survey — ROS-Industrial

However, we really try to include cameras that we personally, or trusted partners, have actually used in an application. So it is not exhaustive, however these units work well.

We have had the experience to benchmark in an application a number of sensors, and the recent RealSense 455 did really well. The nicest unit put out by that team. Framos recently announced a 455 PoE unit, but not sure the future of Framos building around RealSense either. It was not without problems, but you could see Intel selling off to enable others to continue producing the hardware via license.

We will see…

8 Likes
5

yeah the ones I could think besides the Realsense was

I haven’t use the Mynt but I’ve used the structure quite a bit for XR and scanning applications both for mobile and on desktop, they have great software SDKs it also is the main sensor in Misty if you are wondering if it’s been used in robotics applications.

if anything perhaps the folks at Luxonis working on the OpenCVAI OakD camera can be another option as well

3 Likes
6

I’m happy to discuss alternatives, please schedule here: Calendly - Erin Rapacki

I used to work in solid-state LiDAR while trying to sell a RealSense replacement so I became rather familiar with the alternatives, but it depends on your specific application such as: indoors/outdoors, resolution, RGBD or just Depth, IEEE Time Sync, interference mitigation, IP67, dust/fog expectations, durability, range, etc…

I now work on Unity Simulation for roboticists, including 3D camera modeling (noise, optical distortion, etc) for training and testing Spatial AI applications in robotics & video. Check out ZeroSim for ROS (Open Source) ZeroSim - Robotic Simulation Services

My shortlist includes:

I’m usually not a fan of stereo, has too many depth-of-field issues for applications with a continuous range. I’m a fan of iToF if the surface reflectivity is within a narrow range. Some automotive LiDAR (Cepton, Ouster) have qualities for longer range, outdoor, applications but has resolution and cost tradeoffs that might not be worth it for <3m applications.

I wonder when we can rip out the iPAD SPAD…

5 Likes
7

The end of Realsense cameras is a real disappointment for us as we use D435s extensively in our robots. We are definitely working on finding a replacement but none so far which match the following points:

  • Works in the dark (d435 has an ir pattern projector)
  • Wide FOV ( H ~90°)
  • Short range from 0.2m (even with degraded quality)
8

This is incredibly shortsighted on Intel’s part. I just hope they continue manufacturing and providing the units for years to come, even if they aren’t updating the software anymore.

9

I am going to play my “old man yells at cloud” card and say that Orbecc COULD be a good alternative if they improve their open-source drivers.

Their driver use poorly implemented, closed-source filters that waste CPU, when a better implementation COULD fix this easily.

2 Likes
10

I am really disappointed to hear their decision.
The Mynteye team was also disbanded at the end of 2020. Since then RealSense has become the only product series based on active stereoscopy. There were rumors that Mynteye seems to have resumed their product supply. However their previous team members had quite negative comments on their boss.
RGB-D camera based on active stereoscopy is the best choice for obstacle detection, if both outdoor usage and dim scene usage are required. For pure outdoor daytime usage, passive stereo camera such as ZED is fine. For pure indoor usage, ToF and Structured-Light may also meet the requirement. Although their effective range is generally shorter.
Another use case of RealSense is VSLAM. This generally requires attached IMU and internal hardware synchronization. You can easily find some cheap stereo cameras. But most of them have neither IMU nor hardware synchronization. I also tried to build one myself, using IMX219-83 from Waveshare and Nvidia VisionWorks. This one contains IMU but no hardware synchronization. If you are finding a replacement for RealSense Tracking Camera, Indemind is a good choice. They use two grayscale cameras to achieve high frame rate (200 FPS). It’s not good at generating dense depth image.
I think I may ask Waveshare to see if they have interest to improve their product (add hardware synchronization and IR pattern), because it’s really cheap. VisionWorks will also be abandoned by Nvidia, and will be replaced by VPI. VPI uses PVA to estimate stereo disparity, so it does not take up CPU and GPU.
OAK-D is also a good product. But its focus distance seems a bit far away.

4 Likes
11

@borongyuan can you elaborate on the Mynteye situation ? They still sell products but there is no more development ?

12

Regarding MYNT S cameras, do you know if they consume CPU a lot?

13

I’ve been using the OAK-D and it is definitely a good camera but it has a low FOV and their ROS package is still in development. It would be an excellent alternative in the future, but definitely not right now.

1 Like
14

It is passive right ? Do you think an IR projector could enable it to work in the dark ?

15

I have had good success with the Asus Xtion using the openni2 drivers in ROS1. This is similar in cost (~$300 US)to the Oak-D camera. The Xtion is the unit used by TRI on the head camera for the HSR and Stanford on the head camera for Jackrabbot2. The Xtion is limited to indoor and close (0-3m) range. I still would like to see a side by side comparison, of not only the cameras mentioned, but also other depth and stereo cameras. I understand that besides the sensor, the computing overhead of using optical sensors fro both SLAM and object recognition and tracking, is also a major consideration.

16

Has there been an official statement from Intel regarding ending the RealSense line? All I have found are the posts that have been referenced but these are not official Intel comments, thus it would appear to be only a rumor and speculation at this stage.

17

I’ve liked these as hobby cameras, but it looks like they sell a model with a bit better packaging for inclusion into a product. The idea of having AI running on the camera itself I have mixed feelings on

  • One side: Yay! Can be done on the camera and I don’t need to change my compute to support a GPU and deal with that additional complexity in my system.
  • Another side: Can it handle my specific <insert model or need> for my specific product with my team of people knowledgeable with the usual methods of deploying edge models. Can it run fast enough to be an actual replacement to a GPU?

Unquestionably though, if it does non-AI things like some powerful filtering and/or clustering of depth information on the camera and returned the pointcloud filtered/unfiltered and with quality clusters, that itself would be of serious value. Filtering pointclouds when you’re working with many sensors on a robot can be nontrivial on limited compute resources. +1 if they include basic tracking of depth blocks.

2 Likes
18

Compared to many other cameras it has a compute power that does not drag a significant compute power from the host machine. This was a selling point of Realsense at least for me as we could use on embedded ARM machines.

So as long as it can make an ok depth prediction for obstacle avoidance it is a very good alternative to Realsense

1 Like
19

Great list, everyone! Brandon here from Luxonis here. I had somehow missed this thread and @Katherine_Scott kindly pointed it out to me today. (Thanks, @Katherine_Scott !)

To add to this list, we also have a list with more depth/stereo/etc. solutions here.

And thanks everyone for the feedback on OAK-D! Since knowing what is coming down the pike in terms of OAK-D might be helpful here (spoiler: we’re not cancelled) I figured it would be worthwhile to share what we have in the works.

Agreed - our ROS is in development. We think we will have a solid ROS solution in September. Right now basic support is there - but things like manual framerate control, exposure settings, etc. are missing. @saching13 is working on adding these now and then we will get official apt-get installable packages out.

For higher FOV, we don’t support this off-the-shelf, but we have had a slew of folks integrate high-FOV into their own custom products/solutions (using our OAK-SOM in their product). See the ArduCam pin-compatible version here for example. And we now support mesh calibration/rectification/etc. so these wide-FOV can be used for depth/etc.

And this actually can even be installed on the OAK-D PCBA, if you’re so brave to take it out of the enclosure:

image
image1920×2560 205 KB

We’re more than up for making requests as well. So if say this wide FOV + laser emitter/etc. is of interest, we may be able to make it happen.

And for custom solutions, we already do have quite a few end-user-products where such a thing has been done (particularly in small robots). The open-source designs make this a lot easier/faster.

Yes - all our current solutions (Hardware Github here) use passive stereo depth (Census Transform based).

We are working on the following though:

  • OAK-D with IR laser dot projection (active stereo depth) and IR LED blanket illumination. We’ll likely call this OAK-D-PRO.
  • Time of Flight (w/ option to add stereo depth as well - and potentially also active).

Here’s OAK-D-PRO with the laser dot projector and IR blanket illuminator:

image
image936×421 26.4 KB

We’ll release this as soon as we get through laser-safety certification.

And here’s the initial (not-yet-dialed-in) ToF on OAK-D:

image
image1452×622 62.6 KB

The timeline on ToF is a bit less known, but it’s working!

Our mounting/enclosure/etc. on the current OAK-D is not what it should be. (It was my fault; wrong calls on my part). We’ve since listened to feedback from our customers (and thanks for being one!) and corrected this in future designs.

So all of the next-gen we’re working on will have VESA- (75mm-) spaced M4 mounting on the back for easier/more-secure mounting and inclusion of the finished camera into a robot/etc. The current plan is below:

image

It’s often a bit hard to tell what CAD renderings really look like in terms of size, so below gives an idea:

image
image1920×1440 110 KB

So this has the tripod mount on the bottom, and then 2x M4 mounting holes spaced at 7.5cm on the back. (And then of course we have our OAK-SOM series for inclusion of this into custom products/etc.)

Thoughts?

Thanks again!
-Brandon from Luxonis

6 Likes
20

Hello,

I am Vincenzo from Terabee (www.terabee.com). I am Product and R&D Project Manager at Terabee for industrial automation products.

At Terabee we are time-of-flight experts and have recently released a compact and rugged 3D industrial ToF camera, the Terabee 3Dcam VGA. It streams depth / point cloud / IR / quality factor with VGA resolution (640 x 480), 90° × 70° FOV and full aluminium case (<500 gr weight, IP65 and IP67). As a ToF nIR camera, it works very well in the dark.

hand_3Dcam_VGA
hand_3Dcam_VGA1002×621 36.1 KB

We also offer Linux/Windows SDK, GUI, ROS package (Noetic, Melodic) and “open” on-board Linux computing power.

Here the website of the product and the direct link to the spec sheet

Depth-IR-blog-post

You can contact me directly if you want to know more: vincenzo.forte@terabee.com

Of course I am also eager to know from you a first impression on the product itself.

Best,

Vincenzo

5 Likes