Ever wondered how to run a Neural Network on a teeny-tiny embedded device?
Here at micro-ROS, we’re working with @Luxonis-Brandon and his team on the integration of the small turtle with an exciting AIoT (Artificial Intelligence of Things) project, involving the DepthAI BW1092 board by LUXonis. This small device enables interacting with LUXonis DepthAI System-on-Module (SoM) from a microcontroller. And guess who? Our good old friend the ESP32!
DepthAI is a platform built around the Myriad X which combines depth perception, neural inference and object tracking, all accessible via a Python API and a plug/play SoM with open-source hardware. It comes integrated with a variety of platforms, and in the case of the board we use for our project, it brings these capabilities to the embedded world.
The DepthAI BW1092 features three cameras (a general one, plus two allowing for stereoscopic vision), making it possible to run 3D object detection out of the box. In turn, this allows retrieving the metadata coming out of a Neural Network, running inside the Myriad X processor, into the development environment of our small ESP32 MCU. Thanks to the integration with micro-ROS, it will be possible to bring these data to the ROS 2 ecosystem, exposing ROS 2 topics with the results of classification networks with spatial vision coming out of an embedded device.
If we get lucky with the integration we might be seeing soon our next PointCloud message coming directly from the DepthAI spatial vision calculations!
Thank you for putting this together @FraFin ! We’re super excited about the cool things we’ll be able to do together with micro-ROS,. I can’t wait to see little autonomous actuators, all sorts of intelligent AIoT devices working together in real-time, etc. Going to be so cool!
UPDATE: Today at 5 pm CET we’ll be hosting one of our regular EWG meetings - this time with a special guest! @Luxonis-Brandon will be showing us the wonders of their Embedded DepthAI BW1092 - the platform featuring stereoscopic cameras, 3D recognition patterns and classification algorithms, operated by a humble (yet powerful!) ESP32 MCU.
micro-ROS is already running on this amazing device, and we’re looking forward to seeing the results of our efforts to convert the BW1092 metadata into ROS 2 topics!
