At ROSCon Kyoto 2022, we are releasing Isaac ROS Developer Preview 2 with major updates, enhancements, and bug fixes.
Isaac ROS update for ROS2 Humble is available now at github.com/NVIDIA-ISAAC-ROS, including vision-based perception for navigation and cloud to robot task assignment/tracking.
This release includes packages for AI perception, image processing, navigation and adds:
- Mission Dispatch and Client to assign and track tasks from a Cloud | Edge fleet management system to the robot in real operation and simulation. It is pre-integrated with Nav2 and provided as open source.
- Freespace segmentation from an improved BI3D DNN model produces a vision-based occupancy grid in the neighborhood of the robot.
- H.264 video encode and decode hardware accelerated NITROS packages to compress camera data capture and playback for development of AI models and perception functions, compressing 4x 1080p cameras at 30fps (>120fps total) reducing data footprint by ~10x.
- Updated ROS2 Humble + Nav2 container saving >8hrs of compile time for Jetson and x86 + GPU including scanning against NVD to fix security vulnerabilities in the container.
- Bug fixes
Freespace occupancy grid visualization of result computed from RealSense stereo camera pair, without using depth from RealSense
BI3D model from Isaac ROS DP1.1 release in the middle compared to improvement for flat featureless surfaces in this release, Isaac ROS DP2 on the right
Mission Dispatch and Client provide a standard, open source method to assign and track tasks between a fleet management system and ROS2 robots. Dispatch and Client communicate using VDA5050 on top of MQTT wirelessly between the cloud and robot. For more information see Isaac Mission Dispatch & Client w/ VDA5050 open source release for ROS2.
Performance of this release benefits from hardware acceleration additions to Humble and additional optimization.
| Package | ROS software (CPU) Humble Jetson Orin |
Isaac ROS DP2 Humble Jetson AGX Orin |
Isaac ROS DP2 Humble Jetson Orin Nano (8GB) |
Isaac ROS DP2 Humble RTX3060TI + core i7 11th gen |
|---|---|---|---|---|
| AprilTag (720p) | 100fps 9.7ms |
248fps 5.5ms |
82fps 14ms |
600fps 2ms |
| DOPE (object pose DNN) (VGA) |
N/A | 40fps 30ms |
N/A | 84fps 15.4ms |
| Freespace (1080p) |
N/A | 1145fps 1.3ms |
725fps 2ms |
1490fps 0.3ms |
| Image compression H.264 I-Frame only(1080p) |
20fps 49ms |
170fps 17.4ms |
N/A | N/A |
| Image decompression H.264 I-Frame only(1080p) |
102fps(core i7) 29ms |
N/A | N/A | 400fps 2.3ms |
| Image detection people detector (544p) |
N/A | 225fps 7.7ms |
72fps 18ms |
450fps 3.2ms |
| Image segmentation people detector (544p) |
N/A | 260fps 3.7ms |
128fps 6.7ms |
300fps 2ms |
| Proximity segmentation BI3D (576p) |
N/A | 81fps(GPU) 61ms 62fps (DLA) 46ms |
25fps(GPU) 65ms N/A (DLA) |
145fps(GPU) 386ms |
| Rectify node SGM CV (1080p) |
231fps 2.3ms |
730fps 1.5ms |
330fps 3.1ms |
900fps 0.4ms |
| Stereo disparity ESS DNN (1080p) |
N/A | 51fps 17.3ms |
16fps 60ms |
98fps 7.6ms |
| Stereo disparity SGM CV (540p) |
74fps 12.5ms |
144fps (GPU) 7.6ms |
52fps (GPU) 20ms |
380fps (GPU) 3.1ms |
| VSLAM (720p) |
N/A | 250fps 3.1ms |
105fps 10ms |
265fps 5.2ms |
Performance measured includes input node → node under performance test → output node where the publishing rate of the input node is autotuned to discover the peak throughput dropping <1% of the frames. Average fps computed over 5 runs, discarding minimum and maximum runs; latency measured at 30fps publishing rate.
Isaac ROS update is available now at github.com/NVIDIA-ISAAC-ROS and is part of our commitment to provide features and hardware acceleration for autonomous robots.
Clone the repositories you need into your ROS workspace to build from source with colcon alongside your other ROS2 packages and leverage the pre-built ROS2 Humble & Nav2 container. Please note that this release has been tested on the NVIDIA Jetson AGX Orin & Xavier with JetPack 5.0.2 (Ubuntu 20.04).
An incremental update will be around the end of the year, with the next major release at GTC in March of 2023.