Isaac ROS Oct update, Mission Dispatch & Client, Freespace and Image compression

, , , ,
1

At ROSCon Kyoto 2022, we are releasing Isaac ROS Developer Preview 2 with major updates, enhancements, and bug fixes.

Isaac ROS update for ROS2 Humble is available now at github.com/NVIDIA-ISAAC-ROS, including vision-based perception for navigation and cloud to robot task assignment/tracking.

This release includes packages for AI perception, image processing, navigation and adds:

  • Mission Dispatch and Client to assign and track tasks from a Cloud | Edge fleet management system to the robot in real operation and simulation. It is pre-integrated with Nav2 and provided as open source.
  • Freespace segmentation from an improved BI3D DNN model produces a vision-based occupancy grid in the neighborhood of the robot.
  • H.264 video encode and decode hardware accelerated NITROS packages to compress camera data capture and playback for development of AI models and perception functions, compressing 4x 1080p cameras at 30fps (>120fps total) reducing data footprint by ~10x.
  • Updated ROS2 Humble + Nav2 container saving >8hrs of compile time for Jetson and x86 + GPU including scanning against NVD to fix security vulnerabilities in the container.
  • Bug fixes

isaac_ros_dp2_realsense_freespace
Freespace occupancy grid visualization of result computed from RealSense stereo camera pair, without using depth from RealSense

isaac_ros_dp2_bi3d_delta
isaac_ros_dp2_bi3d_delta2880×576 125 KB

BI3D model from Isaac ROS DP1.1 release in the middle compared to improvement for flat featureless surfaces in this release, Isaac ROS DP2 on the right

Mission Dispatch and Client provide a standard, open source method to assign and track tasks between a fleet management system and ROS2 robots. Dispatch and Client communicate using VDA5050 on top of MQTT wirelessly between the cloud and robot. For more information see Isaac Mission Dispatch & Client w/ VDA5050 open source release for ROS2.

Performance of this release benefits from hardware acceleration additions to Humble and additional optimization.

Package ROS software (CPU)
Humble Jetson Orin		 Isaac ROS DP2
Humble Jetson AGX Orin		 Isaac ROS DP2
Humble Jetson Orin Nano (8GB) 		Isaac ROS DP2
Humble RTX3060TI + core i7 11th gen
AprilTag (720p) 100fps
9.7ms		 248fps
5.5ms		 82fps
14ms		 600fps
2ms
DOPE
(object pose DNN) (VGA) 		N/A 40fps
30ms		 N/A 84fps
15.4ms
Freespace
(1080p) 		N/A 1145fps
1.3ms		 725fps
2ms		 1490fps
0.3ms
Image compression
H.264 I-Frame only(1080p) 		20fps
49ms		 170fps
17.4ms 		N/A N/A
Image decompression
H.264 I-Frame only(1080p) 		102fps(core i7)
29ms		 N/A N/A 400fps
2.3ms
Image detection
people detector (544p) 		N/A 225fps
7.7ms		 72fps
18ms		 450fps
3.2ms
Image segmentation
people detector (544p) 		N/A 260fps
3.7ms		 128fps
6.7ms		 300fps
2ms
Proximity segmentation
BI3D (576p) 		N/A 81fps(GPU) 61ms
62fps (DLA) 46ms		 25fps(GPU) 65ms
N/A (DLA) 		145fps(GPU) 386ms
Rectify node
SGM CV (1080p) 		231fps
2.3ms		 730fps
1.5ms		 330fps
3.1ms		 900fps
0.4ms
Stereo disparity
ESS DNN (1080p) 		N/A 51fps
17.3ms		 16fps
60ms		 98fps
7.6ms
Stereo disparity
SGM CV (540p) 		74fps
12.5ms		 144fps (GPU)
7.6ms		 52fps (GPU)
20ms		 380fps (GPU)
3.1ms
VSLAM
(720p) 		N/A 250fps
3.1ms		 105fps
10ms		 265fps
5.2ms

Performance measured includes input node → node under performance test → output node where the publishing rate of the input node is autotuned to discover the peak throughput dropping <1% of the frames. Average fps computed over 5 runs, discarding minimum and maximum runs; latency measured at 30fps publishing rate.

Isaac ROS update is available now at github.com/NVIDIA-ISAAC-ROS and is part of our commitment to provide features and hardware acceleration for autonomous robots.

Clone the repositories you need into your ROS workspace to build from source with colcon alongside your other ROS2 packages and leverage the pre-built ROS2 Humble & Nav2 container. Please note that this release has been tested on the NVIDIA Jetson AGX Orin & Xavier with JetPack 5.0.2 (Ubuntu 20.04).

An incremental update will be around the end of the year, with the next major release at GTC in March of 2023.

4 Likes
2

Reviewing the Docker containers and the instructions for using this, why is it based on Ubuntu 20.04 and a source build of ROS Humble? That combination makes it useless to use any of the existing infrastructure for packages in ROS or even build our existing packages that target Humble and depend on system packages that do not exist on 20.04.

3

Good question. Jetpack supports Ubuntu 20.04, for our target Jetson platforms (trading compilation time for download time).

We appreciate the complexity of compiling Humble from source on Ubuntu 20.04, which is why a pre-built container for ROS2 Humble w/ Nav2 is provided on NGC.

Thanks.

4

There are more packages than just Nav2 a user might want. When they have system dependencies on libraries packaged with Ubuntu 22.04 this becomes a problem where they can’t build for your target without a bunch of work. Why did you not choose to use a ROS version that is supported on Ubuntu 20.04 or upgrade the Jetson platforms to 22.04 if you wanted to use Humble?

1 Like
5

Yes, and we are open to feedback on additional packages we can include into pre-built containers to bootstrap evaluation and development. It does not remove the need to recompile when changes to those packages are made. Which packages would be beneficial to add?

We understand this and experience it ourselves porting changes between versions.

If there are system dependencies on Ubuntu 22.04, that are not present in Jetpack provided Ubuntu 20.04, porting would be required. The amount of work varies on the number and scope of system dependencies. Are there specific gaps you have identified for your work between Jetpack 5.0.2 and what you need from Ubuntu 22.04?

ROS2 works with multiple operating systems, and versions of those operating systems by design. Do you mean ROS2 Humble can only work with Ubuntu 22.04? Or do you mean versions are limited to only variations supported by the build farm?

Some Jetson customers working to ship in production systems are not so quick to switch to new OS versions, where-as some others in earlier phases of development adopt the latest. We need to find a balance between these often conflicting needs. The time invested in compatibility, security, performance, and quality is value offered in Jetpack with Ubuntu 20.04 for building products one can depend on in production.

We worked on improvements to ROS2 Humble for hardware acceleration with OSRF to provide great performance. That makes ROS2 Humble a high order bit to build autonomous robots.

Thanks.

6

On 2023-01-17 at 8am PST we will be hosting a webinar to show you how to use and extend Isaac Mission Dispatch and Mission Client. These tools can help you build cloud connected robots using open standards like MQTT and VDA5050. Click here to register: Isaac ROS Webinar Series