NVIDIA announced the 2022.1 release of NVIDIA Isaac Sim with new features to accelerate development, testing and training of AI robots. Isaac Sim, powered by Omniverse, is a scalable robotics simulation application and synthetic data generation tool that powers photorealistic, physically-accurate virtual environments to develop, test, and manage AI-based robots. Download>>.
Highlights includes:
| Ignition-Omniverse connector with Gazebo | Enables ROS developers to combine the advanced features of Isaac Sim and Gazebo with a single connector. |
| SceneBlox | Isaac Replicator new feature for generating synthetic data scenes procedurally and complex environments automatically. Expedite training AI robots with photorealistic data. |
| Omnigraph | Simplify application development and debugging with visual programming. New ROS pipelines are implemented in Omnigraph. |
| Isaac Cortex | Program task-aware and adaptive skills behaviors for collaborative robots (cobots) tasks as easily as programming game AI using Isaac Cortex decision framework. |
| New Robots and Assets | Quadrupeds: A1, GO1, Anymal; AMR: Obelix; New modular warehouse and conveyor assets |
Bridging Ignition Gazebo and Isaac Sim Gap
In conjunction with Open Robotics, NVIDIA presents ROS developers with a seamless simulation interoperability connector between Open Robotics’s Ignition Gazebo and NVIDIA Isaac Sim.
With the new Ignition-Omniverse Connector, ROS developers using Gazebo can benefit from the best of both virtual worlds such as physically-accurate sensor models, high-fidelity dynamics, developers testing tools, photorealistic and real-time rendering. Three ways in which these interaction can occur includes:
-
Bi-Directional SDF <-> USD file converter: It takes as input an SDF file used in Gazebo and produces as output a USD file used in Isaac Sim with approximately equivalent results; and vice versa, allowing the user to easily migrate assets between Isaac sim and Gazebo. It is available through a component from sdformat. Check out these instructions on how to install it.
-
Simultaneous Multi-Gazebo Simulations: Multiple gazebo simulations can stay live synched through a shared USD stage by connecting to Omniverse’s nucleus server.
-
Hybrid Simulation (Isaac Sim - Ignition Gazebo)
The workflow for the hybrid simulation is explained in the image below.
Figure 1. Ignition-Omniverse Connector Workflow
The connector allows ROS developers to:
- Separate their simulation workload between Ignition and Isaac Sim, with both systems running in parallel. For example, physics can be handled by Ignition, with rendering and sensors handled by Isaac Sim, or vice versa.
- Share live assets data and modifications of the virtual environment between a running Ignition simulation and a running Isaac Sim simulation.
- Visualize the data in the ROS network from both simulators in Rviz2.
- Use ROS to send sensor data to the ROS network or receive some commands to manipulate an asset.
- Support common operations, like model addition, deletion, and pose change.
Check out the code in the gz-omni repository.
Train AI robots with Photo Realistic Synthetic Data
Isaac Replicator is the synthetic data generation tool in Isaac Sim. Synthetic data is very useful in robotics to bootstrap training, address long-tail dataset challenges, and provide unavailable real-world data like speed and direction from synthetic video. Autonomous machines require synthetic data in training to ensure model robustness.
Figure 2. Example of a warehouse procedurally generated using SceneBlox
In the latest release, ROS developers can generate scenes procedurally using the new SDG feature called SceneBlox. SceneBlox can be used to create industrial environments like warehouses automatically. New examples were also added that demonstrate how to generate synthetic data and train a pose estimation model using Replicator.
Omnigraph in Action
Below is a demo and a graph showing how to control the jetbot.
Get started
- Download Isaac Sim.
- Follow Ignition-Omniverse connector with Gazebo tutorials for building your own hybrid robotics application.
- Watch Bridging Ignition and Isaac Sim at GTC 2022 for the implementation and some demos.
- Watch Isaac Replicator Composer Webinar by Matthew Trepte.
- Watch Isaac Cortex GTC presentation by Nathan Ratliff.
- Post your queries on NVIDIA forums.