Synthetic data generation with Isaac SIM tutorial

In developing perception nodes for Isaac ROS, we created a synthetic data generation tool, Replicator Composer, built on Isaac SIM, to accelerate the iterative development process of training DNN models. This tool, developed in python, was released in source with Isaac SIM starting in the 2021.2 release.

Synthetically generated photorealistic datasets with physics include camera image sequences, instance & object segmentation, depth images, disparity images, and bounding boxes.

isaac_replicator_composer_parameters_groundtruths_rgb_29 isaac_replicator_composer_parameters_groundtruths_wireframe_29 isaac_replicator_composer_parameters_groundtruths_semantic_seg_29 isaac_replicator_composer_parameters_groundtruths_instance_seg_29 isaac_replicator_composer_parameters_groundtruths_depth_29 isaac_replicator_composer_parameters_groundtruths_disparity_disparity_right_29 isaac_replicator_composer_parameters_groundtruths_bbox_2d_tight_29

Learn how to get started using Isaac SIM Replicator with the tutorial which demonstrates how to quickly and easily generate a FlyingThings3D style dataset.

All parameters are specified in YAML files, so we can trace what was used for each element of a dataset, and can recreate the same image with updated versions of the tools. Compliance with safety standards requires we can track the version of the tools and inputs to the development process.

A key feature developed for the tool is sensor relative, where objects are placed relative to the sensor instead of relative to the scene. This allows for direct placement of parameterized objects from YAML for targeted test conditions.

The YAML parameterization allows for various levels of randomization or directed coverage, such as gaussian, uniform, choice from a list, or a walk across all elements in a list to achieve targets in training and test as part of our iterative development process.

Datasets are prototyped on developer systems generating 10’s of samples from a set of input YAML parameters to assess results before we scale into a containerized cloud based workflow for >= 10K datasets.

This tool is available for download in source with Isaac SIM.

7 Likes

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.