I would like to share you about my two ROS 2 manipulation projects. They are used deep learning techniques with NVIDIA Jetson GPU acceleration.
IKNet: Inverse kinematics neural networks
IKNet is an inverse kinematics estimation with simple neural networks.
It can be run on Jetson family or PC with/without NVIDIA GPU. The training needs 900MB of GPU memory under default options. That’s why the training can be run even on NVIDIA Jetson Nano 2GB!
This video shows the inference demo using 4 DoF manipulator ROBOTIS OpenManipulator-X. The inference can be solved under 0.3 degrees precision in average and calculated within 1ms thanks to NVIDIA TensorRT.
This is the source code repository. If you are interested in, the README.md is described everything you need. It also has the training datasets and the learning models. Please check it.
Robot gripper control by hand gesture estimation
This demo shows the robot gripper control by a hand gesture estimation. If the hand indicates fist, the gripper makes close. And if the hand indicates pan, the gripper makes open.
The hand gesture estimation is thanks to the ros2_trt_pose_hand .
The project is still under construction but the ongoing PR is here.
I wish the details of the projects and the additional works will be shown on ROSCon 2021. Please stay tuned.