Hey everyone!
We’ve developed an open-source teleoperation device that lets you control a robot arm using just your phone—no extra hardware or app downloads needed! Just move your phone, and watch the robot gripper mimic your motions in real time.
As part of our imitation learning project, we needed a teleoperation device to enhance our process of gathering demonstrations. We tried keyboard, spacemouse, and freedrive, but it for some use-cases it limited us. This approach utilizes WebXR (specifically ARCore) to fuse RGB and IMU data, allowing us to servo the robot arm in 6 DoF (Degrees of Freedom).
ROS2 Support
The project has built-in support for the ROS 2 cartesian_motion_controller but I guess it should easily adapted for MoveIt Servo.
How It Works?
Python Package: The package starts a web server. Open the web app on your smartphone, and it streams the phone’s position in 3D space (position + orientation). Real-Time Control: We use the phone’s pose data to servo the arm.
We packaged this as an open-source project because we believe it could be valuable to the community.
