Turtlebot 3 Deep Reinforcement Learning Navigation

Gi_T · 2025-03-06T21:00:12.659Z

I recently extended the DRL-robot-navigation package by Reinis Cimurs, which trains a TD3 RL model for goal-based navigation, to support the Turtlebot3 and ROS 2.

My fork adds:
Turtlebot 3 support
Updated Gazebo interaction to work with increased real-time factor
ROS 2 support, making it compatible with the latest middleware and tooling
A WIP port to newer Gazebo versions, including Fortress and Harmonic

Repository: GitHub - TannerGilbert/Turtlebot3-DRL-Navigation: Deep Reinforcement Learning for mobile robot navigation in ROS Gazebo simulator. Using Twin Delayed Deep Deterministic Policy Gradient (TD3) neural network, a robot learns to navigate to a random goal point in a simulated environment while avoiding obstacles.

model_test

I wanted to get some feedback on the Gazebo Fortress/Harmonic migration (current branch). As far as I have seen it’s currently not possible to bridge the /world/default/set_pose service without Add SpawnEntity, DeleteEntity, & SetEntityPose Support by retinfai · Pull Request #380 · gazebosim/ros_gz · GitHub getting merged and backported. Therefore I’m currently using the Gazebo Transport package Python package which seems to work fine but when resetting with:

msg = WorldControl()
msg.reset.all = True
self.gazebo_node.request('/world/default/control', msg, WorldControl, Boolean, 1000)

the Turtlebot gets removed and needs to be added again. When using msg.reset.model_only = True the Turtlebot model stays but the environment doesn’t seem to reset properly showing different behaviour than when using Gazebo Classic.

Anupam_Das · 2025-03-06T21:46:29.906Z

@Gi_T This is really interesting work! I have a somewhat related project where I’m training a neural network to perform navigation using a single camera, aiming to improve upon encoder/IMU-based odometry. The goal is to use low-cost hobby robot components and a single ESP32-based camera.

Deep reinforcement learning (DRL) for navigation often relies on LiDAR, which increases costs. Do you think it’s possible to extend DRL-based navigation using only a monocular camera? Monocular SLAM results seem a bit noisy. (GitHub - weixr18/Pi-SLAM: Implementing full visual SLAM on the Raspberry Pi 4B) . Seems do-able as these depth estimation models are getting really good. Latency might be the bottleneck. Im still a little new to DRL techniques in general.

Gi_T · 2025-03-08T09:43:23.245Z

Extending the project to work with different sensors sounds indeed interesting. I’ll have a look if I find the time.

MoffKalast · 2025-03-08T11:19:32.524Z

That’s looking pretty good! Would be great to see some longer videos of it running.

I’ve always wondered how well these methods generalize to different sensors, say with different min/max range, FoV and frequency. Would it be need retraining for a lidar with twice the range?

A colleague of mine did his dissertation on the same topic for mapless navigation. Though that one was based on integrating markov chains with DWA if I recall right.

relies on LiDAR, which increases costs

Arguably cheap 2D lidars are now at price points comparable to cameras, no? There’s a decent selection under $100, from Slamtec, Robotis, LDRobot, etc.

Anupam_Das · 2025-03-09T20:05:16.963Z

Formfactor and cost is still not where it needs to be even compared to the slamtec type 2D Lidars. Think of something like the (late) Anki Cosmo. Trying to keep the total price under $200 for the entire robot. The challenge (naive as it may be) would be to use a single camera like an XIAO ESP32S3 Sense Camera (about $14.00 retail). and it be the only sensor (except for perhaps a simple cliff detection sensor along the base). With good encoder/motor and IMU you can get good enough odometry as shown here: https://www.youtube.com/watch?v=7Wbw5yRmgS8&t=2787s but what if we could improve on this with pre-trained localization model. For now im experimenting with ORBSlam but im it could be very interesting to compare the robustnes of this against a trained navigation model. That being said, the work of Boris Sofman/ANKI is really brilliant given the constraints they had to work with at the time.