gym-gazebo migration to ROS2 with MARA

ROS Projects
1

Dear ROS Community,

I am going to start the task of migrating gym-gazebo to ROS2, give it a new structure and test it with the MARA.

In case you don’t know about gym-gazebo, it is basically a toolkit for developing and comparing reinforcement learning algorithms using ROS and Gazebo. A few robots have been successfully tested in this environment, but the community has focused particularly in the Turtlebot. A couple example videos of this robot: qlearn-sarsa, running an already trained nn. I invite you to also take a look at the github repo.

Everything here is ROS1 based and the general structure is not very convenient. The goal is to migrate to ROS2 and use the under development ros2 version of the gazebo_ros_pkgs. Once this is completed, installation and usage tutorials will be created for the community.

Please feel free to share your thoughts, specially if you are familiar with the ros2 gazebo_ros_pkgs branch. Take a look at the discussion issue for a more technical debate and share your interest if you want to get involved in the project!

4 Likes
2

I’m glad to hear you’re doing this. Have you thought about using openai/gym as a library instead of forking it? Forking it seems to make it near impossible to get updates as openai moves forward. I realize that was the “recommended way” openai had suggested to use their repo, but it seems problematic.

1 Like
3

Sure, I don’t even know when was the last time updates form OpenAI Gym where merged. It makes more sense to update the structure to a more convenient one, while still using the gym library for some features like monitoring the environment. The core idea behind the gazeboEnv superclass and specific Envs for each scenario would be kept the same as is it now, following the OpenAI gym.

4

Interesting, did you struggle due to the fork yourself @mkhansen?

I think that using it as a library is a pretty valid suggestion. @rkojcev and @ahcorde, you guys have been maintaining lots of algorithms and gym-gazebo together with them, what’s your view on this?

5

That is a very valid suggestion. We will look into how to make gym_gazebo more of a stand alone package and use necessary libraries such as openai/gym

2 Likes
6

@vmayoral - I wouldn’t say I struggled, but I did notice that there was a big delta between the forked version you had and the original. There were bugs I ran into in the fork that had already been fixed in the upstream, and due to the big delta, it wasn’t easy to port them. If you maintain a fork, then you probably at least need to rebase it to the main repo on a regular basis to get the latest bug fixes. That in itself is time consuming, which is why I suggested using it as a library instead of forking it.

I hope that answer helps explain my reasoning.

2 Likes
7

Thanks for input @mkhansen, that makes sense. We’ll try our best.

1 Like
8

Just to let everyone know we have released our gym-gazebo2 and ros2learn toolkits, check the announcement at: Check out gym-gazebo2 and ROS2Learn!

1 Like