Prompting TurtleSim from ChatGPT

Put together a demo to prompt TurtleSim through ChatGPT

The demo lets ChatGPT call into ROS services through WebSockets via rosbridge_suite and roslibpy. This ensures that ChatGPT only executes what you allow it to execute. ChatGPT does not run any Python code on your machine. This also relieves a client from ROS dependencies.

Prompt:

Move the turtle left by 2, then rotate 180 degrees, and move back to (5, 5).
Finally, spawn a turtle named turtle2 at (10, 10) and remove turtle1.

Result:
turtlesim

7 Likes

maybe for anyone who comes across this: Would someone be interested in creating a organization on github to work a little on this and to extend it to the ROS ecosystem more broadly in an autogpt-like approach?

I really like this idea of integrating AI and ROS.
I do have a good amount of experience with ROS2 Humble and would like to contribute to your idea as much as I can!

Good luck for the upcoming projects! :smile:

2 Likes

On similar notes, Microsoft have ChatGPT for robotics, repo but geared toward Airsim.

2 Likes

Hey :wave:, this project is really great, and has good scope for the enhancing the HRI. Since you have mentioned about the next level for an autogpt kind of approach, that’s really great and I think rosgpt will give you some inspiration as well. Looking forward to contribute!

1 Like

oh wow lots of existing work already! Thanks for sharing all these

would this approach require the user to implement a new node for each node that chatgpt interfaces? What are the benefits/tradeoffs? Sorry if I get this implementation wrong, only had a brief look.

Isn’t it beneficial to have chatgpt directly interface services? It seems rosliby is a nice way to have chatgpt probe into existing nodes / services / parameters (actions don’t seem to be supported for ros2 yet)

1 Like

so here is an organization that could be used: ros-ai · GitHub

please let me know who would be interested in exploring a little. Just drop your github tag here and I’ll send out invites! Sent out an invite to @kalash so far.

1 Like

mhubii, can you invite me!! my tag: santoshchaganti

`

1 Like

Actually, I didn’t look in much to both of the implementations so far, but rather remembered about this project when you actually mentioned about the different future possibilities of your work. I will need to look in to it actually give an opinion about the tradeoffs. I would love to be part of the org, since this is such promising area with lot of great possibilities and learning opportunities. could you please invite me as well. lonebots that’s my username :wink:

1 Like

we created a Discord so everyone can stay in the loop!

Feel free to join Discord

Looking forward to working a little on these topics!

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.