I am looking for a general solution to the problem of connecting a robot to ROS nodes running inside a k8s cluster. It’s a scenario that I believe will become more and more common in practice, however I could not find any guidelines or best practices anywhere.
Basically the core of the problem is that ROS nodes use TCP/UDP/DDS to communicate with each other on random ports, while k8s wants to expose services on configurable ports and typically over HTTP(S). Ingress controllers have to be configured to route packets from the robot (outside the cluster) to the “right” destination pod. This doesn’t really play well with ROS.
I have looked around here on discourse and played with a few implementations I found online, but I would like to know if I am missing something obvious or if somebody has come up with other solutions.
As of now, I am aware of the following approaches:
Using a VPN (e.g., wireguard, or maybe Husarnet? e.g., Connecting Remote ROS 2 Nodes using Docker & VPN) to create an overlay between the pods subnet/namespace and the robot(s);
Extending the k8s cluster through federation, basically adding a robot as a “remote cluster” (e.g., Robotics Distributed System based on Kubernetes) and using cluster “internal” networking with the appropriate CNI plugin;
Using Ros Bridge (GitHub - RobotWebTools/rosbridge_suite: Server Implementations of the rosbridge v2 Protocol) in Web socket mode, exposing it through a k8s ingress, and running a client on the robot (e.g., rosduct GitHub - uts-magic-lab/rosduct: Proxy to expose remote ROS topics, services and parameters locally thru rosbridge );
Using message brokers (e.g., RabbitMQ) on the k8s cluster and “cloud bridges” to relay ROS messages across nodes connected to different ROS masters (e.g., ROS Routed Networks :: rapyuta.io Documentation );
Using ROS only within robots / cloud and transforming ROS messages to another representation on the internet;
Are you aware or (even better) do you use any other solution in production?
Thank you in advance.
All the best,