Cross-platform ROS+Gazebo development environments?

Training & Education
, ,
1

Hello,

I am in charge of the software team for my university’s competition underwater robotics team (we participate in RoboSub), and I’m trying to improve the experience for new members, particularly for the development environment setup.

Currently we are using ROS Melodic, with uuv_simulator on top of Gazebo for simulation. For a while we have been suggesting that members use a virtual machine to set everything up so they can get Ubuntu 18.04 for ROS Melodic. This past year I’ve created an Ansible playbook that automatically installs everything, and then created virtual machine images that I distribute, that each member then loads onto their computer to speed up the installation process from ~40 minutes to ~10 minutes. That’s the idea at least, but we’ve face many issues.

Most members are using VirtualBox. There are a few assorted issues that we know the solution to but are still annoying when we encounter them. Then there are others like the setup not working on dedicated AMD GPUs.

Is there a better way?

I’ve looked at this thread: What environments do you use for training and courses? - #4 by ruffsl

It seems Docker is a good option, and I was already considering it myself as we want to migrate to ROS 2 but our current hardware does not officially support newer versions of Ubuntu, so I was going to use Docker to achieve the migration.

One idea I had is to have people install Gazebo bare-metal so GPU acceleration should be a breeze, then have it talk to ROS in Docker somehow. Possible issues are (1) Gazebo’s support for Windows is experimental, not sure if it supports M1 Macs, and (2) more involved installation process, and my dream is that the installation would take 10 minutes at the most, with new members able to start writing tutorial code within 20 minutes on their first day.

Key wants/needs (not sure if they are all currently possible):

  • Majority of members are freshman students, many have little or no prior experience in robotics, Linux, Python/C++, etc. I want the setup to be as quick and painless as possible.
  • GPU acceleration is important for Gazebo simulation.
    • Our competition is autonomous, so we could probably get away with recording the simulation results to video, but ideally everyone would be able to run it live at 20-60 fps.
  • Need to support:
    • Windows, macOS, Linux
    • Both Intel and new Arm Apple Silicon Macs
    • GPU acceleration on Intel/AMD integrated graphics, AMD discrete graphics, Nvidia discrete graphics.
  • As a university club we suffer from turnover, so ideally the solution is simple enough that future members can easily learn how to support/maintain it. Ansible and making custom VM images was probably a bad choice on my part for this goal.
2

From a former member of an university robotics team, we solved this problem running ROS and Gazebo in Windows using WSL and VcXsrv Windows X Server. The steup process is quite the same of an Ubuntu environment, with the extra step of setting up the X Server inside windows, which acts effectively acts as the bridge between WSL and Windows

We never tried using GPU acceleration in Windows computers, I would guess you need to configure a GPU-accelerated X session and then make it available in the Gazebo execution context, but I’m not sure

I have a bad experience trying to teach Docker to freshman students, most often than not it ends being a major blocker for inexperienced users. If you have more reasons to dockerize your entire environment beyond simulation, I would definitively follow that path. The initial setup is not that simple, but once is done it really makes life simpler, but not everyone would agree with that. There are plenty of material in the internet about GPU accelerated docker images, so it is easy to find answers when needed, I think the trickiest part would be making the GUI available in the user space (which may not be needed if you are going to run a headless simulation)

At last, a question, have you tried using ROS 2? The support for platforms other than Linux is definitively better, even though is not perfect

3

Thank you for your thoughts and the X server idea.

We never tried using GPU acceleration in Windows computers

Was performance of Gazebo satisfactory via the X server and WSL? By GPU acceleration I mainly mean so that the Gazebo simulation runs at a good speed, not CUDA or non-graphical stuff.

have you tried using ROS 2? The support for platforms other than Linux is definitively better, even though is not perfect

We plan to move to ROS 2 Humble, including moving the simulation. Sorry I did not make that more clear in the original post. Whatever we go with for dev environments doesn’t need to work with ROS 1 Melodic, as we will move everything to ROS 2.

Cross-platform support looks better in ROS 2, but
(1) Ideally I want setup to be a matter of getting whatever prerequisite software (e.g. Docker, VirtualBox), then 1-3 commands. Perhaps this is not reasonable. I would settle for something reliable even if it takes an hour and many steps.
(2) I’m hoping to get around the problem of differences across computers. Personally I am not very familiar with Windows or macOS, and don’t want to deal with getting it to work across all 3 systems.
(3) The macOS version that is supported seems to be somewhat old, and not supported on Arm Apple Silicon Macs.

One of our members pointed out the cross-platform RoboStack package manager. That is another route I’m considering.

4

Some thoughts:

  1. How many teams/participants are there? Is it an option to provide them with a preinstalled PC? This obviously has a cost, but so do the hours you put into it.

  2. Do you really need Gazebo on the students’ computers? Maybe it can run on a university server and they connect to it? This would eliminate the troubles to setup GPU acceleration.

  3. If 2. is an option, maybe mini-pc’s could be a low-cost solution for 1.?

  4. Or: can they dual-boot Ubuntu from a USB SSD? SSD’s are relatively cheap.
    (not a USB stick, USB sticks are slow for random access)

  5. Are you familiar with the development environment of The Construct? They spawn a development environment (including Gazebo etc) which you can access through a web browser. It’s quite nice, albeit sometimes a bit slow to start. Maybe you could set up something similar on a university server?

1 Like
5

How many teams/participants are there? Is it an option to provide them with a preinstalled PC?

We have around 10-15 members on my team, but I’m not sure if that’s our maximum, and during the initial weeks there are 40+ people participating.

A preinstalled PC may be possible but purchasing things is a pain, we don’t have a ton of storage space, and I’m not sure if the university would approve this expense. These are solveable issues, so I will continue thinking about this possible solution and discuss with my team.

dual-boot Ubuntu from a USB SSD

This sounds pretty good. In fact I asked my friend who is taking robotics classes at the university and he said they require students to do dual boot (or at least virtual machine) of Ubuntu, on your computer itself, they do not provide SSDs. Some CS classes require to have Linux installed so I may rely on that. But prepared SSDs could be nice for freshman or people with little disk space.

I don’t think this would work for M1 Macs yet because you can’t boot Linux on them yet, but that is a manageable number of people, and virtual machines tend to work alright.

Do you really need Gazebo on the students’ computers? Maybe it can run on a university server and they connect to it?
…
The Construct

I’ve had this idea before, I will look more into it. A server setup could definitely achieve a very quick and painless setup. I could not find a good place for running GPU servers, but a teammate said the university may be willing to provide this if we ask. I am also working on a related project so maybe I will be able to do this.

Until I’ve got a better idea of our ROS 2 requirements, I think I’ll go with the plan of targeting Linux dual boot or VM, but moving away from custom-built VM images because that has proven to be confusing and painful. Then we will try to purchase some computers for club use because having some Nvidia GPUs will be useful for CUDA regardless. I will also look into a server setup as I’m working on a similar project already.

Thank you, I have a lot to think about. I welcome further ideas if anyone has any.