Automatic Deployment of ROS2 Based System to remote devices: Dual Copy or Containers?

The architecture of Podman makes it much easier to integrate into your service manager than Docker.

With Docker everything is an RPC request to a daemon (that often, but not always, runs as root), making it not suitable for individual containers to be encapsulation into service managers such as systemd, that can usually provide cgroup and other security limits, as well as lifecycle management.

With Podman, the container processes remain children of the parent that launches them (usually the podman run command). This makes it trivial to add via ExecStart to a systemd service, and everything remains in the scope of the service. This container/service can then be integrated into the startup sequence of your system as a whole, enabling you to depend on other system services.

This is particularly beneficial when it comes to systemd’s .device units. You can write a systemd service that will only run when particular hardware is plugged in, starting up the relevant Podman containers and bind-mounting the particular device inside, then gracefully shut down the container and service when it’s removed. It’s almost like a containerised userspace driver :slight_smile:

3 Likes

Throwing in another option: using [GitHub - asherikov/ccws: ROS development environment for (cross-)compilation, testing, linting, documetation and binary package generation. Can be used interactively and in CI.] you can generate deb packages that include the whole ROS stack and install different versions of them in parallel. Switching back and forth should require some reasonable amount of scripting.

(Just Idea)

After I have seen all these information, I think we definitely can do bird of feather at ROSCon 2023? what do you think? any volunteer to lead this BoF?

4 Likes

Hi,

(those who attend to ROSCon 2023)

We will have BoF session at ROSCon 2023 for Deployment of ROS 2 to remote devices in next week.

If you attend to ROSCon 2023, please feel free to join :smile:

We will be sharing experience, ideas and related topics like brain storming casually.
So just drop by if you are interested :exclamation: I hope I can meet everyone in this thread at ROSCon 2023.

thanks,
Tomoya

2 Likes

I know this is ROS2 but for ROS 1 we have been using colcon-bundle to make our artifacts that we then deploy using git and a custom script the points to what tag to download. As colcon-bundle is deprecated in ROS 2 we are looking for a replacement as we are looking to upgrade to ROS 2.
We have been looking a bit at using Nix Does anyone have any experiences with that?

There was a talk about ROS using Nix last weekend on LinuxDays in Prague. Unfortunately, both the presentation and the talk were given in Czech. However, nowadays there are powerful instant translation tools, so maybe someone will be able to e.g. listen to the video with SayHi running on the phone :smiley: Jak na ROS pomocĂ­ Nixu :: LinuxDays 2023 :: pretalx .

I would suggest to go the ROS-with-Nix guru himself: @Mike_Purvis.

See his ROSCon22 presentation: “Better ROS Builds with Nix” (recording, slides).

1 Like

I can briefly share what was done for Beams from Suitable (over 7,000) of those sold and what I am doing at Meta for our robots.

I have a script that builds a custom ubuntu OS image that starts with debootstrap and basically finishes by turning the root image into a read only sqshfs image.

Everything can be put into that image and it is read only. Once it is packaged it can be small, maybe 1 gig, depends on your dependencies.

Now you create a system with a few partitions and the boot partition will be created with the kernel and initrd that you can distribute with the sqshfs or extract it from that image.

Your boot loader, chooses the kernel you want, that kernel mounts the sqshfs image and the system boots and runs from a read only image. You make the magic happen by adding a few tweaks to the initrd process which is pretty easy to do.

Now you might want each system to have some unique properties, like a hostname or a configuration file. You can place that in a different partition or directory and use an overlayfs to create directories that are writable over the underlying root OS image, or you can make changes in a RAM fs and just throw the changes away.

To upgrade the system it downloads a new root image, saves it, puts the kernel files in /boot and makes it available for grub. You can just manipulate some symlinks for this.

It is easy to recover or rollback automatically in the boot loader just by picking an older kernel.

I created this approach because it is efficient for distribution, immutable and easy to recover from failure. The robots query a server and download what is offered to them. All management is done automatically.

4 Likes

To put a long story short, you scripted yourself what openembedded and yocto gives you kind of for free. Interesting approach.

It is also possible to achieve the same with Ubuntu Core, if one wants to stay inside the Ubuntu ecosystem.

I would like to see pointers to the Ubuntu Core equivalent?

Hi Phil

I couldn’t understand correctly the ‘Ubuntu Core equivalent’ meaning.

p.s I’m using ROS foxy

2023년 10월 23일 (월) 오전 3:00, Phil Brown via ROS Discourse <notifications@ros.discoursemail.com>님이 작성:

Dear @Phil_Brown and @ybbaek,

I was replying to @RethinkMatze 's comment, and mainly meant that transactional/differential updates are supported by Ubuntu Core out of the box as well: OTA updates | Ubuntu Core | Ubuntu

I am sorry for the confusion.

Kind regards,
Gergely

Hi @kisg ,

From briefly looking at it, it only seems to be applying to the snaps (read-only, compressed), not the entire OS.
How would you update the base OS?

Thanks and regards,

Matthias

Just throwing another idea, Guix is really neat for this task. Updates are atomic, can do delta updates, you can revert in a second, and you can describe your whole system using a few text files.

I tried (without success yet) to automate the distribution of ROS using guix. Nix was mentioned, I haven’t tried it yet.

1 Like

I disagree. The big benefit of @boilerbots’s approach is that you can still use Ubuntu apt packages to create the system image. This I found much easier than learning about the packaging and layer system of yocto (or buildroot) and you don’t need to compile things from scratch, so it’s much faster.

Hi Everyone,

We at Nervosys have been thinking about the ideal operating system (OS) for robotics for some time.

We agree with those who suggest that Nix offers a close-to-ideal solution for building, deploying, and managing fleets of robotic systems. To take NixOS to the next step of being ideal for intelligent robotic systems, we have created Botnix (“Robot Linux” or “Robotics Linux”), an open-source operating system for robotics based on NixOS:

https://github.com/nervosys/botnix/

We encourage you all to join us in building the future of robotic general intelligence. We are building a free and open community dedicated to AI for robotics. Feel free to reach out here or through the Botnix Discord:

https://discord.gg/FA4UjaB7/

Ad astra,

Adam Erickson, PhD

@Hugal31 Guix packages for ROS would be fantastic. How far did you get with packaging?

In Ubuntu Core, everything is a snap. Not only the applications layer, but also the runtime environment, the kernel, down to the bootloader. Hence all layers benefits from the features offered by snaps (transactional/ota/delta updates, isolation, security, immutability, etc).
On top of that, UC also comes with things like secure boot, full disk encryption, validation sets, recovery mode to name a few. Find out more here.

Just a heads up that Botnix, which is basically just NixOS at this stage, comes with many of the benefits of Snaps in the form of the Nix Store. This is without introducing the overhead of a container system.