Autoware Docker Requirements

Autoware
,
1

There has been some discussion among the maintainers and certain users about the changes being made to the docker repository on Gitlab. As such, @gbiggs, @sgermanserrano, and I wanted to bring about some public discussion about the use cases and requirements to meet them for the Docker containers that are produced using the docker repository. In this case, we’re specifically talking about the Docker images in the generic folder, which can be run natively on x86_64 and armv8a platforms. Below are my thoughts and I would like to hear from others in the community who use these files as much as possible.

Just as a note, our intention is to make the docker repo version-agnostic meaning that it will not be tagged with version releases as the other repos are but should be able to build and run Docker images from the current version as many previous versions as is feasible.

  • entrypoint.sh and run.sh

    • Maintain backward-compatibility with images back to at least 1.9.0 by passing the $USER_ID environment variable containing the user id of the user running the script.
    • Change the autoware (default user in Docker images) user ID and group ID to match those passed in through either the $USER_ID environment variable or the --user flag (industry best-practice).
    • Change the ownership of files in /home/autoware to the new user and group IDs.
    • Provide a flag to bypass the ownership change since it can be dangerous when outside folders are mounted under /home/autoware and also because the ownership change on images containing pre-compiled Autoware takes upwards of 5 minutes on container launch (how many users actually require the permission change?).
    • Account for users launching the container both with and without root privileges (do we want to do this or just enforce one or the other?).
    • Use industry-standard methods for running commands sent to the container as a different user by utilizing tools like gosu.
    • Ensure that, when a command is passed to entrypoint.sh, it runs it in the container as the autoware user, unaltered.
    • Ensure that, when no command is passed to entrypoint.sh, it runs /bin/bash -l to provide a “login-style” script (runs /etc/profile.d and $HOME/.bashrc login scripts) as the autoware user.

  • build.sh

    • Support building latest (master) and previously-released versions of Autoware, back to at least 1.9.0 by passing a version number to the script (e.g. -v).

  • Dockerfiles

    • Utilize Docker-recommended best practices such as replacing commands like RUN su -c "...." $USERNAME with USER $USERNAME and RUN ....
    • Organize the generated layers carefully by combining/splitting commands to keep each layer to a reasonable size and containing a logically-grouped set of commands.
1 Like
2

Also to point out the files inside the generic folder can also be used without modification to build native docker images for arm64v8 architecture, which are currently hosted in https://hub.docker.com/r/autoware/arm64v8/tags

1 Like
3

Could you link to the source files for the Dockerfiles in question (both ai, and auto)?
Also, forgive me if I’m still lost in the migration of things, but the Docker Requirements we discussing here are for autoware.auto, not autoware.ai, correct?

4

@ruffsl - I have updated my original post with links to the files in question. Thanks for the suggestion! In this case, we are specifically talking about Autoware.ai. Autoware.Auto has a pretty robust Docker/CI infrastructure from what I’ve seen but I’m sure it’s open for discussion separately as well.

5

Hi @JWhitleyWork, as per Servando’s comment, can you please update the original post to avoid confusing people. The last major update to docker/generic added native compilation support for both armv8a and x86_64 platforms.

These are the main use cases I see Docker being used for on Autoware (I wonder if people have different use cases):

  1. CI.
  • Simplifies sharing a common system to run build and tests on CI systems
  • Controlled environment that can be easily reproduced by everyone (as opposed to custom machines setup for the project only)
  1. Base image for development.
  • Simplifies the setup for development.
  • Allow developers to switch between versions of Autoware/ROS/Ubuntu without major dependencies on the host system.
  • Ensure Autoware dependencies from system are properly tested as missing packages are easily detected.
  1. Distribute a pre-built Autoware to provide a simple out-of-box experience for people who just want to try out Autoware.

Requirements

  1. Keep “base” and “full” (containing the pre-built Autoware) images separated.
  2. Support easy sharing of data between host/container independent of host’s user
  • Enable users working from companies/universities were LDAP or another centralised user management is used.
  • Enable the use of shared systems (e.g. servers)
  1. Allow inserting a .repos into the “full” build to simplify testing (not sure the current CI already implements something similar
  2. Document the Docker usage
  • The wiki page imported into Gitlab is out of date
6

@filipe.rinaldi - I updated my original post to note that the files in the generic folder will work with both x86_64 and armv8a architectures.

Regarding requirement 3: I’m guessing you’re talking about the ability to add additional repos into the Autoware workspace? If so, this can be added but it would be good to have a Gitlab issue to track this feature request. Would you mind adding one at https://gitlab.com/autowarefoundation/autoware.ai/docker/issues?

Regarding requirement 4: I recently updated the Gitlab wiki Docker pages, prior to the 1.12.0 release. Can you please review https://gitlab.com/autowarefoundation/autoware.ai/autoware/wikis/Docker and https://gitlab.com/autowarefoundation/autoware.ai/autoware/wikis/Generic-x86-Docker and provide feedback? We do need armv8a-specific instructions but they will be very similar to the x86-specific instructions minus the Nvidia information.