--isolated in Docker image

I notice that your ROS2 Docker images are compiled with the --isolated flag. What is the advantage or tradeoff to that flag in that scenario? How does it affect the size of the Docker image?

I used --isolated (as well as other flags) in the Docker images mostly because that’s the ament invocation I use for development.

All the --isolated flag does is tell ament_tools to create an “isolated” install folder rather than a merged one, this means that each package will have it’s own subfolder in the install directory.

Without --isolated:

install
├── bin
├── include
├── lib
├── local_setup.bash
├── local_setup.sh
├── local_setup.zsh
├── opt
├── _order_packages.py
├── setup.bash
├── setup.sh
├── setup.zsh
├── share
└── src

With --isolated:

install
├── actionlib_msgs
│   ├── include
│   ├── lib
│   ├── local_setup.bash
│   ├── local_setup.sh
│   ├── local_setup.zsh
│   ├── _order_packages.py
│   ├── setup.bash
│   ├── setup.sh
│   ├── setup.zsh
│   └── share
...
└── visualization_msgs
    ├── include
    ├── lib
    ├── local_setup.bash
    ├── local_setup.sh
    ├── local_setup.zsh
    ├── _order_packages.py
    ├── setup.bash
    ├── setup.sh
    ├── setup.zsh
    └── share

The main advantage of this approach is that each package will have a different path for its includes, libraries etc (X_INCLUDE_DIRS, X_LIBRARIES…). This allows to make sure the dependencies are properly declared in CMake without relying on the fact that some files from projects that haven’t been find_package'd are present in the install directory.

The drawback is that setting up the environment (source (local_)setup.bash) takes longer as a multitude of directories need to be added to environment variables (PATH, PYTHONPATH, LD_LIBRARY_PATH…).
On Windows with the cmd command line interpreter, environment variables are limited to 2048 characters making this approach unusable.

Other flags used in the docker images that differ from the ros2 installation instructions:

  • --parallel: allows to build concurrently packages whose dependencies are satisfied. We don’t advertise it as we faced some race conditions at install time when not using it in cmbination with --isolated
  • --symlink-install: create symlinks in the install folder instead of copying the files. That allows faster install step as well as removing the need to rerun ament when modifying files that dont require compilation (e.g python scripts) in the source space. This is also not an option on windows as symlinking files requires administrator permissions.
  • --cmake-args -DSECURITY=ON --: compiles Fast-RTPS with DDS-Security enabled

Hope this helps

So you think disk usage is the same or does the isolated build duplicate copies of dependencies?

Have you looked into making the docker images run the local_setup.bash file as part of the image compilation so that it doesn’t require any time during image startup?

A quick test seems to show that --isolated does significantly increase the drive space usage. That seems like something we don’t want for Docker images.

Update: I forgot to test with the symbolic linking. I just did that; the isolated builds are still larger but using symbolic links does mitigate some of the growth.

As @marguedas mentioned before the isolated option should only change the location of installed files and only add a couple of scripts for each package. Please provide more information about the result of your comparison.

I see isolated builds cause a size increase on some of our internal projects. However, I just ran a build comparison on ROS2 itself and saw no significant size increase. I guess it’s a non-issue. Our internal projects probably have poor dependency management; I know that ament’s --parallel flag doesn’t work on our internal projects because of file conflicts.