Is it a good practice to force `RPATH` to be embedded in the installed targets?

mani · February 16, 2017, 12:27am

For one of the packages I maintain (parrot_arsdk), I modified the install rules to install all the libraries to a sub folder of ${CATKIN_PACKAGE_LIB_DESTINATION} (i.e. ${CATKIN_PACKAGE_LIB_DESTINATION}/parrot_arsdk). The rational behind this is to prevent some of the libraries that are custom built as part of the SDK (e.g. curl and libressl) to be picked up by other ROS packages by accident.

When compiling a dependent package (e.g. bebop_autonomy) from source everything is fine. The compiled binary is linked against the correct parrot_arsdk libraries (since RPATH is embedded in the files created in the build/devel space). However when install rules are applied, RPATH information are stripped from the binaries (this is the default behaviour), so unless LD_LIBRARY_PATH is manually set, it is not possible to execute those binaries from the install space.

My workaround for this problem was to add the following line to CMakeLists.txt of the dependent package (bebop_driver):

set(CMAKE_INSTALL_RPATH_USE_LINK_PATH TRUE)

This fixes the issue on my local catkin workspace since the binaries installed into the install space now have full RPATH information embedded in them. However I am not sure if this is the best way to fix this issue and if this approach is going to work when the binaries are built on the build farm. Any comment or suggestion is really appreciated.

PS. This is to confirm that the RPATH information is indeed embedded in the installed targets:

$ ldd catkin_ws/install/lib/libbebop.so  | grep ar
libarcontroller.so => /full/path/to/catkin_ws/install/lib/parrot_arsdk/libarcontroller.so (0x00007fcfcd24f000)

Martin_Guenther · February 16, 2017, 8:27am

As you suspect, this won’t work when binary packages are built on the build farm. The generated binaries will include the full path to where libarcontroller.so was on the build farm, which will not be where it is on the machine where you install the .deb. So while building would work, the installed version will be unusable.

Usually, one would simply ensure that all required libraries are properly installed into a location on the LD_LIBRARY_PATH, but as you said you can’t do that:

The rational behind this is to prevent some of the libraries that are custom built as part of the SDK (e.g. curl and libressl) to be picked up by other ROS packages by accident.

I have a similar problem, where the manufacturer provides precompiled libraries that interfere with system libraries (such as Qt) and therefore must not be on the LD_LIBRARY_PATH for the other packages. My “solution” is to only set the LD_LIBRARY_PATH for the node when it’s launched:

github.com

uos/basler_tof/blob/810c8aee1b396d1ad46e785c1aca3859adf360f9/launch/includes/device.launch.xml#L26


      
          
            <!-- URL specifying the location of calibration data for the camera -->
            <arg name="camera_info_url" default="" />
          
            <group ns="$(arg camera)">
              <node if="$(arg publish_tf)" pkg="tf" type="static_transform_publisher" name="$(arg camera)_base_link"
                args="0 0 0 -$(arg pi/2) 0 -$(arg pi/2) $(arg camera)_link $(arg frame_id) 100" />
          
              <node pkg="basler_tof" type="basler_tof_node" name="$(arg camera)_node" output="screen">
                <!-- This is necessary to find ProducerTOF.cti: -->
                <env name="LD_LIBRARY_PATH" value="$(arg basler_tof_path)/bin:$(arg basler_tof_path)/bin/GigE:$(arg basler_tof_path)/lib64:$(arg basler_tof_path)/lib64/gentlproducer/gtl:$(env LD_LIBRARY_PATH)" />
                <param name="frame_id"  value="$(arg frame_id)"  />
                <param name="device_id" type="str" value="$(arg device_id)" />
                <param name="camera_info_url" value="$(arg camera_info_url)" />
              </node>
            </group>
          </launch>

This is however a very dirty hack (for starters, the env launch file tag has been deprecated since Fuerte), and I would be interested in a proper solution as well.

de-vri-es · February 16, 2017, 9:28am

In general I you shouldn’t set an RPATH on the installed binaries / libraries as a developer. However, if you’re packaging for a specific distribution you know more about the target system than the original developer, and setting an RPATH could be a good solution. The same could be true when doing local installs in your home folder.

In short, my view boils down to this: maintainers / developers shouldn’t mess with RPATHs, but users/packagers can have valid reasons. Of course you may be both at the same time, but you should still keep the separation in mind since other users/packagers may not want an RPATH in the installed targets.

Another option is to use a tool like patchelf to set an RPATH in the precompiled binaries. I tend to do this with closed source binaries that ship a bunch of conflicting libraries. You won’t need to set LD_LIBRARY_PATH then. If you do prefer LD_LIBRARY_PATH but you can’t use an env tag, you can use a wrapper script or call the env command.

mani · February 20, 2017, 5:53pm

Thanks @Martin_Guenther and @de-vri-es for your insights and comments.

@Martin_Guenther

I really liked the env-based method. I’ve never fully understood how env tag is deprecated but is still referenced in the documentation. Does your method work on Kinetic? I’ve seen the wrapper script method mentioned by @de-vri-es in other projects. I think I will go with that solution for now. The downside is running any node via rosrun CLI is going to be a hassle for end users.

In short, my view boils down to this: maintainers / developers shouldn’t mess with RPATHs, but users/packagers can have valid reasons. Of course you may be both at the same time, but you should still keep the separation in mind since other users/packagers may not want an RPATH in the installed targets.

Good point!

tfoote · February 21, 2017, 6:44pm

For setting environment variables you should use the env-loader element of the roslaunch XML machine tag.

Martin_Guenther · February 22, 2017, 8:31pm

In this case, we only want to set the LD_LIBRARY_PATH of one specific node and must leave the other nodes on the same machine untouched. So I don’t think the env-loader element is a good fit.

Martin_Guenther · February 22, 2017, 8:41pm

It definitely still works on Indigo, and I suspect it also works in Kinetic.

Why? They can directly rosrun the wrapper script, so that’s a point for the wrapper script.

On the other hand, the env element has the advantage that you can easily use launch file args to specify the location of the other libraries (like the basler_tof_path arg in my example above).

For a non-deprecated alternative to the env element, I like the suggestion by @de-vri-es to call the env command instead. You could even put it into the launch-prefix element of the roslaunch XML node tag.

mani · March 1, 2017, 7:42pm

Thanks @Martin_Guenther, @de-vri-es and @tfoote for your comments and suggestions. I came up with a solution based on an env-based wrapper script that works with both rosrun and within launch files. I could not get @tfoote’s suggestion of using the <machine> tag to work on a local machine. I second @Martin_Guenther’s point about this suggestion.

Here is the wrapper script I wrote:

github.com

AutonomyLab/bebop_autonomy/blob/adynathos-indigo-devel/bebop_driver/scripts/bebop_env_loader

#!/usr/bin/env bash

# This script prepend the path to parrot_arsdk's libraries to 
# LD_LIBRARY_PATH if it is set.
# 
# Example usage with rusrun
# $ rosrun --prefix "rosrun bebop_driver bebop_env_loader" bebop_driver bebop_driver_node
#
# Example usage within roslaunch
# <node pkg="bebop_driver" name="bebop_driver" type="bebop_driver_node" launch-prefix="$(find bebop_driver)/scripts/bebop_env_loader">
#
# More information: https://discourse.ros.org/t/is-it-a-good-practice-to-force-rpath-to-be-embedded-in-the-installed-targets/1335

if [ $# -eq 0 ]; then
  echo "Usage: bebop_env_loader /path/to/binary [args]"
  exit 1
fi

# If LD_LIBRARY_PATH is empty, do not attempt to do anything.
# Sourcing ROS setup.* files sets this variable

This file has been truncated. show original

It simply looks at LD_LIBRARY_PATH environmental variable. If it is set, it picks the first element, adds parrot_arsdk to the end of that element and adds the modified element to the head of LD_LIBRARY_PATH. Then it executes any specified binary and arguments with the env command using the modified LD_LIBRARY_PATH.

For example if LD_LIBRARY_PATH is set to /opt/ros/kinetic/lib:..., it executes the following command:

$ env LD_LIBRARY_PATH=/opt/ros/kinetic/lib/parrot_arsdk:/opt/ros/kinetic/lib:... /path/to/binary [args]

This way it can be used both by rosrun and within launch files in a quite portable way:

$ rosrun --prefix "rosrun bebop_driver bebop_env_loader" bebop_driver bebop_driver_node

or

<node pkg="bebop_driver" name="bebop_driver" type="bebop_driver_node" launch-prefix="$(find bebop_driver)/scripts/bebop_env_loader">

I did some testing and it seems to be working fine. Is this the solution you were suggesting?

Topic		Replies	Views
Reproducible builds General	4	1086	August 18, 2016
Catkin_virtualenv ROS Projects	0	1814	April 1, 2018
About running catkin_make	1	1476	May 24, 2018
Catkin_make error building package on machine after image install OpenEmbedded	8	9036	May 31, 2020
Catkin_pip v0.2 released General release	2	898	March 30, 2017

Is it a good practice to force `RPATH` to be embedded in the installed targets?

Related topics