Announcing open-sourcing of ROS 2 Parameter Configuration package

[jak]

We at Karelics are excited to announce open sourcing of our Parameter Configuration ROS 2 package

GitHub - Karelics/param_configuration: The tools to make ROS 2 parameter...

The idea for open-sourcing came from this thread in Discourse, as we saw that many ROS users are looking to solve similar problems as we have solved with this package.

Parameter Configuration simplifies parameter handling for single and multi-robot setups by providing two main features:

1. YAML placeholders: Adds support for variables and placeholders in YAML files, which helps to avoid parameter duplication and often eliminates the need to rewrite parameters in launch files.

2. YAML file overlaying: Allows hierarchical parameter loading. It is possible to set device level (robot instance) and robot model level parameters to override default ROS 2 package parameters, making it easier to handle parameters in multi-robot setups.

The package is still in experimental stage, so we’d love to hear about your experiences with it and receive feedback on how we can make it even better. We are calling on the community and companies to join in and collaborate to develop it further. If you have projects where you would like to use this package, we are open to providing support!

[doganulus]

Thank you for sharing your configuration library. Hierarchical parameter loading is a very useful feature, and I find it an exciting topic. I have a few comments and feedback on the design after reading your README and open questions.

On More Layers

To support more layers, one idea could be defining a PATH variable such as

ROS_CONFIG_PATH="~/cfg_dir/device/:~/cfg_dir/model/"

This is very good because we do not need to dictate a particular (single) config directory, and we can easily define config directories that are conventionally spread over the filesystem. For example, this layout uses conventional /opt, /run, and /etc directories.

ROS_CONFIG_PATH="/etc/<namespace>/config:/run/<namespace>/config:/opt/<namespace>/config"

And users can extend/alter them easily:

export ROS_CONFIG_PATH="~/<namespace>/config:${ROS_CONFIG_PATH}"

Substitutions and Evals!

Regarding substitution and eval functionality, I think it is an overkill. If I use your !eval statements, my parameter files would be useless without your tooling. This is something I never want. And this is a big liability for your development too. Now you must carry the responsibility of not only your own company but everyone who uses your tooling. Many problems in open-source projects start from such decisions (ROS I am looking at you).

If you really insist on such advanced features, the template language must be something standardized and widespread, like the GNU utility envsubst or jinja at least. Other utilities and templating languages may be considered as an alternative but it is better to avoid developing a custom format or relying exclusively on Python evals. The value I see here is in the configuration hierarchy, not in the templating. This is even more if we think that Google (jsonnet) and Apple (pkl) already provide extensive solutions for configuration templating.

Testing for Direct Node Execution and Beyond

A not-so-much-documented feature of ROS is the ability to execute node executables directly. I remember it took some time for me to figure out how to pass parameters, but it is as follows:

node-exec --ros-args --params-file my_param.yml

Direct execution does not perform variable substitution, which is a feature of ROS launch. This is another reason I suggest not using $(find-pkg ...) statements in parameter files, but sometimes I see this is abused. Therefore, I want to emphasize that any parameter solution should be functional when executing nodes directly without relying on Launch. I think your library is fine as I can use it directly like this:

config print --config-directory my_cfg_dir /home/user/param_config.yaml
node-exec --ros-args --params-file /home/user/param_config.yaml

It is, of course, better that ROS node executables have such functionality built-in, perhaps via utilizing ROS_CONFIG_PATH. Maybe this would evolve into a more fundamental solution.

Can it be done via a new rcl-yaml-param-parser? Any comment?

[jak]

Thank you for the feedback @doganulus, it is really valuable to us! I’m happy to hear that you see the value of the hierarchical parameter loading.

I like the idea of using env variable to declare the folder locations. It would give freedom for the user to decide how to store the configs, and it also makes the path declaration very simple. From the technical design point of view, the package provides also a possibilty to have different layer types, the available ones now being “RosPackageLayer” and “FileLocationLayer”. New custom layer types can be implemented, which is why we have previously considered to have a config file that describes the number of the layers, layer types, and necessary paths. But we scrapped this idea as it didn’t work well with our directory structure. Maybe a simple env variable as you suggested would be easiest solution!

I share your concerns related to !eval statements and substitutions. They will indeed make these created parameter files only readable with our tool. In our case however, these statements tremendously improved our system, which is why we nevertheless chose to go with this approach. Just to summarize, here are the main benefits it provided us:

• Reduced parameter duplication: Usually we need to set some parameters to have the same value for multiple different Nodes. When modifying the value, failure in changing it in all the places caused unexpected behavior and unnecessary time spent on debugging. Now we can set the parameter just in a single place and read it from there in the other YAML files.
• Launch file quality: In our parameter files we have many parameters that have to be evaluated in runtime, such as env variables and ROS paths. This required us to substitute parameters using ReWrittenYaml from Nav2, creating unnecessary dependency. Also, for some robots we have created a general YAML configuration file that describes the robot dimensions, sensor positions, etc. Previously we would read this YAML file in launch files with our custom Python class, calculate the necessary values (robot radius, footprint, safety zones etc.) and substitute them for all the Nodes that need them. But this method turned out to be complex and difficult to maintain. Now we can do all of this directly in the parameter files using the placeholders.
• Improved Docker deployment: Because we now no longer need to modify the launch files to evaluate parameters in runtime, we can have reusable Docker containers across different robot models. The parameter files are just directly read from the outside of the container.

I agree with you that we should move towards more standardized template languages and investigate if they provide all the features we need to have.

Running the nodes with ros2 run and passing these parameter files has also been an issue to us, as we cannot directly pass them there. This is why we have been mainly launching nodes only via launch files. The config print command right now only prints the configuration in terminal, but I think we should definitely add a command to dump / save the configuration, as you described. It would solve the problem of running the Nodes independently. Thanks for the idea!

What comes to rcl-yaml-parser, I would love to see these features integrated directly into it, as that would make these placeholders usable directly for any Node.

[doganulus]

I understand there are some use cases where we may need templating. Still I think we can solve this problem seperately without committing ourselves to a particular templating technology. Having a well-defined configuration hierarchy may help in that regard.

For example, assume we define a configuration hierarchy, which is configured via ROS_CONFIG_PATH, as follows (from the highest to lowest):

• $HOME/<namespace>/config
• /etc/<namespace>/config
• /run/<namespace>/config
• /opt/<namespace>/config

First, let us agree our packages reside in the following locations after installation:/opt/<namespace>/packages/<pkg_name>

1. The first configuration layer resides under /opt/<namespace>/config:

   • So we symlink /opt/<namespace>/config/<pkg_name> to /opt/<namespace>/packages/<pkg_name>/{config,params} conventionally
   • But also these are easily discardable whenever needed

2. The second layer is dedicated to runtime generated package configuration (from templates):

   • We install templates at /opt/<namespace>/packages/<pkg_name>/templates/config/<node_name>.params.yml.j2
   • At launch, we generate parameter files from templates and place them under /run/<namespace>/config/<pkg_name>
   • This process is handled by a third-party tool or the launcher. We do not commit to a particular templating format but we ensure generated files are at the correct place.

3. The third layer is for host or device configuration.

   • Place host-specific config files under /etc/<namespace>/config/<pkg_name>
   • This is the most suitable place where we can mount external configuration files for containers/device

4. And the user can override all from their home directory.

[jak]

Interesting idea. I think we would need to give it a try to properly see its benefits and possible drawbacks.

If I understood correctly, this approach would basically remove the need for a separate RosPackageLayer, as it would be considered just as a normal FileLocationLayer due to symlinking. The second layer then handles all the placeholders by resolving the template files in the runtime before the ROS Nodes are launched. And the rest of the layers would be just normal overlaying files with overriding parameter values.

Would this approach allow us to use placeholders in the device level configurations?