Work in progress: Chef cookbooks for ROS build farm configuration

With the release of Ubuntu 20.04 comes the final year of public support for Ubuntu 16.04 upon which the existing buildfarm_deployment is based.
The current buildfarm_deployment scripts were themselves ported forward from an earlier version of puppet to just barely work with Puppet 3.8 on 16.04.
The result of this deferred maintenance is that much more work would be needed to update the existing scripts and required modules to Puppet 5.5 used in Ubuntu 20.04.

Additionally, Open Robotics has been using the open source configuration management utility, chef for recent infrastructure products due to its better record of support for all platforms supported by ROS 2.
Thus we’ve been preparing for our own migration to a 20.04 based ROS buildfarm deployment with the creation of a new chef cookbook.

This work has been done in large part by one of the interns at Open Robotics, Maanasa Kotha.

Over the course of the next month we’ll be running test deployments using this cookbook to identify unported features or configuration details and prepare for a migration of the official ROS and ROS 2 build farms.
We would also like to invite interested community members to check out the cookbook and eventually, the example chef repository, which are roughly analogous to the buildfarm_deployment and buildfarm_deployment_config repositories used by the existing infrastructure.

The jenkins and agent recipes are pretty close to complete but the repo recipe needs more work to get reprepro-updater and its configuration added, along with the recent pulp support for RPMs.

In addition to an attempted direct migration of most features, we’ve built these cookbooks with the express goal of resolving a few longstanding issues with the buildfarm_deployment.

  • Enable reconfiguration of the Jenkins node without losing necessary configuration buildfarm_deployment#160
  • Allow more complex authentication/authorization configuration groovy scripting instead of choosing between managing auth by hand or a single configured admin user.
  • Improved testing of the deployment cookbooks with the Test Kitchen infrastructure testing harness.
  • Allowing a single (large) machine to support all three of the cluster roles for quick or small deployments. (Not yet supported)
4 Likes