| ROS Resources: ROS Homepage | Media and Trademarks | Documentation | ROS Index | How to Get Help | Q&A Help Site | Discussion Forum | Service Status |
I am considering building some ROS packages on Mac regularly for catching up broken build/tests on this platform.
The first step would be to do a prototype (https://www.macstadium.com/ looks promising). If it goes OK, this needs to be integrated with ROS Buildfarm.
What do you think - useful, doable?
The main issue is the need for a reproducible build, which is hard to do because macOS doesn’t have docker and no one (outside of Travis-CI that I’m aware of) has automated use of VM’s with checkpoints to emulate a “clean starting state” for each build. This is a crucial step for building packages in my opinion, but AFAIK (it could have changed) macstadium.com only provides hosting on mac minis, not the automation for repeatable jobs. Travis-CI does provide this as continuous integration, and I believe some people are using (abusing?) this to build Homebrew bottles @scpeters might know more about that then me at this point.
If you’re interested in trying anyways, you’ll want to catch up on the existing efforts:
Those two links are the best summaries I can find, but efforts so far have stalled, as just keeping it building from source on macOS with Homebrew has become of a burden than the community appears to be willing to bear.
In order to be integrated into the “ROS buildfarm” (by this I assume you mean build.ros.org), we’ll need a way to provision machines, generate artifacts on a per package basis (bloom does this for debian and fedora packaging), and extend the build farm itself to create the necessary jenkins jobs to actually execute the packaging on the build machines and upload the result somewhere. Once you get to this point, @nuclearsandwich and/or @dirk-thomas might have more to add.
Just glancing at macstadium.com again, it looks like they now support provisioning VM’s on demand and use VMware under the hood:
https://www.macstadium.com/cloud/
Also, I’ll mention that Homebrew does it’s packaging (last time I checked, again could have changed) not by having VM’s, but by deleting the homebrew directory each time before a new build starts. So this is more like ensuring you completely cleanup the system before finishing a job, rather than discarding the changes and going back to a previous docker image or VM snapshot. This might also be possible, but in my experience it’s harder for ROS since we not only install things from Homebrew, but also things from pip for Python dependencies (also possibly Ruby’s gem, etc…).
I maintain the osrf/simulation tap where I build bottles for gazebo and its dependencies. We use a few mac minis with jenkins for the bottle builds.
I have seen folks attempt to build bottles with travis-ci, though I’m not sure what the current status is:
The biggest challenge with maintaining homebrew bottles on a non-core tap is that your bottles can break at any time if a new bottle is built for one of your dependencies (such as boost or protobuf). I have a daily job that tests one of our gazebo bottles, and then we manually rebuild them if we notice that job failing due to an outdated dependency. We try to stay on top of it and don’t see too much downtime, but this is not scalable. I wouldn’t want to be in charge of keep homebrew bottles for all of ROS working.
My initial focus is just building and testing. Packaging will come later if building and testing is successful. I believe both things is going to be useful - knowing that a particular change causes a build break or a test failure on Mac is a useful information for package maintainers.
Yes, doing a clean (or more generally determistic) build is a concern for any CI process, and this work should get some data on the topic.
If you want to avoid building “everything” at once (including all recursive dependencies) you will need packaging to provide the binary packages for the dependencies.
If you want to avoid building “everything” at once (including all recursive dependencies) you will need packaging to provide the binary packages for the dependencies.
Dirk is exactly right. Even if you only want to use travis for CI (which is easy enough to set up), once you have enough dependencies, your build will timeout if you don’t install them from pre-compiled binaries (bottles in the homebrew case).
According to https://docs.travis-ci.com/user/customizing-the-build/#Build-Timeouts the
job’s timeout is 50 minutes. When I was following http://wiki.ros.org/kinetic/Installation/OSX/Homebrew/Source on my computer
for rosinstall_generator desktop ... I think the whole process was done in less than 50 minutes and it includes quite a few packages being built.
I plan to start with something small, see how it works, and then extend if successful.
I think most of the key points have been covered. Any automated build and testing will benefit the project but in order to be widely useful, it needs to be reproducible and make efficient use of resources. You might find the ros2/ci repository interesting. It shows how we build and test ROS 2 from source on https://ci.ros2.org which has several windows and mac machines attached to it. As the project grows we’re starting to feel the pain of this approach as CI times climb.
We would definitely encourage contributions for MacOS support into the ros_buildfarm repository; giving community members the opportunity to deploy it. However since MacOS is not an officially supported platform, it is a small fraction of the user base, and costs for Apple hardware or compute time are significantly higher than other platforms, deploying MacOS to the official infrastructure is not a decision that can be made based solely on the technical capability to do so. Once we have the capability, if an organization would like to sponsor efforts to deploy and support the ongoing maintenance of MacOS infrastructure it could be reconsidered.
I started with Travis, picked up rosdep, and added the building and testing on Mac. The link to the CI page. Then I opened https://travis-ci.org/ros/rosdep to discover that the last build in Travis happened 5 years ago. That’s many changes ago.
Is Travis an option? Having VMs for free for OS projects sounds like a good deal to me.
For smaller packages like rosdep Travis is definitely an option that’s already in use. You’ve found the Travis CI history for the old repository location. It was moved to
So you can see the current status is here: https://travis-ci.org/ros-infrastructure/rosdep
It looks like you’ve already found it, but for others reference there’s now a [PR under review}https://github.com/ros-infrastructure/rosdep/pull/591) for rosdep testing on macOS.
Do you have any information on how you plan to address this pain?
I think what @dirk-thomas said: provide binary packages for the dependencies. You can see my other comments in this thread about that current status for macOS.
I am focusing right now on building python projects on macOs. Example: https://github.com/ros-infrastructure/rosdep/.
.travis folder contains few things, and it will be pretty much the same for every project. I am thinking about moving the content of .travis into a separate repo and adding it to existing repos as a submodule.
What do you think about using submodules for this case?
Normally I dislike using submodules, but given that it’s only required for travis, I think it would be ok. At least I can speak for the repositories I maintain, other maintainers might not feel that same way.
I too dislike submodules, and although they would only be necessary for travis if they are there by default it can be confusing/disruptive for developers. If it’s only necessary for travis it would be simple to package that functionality into a resource that travis would fetch/install during the setup phase, like it does like any other testing/tooling dependencies which are installed via the .travis install segment.
Thanks for the ideas.
MacOS CI for ROS packages would be a great addition, even for core repositories like ros_comm (see e.g. https://github.com/ros/ros_comm/issues/1357#issuecomment-378025569).
You might want to have a look at https://github.com/ros-industrial/industrial_ci, which can be easily added to other repositories without submodules, just loke @tfoote mentioned. Maybe it even makes sense to integrate mac support there, since this is already in use by many packages it seems.
The same goes for ros_buildfarm. Afaik neither of the two offers support for macOS atm. For ros_buildfarm it is certainly planned to also test on macOS and Windows in order to satisfy the need in ROS 2. Once that works on Jenkins it should be feasible to run the same on Travis (with macOS) and AppVeyor.