macOS support in ROS 2 Galactic and beyond

Next Generation ROS
,
1

In all ROS 2 releases up to and including Foxy Fitzroy, macOS is listed as a Tier 1 platform. That means that build and test failures on macOS are considered a serious regression and need to be examined/fixed before the code can go into a ROS 2 release.

However, there are several problems with supporting macOS going forward. The problems in order of severity are:

  1. We currently require that users disable SIP to run ROS 2 on macOS.
  2. Useful tools like RViz2 and the RQt family of tools do not run very well on macOS (see this thread for some details about it).
  3. The binary distribution that we provide for releases goes out-of-date almost the moment we release it. That’s because our binary distribution relies on Homebrew to provide system dependencies, and Homebrew changes extremely quickly. Thus, any library that we build against today has a high likelihood of being changed/upgraded in the near future.
  4. Keeping the macOS build machines working and up-to-date takes a non-trivial amount of effort. This can be partly automated away, but because of the previous point this is actually somewhat difficult.
  5. Upgrades from one macOS version to the next have gotten more difficult over time. This is because we treat macOS as “just another Unix”, but that is increasingly not the case. The result of this is that we are often several macOS versions behind.
  6. We now need to get hardware and do a port to the new macOS M1 hardware.

Because of all of the above problems, we are considering dropping macOS from a Tier 1 platform to a Tier 3 platform for ROS 2 Galactic, due in May 2021. As a Tier 3 platform, we would continue to accept fixes for it, but we would not block releases for regressions in it and it would be up to the community to continue to develop and test on it.

To continue Tier 1 support for macOS we need funding for a dedicated maintainer to support the release process and improve the overall end user experience. If you or someone you know would be interested in staffing/funding this project please let us know by responding to the post or contacting us at info@openrobotics.org. If we do not find a dedicated maintainer for macOS, then we will drop macOS support in Galactic to Tier 2 (if there is an organized team of volunteers willing to do regular maintenance), or Tier 3 (best effort maintenance).

10 Likes
2

I am afraid I don’t have the time to maintain the macOS builds, but I would be sad to see it relegated to a secondary status. I know my sadness isn’t enough for finding the resources one needs to do the work, though … (Perhaps dropping Windows could help? Frees up resources and brings world peace a bit closer :wink:)

The GUIs are indeed quite problematic, but perhaps it could remain tier-1 for at least the non-GUI parts? Those work ok on the latest macOS and with SIP enabled, even if there are some complications caused by SIP for the python tests.

Regarding a port to the Apple M1, I’d say the first signs are positive. In case anyone wants to give it a go, below is roughly what it took for me to get test_communication and test_rclcpp up with Cyclone DDS. No guarantee that these notes are entirely correct, but they should be close enough for the intrepid and curious:

Install homebrew (in /opt/homebrew, following the recommendation for doing a native M1 homebrew installation)

brew edit graphviz -> drop librsvg
* works around “brew install rust” failing (rustup will install a working compiler, but I don't know how to make homebrew use it)

install ROS2 dependencies
* most of them anyway (I haven’t tried Qt or other similarly complicated things)
* don’t bother disabling SIP, there’s no real need

git clone https://github.com/numpy/numpy.git
cd numpy
pip3 install . --no-binary :all: --no-use-pep517
(see https://github.com/numpy/numpy/issues/17807#issuecomment-733863375; better do it outside ros2_ws)

Edit src/osrf/osrf_testing_tools_cpp/osrf_testing_tools_cpp/src/memory_tools/vendor/bombela/backward-cpp/backward.hpp
  #elif defined(__aarch64__) -> add && !defined(__APPLE__)
add another case
  #elif defined(__APPLE__) && defined(__aarch64__)
  error_addr = reinterpret_cast<void *>(uctx->uc_mcontext->__ss.__pc);
(no idea if that suffices to make it work, but it builds)

Start the build:

colcon build --packages-up-to rmw_cyclonedds_cpp test_communication test_rclcpp --cmake-args -DCMAKE_BUILD_TYPE=Debug -DCMAKE_PREFIX_PATH=`brew --prefix` -DOPENSSL_ROOT_DIR=`brew --prefix openssl`

This will fail because of an issue in "mimick", but the sources only show up during the build. Edit:

build/mimick_vendor/mimick-ext-prefix/src/mimick-ext/src/asm/trampoline-aarch64.S
* add .align4 for mmk_trampoline, mmk_trampoline_end
* prefix mmk_trampoline and mmk_trampoline_end with an “_”

then restart the build.
3 Likes
3

I agree with you. Personally, it would be great to just get a kind of minimal installation of macOS similar to ubuntu’s ros-*-ros-base. In fact, installing just the base tools compiles much much quicker and doesn’t require SIP to be disabled. Here is how I used to do it. We could probably just create a vcs file that just contains the base packages.

I’m currently waiting for them to release a 16" MacBook Pro with an Apple Silicon chip to attempt building on that.

1 Like
4

It’s understandable that macOS is probably not the biggest priority for Open Robotics, but I think at least some of the problems listed can be overcome / mitigated:

1. I have built ROS 2 on macOS from source without needing to disable SIP. Is SIP an issue only for binary installs?

3. Couldn’t you create your own tap where you pin packages to a supported version? Maybe combine this with a custom version of brew called, say, ros-brew that is installed in /opt/ros2/? You can set up your own binary packages as well. All of this could be done with MacPorts as well.

5. Major macOS versions change about once a year. Isn’t that roughly similar to the Ubuntu release cycle?

6. I believe you can create universal binaries on Intel machines. If not, it’s only a matter of time before AWS M1-powered instances will be available.

2 Likes
5

We have started building out ROS 1 packages as conda packages for all three platforms (OS X, Windows and Linux). Additionally, the conda packages work on many Linux distributions that ship with a recent enough GLIBC. The RoboStack packages for ROS Noetic and Melodic are here, and on Linux we managed to build everything up to ROS Desktop Full. We do have RViz and more on OS X and Windows, as well.

The RoboStack project relies heavily on automation. We have a package-recipe generator called “vinca” (started by Sean Yen) that automatically generates recipes and then pushes those recipes to “build branches” where they are automatically build using (generated) Azure pipelines for OS X, Windows, Linux and Linux-ARM64. You can find the “magic” here: github.com/robostack/ros-noetic

Besides me, Tobias Fischer and Silvio Traversaro have been doing a lot for this effort. We use the base stack from the conda-forge distribution. Here Silvio explains the benefits of conda really well: https://github.com/robotology/robotology-superbuild/pull/484#issue-501112882

In the future, we’d really like to make the installation of binary ROS 2 packages using conda just as easy. If anyone is interested in helping out, that would be great. We have an open gitter chatroom at gitter.im/RoboStack/Lobby and we did the initial steps for Foxy here https://github.com/robostack/ros-foxy

6 Likes
6

First, thanks to those of you who have responded so far. It is sometimes hard for us to tell who is using the software in which way, so this conversation is valuable.

As a general reply to the comments above: I am certain that the technical issues we currently have with macOS can be solved. The question before us is: who is going to invest in doing that work, and is the return on investment worth it?

Here are some reasons that I think the investment is not worth it:

  1. Because of some of our current problems, I have a hard time seeing how any users could use the binary package of macOS. Thus, all of the users are probably building from source, which immediately limits the number of people who are willing to go that far.

  2. We don’t see a whole lot of bug reports that come in specifically about macOS. Doing a query on github, there were 88 issues opened in the last year that have “macOS” in them. Of those, the ones that were reports specifically about macOS problems (and not opened by an Open Robotics employee) number 23. Compare that to the ~800 issues that have overall been opened by non-Open Robotics employees in the last year, and it is a very small percentage. (There are lots of problems with these numbers, so this is just for a rough comparison)

  3. For downstream users of ROS 2, I think it is reasonable to assume that macOS isn’t deployed on a robot. Thus, the main use is as a debugging/development terminal. But if that’s the case, then I think it is essential that tools like RViz2 and RQt work well, and they currently don’t.

And just to be totally clear: Tier 3 support means we would still accept fixes that pertain specifically to macOS. We just wouldn’t block PRs or overnight CI on bugs in it.

Below are some more specific responses to the replies above.

I think that is hard to communicate to users. And it still doesn’t address the ongoing maintenance of the macOS build machines, and the lack of a useful binary distribution. Nor of the fact that we need to now support two separate architectures for macOS.

Yeah, I actually don’t think the change to the M1 processor will be a huge deal. It’s the stuff that surrounds it: upgrading to Big Sur, building binary distributions for both the M1 and the older x86 macOS, and getting/maintaining hardware.

That seems like a reasonable thing to do in general. The thing that has kept us from doing it in the past is that it is a duplicate of the “variants” in GitHub - ros2/variants: Variants for ROS 2 (implemented as ament packages). But maybe we can figure out a way to have one source of truth for both of those things.

Yeah, those are some good ideas, particularly about creating our own tap. I hadn’t considered that before. If we are going to keep it as Tier 1 support, then I think we would look into some of those.

This is another good idea. If there is a set of community people who can get ROS 2 into conda, that would be a viable path towards keeping macOS working with ROS 2 (in a much nicer-to-install way, as well).

2 Likes
7

I recently (over one year) switch from Linux to macOS because in my community a lot of colleagues are using mac as they are not geeks. We may gain them over ROS if it is easy to install. I agree that the current situation with brew and the source install is exactly the opposite. Conda seems to be a nice technical solution for macOS and may increase the support.

3 Likes
8

For what it’s worth:

Docker introduced a public preview/beta here for the new M1 Mac.
I was able to test nav2 on foxy with this. Worked really great so far.
What I did to reproduce:

  • Pulled some foxy arm64v8 pre-built docker containers. (thanks @ruffsl I guess?)
  • Followed the nav2 instructions to build nav2 from source ontop of a ros-distro-release
  • Increased the RAM for the docker a bit (as I had some strange failures)
  • Run colcon test --packages-select nav2_system_tests (includes some full blown tests with simulation)
  • All tests passed

As this is not directly part of the Tier-1 discussion, I also wanted to add that some ros-distro-base package for Mac would be nice to have for smaller projects/plugins alongside things living in docker or real robots. For me this pretty much boils down to nice GUI/interaction tools (at least on my site).

Is there anyone that has Macs integrated into their robots? I have to admit that working (building, installing, colcon tests) 3 hours on ROS dockers and still having 90% battery is impressive… :smiley:

But mostly on my wish list would be a nice RVIz that runs on MacOS, so I don’t need a Ubuntu VM just for that and have to deal with messy network configurations.

Maybe it would also be possible to provide nice guides that also let RVIz and other tools run in docker and just forward the Xserver?

At least for me, Mac OS is a really nice Interface into my computer. But if the stuff underneath is called docker or bare-metal, I would not care too much. UX outweighs running everything native in my case.
(This is probably a different story for older hardware, but I’m running this basically on a MacBook Air… ARM future processors are just nuts…)

2 Likes
9

I maintain the osrf/simulation to with gazebo and ignition formulae, and it is a fair bit of work, especially managing binary (bottle) rebuilds on an arbitrary schedule due to the rolling nature of homebrew-core. Yes there are some versioned formulae we can pin, but it is not as well supported.

A first step towards making a tap for ros2 would be getting working formulae for at least one DDS implementation.

1 Like
10

FYI, the navigation2 repo includes a reference Dockerfile for building the project from distro images:

https://github.com/ros-planning/navigation2/blob/main/.dockerhub/distro.Dockerfile

You could also resort to using a VNC client over setting up an XQuartz X11 server on MacOS:

I hope stuff like webviz will mature to help ros2 visualization tools circumvent multi platform GUI issues.

5 Likes
11

If it is interesting for you, it is possible to integrate in a docker the novnc client.
It is providing a javascript vnc client which can be started in your browser.
Then everything is integrated in your browser and you don’t have to install/run the XQuartz
I used to teach remotely an introduction to ROS and it worked on my macOS Catalina.
The docker file is here:

Note that it is part of the ROSIN project ROSinOccitanie:

A snapshot here:

GazeboTurtlebot3OnMacThroughFirefox
GazeboTurtlebot3OnMacThroughFirefox3584Ă—2240 558 KB

4 Likes
12

Apple M1 (Arm) Mac is rockstar ROS dev desktop once sorted, faster than AMD Ryzen 3700X for 1KB and larger messages because M1 has more memory bandwidth. Apples-to-apples comparison of Zenoh Rust on M1 red and Ryzen blue. More on Zenoh in the ROS Rust WG Nov 18th recording & slides

image
image1404Ă—936 175 KB
image
image2112Ă—1268 381 KB

5 Likes
13

Has there been updates on the topic? I’m kind of curious how I can help after I submit my dissertation.

1 Like
14

There have been some minor updates. https://github.com/ros2/ros2_documentation/pull/968 proposes a way to work on macOS without disabling SIP. I have no idea how well that works, but it is interesting.

Given that we’ve had no offers of substantial help for macOS, we’re most likely going to mark it as Tier 3 for Galactic: https://github.com/ros-infrastructure/rep/pull/291 .

That being said, if you’d like to make ROS 2 a good experience for macOS users, my suggestion is to work on making the tooling work better on macOS. In particular, rviz2 and the rqt family of tools need to work really well, so that’s where I’d concentrate my effort.

4 Likes
15

Cool, I’ll try and concentrate on this once I finish my dissertation.

1 Like
16

Also I wanted to mention to the group that I was able to get ROS 2 Foxy working on macOS Big Sur (with SIP enabled). Source installations from my experience seem to still work seamlessly since Catalina.

There were major problems jumping from Mojave to Catalina with the Dashing and Eloquent releases, but the compatibility issues have lessened after Foxy with the subsequent versions of macOS. I’ll try and submit some pull requests to fix issues present in this thread you linked earlier as well.

2 Likes
17

Are you able to run tests as well as build successfully? Much of the problem with SIP had to do with tests relying on environment variables that SIP does not allow users to control.

2 Likes
18

So far, I haven’t run into any build issues. However, I don’t do much work with environmental variables that SIP controls. Do you know what some of these environmental variables may be?

1 Like
19

We’ve been working really hard on cross-platform packaging for macOS and Windows. So far, we’ve packaged many of the ROS1 noetic packages, using a different installation tool called conda and conda-forge as our base repositories. We also have working packages for RViz and the entire rqt stack.

You can read more about this effort here: Cross-platform conda packages for ROS | by Wolf Vollprecht | robostack | Feb, 2021 | Medium

We are very much looking forward to work further on ROS 2 Foxy and we’ve started towards this in this repo: GitHub - RoboStack/ros-foxy: ros foxy vinca files

We have RViz 2 working on Linux, but not yet on Windows and macOS, mostly because we ran into some difficulties with building the vendored packages. But with some time investment, that should be fixed quite easily.

It would be really cool to work on this with the ROS community (and the conda-forge community, that packages thousands of data science / scientific packages for Windows, macOS and Linux).

6 Likes
20

I decided to bump this thread because I thought it was relevant.

We could use help reviewing the macOS tutorials for Ignition Edifice.

3 Likes