Wire Breaking Change in rmw_zenoh for ROS 2 Jazzy

ROS ROS Announcements and News
,
5

Hi @peci1,

I want to start by saying we agree that in-release stability is a cornerstone of the ROS ecosystem, and we apologize for the confusion and frustration this situation has caused. I understand why this change was unexpected, and I appreciate you bringing your feedback to us!

Let me first provide some context and then clarify the sequence of events.

Our Rationale for Backporting

Until its promotion to Tier-1 in Kilted, rmw_zenoh was under very active, rapid development. We made no formal stability declarations for it on Jazzy; in fact, our first binary release for Jazzy was only in January of this year. We were transparent in that announcement that it was not yet a stable, core component.

Our primary goal for the Jazzy binaries was to encourage community testing and gather feedback. The feedback we received, particularly regarding performance on resource-constrained systems using zenoh-pico, was instrumental. This led directly to the serialization improvements you’ve noted. Our decision to backport this breaking change was intentional—we believed the significant benefits for performance and embedded interoperability for all users outweighed the risk, given the package’s declared development status at the time.

Clarifying the Sequence of Events

So just to lay things as they are (hopefully I read all the threads about it correctly) : the Jazzy and Humble backports were merged by mistake, right? No big discussion, no announcement… Then “thanks” to a buildfarm bug, people got new version on arm64 and old version on amd64, which caused mayhem. So the new version was reverted from buildfarm, but the main reason was OpenCV challenge (otherwise, I see that probably, instead of reverting, the action would have been to fix the amd64 version to the new one). And then Marco decided (where, when? it’s not documented anywhere? or was it the PMC meeting with still pending notes?) that he will re-release the new version for all platforms in the next sync.

I’d like to clarify a couple things in your message.

  1. No Mistake in Backporting: The backporting of these changes was an intentional decision. As a nascent project under heavy development for Kilted, we chose to include these optimizations in Jazzy so that users could benefit from them. The changes were never reverted in the source branches.

  2. The Revert was Intentional and Proactive: We did create a 0.2.4 release of rmw_zenoh with the breaking changes (ros/rosdistro#45285). However, we proactively reverted it before the sync went out (ros/rosdistro#45556). This was done for two key reasons:

    • To ensure stability for the OpenCV challenge.
    • Crucially, to give us time to write a proper announcement (this post) rather than having the change land silently. We realized that even for a non-Tier-1 package, this level of breaking change required a heads-up.

  3. The Buildfarm Issue: The fact that different architectures (arm64 vs. amd64) temporarily had different versions was an unfortunate and independent buildfarm mishap, which you can read about here. This incident complicated the situation, but it was not the driver for our decision to pause and announce—it was a parallel problem. tldr; arm64 package did not get a chance to rebuild before the sync went out and the version was automatically downgraded in the subsequent sync.

  4. The Decision to Proceed: The decision to move forward with the release now is a maintainer-level decision, not a directive from the ROS Boss (Marco), whose role is to execute the syncs. We are now proceeding with the release after the OpenCV challenge has concluded and after providing the community notice we failed to provide the first time. The code changes were never reverted in the source branches; we simply paused the binary deployment.

Our Commitment to Transparency and Future Stability

Even today, transparency with the ROS community remains our top priority, which is why we made the effort to create this announcement. We believe that the long-term value this change brings to Jazzy users, particularly for performance and seamless communication with embedded systems via zenoh-pico, outweighs the short-term inconvenience. We are incredibly grateful to our early adopters for their testing and feedback, which has been instrumental in improving rmw_zenoh.

Now that rmw_zenoh is Tier-1 in Kilted, we are committed to maintaining wire compatibility and will be extremely cautious about introducing any breaking changes in future released distributions. As mentioned, this backport to Jazzy was an exceptional case, given how actively rmw_zenoh was evolving leading up to Kilted.

We appreciate your understanding and continued feedback as you switch to ROS 2 and using rmw_zenoh for your robots!

3 Likes