Fast Forward Merging rosbag2 master API to Foxy

[matthew-reynolds]

@peci1

API breaks happened before, e.g. Split of joint_state_publisher and joint_state_publisher_gui and the corresponding PR Split jsp and jsp gui by clalancette · Pull Request #31 · ros/joint_state_publisher · GitHub . It confused the user base, but in the end it somehow went through…

I would also like to add that that change violated one promise (API/ABI) in order to remedy another (No GUI dependencies in the robot metapackage) - Kinda stuck between a rock and a hard place, you’re violating a promise one way or another.

In this case, it’s violating a (slightly less rigid, as @emersonknapp pointed out) promise to remedy usability. I’m not saying the situations are entirely dissimilar, but they are slightly different.

I don’t know the ROS2 landscape well enough to give a well informed opinion here, but given the leafy-ness of the package, my opinion leans towards prefering the break.

@33thou

@peci1

I know it sounds like a lot of work, but what about renaming rosbag2 and related packages to something else for Foxy? That would bring the new features to those who want them but keep the promise of stable API/ABI

This seems like a very reasonable solution to the current tension here.

I disagree, I think @gavanderhoorn is correct that fragmentation is a worse evil here. Given how currently unusable rosbag2 is and the (reasonable?) assumption that the package is fairly leaf-like, an API break seems more reasonable to me.

[gbiggs]

I’m in favour of the backport even if it involves breaking API/ABI.

@tomoyafujita mentioned data corruption as one reason to break. I think that missing topics and messages are arguably data corruption issues that need to be fixed.

@gavanderhoorn mentioned bugs. rosbag2 doesn’t work properly, which, like him, I think counts as a bug.

And as @emersonknapp said, the API has not yet reached version 1.0 so the version number doesn’t block breaking changes.

The only thing blocking a breaking change is the rule in ROS that API/ABI does not break within a distribution. I think we need to accept that this rule is not 100% absolute. If there is a strong enough reason, as agreed by the community (which for all intents and purposes can only be defined as “those interested enough to read Discourse and respond”), then in my opinion the break is justified.

[emersonknapp]

I’ll also throw in here that I’m investigating how much of a patch would be needed to break no public APIs. It’s much harder to make that promise about the ABI - but I think we may be in a much better position if we can remove the possibility of breaking anybody’s build.

[tomoyafujita]

The only thing blocking a breaking change is the rule in ROS that API/ABI does not break within a distribution. I think we need to accept that this rule is not 100% absolute.

atm, i was thinking that this is hard guarantee to support the products. but now i understand API/ABI occasionally breaks if necessary. and we can detect that with checking the major version of the packages.

I’m investigating how much of a patch would be needed to break no public APIs.

thanks

[peci1]

What about approaching this from the packaging side? I.e. create a new set of packages with the same package name (or a similar one), like ros2-foxy-rosbag2-fixed, and configure these packages to conflict with the original rosbag2 packages and replace them. Then users could choose whether they stay with the broken but compatible version, or whether they install the fixed but incompatible version. And the old version could issue a loud warning in the console telling the users what can be done to get a better rosbag.

I’m not sure yet how this would work with rosdep, which might try to always install the old version, though…

[peci1]

Or another approach - you could set up an additional PPA that would contain just the fixed rosbag. Everybody could then decide what versions of the packages are better for each specific use-case.

[emersonknapp]

I was able to get the API/ABI report running - https://build.ros2.org/job/Fpr__rosbag2__ubuntu_focal_amd64/126/API_5fABI_20report/

[joespeed]

Yes, backport to Foxy because racecar! Foxy’s current as-is rosbag2 performance is completely unusable for the 500 students and 28 universities in the Indy Autonomous Challenge. ROS 2 Foxy is the recommended software but Foxy rosbag2 can not cope with the data rates for the 3x flash lidar, 6x cameras, 4x radars, GNSS w IMU, drive-by-wire shown here

Current Foxy rosbag2 also does not work for autonomous cargo delivery and other autonomous driving pilots underway.

Indy Autonomous Challenge and other autonomous driving uses is why Apex.AI, TierIV, Bosch, ADLINK worked with Robotec.AI, ROS Tooling WG to contribute the huge improvements that are now available to Foxy users if backported.

[joespeed]

@33thou

We’re using the NVIDIA Jetson which only has Ubuntu 18, so I’m stuck on Eloquent.

Says who? But to be fair that is what most people assume. @spiderkeys and many others run Foxy on Jetson without issue when built from source. And NVIDIA has released ros2 foxy dockers for Jetson. Yes, this would be good for us to make clear in ros2 docs.

[bmagyar]

The road to stability is sometimes a wee bit bumpy.

I’d be very happy to have a fully functioning set of common tools in an LTS.
In lack of that, people will keep rushing onto new versions, now having to migrate to stuff they never intended to in the new distro or worse, go full “Not invented here”…

Perhaps what may help reasoning about the severity of this change is the current size estimate of the Foxy userbase?

[33thou]

@joespeed

Says who? But to be fair that is what most people assume.

People “assume” this because it’s clearly documented in REP 2000 -- ROS 2 Releases and Target Platforms (ROS.org)

Thanks for the heads up that Foxy might work!

+1 to backport

[Timple]

Long time ROS1 user here. By now we have accumulated terabytes of ROS1 bagfiles for debugging, future testing and future possibilities with the lidar data (learning?).

We’re on the verge of using ROS2 for our newest project. Foxy would be obvious choice here since Galactic is still a year away.

Now reading that the rosbag2 system is flaky is really worrisome. I am glad I read it here first.
This could potentially have cost lost of engineering hours to figure out that we lost data, how we lost data and how we should not lose data again.

So for the sake of ROS2 newbies, please allow the current release to be not a regression from ROS1.

[clalancette]

@Timple

We’re on the verge of using ROS2 for our newest project. Foxy would be obvious choice here since Galactic is still a year away.

Just to be clear; Galactic is 3 months away, not a year. The timeline is here: ROS 2 Galactic Geochelone (codename ‘galactic’; May, 2021)

[Timple]

You are absolutely right: ROS Clock

If this backport doesn’t make it, Galactic might be the way to go instead of the recommended Foxy.

[AirGar69]

This would be great to get into FOXY. This is the designated ROS2 version for the Indy Autonomous Challange and the autonomous vehicle has lots of cameras, lidars, radars, GPS/IMU, bunches of sensors not to mention a very underdeveloped software stack. All this means lots and lots of data flowing around and needed to be captured at 200 mph. It is our sense the current RosBag2 just won’t be able to keep up and we’ll have to make some tough decisions not to try to get all the data we’d like for post-processing onto the disk. Univ of Hawaii Autonomous racing team - AIracingTech