| ROS Resources: Documentation | Support | Discussion Forum | Service Status | Q&A answers.ros.org |
I support the backport.
The reason is simple: We have already switched to the rosbag2 version from rolling, even though we remain on foxy for the rest of our system. This is because the version currently in foxy not only has performance issues, but is also lacking many important features in the rosbag2 Python API.
thanks all for sharing requirements and opinions 
the plan was not to break the command line usage
i did not mean only CLI but all interfaces exposed to user application.
Many users have moved on to create their own solutions for record and replay or used rosbag - how is this better than breaking the API/ABI and to help the users to update?
i was wondering who Many users here. i was thinking that we cannot exactly know who actually uses…we eventually break someone’s CI or build system with unexpected modification.
leveldb does not solve all of the issues that the rosbag2 had, especially on the transport level.
we always have (will have) issues and improvement, i wasn’t clear on why and how come this improvement is an exception. at least we should have logics to make that decision, unless it would be easy to break the userspace again.
Also please note that Robotec.AI’s report 1 concludes that the sqlite3 performance is now actually slightly better than leveldb.
yes, i am aware of that. which is good thing for everyone. probably we don’t have to maintain the leveldb implementation anymore! (actually we’ve been considering details and scenarios including aarch64 platform) btw, we are not pushing leveldb at all, we just want to have something works for us.
The answer to the latter is certainly no and I vaguely recall rare situations from the former (e.g. indigo ABI break in roscpp after update? - ROS Answers: Open Source Q&A Forum).
API breaks happened before, e.g. Split of joint_state_publisher and joint_state_publisher_gui and the corresponding PR Split jsp and jsp gui by clalancette · Pull Request #31 · ros/joint_state_publisher · GitHub . It confused the user base, but in the end it somehow went through…
appreciate for pointing these out! so after all it happened before. and according to ROS 2 developer guide and https://semver.org/ we need to bump the major version to break the API. that is said it is no hard guarantee? please correct me if i am wrong, this is really important as in architecture and entire build system.
thanks in advance 
Yes, we have had API and ABI breaks before. But as @Dejan_Pangercic says, it is a pretty rare situation and the problem that gets fixed has to be very severe.
That change was meant to be transparent, but I messed up. However, in my defense, nobody tested it out in the testing repository or ever reported a bug until several months later. (anyway, I don’t want to hijack this thread for this)
For the current situation with rosbag2, I’m still on the fence about it. I agree with @Dejan_Pangercic that rosbag2 in Foxy largely doesn’t meet the needs of users. But it does also feel like breaking the API/ABI here is breaking our promise to users. As @kyrofa says; this thread will reach some users, but many of them will not see it at all. So it is hard to determine what the scale of the breakage will be.
I will note that rosbag2 has not yet declared version 1.0, nor has it declared a REP-2004 quality level for Foxy (or yet on master) - so according to the ROS and semver guidelines we won’t have broken any contract and would only need to bump the minor version. I’d like to declare 1.0 and a quality level for Galactic which I think would create a hard exclusion for doing this ever again.
Just making an argument from the “letter of the law” 
- it’s not a final argument.
I see parallels with the thread over here about whether people should be using the most recent distributions or sticking with older, more stable stuff.
FYI, I’m a new ROS user and have been evaluating it for use in a product I’m working on. We’re using the NVIDIA Jetson which only has Ubuntu 18, so I’m stuck on Eloquent. Recently I tried using rosbag2 but the ros-eloquent-ros2bag version doesn’t even implement --loop??
This was really the final straw and I’ve decided to give up on ROS2 and try ROS1 instead, or potentially skip ROS altogether. Releasing incomplete software under official-sounding names like rosbag2 is a great way to make people like me lose trust in the ecosystem.
This seems like a very reasonable solution to the current tension here.
I would also like to add that that change violated one promise (API/ABI) in order to remedy another (No GUI dependencies in the robot metapackage) - Kinda stuck between a rock and a hard place, you’re violating a promise one way or another.
In this case, it’s violating a (slightly less rigid, as @emersonknapp pointed out) promise to remedy usability. I’m not saying the situations are entirely dissimilar, but they are slightly different.
I don’t know the ROS2 landscape well enough to give a well informed opinion here, but given the leafy-ness of the package, my opinion leans towards prefering the break.
I disagree, I think @gavanderhoorn is correct that fragmentation is a worse evil here. Given how currently unusable rosbag2 is and the (reasonable?) assumption that the package is fairly leaf-like, an API break seems more reasonable to me.
I’m in favour of the backport even if it involves breaking API/ABI.
@tomoyafujita mentioned data corruption as one reason to break. I think that missing topics and messages are arguably data corruption issues that need to be fixed.
@gavanderhoorn mentioned bugs. rosbag2 doesn’t work properly, which, like him, I think counts as a bug.
And as @emersonknapp said, the API has not yet reached version 1.0 so the version number doesn’t block breaking changes.
The only thing blocking a breaking change is the rule in ROS that API/ABI does not break within a distribution. I think we need to accept that this rule is not 100% absolute. If there is a strong enough reason, as agreed by the community (which for all intents and purposes can only be defined as “those interested enough to read Discourse and respond”), then in my opinion the break is justified.
I’ll also throw in here that I’m investigating how much of a patch would be needed to break no public APIs. It’s much harder to make that promise about the ABI - but I think we may be in a much better position if we can remove the possibility of breaking anybody’s build.
The only thing blocking a breaking change is the rule in ROS that API/ABI does not break within a distribution. I think we need to accept that this rule is not 100% absolute.
atm, i was thinking that this is hard guarantee to support the products. but now i understand API/ABI occasionally breaks if necessary. and we can detect that with checking the major version of the packages.
I’m investigating how much of a patch would be needed to break no public APIs.
thanks
What about approaching this from the packaging side? I.e. create a new set of packages with the same package name (or a similar one), like ros2-foxy-rosbag2-fixed, and configure these packages to conflict with the original rosbag2 packages and replace them. Then users could choose whether they stay with the broken but compatible version, or whether they install the fixed but incompatible version. And the old version could issue a loud warning in the console telling the users what can be done to get a better rosbag.
I’m not sure yet how this would work with rosdep, which might try to always install the old version, though…
Or another approach - you could set up an additional PPA that would contain just the fixed rosbag. Everybody could then decide what versions of the packages are better for each specific use-case.
I was able to get the API/ABI report running - https://build.ros2.org/job/Fpr__rosbag2__ubuntu_focal_amd64/126/API_5fABI_20report/
Yes, backport to Foxy because racecar! Foxy’s current as-is rosbag2 performance is completely unusable for the 500 students and 28 universities in the Indy Autonomous Challenge. ROS 2 Foxy is the recommended software but Foxy rosbag2 can not cope with the data rates for the 3x flash lidar, 6x cameras, 4x radars, GNSS w IMU, drive-by-wire shown here
Current Foxy rosbag2 also does not work for autonomous cargo delivery and other autonomous driving pilots underway.
Indy Autonomous Challenge and other autonomous driving uses is why Apex.AI, TierIV, Bosch, ADLINK worked with Robotec.AI, ROS Tooling WG to contribute the huge improvements that are now available to Foxy users if backported.
Says who? But to be fair that is what most people assume. @spiderkeys and many others run Foxy on Jetson without issue when built from source. And NVIDIA has released ros2 foxy dockers for Jetson. Yes, this would be good for us to make clear in ros2 docs.
The road to stability is sometimes a wee bit bumpy.
I’d be very happy to have a fully functioning set of common tools in an LTS.
In lack of that, people will keep rushing onto new versions, now having to migrate to stuff they never intended to in the new distro or worse, go full “Not invented here”…
Perhaps what may help reasoning about the severity of this change is the current size estimate of the Foxy userbase?
People “assume” this because it’s clearly documented in REP 2000 -- ROS 2 Releases and Target Platforms (ROS.org)
Thanks for the heads up that Foxy might work!
+1 to backport 
Long time ROS1 user here. By now we have accumulated terabytes of ROS1 bagfiles for debugging, future testing and future possibilities with the lidar data (learning?).
We’re on the verge of using ROS2 for our newest project. Foxy would be obvious choice here since Galactic is still a year away.
Now reading that the rosbag2 system is flaky is really worrisome. I am glad I read it here first.
This could potentially have cost lost of engineering hours to figure out that we lost data, how we lost data and how we should not lose data again.
So for the sake of ROS2 newbies, please allow the current release to be not a regression from ROS1.
Just to be clear; Galactic is 3 months away, not a year. The timeline is here: ROS 2 Galactic Geochelone (codename ‘galactic’; May, 2021)
You are absolutely right: ROS Clock
If this backport doesn’t make it, Galactic might be the way to go instead of the recommended Foxy.
This would be great to get into FOXY. This is the designated ROS2 version for the Indy Autonomous Challange and the autonomous vehicle has lots of cameras, lidars, radars, GPS/IMU, bunches of sensors not to mention a very underdeveloped software stack. All this means lots and lots of data flowing around and needed to be captured at 200 mph. It is our sense the current RosBag2 just won’t be able to keep up and we’ll have to make some tough decisions not to try to get all the data we’d like for post-processing onto the disk. Univ of Hawaii Autonomous racing team - AIracingTech
