Preparing for Melodic sync 2020-06-09

clalancette · 2020-06-09T15:49:50.963Z

There is currently one regression and 121 packages ready to sync. Except for fixes for the regression, I plan to start holding packages Wednesday, June 10, with a sync later next week.

@smac The one regression in Melodic right now is in slam_toolbox. How would you like to fix that; a revert to an earlier version, or a fix forward?

Please comment here if there are any other outstanding issues I should know about before performing the sync.

smac · 2020-06-09T17:41:29.497Z

@clalancette I noticed that along with a stream of build failures -> successes that have been oscillating in the build farm this week so I didn’t think much of it. There have been no PRs since the last sync so I’m not sure why this is failing.

Here is an example job, anything stick out to you as a problem? http://build.ros.org/job/Mbin_uB64__slam_toolbox__ubuntu_bionic_amd64__binary/46/consoleFull I don’t see anything here actionable. The only 2 things I can find are some pthread not found error which I’ve seen in other jobs before and some Assertion failed: GTEST_LIBRARIES (value is '') which nothing has changed in relation to.

Agreed on fixing it, but its not clear to me what’s wrong to fix.

clalancette · 2020-06-09T18:33:35.359Z

smac:

The only 2 things I can find are some pthread not found error which I’ve seen in other jobs before and

Yeah, you can ignore the pthread thing. It’s part of the debug output of CMake when other things fail before it.

smac:

some Assertion failed: GTEST_LIBRARIES (value is '') which nothing has changed in relation to.

That seems to be the problem to me. I haven’t looked; is it possible that slam_toolbox is missing a dependency on gtest somehow?

smac · 2020-06-09T18:48:32.176Z

That does appear to be the case, but a question is why does this cause an issue now? There have been 2 (maybe even 3) releases in Melodic since that was added without a failure over the last 9 months since gtest was introduced.

Anyhow, I’ll add that to the package.xml and see if that fixes it up, but I’m not hopeful. Edit: PR https://github.com/ros/rosdistro/pull/25388

tfoote · 2020-06-09T19:13:50.341Z

This is the first build that failed. http://build.ros.org/job/Mbin_uB64__slam_toolbox__ubuntu_bionic_amd64__binary/36/

Looking through that list of triggers it looks like there were a couple of recent releasese.

tfoote@snowman4:/tmp Last: [0] (0s Seconds)
$ grep Msrc_uB triggers.txt | sort | uniq
Started by upstream project Msrc_uB__catkin__ubuntu_bionic__source build number 7
Started by upstream project Msrc_uB__message_filters__ubuntu_bionic__source build number 7
Started by upstream project Msrc_uB__python_qt_binding__ubuntu_bionic__source build number 7
Started by upstream project Msrc_uB__rosbag_storage__ubuntu_bionic__source build number 5
Started by upstream project Msrc_uB__rosbag__ubuntu_bionic__source build number 5
Started by upstream project Msrc_uB__rosbash__ubuntu_bionic__source build number 5
Started by upstream project Msrc_uB__rosclean__ubuntu_bionic__source build number 5
Started by upstream project Msrc_uB__roscpp__ubuntu_bionic__source build number 6
Started by upstream project Msrc_uB__roslaunch__ubuntu_bionic__source build number 6
Started by upstream project Msrc_uB__roslib__ubuntu_bionic__source build number 5
Started by upstream project Msrc_uB__rosmake__ubuntu_bionic__source build number 6
Started by upstream project Msrc_uB__rosmaster__ubuntu_bionic__source build number 7
Started by upstream project Msrc_uB__rosmsg__ubuntu_bionic__source build number 7
Started by upstream project Msrc_uB__rosnode__ubuntu_bionic__source build number 5
Started by upstream project Msrc_uB__rosservice__ubuntu_bionic__source build number 5
Started by upstream project Msrc_uB__rostopic__ubuntu_bionic__source build number 5
Started by upstream project Msrc_uB__roswtf__ubuntu_bionic__source build number 5
Started by upstream project Msrc_uB__topic_tools__ubuntu_bionic__source build number 5
tfoote@snowman4:/tmp Last: [0] (0s Seconds)
$ grep Msrc_uB triggers.txt | sort | uniq | wc -l
18

Which appears to follow the following releases:

github.com/ros/rosdistro

ros_comm: 1.14.6-1 in 'melodic/distribution.yaml' [bloom]

ros:master ← dirk-thomas:bloom-ros_comm-0

opened 07:39PM - 29 May 20 UTC

dirk-thomas

+1 -1

github.com/ros/rosdistro

catkin: 0.7.26-1 in 'melodic/distribution.yaml' [bloom]

ros:master ← dirk-thomas:bloom-catkin-1

opened 06:21PM - 21 May 20 UTC

dirk-thomas

+1 -1

github.com/ros/rosdistro

ros: 1.14.9-1 in 'melodic/distribution.yaml' [bloom]

ros:master ← dirk-thomas:bloom-ros-2

opened 09:38PM - 20 Apr 20 UTC

dirk-thomas

+1 -1

github.com/ros/rosdistro

python_qt_binding: 0.4.2-1 in 'melodic/distribution.yaml' [bloom]

ros:master ← dirk-thomas:bloom-python_qt_binding-0

opened 05:54AM - 29 May 20 UTC

dirk-thomas

+1 -1

Looking at the changelogs the change that looks to touch this code path most closely related to the failure:

github.com/ros/catkin

Stops catkin from trying to find C++ libraries if not needed

ros:noetic-devel ← BryceStevenWilley:handle_cmake_none_1080

opened 01:50AM - 30 Apr 20 UTC

BryceStevenWilley

+30 -2

smac · 2020-06-09T19:23:12.231Z

tfoote:

Looking through that list of triggers it looks like there

Awesome, for my future ability to debug issues like this myself, how do I get to a place to be able to search for these triggers – or is this part of the internal OR toolset? Is there some documentation surrounding this?

smac · 2020-06-09T19:52:25.565Z

Adding gtest dependency didn’t seem to fix it http://build.ros.org/job/Mbin_uB64__slam_toolbox__ubuntu_bionic_amd64__binary/48/console. trying to add rosunit as well. New release to follow.

Edit: this isn’t as simple as I thought unfortunately, it might take me a bit to figure this one out.

brycew · 2020-06-09T20:00:40.945Z

Adding dependencies I don’t think should solve the problem. If by the time that the catkin package is found CMake hasn’t deduced that C++ is a language, then it won’t include gtest.

What certainly would solve the problem is adding project(slam_toolbox LANGUAGES CXX) here: https://github.com/SteveMacenski/slam_toolbox/blob/fba7efd47531def4d0255ce43f0bcd75a8aa2da8/slam_toolbox/CMakeLists.txt#L2. You shouldn’t have to do this, but I honestly have no idea if the buildfarm environment is effecting what CMake is doing under the hood.

tfoote · 2020-06-09T20:15:29.496Z

smac:

Awesome, for my future ability to debug issues like this myself, how do I get to a place to be able to search for these triggers – or is this part of the internal OR toolset? Is there some documentation surrounding this?

The triggers are directly from the public Jenkins page I linked to at http://build.ros.org/job/Mbin_uB64__slam_toolbox__ubuntu_bionic_amd64__binary/36/

I copy and pasted the body into triggers.txt and above you can see my invocations of grep, sort, and uniq to filter them down. Then I just cross referenced those source builds to the changes in the rosdistro. And then browsed through the changelogs. Nothing magic, but a bunch of manual work.

With a core change like this the trigger lists are pretty long and the grepping is necessary. Often it’s just immediately readable. For example your most recent release caused a single trigger: http://build.ros.org/job/Mbin_uB64__slam_toolbox__ubuntu_bionic_amd64__binary/48/

If this caused a regression it would be easy to figure out where it came from.

smac · 2020-06-09T20:22:21.967Z

brycew:

What certainly would solve the problem is adding project(slam_toolbox LANGUAGES CXX) here

Doesn’t work – http://build.ros.org/job/Mpr__slam_toolbox__ubuntu_bionic_amd64/106/console still fails with the same Assertion failed: GTEST_LIBRARIES (value is ''). Commit bump to 1.1.5 by SteveMacenski · Pull Request #206 · SteveMacenski/slam_toolbox · GitHub

Mhm this is fun. I’m also staring at the costmap_2d test setup and I can’t find a place where we deviate.

Edit: I’m just disabling tests for melodic for now. There’s only a few tests for some experimental features not used in the main nodes. Its not worth blocking the rest of the package for. I’ll see if this reappears in Noetic and I’ll fix there if necessary & reenable tests in melodic.

dirk-thomas · 2020-06-09T21:47:13.418Z

brycew:

What certainly would solve the problem is adding project(slam_toolbox LANGUAGES CXX) here: https://github.com/SteveMacenski/slam_toolbox/blob/fba7efd47531def4d0255ce43f0bcd75a8aa2da8/slam_toolbox/CMakeLists.txt#L2.

This is not necessary. The default languages are C and CXX - see the CMake documentation: https://cmake.org/cmake/help/latest/command/project.html

Also catkin already exports the gtest dependency (https://github.com/ros/catkin/blob/afb0e2a2ae8c0c34c17a2df7b8e1219792246bf1/package.xml#L34-L35). So since your package build depends on catkin that already ensures that gtest is available.

Also based on the build output gtest is certainly installed:

-- Found gtest sources under '/usr/src/googletest': gtests will be built

My best guess is that the add_subdirectory() call including a separate CMake project which finds catkin itselfmight be the culprit. I would suggest to make sure thatfind_package(catkin REQUIRED)` is called before that in the main CMake file.

clalancette · 2020-06-12T13:38:12.702Z

The sync is out.