Adding Debian as a Tier 2 platform

gbiggs · 2019-09-24T14:17:20.763Z

@tfoote
Following up from some comments during the last TSC meeting and this ancient thread, I’d like to check what tasks need to be done to get Debian Buster as a Tier 2 supported platform for ROS 2 Eloquent.

In particular what infrastructure do we need to set up and what do we need to provide to where for the binary packages to be made available?

miker256 · 2019-09-24T22:23:59.157Z

This would be a good starting point @gbiggs.

https://wiki.debian.org/DebianScience/Robotics/ROS/OnBuster

sousou1 · 2019-09-26T15:46:04.486Z

I am working with @gbiggs for Tier 2 support.

I installed and ran ROS 2 Eloquent with Buster.
So next , I want to create a CI for Buster.

I think the following two are important for Tier 2 support, is it correct?

Create a Debian Agent with ROS Buildfarm and test it with the same frequency as other Tier1 and Tier2.
Provide support for CI failures.

First of all, we plan to build and verify Master, Agent (added Debian), and Repo on private servers.
What should we do as the next step?

And what support should we do for CI failures?

miker256 · 2019-09-26T23:49:28.738Z

sousou1:

Create a Debian Agent with ROS Buildfarm and test it with the same frequency as other Tier1 and Tier2.

Provide support for CI failures.

What are your thoughts on this @nuclearsandwich? I know back in July you worked on Tier 2 support for armhf.

github.com/ros2/ros2

armhf support is incomplete

opened 05:28PM - 03 Jun 19 UTC

closed 06:00PM - 18 Jul 19 UTC

nuclearsandwich

For ROS 2 Dashing we have set armhf as a tier 2 platform with nightly build and …binary archive support provided by https://ci.ros2.org/ as discovered over the weekend the armhf configuration on that service is incomplete and the last couple weeks of armhf builds and archives are actually being built in a 64-bit ARM environment. I have removed the package labeled armhf from the first [Dashing release on GitHub](https://github.com/ros2/ros2/releases/tag/release-dashing-20190531) to avoid confusing folks and opened https://github.com/ros2/ci/pull/299 which gets builds running on armhf in earnest. However there are some unresolved issues there. - [ ] Resolve warnings on armhf ([test CI warnings report](https://ci.ros2.org/view/All/job/test_ci_linux-armhf/8/warnings23Result/new/)) - [ ] Resolve GCC ABI warnings throughout: ``` In file included from /usr/include/c++/7/vector:63:0, from /home/jenkins-agent/workspace/test_ci_linux-armhf/ws/build/osrf_testing_tools_cpp/googletest-1.8.0-extracted/googletest-1.8.0-src/googletest/include/gtest/gtest.h:56, from /home/jenkins-agent/workspace/test_ci_linux-armhf/ws/build/osrf_testing_tools_cpp/googletest-1.8.0-extracted/googletest-1.8.0-src/googletest/src/gtest-all.cc:39: /usr/include/c++/7/bits/stl_uninitialized.h: In function ‘_ForwardIterator std::__uninitialized_copy_a(_InputIterator, _InputIterator, _ForwardIterator, std::allocator<_Tp>&) [with _InputIterator = __gnu_cxx::__normal_iterator<const double*, std::vector<double> >; _ForwardIterator = double*; _Tp = double]’: /usr/include/c++/7/bits/stl_uninitialized.h:287:5: note: parameter passing for argument of type ‘__gnu_cxx::__normal_iterator<const double*, std::vector<double> >’ changed in GCC 7.1 __uninitialized_copy_a(_InputIterator __first, _InputIterator __last, ^~~~~~~~~~~~~~~~~~~~~~ /usr/include/c++/7/bits/stl_uninitialized.h:287:5: note: parameter passing for argument of type ‘__gnu_cxx::__normal_iterator<const double*, std::vector<double> >’ changed in GCC 7.1 /usr/include/c++/7/bits/stl_uninitialized.h:289:63: note: parameter passing for argument of type ‘__gnu_cxx::__normal_iterator<const double*, std::vector<double> >’ changed in GCC 7.1 { return std::uninitialized_copy(__first, __last, __result); } ^ ```

github.com/ros2/ci

Fix armhf build support

ros2:master ← ros2:fix-armhf

opened 05:13PM - 03 Jun 19 UTC

nuclearsandwich

+15 -9

This adds more complete armhf support using the armhf images built for the ROS b…uild farm on the native CI linux-arm workers, which support armv7 (armhf) and armv8 (aarch64) natively but are running as armv8 platforms. Connects to ros2/ros2#721

https://www.ros.org/reps/rep-2000.html#id17

nuclearsandwich · 2019-09-30T22:58:32.284Z

gbiggs:

In particular what infrastructure do we need to set up and what do we need to provide to where for the binary packages to be made available?

I was discussing this with @tfoote just now and we think that for the moment it would be okay if you added a Debian Buster build to ci.ros2.org via pull requests to GitHub - ros2/ci: ROS 2 CI Infrastructure as the shortest path to shareable results. To make developing that PR easier it might make sense to set up a local CI clone which would require at least two virtual machines, one for the master and one for the agent.

sousou1:

Create a Debian Agent with ROS Buildfarm and test it with the same frequency as other Tier1 and Tier2.

Linux builds on ci.ros2.org and ros_buildfarm are run inside Docker containers, so it is not necessary for either setup to have agents running Debian natively and I’d expect there to be some problems doing so since the configuration management for buildfarm_deployment and the instructions available for setting up ci.ros2.org infrastructure all assume Ubuntu 16.04.

miker256:

This would be a good starting point @gbiggs.

https://wiki.debian.org/DebianScience/Robotics/ROS/OnBuster

@miker256 this article uses ROS from Buster repositories and is not going to be much use when supporting mainline ROS 2 development on Debian Buster since they’ll need tool versions from our repositories not Debian’s.

miker256:

What are your thoughts on this @nuclearsandwich? I know back in July you worked on Tier 2 support for armhf.

armhf required additional agents because none of our existing agents, which are amd64 and arm64, had support for the armhf instruction set. Adding support for Debian builds is a matter of configuration, either forking GitHub - ros2/ci: ROS 2 CI Infrastructure or adding a config file to GitHub - ros2/ros_buildfarm_config (this route also has binary release requirements, see Python3.5 support lost in Dashing release · Issue #735 · ros2/ros2 · GitHub).

sousou1 · 2019-10-01T04:47:18.646Z

@nuclearsandwich

Thanks for the response.
We will start building the Debian agent with Docker and incorporating it into Buildfarm.

I have an additional question.

nuclearsandwich:

I was discussing this with @tfoote just now and we think that for the moment it would be okay if you added a Debian Buster build to ci.ros2.org via pull requests to GitHub - ros2/ci: ROS 2 CI Infrastructure as the shortest path to shareable results.

If PR is merged, Debian will be added to ros2/ci. What’s the difference between Tier1 and Tier2?

gbiggs · 2019-10-01T12:58:13.231Z

Thanks for the detailed response, @nuclearsandwich.

nuclearsandwich · 2019-10-01T22:18:11.608Z

sousou1:

If PR is merged, Debian will be added to ros2/ci. What’s the difference between Tier1 and Tier2?

Support tiers are defined in REP-2000.

ros2/ci has many different job types
Right now support tiers describe what may block a release, automated testing frequency requirements, and, on platforms where per-package binaries are available, what type of binaries we provide. In the past it’s been easier to add additional nightly jobs for tier 2 platforms rather than create a separate periodic job type. But only the Tier 1 platforms are triggered as part of a pull request review process and even with nightly jobs running that does not mean that our on-call build farmer is required to respond to issues on other platforms. Once the jobs are running on CI we’ll expect that the community members who want to support Debian will do the work of checking the periodic jobs and getting them on par with the other platforms.

sousou1 · 2019-10-02T02:59:15.596Z

I understand the difference. Thank you for the detailed explanation @nuclearsandwich.

sousou1 · 2019-10-17T10:21:22.929Z

@nuclearsandwich
We prepared the buildfarm using virtual machines, and create a job using https://github.com/ros2/ci, but I have one question.

The sub-module https://github.com/osrf/rticonnextdds-src/ included in https://github.com/ros2/ci seems to be private.
Now, comment out the part of the Dockerfile that adds the license and run the job with the option not to use Connext. Do we have to contract and prepare our own RTI Connext DDS license?

dirk-thomas · 2019-10-17T17:02:23.366Z

The logic in ros2/ci has two options to use Connext:

either install the Debian package available (which implies accepting the embedded license) or install Connext on macOS / Windows machines
or build Connext from source (that is what the osrf/rticonnextdds-src repo is for). Since this repo is private you would need to talk to RTI if you want access to their sources in a similar way.

Sticking to just the first option is probably fine for you.

sousou1 · 2019-10-18T15:25:30.480Z

Thank you for your response. @dirk-thomas
I will try the first option in my local environment.

sousou1 · 2019-10-30T02:16:53.425Z

@dirk-thomas
We modified the Dockerfile and finished testing with jenkins. The build was successful, but some test fail. The test summary is as follows.

Summary: 268 packages finished [1h 4min 57s]
   6 packages failed: ros2action ros2component ros2doctor ros2node ros2service ros2topic
   11 packages had stderr output: ament_lint osrf_pycommon pendulum_control rcl rclcpp rclpy rcutils rqt rviz_default_plugins rviz_rendering test_rclcpp
   9 packages had test failures: rclcpp ros2interface ros2msg ros2param ros2pkg ros2srv rviz_default_plugins rviz_rendering sros2

Should we fix all these errors before sending PR to ros2/ci?

dirk-thomas · 2019-11-15T19:31:56.040Z

sousou1:

Should we fix all these errors before sending PR to ros2/ci?

Please try another build with the latest state. I would assume most of these errors should have been resolved by now.

Please feel free to create the PR whenever you are ready. Once that is the case we will kick off some jobs anyway to check their status.

sousou1 · 2019-11-15T21:54:30.986Z

Thank you for answering. @dirk-thomas

dirk-thomas:

Please try another build with the latest state. I would assume most of these errors should have been resolved by now.

When I tried from the latest commit, test results have been improved.
There are still some errors, but it seems to be caused by the version difference of apt package.

I also created a PR.
https://github.com/ros2/ci/pull/355
Please let me know if there is any additional work.