REP3 proposal for Lunar Loggerhead

marguedas · 2017-01-18T22:49:38.844Z

Hello everyone,

2017 is there and ROS Lunar is around the corner!
A proposal for the minimum requirements and targeted architectures (aka REP3) is available here. This release will maintain the minimum requirements of Kinetic, the main change is that we will stop building packages for i386 architectures.
We would like the feedback of all developers and maintainers about this draft.

Your friendly ROS team

jackie · 2017-01-19T00:48:15.809Z

Interesting, are there usage statistics for i386 downloads vs x64? The Intel Atom family has some 32-bit processors. They might not be the most popular choice, but I would be curious to see statistics on this before the decision to deprecate 32-bit Intel is made.

marguedas · 2017-01-20T18:06:31.130Z

Looking on the Intel website, it looks like the latest 32-bit atom processor they released was in 2011 and is already end of life. The last still supported were launched in 2010 [1]. All new intel processors (Joule/Curie etc) are 64-bit architectures.

Regarding statistics, Debian saw amd64 taking over i386 around mid 2012 and today there is a 3.6:1 ratio [2]. For ROS packages, amd64 downloads exceeded i386 since 2015, last month’s statistics of xenial downloads show amd64 packages downloaded ~20% more than i386 [3]. We do suspect that most i386 downloads are made for computers supporting 64-bit instruction set but running x86 OSs

It would be interesting to see how many people among the community are actually installing ROS on 32-bit processors to make an educated decision.

[1] https://ark.intel.com/fr/search/advanced?s=t&FamilyText=Intel®%20Atom™%20Processor&InstructionSet=32-bit
[2] http://popcon.debian.org/
[3] https://awstats.osuosl.org/reports/packages.ros.org/2016/12/awstats.packages.ros.org.urldetail.html

ghindman · 2017-01-23T19:38:04.966Z

While not likely an issue architecturally, there are still scenarios where 32-bit support is useful, such as low memory configurations (<4GB RAM) where 64-bit creates memory pressure, and safety applications which use 32-bit/64-bit software diversity. I’m sure there are others as well.

rmattes · 2017-01-24T14:47:33.131Z

I have a couple of questions about the mention of Fedora in the REP:

First, what does “recommended support” for Fedora and Debian mean? That the build-from-source process should generally work? Or is there thought of adding binary releases for those distributions?

Second, unless something in Fedora land changes, Fedora 25 will go EOL ~13 months after its release date[1], or around December 2017. Fedora 26, which will be released in the June 2017 timeframe[2], will be EOL around July 2018. Does it make sense to call out specific distribution versions when they won’t all be supported over the same timeframe as the ROS release? Or, in cases where a distributions future plans are unknown that far out, does it make more sense to just make a blanket statement that the earliest and/or latest supported version of that distribution should be supported by ROS?

That said, Fedora is already incompatible with the “Exact or Series” requirements. We’ve already got pcl-1.8 in fedora 24 and higher[3], and Qt 5.6 or higher in fedora 24 and higher[4]. On the other hand, we won’t introduce gazebo 8 or probably opencv-3.2 until fedora 26, as large breaking changes in released fedora versions violate the stable release updates policy[5]. So do they really need to be exact, or are they only enumerating the requirements as dictated by the environments on the “Required Support for” platforms?

[1] https://fedoraproject.org/wiki/Fedora_Release_Life_Cycle#Maintenance_Schedule
[2] https://fedoraproject.org/wiki/Releases/26/Schedule
[3] https://apps.fedoraproject.org/packages/pcl
[4] https://apps.fedoraproject.org/packages/qt5
[5] https://fedoraproject.org/wiki/Updates_Policy#Stable_Releases

tfoote · 2017-01-24T17:10:07.278Z

rmattes:

First, what does “recommended support” for Fedora and Debian mean? That the build-from-source process should generally work? Or is there thought of adding binary releases for those distributions?

This means that we will do a best effort to support users on these platforms but not block if something is not available. It’s expected that building from source should be possible on those platforms. We generate the build metadata automatically for them (aka bloom-release is aware of them and turns them on by default) And we have enabled builds on Debian already in Kinetic and since it’s so close to Ubuntu there’s very little difference. If there’s interest from contributors to get Fedora binary builds working we would be interested in supporting bringing RPM builds online too.

rmattes:

Second, unless something in Fedora land changes, Fedora 25 will go EOL ~13 months after its release date[1], or around December 2017. Fedora 26, which will be released in the June 2017 timeframe[2], will be EOL around July 2018. Does it make sense to call out specific distribution versions when they won’t all be supported over the same timeframe as the ROS release? Or, in cases where a distributions future plans are unknown that far out, does it make more sense to just make a blanket statement that the earliest and/or latest supported version of that distribution should be supported by ROS?

The timing is unfortunate. The primary reason for calling out a specific version is that for our tooling, like bloom we have to pick specific versions for which to generate metadata. We can only support available releases since the dependencies need to be known. Our release is synchronized with the Ubuntu alpha cycle so that we can use their prerelease versions for development and extend our life cycle as far as possible. They have even shorter cycles than Fedora (9 months for non-LTS)

In the past we’ve tried adding platforms to ROS distros and that is as much work as creating a new rosdistro, and actually harder since you are not in a tick-tock cycle so deprecations upstream can box you into a corner. So we can’t plan to add 26 after our release. And the 26 Alpha looks like it won’t be available until mid march. (Fedora 26 Release Schedule )

rmattes:

That said, Fedora is already incompatible with the “Exact or Series” requirements. We’ve already got pcl-1.8 in fedora 24 and higher[3], and Qt 5.6 or higher in fedora 24 and higher[4]. On the other hand, we won’t introduce gazebo 8 or probably opencv-3.2 until fedora 26, as large breaking changes in released fedora versions violate the stable release updates policy[5]. So do they really need to be exact, or are they only enumerating the requirements as dictated by the environments on the “Required Support for” platforms?

It would be good to know what these will be and call them out. The goal is to have a consistent set of core dependencies that maintainers don’t need to worry about supporting multiple dependencies with #ifdefs or other mechanisms.

A few specifics.

PCL on Yakkety and Zesty is already 1.8 as well. We should specify that’s more flexible.
Our target of Gazebo 8 is to synchronize the Ubuntu LTS, with the ROS LTS, with the Gazebo LTS versions. To make this happen we will have to custom package Gazebo8 for the non-LTS ROS releases. And for the LTS releases we will have to at least package for the non-LTS ubuntu versions assuming the the Gazebo LTS is submitted successfully upstream. Though if we’re already building the non-LTS Ubuntu version for the ROS/Gazebo LTS coverage we might as well build new packages for the Ubuntu LTS in case of a patch release to get it out quicker and more consistently. So for Fedora I think it would be best if we packaged Gazebo 8 for Fedora 25 as well. @jrivero Do we already have rpm packaging of gazebo supported?

@rmattes Could you go line by line through the dependencies and start a thread on that line of the PR if there’s a discrepancy that needs to be resolved?

dirk-thomas · 2017-01-24T17:36:39.316Z

tfoote:

So we can’t plan to add 26 after our release. And the 26 Alpha looks like it won’t be available until mid march.

As long as the package names are “known” (which I would expect to be the case in February when they do the rebuild) we should be able to support Fedora 26.

I would even think that it might be a good idea to only support Fedora 26 then since some packages are lower then our currently planned requirements in Fedora 25. A higher version should usually not be a big problem since maintainers commonly try to keep their software compatible with newer releases (the same happens all the time on e.g. Gentoo).

marguedas · 2017-01-25T22:04:29.432Z

marguedas:

For ROS packages, amd64 downloads exceeded i386 since 2015, last month’s statistics of xenial downloads show amd64 packages downloaded ~20% more than i386 [3]. We do suspect that most i386 downloads are made for computers supporting 64-bit instruction set but running x86 OSs

[3] Statistics for packages.ros.org (2016-12) - urldetail

Update on the statistics. The previous statistics were package lookups (apt-get update) and not actual package downloads.
Below is the breakdown of package downloads per architecture over the last few years:

|  Architectures | 07/2014 | 12/2015 | 12/2016 |
------------------------------------------------
|     amd64      | 67.56 % | 82.94 % | 84.01 % |
|     i386       | 28.77 % | 10.52 % |  6.53 % |
|     armhf      |  0.00 % |  2.96 % |  4.66 % |
|     arm64      |  0.00 % |  0.00 % |  1.59 % |
|     source     |  1.05 % |  0.51 % |  0.68 % |

Note: armhf packages have been introduced in 2015 and arm64 packages in 2016
Note2: these statistics are total package downloads, all Linux and ROS distributions combined

EDIT:
Xenial only statistics:

|  Architectures | 12/2016 |
----------------------------
|     amd64      | 22.61 % |
|     i386       |  2.26 % |
|     armhf      |  2.35 % |
|     arm64      |  1.52 % |
|     source     |  0.56 % |

dirk-thomas · 2017-01-25T22:17:39.674Z

Is it possible to get the statistics for Xenial only? Since Ubuntu recommends the 64bit version for that distro it would show how many users intentionally chose i386 instead.

tfoote · 2017-01-25T22:51:14.440Z

Here’s a slightly more detailed view of the 12/2016 statistics with the script slightly modifed to filter for Kinetic only.

| Architecture | 12/2016 |
--------------------------
| i386         | 2.33 %  |
| amd64        | 23.24 % |
| armhf        |  2.35 % |
| arm64        |  1.54 % |
| source       |  0.57 % |
--------------------------
|distro_arch   |         |
--------------------------
| jessie_amd64 |  0.38 % |
| jessie_arm64 |  0.02 % |
| jessie_source|  0.01 % |
| wily_amd64   |  0.27 % |
| wily_i386    |  0.07 % |
| wily_source  |  0.01 % |
| xenial_amd64 | 22.60 % |
| xenial_arm64 |  1.52 % |
| xenial_armhf |  2.35 % |
| xenial_i386  |  2.26 % |
| xenial_source|  0.56 % |
--------------------------

Also I found an article on Ubuntu switching to 64 bit downloads by default as of Saucy, on OMG Ubuntu. Clearly there’s some lag.

dirk-thomas · 2017-01-25T23:10:24.180Z

“Less then 10%” sounds like a much better argument to drop support then the previous global stats. So I am in favor of dropping the i386 builds then since it reduced our farm by almost half

I expect the majority of the currently remaining users could switch to amd64. The few remaining ones will need to build from source (as many more users on various other platforms already do today).

rmattes · 2017-01-29T03:00:06.610Z

At this point, the only discrepancies for Fedora 25 are Gazebo 7 and OpenCV 3.1. Qt 5 versions are usually backwards compatible, so packages targeting Qt 5.3 will probably be fine forh the 5.6 or 5.7 versions in Fedora.

There’s not likely to be any package name changes between f25 and f26, so things like rosdep keys should still work normally. It would likely be safe to target f26 with the buildsystem metadata even before it’s released. Fedora 26 will officially start to “exist” at the end of February when it’s branched from rawhide - at that point, there will be f26 repositories, mock configurations, buildsystem configurations, container images, etc., making it possible to build against it in a pre-alpha state.

I will note that Fedora has a system called copr, which is similar to Ubuntu PPAs. I set up a gazebo-latest copr some time ago, to keep the latest version of Gazebo available in Fedora releases. I kind of let it slip for a while, but I am planning on building gazebo 8 in the copr for all released Fedora versions. It may be easier to point users to that repository than to package it separately.

tfoote · 2017-01-30T06:23:27.525Z

rmattes:

There’s not likely to be any package name changes between f25 and f26, so things like rosdep keys should still work normally. It would likely be safe to target f26 with the buildsystem metadata even before it’s released. Fedora 26 will officially start to “exist” at the end of February when it’s branched from rawhide - at that point, there will be f26 repositories, mock configurations, buildsystem configurations, container images, etc., making it possible to build against it in a pre-alpha state.

If things will be stabilized by February and we don’t expect any changes leading up to that then it sounds like supporting 26 would be fine. I thought that it might be later in the the release cycle.

rmattes:

I will note that Fedora has a system called copr, which is similar to Ubuntu PPAs. I set up a gazebo-latest copr some time ago, to keep the latest version of Gazebo available in Fedora releases. I kind of let it slip for a while, but I am planning on building gazebo 8 in the copr for all released Fedora versions. It may be easier to point users to that repository than to package it separately.

If we do need to start building packages we will also want to setup a standard repository for rpms on the ros.org domain like we have for debian family packages. Presumably we can import from resources like the copr repos.

marguedas · 2017-02-02T01:34:02.258Z

It appears that OpenCV3.2 is very close to OpenCV3.1 and should be backward compatible (based on @vrabaud comment )?
So I’m going to move ahead and change the REP to target f26. We can revisit end of February if anything comes up.

marguedas · 2017-02-08T05:14:50.288Z

Hi everyone,

We’re planning on merging the REP3 proposal at the end of the week. If you have any further comments please submit them by then.

The ROS team