Superflore & Open Embedded

Hey!

I’ve been working on getting superflore to generate bitbake recipes for OpenEmbedded.

This will allow for automated installer generations for an entire ROS distribution (given that the dependencies exist).

Currently, you can see the experimental lunar generation on my fork of meta-ros.

There’s a few issues at the moment, however. I can’t seem to get the cmake directories to install in the image, nor can I get the include folder from ROS root to be present in the image.

I’m sure this is just a variable I failed to set, but all input is welcome!

Another question I’d like some feedback for is the location of different distributions. My current opinion is to put each ROS distro on its own branch, so as to avoid collision of recipes. If anybody has any other suggestions here, please let me know!

I’ll post back with a vcs-importable repos file so you can try and replicate the image I’ve been working on easily.

1 Like

Hi,

thank you for that contribution.

First, some general remarks to your superflore tool:

You should have a look at these three scripts that already can generate recipes:

I actually never tried to use these scripts to generate recipe files, but from my review, I believe they generally work but with certain limiting assumptions of the repository structure, and probably do a good job if you are willing to tweak the generated recipes a bit more with some manual effort.

Furthermore, the pull request https://github.com/bmwcarit/meta-ros/pull/565 is providing a script for updating recipes. In the future, once the pull request is merged, we will continue to use and extend that script for the maintenance activities.

Your remark “given that the dependencies exist” actually points out the current blocker for most recipe generation: Certain (non-ROS packaged) dependencies have no recipes yet, and they are sometimes quite difficult to write. Here just two examples:

From my experience maintaining meta-ros for five years by now, creating the recipe files for meta-ros might be some manual (repetitive) work and it seems to be an issue at first, but in the long perspective, creating the recipe is just a simple one-time effort anyway whereas often the time-consuming activities are to find when recipes break due to upstream changes in meta-oe and meta-openembedded. For that, you just need to dig into the issue, no tool can help you there.

Of course, I still hope that superflore helps us to include more recipes in the layer and can be useful for the maintenance of recipe version updates when the distribution is sync-ed once a month.

Now, second to your specific questions:

  • The ros.bbclass (https://github.com/bulwahn/meta-ros/blob/master/classes/ros.bbclass#L30), lines 30 to 60, state what is packaged from the build/install in which package, e.g., the main package, the dev package etc. It might be that you need to adjust the paths after you modified the catkin tool. To have the cmake and include folder in the image, you probably need to include the dev package in the image recipe.

  • I also believe it is best to have one branch for each ROS distro in meta-ros. I would like to first merge the currently open pull requests, especially the update-script and the sdk support and then we can split it into three branches, indigo, kinetic, lunar, if we get all the recipes work in these branches.

Lukas

1 Like

I actually used these scripts when I wrote the bitbake generator.

It would also be nice to get rosdep keys to resolve these dependencies. As for dependencies that don’t already exist, I run into them from time to time with Gentoo (though most do exist in the gentoo repository).

@bulwahn I’ve had some real progress here with respect to catkin.

At the moment, I think the main issue is some sort of python pack linkage.

 ERROR: cmake-modules-0.4.1-r0 do_configure: Function failed: do_configure (log file is located at /home/allenh1/work/poky/rpi-build/tmp/work/cortexa7hf-neon-vfpv4-poky-linux-gnueabi/cmake-modules/0.4.1-r0/temp/log.do_configure.17964)
ERROR: Logfile of failure stored in: /home/allenh1/work/poky/rpi-build/tmp/work/cortexa7hf-neon-vfpv4-poky-linux-gnueabi/cmake-modules/0.4.1-r0/temp/log.do_configure.17964
Log data follows:
| DEBUG: Executing shell function do_configure
| -- The C compiler identification is GNU 7.2.0
| -- The CXX compiler identification is GNU 7.2.0
| -- Check for working C compiler: /home/allenh1/work/poky/rpi-build/tmp/work/cortexa7hf-neon-vfpv4-poky-linux-gnueabi/cmake-modules/0.4.1-r0/recipe-sysroot-native/usr/bin/arm-poky-linux-gnueabi/arm-poky-linux-gnueabi-gcc
| -- Check for working C compiler: /home/allenh1/work/poky/rpi-build/tmp/work/cortexa7hf-neon-vfpv4-poky-linux-gnueabi/cmake-modules/0.4.1-r0/recipe-sysroot-native/usr/bin/arm-poky-linux-gnueabi/arm-poky-linux-gnueabi-gcc -- works
| -- Detecting C compiler ABI info
| -- Detecting C compiler ABI info - done
| -- Detecting C compile features
| -- Detecting C compile features - done
| -- Check for working CXX compiler: /home/allenh1/work/poky/rpi-build/tmp/work/cortexa7hf-neon-vfpv4-poky-linux-gnueabi/cmake-modules/0.4.1-r0/recipe-sysroot-native/usr/bin/arm-poky-linux-gnueabi/arm-poky-linux-gnueabi-g++
| -- Check for working CXX compiler: /home/allenh1/work/poky/rpi-build/tmp/work/cortexa7hf-neon-vfpv4-poky-linux-gnueabi/cmake-modules/0.4.1-r0/recipe-sysroot-native/usr/bin/arm-poky-linux-gnueabi/arm-poky-linux-gnueabi-g++ -- works
| -- Detecting CXX compiler ABI info
| -- Detecting CXX compiler ABI info - done
| -- Detecting CXX compile features
| -- Detecting CXX compile features - done
| -- Using CATKIN_DEVEL_PREFIX: /home/allenh1/work/poky/rpi-build/tmp/work/cortexa7hf-neon-vfpv4-poky-linux-gnueabi/cmake-modules/0.4.1-r0/build/devel
| -- Using CMAKE_PREFIX_PATH: /home/allenh1/work/poky/rpi-build/tmp/work/cortexa7hf-neon-vfpv4-poky-linux-gnueabi/cmake-modules/0.4.1-r0/recipe-sysroot/opt/ros/lunar;/home/allenh1/work/poky/rpi-build/tmp/work/cortexa7hf-neon-vfpv4-poky-linux-gnueabi/cmake-modules/0.4.1-r0/recipe-sysroot/usr;                        /home/allenh1/work/poky/rpi-build/tmp/work/cortexa7hf-neon-vfpv4-poky-linux-gnueabi/cmake-modules/0.4.1-r0/recipe-sysroot-native/opt/ros/lunar;/home/allenh1/work/poky/rpi-build/tmp/work/cortexa7hf-neon-vfpv4-poky-linux-gnueabi/cmake-modules/0.4.1-r0/recipe-sysroot-native/usr
| -- Found PythonInterp: /home/allenh1/work/poky/rpi-build/tmp/work/cortexa7hf-neon-vfpv4-poky-linux-gnueabi/cmake-modules/0.4.1-r0/recipe-sysroot-native/usr/bin/python-native/python (found version "2.7.13")
| -- Using PYTHON_EXECUTABLE: /home/allenh1/work/poky/rpi-build/tmp/work/cortexa7hf-neon-vfpv4-poky-linux-gnueabi/cmake-modules/0.4.1-r0/recipe-sysroot-native/usr/bin/python-native/python
| -- Using default Python package layout
| -- Found PY_em: /home/allenh1/work/poky/rpi-build/tmp/work/cortexa7hf-neon-vfpv4-poky-linux-gnueabi/cmake-modules/0.4.1-r0/recipe-sysroot-native/usr/lib/python2.7/site-packages/em.pyc
| -- Using empy: /home/allenh1/work/poky/rpi-build/tmp/work/cortexa7hf-neon-vfpv4-poky-linux-gnueabi/cmake-modules/0.4.1-r0/recipe-sysroot-native/usr/lib/python2.7/site-packages/em.pyc
| -- Using CATKIN_ENABLE_TESTING: 0
| -- catkin 0.7.8
| Traceback (most recent call last):
|   File "/home/allenh1/work/poky/rpi-build/tmp/work/cortexa7hf-neon-vfpv4-poky-linux-gnueabi/cmake-modules/0.4.1-r0/build/catkin_generated/generate_cached_setup.py", line 20, in <module>
|     from catkin.environment_cache import generate_environment_script
| ImportError: No module named catkin.environment_cache
| CMake Error at /home/allenh1/work/poky/rpi-build/tmp/work/cortexa7hf-neon-vfpv4-poky-linux-gnueabi/cmake-modules/0.4.1-r0/recipe-sysroot/opt/ros/lunar/share/catkin/cmake/safe_execute_process.cmake:11 (message):
| 
|   execute_process(/home/allenh1/work/poky/rpi-build/tmp/work/cortexa7hf-neon-vfpv4-poky-linux-gnueabi/cmake-modules/0.4.1-r0/recipe-sysroot-native/usr/bin/python-native/python
|   "/home/allenh1/work/poky/rpi-build/tmp/work/cortexa7hf-neon-vfpv4-poky-linux-gnueabi/cmake-modules/0.4.1-r0/build/catkin_generated/generate_cached_setup.py")
|   returned error code 1
| Call Stack (most recent call first):
|   /home/allenh1/work/poky/rpi-build/tmp/work/cortexa7hf-neon-vfpv4-poky-linux-gnueabi/cmake-modules/0.4.1-r0/recipe-sysroot/opt/ros/lunar/share/catkin/cmake/all.cmake:186 (safe_execute_process)
|   /home/allenh1/work/poky/rpi-build/tmp/work/cortexa7hf-neon-vfpv4-poky-linux-gnueabi/cmake-modules/0.4.1-r0/recipe-sysroot/opt/ros/lunar/share/catkin/cmake/catkinConfig.cmake:20 (include)
|   CMakeLists.txt:4 (find_package)
| 
| 
| -- Configuring incomplete, errors occurred!
| See also "/home/allenh1/work/poky/rpi-build/tmp/work/cortexa7hf-neon-vfpv4-poky-linux-gnueabi/cmake-modules/0.4.1-r0/build/CMakeFiles/CMakeOutput.log".
| WARNING: /home/allenh1/work/poky/rpi-build/tmp/work/cortexa7hf-neon-vfpv4-poky-linux-gnueabi/cmake-modules/0.4.1-r0/temp/run.do_configure.17964:1 exit 1 from 'cmake $oecmake_sitefile /home/allenh1/work/poky/rpi-build/tmp/work/cortexa7hf-neon-vfpv4-poky-linux-gnueabi/cmake-modules/0.4.1-r0/cmake_modules-release-release-lunar-cmake_modules-0.4.1-0 -DCMAKE_INSTALL_PREFIX:PATH=/usr -DCMAKE_INSTALL_BINDIR:PATH=bin -DCMAKE_INSTALL_SBINDIR:PATH=sbin -DCMAKE_INSTALL_LIBEXECDIR:PATH=libexec -DCMAKE_INSTALL_SYSCONFDIR:PATH=/etc -DCMAKE_INSTALL_SHAREDSTATEDIR:PATH=../com -DCMAKE_INSTALL_LOCALSTATEDIR:PATH=/var -DCMAKE_INSTALL_LIBDIR:PATH=lib -DCMAKE_INSTALL_INCLUDEDIR:PATH=include -DCMAKE_INSTALL_DATAROOTDIR:PATH=share -DCMAKE_INSTALL_SO_NO_EXE=0 -DCMAKE_TOOLCHAIN_FILE=/home/allenh1/work/poky/rpi-build/tmp/work/cortexa7hf-neon-vfpv4-poky-linux-gnueabi/cmake-modules/0.4.1-r0/toolchain.cmake -DCMAKE_VERBOSE_MAKEFILE=1 -DCMAKE_NO_SYSTEM_FROM_IMPORTED=1 -DCMAKE_PREFIX_PATH='/home/allenh1/work/poky/rpi-build/tmp/work/cortexa7hf-neon-vfpv4-poky-linux-gnueabi/cmake-modules/0.4.1-r0/recipe-sysroot/opt/ros/lunar;/home/allenh1/work/poky/rpi-build/tmp/work/cortexa7hf-neon-vfpv4-poky-linux-gnueabi/cmake-modules/0.4.1-r0/recipe-sysroot/usr;                        /home/allenh1/work/poky/rpi-build/tmp/work/cortexa7hf-neon-vfpv4-poky-linux-gnueabi/cmake-modules/0.4.1-r0/recipe-sysroot-native/opt/ros/lunar;/home/allenh1/work/poky/rpi-build/tmp/work/cortexa7hf-neon-vfpv4-poky-linux-gnueabi/cmake-modules/0.4.1-r0/recipe-sysroot-native/usr' -DCMAKE_INSTALL_PREFIX:PATH='/opt/ros/lunar' -DSETUPTOOLS_DEB_LAYOUT=OFF -DCATKIN_ENABLE_TESTING=0 -DCATKIN_BUILD_BINARY_PACKAGE=1 ${PACKAGECONFIG_CONFARGS} -Wno-dev'
| ERROR: Function failed: do_configure (log file is located at /home/allenh1/work/poky/rpi-build/tmp/work/cortexa7hf-neon-vfpv4-poky-linux-gnueabi/cmake-modules/0.4.1-r0/temp/log.do_configure.17964)
ERROR: Task (/home/allenh1/work/poky/meta-ros/recipes-ros-lunar/cmake-modules/cmake-modules_0.4.1.bb:do_configure) failed with exit code '1'
ERROR: Task (/home/allenh1/work/poky/meta-ros/recipes-ros-lunar/rosgraph/rosgraph_1.13.5.bb:do_configure) failed with exit code '1'

I don’t quite have the Open Embedded experience to debug this yet. Do you have any idea as to what could be wrong here?

Hi @allenh1,

I am sorry but I don’t think I can really help you there. A wild guess from the log is that a catkin-native dependencies is missing somewhere, but I think the problem is more difficult than just that.

If you look at the git history of the catkin.bbclass (https://github.com/bmwcarit/meta-ros/commits/master/classes/catkin.bbclass), you will notice that although I am the maintainer of meta-ros, I actually contributed only very few changes (and more general refactoring) to the catkin.bbclass. The currently best expert for the catkin.bbclass would actually be Dmitry.

@rojkov Can you possibly help here?

Lukas

Hi,

IIRC the real catkin code comes from the catkin-runtime recipe which catkin-native depends on. The original catkin.bbclass adds catkin-native as a dependency. The catkin.bbclass from the lunar branch seems to miss it.