ROS2 Beta1 compiled from source - unicode bug

Hello,

I have been using ROS1 in my lab for a while, and felt like trying ROS2 after its Beta1 release. I began with a bare bones Ubuntu 16.04 container and followed the install from source instructions found here - https://github.com/ros2/ros2/wiki/Linux-Development-Setup

During the build, I got the following error:

+++ Building 'rmw_fastrtps_cpp'
Running cmake because arguments have changed.
==> '. /root/ros2_copy_ws/build/rmw_fastrtps_cpp/cmake__build.sh && /usr/bin/cmake /root/ros2_copy_ws/src/ros2/rmw_fastrtps/rmw_fastrtps_cpp -DBUILD_TESTING=1 -DAMENT_CMAKE_SYMLINK_INSTALL=1 -DCMAKE_INSTALL_PREFIX=/root/ros2_copy_ws/install' in '/root/ros2_copy_ws/build/rmw_fastrtps_cpp'
-- The C compiler identification is GNU 5.4.0
-- The CXX compiler identification is GNU 5.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found ament_cmake: 0.0.0 (/root/ros2_copy_ws/install/share/ament_cmake/cmake)
-- Found PythonInterp: /usr/bin/python3 (found suitable version "3.5.2", minimum required is "3") 
-- Using PYTHON_EXECUTABLE: /usr/bin/python3
-- Override CMake install command with custom implementation using symlinks instead of copying resources
-- Found fastrtps_cmake_module: 0.0.0 (/root/ros2_copy_ws/install/share/fastrtps_cmake_module/cmake)
-- Found rmw: 0.0.0 (/root/ros2_copy_ws/install/share/rmw/cmake)
-- Found rosidl_generator_cpp: 0.0.0 (/root/ros2_copy_ws/install/share/rosidl_generator_cpp/cmake)
-- Found rosidl_typesupport_introspection_c: 0.0.0 (/root/ros2_copy_ws/install/share/rosidl_typesupport_introspection_c/cmake)
-- Found rosidl_typesupport_introspection_cpp: 0.0.0 (/root/ros2_copy_ws/install/share/rosidl_typesupport_introspection_cpp/cmake)
-- Found rosidl_typesupport_cpp: 0.0.0 (/root/ros2_copy_ws/install/share/rosidl_typesupport_cpp/cmake)
-- Found ament_lint_auto: 0.0.0 (/root/ros2_copy_ws/install/share/ament_lint_auto/cmake)
Error parsing '/root/ros2_copy_ws/src/ros2/rmw_fastrtps/rmw_fastrtps_cpp/package.xml':
Traceback (most recent call last):
  File "/root/ros2_copy_ws/install/share/ament_cmake_core/cmake/core/package_xml_2_cmake.py", line 141, in <module>
    main()
  File "/root/ros2_copy_ws/install/share/ament_cmake_core/cmake/core/package_xml_2_cmake.py", line 56, in main
    raise e
  File "/root/ros2_copy_ws/install/share/ament_cmake_core/cmake/core/package_xml_2_cmake.py", line 53, in main
    package = parse_package_string(args.package_xml.read())
  File "/usr/lib/python3.5/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 409: ordinal not in range(128)
CMake Error at /root/ros2_copy_ws/install/share/ament_cmake_core/cmake/core/ament_package_xml.cmake:94 (message):
  execute_process(/usr/bin/python3
  /root/ros2_copy_ws/install/share/ament_cmake_core/cmake/core/package_xml_2_cmake.py
  /root/ros2_copy_ws/src/ros2/rmw_fastrtps/rmw_fastrtps_cpp/package.xml
  /root/ros2_copy_ws/build/rmw_fastrtps_cpp/ament_cmake_core/package.cmake)
  returned error code 1
Call Stack (most recent call first):
  /root/ros2_copy_ws/install/share/ament_cmake_core/cmake/core/ament_package_xml.cmake:49 (_ament_package_xml)
  /root/ros2_copy_ws/install/share/ament_lint_auto/cmake/ament_lint_auto_find_test_dependencies.cmake:30 (ament_package_xml)
  CMakeLists.txt:72 (ament_lint_auto_find_test_dependencies)


-- Configuring incomplete, errors occurred!
See also "/root/ros2_copy_ws/build/rmw_fastrtps_cpp/CMakeFiles/CMakeOutput.log".

<== Command '. /root/ros2_copy_ws/build/rmw_fastrtps_cpp/cmake__build.sh && /usr/bin/cmake /root/ros2_copy_ws/src/ros2/rmw_fastrtps/rmw_fastrtps_cpp -DBUILD_TESTING=1 -DAMENT_CMAKE_SYMLINK_INSTALL=1 -DCMAKE_INSTALL_PREFIX=/root/ros2_copy_ws/install' failed in '/root/ros2_copy_ws/build/rmw_fastrtps_cpp' with exit code '1'
<== Command '. /root/ros2_copy_ws/build/rmw_fastrtps_cpp/cmake__build.sh && /usr/bin/cmake /root/ros2_copy_ws/src/ros2/rmw_fastrtps/rmw_fastrtps_cpp -DBUILD_TESTING=1 -DAMENT_CMAKE_SYMLINK_INSTALL=1 -DCMAKE_INSTALL_PREFIX=/root/ros2_copy_ws/install' failed in '/root/ros2_copy_ws/build/rmw_fastrtps_cpp' with exit code '1'

I looked at the culprit package.xml file and found the following snippet as the issue:

<maintainer email="ricardogonzalez@eprosima.com">Ricardo González</maintainer>

I could figure it out only because it has an á which is usually not present in python’s unicode decoder. Changing the á to a regular a - fixed the error, and rest of the source code compiled perfectly.

Sincere apologies to Ricardo for changing your name. (@richiprosima ) :slight_smile:

Hi @shehz,

This is a problem showing up in containers indeed but not on native installations.
This is because containers don’t handle unicode characters by default. The way we handle it in our containers is to call ENV LANG en_US.UTF-8.

Can you try this out at the beginning of your Dockerfile and post back here with the outcome?

Thanks!

Before stopping the container, I checked the locale:

root@526a88b1e27b:~/ros2_ws# locale
LANG=
LANGUAGE=
LC_CTYPE="POSIX"
LC_NUMERIC="POSIX"
LC_TIME="POSIX"
LC_COLLATE="POSIX"
LC_MONETARY="POSIX"
LC_MESSAGES="POSIX"
LC_PAPER="POSIX"
LC_NAME="POSIX"
LC_ADDRESS="POSIX"
LC_TELEPHONE="POSIX"
LC_MEASUREMENT="POSIX"
LC_IDENTIFICATION="POSIX"
LC_ALL=

You are correct it was the problem of my environment variables. Set it according to your instructions and checked locale again:

root@526a88b1e27b:~/ros2_ws# locale
LANG=en_US.UTF-8
LANGUAGE=en_US:en
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=en_US.UTF-8

Compiling perfectly now. I thought this step update-locale LC_ALL=en_US.UTF-8 LANG=en_US.UTF-8 in the install instructions would have worked inside the container - but it did not.

Thanks a lot @marguedas for pointing out your dockerfiles, saves me a lot of trouble!

Glad it worked!

Thanks for pointing it out, we added the LANG=en_US.UTF-8 to the install instructions hoping that it would fix this particular issue. Apparently it’s not enough. According to this thread the entire update-locale call has no effect in commandline in docker containers, so it seems that it “has to happen” in the Dockerfile before hitting the entry point.