Ros2 LifecycleNode memory corruption issues

Next Generation ROS
1

We are experimenting with ROS2 and the LifecycleNode and started to observe crashes in our code. When we use regular nodes everything seems to be working fine. I tried to use valgrind to see what might be happening and indeed there are many warnings coming out from the tool (see some of the stuff below).

I guess my question is - in what shape is the lifecycle package and are you guys aware of these problems?

Thanks :slight_smile:

Invalid write of size 2
==4948== at 0x4C32723: memcpy@@GLIBC_2.14 (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==4948== by 0x7009998: concatenate (in /home/breh/ros2_ws/build/rcl_lifecycle/librcl_lifecycle.so)
==4948== by 0x7009B68: rcl_lifecycle_com_interface_init (in /home/breh/ros2_ws/build/rcl_lifecycle/librcl_lifecycle.so)
==4948== by 0x700A8AC: rcl_lifecycle_state_machine_init (in /home/breh/ros2_ws/build/rcl_lifecycle/librcl_lifecycle.so)
==4948== by 0x5AF59F7: rclcpp_lifecycle::LifecycleNode::LifecycleNodeInterfaceImpl::init() (in /home/breh/ros2_ws/build/rclcpp_lifecycle/librclcpp_lifecycle.so)
==4948== by 0x5AF3CDF: rclcpp_lifecycle::LifecycleNode::LifecycleNode(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, std::shared_ptrrclcpp::context::Context, bool) (in /home/breh/ros2_ws/build/rclcpp_lifecycle/librclcpp_lifecycle.so)
==4948== by 0x5AF3985: rclcpp_lifecycle::LifecycleNode::LifecycleNode(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, bool) (in /home/breh/ros2_ws/build/rclcpp_lifecycle/librclcpp_lifecycle.so)
==4948== by 0x40B4D1: LifecycleTalker::LifecycleTalker(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, bool) (in /home/breh/ros2_overlay_ws/install/bin/lifecycle_talker)
==4948== by 0x413E62: ZN9__gnu_cxx13new_allocatorI15LifecycleTalkerE9constructIS1_IRA10_KcEEEvPT_DpOT0 (in /home/breh/ros2_overlay_ws/install/bin/lifecycle_talker)
==4948== by 0x413437: ZNSt16allocator_traitsISaI15LifecycleTalkerEE9constructIS0_IRA10_KcEEEvRS1_PT_DpOT0 (in /home/breh/ros2_overlay_ws/install/bin/lifecycle_talker)
==4948== by 0x412855: std::_Sp_counted_ptr_inplace<LifecycleTalker, std::allocator, (__gnu_cxx::_Lock_policy)2>::_Sp_counted_ptr_inplace<char const (&) [10]>(std::allocator, char const (&) [10]) (in /home/breh/ros2_overlay_ws/install/bin/lifecycle_talker)
==4948== by 0x4115B6: std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count<LifecycleTalker, std::allocator, char const (&) [10]>(std::_Sp_make_shared_tag, LifecycleTalker*, std::allocator const&, char const (&) [10]) (in /home/breh/ros2_overlay_ws/install/bin/lifecycle_talker)
==4948== Address 0xe620f3a is 26 bytes inside a block of size 27 alloc’d
==4948== at 0x4C2DB8F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==4948== by 0x7009946: concatenate (in /home/breh/ros2_ws/build/rcl_lifecycle/librcl_lifecycle.so)
==4948== by 0x7009B68: rcl_lifecycle_com_interface_init (in /home/breh/ros2_ws/build/rcl_lifecycle/librcl_lifecycle.so)
==4948== by 0x700A8AC: rcl_lifecycle_state_machine_init (in /home/breh/ros2_ws/build/rcl_lifecycle/librcl_lifecycle.so)
==4948== by 0x5AF59F7: rclcpp_lifecycle::LifecycleNode::LifecycleNodeInterfaceImpl::init() (in /home/breh/ros2_ws/build/rclcpp_lifecycle/librclcpp_lifecycle.so)
==4948== by 0x5AF3CDF: rclcpp_lifecycle::LifecycleNode::LifecycleNode(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, std::shared_ptrrclcpp::context::Context, bool) (in /home/breh/ros2_ws/build/rclcpp_lifecycle/librclcpp_lifecycle.so)
==4948== by 0x5AF3985: rclcpp_lifecycle::LifecycleNode::LifecycleNode(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, bool) (in /home/breh/ros2_ws/build/rclcpp_lifecycle/librclcpp_lifecycle.so)
==4948== by 0x40B4D1: LifecycleTalker::LifecycleTalker(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, bool) (in /home/breh/ros2_overlay_ws/install/bin/lifecycle_talker)
==4948== by 0x413E62: ZN9__gnu_cxx13new_allocatorI15LifecycleTalkerE9constructIS1_IRA10_KcEEEvPT_DpOT0 (in /home/breh/ros2_overlay_ws/install/bin/lifecycle_talker)
==4948== by 0x413437: ZNSt16allocator_traitsISaI15LifecycleTalkerEE9constructIS0_IRA10_KcEEEvRS1_PT_DpOT0 (in /home/breh/ros2_overlay_ws/install/bin/lifecycle_talker)
==4948== by 0x412855: std::_Sp_counted_ptr_inplace<LifecycleTalker, std::allocator, (__gnu_cxx::_Lock_policy)2>::_Sp_counted_ptr_inplace<char const (&) [10]>(std::allocator, char const (&) [10]) (in /home/breh/ros2_overlay_ws/install/bin/lifecycle_talker)
==4948== by 0x4115B6: std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count<LifecycleTalker, std::allocator, char const (&) [10]>(std::_Sp_make_shared_tag, LifecycleTalker*, std::allocator const&, char const (&) [10]) (in /home/breh/ros2_overlay_ws/install/bin/lifecycle_talker)

2

Sorry to hear you encounter issues with it and thanks for bringing that issue forward. The state of the package is pretty much as it was by the end of 2016 for beta1. There was not major development since then.

I can’t reproduce your error locally on my machine, the tutorial runs as expected. Can you provide me a few more details:

  • Are you running the beta1 tutorial?
  • Which DDS implementation?
  • And which ROS2 version? Beta1 binary or master checkout?
  • Can you easily reproduce? Is it a consistent behavior?

We are currently refactoring our allocators in ROS2, and given the backtrace, it looks like the lifecycle implementation does use plain malloc. I’ll open a github issue on that.

3

I build ROS 2 beta 1 from sources. Running it on Linux ubuntu16-vb 4.8.0-41-generic #44~16.04.1-Ubuntu SMP Fri Mar 3 17:11:16 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

The tutorials work, don’t get me wrong, but after seeing some serious crashes in our code, I tried to check the samples with valgrind to see what is happening. Running “valgrind ./lifecycle_talker” always gives me many warnings like that, so I suspect there some issues with the lifecycle code. Running valgrind with some other tutorials not using lifecycle nodes seems to be relatively fine (besides a couple of mem leaks).