Rosimport : python is dynamic

Hi everyone,

Python is a dynamic language, and ROS integration with python should fit the language habitual usage and be dynamic.
Based on that main thought, catkin should be made completely redundant in python.

I have been working last year on two projects as proof of concept for this idea :

With these, the whole artificial catkin build, and source devel/setup.bash, is now totally unneeded for python ROS code. A python dev dev can follow the usual python development flow : create a virtual environment and just run a REPL or some python code. I personally believe that, done well, this has the potential to greatly simplify and speed up python ROS development (usually the main point of using python)

These projects are tested and working, however they are in need of feedback, documentation and general love and care.
So if you are interested in this kind of experiment, give these a try, and let me know what you think !

4 Likes

How does this look to someone who has a workspace with a mix of C++ and Python packages? With Catkin if I have that, and I have messages in the Python package, Catkin will ensure they are available from the C++ package. Based on your description of pyros, I get the impression that it’s only suitable for the equivalent of a pure-Python workspace, and if I have stuff that needs to be shared with C++ packages then I need to deal with that in a workspace for that. Is this correct? It’s not necessarily a bad thing, I just want to be clear on the limitations.

Cool work man! As I also mainly work on ROS via Python I find your work very interesting, altho as I tend to work with teams that also code in differente languages a catkin_ws is usually needed.

You may be interested in my project: https://github.com/uts-magic-lab/message_thief

In there I made a handy function (https://github.com/uts-magic-lab/message_thief/blob/master/recreate_msgs.py#L479 ) that given a message definition (the .msg / .srv) and the package name, generates a fake package that can be compiled in a catkin_ws for any language to use that message. It also has tools to “steal” from a robot these definitions.

I gave a lightning talk about it in ROSCON: https://vimeo.com/236177042#t=20m12s (seems like the link to start at the 20m 12s mark does not work, sorry!).

Indeed these focus primarily on pure-python environments for proper ROS
python integration.

The setup layout assumption that I work under is the usual software one :

  • work on one repo at a time ( or tracking old bugs across different
    packages and versions is a nightmare )
  • know your dependencies and how they integrate with your own code ( or
    upgrading your system will break your code without your knowledge )
  • continuous integration : update/test/deploy as often and as fast as
    possible ( or you will break what already works beyond any hope of repair )

So for ROS and C++/Python that means I usually expect the C++ packages are
built and deployed to the machine when a dev work on a python package (on
its own, in a virtualenv), relying on these packages.

That being said, messages are still the same, and they are located in the
same place, so if someone wanted to go through the trouble of writing a
CMakeLists.txt file for them, he/she could get the C++ code part to compile
and run while having the python code working there, all from the same
workspace. But I see more risks than benefits in doing that, so I dont want
to encourage it. Very few actually know or even care about how dependency
trees are resolved in static and dynamic time for both C++ and Python and
are able to solve a problem that may occur with a mixed complex setup.

Also I understand that the usual way to do things with ROS is to have a
separate dedicated package for message definitions only, so in that case it
is not an issue, even in the same workspace.

rosimport is known to work only at the development phase so far. I still
need to work out some details for packaging (with pip) python packages that
use it.

1 Like

Interesting stuff !

Thinking about your TODO for instant connection to a robot, it looks like you got the “discovering” messages part working already there, and you could use a similar trick than rosimport to import the messages in your code (instead of generating a workspace)

You might want to have a look at https://github.com/pyros-dev/rosimport/blob/master/rosimport/_ros_generator.py
Maybe there are some parts we could factor out, or maybe patch or standardise some kind of API in genpy and genmsg via a REP, to enable this kind of dynamic tools…

Thanks for putting this up @asmodehn, I was really interested in your work in pyros, and found catkin_pip to be a very useful jumping off point.

We were faced with related challenges w.r.t shipping python packages, and I ended up breaking out and releasing our solution (https://github.com/locusrobotics/catkin_virtualenv/).

It comes at it from the opposite direction, which is bundling a virtualenv with dependencies in each package as part of the CMake build. I’m also looking forward to poking at snappy instead, which may offer a better layer in which to bundle a virtualenv (bunch of ROS packages + one agglomerated virtualenv inside a standalone snap).

Catkin_virtualenv look interesting, I shall try it sometimes… But I hope that eventually we will get rid of the whole build process, completely redundant in python.