ROS1 shim library development proposal

In this thread with @dirk-thomas & @Ingo_Lutkebohle the idea of making a shim library to transparently have ROS1 API call ROS2 API underneath.

I’m starting the thread so we can discuss how doable this is and what are the limitations we will find (given that ROS2 structure does not exactly match ROS1).

I think the aim would be to have both a C++ and a Python shim, I’m up for doing work on the Python part (which I’m way more familiar with).

In Python a few dirty monkey-patching tricks on rospy may get something working relatively quickly without doing a full rospy client shim.

I don’t know what could be the approach to minimize or straight away remove the need to touch code on the C++ side.

I guess all the stuff that needs to be shimmed is:

  • Node creation/management
  • ROS time
  • ROS console
  • Msg & Srv
  • Topics
  • Services
  • Parameters

I think actionlib, dynamic reconfigure, TF, etc all build upon those so most stuff should work.

2 Likes

The option of modifying roscpp and rospy is likely faster and more backwards compatible. It might also help with the other maintenance efforts on ROS1.

That said, if we could eventually transition to something built on top of rcl, it would allow is to get rid of the legacy codebase.

Steps for roscpp, with several design choices to make that also apply to rospy.

  1. For hacking the old codebase, we need a rosgraph replacement as well, to get the info about the system. It might be a good start to have a rosmaster node, maybe based on the ROS2 info server (not sure what that’s called right now). In a later step, we could replace the roscpp graph calls with queries of the rmw info directly. Or maybe it would be easier to do that right away, that would be something look at.

  2. rostime and rosconsole I would have to look at in l more detail.

  3. In roscpp, the transport is already pluggable, so it “should” be straightforward to replace with rmw.

Topics and services are a fairly small layer above that, maybe we can even reuse the existing ones.

  1. Parameters live in the node in ROS2, which is not a big issue in itself, but we would also need to provide the parameter node interface. Or we could support the old parameter server in the master, which would again be more backwards compatible but is a break from how ROS2, particularly its launch system does it

  2. ROS2 node interfaces are required for full integration with ROS2. We have a port of the ROS2 lifecycle to ROS1 already, which could be used to build the lifecycle interface. Not sure about the rest.

Btw, all that’s just of the top of my head right now, because I’m traveling with bad Internet access and can only email irregularly. As well as having family things to do :wink: Expect more interaction starting from 10th of September.

However, bouncing around some of the design questions should be possible. I’ll try my best.

Cheers, Ingo

1 Like

It’s old, out of date, and full of dragons, but @codebot and I did some experimental hacking on a ROS 1 shim a couple of years ago:

I can’t recommend building on it, but there it is in case it’s helpful in seeing the direction we were headed. We got a few things working with that approach to shimming.

1 Like

I’d like to be clear on the goal here. Is it:

  • to allow a ROS 1 node to run on top of the ROS 2 code base so that node can be used as-is for ever more?
  • to allow a ROS 1 node and a ROS 2 node to interact?
  • to allow a node to work seamlessly with pure ROS 1 or on top of the ROS 2 code base depending on how you compile it?
  • to provide an easy path to gradually migrate a code base from ROS 1 to ROS 2?
  • to make it easier to use legacy, unmaintained ROS 1 nodes without porting them?
  • something else?
  • all of the above?

To be honest I’m not entirely convinced that a shim would increase the speed or ease of adoption of ROS 2. Interoperation is already achieved using the bridge, and maintaining a ROS2-based implementation of ROS1 seems like adding a third thing to maintain without sufficient benefit. The only place I think the shim is a long-term benefit is using legacy ROS 1 code without needing to maintain the complete ROS 1 code base - and I’d still prefer that such code is ported instead.

Still, if it’s going to happen then I’d like to see it done right. :upside_down_face:

In general, I think it would be easier to take one API at a time and, using just the headers, provide alternative implementations that use ROS 2 facilities instead of the old ROS 1 facilities. In many places suitable defaults for ROS 2 parameters, such as QoS, need to be decided; they need to mimic as close as possible the ROS 1 behaviour.

I think that providing a master implementation using a ROS 2 node works well. The graph info daemon in ROS 2 should provide the information that rosgraph needs.

Logging works a little differently in ROS 2 but the API is I think wrappable in rosconsole’s API. The functionality is close enough.

The ROS1 parameters API would need to be reimplemented to use the ROS2 parameter API but I don’t think that would be too difficult. The more difficult part is deciding how to provide the ROS1-style behaviour. I think it would be relatively trivial to recreate the parameter server using a ROS2 node and have the shimmed roscore launch that node instead of the old parameter server.

A ROS 1 node doesn’t require a life cycle. This is starting to get into “turn ROS 1 nodes into ROS 2 nodes” territory, which means adding new APIs to ROS 1. I don’t think that is a useful goal. If someone wants their ROS 1 code to operate on top of the ROS 2 code base and benefit from all the useful features of ROS 2, they should port it to ROS 2. Otherwise the shim will contribute to keeping the community split across two separate APIs rather than on a path to eventually converging on ROS 2. The shim should be limited only to mimicing existing ROS 1 behaviour using ROS 2 as the underlying middleware.

1 Like

Hi Geoff,

Good questions. Short term, I have two issues I’d like to solve: 1) The ROS1 middleware has no security, and is therefore problematic in products. Many other issues I can address on a per-node basis, but security is reallu pervasive. 2) the ROS2 ecosystem lacks some features, particularly in libraries, that ROS1 has.

Long term, I’d like to have joint community efforts as much as possible. The bridge is problematic in this respect, mostly because of tooling. A shim for using some ROS1 libs is also problematic, but less so, IMO.

So, I would describe the goal as being able to use a ROS1-API-based node implementation in a ROS2 system as seamlessly as possible, including launching it using launch, etc.

Where to go from there, e.g. doing a full port, or keeping it like that, I can’t really say right now. I would expect that the advantages of ROS2 eventually lead to a full port. However, the shim gives users much more freedom on when to do so.

One general comment here: When discussing porting, please do not forget QA. QA for a product is quite different from what many in academia are used to. Porting a full product in one big effort is essentially a no-go, particularly, but not only, for the many applications where only a small number of units are ever sold.

Thus, a shim that starts from the existing ROS1 libs and their semantics stands a much better chance than reimplementing these APIs based on ROS2. Changing the middleware is enough of an issue as a first step. We can then gradually replace more over time, once we have a better idea of the QA issues.

Cheers,

Ingo

1 Like

I definitely agree that this is a limitation with an outsized influence. However, I’m interested to hear what you think about why SROS is not sufficient. (I have my own opinions, but I’m not a security expert and want to hear yours.)

I also agree that many libraries are lacking. It would be nice to see things like MoveIt come to ROS2 rapidly, and as a major library it might do, but there are lots of small libraries that are also useful and are less likely to be ported soon.

What sort of tooling is the bridge a problem for?

“Launching it using launch” could imply many things, from ros2 launch mimicking the simple process starting of roslaunch to having a ROS1 node appear to be and behave as a native ROS2 node. I don’t see a problem with the former, but I feel that the latter is going beyond what is necessary. I feel that if a node needs to behave indistinguishably from a ROS 2 node, it should be ported.

I understand the QA concerns of industry. However I disagree that a shim that “starts from the existing ROS1 libs and their semantics stands a much better chance than reimplementing these APIs based on ROS2”. By replacing the underlying framework with ROS 2 you are making major changes in the behaviour of the node. It’s not just a matter of swapping out the communications middleware, you are also suggesting things like changing the behaviour of a node to meet the ROS 2 semantics via the shim. Interoperation via having the same middleware is one thing, but behaving like a ROS 2 node, which is what is implied by “being able to use a ROS1-API-based node implementation in a ROS2 system as seamlessly as possible” involves major changes in the behaviour of the node even though the node’s source itself has not changed. Changes range from how parameters are delivered to how execution of the node is managed. I think the QA effort involved would be less, but not significantly so.

Nevertheless, I do agree that even if QA is not reduced then re-implementation time is reduced by having a simple shim that provides some level of interoperability between a ROS 1 system and a ROS 2 system without using a bridge, which is a benefit. I just disagree on how far the shim needs to go.

1 Like

Good questions. Short term, I have two issues I’d like to solve: 1) The ROS1 middleware has no security, and is therefore problematic in products. Many other issues I can address on a per-node basis, but security is reallu pervasive. 2) the ROS2 ecosystem lacks some features, particularly in libraries, that ROS1 has.

I don’t know what is the best and most realistic path to achieve them, but I personally think the long term goals are twofold:

  1. Remember that this thread was started from a call-to-help for maintaining the core ROS1 libraries. In order for the shim to have any impact on that front, it must eventually be able to replace the ROS1 core library implementation with an implementation based on ROS2. This implementation must be able to mimic the behaviour of ROS1 to a very large degree in order to be an acceptable replacement. The users of the shim are those that want to or need to keep using ROS1, for whatever reason. Their node implementations don’t need to change, since the API is unchanged. Only then can maintenance of the current ROS1 implementation be dropped. Until it is complete, there is little benefit for “reduced maintenance effort”. I have no idea how realistic this is, but it is a big task that can probably not be called a short-term goal. However, if reducing ROS1 maintenance efforts is a goal, this has to be kept in mind.

  2. Different from the first point is minimizing the migration effort to use ROS1 libraries in a ROS2 context. The bridge fulfills this need to some degree, but it seems people feel like it is not enough (I can’t comment on that). With ROS2 being largely incompatible, the fear of some people has been that the community is splitting between ROS1 and ROS2 with little synergies between them, and while OSRF is focusing most efforts on ROS2, ROS1 is left behind. Think of all the small libraries and nodes that are available in ROS1, but are very unlikely to be ported to native ROS2, since the effort of porting and maintaining two implementations is just too much. In ROS1 there was even talk of adopting a rolling-release model and there is a team of maintainers just for re-releasing working/useful packages, that are not even being re-released into the latest distro by themselves, so you cannot expect that anyone will re-write the majority of these packages for ROS2. Here the shim would allow these nodes to be compiled and used in a ROS2 context, without any or with very little code changes, while at the same time the same code-base continues to work with ROS1 (no matter if with the current ROS1 middleware, or with a ROS2-backed version). This way you would hope to channel maintenance efforts for both ROS1 and ROS2 to benefit each other, since they work on the same code base. Again, the semantics are tricky, since from the node’s perspective they has to be largely the same to ensure that it can work as it was written. At the same time you would like good interoperability with native ROS2 nodes, therefore some semantics will probably have to change in some respects when compiling for ROS2. This need will probably be hard to balance.

I don’t know how realistic either of the two goals really are, but if either or both are not even considered potential mid-/long-term goals, this should be stated from the beginning so expectations of where this is headed don’t get mixed up. If they are considered potential mid-/long-term goals, any short-term effort should ideally aligned with working towards them.