C++ AnySubscriber and AnyPublisher ROS Babel Fish now available

General
1

Greetings everyone,

I’m here to announce the open-source release of ros_babel_fish which enables you to subscribe to and publish on topics with message types unknown at compile time.
Calling services and providing a service server is also supported.
The main focus is on usability, and the message structure is retained.
There are no limitations (that I know of) regarding the complexity or size of the message, e.g., array size.
Dependencies are limited to OpenSSL (for MD5), roscpp and roslib.

For a benchmark when decoding an unknown message, click here. The library is benchmarked against the existing solution for subscribing to unknown message types ros_type_introspection.

My use case is a plugin for QML that enables publishing, subscribing and calling services directly in QML.
This plugin will be released in the very near future.

It’s currently in alpha state and bugs are expected. Tests for ~95% line coverage are included.
The library was developed and tested using Ubuntu 18.04 and ROS Melodic.

License: MIT

I’m looking forward to hearing your feedback and use cases!

Kind regards,
Stefan Fabian

2 Likes
2

Thanks for sharing this. Does this package something different than variant_topic_tools? Or does it have the better performance (I never checked the performance of variant_topic_tools)?

3

Good question!
To be quite honest, I hadn’t even heard about variant_topic_tools until your comment.
I’ve had a quick look into it, but given the lack of any concise examples or tutorials, it’s hard to tell.
Can’t speak of performance without evaluating it.
However, it looks like it creates a copy of the message’s content for everything, whereas ROS Babel Fish will lazy copy large primitive arrays and only reference the position in the message buffer. This might evaluate messages with large array fields such as images or point clouds faster but given the maturity and judging from a quick glance, good code design of variant_topic_tools, I assume it might perform better in other use cases.
Without examples, I can only guess the differences based on the available documentation.
I assume the main functionality of variant_topic_tools is to subscribe to an unknown message type, modify and republish it?
I haven’t seen any mechanism to load message types where you do not have the definition which is something this library can do by looking up the definition in your workspace similar to the rosmsg/rossrv utilities.
Also, this library supports both calling services and providing a service server, though, I must admit that I have currently no use case for the latter. I haven’t seen anything indicating support for this in variant_topic_tools.

Action clients are also on my TODO list.

One of the advantages of this library is that it provides examples on how to use it and hopefully, convenient access functions that promote less and easier to understand code.

4

Thanks for all the good work @StefanFabian :slightly_smiling_face:

I had a similar question. Have you looked at ros_type_introspection, this package has similar use case. Does babel_fish offer some extra features, haven’t taken a look at either one of these yet so not an expert, maybe you can shed some light.

5

As @Asif_Sattar mentioned, ros_type_introspection is very similar, even if babel fish seems to have a “nicer API”.

ros_type introspection is focused on achieving the fastest possible performance and it is currently used by PlotJuggler.

Something that “surprise” me is that you mention that you don’t need to know the message at compilation time but only at run-time.

This is not exactly true, because you have a chicken-and-egg problem, in my opinion.

Let’s consider your example

std::cout << "Position: " << compound["position"]["x"].value<double>() << ", " 
          << compound["position"]["y"].value<double>() << ", "
          << compound["position"]["z"].value<double>() << std::endl;
  std::cout << "Orientation: " << compound["orientation"]["w"].value<double>() << ", "
            << compound["orientation"]["x"].value<double>() << ", "
            << compound["orientation"]["y"].value<double>() << ", " 
            << compound["orientation"]["z"].value<double>() << std::endl;

In your source code you are making a strong assumption about the hierarchy of the message.
You know exactly the fields of the message and the hierarchical composition.

Therefore, you can’t really say that you can deal with “any” message.

Am I missing something? What do you think?

6

By the way, just for fun, one of these days we should build a benchmark that compares variant_topic_tools, ros_type_introspection and ros_babel_fish.

The latter definitively wins the price as the “most catchy name” :wink:

1 Like
7

Similar to how ros_type_introspection works, it would seem it’s possible to iterate over fields in messages (see here fi).

That would make the strong assumption something specific to the example you refer to.

1 Like
8

I didn’t want to get technical here and only give an introduction with what I believe are the most important infos. But as I’ve mentioned, I’ve benchmarked it against ros_type_introspection, so, yes I do know about it :wink:
I have actually started with using ros_type_introspection but I needed something to be able to use messages in QML and it should be able to publish messages as well which is something that to my knowledge ros_type_introspection was not designed to do.
The former was already possible but kind of a hassle because the flattening of the message had to be reverted.
Ultimately, it has a different design philosophy due to a completely different main use case. Whereas ros_type_introspection was primarily designed with a focus on the visualization of values in PlotJuggler (I assume) and therefore, it makes sense to flatten the hierarchy and the focus is more on speed than ease of use.
I, on the other hand, want to preserve the message hierarchy (and large array fields).
So both have their separate use cases.

1 Like
9

Of course, but you need to realize that the focus here was to use ROS messages in QML where you may know the message types but you can’t include them or otherwise use them without significant extra work.
Also you can theoretically just traverse the message and fill the fields you want. For example, you could search a message for pose messages and fill the appropriate values.

Have a look at the troll_node in the example folder which traverses the messages hierarchy and replaces all string fields with the lyrics of Rick Astley’s Never Gonna Give You Up.

PS: Since I know that you value performance a lot, you may want to have a look at BabelFishMessage which is similar to topic_tools::ShapeShifer but exposes the buffer field so you don’t have to copy the message buffer. This could improve your performance for large messages significantly.

10

The former was already possible but kind of a hassle because the flattening of the message had to be reverted.

Indeed, this is the most important strength and weakness of my software :smiley:

Anyway, as I said before, great work.
If BabelFish offers any performance improvement in my use case, I will be very happy to use it.

1 Like
11

As the benchmark shows, it currently does allow quicker access for larger messages but I assume that if you remove the unnecessary copy of the message buffer by creating your own version of the shape shifter message (you can copy and modify the code for the BabelFishMessage, if you want), your code might perform better for your use case.
There are some simple performance improvements that I could implement that would benefit performance for your use case, though. For example, the location of a value in the message could be evaluated into a formula that can be quickly evaluated without decomposing the entire message.
If you’re only interested in a particular value for plotting, that would certainly mean a big performance improvement.
Not sure, if you did something like that already.

Another reason why I couldn’t use ros_type_introspection that I just remembered is that it uses a custom version of abseil-cpp which clashes with our workspace since another piece of software builds abseil from source as a dependency.
That’s also an advantage of this library which comes with almost no dependencies.
I could remove the OpenSSL dependency if someone implements MD5 under MIT License or another compatible open source license.

12

If only that would be true :stuck_out_tongue:

I have been thinking about this a LOT and the answer is NO, it is not possible in the general case, because whenever you have a vector of vectors (complex types) you can not make any of these assumptions.

But I would love to be proven wrong :smiley:

Another reason why I couldn’t use ros_type_introspection that I just remembered is that it uses a custom version of abseil-cpp which clashes with our workspace

Using abseil is very nice but, as you mentioned, it create this kind of problems. I am not interested to develop ros_type_introspection any further (because it serves me well) but if I do, I will try to fix the problem with abseil.

13

And I like challenges :wink:
It’s not always at the same location, that’s true and that’s why I said you can build up a formula.
Luckily, I currently have a lot of free time to use.

14

Thanks for sharing mr. Fabian.

Very good job.

1 Like
15

Done. Please check the updated benchmark. I’ll see if I can do a proper benchmark but probably not in the next couple of days. If anyone else wants to do a proper benchmark, I’d gladly help with the usage of this library.

Also, check the README for an introduction to the new MessageExtractor and these tests for some usage examples and complex tests involving a message with multiple arrays, some variable length, some fixed and string arrays and arrays of complex types.

It’s probably the fastest possible way to extract values without knowing the actual type (removing the error checks would make it a bit faster but I’m a fan of defensive programming).

16

Nice, I will have a look and add my own benchmarks :+1:

1 Like
17

Just a quick update since the benchmark links are dead.
Benchmarks were moved to a separate repo.
It was also divided into benchmarks from bag and benchmarks using Google’s benchmark library.

18

challenge_accepted

A pull request to your benchmark repo is on the way…

20

I just pushed the new commit to ros_type_intospection, i.e. the option to save large blobs either as a copy or a reference.
This is the result of the benchmark, where RTI means ros_type_introspection and RBF stands for ros_babel_fish.

-------------------------------------------------------------------------
Benchmark                               Time             CPU   Iterations
-------------------------------------------------------------------------
RTI_ParseMessageDefinitionPose       92.2 us         92.4 us         7266
RBF_ParseMessageDefinitionPose       5721 us         5721 us          122
RTI_ParseMessagePose                  375 ns          375 ns      1877861
RBF_ParseMessagePose                 1991 ns         1991 ns       350942
RTI_ParseMessageJointState            618 ns          618 ns      1131938
RBF_ParseMessageJointState           1844 ns         1844 ns       382951
RTI_ParseMessageOdom                 1232 ns         1232 ns       557055
RBF_ParseMessageOdom                 6160 ns         6159 ns       112515
RTI_ParseMessageTF                   1809 ns         1809 ns       384416
RBF_ParseMessageTF                  10337 ns        10337 ns        69067
RTI_ParseMessagePointcloud            581 ns          581 ns      1195044
RBF_ParseMessagePointcloud           3291 ns         3291 ns       214058
RTI_ParseMessageFullHDImage           327 ns          327 ns      2141815
RBF_ParseMessageFullHDImage          1929 ns         1929 ns       360447

If the only problem with RTI is the mess I do with abseil_cpp, then there are easy ways to fix that, without rewriting it from scratch.

I don’t say this as a critic to you, @StefanFabian, it is nice that people try their own solutions. But collaboration is what makes open source great :slight_smile:

21

I do agree with this but I just want to clarify for anyone reading this that this is not the case for RBF (because your comment may be interpreted as if that was the case).
RBF follows a different design choice which keeps the message hierarchy rather than flattening the message and it allows for the sending of messages even without ever having received a message of that type.
It is intended for use cases like UI buttons that can be configured to send a specific message or call a service on button press that can be configured using xml files.
But I’m not gonna repeat all the differences, I’ve listed them somewhere above in this thread.

PS: It’s also currently not optimized for speed at all, so I’m not surprised that RTI is faster with the zero-copy mechanism I suggested.