CycloneDDS roadmap - multi-network, async, content filtering, QL1, et al

joespeed · November 26, 2020, 11:53pm

Eclipse Cyclone DDS contributors want your input for roadmap to support the ROS community. We’ve had many inquiries about what is planned so wanted to show where to find the roadmap. We would love your comments and input here and via github issues.

CycloneDDS 0.8, Dec 2020

C++11 APIs (GA)
New APIs to access to serialized CDR data
New APIs for configuring the deployment of Cyclone DDS based application
Multi-Network interface cards support (Consolidation)
Content filtering support for C++
IDL compiler front-end & back end support of the Xtypes annotations
Topic and data type discovery
ROS 2 Quality Level 2
- Automated performance and regression testing
- Formal feature list also showing test coverage
- APIs marked stable (each API in formal feature list), evolving or experimental

CycloneDDS 0.9, March 2021

Integration of Durability Service for Transient and Persistent data
Shared memory transport support … including zero-copy API
Python APIs (GA)
Internet-scale deployment support
DDS-Xtypes APIs
Asynchronous mode of operation
ROS 2 Quality Level 1

CycloneDDS 1.0 June 2021

Static discovery
Static memory allocation
Writer side filtering support
Content Querying APIs

Long-term milestones

Network Mobility support
C# language binding support (DDS C# API)
Java language binding support (Java 5 Language PSM for DDS)
Time-sensitive Networking support DDS-TSN
Rust language binding support
Certifiable DDS
FACE 3 support
Network Scheduling and Federated architecture support
TrustZone support for DDS Security

github.com

eclipse-cyclonedds/cyclonedds/blob/master/ROADMAP.md

# Cyclone DDS Roadmap

## Introduction

The Eclipse Cyclone DDS project aims at implementing the OMG DDS core standards, namely:

 * [Data Distribution Service][1] (DDS)
 * [DDS Interoperability Wire Protocol][2] (DDSI-RTPS)
 * [DDS Security][3] (DDS-SECURITY)
 * [Extensible and Dynamic Topic Types for DDS][1] (DDS-XTypes)

We ultimately aim to comply with the latest version of above OMG standards. Cyclone DDS supports currently the following language bindings for the DCPS APIs:

 * C language
 * C++ language ([ISO/IEC C++ 2003 Language PSM for DDS][5]) (DDS-PSM-Cxx)

The community is welcome to add new language bindings and contribute to the implementation of other OMG specifications.

The roadmap presented here is indicative and subject to change.

This file has been truncated. show original

JWhitleyWork · November 25, 2020, 4:38pm

What is the current status of zero-copy transfers? If it is not yet implemented in Cyclone, where does it fall in this roadmap?

mikeferguson · November 25, 2020, 7:11pm

Very glad to see “Multi-Network Interface card support” - this is about the only thing that has tripped me up with CycloneDDS. Will this magically work with rmw wrapper once CycloneDDS 0.8 goes out, or will it require updates to the RMW to use it?

sjames · November 26, 2020, 4:49am

Exciting roadmap!

Static memory allocation looks interesting. Will also be nice if language bindings can override the allocator.
TrustZone. I’m curious about this. Shouldn’t this interface be handled by security plugins? As far as I know, there is no standard software interface to a TrustZone.
Any plans to support the DDS-RPC spec?

Cheers!
Sojan

tomoyafujita · November 26, 2020, 5:00am

This is really nice, since we have been working on ContentFilteredTopic Design and RMW Proposal. thank for sharing the roadmap

eboasson · November 26, 2020, 3:32pm

There is a shared memory prototype that makes a copy in shared memory, for true zero copy the application needs to reserve the space rather than rely on Cyclone to do it internally, so it needs some work still (mostly in the periphery of the code base, but still).

If you look at the performance curve you get with that extra copy in, it hurts. So in my view “shared memory” support really means: “zero-copy shared memory support”. Otherwise, it is just isn’t fun

eboasson · November 26, 2020, 3:36pm

It’ll “magically” work. All these details are handled by the configuration XML, and so it mostly depends on how far I am willing to go in changing the default behaviour. Should I decide to stick with the current behaviour (just one interface) for the time being, you might have to instruct Cyclone to do something else via that configuration XML.

(Actually, there is a working prototype: https://github.com/eboasson/cyclonedds/tree/multi-home-red-netw — feel free to play with it. I can only help for getting it right.)

eboasson · November 26, 2020, 3:42pm

Static memory allocation looks interesting. Will also be nice if language bindings can override the allocator.

Good point. I need to think about that, but I can see where you’re coming from

TrustZone. I’m curious about this. Shouldn’t this interface be handled by security plugins? As far as I know, there is no standard software interface to a TrustZone.

Yes, so it is plans for plugins that forward the operations to code running inside the TrustZone. But this is still a bit vague as to the details.

Any plans to support the DDS-RPC spec?

Hmm … not really. It is an abomination that takes a pub/sub-based shared data space and then abuses it to do point-to-point request-reply, and in typical DDS specification-style it also implies a lot of work. There are more valuable things to be done, I’d say. In my opinion, it makes far more sense to use DDS to publish server locators, then use a standard RPC mechanism to invoke those services.

But if someone were to contribute an implementation (it’s mostly preprocessing anyway) of course I would be grateful!

JWhitleyWork · November 26, 2020, 5:19pm

Thanks for the info, @eboasson! No ETA on that at this time, then?

eboasson · November 26, 2020, 8:01pm

I’d say March 2021, but I can see how one could read my initial response differently. Sorry about that!

joespeed · November 27, 2020, 12:39am

… with built-in Eclipse iceoryx. Heard good things from people that tried it. I think up to 4x improvement for point clouds & images. March release with zero-copy API will improve significantly upon that.

github.com

eclipse-cyclonedds/cyclonedds/blob/iceoryx/docs/manual/shared_memory.rst

.. _`Shared Memory`:

#############
Shared Memory
#############

The documentation is to describe the Proof of Concepts detail of supporting shared memory in Cyclone DDS.
The shared memory implementation is based on `Eclipse iceoryx <https://projects.eclipse.org/proposals/eclipse-iceoryx>`_.

*****
Build
*****

The following step is under Ubuntu 18.04, but other Linux distribution should also work, too.

Before using iceoryx, there are some dependecies which should be installed.
These packages are used by iceoryx.

.. code-block:: bash

This file has been truncated. show original

michael-poehnl · November 27, 2020, 6:14pm

Can’t wait to see Cyclone 0.9 in March. The full power of true zero-copy shared memory communication will be leveraged when the API is adapted accordingly and the ROS2 loaned messages are used.

Does “Topic and data type discovery” will include https://github.com/eclipse-cyclonedds/cyclonedds/issues/42? That would be good news too

eboasson · November 30, 2020, 3:38pm

Does “Topic and data type discovery” will include https://github.com/eclipse-cyclonedds/cyclonedds/issues/42?

Yep, it does

connoranderson · November 30, 2020, 7:40pm

Yes, this would be great!

joespeed · November 30, 2020, 7:53pm

@connoranderson CycloneDDS Rust support is well progressed with these Rust bindings and zenoh-plugin-dds

MichNiez · December 1, 2020, 3:57pm

Certifiable DDS

I know it is a long term goal but can you comment how you envision reaching safety certification?

spiderkeys · December 2, 2020, 5:52pm

Overall, looks like a good roadmap! Wanted to weigh in with some +1’s and comments on other features that I have utilized or found useful over the years working with DDS.

First with the +1’s on features that I think are really important and that we have an immediate need for within our company’s stack:

Network/IP Mobility (similar to RTI’s https://www.rti.com/blog/going-mobile-with-connext-dds-5.3)
Multi-Network interface cards support
SHMEM & Zerocopy SHMEM support
C++11 and Python APIs/bindings
Static discovery
Static allocation

The multi-network and IP mobility features are critical for us in the long run, especially given that our robots and user devices frequently roam across different LAN networks, often times residing on multiple at the same time and sometimes having both static and DHCP addresses on a single network interface. We’ve already run into a couple of issues around discovery and consistency in maintaining ROS2 connectivity between nodes when using non-RTI RMW implementations, so this will be critical for us in ensuring that in all of the types of network environments and configurations that our customers use, the RMW is well supported and behaves correctly.

SHMEM/Zerocopy, of course, is of huge importance, given that we are deploying high-bandwidth, multi-camera systems on embedded processors like the Jetson Nano, Xavier, etc.

Some additional suggestions:

Support for chainable transport plugins via a transport plugin API, similar to what RTI provides support for:
- https://www.rti.com/hubfs/docs/DDS_Over_Low_Bandwidth.pdf
- https://community.rti.com/static/documentation/connext-dds/6.0.1/doc/api/connext_dds/transport_plugin/group__NDDS__TransportPluginComponent.html
- This feature is very useful when trying to customize or optimize existing transports, and I have used this before to enable features like tunneling DDS data transparently through other protocols and technologies (such as funneling the end RTPS traffic for multiple participants through a single UDP port/address, or a custom SHMEM interface, GPU/hardware interface, etc) for the purpose of teleoperations and high-performance processing. It seems like Zenoh is one good method for providing this kind of routing, but there is some overhead in having to pass through another first-class DDS entity to create this type of flexibility, so it would be great if there was a flexible API for building these features on top of the DDS transport system as plugins.
Not exactly a CycloneDDS feature, but in the Cyclone RMW it would be great to have access to features at node-creation time that allow programmatic configuration of aspects like specifying partitions on the Publisher/Subscriber entities. I realize that this is better discussed in the context of the “how do we provide ROS2 users a common interface to DDS features” discussion, but in the meantime, I think it might be easy to streamline some of these “extensions” into the vendor RMW layers. Partitions in particular are of critical importance to us to support efficient multi-robot <> multi-user-device (i.e., apps running on mobile devices & laptops) environments.

Love the work that the Cyclone team has been doing so far, and look forward to the next couple of milestones! Only request I would emphasize (selfishly) is that the network mobility work be prioritized a bit more highly, if possible

eliasdc · December 11, 2020, 10:45am

Thanks for the update.

We have been trying out Cyclone DDS and found it has a big improvement over FastDDS with regards to larger and distributed setups (multiple machines). If we can we’ll share some of our testing results but it still needs some clean-up.

In order to fully adopt Cyclone DDS we require the Multi-Network interface cards support, we’ll try out the https://github.com/eboasson/cyclonedds/tree/multi-home-red-netw branch and report our feedback/experience.