Lanelet2 ROS Message Design

mitsudome-r · 2019-07-17T10:17:01.413Z

Currently, there is no ROS message defined for Lanelet2. The official Lanelet2 seems to take some time to define them. https://github.com/fzi-forschungszentrum-informatik/Lanelet2/issues/7#issuecomment-458691411

However, it is urgent to replace current vector map format with Lanelet2 since the closed format is slowing down the development of other modules (e.g. perception and planning). Therefore, I suggest we define ROS message with Autoware community and use it for the meantime. Perhaps we can create PR to official repository if it works well within Autoware.

For Lanelet2 ROS message definition, I have considered three approach so far.

Multiple Topics Approach
This is similar approach with current vector_map_msgs. Map data is divided into multiple topics based on object types(point, line, lane, signal, traffic sign, etc.), and other nodes only subscribe to interested object. This keeps away from different nodes having all copy of map data which increases memory efficiency.

If we are going to use similar design for Lanelet2, we probably would divide topics into premitives: point, linestring, lanelet, and regulatory elements. (See here for lanelet2 primitives https://github.com/fzi-forschungszentrum-informatik/Lanelet2/blob/master/lanelet2_core/doc/LaneletPrimitives.md#lanelet-primitives) One of the concerns is that if we are going to use Lanelet2 library, then a node would need to subscribe all the topics anyway and have no merits of having separate topics.

Merits:
- Memory efficient when there are many nodes subscribing to map data (but not likely with Lanelet2)
- Easy to look for specific object from rostopic echo
Demerits:
- Needs to synchronize the topics if there are any updates at run time
- Needs to subscribe many topics to obtain map data as a whole

Single Topic Approach (XML)
Map will be stored as xml data as string member of the message. Lanelet2 has functions to write maps into OSM(XML) format file, and we can write into ROS string message instead of writing into filestream. This way, we can convert the message back to LaneletMap(C++ class used in Lanelet2 library) just like loading map from a file. Developers can also check data with rostopic echo and read the message data in XML format. I do have concern that output rostopic echo do not process newline, and it just outputs the code itself(’\n’). Therefore, although output of the command is in XML format, it is very hard to comprehend.

merit:
- no need for synchronization (single topic)
- can reuse code from Lanelet2 writer/parser
- message data will be exactly same as loaded osm file
demerit:
- data size would be relatively large since they are in text format

Single Topic Approach (Binary)
This approach would be same as the second approach, except that the data will be stored as binary instead of XML. Lanelet2 also has function to write maps in binary format(BinWriter class), so we can also reuse the code for this approach as well. Since the data will be binary, data size should be smaller, but output from rostopic echo would be totally incomprehensible. However, I think this wouldn’t be a problem if we have good visualization in RVIZ. (In fact, pointcloud data is using binary format for ROS Message and have been working so far).

merit:
- no need for synchronization (single topic)
- can reuse code from Lanelet2 writer/parser
- data size will be smaller than XML data
demerit:
- rostopic echo output would be incomprehensible

I prefer the third approach since I never used rostopic echo to track down the issue related to vector map in my experience. I have used to check if the topic is published or not, but I never used it to actually check the content of topic data.

Any suggestions or feedback would be appreciated.

simon-t4 · 2019-07-18T10:36:13.009Z

Thanks for opening the discussion Mitsudome-san.

I agree on your comments, working with the binary topic has been the most easy so far.

For 2) there is also the need to project back into long/lat to write the xml file, and then on the subscriber to project into the lanelet map structure again. Not such a big problem, but for MGRS projection there was some issue with the MGRS base code sharing for reverse projection? Anyway, it introduces a layer of processing not needed by binary format.

For 3) is it an option to publish the full map, and nodes that desire subsets of the map (either geometric or primitive subsets) could use a ROS service to ask for a defined subset - the map loader could then construct a purpose built lanelet map of the subset, and send in single topic format?

Similarly for 1) rather than define individual message types and filling out all information, subset maps (containing for instance just point primitive layer) could be generated and published as subset lanelet maps and recombined if necessary at the subscriber. This would simplify the message structure and take advantage of the existing API to construct messages.

Hatem · 2019-07-18T14:35:46.434Z

Thank you for the detailed explanation,
I agree with you, option 3 is good.
also lets consider that in the future map information will be updated regularly, such as dynamic maps , so we semantic based ROS topic.
currently in OpenPlanner I use option one, although we can solve the multiple topics problem by making one message holding the whole map data (or part of the map) .
the processing cost here is only rebuilding the network to achieved faster search.
example of the functions needed for planning:

Find road boundary around the vehicle.
Find next traffic light position.
Find next sign , stop line , …
Extract current lane information (center line waypoints, direction, overtaking, left and right lanes, …)

it will be good if we can build the road network and send the map as ros topic, one problem with that is data size. but if we consider a map server who send small part of the map frequently then this is the best solution in my opinion.

gbiggs · 2019-07-29T00:12:00.897Z

Please consider in your options:

Use of a map server with services to provide a subset of the map based on certain criteria (map in a specific area, lanes along a route, etc.), as well as a service that can provide the whole map.
Whether you can send structured map data rather than wrapping an XML or binary chunk in a ROS message, because this would give more flexibility for other nodes to use the data even without the lanelets2 library, and would ease debugging for those who wish to debug at the topic level. This may require modifying the lanelets2 library but you should give this consideration.
Whether using structured map data in ROS messages with that structure oriented towards use rather than source format would give more flexibility to support different map formats in the future. In other words, make your ideal map message(s) rather than lanelet2 messages.

Please also state your assumptions, requirements, and detailed use cases that will drive the decision on what sort of messages to create.

mitsudome-r · 2019-07-30T09:19:40.499Z

Thank you for your comment.
Here are some of the information that I can think of

Assumptions:

All nodes that access to map data should use Lanelet2 library. Although you mentioned about flexibility of using data without lanelet2 library, I think we should access to the map through the library if you don’t have any critical reasons. This would keep us from having similar codes everywhere in Autoware.

Requirements:

Message data must contain enough information to reconstruct LaneletMap data structure
Message should be flexible enough to be used in different use cases. e.g. message should be able to express custom tags without changing message definition
Message should be scalable. e.g. It should be able to represent subset of a map.

Detailed Use Case:

retrieving traffic light related to current lane
calculating route to goal from current position
retrieving crosswalk area
retrieving intersection information (yield, right of way)
finding stop lines
retrieve geometrical subset of map

Angelo_Mastroberardi · 2019-07-30T17:31:56.104Z

I overall agree, but need to add a couple of things

Assumptions:

if we need to access the map via lanelet2 library, we need to keep the file into lanelet2-OSM instead of plain vanilla OSM

Detailed Use Case:

retrive the nearest parking space available from the map, or a section of it

simon-t4 · 2019-07-31T01:27:37.009Z

Some thoughts on Geoff’s suggestions.

Regarding Structured vs Unstructured messages: I initially started implementing structured messages with each lanelet2 primitive having its own message type. The only problem at the time was that it was convoluted accessing the underlying data from the lanelet2 API to build each message, but now we have more experience with the API it should be relatively easy. As far as usage goes however, I don’t know if it makes sense to structure the data in messages.

The structure does not delineate any useful subsets of the data. The hierarchical nature of the format means that it hard to imagine requesting subsets of the map information that does not contain most information (i.e most requests will be at lanelet level, which requires points, linestrings and related reg.elems anyway). A lanelet message by itself does not have any spatial information, just attribute data. Conversely just sending points has little meaning either. I would argue using structured messages to send subsets of map (based on structure of format as opposed to geometry deffined subsets) would not be particularly useful. In a similar manner, nodes using the map data without lanelet2 library would have to reconstruct the lanelet data for it to be useful, leading to replication of code as Mitsudome-san mentioned. Perhaps excluding the lanelet2 library would be desirable for very lightweight nodes?
The other consideration is that structured messages could be useful for topic level debugging. My initial reaction is that map data is only coherent viewed at higher levels, and that viewing the contents of individual primitives may not be so helpful for debugging. I it might be useful to verify that well formed data is being sent, or that a particular type of lane is being sent. Perhaps visualization tools would be more helpful in this regard? However, I certainly can appreciate that topic based debugging could be useful, and am open to including structured messages if others think in necessary
We could support both? For certain cases it might make sense to debug using the structured messages, or that structured messages might encapsulation useful subsets of map data. In other cases binary data message is fast and easy.

Regarding an ideal structure for map messages oriented towards use, I conceptually agree, but in practice, how we use data is largely defined by the internal format. The messages being passed are being consumed through the internal format. If it makes sense to build a ideal message format, then why not build an ideal map format?

JWhitleyWork · 2019-07-31T05:01:01.772Z

I disagree that much code duplication would occur even if every node that utilized map data in an unstructured Lanelet2 format has to do some sort of decoding - this is the purpose of shared libraries. A single encode and a single decode function should exist (or be created) in a shared library which can be linked against by any nodes needing to handle Lanelet2 format data. The decode function would return a class structure holding the in-memory representation of the map (or subset of the map) which can be manipulated as-needed by the consuming node and the encode function would simply return the encoded, unstructured data which can be published. Minimal code duplication.

On a related note, I agree with @simon-t4 - a structured representation of the data being passed around doesn’t serve much practical purpose with the design of Lanelet2 having such a strong decoupling of the primitives from what a grouping of primitives represent. The overhead inherent in encoding/decoding them as structured messages and the additional bandwidth required for the metadata are also non-trivial. Given that network I/O is second only to disk I/O in order of the most expensive operations you can perform on a modern computer, I lean heavily toward unstructured transport. I also think the actual implementation of how we access the map or a subset of the map (making requests to a server, segmenting the map, etc.) are somewhat irrelevant to representation of the data at the transport layer. Whether we split this up into multiple nodes or have a single server handling the map, they are still passing around the same basic datum and, in the case of Lanelet2, those datum are not easily reconstructed into the objects they represent without the structure of the Lanelet2 class objects.

simon-t4 · 2019-07-31T09:09:55.325Z

@JWhitleyWork Hi, I guess the idea of code duplication concern arose because of the suggestion that some nodes might not want to link to the lanelets shared library, and instead work directly on the message data. If a shared library is to be linked anyway, we may as well use lanelets, or might there be a need for a lighter encoder/decoder lib with less dependencies?

gbiggs · 2019-08-02T14:51:26.669Z

mitsudome-r:

Assumptions:

All nodes that access to map data should use Lanelet2 library. Although you mentioned about flexibility of using data without lanelet2 library, I think we should access to the map through the library if you don’t have any critical reasons. This would keep us from having similar codes everywhere in Autoware.

If you do not know how the data will be used, you should not make this assumption.

mitsudome-r:

Message data must contain enough information to reconstruct LaneletMap data structure

This places no requirement on the message data itself being in the Lanelets structure.

mitsudome-r:

Message should be flexible enough to be used in different use cases. e.g. message should be able to express custom tags without changing message definition

Define those use cases.
Be very careful about allowing custom data in messages. You need to have a strategy in place for ensuring correct behaviour in the face of incorrect data.

mitsudome-r:

Message should be scalable. e.g. It should be able to represent subset of a map.

Although I agree with this, you need to define why it is necessary so you know exactly what a “subset of a map” means.

mitsudome-r:

Detailed Use Case:

retrieving traffic light related to current lane

calculating route to goal from current position

retrieving crosswalk area

retrieving intersection information (yield, right of way)

finding stop lines

retrieve geometrical subset of map

These are not detailed. I recommend you follow a use case template. It will help you fill out the necessary detail. A good example is the template designed by Alistair Cockburn many years ago. Martin Fowler has also produced one, and I’m sure there are thousands of other templates.

Keep in mind that this is just the start. From your use cases, you will need to derive requirements.

mitsudome-r · 2019-08-08T17:12:25.540Z

Thank you for your reply and sorry for my late response.

You are right that we should not design message without defining use cases first. I have started writin use cases of map information with Cockburn’s template(with some modifications).

docs.google.com

Use Cases of HD Map

TEMPLATE Use case name: <the name should be the goal as a short active verb phrase> Context of use: <a longer statement of the goal, if needed, its normal occurence conditions> Level: <one of: summary, user-goal, subfunction> Primary Actor: <a role...

I wrote them based on functions of current Autoware and functions that are likely implemented in the future, although it is not finished yet. We are planning to discuss about use cases and requirements in detail in the Map WG so hopefully we can provide more sophisticated use cases in next few weeks. Then, we can continue our discussion on requirements and designs.

Having said that, I would like to start Lanelet2 integration with unstructured(binary) message approach for v1.13, as it seems to be most supported message type so far. I believe fundamental purpose of replacing Aisan vector map implementation is to make Autoware’s HD Map format public (both physical storage format and internal data), and having best message design is not crucial to the purpose as long as it transfers sufficient information. Current closed format is likely to discourage developers to create new features or even enter into discussions like this, and dropping Lanelet2 features from v1.13 could be a huge loss to the community. Therefore, I suggest to start implementation with binary message in order to hedge the risk. I am aware that the deadlines for next release is longer than last release, but I would like to submit merge requests in advance because there are many nodes that uses vector_map_info/* topics that needs to be replaced.

I understand the importance of following engineering process, and we should continue designing best map message for Autoware in this thread as well as in Map WG. We will do our best to keep implementation of Lanelet2 nodes for v.1.13 to be independent from message structure so that we can replace it easily once we have the new message design.
How does this sound to you?

Angelo_Mastroberardi · 2019-08-09T15:03:50.347Z

Hi @mitsudome-r,

so, if we go for option (2) or option (3) we have something like that right ?

image.png1052×426 25.8 KB

So, because we are using the same library we don’t need a big structure of ROS msg (it would duplicate the work), and just use lanelet to manage errors in read/write a msg.

About the errors. should we have a way to send a feedback to the server in case the map delivered (XML or BIN) is good or bad ?

Also … should the msg be a requrest/response or pub/sub ? in our case, for parking garage maps, request response is good enough (you ask for the map of a given area when you are getting closer to it). Is it similar to what you are doing ?

mitsudome-r · 2019-08-15T10:16:17.436Z

So, because we are using the same library we don’t need a big structure of ROS msg (it would duplicate the work), and just use lanelet to manage errors in read/write a msg.

That was my idea.

About the errors . should we have a way to send a feedback to the server in case the map delivered (XML or BIN) is good or bad ?

I am not sure if we should have this feedback. If we are going to have one, then the server should do something when they get the error message, but I don’t know if there is anything server can do about it. Maybe sending the topic again if it is pub/sub? Also, what do you mean by good or bad? Are you talking about corruption of data?

Also … should the msg be a requrest/response or pub/sub ? in our case, for parking garage maps, request response is good enough (you ask for the map of a given area when you are getting closer to it). Is it similar to what you are doing?

Our work on replacing current Aisan vector map implementation is done by pub/sub. This is just because current implementation is done that way. We should probably think about it again once we have all requirements listed up.

gbiggs · 2019-08-15T12:57:49.244Z

mitsudome-r:

I understand the importance of following engineering process, and we should continue designing best map message for Autoware in this thread as well as in Map WG. We will do our best to keep implementation of Lanelet2 nodes for v.1.13 to be independent from message structure so that we can replace it easily once we have the new message design.
How does this sound to you?

If you want to do it that way for Autoware.AI, then I don’t really mind. But remember two important things:

The Maps Working Group is for the Autoware Specification and its reference implementation, Autoware.Auto. You cannot ask the Maps WG to make decisions based on a need from Autoware.AI.
If you make a hasty design decision now to get something out soon, you are encruing technical debt. This is an accepted software engineering practice, but in this case you will be paying back that debt by re-implementing all your Lanelet2-related stuff again later on.

Angelo_Mastroberardi · 2019-08-15T17:06:01.194Z

ok, thanks for the explanation. It’s clear.

For the feedback, yes I mean the client could notify the server if it gets corrupted data (basically a map it cannot read).

In that case, the server could
(1) loop and resent the map until the client has a good map, or until it made N-failed-attempts,
(2) notify another component, say ErrorHandler, or a human to log the error so that developers can investigate why in that area / to that client the map breaks.

Maybe it’s not necessary in this phase, but when the product is fully developed and used, it’s a good tool for crisis management.

kenji-miyake · 2019-08-18T12:11:35.078Z

Hello, excuse me for cutting in. Please let me write my idea.

As for data corruption and feedback, what is the reason of data corruption?

In my understanding, generally, if the sender sent correct data, the receiver will: 1. receive correct data, or 2. not receive any data.
It won’t receive corrupted data because it is detected in the transport layer.

Or, if the map file itself is corrupted, it should be detected and fixed in the verification step, not in the runtime, and the sender should not send the data.

So my opinion is,

A simple resending feature is required because sometimes sending data might fail, but probably it can be done by ROS/ROS2 features.
No excessive feature is not necessary.

As for pub/sub or server, it depends on the target, Autoware.ai or Autoware.Auto.

For Autoware.ai, since the map node sends whole map data, I think using Service is more natural than pub/sub.
However, since the current implementation uses pub/sub, to keep using pub/sub might be better, considering the re-implementation cost.

For Autoware.Auto, it should be drastically redesigned for more safety, including Lanelet2 library.
I think sending structured data using pub/sub is best for now, but as we discussed at the last meeting, let’s consider this using the AVP use case.

kenji-miyake · 2019-08-18T12:13:26.609Z

By the way, I feel we need to have the same understanding of this WG’s goals.
Since map-related things have many aspects and the best solution depends on the target, we have to clarify what target we refer to.

Therefore, please let me summarize the goal(for .AI/.Auto), current status, and milestones of this WG.
I will write them maybe tomorrow in this thread: Autoware Map Data and Formats working group: Call for participation.