[URDF-NG] Parser tools wishlist

I have a few ideas for the “next generation” of robot description format parsing tools, inspired by the recent community meeting, as well as discussions on Github and in person. This is a big, somewhat freeform brainstorm post; please reply your clarification questions, conceptual corrections, general concerns, or ideas for enhancements.

I’ll try to consistently use the following terms:

  • URDF refers to the URDF specification
  • urdfdom refers to the parser library (yes, there is also a Python urdf parser, urdf_parser_py; just group them together for now as doing the same thing for different libraries)
  • SDF refers to the SDF specification
  • sdformat refers to the parser library

I’m also trying to keep the language agnostic to the different the paths of: making major changes to URDF, switching to using SDF across the ROS ecosystem and making some changes to SDF, or writing a new thing entirely. We haven’t made a decision regarding these paths yet, and this thread is not about which specification to use, but instead architectural changes to be made to the parser library we use in ROS. One of the points I’m trying to make is that I think a parser library is orthogonal to its specification and can be its own modular component. The specification is data and the parser library is code.

Here are my major points:

  • A schema should be a required input to a parser

The URDF “specification” refers to an outdated set of ROS Wiki pages. urdfdom is both ahead and behind the documented specification.

The advantages of sdformat’s approach to using a schema include: the ability to autogenerate documentation on sdformat.org and a relatively minimal parser library that uses the schema for
validation, rather than hardcoding the logic of the specification in the parser.

One disadvantage that William brought up when we discussed this is that URDF might not actually be describable by an XML-based way of specifying the schema, or at least, not elegantly describable.

An example clause from URDF which would be difficult to describe through an XML schema would be:
‘The “limit” element of the “joint” element can only have “lower” and “upper” attributes IF the “type” of the joint is not “continuous”.’
(paraphrased from http://wiki.ros.org/urdf/XML/joint)

Conditions would be needed to express such clauses, which leads to a potentially messy and

  • Extensibility

There should be support for downstream packages to provide an extension of the schema by writing a new sub-element of the schema, or an element that includes existing elements.This is a feature supported in neither urdfdom nor sdformat.

I appreciate that SDF has a “plugin” element for any freeform XML that the user wants to be parsed. But a lot of custom boilerplate is needed to parse the elements out of a “plugin” element and check them for errors. There should be an easy way for external packages to add new elements to the schema which the parser library can validate, minimizing boilerplate and eliminating upstream changes.

For example, consider this set of packages:
Package “urdf_schema” provides the schema for URDF. It has no dependencies on other packages in this set.
Package “sensor_schema”, which depends on “urdf_schema”, provides a schema for sensors, an optional sub-element of a robot in URDF. (I don’t know what the schema language will look like yet. If it looked like SDF, this package might have a file that looks like this).
Package “depth_camera_schema” depends on “sensor_schema” and describes a sensor type, an RGBD camera, which is represented as a child of “sensor”.
Package “simulation_schema” describes simulation worlds, with a URDF robot element as an optional sub-element (like SDF). It depends on “urdfdom”.
Once a piece of software includes a package with a schema extension, it can use the parser library to parse XML that follows the schema (for example, to simulate a robot with a depth camera).

The most important challenge here is the requirement that schema extensions can insert themselves
into existing elements (e.g. adding a sensor as a sub-element of a robot) without the other packages knowing about the extension.
This may sound impossible/not a good idea, but I believe it is important to avoid users forking urdf when they want to add new possible elements of a robot that are not represented in the mainline spec.

A possible concern about allowing more extensibility is fracturing the standard. This is a valid concern but I think that higher flexibility is worth it to allow new elements to be parsed without forking. It offers a path for new elements to be quickly implemented, distributed, and tested before they are merged into the mainstream “urdf_schema” package. The federated model works pretty well for ROS, e.g. for message descriptions. Robot description formats could be federated in a similar way by having a common URDF schema package and schema packages further down the package tree for extending the spec.

  • Code generation for syntactic sugar

This point is less important than the other two, but I think it would still be a helpful feature of the parser engine. Inspired by code generation for ROS messages: auto-generate a class with accessors and mutators to the element’s attributes/children, and conversion functions to/from XML for the elements described by the schema.

SDF doesn’t really do this right now; it provides getElement("key name") methods for navigating an SDF graph. Gazebo classes then retrieve information from SDF on initialization. For example, Gazebo::Joint has an SDF::Element from which it reads the child elements and attributes of the XML .

With this new feature, the boilerplate to initialize a class from an XML element would be auto-generated, as well as the data representation itself.

1 Like

Dear Jackie,

I very much like the idea, to have the URDF specification described by a schema and the parser be auto-generated from the schema. I’ve used xsdcxx before to this end, which provides code-generation for C++ data-types derived from XSD types as well as appropriate parsing.
xsd/e (http://www.codesynthesis.com/projects/xsde/) seems to be a follow up to this with smaller footprint.
However, I share William’s concerns about the limited description capabilities of e.g. XSD. In particular, it was difficult / impossible to leave the order of elements unspecified while simultaneously specifying various required counts for them.
Otherwise, It exactly resembles what you suggest.

Conceptually, I think we need to distinguish between different applications: While, e.g. Gazebo, can easily manage modularity / extensibility by plugins, because appropriate new elements get directly parsed by a plugin lib into classes derived from some abstract interface. I.e. Gazebo itself doesn’t need to know about the detailed content of the new element.
On the other hand, typcial usage for URDF, is that an application needs to parse the XML description into a data structure for further use (in contrast to an instance of a base class). Hence, for an application it is not sufficient to load a plugin for parsing the element, but the application also needs to know about the data structure, the parser is parsing into.

My proposals to urdfdom (#84) and urdf (#125) exactly allow for this: While the base liburdfdom sensor provides some convenience methods to parse any sensors, applications can request additional elements to be parsed by dynamically loaded libs (managed by pluginlib). By also publishing the header files of the corresponding data types, the application can easily access the parsed data structure.
An example for this approach can be found there: https://github.com/ubi-agni/urdf_tactile.

Sorry, I was not able to include more than two links.

From the point of view of implementing tools, having a schema available would be a huge boost. It would lead to greater consistency and make it easier to maintain tools that need to parse URDF files.

XSD looks pretty cool! Thanks for the link, Robert.

Thinking about XML description vs data structure: It seems as if the in-memory data structure should mirror the format of the XML element.

In the system I proposed in the OP (which I haven’t implemented yet), the data structure gets generated by the parser and then that class would have to be injected into the user’s code (the dependent package), by linking against the library compiled from the generated code. Thus the parser is imposing a data structure on the user code.

In your proposal and implementation, the user provides the bridge between the parser and the data structure via pluginlib.

I would summarize the tradeoff between these two approaches as: more standardization and code reuse vs. more flexibility and individual/fine-grained control.

I’m not sure which approach is better, and it’s possible to support both (though it would take more engineering effort).

Tully raised the hyperlink limit for “new” users to 5, sorry about the inconvenience.