New REP proposal: Human-Robot Interaction in ROS (ROS4HRI)

Dear ROS community,

Following REP-1 recommendations for new REP submissions, I would like to gauge interest in a new REP to create a set of conventions around HRI (human-robot interaction) application scenarios.

Abstract

This new ‘ROS4HRI’ REP proposal aims at providing a set of conventions and common interfaces for Human-Robot Interaction (HRI) scenarios. This interface is designed to promote interoperability and reusability of core functionality between the many HRI-related software tools, from skeleton tracking, to face recognition, to natural language processing.

By following the naming conventions and leveraging the interfaces defined in this REP, both tools and libraries can be designed to be reusable between different frameworks and experiments. Importantly, the REP does not mandate specific tools or algorithms to perform human perception/social signal recognition per se. It only specify naming conventions and interfaces between these nodes.

These interfaces are designed to be relevant for a broad range of HRI situations, from crowd simulation, to kineastetic teaching, to social interaction.

Rationale

ROS is widely used in the context of human-robot interactions (HRI). However, to date, not a single effort has been successful at coming up with broadly accepted interfaces and pipelines for that domain, as found in other parts of the ROS ecosystem (for manipulation or 2D navigation for instance). As a result, many different implementations of common tasks (skeleton tracking, face recognition, speech processing, etc) cohabit, and while they achieve similar goals, they are not generally compatible, hampering the code reusability, experiment replicability, and general sharing of knowledge.

In order to address this issue, this REP aims at structuring the whole “ROS for HRI” space by creating an adequate set of ROS messages and services to describe the software interactions relevant to the HRI domain, as well as a set of convention (eg topics structure, tf frames) to expose human-related information.

The REP aims at modeling these interfaces based on existing, state-of-the-art algorithms relevant to HRI, while considering the broad range of application scenario in HRI.

It is hoped that such an effort will allow easier collaboration between projects and allow a reduction in duplicate efforts to implement the same functionality.

Items covered by the proposed REP

  • human modeling, as a combination of a permanent identity (person) and transient parts that are intermittently detected (eg face, skeleton, voice);
  • topic naming conventions under the /humans/ topic namespace;
  • 3D tf frame conventions (naming, orientation – compatible with REP120 where possible)
  • representation of group interactions (groups, mutual gaze)

A detailed proposal was presented at IROS2021: ROS for Human-Robot Interaction | IEEE Conference Publication | IEEE Xplore

Reference implementation

The reference implementation will include:

  • a set of HRI-related ROS messages: hri_msgs;
  • libhri, a library that eases the access to human-related signals (providing a HRIListener inspired by tfListener);
  • a reference open-source pipeline that will include:
    • face detection and gaze estimation
    • multi-body 3D pose estimation
    • voice activity detection and speaker diarization
    • sound source localisation
    • ASR
  • rviz plugins to visualise human-related information like 3D skeletons, face & body regions of interest
23 Likes

This would definitely be a valuable enhancement!

1 Like

Sounds awesome! Please sign me up!!

@dirk-thomas @tfoote @clalancette (or someone else :wink: ): what would be the next steps? I have a draft of the REP ready to share. Should I make a PR on ros-infrastructure? Do I need a (provisional?) REP number? Thanks!

1 Like

Yes, please open a PR to GitHub - ros-infrastructure/rep: ROS Enhancement Proposals . As for which number to use, since this looks like a general proposal of standards, I’ll suggest using REP number 155. Please make sure to put the term REP-155 somewhere in the title of the PR so it is easy to see that it is now “provisionally” assigned.

Happy to share that the draft of the REP has been sent as a PR to the ros-infrastructure/rep repository:

Feedback from the community very welcome!

7 Likes

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.