ROS Resources: Documentation | Support | Discussion Forum | Service Status | Q&A answers.ros.org

Definition of data recording feature for Autoware.Auto

Hi,

This post is to discuss the definition of data recording feature for Autoware.Auto, which comes from the epic on the Gitlab.

First of all, I want to figure out the purpose(what, when, what for) of the data recording feature.
After that, we’ll discuss “how to” and “where to” record the data.

I have shortly studied the definition of the existing EDRs(Event Data Recorders), and EDR-AD(Event Data Recorder for Autonomous Driving) which has been proposed recently.
For example, the definition of EDRs by the US government is as below.

What

  • Some data related to vehicle dynamics(Delta–V, Speed, Engine throttle, Service brake, Steering input, ABS activity, etc) and driver’s safety status(Safety belt status, Frontal airbag deployment, etc)

When

  • Roughly -5sec to +300ms relative to the crash impact

What for

  • Post-crash investigations and for analysis of safety equipment performance

The definition of EDR-AD by CLEPA is as below.
What

  • Generic events(Event type, Timestamp, Location) in AD system for driving log or testing
  • The same data as EDRs for crash event
  • Additional data related to rule violations(driving manoeuvers, Camera images, GPS time/ position, AD active state) for malfeasance event

When

  • While AD system activated for driving log or testing
  • Roughly -30sec to +10sec relative to the crash event
  • When AD system detect any rule violations for malfeasance event

What for

  • Disculpation/Exoneration of the driver in case of infringements (including non-accident events)
  • Accident situation: if AD system was activated
  • Product liability, product monitoring obligation, quality assurance and product development for OEMs and suppliers.
  • Supply of factual data for legal proof.

I think the concept of EDR-AD is a good example for us to discuss the data recording feature in Autoware.Auto. So I am thinking to break down the required data elements and map them to data in Autoware.Auto. However, I just hit a basic question.

  • Should we assume that a generic EDR is already equipped on the vehicle?
    If yes, synchronization between the data recorded in the EDR the one in Autoware.Auto would be the key point.
    If no, this feature should contain all features of EDR-AD(including generic EDR).

Does anyone have an answer to it? Any other suggestions or feedback would be appreciated.

The following links would be helpful for reference.

4 Likes

This is a critical feature for any AV that will be certified for road use. The UK perspective on this question is put forward in the 2019 Code of Practice for Automated Vehicle Trialling.

I’ve had some interactions with the Law Commission trying to work out the relationship between privacy and the requirement to store data, and also about the length of time that data needs to be stored. These are difficult questions.

Thanks. I’ll check the document you shared.

As far as I studied in several documents including the one which @Brian_Holt shared for me, we can say:

  • Autonomous driving vehicles should obviously implement EDR-like features the same as manned vehicles in case of a crash event.
  • Autonomous driving vehicles should record additional data related to AD systems so that any third party can determine the responsibility of the driver after infringements or incidents (including non-accident events).
  • The recorded data related to AD features can contain visual and audio data that should be handled with relevant data protection or privacy laws in each region.

However, I also find some difficulty to define the requirement for this data recording feature because:

  • Requirements for data recording features for AD systems are not as clear as ones for EDRs.
  • It is not clear for me if we should expect an existing EDR equipped on the vehicle or not.
  • Appropriate handling visual and audio data will change depends on the laws in each region

My current idea to overcome the above is:

  • Define a typical use case and fundamental functions to record and store data in Autoware specification.
  • The fundamental functions will contain:
    ○ receiving vehicle data in the legacy in-vehicle bus like CAN via vehicle interface
    ○ external interfaces to start and stop recording
    ○ time synchronization between vehicle data and AD system data (if needed)
    ○ configuring a set of data elements to record
    ○ configuring the persistency and durability for each data element
    ○ encryption of the recorded data
  • Categorize data elements to record(e.g. we can roughly categorize them into 3 types of data as “vehicle data”, “AD system data” and “privacy data”)
  • Define a set of default persistency and durability for each data category
  • Implement a minimum set of recording functions on Autoware.Auto as a reference implementation.

Will it work for our users? Any other opinions or feedback would be appreciated.

My understanding of the intent behind EDRs is to exactly this. Ideally the information stored should be sufficient to reconstruct the scenario in simulation and I would recommend that goal as a guide for this work.

While basic telemetry (pose, velocity, acceleration, status of indicators, horn, lights, etc) do not consume much space, sensor data such as LiDAR and camera is very space-heavy. Obviously the system will not know when the incident is going to happen, and the requirement to collect -30s to +10s means that the system will need to be constantly recording. This is probably best implemented using a circular buffer, and storing the contents of the buffer when an incident occurs.

Thanks, @Brian_Holt. I agree with your opinion.

And at the last WG meeting, @gbiggs gave me a suggestion that Autoware should assume an EDR is not present and record everything.

After that, I refined the definition of this feature. I would be happy to hear any feedback.

What

  • Any CAN data available via the vehicle interface
    (the set of data elements to record should be configurable)
  • Any ROS2 topic and parameters available via the standard ROS2 APIs
    (the set of data elements to record should be configurable)
  • Typical system logs generated by the operating system outside of Autoware
    (e.g. date, ps, env, syslog, dmesg, netstat in Linux)
  • Identification data of the running system
    (e.g. software release version, system mode, system status)

When

  • -30sec to +10sec relative to the trigged event.
    (Autoware should record all of the data constantly in a buffer while activated and store the content in persistent storage when triggered.)

What for

  • Provide the appropriate evidence so that any third party can determine the responsibility of the AD system after infringements or incidents (including non-accident events). Ideally, it can reconstruct the scenario in the simulation with the stored data sufficiently.

Other considerations
Autoware should provide an appropriate security function considering the following requirements.

  • The stored data should not be changed by anyone after the incident.
  • The stored audio and visual data should be treated under relevant privacy laws.
1 Like

@Adrian_Bedford do you have any more thoughts on the topic?

You’ve raised many points here so I comment in several posts:
I reply to “-30sec to +10sec relative to the trigged event”

Sections 5.12 to 5.16, pages 19 and 20 of this UK document:

Normal recording frequency minimum is 10Hz, but “In the event of an incident, an event data recorder should be able to capture a suggested minimum period of 30 seconds before the incident, and 15 seconds after. It is recommended that the minimum recording frequency is 50Hz”

“The stored audio and visual data should be treated under relevant privacy laws.”


This article implies extra precautions if the data is to be used for prosecutions.

Some vehicle may not have the hardware installed to enable the recording of all the data suggested. For example, an ordinary StreetDrone does not provide for the autonomous switching of lights or indicators, nor the recording of their state. It is now provided as an option or an upgrade.