This post is to discuss the definition of data recording feature for Autoware.Auto, which comes from the epic on the Gitlab.
First of all, I want to figure out the purpose(what, when, what for) of the data recording feature.
After that, we’ll discuss “how to” and “where to” record the data.
I have shortly studied the definition of the existing EDRs(Event Data Recorders), and EDR-AD(Event Data Recorder for Autonomous Driving) which has been proposed recently.
For example, the definition of EDRs by the US government is as below.
What
Some data related to vehicle dynamics(Delta–V, Speed, Engine throttle, Service brake, Steering input, ABS activity, etc) and driver’s safety status(Safety belt status, Frontal airbag deployment, etc)
When
Roughly -5sec to +300ms relative to the crash impact
What for
Post-crash investigations and for analysis of safety equipment performance
The definition of EDR-AD by CLEPA is as below. What
Generic events(Event type, Timestamp, Location) in AD system for driving log or testing
The same data as EDRs for crash event
Additional data related to rule violations(driving manoeuvers, Camera images, GPS time/ position, AD active state) for malfeasance event
When
While AD system activated for driving log or testing
Roughly -30sec to +10sec relative to the crash event
When AD system detect any rule violations for malfeasance event
What for
Disculpation/Exoneration of the driver in case of infringements (including non-accident events)
Accident situation: if AD system was activated
Product liability, product monitoring obligation, quality assurance and product development for OEMs and suppliers.
Supply of factual data for legal proof.
I think the concept of EDR-AD is a good example for us to discuss the data recording feature in Autoware.Auto. So I am thinking to break down the required data elements and map them to data in Autoware.Auto. However, I just hit a basic question.
Should we assume that a generic EDR is already equipped on the vehicle?
If yes, synchronization between the data recorded in the EDR the one in Autoware.Auto would be the key point.
If no, this feature should contain all features of EDR-AD(including generic EDR).
Does anyone have an answer to it? Any other suggestions or feedback would be appreciated.
The following links would be helpful for reference.
I’ve had some interactions with the Law Commission trying to work out the relationship between privacy and the requirement to store data, and also about the length of time that data needs to be stored. These are difficult questions.
As far as I studied in several documents including the one which @Brian_Holt shared for me, we can say:
Autonomous driving vehicles should obviously implement EDR-like features the same as manned vehicles in case of a crash event.
Autonomous driving vehicles should record additional data related to AD systems so that any third party can determine the responsibility of the driver after infringements or incidents (including non-accident events).
The recorded data related to AD features can contain visual and audio data that should be handled with relevant data protection or privacy laws in each region.
However, I also find some difficulty to define the requirement for this data recording feature because:
Requirements for data recording features for AD systems are not as clear as ones for EDRs.
It is not clear for me if we should expect an existing EDR equipped on the vehicle or not.
Appropriate handling visual and audio data will change depends on the laws in each region
My current idea to overcome the above is:
Define a typical use case and fundamental functions to record and store data in Autoware specification.
The fundamental functions will contain:
○ receiving vehicle data in the legacy in-vehicle bus like CAN via vehicle interface
○ external interfaces to start and stop recording
○ time synchronization between vehicle data and AD system data (if needed)
○ configuring a set of data elements to record
○ configuring the persistency and durability for each data element
○ encryption of the recorded data
Categorize data elements to record(e.g. we can roughly categorize them into 3 types of data as “vehicle data”, “AD system data” and “privacy data”)
Define a set of default persistency and durability for each data category
Implement a minimum set of recording functions on Autoware.Auto as a reference implementation.
Will it work for our users? Any other opinions or feedback would be appreciated.
My understanding of the intent behind EDRs is to exactly this. Ideally the information stored should be sufficient to reconstruct the scenario in simulation and I would recommend that goal as a guide for this work.
While basic telemetry (pose, velocity, acceleration, status of indicators, horn, lights, etc) do not consume much space, sensor data such as LiDAR and camera is very space-heavy. Obviously the system will not know when the incident is going to happen, and the requirement to collect -30s to +10s means that the system will need to be constantly recording. This is probably best implemented using a circular buffer, and storing the contents of the buffer when an incident occurs.
And at the last WG meeting, @gbiggs gave me a suggestion that Autoware should assume an EDR is not present and record everything.
After that, I refined the definition of this feature. I would be happy to hear any feedback.
What
Any CAN data available via the vehicle interface
(the set of data elements to record should be configurable)
Any ROS2 topic and parameters available via the standard ROS2 APIs
(the set of data elements to record should be configurable)
Typical system logs generated by the operating system outside of Autoware
(e.g. date, ps, env, syslog, dmesg, netstat in Linux)
Identification data of the running system
(e.g. software release version, system mode, system status)
When
-30sec to +10sec relative to the trigged event.
(Autoware should record all of the data constantly in a buffer while activated and store the content in persistent storage when triggered.)
What for
Provide the appropriate evidence so that any third party can determine the responsibility of the AD system after infringements or incidents (including non-accident events). Ideally, it can reconstruct the scenario in the simulation with the stored data sufficiently.
Other considerations
Autoware should provide an appropriate security function considering the following requirements.
The stored data should not be changed by anyone after the incident.
The stored audio and visual data should be treated under relevant privacy laws.
You’ve raised many points here so I comment in several posts:
I reply to “-30sec to +10sec relative to the trigged event”
Sections 5.12 to 5.16, pages 19 and 20 of this UK document:
Normal recording frequency minimum is 10Hz, but “In the event of an incident, an event data recorder should be able to capture a suggested minimum period of 30 seconds before the incident, and 15 seconds after. It is recommended that the minimum recording frequency is 50Hz”
“The stored audio and visual data should be treated under relevant privacy laws.”
This article implies extra precautions if the data is to be used for prosecutions.
Some vehicle may not have the hardware installed to enable the recording of all the data suggested. For example, an ordinary StreetDrone does not provide for the autonomous switching of lights or indicators, nor the recording of their state. It is now provided as an option or an upgrade.
I think this requirement means we should set the default recording rate at 50Hz in a buffer because we expect that this feature should be used in the post-incident analysis.
This article is interesting. It also says that the definition of “sensitive data” that “users" can delete is still ambiguous and should be clarified by the authorities in the future.
So I think we should let the distributors of Autoware.Auto to configure the specific set of data elements to record, based on the regulations they have to follow in their region.
Yes, I agree. So the distributors of Autoware.Auto should be able to configure the specific set of data elements to record, based on their hardware specifications, too.
This point hasn’t been mentioned yet but I think it should be to be clear.
The data recording feature will be a part of Autoware, but that doesn’t mean it should be based on ROS 2 nodes for its architecture. This feature is going to be quite important for safety and so it should be as simple as possible. I think that therefore we should have a data recorder architecture that has minimal dependencies, and that we can add a sub-component to responsible for managing the interaction with ROS 2 to get the ROS 2 data. This sub-component could well not even use ROS 2, instead directly using a DDS client library to get the data.
@gbiggs
Thanks, I agree with your opinion. I’ll keep it in my mind to design this feature as simple as we can so that this feature works correctly even if some critical errors happen in ROS 2 layers. The same design rule should be also applied in recording vehicle data and system logs.
I was told that the Working Party on Automated/Autonomous and Connected Vehicles (GRVA) has been established under the WP29 this year. I will watch the activities in this working party because I expect the upcoming regulations in the participating countries will follow the outcome from it.
The working party proposes the two type of equipment
EDR(Event Data Recorder)
DSSAD(Data Storage System for Automated Driving)
EDR is the existing one, which records pre-crash and post-crash data when triggered by airbag deployment. It is only used for post-crash analysis.
DSSAD is the new one, which is supposed to record the specific AD data set(e.g. significative interaction between the AD system and the human driver) continuously for several months. It is used to determine who took control of the car when the incident happened.
As far as reading their documents, it seems like to me that people in the working party gave up to record space-heavy data in DSSAD due to storage capacity and privacy protection, and decided to focus on the very limited data.
So, I want to ask you again whether we should expect to record space-heavy data such as LiDAR and camera in Autoware.auto. Recording such data should be obviously important for debugging but it may not be required by regal regulations in the near future. @gbiggs, @Brian_Holt, @Adrian_Bedford
Any thoughts?
Does this mean “all data for several months” or “continuously record, and remain operational for several months”?
Why is several months of data necessary to determine the immediate time around an accident? A vehicle is likely to start and stop dozens of times in several months. This seems like an odd requirement to me.
I think that even if the data is not required by any legal regulations, we still are obligated to record all sensor, control and state data for the immediate time around an incident. Otherwise it would become impossible to investigate the cause of the incident and determine if Autoware was the cause, and if so what in Autoware needs to be fixed. This is a basic requirement of any safety-critical system.
It means the former “all data for several months”. However, “all data” will be a small set of AD specific events.
They expect that some kinds of accidents cannot be detected by the AD system itself but should be claimed by others afterward. For example, they assume accidents such as traffic offenses, collisions with very small impacts and unsafe maneuvers without collisions.
So the document says DSSAD should store data with timestamps for a long period(in the last X-months) and be able to deliver them when requested by an authorized entity. I have to check how reliable the recorded data without exact time synchronization between other computing nodes would be.
OK. It makes sense. We can also add any data elements that Autoware processed for the immediate time around an incident as an EDR-like feature.
OK, that makes more sense now. That implies that our data logging feature needs to have levels of retention that can be set on a per-data-stream basis.