#SDIA23 Update: Research award winner Dr Nick Reed developing Digital Commentary Driving concept to assess self-driving safety
Welcome to #SDIA23 Updates, a new series exploring what our reigning Self-Driving Industry Award champions have been working on recently. First up: winner of the 2023 award for Research, Reed Mobility.
In this category, the judges were looking for examples of exceptional academic studies and/or market analysis. Funded by the Rees Jeffreys Road Fund, Reed Mobility led a project exploring public attitudes and expectations towards the ethical behaviours of self-driving vehicles.
Since the awards in November, Reed Mobility founder, Dr Nick Reed, has continued his work as Chief Road Safety Adviser to National Highways – including plans to reduce deaths and serious injuries on England’s strategic road network (SRN), and supporting activity in relation to connected and automated mobility (CAM), smart motorways and cybersecurity.
He recently became a founding member of the Department for Transport’s College of Experts, was appointed a trustee to the Road Safety Trust, and joined the Advisory Board of Partners for Automated Vehicle Education (PAVE) UK.
As if all that weren’t enough, Dr Reed also found time to update BSI’s CAM Vocabulary…
… and continued his work with colleagues there to develop a technique for assessing automated vehicle (AV) safety performance – Digital Commentary Driving (DCD) – as he explains here:
Self-driving data
NR: The 20th century economist, William Deming, is quoted as saying “In God we trust, all others bring data”. This captures his sense that when it comes to important decisions, gut feel and belief are not enough; objective evidence in the form of data is necessary to support decision-making.
My work that won the research category of the Self-Driving Industry Awards 2023 identified that trust was the most important value to the public in their appreciation of automated vehicles and that this trust is encapsulated by four key attributes. AVs should:
- Be governed by a clear, legal framework;
- Be at least as safe as a good human driver;
- Protect other road users at least as well as they protect their occupants;
- Share data with stakeholders to improve safety.
These principles are enshrined in the recently passed Automated Vehicles Act 2024, but what will be the data that enables us to trust that they will be safe? There are many ways that this question can be approached. The Act provides some signposts, starting with two key principles:
- authorised automated vehicles will achieve a level of safety equivalent to, or higher than, that of careful and competent human drivers, and
- road safety in Great Britain will be better as a result of the use of authorised automated vehicles on roads than it would otherwise be.
These make intuitive sense but what objective data would satisfy Deming and show that AVs are adhering to these principles? Principle (a) is challenging because there is no agreed definition of careful and competent driving (although DVSA’s National Standards for Driving are a good start); it is not clear how we could determine that an automated vehicle is behaving carefully and competently and therefore what data we should be collecting in order to prove it.
For principle (b), the answer appears more straightforward. We could look at collision rates of AVs (i.e. crashes per distance travelled) and compare that to collision rates for human drivers in similar vehicles on similar journeys and, if AVs achieve a lower crash rate, we can say that road safety is better. However, there are nuances here too. Let’s say AVs were found to be 10% safer than human driven vehicles – would we consider road safety to have improved if utilisation went up by 20%?
Although each individual AV trip would be relatively safer than that completed by a human driver, the overall level of exposure would mean an increase in the absolute number of crashes. Furthermore, an AV service might attract customers who previously completed a similar journey by train. Rail travel is estimated to be 20× safer than human driving so shifting to AVs would increase the global risk of injury even if the AVs were significantly safer than human drivers on the same trip.
My work with colleagues from BSI to develop a potential technique for assessing AV safety performance may offer a solution. We looked at the ways that we assess the safety of advanced human drivers and the metrics used to assess safe performance of mobile robots. Bridging these worlds, we proposed the concept of Digital Commentary Driving (DCD). This is a standardised protocol for the collection of data from AVs requiring the collection of the data that an AV must be using in order to drive safely.
This includes the current status of the vehicle (e.g. speed, steering angle, brake application, accelerator application, current heading, software version etc.), perception of the surrounding environment (e.g. fixed objects, moving objects) and predictions of their future movement (e.g. desired future heading, desired speed etc.).
Since DCD data covers the essential features necessary for careful and competent driving, it cannot compromise commercially sensitive information about the way a vehicle is being controlled. There is no presumption over how this data is arrived at by the AV systems. DCD does not prescribe the hardware (e.g. an AV might use one or more of cameras, lidar, radar, ultrasound, V2X communication etc.) or the software (e.g. end-to-end deep learning or rules-based approaches etc.) involved – it only requires the sharing of standardised data regarding the perceptions, decisions and actions of the AV.
Furthermore, AV companies would only be required to share data on the performance of their vehicles with an authorised regulator who would hold it securely for analysis pursuant to safety performance.
Of course, the collection of DCD data in itself does not tell us what it means for an AV to be a careful and competent driver. However, it does start to provide a consistent dataset that will enable objective analysis of driving performance by AVs from all developers and benchmarks to be established that set expectations around what it means to drive safely.
This may start the process of building the trust so valued by the public – and would perhaps satisfy Deming’s expectation for objective data to support critical decision making.
