Update on NIST’s Automated Driving Systems Safety Measurement Technical Working Group

Measurement science and standards are needed to support the safe and predictable operation of future automated vehicles (AVs). These vehicles have great potential to significantly impact our daily lives and improve the competitiveness of our economy.

NIST continues to build on a successful first-year Strategic and Emerging Research Initiatives (SERI) project, NIST and Autonomous Vehicles. (See 2022 NIST AV workshop and report.) NIST is supporting this expanded second-year SERI-funded, NIST Automated Vehicle Program to address system performance and measurement methods for AV sensors/perception, artificial intelligence (AI), cybersecurity, and AV communications. NIST is designing and establishing a system interaction testbed, which will involve both virtual and physical environments. This project brings together expertise from across NIST Laboratories and provides the basis for expanded stakeholder interactions, including a planned NIST AV workshop in September 2023.

Since 2019, NIST’s research on automated vehicles has been informed by its Automated Driving Systems Safety Measurement Technical Working Group, which meets regularly. Participants include representatives from over half of the automotive industry’s original equipment manufacturers; US Department of Transportation; several state transportation departments; and research universities such as Carnegie Mellon, University of California, University of Michigan, and Virginia Tech.

A Brief Review: the Technical Working Group initially confirmed the need for a consensus-based measurement of automated vehicles safety, as indicated in NIST Special Publication 1900-320, Workshop Report: Consensus Safety Measurement Methodologies for Automated Driving System-Equipped Vehicles. The Technical Working Group subsequently aided development of NIST’s Special Publication 1900-301, Automated Driving System Safety Measurement Part I: Operating Envelope Specification, which presented a novel approach to safety measurement.

Today’s Efforts: The Technical Working Group seeks to determine what constitutes safe driving. This includes:

Determining ways of assessing whether automated vehicles meet the specifications of key driving behaviors or maneuver(s): Such behaviors include navigating an intersection, passing another vehicle, etc. These behaviors require specifications for their implementations and assessment. For example, staying in the center of a lane entails maintaining a speed within legal limits, and not deviating more than six inches from center line. Evaluating driving behaviors against their specifications will be used to determine and measure safe driving.

Assessing Human Trust in AI: Researchers in psychology and human factors assist this effort. Human trust is complex, evolving, and individual. Without passengers’ and vulnerable road user trust in automated vehicles, acceptance and adoption will be more difficult. A large part of the issue is replacing the human decision-maker with AI. This activity is developing a computer-digestible language for expressing human expectations of autonomous systems performance, such as those used to automate the driving task.

Those interested in joining the NIST Automated Driving Systems Safety Measurement Technical Working Group should contact Dr. Edward Griffor by email at edward.griffor [at] nist.gov (edward[dot]griffor[at]nist[dot]gov).