U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

EVENTS

ADS Safety Measurement and Operational Design Domain

Share

 

NIST is closely monitoring guidance from Federal, State, and local health authorities on the outbreak of COVID-19. To protect the health and safety of NIST employees and the American public they continue to serve, NIST has decided to make the ADS Safety Measurement and Operational Design Domain virtual only. For more information on COVID-19, please visit:  cdc.gov/covid19.
 

NIST will hold a workshop on Automated Driving System Safety Measurement and Operational Design Domain (ADS; SAE J3016) on July 7-8, 2020. The automotive industry is planning to deploy automated driving systems with most major manufacturers (OEMs) targeting the 2020s. Community consensus around reliable, broadly-acceptable performance metrics for assessing ADS safety can facilitate the successful achievement of these deployments. This workshop will explore, with the ADS community,  methods for analyzing the Operational Design Domain of an ADS-equipped vehicle into fundamental measurements.

 

ADS-ODD Technical Working Group

Would you like to be involved in the ADS-ODD Technical Working Group. The ADS-ODD Technical Working Group will meet virtually biweekly starting on April 22, 2020 and will discuss alternative taxonomies or frameworks for analyzing the Operation Design Domain of an ADS and write a technical brief in preparation for the July 7-8, 2020 workshop. If so, please email edward.griffor [at] nist.gov (edward[dot]griffor[at]nist[dot]gov).

 

ADS-ODD Pre-Read Materials

RAND

Measuring Automated Vehicle Safety

Forging a Framework by Laura Fraade-Blanar, Marjory S. Blumenthal, James M. Anderson, Nidhi Kalra, https://www.rand.org/pubs/research_reports/RR2662.html#:~:text=No%20standard%20definition%20of%20safety,can%20include%20animals%20and%20property 

 

Intel

Responsibility-Sensitive Safety (RSS), Intel Mobileye, https://www.mobileye.com/responsibility-sensitive-safety/ 

 

University of Warwick

The Curious Case of Operational Design Domain: What it is and is not? by Siddartha Khastgir, https://medium.com/@siddkhastgir/the-curious-case-of-operational-design-domain-what-it-is-and-is-not-e0180b92a3ae 

 

USDOT-NHTSA

AV 2.0 to AV 4.0 Activities, https://www.nhtsa.gov/vehicle-manufacturers/automated-driving-systems 

 

A Framework for Automated Driving System Testable Cases and Scenarios Overview, https://www.transportation.gov/av/research 

 

Voluntary Safety Self-Assessment (guidance, template and industry submissions), https://www.nhtsa.gov/automated-driving-systems/voluntary-safety-self-assessment 

 

SAE International

Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles J3016_201806, https://www.sae.org/standards/content/j3016_201806/ 

 

AVSC Best Practice for Describing an Operational Design Domain: Conceptual Framework and Lexicon, https://www.sae.org/standards/content/avsc00002202004/

 

Scenarios (Pegasus)

Pegasus Project, https://www.pegasusprojekt.de/en/home 

 

“Defining and Substantiating the Terms Scene, Situation, and Scenario for Automated Driving,” by S. Ulbrich, T. Menzel, A. Reschka, F. Schuldt, and M. Maurer, https://ieeexplore.ieee.org/document/7313256 

NIST ADS

This workshop will explore methods for evaluating automated driving system (ADS) safety using fundamental measurements derived from the provisions of the relevant operational design domain (ODD).

Agenda: Day 1 - 7 July 2020

  • Welcome and Introductions - 11-11:10 am (Dave Wollman - NIST)

Speaker Panel

  • Introduction to ADS Safety Concepts and their Relationships: ODD, OES and Scenarios (Ed Griffor - NIST) (11:10 - 11:25am)
  • What is an Operational Design Domain (ODD) and how is it used? (Barb Wendling – Fmr. Principal Engineer Automated Driving, MBRDNA) (11:25 - 11:40 am)
  • Scenarios and Operational Design Domain: A confusing but necessary relationship (Siddartha Khastgir – WMG, University of Warwick, UK) (11:40 - 11:55 am)
  • Are new measurement concepts needed to translate between ODD and scenarios? Operational Envelope Specification (OES) and Operational Design Conditions (ODC) (Chris Greer - NIST) (11:55 – 12:10)
  • Q&A (12:10 – 12:30 pm)

Lunch/Break (12:30 am - 1:00 pm)

Breakouts

  • Charge to Breakout Sessions (Ed Griffor - NIST) (1:00 pm - 1:10 pm)

Breakout Sessions (in parallel 1:10 - 2:30 pm)

Breakout #1:      Definitions of ODD, OES and Scenarios

Breakout #2:      Relationships between ODD, OES and Scenarios

Breakout Summaries (2:30 - 2:45 pm)

Day 1 Closing (2:45 pm)

Agenda: Day 2 - 8 July 2020

  • Welcome and Review of Day 1 Results - 11-11:10 am (Dave Wollman - NIST)

Speaker Panel

  • Summary of IEEE P2843 Work (Jack Weast – Intel and IEEE P2843 Chair) (11:10 - 11:25 am)
  • ODD: a prospective look at perspectives (Ed Straub – SAE International) (11:25 - 11:40 am)
  • “What’s data got to do with it” (Michelle Chaka, VTTI) (11:40 - 11:55 am)
  • Q&A (11:55 am – 12:15 pm)

Lunch/Break (12:15 pm – 1:00 pm)

Breakouts

  • Charge to Breakout Sessions (Ed Griffor - NIST) (1:00 pm - 1:10 pm)

Breakout Sessions (in parallel 1:10 - 2:30 pm)

Breakout #1 and #2

Both breakout groups will consider the question: What are the desirable properties and measurement inputs of an OES or ODC? 

    1. Examples of desirable properties might include measurability, composability, comprehensiveness, and reproducibility.
    2. Examples of measurement inputs might include testing regimes (simulation/track/public roads), outcomes-oriented measurement methods (e.g. safe completion of an unprotected left turn), OERDC quality (degree to which the physical world and ADS virtual representation align), etc.

Breakout Summaries (2:30 - 2:50 pm)

Next Steps (2:50– 3:00 pm)

Day 2 Closing (3:00 pm)

Created February 20, 2020, Updated July 6, 2020