Skip to main content
U.S. flag

An official website of the United States government

Dot gov

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Https

Secure .gov websites use HTTPS
A lock ( ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

OpenASR Challenge

The goal of the OpenASR (Open Automatic Speech Recognition) Challenge is to assess the state of the art of ASR technologies for low-resource languages.

The OpenASR Challenge is an open challenge created out of the IARPA (Intelligence Advanced Research Projects Activity) MATERIAL (Machine Translation for English Retrieval of Information in Any Language) program that encompasses more tasks, including CLIR (cross-language information retrieval), domain classification, and summarization. For every year of MATERIAL, NIST supports a simplified, smaller scale evaluation open to all, focusing on a particular technology aspect of MATERIAL. CLIR technologies were the focus of the first open challenge in 2019, OpenCLIR. In 2020, the focus is on ASR. The capabilities tested in the open challenges are expected to ultimately support the MATERIAL task of effective triage and analysis of large volumes of text and audio content in a variety of less-studied languages.

OpenASR20 Challenge

The first OpenASR Challenge associated with MATERIAL, OpenASR20, was opened for registration in July 2020, with an evaluation period in November 2020. It featured ASR evaluation opportunities for these ten low-resource languages:

  • Amharic
  • Cantonese
  • Guarani
  • Javanese
  • Kurmanji Kurdish
  • Mongolian
  • Pashto
  • Somali
  • Tamil
  • Vietnamese

It was implemented as a track of NIST’s OpenSAT (Open Speech Analytic Technologies) evaluation series, using the OpenSAT web server for registration, data access, submission, and scoring purposes.

The evaluation plan posted in the Documentation section below describes the OpenASR20 in detail.

Registration

Registration for the OpenASR20 Challenge is now closed.

Documentation and Resources

Results

The OpenASR20 evaluation was conducted in November 2020. A brief overview of the results will be available soon. OpenASR20 participants, as well as others working in the low-resource ASR problem space, are strongly encouraged to submit their work to an OpenASR special session at INTERSPEECH 2021. Please see the OpenASR20 and Low-Resource ASR Special Session at INTERSPEECH 2021 Call for Papers. This special session also welcomes contributions from others working in the low-resource ASR problem space who did not participate in OpenASR20.

Contact

Please email openasr_poc@nist.gov for any questions or comments regarding the OpenASR Challenge. 

Created June 10, 2020, Updated February 24, 2021