NIST logo
*

NIST Open Machine Translation 2012 Evaluation (OpenMT12)

The NIST Open Machine Translation 2012 Evaluation (OpenMT12) was implemented according to the OpenMT12 evaluation plan. It took place in the spring of 2012, followed by a summer of 2012 workshop.

Highlights

  • Evaluation on prior years' Arabic-to-English and Chinese-to-English Progress test data, with the data to be made available to participants after OpenMT12
  • Evaluation on a parallel data set based on prior years' Progress tests for Arabic, Chinese, Dari, Farsi, and Korean to English, in two source data styles
  • Evaluation on new domain data for Chinese-to-English.
  • Support for both a single system and a system combination track (provided sufficient interest by the participants)
  • Evaluation by automatic metrics and coordination of volunteer human assessments using a new tool to rank alternative translations

Documentation

Schedule

  • (See Table 1 of eval plan): Training data off-limits periods
  • October 28 2011: Evaluation plan available
  • October 28 2011 – February 3 2012: Registration period (early registration highly encouraged)
  • November 4 2011: Training data available from LDC
  • January 16 – February 24 2012: Dry run period (early submission highly encouraged); output due at NIST February 24 11.59am EST
  • April 2 – 6 2012: Main evaluation period; output due at NIST April 6 11.59am EDT
  • April 16 – 20 2012: System combination evaluation period; output due at NIST April 20 11.59am EDT
  • April 20 2012: Preliminary release of main evaluation results to participants
  • April 27 2012: System descriptions due
  • April 30 – June 1 2012: Human assessment period
  • June 27 – 28 2012: Workshop in the Washington DC area
  • August 31 2012: Official public release of results

Software and Resources

Contact

mt_poc@nist.gov