Metrics for Machine Translation 2010 Evaluation

Metrics for Machine Translation 2010 Evaluation (MetricsMaTr10)

MetricsMaTr10 was implemented in accordance with the MetricsMaTr10 evaluation plan. It took place in the spring/summer of 2010 as part of the Joint Fifth Workshop on Statistical Machine Translation and MetricsMaTr10, a workshop of ACL 2010.

Please see the main MetricsMaTr page for a general description of the MetricsMaTr challenge series.

Documentation

• Evaluation plan, v1-3, updated April 19 2010
• NIST MetricsMaTr10 Official Release of Results, updated April 12 2011
• NIST WMT10/MetricsMaTr10 workshop slides, public v1-1, October 22 2010

Previous MetricsMaTr Resources

• MetricsMaTr08 development set automatic metric scores (baseline and MetricsMaTr08 submitted metrics)
• WMT08 (ACL08 Third Workshop on Statistical Machine Translation) test set: machine translations, reference translations, and automatic metric scores (baseline and MetricsMATR08 submitted metrics)

Schedule

Please see the Joint Fifth Workshop on Statistical Machine Translation and MetricsMaTr10 website regarding dates for voluntary submissions of metric descriptions as short papers to ACL 2010.

• January 11 2010: MetricsMaTr08 development data set re-release for MetricsMaTr10
• March 26 2010: Metric submission commitment due at NIST
• March 26 - May 14 2010: Metric submission period, metrics must be installed and operational at NIST by May 14, 5pm EDT
• June 18 2010: Informal metric descriptions due at NIST
• July 15 - 16 2010: Joint Fifth Workshop on Statistical Machine Translation and MetricsMaTr10 (at ACL 2010 in Uppsala, Sweden)

Contact

mt_poc [at] nist.gov (mt_poc[at]nist[dot]gov)

You can subscribe to the MetricsMaTr mailing list hosted by NIST, metricsmt_list [at] nist.gov (metricsmt_list[at]nist[dot]gov), by sending e-mail to listproc [at] nist.gov (listproc[at]nist[dot]gov). Put "subscribe metricsmt_list" in the body.