NIST logo

Publication Citation: Findings of the 2010 Joint Workshop on Statistical Machine Translation and Metrics for Machine Translation

NIST Authors in Bold

Author(s): Chris Callison-Burch; Philipp Koehn; Christof Monz; Kay Peterson; Mark A. Przybocki; Omar F. Zaidan;
Title: Findings of the 2010 Joint Workshop on Statistical Machine Translation and Metrics for Machine Translation
Published: July 20, 2010
Abstract: This paper presents the results of the WMT10 and MetricsMATR10 shared tasks, which included a translation task, a system combination task, and an evaluation task. We conducted a large-scale manual evaluation of 104 machine translation systems and 41 system combination entries. We used the ranking of these systems to measure how strongly auto- matic metrics correlate with human judgments of translation quality for 26 metrics. This year we also investigated increasing the number of human judgments by hiring non-expert annotators through Amazon‰s Mechanical Turk.
Proceedings: ACL 2010 Joint Fifth Workshop on Statistical Machine Translation and MetricsMaTr
Pages: 33 pp.
Location: Uppsala, -1
Dates: July 15-16, 2010
Keywords: machine translation; mt; evaluation; metrology
Research Areas: Information Technology, Software Testing Metrics
PDF version: PDF Document Click here to retrieve PDF version of paper (610KB)