| Author(s): |
Chris Callison-Burch; Philipp Koehn; Christof Monz; Kay Peterson; Mark A. Przybocki; Omar F. Zaidan; |
| Title: |
Findings of the 2010 Joint Workshop on Statistical Machine Translation and Metrics for Machine Translation |
| Published: |
July 20, 2010 |
| Abstract: |
This paper presents the results of the WMT10 and MetricsMATR10 shared tasks, which included a translation task, a system combination task, and an evaluation task. We conducted a large-scale manual evaluation of 104 machine translation systems and 41 system combination entries. We used the ranking of these systems to measure how strongly auto- matic metrics correlate with human judgments of translation quality for 26 metrics. This year we also investigated increasing the number of human judgments by hiring non-expert annotators through Amazon’s Mechanical Turk. |
| Proceedings: |
ACL 2010 Joint Fifth Workshop on Statistical Machine Translation and MetricsMaTr |
| Pages: |
33 pp. |
| Location: |
Uppsala, -1 |
| Dates: |
July 15-16, 2010 |
| Keywords: |
machine translation; mt; evaluation; metrology |
| Research Areas: |
Information Technology, Software Testing Metrics |
| PDF version: |
 Click here to retrieve PDF version of paper (596KB) |
