Take a sneak peek at the new NIST.gov and let us know what you think!
(Please note: some content may not be complete on the beta site.).
NIST Authors in Bold
|Author(s):||Ellen M. Voorhees;|
|Title:||Contradictions and Justifications: Extensions to the Textual Entailment Task|
|Published:||June 16, 2008|
|Abstract:||The third PASCAL Recognizing Textual Entailment Challenge (RTE-3) contained an optional task that extended the main entailment task by requiring a system to make three-way entailment decisions (entails, contradicts, neither) and to justify its response. Contradiction was rare in the RTE-3 test set, occurring in only about 10% of the cases, and systems found accurately detecting it difficult. Subsequent analysis of the results shows a test set must contain many more entailment pairs for the three-way task than the traditional two-way task to have equal confidence in system comparisons. Each of six human judges representing eventual end users rated the quality of a justification by assigning "understandability" and "correctness" scores. Ratings of the same justification across judges differed significantly, signaling the need for a better characterization of the justification task.|
|Conference:||46th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies (ACL 2008)|
|Proceedings:||Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies (ACL 2008)|
|Dates:||June 15-20, 2008|
|Keywords:||natural language processing, textual entailment|
|PDF version:||Click here to retrieve PDF version of paper (80KB)|