Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Contradictions and Justifications: Extensions to the Textual Entailment Task

Published

Author(s)

Ellen M. Voorhees

Abstract

The third PASCAL Recognizing Textual Entailment Challenge (RTE-3) contained an optional task that extended the main entailment task by requiring a system to make three-way entailment decisions (entails, contradicts, neither) and to justify its response.  Contradiction was rare in the RTE-3 test set, occurring in only about 10% of the cases, and systems found accurately detecting it difficult.  Subsequent analysis of the results shows a test set must contain many more entailment pairs for the three-way task than the traditional two-way task to have equal confidence in system comparisons.  Each of six human judges representing eventual end users rated the quality of a justification by assigning "understandability" and "correctness" scores.  Ratings of the same justification across judges differed significantly, signaling the need for a better characterization of the justification task.
Proceedings Title
Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies (ACL 2008)
Conference Dates
June 15-20, 2008
Conference Location
Columbus, OH
Conference Title
46th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies (ACL 2008)

Keywords

natural language processing, textual entailment

Citation

Voorhees, E. (2008), Contradictions and Justifications: Extensions to the Textual Entailment Task, Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies (ACL 2008), Columbus, OH, [online], https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=152115 (Accessed March 29, 2024)
Created June 16, 2008, Updated February 19, 2017