Text Analysis Conference

The Text Analysis Conference (TAC) is a series of evaluation workshops organized to encourage research in Natural Language Processing and related applications, by providing a large test collection, common evaluation procedures, and a forum for organizations to share their results. TAC comprises sets of tasks known as "tracks," each of which focuses on a particular subproblem of NLP. TAC tracks focus on end-user tasks, but also include component evaluations situated within the context of end-user tasks.

TAC 2012 focuses on Knowledge Base Population (KBP). The goal of Knowledge Base Population is to promote research in automated systems that discover information about named entities as found in a large corpus and incorporate this information into a knowledge base. TAC 2012 fields tasks in three areas, all aimed at improving the ability to automatically populate knowledge bases from text:

1. Entity-Linking: Given a name (of a Person, Organization, or Geopolitical Entity) and a document containing that name, determine the KB node for the named entity, adding a new node for the entity if it is not already in the KB. The reference KB is derived from English Wikipedia, while source documents come from a variety of languages, including:
• English
• Chinese
• Spanish
2. Slot-Filling: Given a named entity and a pre-defined set of attributes ("slots") for the entity type, augment a KB node for that entity by extracting all new learnable slot values for the entity as found in a large corpus of documents. The reference KB is derived from English Wikipedia, while source documents come from a variety of languages, including:
• English
• Spanish
A diagnostic task, Slot Filler Validation, will be to determine whether a candidate filler in a document is a correct slot-filler for a given entity.
3. Cold Start Knowledge Base Population: Given a KB schema with an empty knowledge base, build the KB from scratch by mining a large text collection.

Additionally, TAC continues support for tracks in Summarization and Recognizing Textual Entailment (RTE). The next Summarization and RTE evaluations are planned for early 2013. An interim Workshop on Evaluation Metrics and System Comparison for Automatic Summarization (WEAS) will be held at NAACL-HLT on June 8, 2012, in Montreal, Canada.