TAC 2014 Tracks
Slot Filling (SF)
Call for Participation
TAC 2014 Workshop
TAC KBP 2014 Event Track
The goal of the TAC KBP Event track is to extract information about events such that the information would be suitable as input to a knowledge base. The Event Argument Extraction task aims to extract information about entities (and times) and the role they play in an event. Participating systems will extract tuples that include (EventType, Role, Argument). EventType and Role will be drawn from an externally specified ontology that is based on ACE 2005. Arguments will be strings from within a document representing the canonical (most-specific) name or description of the entity.
While this task does not require the reification of events (or linking the different arguments of an event), systems developed for this task will support KB-queries that link entities to participation in an event-- for example "List ORGANIZATIONs with a PURCHASER role". When combined with other KBP technologies (i.e. slot-filling and entity-linking), more complex, multi-hop queries are possible-- for example, "List ORGANIZATIONSs with MEMBERs who have served as an ATTACKER".
This is a new task in 2014 and will be evaluated in English only. In 2015, we expect to extend to additional TAC languages and add an evaluation of a system's ability to reify events and connect their arguments.
Systems will be given a 500-1,000 document corpus and asked to extract instances of arguments that play a role in some event. The corpus will be a mix of newswire and discussion forum documents. A discussion forum document may contain multiple posts. The corpus will be manually and automatically filtered to ensure at least a few instances of all event-types. The discussion-forum posts will be automatically filtered to identify those posts that are not simply reposts of newswire documents. Very long discussion-forum threads will be truncated.
Systems will be evaluated on F-Score of (EventType, Role, NormalizedArgumentString, Realis) over the pooled assessments. The assessment pool for each document will include a human-participant. The human-participants submission will be assessed in the same manner as the system submissions.
While the ACE 2005 event annotation is being provided to all participants, this task diverges from ACE in some cases. One example of divergence is the addition of correct answers derived through inference/world knowledge. This evaluation will treat as correct some cases that were explicitly excluded in ACE 2005.
Track CoordinatorMarjorie Freedman (BBN, email@example.com)
NIST is an agency of the
U.S. Department of Commerce
Last updated: Monday, 27-Oct-2014 13:25:44 EDT
Comments to: firstname.lastname@example.org