Go to the NIST home page Go to the TAC Home Page TAC Banner

Return to TAC Homepage

KBP 2013 Tracks
  Entity Linking
  Slot Filling (SF)
  Temporal SF
  Spanish SF
  Sentiment SF
  SF Validation
  Cold Start

Call for Participation
Track Registration
Schedule
Data
Reporting Guidelines
Workshop
Organizers




CALL FOR PARTICIPATION

Text Analysis Conference
Knowledge Base Population 2013

Evaluation: February-November, 2013
Workshop: November 18-19, 2013

http://www.nist.gov/tac/2013/KBP/
Follow us on twitter @TACKBP2013

Conducted by:
U.S. National Institute of Standards and Technology (NIST)

With support from:
U.S. Department of Defense


INTRODUCTION

The Text Analysis Conference (TAC) is a series of evaluations and workshops organized to promote research in Natural Language Processing and related applications, by providing a large test collection, common evaluation procedures, and a forum for organizations to share their results.

The goal of TAC Knowledge Base Population (KBP) is to develop and evaluate technologies for building and populating knowledge bases (KBs) about named entities from unstructured text. KBP systems must either populate an existing reference KB, or else build a KB from scratch.

You are invited to participate in TAC KBP 2013. Organizations may choose to participate in any or all of the TAC KBP 2013 tasks. NIST provides test data for each KBP task, and participants run their NLP systems on the data and return their results to NIST for evaluation. TAC KBP culminates in a November workshop at NIST in Gaithersburg, Maryland, USA.

All results submitted to NIST are archived on the TAC web site, and all evaluations of submitted results are included in the workshop proceedings. Dissemination of TAC work and results other than in the workshop proceedings is welcomed, but the conditions of participation specifically preclude any advertising claims based on TAC results.

TRACKS

TAC KBP 2013 offers shared tasks in 7 tracks, all aimed at improving the ability to automatically populate knowledge bases from text:
  1. Entity Linking
    The entity linking task is to link names in a document collection to entities in a reference KB, or to new named entities discovered in the document collection. Tasks are offered in English, Spanish, and Chinese.
    Track coordinator: James Mayfield (james.mayfield@jhuapl.edu)
    Track home page: http://www.nist.gov/tac/2013/KBP/EntityLinking/

  2. English Slot Filling
    The slot filling task is to search a document collection to fill in values for predefined slots (attributes) for a given entity in a reference KB.
    Track coordinator: Mihai Surdeanu (msurdeanu@email.arizona.edu)
    Track home page: http://surdeanu.info/kbp2013/

  3. Temporal Slot Filling
    The goal of temporal slot filling is to augment a KB with temporal constraints on slot filling relations.
    Track coordinator: Mihai Surdeanu (msurdeanu@email.arizona.edu)
    Track home page: http://surdeanu.info/kbp2013/

  4. Cross-lingual Spanish Slot Filling
    The Spanish slot filling track extends English slot filling to the cross-lingual paradigm, in which English and Spanish documents are used to populate an English reference KB.
    Track coordinator: Jason Duncan (crosslingualsf@gmail.com)
    Track home page: http://www.nist.gov/tac/2013/KBP/SpanishSF/

  5. Sentiment Slot Filling
    The overall goal of the Sentiment Slot Filling track is to assess the quality of detectors for scoped and attributed sentiment.
    Track coordinator: Margaret Mitchell (m.mitchell@jhu.edu)
    Track home page: http://www.nist.gov/tac/2013/KBP/SentimentSF/

  6. Slot Filler Validation
    The Slot Filler Validation track focuses on the refinement of output from slot filling systems by either combining information from multiple slot filling systems, or applying more intensive linguistic processing to validate individual candidate slot fillers.
    Track coordinator: Hoa Dang (hoa.dang@nist.gov)
    Track home page: http://www.nist.gov/tac/2013/KBP/SFValidation/

  7. Cold Start KBP
    The Cold Start track integrates entity linking and slot filling to build a knowledge base from scratch.
    Track coordinator: James Mayfield (james.mayfield@jhuapl.edu)
    Track home page: http://www.nist.gov/tac/2013/KBP/ColdStart/

REGISTRATION

Organizations wishing to participate in any of the TAC KBP 2013 tracks are invited to register online by June 15, 2013. Participants are advised to register and submit all required agreement forms as soon as possible in order to receive timely access to evaluation resources, including any sample and training data. Registration for a task does not commit you to participating in the task, but is helpful to know for planning. Late registration will be permitted only if resources allow. Any questions about conference participation may be sent to the TAC project manager: tac-manager@nist.gov.

Track registration: http://www.nist.gov/tac/2013/KBP/registration.html

WORKSHOP

The TAC KBP 2013 workshop will be held November 18-19, 2013, in Gaithersburg, Maryland, USA. The workshop is a forum both for presentation of results (including failure analyses and system comparisons), and for more lengthy system presentations describing techniques used, experiments run on the data, and other issues of interest to NLP researchers. KBP track participants who wish to give a presentation during the workshop will submit a short abstract in September describing the experiments they performed. As there is a limited amount of time for oral presentations, the abstracts will be used to determine which participants are asked to speak and which will present in a poster session.

SCHEDULE

    TAC KBP 2013 Schedule
    By early June 2013Track guidelines posted
    By early June 2013Distribution of document collections
    June 15, 2013Deadline for registration for track participation
    July - September 2013Track evaluation windows (varies by track)
    Late September 2013Release of individual evaluated results to participants
    Late September 2013Deadline for workshop presentation proposals
    Mid October 2013Notification of acceptance of presentation proposals
    October 27, 2013Deadline for system reports (workshop notebook version)
    November 18-19, 2013TAC KBP 2013 workshop in Gaithersburg, Maryland, USA
    Early February 2014Deadline for system reports (final proceedings version)

ORGANIZING COMMITTEE

Javier Artiles (Rakuten Institute of Technology, javart@gmail.com)
Hoa Trang Dang (NIST, hoa.dang@nist.gov)
Jason Duncan (U.S. Department of Defense, crosslingualsf@gmail.com)
Joe Ellis (LDC, joellis@ldc.upenn.edu)
Kira Griffitt (LDC, kiragrif@ldc.upenn.edu)
Ralph Grishman (New York University, grishman@cs.nyu.edu)
Heng Ji (City University of New York, hengji@cs.qc.cuny.edu)
Xuansong Li (LDC, xuansong@ldc.upenn.edu)
James Mayfield (Johns Hopkins University, james.mayfield@jhuapl.edu)
Paul McNamee (Johns Hopkins University, paul.mcnamee@jhuapl.edu)
Margaret Mitchell (Johns Hopkins University, m.mitchell@jhu.edu)
Boyan Onyshkevych (U.S. Department of Defense, baonysh@tycho.ncsc.mil)
Stephanie Strassel (LDC, strassel@ldc.upenn.edu)
Mihai Surdeanu (University of Arizona, msurdeanu@email.arizona.edu)


NIST is an agency of the
U.S. Department of Commerce

privacy policy / security notice / accessibility statement
disclaimer
FOIA

Last updated: Thursday, 06-Jun-2013 16:08:04 EDT
Comments to: tac-web@nist.gov