NIST IAD DSE Pilot Evaluation Plan
Version 1.8, Updated on September 21, 2016
This document describes the plan for the National Institute of Standards and Technology (NIST) In-
formation Access Division (IAD) Data Science Evaluation (DSE) Series Pilot Evaluation to be held in
fall 2016. This pilot evaluation is a continuation and extension of the Data Science Pre-Pilot Evalua-
tion that was run in 2015, and precedes the first full evaluation of the DSE series. The primary goals
of the pilot are to:
I further develop and exercise the evaluation process at NIST in the context of Data Science,
I provide participants the opportunity to exercise the evaluation process prior to participating in
I serve as an archetype for the development of future evaluation tasks, datasets, and metrics,
I establish baseline performance measurements,
I bring to the fore new measurement methods and techniques that might be applied to a broad
range of use cases, regardless of data type and structure.
The pilot evaluation is set in the automotive traffic domain. This is because the domain is relevant to
a broad audience and because large amounts of data are pub...