March 17–18, 2016
National Institute of Standards and Technology (NIST), Gaithersburg, MD
The Information Access Division (IAD)/Multimodal Information Group of the National Institute of Standards and Technology (NIST) held a workshop on the Data Science Evaluation (DSE) series within the new Data Science Research Program (DSRP).
The DSRP provides a forum for community-wide evaluations and supports research progress in the burgeoning field of data science. The DSE is a central component of the DSRP and serves as a broad forum for evaluation that will occur annually in support of data-science research.
In the fall of 2015, IAD hosted a small-scale pre-pilot evaluation in the traffic domain, meant to exercise a range of evaluation metrics and measurement methods. In 2016, IAD will follow up with an open pilot evaluation in the same domain. Proposals for tracks in the DSE will lead to a series of full-scale data science evaluations in 2017.
This workshop program included:
- A description and discussion of the upcoming public NIST IAD Pilot Evaluation
- Presentations from Track Coordinator Finalists for potential tracks to be run in the full-scale evaluation in 2017
- Discussions pertaining to Data Science Challenge Problems
- Summary and analysis of the NIST IAD Pre-Pilot evaluation results
For additional questions, please contact us at datascience [at] nist.gov.
Registration is now closed.
Slides from the workshop are available below, with each presentation linked to the item. Slides are given in chronological order.
- Introduction and Workshop Goals
- Overview of NIST IAD Data Science Research Program
- Overview of NIST IAD Data Science Evaluation Series
- Summary of Data Science Pre-Pilot Evaluation
- NIST Perspective on Lessons Learned from Pre-Pilot
- UF Perspective on Lessons Learned from Pre-Pilot, Part 1
- UF Perspective on Lessons Learned from Pre-Pilot, Part 2
- Introduction to Track Coordinator Finalist Presentations
- DSE Track Proposal #1: Manual vs. Automated Techniques for Estimating Security Risks
- Introduction to Breakout Sessions
- Infrastructure Breakout Introduction
- Infrastructure Breakout Report
- Evaluation Breakout Introduction
- Evaluation Breakout Report
- Benchmarking Breakout Introduction
- Benchmarking Breakout Report
- DSE Track Proposal #2: Data Science for Plant Identification with Remote Sensing
- Overview of Pilot Evaluation
- Challenge Problems Introduction: Overview
- Challenge Problems Introduction: Alignment
- Challenge Problems Introduction: Cleaning
- Challenge Problems Introduction: Prediction
- Concluding Remarks