Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Data Science Evaluation Workshop

March 17–18, 2016

National Institute of Standards and Technology (NIST), Gaithersburg, MD

The Information Access Division (IAD)/Multimodal Information Group of the National Institute of Standards and Technology (NIST) held a workshop on the Data Science Evaluation (DSE) series within the new Data Science Research Program (DSRP).

The DSRP provides a forum for community-wide evaluations and supports research progress in the burgeoning field of data science. The DSE is a central component of the DSRP and serves as a broad forum for evaluation that will occur annually in support of data-science research.

In the fall of 2015, IAD hosted a small-scale pre-pilot evaluation in the traffic domain, meant to exercise a range of evaluation metrics and measurement methods. In 2016, IAD will follow up with an open pilot evaluation in the same domain. Proposals for tracks in the DSE will lead to a series of full-scale data science evaluations in 2017.

This workshop program included:

  • A description and discussion of the upcoming public NIST IAD Pilot Evaluation
  • Presentations from Track Coordinator Finalists for potential tracks to be run in the full-scale evaluation in 2017
  • Discussions pertaining to Data Science Challenge Problems
  • Summary and analysis of the NIST IAD Pre-Pilot evaluation results

For additional questions, please contact us at datascience [at] nist.gov (datascience[at]nist[dot]gov).

Registration 

Registration is now closed.

Program

Workshop Agenda.

Slides from the workshop are available below, with each presentation linked to the item. Slides are given in chronological order.

  1. Introduction and Workshop Goals
  2. Overview of NIST IAD Data Science Research Program
  3. Overview of NIST IAD Data Science Evaluation Series
  4. Summary of Data Science Pre-Pilot Evaluation
  5. NIST Perspective on Lessons Learned from Pre-Pilot
  6. UF Perspective on Lessons Learned from Pre-Pilot, Part 1
  7. UF Perspective on Lessons Learned from Pre-Pilot, Part 2
  8. Introduction to Track Coordinator Finalist Presentations
  9. DSE Track Proposal #1: Manual vs. Automated Techniques for Estimating Security Risks
  10. Introduction to Breakout Sessions
  11. Infrastructure Breakout Introduction
  12. Infrastructure Breakout Report
  13. Evaluation Breakout Introduction
  14. Evaluation Breakout Report
  15. Benchmarking Breakout Introduction
  16. Benchmarking Breakout Report
  17. DSE Track Proposal #2: Data Science for Plant Identification with Remote Sensing
  18. Overview of Pilot Evaluation
  19. Challenge Problems Introduction: Overview
  20. Challenge Problems Introduction: Alignment
  21. Challenge Problems Introduction: Cleaning
  22. Challenge Problems Introduction: Prediction
  23. Concluding Remarks

 

Created January 13, 2016, Updated July 10, 2017