Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

2020 Differential Privacy Temporal Map Challenge

Differential Privacy Temporal Map Challenge logo

The NIST, PSCR Differential Privacy Temporal Map Challenge ran from October 2020 through June 2021 awarding $129,000 in cash prizes. The goal of the challenge was to seek innovative algorithms to de-identify public safety-related data with a privacy guarantee. The challenge also sought novel methods of evaluating the quality of synthetic data.

You can try out your own solution using SDNist, an open source Python implementation of our data and scoring metrics. 
 
The challenge was highly successful with more than 70 unique algorithms submissions across all three sprints of the challenge. Four of those algorithms have been open sourced (links in winners table below). Three solutions participated in the Development Contest, where teams were coached by NIST experts to improve the robustness and documentation of their code, creating easy-to-use implementations of sophisticated differential privacy algorithms.
 
The challenge was implemented by DrivenData with assistance from HeroX. Christine Task from Knexus Research Corporation served as the program’s technical lead. Gary Howarth served as the prize manager.

Winners

Algorithm contest

Visit the Challenge data and scoring code repository

Team Total Awards Open Sourced Development Contest APA Citation
N-CRiPT $44,000 -- --  
Minutemen $48,000 Yes Repository link McKenna R. (2021). Adaptive Granularity Mechanism (version 1.0). URL: https://github.com/ryan112358/nist-synthetic-data-2021
DPSyn $38,000 Yes Repository link Chen A., Li N., Li Z., Wang T. (2021). DPSyn: An algorithm for synthesizing microdata for data analysis while satisfying differential privacy (version 1.0). URL: https://github.com/agl-c/deid2_dpsyn
jimking100 $24,000 Yes Repository link King, J. (2021). Privitized Histograms (Version 1.0.0) [Computer software]. https://github.com/JimKing100/PrivacyHistos
Duke Privacy Team $12,000 -- --  
GooseDP $9,000 Yes --  
SyrDP $5,000 -- --  
3401 Walnut $1,000 -- --  

Metrics contest

Submission Place/Prize Amount
MGD: A Utility Metric for Private Data Publication 1st $5,000
Practical DP Metric 2nd $3,000
Confusion Matrix Metric 2nd $3,000
Bounding Utility Loss via Classifiers 3rd $2,000
Confusion Matrix Metric People's Choice Award $1,000

Challenge details

Large data sets containing personally identifiable information (PII) are exceptionally valuable resources for research and policy analysis in a host of fields supporting America's public safety agencies such as emergency planning and epidemiology.

Temporal map data—information that is geographically situated and may change over time—is of particular interest to the public safety community in applications such as optimizing response time and personnel placement, natural disaster response, epidemic tracking, demographic data and civic planning. Yet, the ability to track a person's location over a period of time presents particularly serious privacy concerns.

The Differential Privacy Temporal Map Challenge invited solvers to develop algorithms and metrics that preserve data utility while guaranteeing individual privacy is protected

Participants competed in a series of coding sprints using differential privacy methods on temporal map data. The goal was to create a privacy-preserving dashboard map that shows changes across different map segments over time. 

Challenge elements

The NIST PSCR Differential Privacy Temporal Map Challenge follows on the success of the 2018 Differential Privacy Synthetic Data Challenge, extending the reach and utility of differential privacy algorithms.

  • Temporal map data: The challenge will feature public safety data sets with geographic and temporal elements. Solutions will seek to satisfy differential privacy while preserving characteristics of original data sets as much as possible, including sequential data and geographic characteristics.
  • Utility metrics: Participants submitted white papers on new and innovative ways of measuring the accuracy of data sets produced by differential privacy algorithms.
  • De-identification algorithms: Participants built algorithms to preserve data accuracy while guaranteeing privacy on temporal map data sets.
  • Open source: Beyond expanding the types of data that can be made differentially private, this challenge incentivized teams to open source their resulting software. 
  • Software Development: Following the algorithm contest, NIST and outside privacy and software experts worked with teams to refactor and better document solutions to create easy-to-implement differential privacy libraries.

Algorithm Sprint Descriptions

Sprint 1

De-ID Prize Challenge Sprint 1 Map

Baltimore 911 Incidents
Highly variable PS data
Training data: 2019
Evaluation data: 2016 & 2020
Problem Description | Results

 

Sprint 2

De-ID Prize Challenge Sprint 2 Map

American Community Survey (US Census)
Complex demographic information
Training data: IL + OH
Evaluation data: NY + PA & NC+SC+GA
Problem Description | Results

 

Sprint 3

De-ID Prize Challenge Sprint 3 Map

Chicago Taxi Rides
Linked trip information
Training data: 2019
Evaluation data: 2016 & 2020
Problem Description |  Results

Resources and links

To find out more about the challenge and winners, visit Challenge.gov.

Created September 1, 2020, Updated July 13, 2022