NIST TC4TL Challenge

Summary

One of the keys to managing the current (and future) epidemic is notifying people of possible virus exposure so they can isolate and seek treatment to limit further spread of the disease. While manual contact tracing is effective for notifying those who may have been exposed, it is believed that automated exposure notification will be a necessary addition as societies open up. Current approaches to automated exposure notification rely on using Bluetooth Low Energy (BLE) signals (or chirps) from smartphones to detect if a person has been too close for too long (TC4TL) to an infected individual. However, the received signal strength indicator (RSSI) value of Bluetooth chirps sent between phones is a very noisy estimator of the actual distance between the phones and can be dramatically affected in real-world conditions by i) where the phones are carried, ii) body positions, ii) physical barriers, and iv) multi-path environments, to mention a few. To better characterize the effectiveness of range and time estimation using the BLE signal, many research organizations around the world are collecting Bluetooth handshake data as well as other phone sensor data (e.g., accelerometer, gyroscope, proximity) between various types of phones with simulated real-world variability. The best hope for a solution to this difficult and important problem is to leverage the world-wide research community with common tasks, data, and success metrics that allow for the exchange of and building on collective ideas and approaches.

Objectives

The National Institute of Standards and Technology (NIST), in coordination with the MIT PACT project, is organizing a TC4TL detector evaluation to facilitate this research effort. The evaluation serves the following objectives:

• to explore promising new ideas in TC4TL detection using BLE signal,

• to support the development of advanced technologies incorporating these ideas,
• to measure performance of the state-of-the-art TC4TL detectors.

 It is intended to be of interest to all researchers in the machine learning community interested in the TC4TL detection problem using BLE signals. To this end the evaluation is designed to be simple, to focus on core technology issues, to be fully supported, and to be accessible to those wishing to participate.                                 

Tasks and Metrics

The basic task in the NIST TC4TL Challenge is estimating the distance and time between two phones given a series of RSSI values along with other phone sensor data. These distance and time estimates will be converted into contact event hypothesis labels, i.e., TC4TL or not-TC4TL, using two parameters, namely distance (D) and time (T) that define a TC4TL event. Reference TC4TL labels are generated using true distance and time from contact events. Hypothesized labels are compared to reference labels and probability of false negative (P_miss) and probability of false positive (P_fa) are calculated. A normalized decision cost function (nDCF) combines these two errors into a single value using weights reflecting the relative cost of each type of error.

Data

To obtain samples of data that has been collected by the PACT consortium so far, or to contribute to the data collection effort, please visit:

https://mitll.github.io/PACT/datasets.html

Evaluation Plan

NIST Pilot TC4TL Challenge Evaluation Plan

Registration

Please visit: https://tc4tlchallenge.nist.gov

For more information about the challenge please send questions to tc4tl_poc [at] nist.gov (tc4tl_poc[at]nist[dot]gov). For the TC4TL Challenge discussion please visit our Google Group.