In light of COVID-19 guidance from Federal, State and local health authorities, NIST has made temporary adjustments to staffing and operations to protect the health and safety of NIST employees and the public. As a result of these changes, ALL frvt evaluation tracks submissions are temporarily suspended. Thank you for your understanding.
NIST will run a series of tests toward quantifying face recognition accuracy for people wearing masks. Our approach will be to apply masks to faces digitally, i.e. using software to apply a synthetic mask. This approach leverages current large datasets. We intend to initially test 1:1 verification algorithms. We will test algorithms already submitted to FRVT 1:1, and we will also invite submission of new algorithms that claim to handle masks. We will first mask only the probe image, leaving the reference photo as is. Later, we will consider the effect of masking both images. We will quantify the effect of masks on both false negative and false positives match rates and issue a public report.
The timing for this activity is uncertain because FRVT is currently closed (see above). We have done considerable preparatory work and anticipate issuing a public report shortly after resuming FRVT. We will accept FRVT algorithms from developers only after FRVT resumes. We will announce resumption of FRVT to FRVT-NEWS group subscribers. [Subscribe]
A new FRVT report released as NISTIR 8280 - FRVT Part 3: Demographic Effects on December 19th, 2019, describes and quantifies demographic differentials for contemporary face recognition algorithms. NIST has conducted tests to quantify demographic differences for nearly 200 face recognition algorithms from nearly 100 developers, using four collections of photographs with more than 18 million images of more than 8 million people. Using both one-to-one verification and one-to-many identification algorithms submitted to NIST, the report found empirical evidence for the existence of a wide range of accuracy across demographic differences in the majority of the current face recognition algorithms that were evaluated.