FRVT 1:N Status
Results for algorithms submitted to NIST in late 2018 (Phase 3 of FRVT 1:N 2018) have been published as NIST Interagency Report 8271, along with supplemental algorithm report cards. Results for algorithms submitted to Ongoing FRVT 1:N (after June 2019) will be published shortly.
The FRVT 2018 will measure advancements in the accuracy and speed of one-to-many face recognition identification algorithms searching enrolled galleries containing at least 10 million identities. The evaluation will primarily use standardized portrait images, and will quantify how accuracy depends on subject-specific demographics and image-specific quality factors.
How To Participate
Minimum FRVT 1:1 accuracy requirements: In order to participate in FRVT 1:N, participants must submit at least one algorithm to the FRVT 1:1 track, and its accuracy must exceed at least one of the following:
- VisaMC dataset: FNMR of 0.025 or less at FMR=0.0001 OR
- Mugshot dataset: FNMR of 0.025 or less at FMR=0.00001
Once an organization submits a 1:1 algorithm that exceeds the minimum accuracy requirements, that organization may submit algorithms to the 1:N track thereafter.
Summary: To participate in this evaluation, developers need to submit a participation agreement to NIST, wrap their software behind the published C++ API, run their libraries through the provided validation package (which creates a submission package), encrypt the package, and provide a download link for the encrypted submission package. More details are provided below.
- API Document [last update: 2019-04-01]: A new API document has been published. All FRVT APIs reference the supporting FRVT General Evaluation Specifications, which includes hardware and operating system environment, software requirements, reporting, and common data structures that support the APIs. Developers must ensure that their submission conforms to the API specifications.
- Validation [last update: 2019-05-29]: An updated validation package has been published to reflect the API changes. All participants must run their software through the validation package prior to submission. The purpose of validation is to ensure consistent algorithm output between your execution and NIST's execution.
- Participation Agreement [last update: 2019-04-22]: A new Participation Agreement for the FRVT Ongoing 1:N track has been published. All organizations wishing to participate in the 1:N track must submit a new Participation Agreement to NIST.
- Encryption Instructions [last update: 2017-11-20]: All submissions must be properly encrypted and signed before transmission to NIST. This must be done according to these instructions using the FRVT Ongoing public key linked from this page. Participants must email their public key to NIST. The participant's public key must correspond to the participant's public-key fingerprint provided on the signed Participation Application.
- Submission Instructions [last update: 2017-11-20]: Encrypted files below 20MB can be emailed to NIST at firstname.lastname@example.org. Encrypted files above 20MB can be provided as a download link from a generic http webserver (e.g., Google Drive). We cannot accept Dropbox links. NIST will not register, or establish any kind of membership, on the provided website. Participants can submit their algorithm(s) as soon as the signed participation agreement is sent to NIST. There is no need to wait for NIST confirmation of the received agreement.