Official websites use .gov
A .gov website belongs to an official government organization in the United States.
Secure .gov websites use HTTPS
A lock (
) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.
FRVT Ongoing Test Page
FRVT 1:1 [Latest Report] [Agreement] [API] [Validation] [Encryption] [FAQs]
FRVT 1:N [Homepage] [API]
FRVT Status
NIST will be accepting submissions again for Ongoing FRVT 1:1 between October 1 - 19, 2018.
The Latest Report includes results for thirty eight algorithms applied to six datasets. The report, which is a draft and open for comments, will be updated on a monthly basis as algorithms and computations complete, as datasets are added, and as new analyses are developed.
To receive update notifications, subscribe by email via frvt-news-request [at] nist.gov (subject: subscribe) (subscribe).
FRVT Leaderboard
The leaderboard shows the top performing algorithms measured on false non-match rate (FNMR) across several different datasets. FNMR is the proportion of mated comparisons below a threshold set to achieve the false match rate (FMR) given in the header on the fourth row. FMR is the proportion of impostor comparisons at or above that threshold.
| # | Developer | VISA Photos FNMR@ FMR ≤ 1e-06 |
VISA Photos FNMR@ FMR ≤ 0.0001 |
MUGSHOT Photos FNMR@ FMR ≤ 0.0001 |
WILD Photos FNMR@ FMR ≤ 0.0001 |
CHILD EXP Photos FNMR@ FMR ≤ 0.01 |
Submission Date |
|---|---|---|---|---|---|---|---|
| 1 | yitu-000 | 0.0331 | 0.0217 | 0.0171 | 0.4315 | 0.5867 | 2017_05_23 |
| 2 | vocord-002 | 0.0342 | 0.0131 | 0.0192 | 0.94839 | 0.76222 | 2017_06_07 |
| 3 | visionlabs-002 | 0.0513 | 0.0142 | 0.0203 | 0.3973 | 0.5565 | 2017_09_08 |
| 4 | fdu-000 | 0.0584 | 0.0163 | 0.0214 | 0.5499 | -41 | 2017_11_22 |
| 5 | ntechlab-002 | 0.0655 | 0.0215 | 0.0236 | 0.3242 | 0.4591 | 2017_08_23 |
| 6 | tongyitrans-002 | 0.0666 | 0.03010 | 0.03917 | 0.72523 | 0.74617 | 2017_07_15 |
| 7 | tongyitrans-001 | 0.0727 | 0.03819 | 0.04118 | 0.70420 | 0.74316 | 2017_04_01 |
| 8 | ntechlab-001 | 0.0838 | 0.0258 | 0.03011 | 0.3191 | 0.4722 | 2017_05_10 |
| 9 | tevian-000 | 0.1299 | 0.03617 | 0.0225 | 0.5008 | -40 | 2017_12_10 |
| 10 | morpho-000 | 0.13410 | 0.0269 | 0.0289 | 0.89336 | 0.84633 | 2017_07_11 |
| 11 | vocord-001 | 0.14111 | 0.03515 | 0.06330 | 0.65414 | 0.69511 | 2017_04_21 |
| 12 | 3divi-002 | 0.15412 | 0.0216 | 0.03315 | 0.4957 | -39 | 2017_10_20 |
| 13 | 3divi-001 | 0.15413 | 0.0204 | 0.03012 | 0.4926 | 0.5494 | 2017_06_22 |
| 14 | yisheng-001 | 0.16014 | 0.03212 | 0.04829 | 0.62811 | 0.75620 | 2017_08_22 |
| 15 | neurotechnology-002 | 0.16615 | 0.03616 | 0.04119 | 0.4274 | 0.6838 | 2017_11_02 |
| 16 | visionlabs-001 | 0.18016 | 0.03011 | 0.0247 | 0.59110 | 0.5616 | 2017_06_12 |
| 17 | innovatrics-001 | 0.18317 | 0.03413 | 0.04322 | 0.64313 | 0.81530 | 2017_07_31 |
| 18 | rankone-003 | 0.18418 | 0.03820 | 0.0288 | 0.67415 | -37 | 2017_11_28 |
| 19 | innovatrics-000 | 0.19119 | 0.03414 | 0.04627 | 0.72022 | 0.85234 | 2017_07_31 |
| 20 | yisheng-000 | 0.19920 | 0.03718 | 0.02910 | 0.68618 | 0.74918 | 2017_08_22 |
Additional algorithms not listed in the leaderboard can be found in Table 2 of our latest FRVT report.
Prior editions of the report: 2017-10-03 | 2017-08-25 | 2017-08-07 | 2017-07-31 | 2015-05-15 | 2017-05-01 | 2017-04-12 | 2017-04-03 | 2017-03-23.
Developers may submit their algorithms at any time by following instructions below. Results will be posted as soon as algorithms successfully complete the tests.
Overview
NIST is starting a new evaluation of face recognition technologies starting in February 2017. Unlike our previous evaluations, this activity will be conducted on an ongoing basis: The evaluation will remain open indefinitely such that developers may submit their algorithms to NIST whenever they are ready, but no more frequently than three calendar months. The algorithms will be evaluated rapidly on a first-come-first-served basis, following our MINEX III evaluation of fingerprint recognition implementations. Performance results will be posted to the NIST website as soon as they are ready. This approach more closely aligns evaluation with development schedules; this improves over the two to four year interval between past FRVT tests.
Goals
The FRVT is aimed at measurement of the performance of automated face recognition technologies applied to a wide range of civil, law enforcement and homeland security applications including verification of visa images, de-duplication of passports, recognition across photojournalism images, and identification of child exploitation victims. In all cases the input image will contain one face only. Our performance reports will include measurements of accuracy, speed, storage and memory consumption, and resilience. NIST will report the dependence of performance on the properties of the images and the subjects. In its initial form, FRVT has one assessment track, for face verification.
1:1 Verification Track
- Participation Agreement [last update: 2017-02-10]: FRVT is conducted by NIST, an agency of the United States Government. Participation is free of charge. FRVT is open to a global audience of face recognition developers. All organizations who seek to participate in FRVT must sign and submit all pages of this Participation Agreement.
- API Document [last update: 2017-09-18]: The final version of the 1:1 Verification API document is now available. Developers must ensure that their submission conforms to the document's specifications.
- Validation [last update: 2017-02-15]: The validation package is now available here. All participants must run their software through the provided validation package prior to submission. The purpose of validation is to ensure consistent algorithm output between your execution and NIST's execution.
- Encryption Instructions[last update: 2017-09-11]: All submissions must be properly encrypted and signed before transmission to NIST. This must be done according to these instructions using the FRVT Ongoing public key linked from this page. Participants must email their public key to NIST. The participant's public key must correspond to the participant's public-key fingerprint provided on the signed Participation Application.
- Submission Instructions[last update: 2017-11-09]: Encrypted files below 20MB can be emailed to NIST at frvt [at] nist.gov (frvt[at]nist[dot]gov). Encrypted files above 20MB can be provided as a download link from a generic http webserver (e.g., Google Drive). We cannot accept Dropbox links. NIST will not register, or establish any kind of membership, on the provided website. Participants can submit their algorithm(s) as soon as the signed participation agreement is sent to NIST. There is no need to wait for NIST confirmation of the received agreement.
-
FAQs [last update: 2017-01-30]: Ongoing responses to a number of questions regarding the evaluation are addressed in our FAQs document.
-
Important Dates [last update: 2018-09-26]: The submission window for Ongoing FRVT 1:1 will reopen from October 1 - 19, 2018.
Contact Information
Inquiries and comments may be submitted to frvt [at] nist.gov (frvt[at]nist[dot]gov).
frvt-news-request [at] nist.gov (subject: subscribe) (Subscribe) to the FRVT mailing list to receive emails when announcements or updates are made.