In conjunction with IARPA, NIST is running the Fusion of Face Recognition Algorithms (FOFRA) 2018 Prize Challenge to advance face recognition on unconstrained images. The 2018 challenge tasks developers with fusing the outputs of multiple face recognition algorithms applied to the same input images. Fusion can be conducted on templates, before recognition, or on scores produced by recognition, with prizes awarded for best improvements to verification and identification accuracy. Thus, IARPA will award cash prizes in four categories: template- and score-level fusion applied to verification and identification, with extra awards for open-source solutions. See IARPA's challenge website for prize and eligibility details.
NIST will provide training data, i.e. sets of templates and scores obtained from running face recognition algorithms on common input images. For all combinations of fused algorithms, FOFRA participants should develop score and template fusion methods that improve verification and identification accuracy. Participants are required to submit fusion schemes to NIST, either as a binary library implementing a simple C++ API, or as open-source R code also with a simple API. The fusion schemes must support fusion of outputs from the particular face recognition algorithms provided in the training data.
Details are provided in these slides. Developers are encouraged to consult the sizable academic literature, and to publish their methods. We note that only a minority of papers consider fusion of identification outputs, even fewer in an open-universe, and similarly only a different minority address template level fusion. FOFRA's largest prizes will be awarded to template fusion for identification.
IARPA will award prizes to those fusion implementations that afford the best improvements in recognition accuracy, as measured in trials performed by NIST. Those trials will involve the application of participant-supplied fusion schemes to outputs from known face recognition algorithms applied to imagery sequestered at NIST. Extra awards will be given to open-source solutions. Open-source implementations must be made freely available on the internet.
Participation is open to universities, corporations and other organizations worldwide. There is no charge for participation. See IARPAs challenge website for prize and eligibility details.
- API Document [last update: 2018-07-11]
The final API document is now available. Developers must ensure that their submission conforms to the document's specifications.
Participation Agreement [last update: 2018-05-21]
FOFRA is conducted by NIST, an agency of the United States Government. All organizations who seek to participate in FOFRA must sign and submit all pages of this participation agreement.
Training Data & Validation Package [last update: 2018-05-21]
Provided as a part of this challenge is training data, which consists of 1) 1:1 match scores, 2) 1:N candidate lists and 3) feature vectors from three different algorithms for the purposes of fusion development. Participants are to take the provided training data to develop fusion scheme(s) for the three algorithm scores and feature vectors. As a part of the validation process, the developer’s submission will be run on the training data, and the output(s) will be submitted to NIST, along with the implementation. The validation package can be obtained by request here. All participants must run their software on the training data provided in the validation package prior to submission. The purpose of validation is to ensure reproducibility. Our execution of your algorithm must produce identical outputs to your execution.
- Encryption Instructions [last update: 2018-05-21]:
All submissions must be properly encrypted and signed before transmission to NIST. This must be done according to these instructions using the FRVT Ongoing public key linked from this page. Participants must email their public key to NIST. The participant's public key must correspond to the participant's public-key fingerprint provided on the signed Participation Application.
- Submission Instructions [last update: 2018-05-21]:
Encrypted files below 20MB can be emailed to NIST at firstname.lastname@example.org. Encrypted files above 20MB can be provided as a download link from a generic http webserver (e.g., Google Drive). We cannot accept Dropbox links. NIST will not register, or establish any kind of membership, on the provided website. Participants can submit their algorithm(s) as soon as the signed participation agreement is sent to NIST. There is no need to wait for NIST confirmation of the received agreement.
FAQs [last update: 2018-05-21]:
Ongoing responses to a number of questions regarding the evaluation will be addressed in our FAQs document.
2018-June 06: Deadline for comments on API and other documents
2018-Aug-06: Final algorithm submission deadline
2018-Oct: Announcement of winners
Inquiries and comments may be submitted to FOFRA2018@nist.gov.