NIST Proprietary Fingerprint Template (PFT) Evaluation
The NIST PFT (2003 - February 2010) evaluation is no longer accepting SDKs for evaluation it has been replaced with PFTII evaluation.
Since June of 2003, NIST has been conducting evaluations of fingerprint-based biometric matching systems using vendor supplied SDKs and comparing these systems to our internal fingerprint matching system. Fingerprint matching systems from vendors are being evaluated to insure that the accuracy of the matchers used in various existing and planned government systems (including US-VISIT) are comparable to the most accurate available COTS products. The main result obtained from these evaluations is an estimate of how well commercial products performed one-to-one matching for verification over a wide range of fingerprint image qualities. The relative accuracy of thumbs and index fingers is also investigated.
Unlike the Fingerprint Vendor Technology Evaluation, FpVTE , these evaluations are ongoing and new SDKs can be included in the test at any time. There are two additional differences between the PFT tests and FpVTE. First, the PFT evaluations are performed on larger samples of each dataset. Second, during the FpVTE testing, each vendor was provided a set of N images and asked to return a matrix of NxN matched scores. The program to perform the testing in FpVTE was written by each vendor and run on the vendor's own computer hardware. In PFT testing, each call to the matcher function compares two fingers at a time and returns a score to the application that was written by NIST. The application that performs the matches uses SDK libraries supplied by each vendor and run on NIST computer hardware.
The final report on the first round of the PFT evaluations performed at NIST is available. This report describes the testing procedures used to evaluate each vendor's matching algorithms. Performance ROC plots are included for each SDK tested. The same techniques for testing and presenting results as described in this report will be used to evaluate any algorithms that are submitted to NIST for testing. The report name is NISTIR 7221 "Studies of One-to-One Fingerprint Matching with Vendor SDK Matchers".
In order to assist you in the testing of your algorithm before submitting it to NIST, we can also provide you with a CD-ROM of fingerprint images. This CD contains fingerprint images from about 180 different subjects. For each subject a pair of tenprint records are included. Each record has 10 rolled images, two plain thumb images, and eight plain images segmented from the two 4-finger slap images. This amounts to having two flat and two rolled images from each finger. With the use of the fingerprint images on this CD you can determine the performance of your system and compare it to those ROC plots of the same data from other systems reviewed in the NISTIR 7221 report.
This can be a very valuable tool for testing your algorithm. It can build confidence in your product. On the other hand, if your algorithm is not ready for NIST evaluation, it can save you the time and resources involved with our testing procedure. A small sample from this CDROM is available for download as a zipfile[2,395K]. If you would like a copy of the CD, send an email request for a copy to craig.watson(at)nist.gov. Include your mailing address (ground mail) and phone number.
Updated Tables and Plots (April 5, 2010).
These tests were performed for the Department of Homeland Security in accordance with section 303 of the Border Security Act, codified at 8 U.S.C. 1732. Specific hardware and software products identified in this report were used in order to adequately support the development of technology to conduct the performance evaluations described in this document. In no case does such identification imply recommendation or endorsement by the National Institute of Standards and Technology, nor does it imply that the products and equipment identified are necessarily the best available for the purpose.