Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Comparing Face Recognition Algorithms to Humans on Challenging Tasks



P J. Phillips, Alice O'Toole, Xiaobo An, Joseph Dunlop, Vaidehi Natu


We compared face identifcation by humans and machines using images taken under a variety of uncontrolled illumination conditions in both indoor and outdoor settings. Natural variations in a person's day-to-day appearance (e.g., hair style, facial expression, hats, glasses, etc.) contributed to the difficulty of the task. Both humans and machines matched the identity of people (same or different?) in pairs of frontal view face images. The degree of diculty introduced by photometric and appearance-based variability was estimated using a face recognition algorithm created by fusing three top-performing algorithms from a recent international competition. The algorithm computed similarity scores for a constant set of same-identity and different-identity pairings from multiple images. Image pairs were assigned to good, moderate, and poor accuracy groups by ranking the similarity scores for each identity pairing, and dividing these rankings into three strata. This procedure isolated the role of photometric variables from the effects of the distinctiveness of particular identities. Algorithm performance for these constant identity pairings varied dramatically across the groups. In three experiments, humans matched image pairs from the good, moderate, and poor conditions, rating the likelihood that the images were of the same person (1: sure same - 5: sure different). Algorithms were more accurate than humans in the good and moderate conditions, but were comparable to humans in the poor accuracy condition. To date, these are the most variable illumination- and appearance-based recognition conditions on which humans and machines have been compared. The finding that machines were never less accurate than humans on these challenging frontal images suggests that face recognition systems may be ready for applications with comparable difficulty. We speculate that superiority of machines over humans in the less challenging conditions may be due to the machine use of view-sp
ACM Transactions on Applied Perception


algorithms, human factors, Verification, Experimentation


Phillips, P. , O'Toole, A. , An, X. , Dunlop, J. and Natu, V. (2012), Comparing Face Recognition Algorithms to Humans on Challenging Tasks, ACM Transactions on Applied Perception, [online], (Accessed July 17, 2024)


If you have any questions about this publication or are having problems accessing it, please contact

Created October 17, 2012, Updated February 19, 2017