Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Evaluating automatic face recognition systems with human benchmarks

Published

Author(s)

P J. Phillips, Alice O'Toole

Abstract

Human face recognition skills are often considered the gold standard against which machines must compete. Over the last two decades, however, international tests of computer-based face recognition algorithms have shown steady improvements in accuracy with increasingly challenging photometric conditions. Indeed, the most recent comparisons between humans and algorithms show that the best algorithms compete favorably with humans recognizing frontal images of faces—even across substantial changes in illumination, facial expression, and appearance. We review these comparisons considering both quantitative and qualitative benchmarks for evaluating performance on identification tasks. We also address the question of how to statistically fuse the judgments of humans and machines to improve performance over either “system” operating alone. On the qualitative dimension, studies have shown that the long-standing human challenge of recognizing people of different races and ethnicities has parallels in machine vision. We discuss complex problems this poses for predicting how well computer-based systems will operate in environments with variable demographic diversity (e.g., airport). In summary, we argue that computer-based face recognition systems are now at the level of humans recognizing unfamiliar faces. The next challenge for machines is to begin to operate with the accuracy and robustness humans show for familiar face recognition.
Citation
Forensic Facial Identification: Theory and Practice of Identification from Eyewitnesses, Composites and CCTV
Publisher Info
John Wiley & Sons, Ltd, West Sussex, -1

Citation

Phillips, P. and O'Toole, A. (2015), Evaluating automatic face recognition systems with human benchmarks, Forensic Facial Identification: Theory and Practice of Identification from Eyewitnesses, Composites and CCTV, John Wiley & Sons, Ltd, West Sussex, -1, [online], https://doi.org/10.1002/9781118469538.ch11 (Accessed February 24, 2024)
Created April 10, 2015, Updated November 10, 2018