The most accurate face recognition algorithms have demonstrated the capability to confirm airline passenger identities while making very few errors, according to recent tests of the software conducted at the National Institute of Standards and Technology (NIST).
The findings, released today as Face Recognition Vendor Test (FRVT) Part 7: Identification for Paperless Travel and Immigration (NISTIR 8381), focus on face recognition (FR) algorithms’ performance under a particular set of simulated circumstances: matching images of travelers to previously obtained photos of those travelers stored in a database. This use of FR is currently part of the onboarding process for international flights, both to confirm a passenger’s identity for the airline’s flight roster and also to record the passenger’s official immigration exit from the United States.
The results indicate that several of the FR algorithms NIST tested could perform the task using a single scan of a passenger’s face with 99.5% accuracy or better — especially if the database contains several images of the passenger.
“We ran simulations to characterize a system that is doing two jobs: identifying passengers at the gate and recording their exit for immigration,” said Patrick Grother, a NIST computer scientist and one of the report’s authors. “We found that accuracy varies across algorithms, but that modern algorithms generally perform better. If airlines use the more accurate ones, passengers can board many flights with no errors.”
Previous FRVT studies have focused on evaluating how algorithms perform one of two different tasks that are among FR’s most common applications. The first task, confirming that a photo matches a different one of the same person, is known as “one-to-one” matching and is commonly used for verification work, such as unlocking a smartphone. The second, determining whether the person in the photo has a match in a large database, is known as “one-to-many” matching.
This latest test concerns a specific application of one-to-many matching in airport transit settings, where travelers’ faces are matched against a database of individuals who are all expected to be present. In this scenario, only a few hundred passengers board a given flight. However, NIST also looked at whether the technology could be viable elsewhere in the airport, specifically in the security line where perhaps 100 times more people might be expected during a certain time window. (The database was built from images used in previous FRVT studies, but the subjects were not wearing face masks.)
As with previous studies, the team used software that developers voluntarily submitted to NIST for evaluation. This time, the team only looked at software that was designed to perform the one-to-many matching task, evaluating a total of 29 algorithms.
Among the report’s findings are:
Grother said that the study does not address an important factor: the sort of camera that an FR system uses. Because airport environments differ, and because the cameras themselves operate in different ways, the report offers some guidance for tests that an airline or immigration authority could run to complement the NIST test results. Such tests would provide accuracy estimates that reflect the actual equipment and environment where it is used.
“We do not focus on cameras, which are an influential variable,” he said. “We recommend that officials conduct the other tests we outline so as to refine their operations.”