U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

PUBLICATIONS

It's Not Always What the Eye Can See – Challenges in the Evaluation of Augmented Reality

Published

Author(s)

Kurtis Goad, Kevin C. Mangold, Susanne M. Furman

Abstract

Augmented reality (AR) technology is developing at a fast pace. Usability evaluation methodologies for AR need to be updated to accommodate the increasing complexity of how people interact with AR technology. Eye tracking metrics, which have been historically successful in traditional, often computer-based technology, have been underutilized when evaluating AR devices. Literature review shows documentation of methods used to evaluate AR since 2000. But these reviews do not address the reasons why evaluators have chosen or excluded certain methods, like eye tracking, and did not address evaluations performed that were not published. This exploratory study addressed these gaps by deploying semi-structured interviews to gain an understanding of the experiences professionals have when evaluating AR technology. Our results show that participants rely most heavily on user feedback and questionnaires/surveys during evaluation. Most participants did not use eye tracking methods. They cited a number of challenges evaluating AR technology and the use of eye tracking including difficulties in data collection and a lack of consistency and standards across devices.
Citation
NIST Interagency/Internal Report (NISTIR) - 8489
Report Number
8489
NIST Pub Series
NIST Interagency/Internal Report (NISTIR)
Pub Type
NIST Pubs

Download Paper

https://doi.org/10.6028/NIST.IR.8489
Local Download
Virtual / augmented reality and Usability and human factors

Citation

Goad, K. , Mangold, K. and Furman, S. (2023), It's Not Always What the Eye Can See – Challenges in the Evaluation of Augmented Reality, NIST Interagency/Internal Report (NISTIR), National Institute of Standards and Technology, Gaithersburg, MD, [online], https://doi.org/10.6028/NIST.IR.8489, https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=956488 (Accessed April 28, 2024)

Created September 19, 2023