Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Performance Evaluation of Autonomous Mobile Robots

Published

Author(s)

Adam S. Jacoff, Elena R. Messina, John Evans

Abstract

The National Institute of Standards and Technology (NIST) has initiated a program to develop quantitative metrics for machine intelligence. One of the possible approaches to evaluating machine intelligence is task-based performance testing, like a mouse in a maze. A series of application-specific testbeds is envisioned. NIST has created a set of reference test arenas for evaluating the performance of autonomous mobile robots performing urban search and rescue tasks. Robots must explore the maze-like test course, map the environment, find the simulated victims, and then report back their findings. These arenas have been used for competitions at the American Association for Artificial Intelligence (AAAI) and the International Joint Conference on Artificial Intelligence (IJCAI) meetings the last two summers, and have now been adopted for the newly emerging RoboCupRescue competitions held internationally each year. In this paper, we describe our approach toward developing performance metrics for autonomous mobile robots through standardized testing within representative environments and objective performance evaluations. Our intent is to help accelerate the robotic research communities' advancement of mobile robot capabilities, thereby improving the effectiveness of robots performing within industrial settings, hazardous environments, and in exploration.
Citation
Industrial Robot
Volume
29

Keywords

Evaluation, Perception, Performance, Robots

Citation

Jacoff, A. , Messina, E. and Evans, J. (2002), Performance Evaluation of Autonomous Mobile Robots, Industrial Robot, [online], https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=821680 (Accessed November 30, 2023)
Created February 1, 2002, Updated February 17, 2017