Skip to main content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Performance Evaluation of Autonomous Mobile Robots

Published

Author(s)

Adam S. Jacoff, Elena R. Messina, John Evans

Abstract

The National Institute of Standards and Technology (NIST) has initiated a program to develop quantitative metrics for machine intelligence. One of the possible approaches to evaluating machine intelligence is task-based performance testing, like a mouse in a maze. A series of application-specific testbeds is envisioned. NIST has created a set of reference test arenas for evaluating the performance of autonomous mobile robots performing urban search and rescue tasks. Robots must explore the maze-like test course, map the environment, find the simulated victims, and then report back their findings. These arenas have been used for competitions at the American Association for Artificial Intelligence (AAAI) and the International Joint Conference on Artificial Intelligence (IJCAI) meetings the last two summers, and have now been adopted for the newly emerging RoboCupRescue competitions held internationally each year. In this paper, we describe our approach toward developing performance metrics for autonomous mobile robots through standardized testing within representative environments and objective performance evaluations. Our intent is to help accelerate the robotic research communities' advancement of mobile robot capabilities, thereby improving the effectiveness of robots performing within industrial settings, hazardous environments, and in exploration.
Citation
Industrial Robot
Volume
29

Keywords

Evaluation, Perception, Performance, Robots
Created February 1, 2002, Updated February 17, 2017