Performance Evaluation of Autonomous Mobile Robots

Published

February 1, 2002

Author(s)

Adam S. Jacoff, Elena R. Messina, John Evans

Abstract

The National Institute of Standards and Technology (NIST) has initiated a program to develop quantitative metrics for machine intelligence. One of the possible approaches to evaluating machine intelligence is task-based performance testing, like a mouse in a maze. A series of application-specific testbeds is envisioned. NIST has created a set of reference test arenas for evaluating the performance of autonomous mobile robots performing urban search and rescue tasks. Robots must explore the maze-like test course, map the environment, find the simulated victims, and then report back their findings. These arenas have been used for competitions at the American Association for Artificial Intelligence (AAAI) and the International Joint Conference on Artificial Intelligence (IJCAI) meetings the last two summers, and have now been adopted for the newly emerging RoboCupRescue competitions held internationally each year. In this paper, we describe our approach toward developing performance metrics for autonomous mobile robots through standardized testing within representative environments and objective performance evaluations. Our intent is to help accelerate the robotic research communities' advancement of mobile robot capabilities, thereby improving the effectiveness of robots performing within industrial settings, hazardous environments, and in exploration.

Citation

Industrial Robot

Volume

29

Pub Type

Journals

Download Paper

Local Download

Keywords

Evaluation, Perception, Performance, Robots

Citation

Jacoff, A. , Messina, E. and Evans, J. (2002), Performance Evaluation of Autonomous Mobile Robots, Industrial Robot, [online], https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=821680 (Accessed April 18, 2024)