NOTICE: Due to a lapse in annual appropriations, most of this website is not being updated. Learn more.
Form submissions will still be accepted but will not receive responses at this time. Sections of this site for programs using non-appropriated funds (such as NVLAP) or those that are excepted from the shutdown (such as CHIPS and NVD) will continue to be updated.
An official website of the United States government
Here’s how you know
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
Secure .gov websites use HTTPS
A lock (
) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.
Benchmarking and Standardization of Intelligent Robotic Systems
Published
Author(s)
Rajmohan Madhavan, R Lakaemper, Tamas Kalmar-Nagy
Abstract
From mundane and repetitive tasks to assisting first responders in saving lives of victims in disaster scenarios, robots are expected to play an important role in our lives in the coming years. Despite recent advances in mobile robotic systems, lack of widely accepted performance metrics and standards hinder the progress in many application areas such as manufacturing, healthcare, and search and rescue. In this paper, we outline the importance of the development of standardized methods and objective performance evaluation/benchmarking of existing and emerging robotic technologies. We provide a survey of significant past efforts by researchers and developers around the globe and discuss how we can leverage such efforts in advancing the state-of-the-art. Using an example of designing a 'standard' evaluation toolkit for robotic mapping, we illustrate some of the problems faced in developing objective performance metrics whilst accommodating the requirements and restrictions imposed by the intended domain of operation and other practical considerations.
Proceedings Title
Proceedings of the 14th International Conference on Advanced Robotics (ICAR 2009), Munich, Germany, June 22 - 26, 2009.
Madhavan, R.
, Lakaemper, R.
and Kalmar-Nagy, T.
(2010),
Benchmarking and Standardization of Intelligent Robotic Systems, Proceedings of the 14th International Conference on Advanced Robotics (ICAR 2009), Munich, Germany, June 22 - 26, 2009., Munich, -1, [online], https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=904625
(Accessed October 15, 2025)