Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Performance Evaluation of Intelligent Systems at the National Institute of Standards and Technology (NIST)

Published

Author(s)

Craig I. Schlenoff, Harry A. Scott, Stephen B. Balakirsky

Abstract

The ability to comprehensively evaluate the quantitative and qualitative performance of an intelligent system is critical to accurately predicting how it will perform in various situations. The design of such evaluations is often as much of a research challenge as is the design of the intelligent systems themselves. Over the past decade, the Intelligent Systems Division (ISD), a part of the National Institute of Standards and Technology (NIST), has been at the forefront of assessing the performance of the various intelligent systems. This paper will give a broad overview of some of the evaluation efforts that have been pursued by ISD over the past few years, including performance evaluation of emergency response robots, sensor systems on unmanned ground vehicles, speech-to-speech translation systems, and the development of performance metrics for mixed-palletizing through the use of a simulation environment.
Citation
International Test and Evaluation Association (ITEA) Journal

Keywords

US&R robot performance, TRANSTAC, mixed-palletizing, simulation, unmanned vehicle, tracking moving objects, performance evaluation, metrics

Citation

Schlenoff, C. , Scott, H. and Balakirsky, S. (2010), Performance Evaluation of Intelligent Systems at the National Institute of Standards and Technology (NIST), International Test and Evaluation Association (ITEA) Journal, [online], https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=906568 (Accessed April 25, 2024)
Created December 31, 2010, Updated February 19, 2017