Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

2009 Performance Metrics for Intelligent Systems (PerMIS'09) Workshop

The Performance Metrics for Intelligent Systems workshop is the only one of its kind dedicated to defining measures and methodologies of evaluating performance of intelligent systems. Started in 2000, the PerMIS series focuses on applications of performance measures to practical problems in commercial, industrial, homeland security, and military applications. It has proved to be an excellent forum for discussions and partnerships, dissemination of ideas, and future collaborations between researchers, graduate students, and practitioners from industry, academia, and government agencies.

The main theme of the ninth iteration of the workshop, PerMIS'09, seeks to address the question: Does performance measurement accelerate the pace of advancement for intelligent systems? In addition to the main theme, as in previous years, the workshop will focus on applications of performance measures to practical problems in commercial, industrial, homeland security, and military applications. Topic areas include, but are not limited to:

  • Defining and measuring aspects of a system:
    • The level of autonomy
    • Human-robot interaction
    • Collaboration & coordination
    • Taxonomies
    • Biologically inspired models
  • Evaluating components within intelligent systems:
    • Sensing and perception
    • Knowledge representation, world models, ontologies
    • Planning and control
    • Learning and adapting
    • Reasoning
  • Infrastructural support for performance evaluation:
    • Testbeds and competitions for intercomparisons
    • Instrumentation and other measurement tools
    • Simulation and modeling support
  • Technology readiness measures for intelligent systems
  • Benchmarks and applied performance measures in various domains, e.g.,
    • Industrial and manufacturing systems
    • Intelligent transportation systems
    • Emergency response robots (e.g., search and rescue, bomb disposal)
    • Homeland security systems
    • Defense robotics
    • Intelligent systems for hazardous environments (e.g., nuclear remediation)
    • Smart grid
    • Space robotics
    • Medical robotics & assistive devices
The Proceedings of PerMIS are indexed by INSPEC, Compendex, ACM Digital Library, and are released as a NIST Special Publication. Selected papers from past workshops have been published as special issues with the Journal of Integrated Computer-Aided Engineering (PerMIS'04) and the Journal of Field Robotics (PerMIS'06) and as a Springer Edited Book Volume (PerMIS'08).

Past Workshops


NIST, DARPA, IEEE Technical Committee on Performance Evaluation and Benchmarking of Robotic and Automation Systems (TC-PEBRAS), IEEE Washington Section Robotics and Automation Society (RAS) Chapter, and the Association for Computing Machinery Special Interest Group on Artificial Intelligence (ACM-SIGART)

CALL for Papers

Plenary Addresses

PerMIS Program (pdf)

Program at a Glance (pdf)

Digests (pdf)

Final Proceedings (pdf)

Created November 9, 2010, Updated September 21, 2016