Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Performance Metrics for Intelligent Systems (PerMIS'10) Workshop

The 2010 Performance Metrics for Intelligent Systems workshop will be the tenth in a series dedicated to defining measures and methodologies of evaluating performance of intelligent systems. Started in 2000, the PerMIS series focuses on applications of performance measures to applied problems in commercial, industrial, homeland security, and military applications.

PerMIS'10 will emphasize the key role of performance assessment in developing intelligent systems that can co-exist with humans towards improving the quality of our lives intertwined with automation. Adaptability to human-centered collaboration, the ability to cope with unstructured, dynamic environments, and keeping humans out of harm's way have been widely accepted as critical prerequisites. Designing such flexible, smart, and safe systems requires that their performance be quantifiable thereby facilitating emerging technologies and societal acceptance.

Selected papers from PerMIS'10 will be considered for publication in a special issue with the International Journal of Intelligent Control and Systems. A National Science Foundation Poster Session is also being planned with travel support to undergraduate and graduate students. The Proceedings of PerMIS are indexed by INSPEC, Compendex, ACM Digital Library, and are released as a NIST Special Publication.

In relation to the main theme, topic areas include, but are not limited to:

  • Defining & Measuring Aspects and Capabilities of a Co-X (e.g. Co-Workers, Co-Inhabitants, ...) System:
    • Human-Robot Interaction, Collaboration and Coordination
    • Mobility
    • Responsiveness, Reliability, Trustworthiness, Interchangeability, Durability
    • Variable Levels of Autonomy
    • Taxonomies
    • Biologically Inspired Models
  • Evaluating Components within Intelligent Systems:
    • Sensing and Perception
    • Knowledge Representation, World Models, Ontologies
    • Planning and Control
    • Learning, Adapting and Reasoning
  • Underlying Infrastructural Support for Performance Assessment:
    • Testing and Evaluation (including testbeds and competitions for inter-comparisons)
    • Instrumentation and Other Measurement Tools
    • Simulation and Modeling
  • Technology Readiness Measures for Intelligent Systems
  • Benchmarks and Applied Performance Measures in Various Domains:
    • Manufacturing, Logistics, and Industrial Systems
    • Service: Domestic, Mining, Agriculture, ...
    • Intelligent Transportation Systems
    • Defense and Security
    • Emergency Response Robots (e.g. search and rescue, bomb disposal)
    • Intelligent systems for Hazardous Environments (e.g. nuclear remediation)
    • Smart Grid
    • Space Robotics
    • Medical & Healthcare

Past Workshops


NIST, DARPA, NSF, Association for Computing Machinery Special Interest Group on Artificial Intelligence (ACM-SIGART), IEEE Technical Committee on Performance Evaluation and Benchmarking of Robotic and Automation Systems (TC-PEBRAS), IEEE Wash./NoVA Sensors Council Chapter



Student Poster Session

Plenary Addresses

Submission Information

Travel and Accommodations

Special Sessions

PerMIS Program (pdf)

Program at a Glance (pdf)

Digests (pdf)

Created March 25, 2010, Updated September 21, 2016