Official websites use .gov
A .gov website belongs to an official government organization in the United States.
Secure .gov websites use HTTPS
A lock (
) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.
As expectations for intelligent systems continue to grow, the need for quantitative evaluation of system performance becomes more critical. This third workshop in a series will bring together leading researchers to address methods for measuring the abilities of intelligent systems.
We wish to discuss ideas for quantitative engineering approaches to measuring intelligence. Performance tests and competitions are in this class. These measure the overall system performance in structured situations. There are mathematical approaches to quantifying the abilities of a system, be it through complexity measures, entropy computations, or other calculations of either internal factors or external manifestations.
Many issues remain. Must performance tests be domain specific? How can tests be propagated throughout the community? Is it reasonable to expect that researchers publish their results? Can systems with fundamentally different designs be compared? Who determines what the criteria for evaluation, or "success" are?
Among the topic areas to be considered for this workshop are:- Adaptive and Learning Systems
- Unmanned Autonomous Systems
- Knowledge Intensive Subsystems
- Cognitive and Neural Modelling
- Large Systems with Human-Computer Interaction for Decision Making
- Evolutionary Computations and Activities
- Hierarchical and Distributed Controllers with Elements of Autonomy
- Image Processing, Classification and Interpretation
- Cooperating Autonomous Robots
- Multi Agent Systems
- Optimization, Heuristics and Search Methods
- Pattern Recognition and Classification
- Behavior based Control
- Self-organizing Systems
- Measuring Systems for Integration Purposes
- Heuristic Interpretation of test results
- Automated Interpretation of test results
- Modeling of Neuro-biological Autonomic Systems
- Mapping Design Specifications into Performance of Intelligent Systems
- Understanding Incomplete and Ambiguous Assignments
- Interpreting and Performing the Assignment Under Conditions of Reduced Technological Capabilities
- How Performance Depends on Knowledge Representation · Multiresolutional Ontology of Performance
- Linkage Between Multiple Sensor Modalities and Performance in Intelligent Systems
- Can Natural Language Communication with an Intelligent System Affect Performance?
- Development of SELF in Intelligent Systems
Sponsors
The National Institute of Standards and Technology
Defense Advanced Research Projects Agency
Institute of Electrical and Electronics Engineers Control Systems Society
National Aeronautics & Space Administration
In Cooperation with
The IEEE Neural Network Council
Call for Papers (pdf)