Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Performance Assessment Framework for Robotic Systems

Summary

Most manufacturing assembly tasks are not readily automatable today due to robots’ limited perception, mobility, and dexterity. The challenge of engineering and implementing successful robotic assembly work cells is exacerbated by the lack of tools for creating assembly-centric performance models encompassing an integrated system. Characterizing and accurately predicting robot system performance is technically hard due to the complex unmodeled interactions amongst the perception, mobility, and dexterity components of a robot system and the broad range of assembly operations. Currently, the successfully-automated assembly tasks are limited implementations that require significant fixturing and tooling, resulting in high investment and limited process flexibility. To advance the adoption of robots for assembly requires overcoming many technical hurdles, including assurance of new safety systems that enable humans and robots to work together. The new technical idea is to compose assembly task-driven measures of component performance (perception, mobility, dexterity, and safety) into robot system autonomous assembly performance models. For this project, NIST will deliver a suite of test methods for perception, mobility, dexterity, and safety components of robots derived from an assembly operation taxonomy and requirements, along with a methodology and tools for composing the resulting measurements into robot system performance models. This will provide manufacturers with the currently-missing data and tools for assessing and assuring implementations of robot systems within their smart manufacturing applications, thereby reducing the risk of adopting this key disruptive technology. Small and medium enterprises, as well as large manufacturers, will be able to use robot systems that enable them to increase the variety of products and shorten their product cycles to meet customer demands.

Description

Performance Assessment Framework for Robotics Systems

Objective:  Enable manufacturers to assess and assure robot system assembly task performance by delivering a methodology and tools for characterizing and composing performance of perception, mobility, dexterity, and safety components.

Technical Idea:  The new technical idea is to compose assembly task-driven measures of component performance (perception, mobility, dexterity, and safety) into robot system autonomous assembly performance models. Performance measures will provide manufacturers and systems integrators with data for selecting the appropriate system components. The plan is to create:

  • Framework: A decomposition of typical assembly operations into the constituent robotic subsystem capabilities that will drive the research plans for the perception, mobility, dexterity, and safety thrusts.
  • Capability Models: A representational framework for modeling performance within each component area that will enable analyses and studies to increase understanding of performance of the overall system within industrial settings.
  • Test Methods: A suite of test methods for perception, mobility, dexterity, and safety components of robots derived from an assembly operation taxonomy and requirements, along with a methodology and tools for composing the resulting measurements into robot system performance models.

Research Plan:  

  • Framework: The research plan entails development of a performance modeling and analysis framework that utilizes the results from parallel efforts to define key component metrics and test methods for robotic assembly operations. Central to the research plan is a testbed integrating robot arms, mobile robot bases, hands and grippers, and a range of sensor systems. The reconfigurable testbed will support experiments in advanced perception, mobility, dexterity, and safety for robotic assembly. The testbed will be designed to support increasing levels of experimentation complexity in the assembly domain, starting with picking and placing parts for kit-building in static environments and ending, after 5 years, with fastening and joining operations in dynamic unstructured environments.
  • Capability Models: The project will produce a robot systems capability model detailing a taxonomy of assembly tasks, decomposed into sub-tasks to define performance metrics and measurement methods for the perception, mobility, dexterity, and safety components of the robot system. This capability model will guide and focus the development of robot systems for use in assembly-centric manufacturing. Test method suites of increasing complexity will be developed:
    • measuring robot performance in identifying, localizing, and tracking parts and other entities in its surroundings and purposefully grasping parts. This test method suite will provide manufacturers and robot developers with the means of assessing and assuring robot performance in assembly tasks using general-purpose grippers and hands.
    • measuring robot performance in joining and fastening operations, including use of assembly tools. This test method suite will provide manufacturers and robot developers the ability to assess and assure a robot systems’ performance in assembly operations using tools.
    • measuring the performance of mobile robots and mobile manipulators in safely navigating unstructured and dynamic environments. Through this test suite, NIST will provide industry with the means of assessing and assuring the performance and safety of mobile robots and manipulators in support of assembly-centric operations.
  • Test Methods: At the end of the project lifetime, the results will be consolidated into a general methodology and tool set for composing perception, mobility, dexterity, and safety performance into robot system autonomous assembly performance models. This will enable manufacturers to assess and assure robot system task performance in smart manufacturing assembly-centric applications.
Created March 21, 2014, Updated December 17, 2018