NIST logo
*
Bookmark and Share

Performance Assessment Framework for Robotic Systems

Summary:

Most assembly tasks are not readily automatable today due to robots’ limited perception, mobility, and dexterity.1,2 Furthermore, there are no means of creating assembly-centric performance models for use in work cell engineering in order to ensure successful implementations. Characterizing and accurately predicting robot system performance is technically hard due to the complex unmodeled interactions amongst the perception, mobility, dexterity, and safety components of a robot system and the broad range of assembly operations. Currently, the small subset of assembly tasks that have been automated are custom implementations requiring significant fixturing and tooling, resulting in high investment and limited process flexibility.3 The new technical idea is to compose assembly task-driven measures of component performance (perception, mobility, dexterity, and safety) into robot system autonomous assembly performance models. NIST will deliver a suite of test methods for perception, mobility, dexterity, and safety components of robots derived from an assembly operation taxonomy and requirements, along with a methodology and tools for composing the resulting measurements into robot system performance models. The assembly-relevant performance measures will provide manufacturers with the currently-missing data and tools for assessing and assuring implementations of robot systems within their smart manufacturing applications, thereby reducing the risk of adopting this key disruptive technology. Small and medium enterprises, as well as large manufacturers, will be able to use robot systems that enable them to increase the variety of products and shorten their product cycles to meet customer demands. NIST will blend its proven process for creating application-driven performance requirements and measurements4 with its expertise in measurement science for manufacturing robotics and autonomy metrics to address the challenges of robot systems for autonomous assembly tasks.

Description:

Objective:

Enable manufacturers to assess and assure robot system assembly task performance by delivering a methodology and tools for characterizing and composing performance of perception, mobility, dexterity, and safety components by 2018.

What is the new technical idea?

The new technical idea is to compose assembly task-driven measures of component performance (perception, mobility, dexterity, and safety) into robot system autonomous assembly performance models. A decomposition of typical assembly operations into the constituent robotic subsystem capabilities will drive the research plans for the perception, mobility, dexterity, and safety thrusts. A representational framework for modeling performance within each component area will enable analyses and studies to increase understanding of performance of the overall system within industrial settings. NIST will deliver a suite of test methods for perception, mobility, dexterity, and safety components of robots derived from an assembly operation taxonomy and requirements, along with a methodology and tools for composing the resulting measurements into robot system performance models.

What is the research plan?


The research plan entails development of a performance modeling and analysis framework that utilizes the results from parallel efforts to define key component metrics and test methods for robotic assembly operations. Central to the research plan is a testbed that will support experiments in advanced perception, mobility, dexterity, and safety for robotic assembly. The testbed will be designed to support increasing levels of experimentation complexity in the assembly domain, starting with picking and placing parts for kit-building in static environments and ending, after 5 years, with fastening and joining operations in dynamic unstructured environments.

In addition to contributing to the modeling of performance in their component areas for use in the compositionality framework, outputs from the four component thrusts (perception, mobility, manipulation, and safety) will include reports and journal publications documenting metrics and test methods as well as artifacts (designs or physical replicas) for use by stakeholders.

The first year will produce a robot systems capability model detailing a taxonomy of assembly tasks, decomposed into sub-tasks to define performance metrics and measurement methods for the perception, mobility, dexterity, and safety components of the robot system. This capability model will guide and focus the future development of robot systems for use in assembly-centric manufacturing. Test method suites of increasing complexity will be developed in the subsequent three years:
  • In the second year, NIST will develop a test method suite that measures robot performance in identifying, localizing, and tracking parts and other entities in its surroundings and purposefully grasping parts. This test method suite will provide manufacturers and robot developers with the means of assessing and assuring robot performance in assembly tasks using general-purpose grippers and hands.
  • The third year will focus on producing a test method suite that measures robot performance in joining and fastening operations, including use of assembly tools. This test method suite will provide manufacturers and robot developers the ability to assess and assure a robot systems’ performance in assembly operations using tools.
  • In the fourth year, NIST will build a test method suite that measures the performance of mobile robots and mobile manipulators in safely navigating unstructured and dynamic environments. Through this test suite, NIST will provide industry with the means of assessing and assuring the performance and safety of mobile robots and manipulators in support of assembly-centric operations.
At the end of the project lifetime, the results of the first four years will be consolidated into a general methodology and tool set for composing perception, mobility, dexterity, and safety performance into robot system autonomous assembly performance models. This will enable manufacturers to assess and assure robot system task performance in smart manufacturing assembly-centric applications.

1 “A Roadmap for US Robotics: From Internet to Robotics (V2),” CCC and CRA (NSF-funded), 2013, http://www.us-robotics.us/
2 M. Hedelind, S. Kock, “Requirements on Flexible Robot Systems for Small Parts Assembly – A Case Study”, in proceedings of International Symposium on Assembly and Manufacturing, 25-27 May, Tampere, Finland, 2011.
3 R. Menassa, “Robonaut2 and Next-Generation Industrial Robots” presentation at the 2011 RoboBusiness Leadership Summit.
4 Nelson, K. and Enright, C., “Robots to the Rescue,” ASTM Standardization News, May/June 2013.

Performance Assessment Framework for Robotics Systems

Start Date:

October 1, 2013

Lead Organizational Unit:

el
Contact
Elena Messina, Project Leader

301 975 3510 Telephone
301 990 9688 Fax  

100 Bureau Drive, M/S 8230
Gaithersburg, MD 20899