Performance of Human-Robot Interaction

Summary

In terms of collaborative robotics, the widespread adoption of robots in historically manual manufacturing environments (which are subject to high product turnover, short production runs, and high variability in equipment configurations) is limited by the robots’ inability to effectively and safely integrate and interact with the existing human labor.  Instead, so-called collaborative robots are relegated to secluded operations with minimal contact with the workforce.  The robots’ inability to communicate with, understand the intention of, and establish a mutual understanding of the environment and situation with human coworkers decreases the robots’ usefulness in collaborative teams consisting of both robots and people.  This limitation is driven by both the absence of tools and protocols needed for effectively describing and measuring human-robot interactions, an incomplete collection of metrics for assessing human-robot teaming performance, and insufficient protocols for enabling more intuitive interfacing with robotic tools.  The Performance of Human-Robot Interaction project will expand on the work toward collaborative robot performance and safety established in the previous Performance of Collaborative Robot Systems project (FY14-FY18), and will provide the methods, protocols, and metrics necessary to evaluate the interactive and teaming capabilities of robot systems, and will use a task-driven decomposition of manufacturing processes to assess and assure the safety and effectiveness of human-robot collaborative teams.  This collection of methods, protocols, and metrics will enable integrators and end-users to maximize the effectiveness and efficiency of collaborative human-robot teams in production processes, impacting both large-scale companies designing and repurposing hybrid manufacturing workflows, and smaller companies looking to begin introducing automated tools into manual processes.

Description

To deliver a suite of test methods, protocols, and information models to enable effective, human-robot collaboration in manufacturing, and advance interactive robot technologies to facilitate the safe and efficient teaming of people and robots that maximally leverages the strengths and capabilities of each toward meeting production goals.

What is the Technical Idea?

To achieve the specified objective, the Performance of Human-Robot Interaction project will focus on developing a collective metrology suite consisting of test methods, metrics, systems models, software libraries, and algorithms to evaluate the robots’ capabilities that enable the successful integration into human-robot teams, and their contribution toward collaborative manufacturing goals.  This collective metrology suite will enable technology developers, integrators, and end users of collaborative robot technologies to:

• Integrate, evaluate, and optimize sensors and algorithms for process, object, and intent recognition to support both situation awareness and collaborative operation safety.  We will develop test methods and metrics, labeled data sets, and virtual models that can be used to assess and assure safe and effective robot system behavior in supportive collaborative operations.
• Provide intuitive plug-and-play capabilities of robots into collaborative teams with human operators, enabling faster integration and acceptance in the manufacturing process.  We will develop prototype protocols for robot systems that enable intuitive interaction, control, and programming for human-robot collaborative tasks.
• Assess and optimize the actions of robot systems and presentation of robot feedback to encourage trust and establishing common ground with human operators in collaborative tasks.  We will develop guidelines and documented best practices for the design of physical appearances, behaviors, and feedback mechanisms that encourage ease-of-use and integration, and minimize robot intent ambiguity.
• Develop new behaviors and capabilities of interactive robot systems to support new operator training, team adaptability to process change and uncertainty, and responsiveness and utility in high-impact situations.  We will develop new protocols and test methods for the evaluation of human preferences and experience, as well as mechanisms for evaluating the effectiveness of information sharing protocols for efficient communication.
• Employ robots capable of automatically and safely adapting to operator preferences, experience, attention, and actions relevant to the collaborative task.  We will develop new test methods and metrics for the evaluation of human-aware collaborative robots, including metrology for the evaluation of operator intent and motions as they pertain to safe and effective human-robot collaboration.
• Evaluate the effectiveness of human-robot teaming as it pertains to the quality of the work being performed by the combined effort to optimally direct and deploy collaborative robot systems in the manufacturing process.  We will expand upon previous work completed on the development of metrics for effective human-machine interaction and interfaces, and develop new test methods for the real-time assessment of human-robot teaming efficiency and effectiveness.
• Share and receive instructions, status updates, and diagnostic data using methods intuitive to both sides of the collaborative human-robot team using this project’s resulting models, metrics and software libraries.  We will develop new test methods and software tools for assessing communication efficacy as a function of information timing, quality, and format, which can be used to optimize the presentation of task-relevant information to human operators.
• Ensure that robot systems are able to accurately and reliably identify and locate people in the workspace, and distinguish them from other robots, tools, and objects.  The ability to distinguish between people and objects within the work environment is outside of the realm of current technology, so we will work to develop new test methods and metrics for the evaluation of sensor systems that are capable of classifying objects within the work volume as human or not.
• Assess and assure the efficiency and effectiveness of employing collaborative robots in flexible factory environments such that the costs and benefits of integrating robots into collaborative teams are optimally balanced in favor of the end user.  We will develop new metrics and test methods for the evaluation of the impacts of employing collaborative robots in manufacturing environments (including the ease of integration, the cost of programming, impacts on team process performance, and the impacts on risk assessments and safety protocols).

What is the Research Plan?

This research plan focuses on five principal capabilities of robot systems that collectively contribute to human-robot integration and teaming:  1) modeling complex, human-occupied spaces to maintain awareness of operator attention and safety; 2) establishing trust in human-robot teams by means of the robot’s design and capabilities; 3) adapting to uncertainty and external cues while working in collaborative human-robot teams; 4) human-specific sensing and modeling; and 5) assessing the quality and impacts of physical and cognitive interactions between peoples and robot systems.  Each of these focal areas builds upon the capabilities defined or developed in its predecessor.  Collectively, they will comprise a total suite of test methods, metrics, and protocols to assess the human-robot interaction performance of collaborative robot systems.  For each phase of development, the test methods, metrics, and protocols will be evaluated using the NIST collaborative robotics testbed.

• World modeling:  We will deliver test methods and metrics to evaluate models of sensors and systems that support both situation awareness in human-robot interaction and collaborative operation safety.  These test methods and metrics will be developed through a focused engineering process, including the development of prototype models in NIST laboratories.  We will focus on intent prediction and state representations of both robots and humans in the environment as they pertain to collaborative manufacturing tasks, and will present metrology that measures the accuracy of these models as compared with known ground truths.
• Establishing trust:  We will deliver protocols and test methods for assessing the design and use of robot appearance and behavior in collaborative human-robot teaming.  These test methods will quantify the impacts of the robot’s appearance and behavior on human-robot relations and interactions, and assess the acceptance and trust of robotic collaborators in manufacturing operations.  This investigation will drive future efforts in metrology efforts for teaming-related artificial intelligence (AI) implementations, behavioral models, and artifact designs.  Guidelines for the use of the robot’s outward appearance (i.e., how it presents itself, and how its physical construction impacts operator acceptance), actions, and presence toward effective teaming will also be developed based on literature reviews, laboratory experiments, and interactions with relevant human-robot interaction communities.
• Adaptation to uncertainty:  Within the realm of human-robot interaction, there exists a great deal of uncertainty with regards to human location, intent, and activity.  There also exists uncertainty in terms of teaming performance, process quality and progression, world state, and safety.  We will deliver protocols and AI tools for adaptive teaming of human-robot and robot-robot pairings in manufacturing operations.  These products will be targeted toward enabling ease-of-integration and ease-of-use of robot systems in collaborative teams by addressing human-intent and activities as they pertain to teaming operations.  Test methods and metrics will be produced to assess the performance impacts such tools will have on the progress and quality of work done by these integrated teams.  The protocols and tools will be developed based on an assessment of the needs and capabilities of robots and humans in the collaborative teams for the successful  completion of the collaborative task, and an associated taxonomy will be developed to capture and encapsulate this information.  Metrics for assessing ease of integration and use of adaptive robot systems in the collaborative teams will also be developed.  
• Human-specific sensing:  Current safety-rated sensors used in manufacturing environments are incapable of distinguishing between people and work objects, and must infer which is which based on operational context.  This necessitates the meticulous engineering of work cells to prevent operator access.  However, injuries (sometimes fatal) still occur, and as a result emerging collaborative robot technologies are incapable of effectively working with human collaborators because they cannot determine what is a work piece, and what is the human collaborator.  We will deliver test methods, metrics, and prototype systems of human-like artifacts for the assessment of human-specific sensing, safety, and interaction.  Validated, biomimetic artifacts, being necessary for the safe verification and validation of human-robot interaction in manufacturing environments, do not exist outside of laboratory prototypes.  We will produce a compendium of human-specific sensing systems, and develop recommendations and specifications for the generation of artifacts for the evaluation of human recognition, identification, localization, and tracking.  Prototype test methods and artifacts will be developed to assess the performance of example sensor systems and validate the artifact guidelines.
• Interaction impacts:  We will deliver test methods that measure the quality of physical and cognitive interactions between people and robot systems, and the impacts that human-machine interfaces have on the types, quantity, effects, and safety of these interactions.  Combined with the previous four capabilities, these test methods will provide a comprehensive understanding of the effects that human-robot interactions have on the collaborative team, the individual contributors, and the collaborative task.  Metrology for evaluating, verifying, and validating the impacts of human-robot teaming will be produced, and will be used to influence the continued development and support of robot performance and safety standards.