Knowledge Driven Planning and Modeling

This project will aid in the development of measurement science and standards to enable advancements in planning by robots in scenarios relevant to manufacturing, starting with kitting and aiming towards assembly. Plans enable a robot to change its actions to deal with uncertainty in its environment and to rapidly switch to new tasks. Plans are based on models of the current environment, predictions about the future, and a priori knowledge of causal relationships between current actions and results. The breadth and usability of knowledge in these models is one of the main factors that constrains the flexibility and performance of manufacturing planning systems. Currently, there is no accepted standardized way to represent this knowledge, to reason with this knowledge, or to measure the performance of these systems.

Objective:

To develop the measurement science and standards for planning and modeling by robots so that they are able to be more quickly re-tasked and are more flexible and adaptive by the end of FY2014.

What is the new technical idea?

The new technical idea is to aid in the development of standard data abstractions for maintaining and sharing knowledge of the world and planning actions in it. Performance measurement techniques and artifacts will be developed to enable manufacturers to use these standards and abstractions in a cost-effective manner. The lack of sensor processing to detect objects in the world and details about environmental conditions as well as an abstraction that is capable of storing and reasoning over this knowledge has led current manufacturing robots to have little or no understanding of the world around them and no capability to dynamically change their actions if the environment changes. This restricts them to operate in highly constrained environments and makes it difficult to change from one task to another. The Robot Perception for Identifying and Locating Parts for Assembly and Robot Perception for Workspace Situational Awareness projects are addressing the sensor processing aspects of this problem. This project is addressing the knowledge abstraction aspects of the problem. The primary challenges that may be solved by a unified knowledge abstraction are:

• Lack of agility that would allow robots to be quickly and easily re-tasked. With current teach pendant programming of robots, it may take an order of magnitude longer to program and configure the system than the task would take to complete by hand.
• Lack of adaptability that would allow robots to cope with part and environmental variations. Many subcomponents have component-by-component or lot-by-lot variations. Today’s high-precision robotic systems are not able to adapt automatically to such variations.
• Lack of flexibility in planning and perception systems that would allow them to cope with unexpected events or failures. A standard framework for representing knowledge would enable interoperation of planning and perception thus promoting knowledge reuse and flexible operation.
• Lack of performance measures to determine a system’s ability to be flexible, adaptable, and agile. Here a better understanding of what enables agility and how to measure it are needed.
  
• Lack of a standard framework for representing knowledge necessary to allow flexibility in robotic systems.

Initially, an open-source simulation engine is being used to simulate plan execution in content rich worlds. This will be augmented to include semantic labels and virtual sensors capable of detecting and reporting relevant environmental features. Repeatable scenarios will allow for the comparison of planning systems that are capable of exploiting this rich content. In FY2014, work in the project will migrate to real robotic hardware.

What is the research plan?

While knowledge abstractions are needed across manufacturing, a program-wide decision was made to start addressing the problem by focusing on assembly, and in particular on the bin-picking problem. This project will focus on a generalization of the bin-picking problem known as kitting. The use of kitting allows us to model bin-picking from multiple bins, as well as complex planning and optimization needed for the construction of kits. The techniques developed here will be directly applicable to the general assembly problem. The work focuses on three key areas:

Knowledge Representation: Perform research into knowledge abstractions that will aid in the development of standard representations for world knowledge and plan knowledge, and the related performance evaluation criteria. We will design a comprehensive model that is able to represent knowledge for the general class of manufacturing problems in the area of rapid re-tasking for general assembly tasks. We will work in cooperation with the IEEE Standards Working Group on Knowledge Representation for Robotics and Automation to create a standard for knowledge representation for robotic applications and will provide a test implementation of the proposed standard to validate its usability. We will also derive methods of defining the performance requirements for the knowledge and will develop performance methods and metrics.

Planning: Develop methods to compare planning algorithms that utilize the representations developed above to address next generation robotics for the class of manufacturing problems in the area of adaptable and reconfigurable assembly. Under this task, we will examine input/output standards and performance measures for planning systems.

Simulation: Develop a simulated manufacturing test method that is capable of verifying the performance of rapid re-tasking using our performance measures. This test method will validate the utility of our interfaces and performance measures. In order to populate the semantic world representation, the existing simulation system (called USARSim) will be augmented and expanded to include new virtual sensing and representation capabilities. The simulation system will provide representations that are not yet possible to obtain from the real sensing systems.  It therefore provides a high-fidelity but more advanced testbed for developing performance requirements and validating test methods.

Recent Results:
Outputs:

1. A workshop at the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) established our relationship with IEEE Robotics and Automation Systems Standards Committee Ontologies for Robotics and Automation Study Group (now a working group) and updated the group on our work in representations.
2. Created and circulated a planning document in the USARSim community that outlines an implementation strategy for developing the ability to include high-level semantic knowledge in the simulation. Making high-level semantic knowledge available allows for the simulation of yet-to-be-developed sensor processing systems that are capable of reliably reporting an object’s type and pose. The simulated sensor outputs may be intentionally corrupted with noise to resemble expected sensor processing output.
3. Developed the integrated system outlined above that allows virtual sensing of high-level semantic knowledge to be extracted from the simulation and delivered it to the USARSim community for comment.
4. Developed a simulation-based planning system using leading techniques from the domain independent planning community for creating static plans for adaptable and reconfigurable assembly. Due to the standard planning representation that we have adopted, several different planning approaches can be compared and evaluated. In order to allow other members of the IEEE Standards Working Group to easily experiment with our data abstraction, we created a tool set that allows planning and sensor processing systems to easily query and utilize the information stored in the world model.

Outcomes:

1. Project Authorization Request (PAR) accepted by IEEE Robotics and Automation Systems Standards Committee, resulting in the formation of the Ontologies for Robotics and Automation Working Group.
2. Held an industry workshop where representatives from the kitting, packaging, and palletizing industries came together to discuss common problems in component placement. As a result of this workshop, a NIST IR was produced, a mailing list that allows interested parties to stay in touch was created, and an industrial working group to advise the project was formed. This occurred at the MODEX show on 6-12 Feb in Atlanta, GA.
3. A draft representation for semantic world knowledge that is capable of supporting a priori and static knowledge gathered from sensors and databases was delivered to the IEEE Standards Working Group. This knowledge supports next generation robotics for manufacturing problems in the area of component placement.

Standards and Codes:

We are working with the NIST-led IEEE Working Group on Knowledge Representation for Robotics and Automation. As part of this project’s efforts, we are working with industry to produce new standards and performance measures for Knowledge Representation.