Industrial Artificial Intelligence Management and Metrology (IAIMM)

Summary

Industrial Artificial Intelligence (IAI), or Artificial Intelligence applied to industry applications, is defined by its requirement to fulfill an explicit need of a system, while both utilizing and being bounded by the limitations and capabilities of that system. Performance and evaluations of an IAI have no meaning outside the context of its impact on a system and users.

The Industrial AI Management and Metrology (IAIMM) project develops and deploys measurement science to advance adoption and management of Artificial Intelligence (AI) systems in industrial environments to improve the productivity, resiliency, security, and the sustainability of manufacturing operations and supply chains. The educational barrier to entry of many IAI systems paired with a lack of standard evaluation tools and management methods has led to hesitation, mistrust, and misapplication of IAI systems in manufacturing enterprises. Industry requires trusted tools that can be used to readily assess the value of an IAI system within its application domain and enterprise. This is a particular issue to Subject Matter Experts (SMEs) who do not have resources necessary to independently develop these assessments.

Our project objective is to: 

1.  Facilitate trust-based adoption of Industrial Artificial Intelligence tools (IAI).
2.  Effectively capture and convey the value of using IAI tools and their impacts through intuitive, risk-aware metrics and procedures.
3.  Enable better, more effective use of IAI tools by developing deployment and evaluation best practices.
    

Description

Industrial AI uses Physics, Data Insights, and Human Observations + Intuition to Create Actionable Intelligence for Informed Decision Support 

Credit: NIST

IAIMM has identified IAI systems for decision making, planning, and control in manufacturing as a prime candidate for better Standard Operating Procedures (SOPs) centered on both use and evaluation. The specific use case of multi-stage manufacturing presents a broad scope of application to accentuate and explore various processes, policies, and pain points within the US manufacturing industry. Many of these challenges center around simulation, collection, and interchange of related connected and disparate data from both equipment and operators.

Exploring and enhancing IAI use in connected multi-stage process use cases can be a direct path to improve the productivity, resiliency, security, and sustainability of manufacturing operations and enterprises across the supply chain. Special interest will be placed upon IAI connected to smart manufacturing processes using Industrial Internet of Things (IIOT) for information collection and communication.

Three Major Efforts

• Sim PROCESD
  • Graph based discrete event simulator for multistage manufacturing product and equipment maintenance
• Collaborative Robotic Operations Work-cell for Process Monitoring
  • Physical multistage manufacturing setup for generating multifaceted operations and production data
• Human Generated Data Analytics
  • Development of methods for incorporation of technical language and human observations into analytical models

Purpose and Goals
This project seeks to provide standard best practice tools and procedures for evaluating, using, and setting expectations for IAI systems to foster appropriate levels of trust in IAI systems. This work entails exploration of data used by IAI, the methods of application, and the relevant domains and stakeholders affected by IAI use. We seek to lower barriers of understanding and interrogation such that users and decision makers of all levels can determine the potential risks and likely impacts of an IAI tool or solution.  Identified metrics and methods of evaluation will provide decision makers clear justification for investing in and maintaining IAI tools. Evaluations will focus on systems through enterprise-level impacts that can be intuitively translated to both floor-level operators and management decision makers. 

• Qualify and evaluate AI/ML tools and methods for industry
  • Risk-based testing methods to evaluate impact of AI-based monitoring tools
  • Domain-centric testing warranted by specialized domains of application
  • Intuitively informative metric generation
  • Showcasing of both business value and engineering benefit, especially of utilizing closed-box tools within risk management processes

As the manufacturing industry produces increasing volumes of diverse data, stakeholders need robust evaluation methods to assess the impacts of IAIs and their ability to realize value from this data. The diverse breadth of manufacturing process information includes but is not limited to: equipment data, design data, execution data, part quality data, systems interaction data, human generated feedback data, process performance data, and more. 

• Integrate and utilize disparate or heterogeneous industrial data sources into AI/ML tools 
  • Unstructured data processing and evaluation
  • Hard/soft sensor data fusion
  • Human-agent communication (human-in-the-loop learning)
  • Asynchronous data and tools
  • Data management, provenance, and utilization

Advanced manufacturing seeks to close the gap between human operators and IAIs that feed off the growing connectivity provided by IIoT technologies. Clear communication of IAI performance expectations as well as of internal logic and reasoning of these systems to human users is key for building appropriate levels of trust. The ability of systems to augment and enhance human performance is directly related to the system’s ability to communicate to the user. Additionally, the value that system provides must be effectively measured and communicated to decision makers. Towards this end, this research seeks identification and development of simulation environments and evaluation tools suitable to IAI and manufacturing specifically.

• Produce testbed environments for algorithm testing, data generation, and procedural development 
  • Open-source, discrete-event simulator of manufacturing operations & maintenance process, serving as a testbed for algorithm testing and demonstration of simulation usability
  • Benchtop physical setup for multi-stage part process and quality data collection
  • Partner with external collaborators for publicly available datasets

IAIMM will develop test methods, standards, toolkits, models, datasets, industry pilots, and build communities of interest to advance manufacturing management of IAI systems with a focus on data use, interpretability, interoperability, and system level impact. IAIMM outputs will lower the barrier to incorporate new technologies and analysis methods into existing operations. The outcomes of the IAIMM program will enable trusted, understandable, and reproducible analysis workflows across engineered products, manufacturing processes, production systems, enterprises, and supply chains to improve decision making.

• Pursue stakeholder interactions and information gathering
  • Host roundtable and panel discussions to facilitate assessment of needs and gaps
  • Disseminate testing software/procedures and solicit feedback from users
  • Publish findings and best practice recommendations in appropriate peer reviewed publications

Relevant Publications:

1. Sharp, Michael; Observations on developing reliability information utilization in a manufacturing environment with case study: robotic arm manipulators The International Journal of Advanced Manufacturing Technology 102 9 3243-3264 2019 Springer London
2. Weiss, Brian A; Sharp, Michael; Klinger, Alexander; Developing a hierarchical decomposition methodology to increase manufacturing process and equipment health awareness Journal of manufacturing systems 48 96-107 2018 Elsevier
3. Sharp, Michael; Weiss, Brian A; Hierarchical modeling of a manufacturing work cell to promote contextualized PHM information across multiple levels Manufacturing letters 15 46-49 2018 Elsevier
4. Hedberg Jr, Thomas D; Sharp, Michael E; Maw, Toby MM; Rahman, Mostafizur M; Jadhav, Swati; Whicker, James J; Feeney, Allison Barnard; Helu, Moneer; Design, manufacturing, and inspection data for a three-component assembly Journal of Research of the National Institute of Standards and Technology 124 1 2019 National Institute of Standards and Technology
5. Sharp, Michael; Brundage, Michael P; Sprock, Timothy; Weiss, Brian A; Selecting Optimal Data for Creating Informed Maintenance Decisions in a Manufacturing Environment Model-Based Enterprise Summit 2019   2019 
6. Sharp, Michael; Sexton, Thurston; Brundage, Michael P; Toward semi-autonomous information: Extraction for Unstructured Maintenance Data in Root Cause Analysis IFIP International Conference on Advances in Production Management Systems   425-432 2017 Springer, Cham
7. Sharp, Michael; Ak, Ronay; Hedberg Jr, Thomas; A survey of the advancing use and development of machine learning in smart manufacturing Journal of manufacturing systems 48 170-179 2018 Elsevier
8. Navinchandran, Madhusudanan; Sharp, Michael E; Brundage, Michael P; Sexton, Thurston B; Studies to predict maintenance time duration and important factors from maintenance workorder data Proceedings of the Annual Conference of the PHM Society 11   2019 
9. Sprock, Timothy; Sharp, Michael; Bernstein, William Z; Brundage, Michael P; Helu, Moneer; Hedberg, Thomas; Integrated Operations management for distributed manufacturing IFAC-PapersOnLine 52 13 1820-1824 2019 Elsevier
10. S Nagahara, TA Sprock, MM Helu. "Toward data-driven production simulation modeling: dispatching rule identification by machine learning techniques", Procedia CIRP 81 (2019), 222-227
11. Dadfarnia, Mehdi; Sharp, Michael; Sprock, Timothy; Understanding and Evaluating Naive Diagnostics Algorithms Applicable in Multistage Manufacturing from a Risk Management Perspective International Manufacturing Science and Engineering Conference 84263 V002T07A042 2020 American Society of Mechanical Engineers
12. Sprock, Timothy; Brundage, Michael P; Bernstein, William Z; Sexton, Thurston; Sharp, Michael; Mitigating Disruption in Production Networks through Dynamic Scheduling Enabled by Integrated Enterprise Data     2020 ASTM International
13. M Helu, T Sprock, D Hartenstine, R Venketesh, W Sobel. "Scalable data pipeline architecture to support the industrial internet of things", CIRP Annals 69 (2020), 385-388
14. Sharp, ME; Hedberg Jr, TD; Bernstein, WZ; Kwon, S; Feasibility study for an automated engineering change process International Journal of Production Research 59 16 4995-5010 2021 Taylor & Francis
15. Navinchandran, Madhusudanan; Sharp, Michael E; Brundage, Michael P; Sexton, Thurston B; Discovering critical KPI factors from natural language in maintenance work orders Journal of Intelligent Manufacturing   19-Jan 2021 Springer US
16. Hedberg Jr, Thomas D; Sharp, ME; Maw, TMM; Helu, MM; Rahman, Mostafizur M; Jadhav, Swati; Whicker, James J; Barnard Feeney, A; Defining requirements for integrating information between design, manufacturing, and inspection International Journal of Production Research   21-Jan 2021 Taylor & Francis
17. Brundage, Michael P; Sharp, Michael; Pavel, Radu; Qualifying Evaluations from Human Operators: Integrating Sensor Data with Natural Language Logs PHM Society European Conference 6 1 9-Sep 2021 
18. Sharp, Michael; Dadfarnia, Mehdi; Sprock, Timothy; Thomas, Douglas; Procedural Guide for System-Level Impact Evaluation of Industrial Artificial Intelligence-Driven Technologies: Application to Risk-Based Investment Analysis for Condition Monitoring Systems in Manufacturing Journal of Manufacturing Science and Engineering 144 7 2022 American Society of Mechanical Engineers Digital Collection