Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Procedural Guide for System-Level Impact Evaluation of Industrial Artificial Intelligence-Driven Technologies: Application to Risk-Based Investment Analysis for Condition Monitoring Systems in Manufacturing

Published

Author(s)

Michael Sharp, Mehdi Dadfarnia, Timothy A. Sprock, Douglas Thomas

Abstract

Industrial artificial intelligence (IAI) and other analysis tools with obfuscated internal processes are growing in capability and ubiquity within industrial settings. Decision-makers share their concern regarding the objective evaluation of such tools and their impacts at the system level, facility level, and beyond. One application where this style of tool is making a significant impact is in Condition Monitoring Systems (CMSs). This paper addresses the need to evaluate CMSs, a collection of software and devices that alert users to changing conditions within assets or systems of a facility. The presented evaluation procedure uses CMSs as a case study for a broader philosophy evaluating the impacts of IAI tools. CMSs can provide value to a system by forewarning faults, defects, or other unwanted events. However, evaluating CMS value through scenarios that did not occur is rarely easy or intuitive. Further complicating this evaluation are the ongoing investment costs and risks posed by the CMS from imperfect monitoring. To overcome this, an industrial facility needs to regularly and objectively review CMS impacts to justify investments and maintain competitive advantage. This paper's procedure assesses the suitability of a CMS for a system in terms of risk and investment analysis. This risk-based approach uses the changes in the likelihood of good and bad events to quantify CMS value without making any one-time pointwise estimates. Fictional case studies presented in this paper illustrate the procedure and demonstrate its usefulness and validity.
Citation
ASME Journal of Manufacturing Science and Engineering
Volume
144
Issue
7

Keywords

manufacturing simulation, evaluating algorithms, cost-benefit analysis, industrial AI tools, condition monitoring systems, inspection and quality control, modeling and simulation, monitoring and diagnostics

Citation

Sharp, M. , Dadfarnia, M. , Sprock, T. and Thomas, D. (2021), Procedural Guide for System-Level Impact Evaluation of Industrial Artificial Intelligence-Driven Technologies: Application to Risk-Based Investment Analysis for Condition Monitoring Systems in Manufacturing, ASME Journal of Manufacturing Science and Engineering, [online], https://doi.org/10.1115/1.4053155 (Accessed November 30, 2024)

Issues

If you have any questions about this publication or are having problems accessing it, please contact reflib@nist.gov.

Created December 16, 2021, Updated October 20, 2022