Industries may use "condition monitoring systems" (CMS) to monitor machines and enterprises – like production lines – for detrimental or unexpected changes in order to mitigate any faults, failures, and other costly scenarios. However, users are often hesitant to adopt these systems because it is unclear what their return on investment would be. A CMS’s value is derived from preventing unwanted events and calculating costs of those prevented events is difficult. Condition monitoring systems that use artificial intelligence are even harder to evaluate, because when intellectual property concerns prevent disclosure of their internal logic, operators fear they may behave in unexpected ways.
To help evaluate condition monitoring systems for their ability to prevent risks, NIST researchers published Key Elements to Contextualize AI-Driven Condition Monitoring Systems towards Their Risk-Based Evaluation. The paper was recently presented at the 2022 5th IEEE International Artificial Intelligence for Industries Conference.
The paper helps potential users determine the differences in risks with and without condition monitoring systems. The differences help indicate benefit to the industrial system directly from adding the risk-mitigating capabilities of the CMS. To this end, the paper provides:
The paper then breaks down the key elements that help determine how the condition monitoring system is designed and operated, and which can be used in its evaluation:
The paper shows how these key elements can be used to construct an evaluation process that captures the impact of condition monitoring systems.