Online Improvement of Condition-based Maintenance Policy via Monte Carlo Tree Search

Published

June 30, 2021

Author(s)

Michael Hoffman, Eunhye Song, Michael Brundage

Abstract

Often in manufacturing systems scenarios arise where the demand for maintenance exceeds the capacity of maintenance resources. This leads to the problem of allocating the limited resources among machines competing for them. This maintenance scheduling problem can be formulated as a Markov Decision Process (MDP) with the goal of finding the optimal dynamic maintenance action given the current system state. However, as the system becomes more complex, solving an MPD suffers from the curse of dimensionality. To overcome this, we propose a two-stage approach that first optimizes a static condition-based maintenance (CBM) policy using the Gaussian Markov Improvement Algorithm (GMIA), then improves the policy online via Monte Carlo Tree Search (MCTS). The static policy significantly reduces the state space of the online problem by allowing us to ignore machines that are not sufficiently degraded. Further, we formulate MCTS to seek a maintenance schedule that maximizes the long-term production volume of the system to reconcile the conflict between maintenance and production objectives. We demonstrate that the resulting online policy is an improvement over the static CBM policy found by GMIA.

Citation

IEEE Transactions on Automation Science and Engineering

Pub Type

Journals

Download Paper

Local Download

Keywords

Condition-based maintenance, Gaussian Markov Improvement Algorithm, Monte Carlo Tree Search

Citation

Hoffman, M. , Song, E. and Brundage, M. (2021), Online Improvement of Condition-based Maintenance Policy via Monte Carlo Tree Search, IEEE Transactions on Automation Science and Engineering, [online], https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=929886 (Accessed October 9, 2025)