Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Online Improvement of Condition-based Maintenance Policy via Monte Carlo Tree Search



Michael Hoffman, Eunhye Song, Michael Brundage


Often in manufacturing systems scenarios arise where the demand for maintenance exceeds the capacity of maintenance resources. This leads to the problem of allocating the limited resources among machines competing for them. This maintenance scheduling problem can be formulated as a Markov Decision Process (MDP) with the goal of finding the optimal dynamic maintenance action given the current system state. However, as the system becomes more complex, solving an MPD suffers from the curse of dimensionality. To overcome this, we propose a two-stage approach that first optimizes a static condition-based maintenance (CBM) policy using the Gaussian Markov Improvement Algorithm (GMIA), then improves the policy online via Monte Carlo Tree Search (MCTS). The static policy significantly reduces the state space of the online problem by allowing us to ignore machines that are not sufficiently degraded. Further, we formulate MCTS to seek a maintenance schedule that maximizes the long-term production volume of the system to reconcile the conflict between maintenance and production objectives. We demonstrate that the resulting online policy is an improvement over the static CBM policy found by GMIA.
IEEE Transactions on Automation Science and Engineering


Condition-based maintenance, Gaussian Markov Improvement Algorithm, Monte Carlo Tree Search


Hoffman, M. , Song, E. and Brundage, M. (2021), Online Improvement of Condition-based Maintenance Policy via Monte Carlo Tree Search, IEEE Transactions on Automation Science and Engineering, [online], (Accessed May 21, 2024)


If you have any questions about this publication or are having problems accessing it, please contact

Created June 30, 2021, Updated October 14, 2021