Online Improvement of Condition-based Maintenance Policy via Monte Carlo Tree Search
Michael Hoffman, Eunhye Song, Michael Brundage
Often in manufacturing systems scenarios arise where the demand for maintenance exceeds the capacity of maintenance resources. This leads to the problem of allocating the limited resources among machines competing for them. This maintenance scheduling problem can be formulated as a Markov Decision Process (MDP) with the goal of finding the optimal dynamic maintenance action given the current system state. However, as the system becomes more complex, solving an MPD suffers from the curse of dimensionality. To overcome this, we propose a two-stage approach that first optimizes a static condition-based maintenance (CBM) policy using the Gaussian Markov Improvement Algorithm (GMIA), then improves the policy online via Monte Carlo Tree Search (MCTS). The static policy significantly reduces the state space of the online problem by allowing us to ignore machines that are not sufficiently degraded. Further, we formulate MCTS to seek a maintenance schedule that maximizes the long-term production volume of the system to reconcile the conflict between maintenance and production objectives. We demonstrate that the resulting online policy is an improvement over the static CBM policy found by GMIA.
IEEE Transactions on Automation Science and Engineering
, Song, E.
and Brundage, M.
Online Improvement of Condition-based Maintenance Policy via Monte Carlo Tree Search, IEEE Transactions on Automation Science and Engineering, [online], https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=929886
(Accessed December 5, 2021)