Predictive Maintenance Decision Making Based on Reinforcement Learning in Multistage Production Systems

Abstract

Predictive maintenance has become increasingly prevalent in modern production systems that are challenged by high-mix low-volume production and short production life cycle. It is very helpful to prevent costly equipment failures, and reduce significant production loss caused by unscheduled machine breakdown. Although important, decision models for joint predictive maintenance and production in manufacturing systems have not been fully explored. Therefore, we propose a reinforcement learning based decision model, that brings together production system modeling and approximate dynamic programming. We start from the development of a state-based model by analyzing the dynamics of a multistage production system with predictive maintenance. It provides an approach to quantitatively evaluate the various disruptions as well as the maintenance decision’s impact on production. Then a reinforcement learning method is proposed to explore optimal maintenance policies, that optimize the production and maintenance cost. To further improve the performance of the production system, machine stoppage bottlenecks are defined. An event-based indicator is proved to identify bottlenecks with production data. We test the proposed models in simulation case studies. The proposed predictive maintenance decision model is compared with three policies, which are state-based policy (SBP), time-based policy (TBP) and greedy policy (GP). The numerical studies show that the proposed decision model outperforms the policies, and it has the lowest system cost that is 9.68%, 39.07%, and 39.56% lower than SBP, TBP, and GP, respectively. In addition, the research shows that bottleneck identification and mitigation could help manufacturing systems to achieve more than 9.00% throughput improvement.