Dynamic Maintenance for a Large Scale Identical Parallel Manufacturing Systems Using Reinforcement Learning

Abstract

In this paper, we propose a Machin Learning (ML)-based framework for maintenance decision making for multi-unit system. More specifically, we propose Reinforcement Learning (RL) approach for dynamic maintenance model for multi- component parallel system subject to stochastic degradation and random failures. Deterioration of each unit occurs independently according to a three-state homogenous Markov process such that each unit has three states, namely, healthy, unhealthy, and failure state. The interaction among system states are modeled based on Birth/Birth-Death process. The overall system state is defined based on different combination of individual component state. The optimal maintenance policy for the system is obtained by modeling the problem as Markov Decision Process (MDP) and Q-learning algorithm with focus on cost minimization is applied as a solution methodology. In comparison to tradition MDP approaches, proposed RL solution is more effective and practical in terms of time and cost savings. Specifically, when the state-space of the problem is large, traditional MDP is note capable to converge to the optimal policy in a timely fashion. Therefore, there is a is the decisive need for development of RL-based solution for maintenance decision making. A numerical example is provided which demonstrates how the RL can be used to find the optimal maintenance policy for the system under study.