Dynamic maintenance scheduling approach under uncertainty: Comparison between reinforcement learning, genetic algorithm simheuristic, dispatching rules

Abstract

Maintenance planning and scheduling are an essential part of manufacturing companies to prevent machine breakdowns and increase machine uptime, along with production efficiency. One of the biggest challenges is to effectively address uncertainty (e.g., unexpected machine failures, variable time to repair). Multiple approaches have been used to solve the maintenance scheduling problem, including dispatching rules (DR), metaheuristics and simheuristics, or most recently reinforcement learning (RL). However, to the best of our knowledge, no study has ever studied to what extent these techniques are effective when faced with different levels of uncertainty. To overcome this gap in research, this paper presents an approach by analyzing the impact of categorized levels of uncertainty, specifically high and low, on the failure distribution and time to repair. Upon the formalization of the maintenance scheduling problem, the experiments conducted are performed in simulated scenarios with different degrees of uncertainty, and also considering a real-life manufacturing use case. The results indicate that rescheduling based on a genetic algorithm (GA) simheuristic outperforms RL and DR in terms of total machine uptime, but not in terms of the mean time to repair when configured with high re-optimization frequencies (i.e., hourly re-optimization), but rapidly underperforms when the re-optimization frequency decreases. Furthermore, our study demonstrates that GA-simheuristic is highly computationally demanding compared to RL and rule-based policies.