Integrated scheduling and flexible maintenance in deteriorating multi-state single machine system using a reinforcement learning approach

Abstract

This paper studies an integrated optimization problem of production scheduling and flexible preventive maintenance (PM) in a multi-state single machine system with deteriorating effects. A flexible PM strategy is proposed to proactively cope with machine failures while ensuring relatively regular PM intervals, which is composed of time-based PM (TBPM) and condition-based PM (CBPM). TBPM is conducted within every flexible time window and CBPM is implemented immediately after the most deteriorated yet still functional state. An illustrative case is presented using the enumeration approach to demonstrate the integration of production scheduling and machine maintenance. Then, Q-learning-based solution framework (QLSF) is further designed with proper state and action sets and reward functions to facilitate the determination of appropriate production scheduling rule under the constraint of the flexible maintenance. Numerical experiments show that the proposed QLSF outperforms the other four state-of-the-art scheduling rules in different scenarios. Moreover, the performance of the proposed flexible PM strategy is also examined and validated in comparison with three candidate maintenance strategies, i.e., run-to-failure corrective maintenance (CM), combination of TBPM and CM, and CBPM. The proposed flexible maintenance and solution approach can enrich the relevant academic knowledge base, and provide managerial insights and guidance in practical production systems.