Counterfactual-attention multi-agent reinforcement learning for joint condition-based maintenance and production scheduling

Abstract

Maintenance and production scheduling are interactive activities that should be considered simultaneously to maintain production systems’ reliability and high production delivery rate. This study investigates a problem of joint maintenance and production scheduling in a multi-stage hybrid flow shop experiencing machine deterioration processes. To address the structural dependency among production stages and machines, a decentralized partially observable Markov decision process (Dec-POMDP) is considered. However, the large state and action space pose a challenge for existing reinforcement learning methods to provide satisfactory solutions. To overcome this problem, the study proposes a counterfactual attention multi-agent reinforcement learning (CAMARL) solution framework that comprises three functional modules: The attention mechanism module, which adaptively shrinks the dimension of the state space; the action abstraction representation module that deals with the high-dimension of the action space; and the coordination control unit, which accelerates the exploration for the optimal production policy. Numerical experiments demonstrate the effectiveness of the proposed method by comparing it with seven benchmarks under different production scenarios.