A multi-agent reinforcement learning model for inventory transshipments under supply chain disruption

Abstract

The COVID-19 pandemic has significantly disrupted global Supply Chains (SCs), emphasizing the importance of SC resilience, which refers to the ability of SCs to return to their original or more desirable state following disruptions. This study focuses on collaboration, a key component of SC resilience, and proposes a novel collaborative structure that incorporates a fictitious agent to manage inventory transshipment decisions between retailers in a centralized manner while maintaining the retailers’ autonomy in ordering. The proposed collaborative structure offers the following advantages from SC resilience and operational perspectives: (i) it facilitates decision synchronization for enhanced collaboration among retailers, and (ii) it allows retailers to collaborate without the need for information sharing, addressing the potential issue of information sharing reluctance. Additionally, this study employs non-stationary probability to capture the deeply uncertain nature of the ripple effect and the highly volatile customer demand caused by the pandemic. A new Reinforcement Learning (RL) algorithm is developed to handle non-stationary environments and to implement the proposed collaborative structure. Experimental results demonstrate that the proposed collaborative structure using the new RL algorithm achieves superior SC resilience compared with centralized inventory management systems with transshipment and decentralized inventory management systems without transshipment using traditional RL algorithms.