Deep Reinforcement Learning toward Robust Multi-echelon Supply Chain Inventory Optimization

Abstract

Multi-echelon supply chains (SC) are highly complex systems with many inherent uncertainties, including customer demands and transportation time from one location to another. In addition to the complex SC structures, inventory decisions made at different stages affect each other. Maintaining global-level optimal inventory levels along the entire supply chain that is robust to changing business needs remains challenging. In this work, we have developed a simulation environment, GymSC, for reinforcement learning of multi-echelon SC inventory policies. Using a series of model-free deep reinforcement learning algorithms, we trained dynamic SC policies that have significantly improved performance when compared with popularly used heuristics. The robustness of the learned policy is demonstrated by its adaptability to a previously unseen environment with non-stationary customer demands. The presented work showcases the effectiveness and robustness of deep reinforcement learning in solving important practical SC optimization problems. In addition, we present this configurable simulation environment as a platform for testing existing and new algorithms to develop robust inventory policies with a goal to encourage further integration of deep reinforcement learning into the SC management problems as a promising alternative for addressing inventory optimization with uncertainties.