A Deep Reinforcement Learning Approach for Inventory Control under Stochastic Lead Time and Demand

Abstract

In the last few years, the deep learning paradigm has experienced huge success in various machine learning research areas like computer vision, drug discoveries, natural language processing, and combinatorial optimizations. Moreover, the world has witnessed remarkable achievements when combining deep learning with reinforcement learning (now known as Deep Reinforcement Learning) in the areas like robotics, video games, business, and healthcare. One of the strongest parts of Deep Reinforcement Learning (DRL) is the ability to solve sequential decision-making problems. The inventory control problem is one such field where DRL can be applied to learn the optimal ordering policy to minimize the total inventory cost. In this paper, a linear supply chain model is considered with stochastic lead time and demand. The problem is then modeled into Markov Decision Processes (MDP). We then designed three different agents: Q-learning agent, Deep Q-network (DQN, also known as Deep Q-Learning), and (R, S) policy-based agent. The Q-learning and DQN agents were trained and evaluated. The (R, S) policy is used as a baseline as it is one of the most popular policies in business organizations. In comparison to traditional reinforcement learning (i.e Q-learning) and rule-based learning (i.e. (R, S) policy), the DQN model performs better in making the optimal ordering decision so that the total cost is minimized.