Deep reinforcement learning for demand fulfillment in online retail

Abstract

A distinctive feature of online retail is the flexibility to ship items to customers from different distribution centers (DCs). This creates interdependence between DCs and poses new challenges in demand fulfillment to decide from which DC to satisfy each customer demand. This paper addresses a demand fulfillment problem in a multi-DC online retail environment where demand and replenishment lead time are stochastic. The objective of the problem is to minimize the long-term operational costs by determining the source DC for each customer demand. We formulate the problem as a semi-Markov decision process and develop a deep reinforcement learning (DRL) algorithm to solve the problem. To evaluate the performance of the DRL algorithm, we compare it with a set of heuristic rules and exact solutions obtained by linear programming. Numerical results show that the DRL policy performs equally well with the most competitive heuristic on complete pooling DC networks and outperforms all the heuristics on partial pooling DC networks. Additionally, by analyzing the transshipment ratio of the best-observed policies, we provide managerial insights regarding the circumstances in which transshipment is more favorable.