Neuroevolution reinforcement learning for multi-echelon inventory optimization with delivery options and uncertain discount

Abstract

The advanced information technology has enabled supply chain to make centralized optimal decision, allowing to make a global optimal solution. However, dealing with uncertainty is important in inventory management. Besides demand and supply uncertainties, supplier discounts also often arise unexpectedly. Further, suppliers or third-parties typically offer various delivery options in which trade-off occurs between cost and lead time. Thus, this study introduces new problem namely Multi-Echelon Inventory Optimization with Delivery Options and Uncertain Discount (MEIO-DO-UD). As a solution, Neuroevolution Reinforcement Learning (NERL) framework is developed for minimizing total system cost. The environment is modeled via System Dynamics (SD) and actor is presented by integration of Artificial Neural Network and Evolutionary Algorithm (EA), creating an effective decision-making model under dynamic uncertainty. The experimental study has been conducted where two different supply chain networks are given namely serial and divergence. Three EA algorithms are compared namely Differential Evolution (DE), Memetic Algorithm (MA), and Evolution Strategy (ES). Furthermore, NERL is also compared with the EA-optimized classical continuous review model namely (s,Q). The result shows that regardless what EA type is used, the proposed NERL always outperforms EA-optimized (s,Q) model. The more complex the problem, the further improvement can be made i.e. cost reduction up to 58%, followed by the fill rate improvement. The result also shows that NERL can avoid overfitting. Managerial implications are highlighted where NERL provides the more stable inventory level among all supply chain partners and bull-whip effect can be damped.