Sequential auction for cloud manufacturing resource trading: A deep reinforcement learning approach to the lot-sizing problem

Abstract

Cloud manufacturing is a rapidly growing trend in modern manufacturing, which has transformed the traditional operations and value chain structure of enterprises. It is crucial to develop a rational and effective trading mechanism of cloud manufacturing resources to meet the ever-increasing demands in this new environment. This paper proposes a sequential auction-based paradigm for the trading of manufacturing resources. The main challenge of the design of the paradigm is to determine the dynamic lot size of resources allocated to each auction considering the uncertainty of arriving demands. To achieve this, we first develop a competitive game model to identify optimal bidding strategies of arrived customers and estimate the expected revenue for each round with the given lot size. Secondly, we construct a Markov decision process (MDP) model to characterize the dynamics of the arrival of stochastic demand and the inventory transition of the manufacturing resources in sequential auctions. Lastly, we leverage a data-driven approach by integrating machine learning with an offline deep reinforcement learning (RL) approach. Specifically, we employ a long short-term memory (LSTM) model to predict forthcoming demands in the environment and develop the deep Q-network (DQN) learning algorithm to optimize lot-sizing policy by interacting with the well-learned LSTM environment model. Our simulation experiments validate the effectiveness of our approach and some management insights are given.