Optimal Look-ahead Control of CSPS System by Deep Q-Network and Profit Sharing

Abstract

The look-ahead control problem in the conveyor-serviced production station (CSPS) system is an important topic of research in an intelligent production line. Some studies has applied various kinds of intelligent algorithms to reduce the average discount cost of the system. There are three goals in this paper. We proposed the algorithm based on Deep Q-network (DQN), applied to the CSPS system to achieve an effective reduction of the average cost. The oscillations in the learning process is reduced through the profit sharing (PS) algorithm. Moreover, we not only proposed the algorithm for CSPS system with DQN, but also further optimized the intelligent look-ahead control of CSPS system by combining it with PS, thereby enabling the processing rate of each production station in the CSPS system to achieve optimal. The learning efficiency, convergence result and stability during the learning process were used as the main evaluation criteria to analyze the experimental results. The results show that the combination of DQN and PS can effectively optimize the performance of look-ahead control when the parameters are selected reasonably.