Coordinated Look-ahead Control of Multiple CSPS System by Multi-agent Reinforcement Learning

Abstract

The optimal control problem of a multiple conveyor-serviced production station (CSPS) system is concerned.The objective is to maximize the part-processing rate of the entire system by choosing a suitable look-ahead control strategy for each CSPS.According to the reaction-diffusion mechanism of multi-agent systems,the original performance function of each agent is first modified by introducing an item with a diffusion function that denotes the interaction of local information (The original item is assumed to have a reaction function).Then,combined with the concept of performance potentials,a multi-agent algorithm,i.e.,Wolf-PHC algorithm,is proposed to derive the coordinated look-ahead control strategy for systems with either discounted or average performance criteria,where the decision epoch of each agent is asynchronous.Finally,a simulation example is used to illustrate the effectiveness of the algorithm,and the simulation results show that due to the modification of the performance functions,the contributions of all the stations are well balanced,and the part-processing rate of the entire system is increased significantly.