Policy manifold generation for multi-task multi-objective optimization of energy flexible machining systems

Abstract

Contemporary organizations recognize the importance of lean and green production to realize ecological and economic benefits. Compared with the existing optimization methods, the multi-task multi-objective reinforcement learning (MT-MORL) offers an attractive means to address the dynamic, multi-target process-optimization problems associated with Energy-Flexible Machining (EFM). Despite the recent advances in reinforcement learning, the realization of an accurate Pareto frontier representation remains a major challenge. This article presents a generative manifold-based policy-search method to approximate the continuously distributed Pareto frontier for EFM optimization. To this end, multi-pass operations are formulated as part of a multi-policy Markov decision process, wherein the machining configurations witness dynamic changes. However, the traditional Gaussian distribution cannot accurately fit complex upper-level policies. Thus, a multi-layered generator was designed to map the high-dimensional policy manifold from a simple Gaussian distribution without performing complex calculations. Additionally, a hybrid multi-task training approach is proposed to handle the mode collapse and large task difference observed during the improvement of the generalization performance. Extensive computational testing and comparisons against existing baseline methods have been performed to demonstrate the improved Pareto frontier quality and computational efficiency of the proposed algorithm.