The marriage of operations research and reinforcement learning: Integration of NEH into Q-learning algorithm for the permutation flowshop scheduling problem

Abstract

The permutation flowshop scheduling problem (PFSP) attracted much interest from the operations research (OR) community, resulting in various heuristic and metaheuristic methods over the past half-century. However, given the hard nature of the PFSP, efficient algorithms rely on the configuration of initial solutions and sophisticated heuristics. Combining OR and artificial intelligence (AI), this paper investigates the marriage of OR and reinforcement learning for the PFSP. A novel method integrating NEH into the Q-learning algorithm is proposed for solving the PFSP with makespan criterion. With the refinement of action selection by strengthening good actions and weakening bad actions, the proposed Q-NEH algorithm shows a better learning and convergence rate, and outperforms the Q-learning algorithm for all 120 instances in Taillard’s benchmark dataset, with a significantly decreased average relative learning error (RLE) from 13.05 % to 1.73 %. Furthermore, by comparing the performances of the Q-NEH algorithm with related state-of-art benchmark algorithms in a wide range set of instances, it further confirms the superiority and stability of the Q-NEH algorithm in statistics. With a total of 2570 independent tests in two phases of experimental evaluations, the superiority of the Q-NEH algorithm for the PFSP suggests that the integration of OR and AI is a promising research direction for solving complex scheduling problems.