Applying Reinforcement Learning to Plan Manufacturing Material Handling Part 2

Abstract

Applying machine learning to improve the efficiency of complex manufacturing processes, particularly logistics and material handling, can be a challenging problem. The interconnectedness of the multiple components that compose such processes and the typically large number of variables required to specify procedures and plans within those processes combine to make it very difficult to map the details of real-world manufacturing processes to an abstract mathematical representation suitable for machine learning methods. In this paper, we report on the application of machine learning methods, in particular reinforcement learning, to generate increasingly efficient plans for material handling to satisfy temporally varying product demands in a representative manufacturing facility. The essential steps in the research included defining a formal representation of a realistically complex material handling plan, defining a set of suitable two-stage plan change operators as reinforcement learning actions, implementing a simulation-based multi-objective reward function that considers multiple components of material handling costs, and abstracting the many possible material handling plans into a state set small enough to enable reinforcement learning. Extensive experimentation with multiple starting plans showed that the reinforcement learning process could consistently reduce the material handling plans' costs over time. This work may be one of the first applications of reinforcement learning with a multi-objective reward function to a realistically complex material handling process. This paper first provides an explanation of how the material handling plans and rewards were abstracted into a manageable state set. It then details the various initial plans and experimental trials used to test the plans. Finally, it reports the results of those experimental trials, including the plan change policies learned and the reductions in material handling costs achieved.