Applying Reinforcement Learning to Plan Manufacturing Material Handling Part 1

Abstract

Applying machine learning to improve the efficiency of complex manufacturing processes, particularly logistics and material handling, can be a challenging problem. The interconnectedness of the multiple components that compose such processes and the typically large number of variables required to specify procedures and plans within those processes combine to make it very difficult to map the details of real-world manufacturing processes to an abstract mathematical representation suitable for machine learning methods. In this paper, we report on the application of machine learning methods, in particular reinforcement learning, to generate increasingly efficient plans for material handling to satisfy temporally varying product demands in a representative manufacturing facility. The essential steps in the research included defining a formal representation of a realistically complex material handling plan, defining a set of suitable two-stage plan change operators as reinforcement learning actions, implementing a simulation-based multi-objective reward function that considers multiple components of material handling costs, and abstracting the many possible material handling plans into a state set small enough to enable reinforcement learning. Extensive experimentation with multiple starting plans showed that the reinforcement learning process could consistently reduce the material handling plans' costs over time. This work may be one of the first applications of reinforcement learning with a multi-objective reward function to a realistically complex material handling process. This paper first provides brief background on material handling and reinforcement learning. It then details the formal mathematical representation of a realistic material handling plan for a manufacturing facility and specifies the reinforcement learning operators to be applied to the plan representation. This paper (part 1 of 2) is the first of two companion papers; the second describes the experimentation and results [1].