Machining sequence learning via inverse reinforcement learning

Abstract

In recent years, process planning automation has been strongly promoted in conjunction with the development of information technology (IT). In manufacturing industries, machining sequencing is one of the elements of computer-aided process planning (CAPP) systems where it has a significant impact on the quality and cost of machined components. Therefore, effective and robust planning rules are essential for practical CAPP systems, and various metrics and constraints have been proposed to facilitate the creation of those rules. However, since it is challenging to address explicit factors such as interference between rules, processing difficulties, and manageability, discrepancies that require manual corrections often arise between the generated sequences and the planner's intentions. To resolve this problem, we propose a method of acquiring rules that reproduce the planner's decisions by inverse reinforcement learning (IRL). To apply the IRL process, we focus on identifying a machining sequence that characterizes the planner's decision based on past production processes and interviews with experts. This machining sequence can then be represented using a Markov decision process (MDP) when changing the workpiece shape, which enables the application of IRL. Additionally, to reflect the drawing information in the sequence decision, the workpiece shape is represented as a graph with attached tolerance and roughness values. The graphed machining sequence is then inputted to the graph, where convolutional networking and training are performed. We verified the validity of our proposed method using a small dataset.