Optimizing the microstructure in open-die forgings using reinforcement learning

Abstract

Abstract. The open-die forging process can produce large workpieces with excellent material properties that can be used for heavy-duty applications like turbine shafts. The mechanical properties result from the microstructure, which in turn directly results from the process route. Since in open-die forging processes commonly hundreds of unique forming operations are carried out, numerous process routes lead to the same final geometry but produce different microstructures. This is why the prior design of an optimal pass schedule is essential to ensure good mechanical properties of open-die forgings. In the past reinforcement learning (RL) was already used to design optimized pass schedules for open-die forging that achieve the desired geometry, utilize the available press force, and reduce the number of passes. Furthermore, the design of a single pass schedule only took a few seconds which creates opportunities for the use of RL in control systems e.g. for the microstructure. This is why in this publication an existing RL algorithm is extended so that microstructure can be included in the optimization. Within this process, a microstructure model is integrated into the RL algorithm and the reward function (defines the goal of the training process) was extended in two steps to also rate the achieved average grain size continuously dependent on the temperature of the workpiece. In addition, the RL implementation was changed to ensure the production of the desired final geometry leading to a decrease in the complexity of the optimization problem. Thus, both an improvement of the designed pass schedules and a significant reduction of the training time of the RL algorithm was achieved.