Incorporating domain knowledge into reinforcement learning to expedite welding sequence optimization

Abstract

Welding Sequence Optimization (WSO) is very effective to minimize the structural deformation, however selecting proper welding sequence leads to a combinatorial optimization problem. State-of-the-art algorithms could take more than one week to compute the best sequence for an assembly of eight weld beads which is unrealistic for the early stages of Product Delivery Process (PDP). In this article, we develop and implement a novel Reinforcement Q-learning algorithm for WSO where structural deformation is used to compute reward function. We utilize a thermo-mechanical Finite Element Analysis (FEA) to predict deformation. The exploration–exploitation dilemma has been tackled by domain knowledge driven ε-greedy algorithm into Q-RL which helps to expedite the WSO and we call this novel algorithm as DKQRL. We run welding simulation experiment using well-known Simufact® software on a typical widely used mounting bracket which contains eight welding beads. DKQRL allows the reduction of structural deformation up to ∼71% and it substantially speeds up the computational time over Modified Lowest Cost Search (MLCS), Genetic Algorithm (GA), exhaustive search, and standard RL algorithm. Results of welding simulation demonstrate a reasonable agreement with real experiment in terms of structural deformation.