Adaptive optimal control of stencil printing process using reinforcement learning

Abstract

The stencil printing process (SPP) is a critical operation in surface mount technology (SMT) because it contributes to 60% of soldering defects. The complex relationships between solder paste volume transfer efficiency (TE) and the SPP variables make the control of the solder paste volume TE during production a challenging problem. This research aims to optimize the stencil printing parameters in real time to control the solder paste volume TE and increase the first-pass yields of printed circuit boards (PCBs). A Reinforcement learning (RL) approach, specifically Q-learning, is used to control and maintain the volume TE within the spec limits in an optimal adaptive control system. RL deals with the problem of building a control system or an autonomous agent that can learn how to take the proper actions to reach its objectives through interaction with its environment. The application of RL in SPP is not yet fully explored; therefore, this study investigates the impacts of applying Q-learning to control the volume TE in real time. The proposed control systems capture the induced variations in the SPP for two printing directions and consequently adjust the significant and easy-to-change printing parameters in real time. Two types of Q-learning are explored: Q-table that uses a tabular format to store the Q-values and Q-network that uses an artificial neural network (ANN) to approximate the Q-value function. Moreover, a new heterogeneous reward function-based clustering is proposed, which is integrated into the Q-network to enhance its performance. The results show that the developed control systems can learn the optimal policy and take the proper actions to transit from initial states to terminal states. The proposed control systems using Q-network with a function approximator and heterogeneous reward function converge fully much faster than Q-table using continuous state space. Moreover, Q-network control systems are capable to transit more states to terminal states with a lower number of actions when compared to Q-table control systems.