Adaptive Weight Tuning of EWMA Controller via Model-Free Deep Reinforcement Learning

Abstract

Exponentially weighted moving average (EWMA) controllers have been extensively studied for run-to-run (RtR) control in semiconductor manufacturing processes. However, the EWMA controller with a fixed weight struggles to achieve excellent performance under unknown stochastic disturbances. To improve the performance of EMWA via online parameter tuning, an intelligent strategy using deep reinforcement learning (DRL) technique is developed in this work. To begin with, the weight adjusting problem is established as a Markov decision process. Meanwhile, simple state space, action space and reward function are designed. Then, the classical deep deterministic policy gradient (DDPG) algorithm is utilized to adjust the weight online. Moreover, a quantile regression-based DDPG (QR-DDPG) algorithm is further verified the effectiveness of the proposed method. Finally, the developed scheme is implemented on a deep reactive ion etching process. Comparisons are conducted to show the superiority of the presented approach in terms of disturbance rejection and target tracking.