Exponentially Weighted Moving Average Controller Research Articles

Reinforcement learning for process control with application in semiconductor manufacturing

Process control is widely discussed in the manufacturing process, especially in semiconductor manufacturing. Due to unavoidable disturbances in manufacturing, different process controllers are proposed to realize variation reduction. Since Reinforcement Learning (RL) has shown great advantages in learning actions from interactions with a dynamic system, we introduce RL methods for process control and propose a new controller called RL-based controller. Considering the fact that most existing run-to-run (R2R) controllers mainly rely on a linear model assumption for the process input–output relationship, we first discuss theoretical properties of RL-based controllers based on the linear model assumption. Then the performance of RL-based controllers and traditional R2R controllers (e.g., Exponentially Weighted Moving Average (EWMA), double EWMA, adaptive EWMA, and general harmonic rule controllers) are compared for linear processes. Furthermore, we find that the RL-based controllers have potential advantages to deal with other complicated nonlinear processes. The intensive numerical studies validate the advantages of the proposed RL-based controllers.

Adaptive Weight Tuning of EWMA Controller via Model-Free Deep Reinforcement Learning

Exponentially weighted moving average (EWMA) controllers have been extensively studied for run-to-run (RtR) control in semiconductor manufacturing processes. However, the EWMA controller with a fixed weight struggles to achieve excellent performance under unknown stochastic disturbances. To improve the performance of EMWA via online parameter tuning, an intelligent strategy using deep reinforcement learning (DRL) technique is developed in this work. To begin with, the weight adjusting problem is established as a Markov decision process. Meanwhile, simple state space, action space and reward function are designed. Then, the classical deep deterministic policy gradient (DDPG) algorithm is utilized to adjust the weight online. Moreover, a quantile regression-based DDPG (QR-DDPG) algorithm is further verified the effectiveness of the proposed method. Finally, the developed scheme is implemented on a deep reactive ion etching process. Comparisons are conducted to show the superiority of the presented approach in terms of disturbance rejection and target tracking.

Design of a run‐to‐run controller based on discrete sliding‐mode observer

AbstractIn this study, a run‐to‐run controller based on a discrete sliding‐mode observer (DSMO) is presented for photolithography manufacturing processes. A second‐order DSMO is introduced to estimate the process disturbances caused by tool aging and preventive maintenance. The proposed controller is constructed by applying DSMO estimation to the deadbeat control law and the stable region is discussed. Numerical simulations and an industrial case demonstrated the performance of the proposed algorithm. It is observed that the proposed controller has a superior performance over the exponentially weighted moving average (EWMA) and double EWMA (dEWMA) controller under the first‐order integrated moving average disturbance with shift or drift. This superiority is maintained when the system contains a certain plant‐model mismatch. Moreover, the tolerance of the proposed controller for metrology delay is better than that of the dEWMA controller.

Run-to-run control of batch production process in manufacturing systems based on online measurement

Run-to-run control is an important and effective method for quality control of batch production process in manufacturing systems. Most of current run-to-run control methods are investigated based on offline sampled measurement due to the limitation of measurement technology. The sampled measurements cannot represent each run, thus lead to performance deterioration of run-to-run control methods. The development of new measurement technologies provides great opportunities to improve product quality, since all data of each run is obtained by online measurement. This paper presents a systematic run-to-run control method for batch production process in manufacturing systems based on online measurement. Firstly, a single-input-single-output process model of multiple workpieces (MSISO) is built and a MSISO double exponentially-weighted-moving-average (EWMA) controller is proposed. The proposed MSISO double EWMA controller updates control recipe of current run based on all output data of its previous run. Then, the stability analysis of the proposed MSISO double EWMA controller is conducted to obtain the sufficient conditions for the asymptotic stability of the control process in manufacturing systems. Mean square error (MSE) is used to evaluate the quality control performance. The results of simulation experiments and a case study show that the proposed method can effectively improve the control performance for batch production process in manufacturing systems when compared to other run-to-run-based methods.

Compensating Misalignment Using Dynamic Random-Effect Control System: A Case of High-Mixed Wafer Fabrication

It is vital to have an exclusive modification in semiconductor production process because of meeting differentiated customer demands in dynamic and competitive global minuscule semiconductor technology market and the highly complex fabrication process. In this paper, we propose a control system based on the dynamic mixed-effect least-square support vector regression (LS-SVR) control system for overlay error compensation with stochastic metrology delay to minimize the misalignment of the patterning process. Moreover, for the stability of the control system in the presence of metrology delay and to deal with nonlinearity among the overlay factors, the novel Lyapunov-based kernel function is merged with the LS-SVR controller. The proposed controller’s operation has been validated and implemented by a major semiconductor manufacturer in Taiwan. The experiments are verified that mixed-effect LS-SVR controller has the higher validity and higher efficiency in comparison with the exponentially weighted moving average (EWMA) and threaded EWMA controllers which had been previously implemented at the company or applied in similar studies. Note to Practitioners —Due to high production complexity in semiconductor manufacturing process, a meticulous and intelligent process control is needed to achieve higher throughput and customer satisfaction. Monitoring a complex system is challenging because the process components and variables operate autonomously and interoperate with other manufacturing segments. This paper proposes a novel run-to-run (R2R) control system to compensate the overlay error during the photolithography process that efficiently deals with the high-mixed manufacturing environment and metrology delay.

An Extended State Observer-Based Run to Run Control for Semiconductor Manufacturing Processes

In this paper, an extended state observer (ESO)-based run to run (RtR) controller is presented to suppress various stochastic disturbances emerging in the semiconductor manufacturing process. Using the internal model control theory, the relationship between the ESO-RtR controller and the exponentially weighted moving average (EWMA) controller, and the relationship between the ESO-RtR controller and double EWMA controller are discussed. The stable region of the proposed ESO-RtR controller is calculated and discussed by using the Jury’s test. Furthermore, the ESO-RtR controller is extended to a nonlinear form (i.e., nonlinear ESO-RtR controller) to improve its disturbance compensation capability. Finally, numerical simulations and an industrial example are provided to demonstrate the effectiveness of the proposed algorithm.

Improved Variable EWMA Controller for General ARIMA Processes

The exponentially weighted moving average (EWMA) controller is a popular run-to-run control scheme in semiconductor manufacturing because of its effectiveness and simplicity. In this paper, we propose an improved variable EWMA controller design method. The optimal discount factor of the EWMA controller is determined by minimizing the mean square error of process output at each run. The proposed method overcomes some potential problems of Tseng's method and can be applied to general autoregressive integrated moving average processes. A numerical study is performed to investigate the performance and sensitivity of the proposed method. It is shown that the proposed method is more effective and robust than the conventional EWMA controller when the process parameters are estimated with uncertainty.

Optimal setup adjustment and control of a process under ARMA disturbances

Process adjustment uses information from past runs to adjust settings for the next run and bring the output to its target. The efficiency of a control algorithm depends on the nature of the disturbance and dynamics of the process. This article develops a control algorithm when the disturbance is a general ARMA(p, q) process, in the presence of measurement error and adjustment error together with a random initial bias. Its optimality property is established and the stability conditions are derived. It is shown that the popular Exponentially Weighted Moving Average (EWMA) controller is a special case of the proposed controller. In addition, Monte Carlo simulations are conducted to study the finite sample behavior of the proposed controller and compare it with the proportional–integral–derivative controller when the disturbance is an ARMA(1,1) process and with the EWMA controller when the disturbance is an IMA(1,1) process. The ARMA controller is also implemented to control an ARMA(2,1) disturbance and its performance is compared with the other two controllers. All of the results reflect the new controller’s superiority when multiple sources of uncertainty exist or a general ARMA(p, q) disturbance is incurred.

Quasi-minimum mean square error run-to-run controller for dynamic models

The Exponentially Weighted Moving Average (EWMA) feedback controller is a popular model-based run-to-run feedback controller that primarily uses data from previous process runs to adjust the settings of the next run. The long-term stability conditions and the transient performance of the EWMA controller have received considerable attention in the literature. Most of these studies have assumed that the process Input–Output (I-O) relationship is static and simply considered colored noise models for the process disturbance. However, process dynamics and disturbance dynamics may occur simultaneously. Under this circumstance, using EWMA-based controllers will usually lead to an unsatisfactory performance. To overcome this weakness, this article first proposes a quasi-minimum mean square error controller. The theoretical results of the long-term stability conditions are derived under a first-order transfer function model together with a general disturbance model. Furthermore, comprehensive simulation studies are conducted to compare the proposed controller with existing popular controllers. The results demonstrate that the proposed controller outperforms those popular controllers for most cases, even when the process I-O model is mis-specified or the process parameters are not precisely estimated. Supplementary materials are available for this article. Go to the publisher's online edition of IIE Transactions for the following supplemental resources: Tables 5 to 7 and Appendix.

Modified EWMA controller subject to metrology delay

The resource and capacity limitations of the metrology equipment used in the creation of integrated circuits result in delay being a common issue for the practical implementation of run-to-run control scheme. In the literature, several papers have provided the means of using metrology delay data or Virtual Metrology (VM) prediction models on the transient behavior and asymptotic stability of Exponentially Weighted Moving Average (EWMA) controllers. However, these procedures still suffer from large bias and/or variation of the process response variable. To overcome this difficulty, a modified EWMA controller is proposed that adjusts the process by using both the metrology delay data and VM information. The analytical expression of the process output of this developed controller and its long-term stability along with short-term output performance are derived. Furthermore, under some specific parameter settings, a more comprehensive study is presented to illustrate that the proposed controller has the capability to reduce the total mean square error of process response variable compared with existing controllers. [Supplementary materials are available for this article. Go to the publisher's online edition of IIE Transactions for the Appendices to this article.]

An Adaptive-Tuning Scheme for G&P EWMA Run-to-Run Control

Run-to-run (RtR) control is an important method for improving process capability. The most common form of RtR controllers are exponentially weighted moving average (EWMA) controllers. The performance of EWMA RtR controllers is affected by the values of the selected tuning parameter. In practice, the tuning parameter usually remains unchanged, resulting in suboptimal performance. In this paper, we propose an adaptive-tuning method for a group and product (G&P) EWMA controller to improve the control performance. The G&P EWMA controller is developed for mixed run processes. We show that the optimum-tuning parameters for the next run of this G&P EWMA controller are obtained online using a window of historical input-output data. The performance improvement due to the proposed method is demonstrated by a simulation example and an industrial application.

Assessing Measurement Noise Effect in Run-to-Run Process Control: Extends EWMA Controller by Kalman Filter

Recently, the Exponentially Weighted Moving Average (EWMA) controller has become a popular control method in Run-to-Run (RtR) process control, but the issue of measurement noise from metrology tools has not been addressed in RtR EWMA controllers yet. This paper utilizes a Kalman Filter (KF) controller to deal with measurement noise in RtR process control and investigates the output properties for steady-state mean and variance, and for closed-loop stability. Five disturbance models modeling semiconductor process disturbances are investigated. These disturbance models consist of Deterministic Trend (DT), Random Walk with Drift (RWD), Integrated Moving Average process (IMA(1,1)), AutoRegressive Moving Average (ARMA(1,1)), and Autoregressive Integrated Moving Average (ARIMA(1,1,1)). Analytical results show that a KF controller can be considered as an extended version of a RtR EWMA controller. In particular, the EWMA controller is a special case of KF in a filtering form without the capability of measuring noise. Simulation results also show that the KF has a better ability to deal with measurement noise than the EWMA controller.

Multiple-input multiple-output double exponentially weighted moving average controller using partial least squares

This paper applies the partial least squares (PLS) method to the multiple-input multiple-output (MIMO) semiconductor processes in the run-to-run (R2R) control practice. Due to the property of batch processing, the semiconductor manufacturing processes frequently exhibit high multicollinearity among input variables and dependency among output variables. These two effects will typically cause variance inflation of the regression coefficient estimates which are utilized in triggering or updating the R2R controller. Furthermore, the process nonlinearity is also likely to occur in some semiconductor processes. As the nonlinearity exists, the performance of the exponentially weighted moving average (EWMA) controller is not adequate and becomes aggravated after a few transient runs. The PLS method is, essentially, well suited for situations where multicollinearity is present among input variables. To rectify the aforementioned difficulties that might realistically take place in practice, the PLS method is considered in this paper a potential estimation alternative to the standard regression method. Three types of R2R simulation studies are conducted to verify the advantages of the PLS method. The simulation results show that using the PLS method as the model-building technique helps the EWMA controller to yield more consistent and robust control outputs than purely using the conventional EWMA controller.

Properties of EWMA Controllers With Gain Adaptation

Exponentially weighted moving average (EWMA) controllers are the most commonly used run-to-run controllers in the semiconductor industry. Using a linear model, an controller can be implemented in two different ways: either process gain or process intercept can be updated using statistics at each run. The most commonly used controller formulation is to keep the process gain as its off-line estimate and update the intercept term at each run. We term this formulation as EWMA controller with intercept adaptation (EWMA-I), and its properties have been extensively studied and well understood. We term the other implementation, i.e., keeping the intercept term as its off-line estimate and updating the process gain at each run, as EWMA with gain adaptation (EWMA-G). Despite the fact that gain variation and adaptation is typical in the semiconductor industry, little research has been done to investigate the properties of an EWMA-G controller, such as its stability and sensitivity. In this work, the stability and sensitivity properties of an EWMA-G controller are investigated and compared with those of an EWMA-I controller. Both stationary and drifting processes are considered. In addition, the expressions of the process outputs are derived and the output variances for stochastic processes are evaluated. Simulation examples are given to illustrate these properties and the relevance of these properties to semiconductor process control is discussed. The analysis results will provide some useful guidance on the industrial applications of the controllers.

Compensating for the Initialization and Sampling of EWMA Run-to-Run Controlled Processes

The exponentially weighted moving average (EWMA) filter is commonly used for state estimation of run-to-run controllers in semiconductor manufacturing. It is widely known that, when at steady state, the EWMA filter provides the minimum mean square error (MSE) forecast for an integrated moving average (IMA) process. The forecast, however, is optimal if and only if every output of the IMA process is measured. If an EWMA controller is implemented utilizing sampled process data, then it is necessary to retune the controller to maintain optimal performance. Furthermore, in practice, the complex interacting selection criteria of advanced sampling applications often cause measurement frequency to be highly irregular. In this paper, a sampling compensation algorithm (SCA) is derived based on the minimum-norm IMA (MNIMA) forecast. The algorithm provides the minimum mean-square-error (MMSE) forecast of an IMA process for irregularly sampled processes and is robust to initialization.

Threaded EWMA Controller Tuning and Performance Evaluation in a High-Mixed System

The exponentially weighted moving average (EWMA) controller is a very popular run-to-run (RtR) control scheme in the semiconductor industry. However, in any typical step of semiconductor process, many different products are produced on parallel tools. RtR control is usually implemented with a ldquothreadedrdquo control framework, i.e., different controllers are used for different combinations of tools and products. In this paper, the problem of EWMA controller tuning and performance evaluation in a mixed product system are investigated by simulation and time-series analysis. It was found that as the product frequency changed, the tuning guidelines of a threaded EWMA controller were different for different types of tool disturbances. For a stationary ARMA(1,1) noise, the tuning parameter lambda should be decreased as product frequency decreases. If the tool exhibits nonstationary tool dynamics, e.g., ARIMA(1,1,1) noise, the tuning parameter should increase as the product frequency decreases.

Stability analysis and optimal tuning of EWMA controllers – Gain adaptation vs. intercept adaptation

Exponentially weighted moving average (EWMA) controllers are the most commonly used run-to-run controllers in semiconductor manufacturing industry. An EWMA controller can be implemented in two different ways. One way is to keep the process gain as its off-line estimate and update the intercept term at each run, which is termed EWMA with intercept adaptation; the other is to keep the intercept term as its off-line estimate and update the process gain at each run, which is termed EWMA with gain adaptation. Despite the fact that gain variation and adaptation is typical in semiconductor industry, most EWMA formulations are for intercept adaptation and few results exist on the stability and sensitivity of EWMA with gain adaptation. In this paper, we propose a general formulation to analyze the stability of both EWMA controllers. The proposed state-space representation not only reveals the similarities and differences between two types of EWMA controllers, but also explains why the stability conditions for both types of EWMA controllers are independent of process disturbances. In addition, we propose a general framework that unifies the analysis of the optimal control performance for both types of EWMA controllers. The proposed framework is different from existing approaches in that it decouples the state estimation from the control law, and derives the optimal weighting based on the state estimation performance. The proposed framework significantly simplifies the analysis procedure, especially for EWMA with gain adaptation. Using this framework, we derive the optimal EWMA weighting through solving the discrete-time algebraic Riccati equation (DARE) for various process disturbances that are encountered in semiconductor manufacturing industry. Simulation examples are given to illustrate the optimality of the EWMA weighting derived using the framework. Some practical aspects of controller tuning are also discussed based on the simulation results.

Stability analysis of single EWMA controller under dynamic models

The Exponentially Weighted Moving Average (EWMA) feedback controller is a popular model-based run-to-run controller which primarily uses data from previous process runs to adjust settings for the next run. The long-term stability conditions of EWMA controllers for this closed-loop system have received considerable attention in the literature. Most of the reported results are obtained under the assumption that the process I-O (Input-Output) relationship follows a static model. Generally speaking, the effect of the input recipe on the output response can be carried over several periods. In this paper, focusing on a first-order dynamic I-O model and assuming that the process disturbance follows a general ARIMA series, a systematic approach to address this control problem is proposed. First, the long-term stability conditions of a single EWMA controller are investigated. Then, the determination of sample size to allow the design of a single EWMA controller for dynamic models is considered. Under the assumption that the process I-O variables follow a bivariate normal distribution, a formula to calculate sample sizes that allow the stability condition to be met with a minimum probability protection is derived. Finally, the effects of dynamic parameters on the determination of sample size are considered. [Supplementary materials are available for this article. Go to the publisher's online edition of IIE Transactions for the following free supplementary resource: Appendix]

A General Harmonic Rule Controller for Run-to-Run Process Control

The existence of initial bias in parameter estimation is an important issue in controlling short-run processes in semiconductor manufacturing. Harmonic rule has been widely used in machine setup adjustment problems. This paper generalizes the harmonic rule to a new controller called general harmonic rule (GHR) controller in run-to-run process control. The stability and optimality of the GHR controller is discussed for a wide range of stochastic disturbances. A numerical study is performed to compare the sensitivity of the GHR controller, the exponentially weighted moving average (EWMA) controller and the variable EWMA controller. It is shown that the GHR controller is more robust than the EWMA controller when the process parameters are estimated with uncertainty.

Smith–EWMA run-to-run control schemes for a process with measurement delay

The Exponentially Weighted Moving Average (EWMA) controller is a popular run-to-run controller in the semiconductor manufacturing industry. The controller adjusts input based on measurement information from previous runs. EWMA controllers can guarantee satisfactory results in many cases; however, when there is a measurement delay in the process, the stability properties and performance of the EWMA controller cannot be guaranteed. In order to maintain the satisfactory outcomes of EWMA controllers, a Smith predictor control scheme is introduced, created particularly for time delay systems in control theory, into EWMA controllers. A modification of the EWMA controller, called the Smith–EWMA run-to-run controller, is proposed. Comparisons between the stability properties of Smith–EWMA and EWMA run-to-run controllers are studied. Moreover, a performance comparison with the EWMA and recursive-least-square controllers under disturbance conditions based on simulation is conducted. The results show that when there is a measurement delay, the proposed Smith–EWMA run-to-run controllers enlarge the stability region and also achieve better performance when there is serious metrology delay and model uncertainty under process disturbances. [Supplementary materials are available for this article. Go to the publisher's online edition of IIE Transactions for the following free supplemental resource: Appendix]