Data-driven automated control algorithm for floating-zone crystal growth derived by reinforcement learning

Abstract

The complete automation of materials manufacturing with high productivity is a key problem in some materials processing. In floating zone (FZ) crystal growth, which is a manufacturing process for semiconductor wafers such as silicon, an operator adaptively controls the input parameters in accordance with the state of the crystal growth process. Since the operation dynamics of FZ crystal growth are complicated, automation is often difficult, and usually the process is manually controlled. Here we demonstrate automated control of FZ crystal growth by reinforcement learning using the dynamics predicted by Gaussian mixture modeling (GMM) from small numbers of trajectories. Our proposed method of constructing the control model is completely data-driven. Using an emulator program for FZ crystal growth, we show that the control model constructed by our proposed model can more accurately follow the ideal growth trajectory than demonstration trajectories created by human operation. Furthermore, we reveal that policy optimization near the demonstration trajectories realizes accurate control following the ideal trajectory.