32‐3: Heterogeneous Resource Constrained Reinforcement Learning Photolithography Scheduler with Heterogeneous Graph Attention Network

Abstract

In mass manufacturing of OLED displays, photolithography process is a critical bottleneck in production capacity. As a result, effective scheduling of the photolithography process is crucial to the overall throughput, productivity, and efficiency of the production. The processing of OLED panels depends upon heterogeneous resources. Aside from that, the complexity, stochastic nature, and highly dynamic characteristic of the processpresent considerable challenges. To address these, we propose a new framework employing a heterogeneous graph neural networkbased Rainbow algorithm. This framework optimizes the schedule used in photolithography process. Considering the constraints on machines, masks, and the stochastic nature of arrival processes, we optimize for maximizing productivity while minimizing the associated costs. We model the interactions among product lot steps, machines, and masks as a heterogeneous graph. This graph encodes the information of lot steps, machines, and masks into distinctive nodes. We then implement a Graph Attention Networkbased architecture for deep representation learning. This transforms state information into node embeddings for each step of each product lot, facilitating decision‐making. Reinforcement learning agents leverage these embeddings to prioritize products via a parameterized Q‐function. Our extensive experiments demonstrate the superior performance of our approach across multiple evaluation metrics in both static and dynamic environment benchmarks, with a higher winning rate and scheduling reward return when compared to existing reinforcement learning methods.