Practical Q-Learning-Based Route-Guidance and Vehicle Assignment for OHT Systems in Semiconductor Fabs

Abstract

We introduce and analyze vehicle-allocation algorithms for overhead hoist transport (OHT) systems in semiconductor wafer-fabrication facilities (fabs). OHT is the most widely used type of automated material-handling system in fabs, comprising hundreds of vehicles delivering lots between processing machines. Timely transport unit delivery by OHT systems are critical to the efficient overall operation of a modern fab. We first describe the limitations of current OHT vehicle-allocation algorithms, and then detail an improved system that can be implemented in practical environments to manage the operation of hundreds of OHT vehicles. Our proposed system is based on reinforcement learning. In particular, it uses Q-learning-based dynamic vehicle-allocation algorithms to examine traffic conditions and then allocate an OHT vehicle to a delivery job. We propose multiple algorithms based on the Q-learning approach and compare their performance with that of conventional allocation approaches, which reveals the appropriate algorithms for use in industry. We demonstrate that our algorithms are efficient and sufficiently fast to be used in a practical large-scale setting.