Reinforcement Learning-based Real-time Scheduling Under Random Machine Breakdowns and Other Disturbances: A Case Study

Abstract

In practice, production lines are dynamic and subject to several disruptions, unforeseen events, and requirements. Examples of such disruptions and events include random machine breakdowns, new order arrivals, order cancellations, due date changes, and shortage of material. Production schedules are adapted to such events by conducting rescheduling continuously, using real-time information about the current status of work-in-progress, machines, and resources on a shop-floor. This level of connectivity and real-time information sharing is achieved with the help of advanced initiatives in manufacturing technologies and industrial informatics such as Industry 4.0. Industry 4.0, driven by many emerging technologies, such as cyber-physical systems (CPS), internet of things (IoT), and internet of services (IoS), delivers real-time actionable data for smart decision-making in manufacturing. Several optimization approaches have been proposed to take advantage of such technology by incorporating the use of real-time information in the optimization process. Recently, with the increasing power of new machine learning (ML) algorithms in solving real-world problems, several ML approaches have been introduced to production planning and scheduling.In this paper, to achieve the Industry 4.0 vision in production control, we apply a reinforcement learning (RL) approach to real-time scheduling (RTS). The proposed RL based RTS uses a multiple dispatching rules (MDRs) strategy to enhance the production performance. A case study of a smart manufacturing firm is considered to apply the proposed approach. The firm is located in Ontario (Canada) and specializes in thermoplastic injection molding of various components and assemblies.The production schedules on the shop floor are sensitive to the changes resulting from random breakdowns and their associated maintenance activities. The production managers are using the data from the continuous monitoring system to update production schedules. The updating process is conducted manually based on their knowledge and a single dispatching rule (SDR) strategy. We believe that the proposed RTS system will help the company utilize the installed Industry 4.0 concepts and achieve the Industry 4.0 vision in the production control.The performance of the proposed RTS system is compared to the current strategy applied in the company. Results show the efficiency of the proposed RTS system compared to the current strategy.