Deep CNN-Based Materials Location and Recognition for Industrial Multi-Crane Visual Sorting System in 5G Network

Abstract

Intelligent manufacturing is a challenging and compelling topic in Industry 4.0. Many computer vision (CV)-based applications have attracted widespread interest from researchers and industries around the world. However, it is difficult to integrate visual recognition algorithms with industrial control systems. The low-level devices are controlled by traditional programmable logic controllers (PLCs) that cannot realize data communication due to different industrial control protocols. In this article, we develop a multi-crane visual sorting system with cloud PLCs in a 5G environment, in which deep convolutional neural network (CNN)-based character recognition and dynamic scheduling are designed for materials in intelligent manufacturing. First, an YOLOv5-based algorithm is applied to locate the position of objects on the conveyor belt. Then, we propose a Chinese character recognition network (CCRNet) to significantly recognize each object from the original image. The position, type, and timestamp of each object are sent to cloud PLCs that are virtualized in the cloud to replace the function of traditional PLCs in the terminal. After that, we propose a dynamic scheduling method to sort the materials in minimum time. Finally, we establish a real experimental platform of a multi-crane visual sorting system to verify the performance of the proposed methods.