Multimodal Fusion Convolutional Neural Network With Cross-Attention Mechanism for Internal Defect Detection of Magnetic Tile

Abstract

The internal defect detection of magnetic tile is extremely significant before mounting. Currently, this task is completely realized by manual operation in the magnetic tile manufacturing industry, which results in inefficiency and diseconomy. In this work, we develop an intelligent system based on the acoustic sound for internal defect detection of magnetic tile to overcome these drawbacks. Due to the non-Gaussian and non-stationary characteristics of the acoustic sound, adopting the single modality of the data for internal defect detection of magnetic tile cannot achieve good accuracy. Therefore, we design a multimodal fusion convolutional neural network (MMFCNN) for internal defect detection of magnetic tile. We train the network in an end-to-end way. Our proposed MMFCNN consists of three blocks, i.e., feature extraction block, feature fusion block and internal defect detection block, whose purposes are to extract features from generated modal data, fuse multimodal feature maps and analyze whether the magnetic tile has internal defects, respectively. Moreover, to realize the information interaction and emphasize more representative information at feature extraction stage, we propose a novel attention mechanism, i.e., cross-attention mechanism. Extensive experimental results demonstrate our proposed MMFCNN is effective for internal defect detection of magnetic tile. Our code is available at https://github.com/Clarkxielf/Multimodal-Fusion-Convolutional-Neural-Network-for-Internal-Defect-Detection-of-Magnetic-Tile.