Multi-expert Attention Network with Unsupervised Aggregation for long-tailed fault diagnosis under speed variation

Abstract

The great achievements of intelligent fault diagnosis technique are based on the balance of different health conditions. However, in practical engineering, difficulty in acquisition of fault signals results in the long-tailed distribution of data which leads to overfitting problems. Meanwhile, domain shift caused by speed variation further deteriorates the reliability of the model. To overcome these challenges, a Multi-expert Attention Network with Unsupervised Aggregation (UA-MAN) is proposed for long-tailed fault diagnosis under speed variation. Specifically, each expert network consists of Transformer blocks and utilizes the global dependency modeling capability of self-attention calculation to suppress the domain shift. To compensate for the lack of self-attention calculation for detailed feature acquisition, a convolutional network with residual connection is designed as the shared backbone before each expert. Additionally, the expert networks are trained with different loss functions which allows each expert can adapt to diverse class distributions. Finally, an unsupervised contrastive learning technique is developed to aggregate experts to handle the test dataset with unknown class distribution. The superiority and reliability of the proposed method is verified under different class distributions in two datasets. Furthermore, ablation experiments demonstrate that unsupervised aggregation adapt to the varied distribution of the test set effectively.