Towards better benchmarking using the CWRU bearing fault dataset

Abstract

This paper investigates the use of the Case Western Reserve University (CWRU) bearing dataset for benchmarking bearing fault diagnosis convolutional neural networks (CNNs) in a domain shift problem. The common method for using the CWRU dataset for demonstrating domain shift is described and a potential flaw is identified. It is argued that the accepted procedure of constructing training and testing datasets with different operating conditions does not constitute a useful domain shift problem since the same physical bearings exist in both training and testing sets. To remedy this while using the CWRU dataset, an alternative benchmarking framework is proposed that constructs training and testing datasets with independent sets of bearings. The original and the proposed benchmarking frameworks are compared by training a set of commonly cited diagnosis CNNs within each framework. The results indicate that the original framework allows CNNs to learn features related to specific bearings and may not be able to generalize for different bearings. It is also found that using existing state-of-the-art deep CNNs from other fields in machine learning research may currently present a more efficient option than developing custom CNN architectures for diagnosis when large machine fault datasets are unavailable.