CapsNet-Enabled Multimodal Data Fusion: A Case Study in RUL Estimation of Turbofan Engine

Abstract

Deep learning-based methods can perform well in remaining useful life (RUL) prediction, which alleviates the undesired economic losses and casualties caused by sudden failures of mechanical components. However, with the increased complexity of the equipment, the demand for multimodal data processing increased as well, the key problem of which is the fusion of multimodal data. Throughout the literature related to this topic, capsule neural network (CapsNet) as a novel deep learning technique, can be promising due to its advanced concept of grouping neurons into capsules. Specifically in those papers, the capsule layers simply served as a feature extractor and cannot function up the potential in multimodal data fusion and representation. Thus, a novel CapsNet-enabled multimodal data fusion method is proposed for RUL estimation of turbofan engines. First, multimodal data are converted in each modal channel as deep features by the residual convolutions, and then each modal feature is independently constructed as a modal-wise primary capsule layer. Second, the multimodal features are fused during the routing process in a capsule-based regressor whose last layer contains one capsule. And the Euclidean norm of the pose matrix of the final capsule is regarded as the output value of RUL. The performance of the proposed framework is validated on a public dataset of turbo engines The experimental results demonstrate that the proposed CapsNet-enabled multimodal data fusion method has a competitive performance compared to other deep learning-based methods.