Continuous video stream pixel sensor: A CNN‐LSTM based deep learning approach for mode shape prediction

Abstract

Modal analysis has emerged as a globally accepted tool to formulate and optimize the behavioral functions of engineering structures, which assists in assessing structural failure and laying out a plan for their maintenance. Modal analysis aims at determining the frequencies, damping ratios, and mode shapes of the system under excitation. However, conventional mode shape measurement methods like contact sensors are prone to precision and accuracy issues owing to the sensor's weight and low spatial resolution. In this paper, we improve upon various existing methods for mode shape determination and introduce the idea of a full-field pixel sensor for mode shape prediction. The proposed computer vision-based deep learning architecture predicts the mode shape of a vibrating structure with significant precision. Besides, a ModeShape dataset consisting of the vibration recording video and finite element analysis (FEA) based label has been curated. Specifically, we introduce a convolutional neural network, long short-term memory (CNN-LSTM) computer vision-based non-contact vibration measurement technique for automated mode shape prediction. The key idea is to use each pixel of a RGB camera as a sensor and process the captured spatio-temporal data to enable mode shape prediction. Our CNN-LSTM model takes the video streams of a vibrating structure as input and yields the fundamental mode shapes. The proposed technique is non-invasive and can extract information at relatively high spatial density. The CNN-LSTM model is proficient by utilizing experimental outcomes. The robustness of the deep learning model has been scrutinized by utilizing specimens of an assortment of different materials and fluctuating dimensions.