Time-frequency Image Research Articles

A Deep Learning-Based Method for Bearing Fault Diagnosis with Few-Shot Learning

To tackle the issue of limited sample data in small sample fault diagnosis for rolling bearings using deep learning, we propose a fault diagnosis method that integrates a KANs-CNN network. Initially, the raw vibration signals are converted into two-dimensional time-frequency images via a continuous wavelet transform. Next, Using CNN combined with KANs for feature extraction, the nonlinear activation of KANs helps extract deep and complex features from the data. After the output of CNN-KANs, an FAN network module is added. The FAN module can employ various feature aggregation strategies, such as weighted averaging, max pooling, addition aggregation, etc., to combine information from multiple feature levels. To further tackle the small sample issue, data generation is performed on the original data through diffusion networks under conditions of fewer samples for bearings and tools, thereby increasing the sample size of the dataset and enhancing fault diagnosis accuracy. Experimental results demonstrate that, under small sample conditions, this method achieves higher accuracy compared to other approaches.

Deep learning-based multilabel compound-fault diagnosis in centrifugal pumps

In this study, the issue of centrifugal pumps used in marine engineering is addressed, as they are susceptible to malfunction owing to long-term operation in highly corrosive seawater and extreme weather conditions, resulting in operational interruptions and safety risks. We propose a high-precision intelligent fault-diagnosis method for multiple fault types based on deep learning. In this method, continuous wavelet transform is firstly employed to extract signal time–frequency domain features. Subsequently, the Swin transformer model is used to process the wavelet time–frequency images converted from signals. Finally, multilabel classification methods are combined to diagnose various complex faults. The effectiveness of the proposed method is validated using a dataset obtained from simulation experiments pertaining to centrifugal-pump faults. The results show that the proposed method achieves 100% accuracy in diagnosing 27 types of faults and provides excellent diagnosis even under limited compound-fault samples, thus offering an efficient and practical method for fault diagnosis in centrifugal pumps used in marine engineering.

Research on an aeroengine bearing fault diagnosis method based on WMSST-CNN-BKA-LSSVM

Abstract To effectively extract the fault feature from small sample datasets and improve the accuracy of the model, an aeroengine bearing fault diagnosis method is proposed in this research based on wavelet multi-synchrosqueezed transform-convolutional neural network-black-winged kite algorithm-least squares support vector machine (WMSST-CNN-BKA-LSSVM). WMSST was used to perform modal decomposition on the collected fault data to obtain the time-frequency images. The time-frequency images were input into CNN for fault data feature extraction. The results of the CNN fully connected layer were used as input to the LSSVM, and the BKA was used to optimize the key parameters of the LSSVM to classify aeroengine bearing faults. The method was verified by aeroengine bearing test data from the Harbin Institute of Technology. The research showed that the accuracy is 99.86% when the faulty test samples are 70%, and the accuracy is 93.90% when the faulty samples are 30%. The accuracy showed the applicability of this method in small sample fault diagnosis. By comparing the method with a one-dimensional convolutional neural network (1D-CNN) and the convolutional neural network - long short-term memory (CNN-LSTM), it is found that the accuracy has been dramatically improved.

Diffusion models-based motor imagery EEG sample augmentation via mixup strategy

Deep representation learning has been widely explored for decoding motor imagery electroencephalogram (MI-EEG) to build EEG-tailored brain-computer interfaces. Due to the labor-intensive and time-consuming recording to MI-EEG, recently BCIs were suffered from sample scarcity, especially for the limitation of sample diversity. To solve this issue, we proposed a novel sample augmentation method, Diffusion models-based EEG Sample Augmentation method via Mixup strategy (DESAM), to solve sample scarcity and improve sample diversity for both multivariate time-series and time–frequency images forms. To reduce the characteristics conflicts brought by the augmented samples, a mixup strategy with data weighting was proposed for both forms. To validate the efficacy of sample augmentation by DESAM, we conducted comparative experiments on three publicly available MI-EEG datasets, and reported the average decoding accuracies and Kappa values on augmented samples by various deep learning models. For the BCI Competition IV datasets 2a, 2b, and 1, the two forms of DESAM sample augmentations achieved improvements for both accuracies and Kappa values. Ablation studies have demonstrated the necessity and significance of our DESAM method, and it makes a possible manner to solve sample scarcity issue of MI-EEG.

Detection of Early Browning in Pears Based on Time–Frequency Images of Vibro-Acoustic Signals and Improved MobileNetV3

Detection of Early Browning in Pears Based on Time–Frequency Images of Vibro-Acoustic Signals and Improved MobileNetV3

The Research of Intra-Pulse Modulated Signal Recognition of Radar Emitter under Few-Shot Learning Condition Based on Multimodal Fusion

Radar radiation source recognition is critical for the reliable operation of radar communication systems. However, in increasingly complex electromagnetic environments, traditional identification methods face significant limitations. These methods often struggle with high noise levels and diverse modulation types, making it difficult to maintain accuracy, especially when the Signal-to-Noise Ratio (SNR) is low or the available training data are limited. These difficulties are further intensified by the necessity to generalize in environments characterized by a substantial quantity of noisy, low-quality signal samples while being constrained by a limited number of desirable high-quality training samples. To more effectively address these issues, this paper proposes a novel approach utilizing Model-Agnostic Meta-Learning (MAML) to enhance model adaptability in few-shot learning scenarios, allowing the model to quickly learn with limited data and optimize parameters effectively. Furthermore, a multimodal fusion neural network, DCFANet, is designed, incorporating residual blocks, squeeze and excitation blocks, and a multi-scale CNN, to fuse I/Q waveform data and time–frequency image data for more comprehensive feature extraction. Our model enables more robust signal recognition, even when the signal quality is severely degraded by noise or when only a few examples of a signal type are available. Testing on 13 intra-pulse modulated signals in an Additive White Gaussian Noise (AWGN) environment across SNRs ranging from −20 to 10 dB demonstrated the approach’s effectiveness. Particularly, under a 5−way5−shot setting, the model achieves high classification accuracy even at −10dB SNR. Our research underscores the model’s ability to address the key challenges of radar emitter signal recognition in low-SNR and data-scarce conditions, demonstrating its strong adaptability and effectiveness in complex, real-world electromagnetic environments.

A Time-Frequency Domain Analysis Method for Variable Frequency Hopping Signal.

A variable frequency hopping (VFH) signal is a kind of frequency hopping (FH) signal that varies both in frequency and dwell time. However, in radio surveillance, the existing methods for unidentified signals using VFH cannot be effectively handled. In this paper, we proposed an improved joint analysis method based on time-frequency domain features, which adopts multi-level processing to solve the time-frequency domain feature analysis problem of the VFH signal. First, the received signal is pre-processed by Short-Time Fourier Transform (STFT) and binarization, and a highly discriminative time-frequency image is obtained; then, the fixed frequency signal is removed based on the feature of connected domains, and the conventional frequency hopping (CFH) signal is removed by density-based spatial clustering of applications with noise (DBSCAN); finally, the overlapping region is cropped by the joint energy peak time-domain continuity properties. After the above multi-level joint processing method, the problem of VFH signal processing is effectively solved. The simulation result shows that the Mean Square Error (MSE) between the output results and the time-frequency image of the original VFH signal tends to be close to 0 when the Signal-to-Noise ratio (SNR) is 5 dB.

Unsupervised motor incipient fault detection using lightweight network and orthogonal low-rank embedding

Abstract Electrical motor is a key component in industrial systems. Detecting incipient fault of motor is critical to system reliability. However, the characteristics of faults and normal stages are difficult to distinguish, Meanwhile, the lightweight requirements of the model and the imbalance of the samples also make incipient fault detection challenged. To address these problems, this paper proposes an unsupervised fault detection method combines lightweight network and orthogonal low-rank embedding (OLE). The raw signals are firstly transformed into time-frequency images and fed into lightweight convolution network for feature extraction. Then, the features are clustered into orthogonal subspaces to enhance inter-class separability. Finally, a detection module based on distance metric is designed to identify the incipient fault of motor. The effectiveness of the proposed method is validated on four industrial motor dataset and compared with other methods.

Mechanism-Based Fault Diagnosis Deep Learning Method for Permanent Magnet Synchronous Motor.

As an important driving device, the permanent magnet synchronous motor (PMSM) plays a critical role in modern industrial fields. Given the harsh working environment, research into accurate PMSM fault diagnosis methods is of practical significance. Time-frequency analysis captures the rich features of PMSM operating conditions, and convolutional neural networks (CNNs) offer excellent feature extraction capabilities. This study proposes an intelligent fault diagnosis method based on continuous wavelet transform (CWT) and CNNs. Initially, a mechanism analysis is conducted on the inter-turn short-circuit and demagnetization faults of PMSMs, identifying and displaying the key feature frequency range in a time-frequency format. Subsequently, a CNN model is developed to extract and classify these time-frequency images. The feature extraction and diagnosis results are visualized with t-distributed stochastic neighbor embedding (t-SNE). The results demonstrate that our method achieves an accuracy rate of over 98.6% for inter-turn short-circuit and demagnetization faults in PMSMs of various severities.

Specific Emitter Identification Algorithm Based on Time–Frequency Sequence Multimodal Feature Fusion Network

Specific emitter identification is a challenge in the field of radar signal processing. Its aims to extract individual fingerprint features of the signal. However, early works are all designed using either signal or time–frequency image and heavily rely on the calculation of hand-crafted features or complex interactions in high-dimensional feature space. This paper introduces the time–frequency multimodal feature fusion network, a novel architecture based on multimodal feature interaction. Specifically, we designed a time–frequency signal feature encoding module, a wvd image feature encoding module, and a multimodal feature fusion module. Additionally, we propose a feature point filtering mechanism named FMM for signal embedding. Our algorithm demonstrates high performance in comparison with the state-of-the-art mainstream identification methods. The results indicate that our algorithm outperforms others, achieving the highest accuracy, precision, recall, and F1-score, surpassing the second-best by 9.3%, 8.2%, 9.2%, and 9%. Notably, the visual results show that the proposed method aligns with the signal generation mechanism, effectively capturing the distinctive fingerprint features of radar data. This paper establishes a foundational architecture for the subsequent multimodal research in SEI tasks.

A zero-shot quantitative evaluation model for subsurface defects size based on ultrasonic nondestructive testing

Existing defect quantitative evaluation models requires collecting and labeling all types of defect data for training. However, such models have a poor ability for new defect, and collecting and labeling data is a costly task. A zero-shot evaluation model is proposed to solve the questions. First, a Wavelet Packet Transform-based method is developed to convert defect ultrasonic signals into time–frequency images. Second, we define defect semantics from length, width, and height to describe defects of different sizes. Third, the ResNet with embedded Brownian Distance Covariance matrix and multi-dimension parallel framework is designed to extract the image features. Then, an autoencoder is trained to embed the defect semantic into the feature space. Finally, the dimension of defect is evaluated by measuring the similarity between the features and the category centroids. Experimental results reveal that our model has better ability to evaluate the dimensions of new defect compared to others models.

Compact convolutional transformers- generative adversarial network for compound fault diagnosis of industrial robot

The safe operation of Industrial robots is a major concern in intelligent manufacturing. Accurate compound fault diagnosis is essential to the safe operation of industrial robots, while it is challenging to achieve since the compound fault samples are hard to be collected. Generative adversarial network (GAN) is a useful tool for addressing the data imbalance issue. However, the computation efficiency of GAN in addressing the data imbalance issue has not been investigated. Hence, this study proposes a lightweight GAN named compact convolutional Transformers-GAN (CCT-GAN) to alleviate the data imbalance issue in compound fault diagnosis modelling. Firstly, the feedback current signals collected from the industrial robot are transformed into time-frequency images via continuous wavelet transformation (CWT). Secondly, CCT-GAN is designed to achieve high-quality fake data generation and compound fault diagnosis modelling without large computational costs. Thirdly, the relation between a single fault and the compound fault is considered in the compound fault diagnosis modelling via multi-hot representation to alleviate the data imbalance issue. An experimental study based on the real-world compound fault dataset of industrial robots reveals that the proposed CCT-GAN shows merits in compound fault diagnosis modelling in comparison with the prevailing algorithms. The results indicate that CCT-GAN can performance of compound fault diagnosis when only 100 data samples from each compound fault category are available.

Spectrum Sensing Method Based on STFT-RADN in Cognitive Radio Networks.

To address the common issues in traditional convolutional neural network (CNN)-based spectrum sensing algorithms in cognitive radio networks (CRNs), including inadequate signal feature representation, inefficient utilization of feature map information, and limited feature extraction capabilities due to shallow network structures, this paper proposes a spectrum sensing algorithm based on a short-time Fourier transform (STFT) and residual attention dense network (RADN). Specifically, the RADN model improves the basic residual block and introduces the convolutional block attention module (CBAM), combining residual connections and dense connections to form a powerful deep feature extraction structure known as residual in dense (RID). This significantly enhances the network's feature extraction capabilities. By performing STFT on the received signals and normalizing them, the signals are converted into time-frequency spectrograms as network inputs, better capturing signal features. The RADN is trained to extract abstract features from the time-frequency images, and the trained RADN serves as the final classifier for spectrum sensing. Experimental results demonstrate that the STFT-RADN spectrum sensing method significantly improves performance under low signal-to-noise ratio (SNR) conditions compared to traditional deep-learning-based methods. This method not only adapts to various modulation schemes but also exhibits high detection probability and strong robustness.

Signal simulation based on improved doppler filter

Abstract In satellite communication, the signal is transmitted through a wireless channel, and distorted signals will appear under the influence of the actual environment. In this paper, an improved Doppler filter is proposed by mathematically fitting the actual signals, and the similarity of the time-frequency map of the signals output by this improved filter reaches 94.357% of the standard value, realizing the simulation of a kind of aberration appearing in the actual environment on the time-frequency image and finally realizing the self-adaptation.

An Integrated Hybrid Convolutional-Support Vector Machine (CSVM) Model with the CWT Method for Hearing Disorder Detection using AEP Signals

Hearing disorder is the most widespread sensory disability worldwide, impairing human communication and learning. An early and accurate hearing disorder detection system using an electroencephalogram (EEG) is the appropriate technique for dealing with this concern. The most significant modality for diagnosing hearing deficiency among EEG control signals is the auditory evoked potential (AEP), which is generated in the cortical region of the brain through auditory stimulus. This study aims to develop an efficient approach for detecting hearing disorders. For this purpose, this study has designed a hybrid model based on the convolutional operation of CNN and the SVM classifier. Initially, the CWT method was utilized to transform the raw AEP signals into time-frequency images. Then, the extracted features were classified using the proposed CSVM model. To test the robustness of the proposed model, this study also implemented a convolutional neural network (CNN) and support vector machine (SVM) with the same parameters. The experimental results with the hybrid CSVM model showed superior performance on the publicly available AEP dataset by achieving 94.48% testing accuracy, 96.40% precision, 92.96% recall, 94.65% F1 score, 88.95% Cohen Kappa score, which indicates that the proposed hybrid model could be used for early hearing disorder detection. Future enhancements will concentrate on identifying different hearing-signal-based data and the cloud-based, automated classification of AEP signals.

Unbalanced Data-Based Fault Diagnosis Method of Bearing Utilizing Time-Frequency DCGAN Processing

Aiming at the unbalanced datasets of bearing fault samples, a fault diagnosis method based on time-frequency conversion and processing of bearing vibration signals based on deep convolutional generative adversarial network (DCGAN) is proposed in this paper. Firstly, through short-time Fourier transform(STFT), the vibration signals are converted into time-frequency images, and then time-frequency images are input into DCGAN to expand the fault samples.Secondly, the expanded fault samples are evaluated for image quality through the comprehensive method of peak signal-to-noise ratio(PSNR) and structural similarity(SSIM). Thirdly, using the Canny edge detection algorithm to extract features of the time-frequency image, and using the obtained binary image as the feature.Finally, k-nearest neighbor algorithm is used for classification to testify the superiority of time-frequency DCGAN processing. The experimental results show that the expanded samples can effectively improve the imbalance of the samples and improve the accuracy of fault diagnosis of bearing.

Research on a Bearing Fault Diagnosis Method Based on an Improved Wasserstein Generative Adversarial Network

In this paper, an advanced Wasserstein generative adversarial network (WGAN)-based bearing fault diagnosis approach is proposed to bolster the diagnostic efficacy of conventional WGANs and tackle the challenge of selecting optimal hyperparameters while reducing the reliance on sample labeling. Raw vibration signals undergo continuous wavelet transform (CWT) processing to generate time–frequency images that align with the model’s input dimensions. Subsequently, these images are incorporated into a region-based fully convolutional network (R-FCN), substituting the traditional discriminator for feature capturing. The WGAN model is refined through the utilization of the Bayesian optimization algorithm (BOA) to optimize the generator and discriminator’s semi-supervised learning loss function. This approach is verified using the Case Western Reserve University (CWRU) dataset and a centrifugal pump failure experimental dataset. The results showed improvements in data input generalization and fault feature extraction capabilities. By avoiding the need to label large quantities of sample data, the diagnostic accuracy was improved to 98.9% and 97.4%.

A Gradient-Based Optimizer Method for Improving the Feature Extraction Capability of Network Architecture and Its Application to Condition Monitoring of Rotating Machinery

Aiming at the problem of inaccurate feature extraction under the massive data of rotating machinery, this paper proposes a feature extraction capability enhancement method based on Gradient-Based Optimizer for trusted network architecture to monitor the conditions of bearings, which firstly transforms the original signal into a wavelet time-frequency image and visualizes the high-dimensional and difficult-to-visualize data in low dimensions. Then, different convolutional neural network architectures are tested by implanting different noise intensities to determine the most robust network architecture. Secondly, this paper adopts the grid method as well as the swarm optimization algorithm to guide the parameters of the model to achieve the best feature extraction effect, compares the optimization effect of several swarm optimization algorithms horizontally, and determines the most effective GBO algorithm among them. Finally, to improve the credibility of the best network model, this paper combines the grad-cam method with the trusted fusion method to improve the accuracy and reliability of the model. Validated by the bearing data of Case Western Reserve University, it shows that the proposed model in this paper has good credibility and accuracy.

Intra-Pulse Modulation Recognition of Radar Signals Based on Efficient Cross-Scale Aware Network.

Radar signal intra-pulse modulation recognition can be addressed with convolutional neural networks (CNNs) and time-frequency images (TFIs). However, current CNNs have high computational complexity and do not perform well in low-signal-to-noise ratio (SNR) scenarios. In this paper, we propose a lightweight CNN known as the cross-scale aware network (CSANet) to recognize intra-pulse modulation based on three types of TFIs. The cross-scale aware (CSA) module, designed as a residual and parallel architecture, comprises a depthwise dilated convolution group (DDConv Group), a cross-channel interaction (CCI) mechanism, and spatial information focus (SIF). DDConv Group produces multiple-scale features with a dynamic receptive field, CCI fuses the features and mitigates noise in multiple channels, and SIF is aware of the cross-scale details of TFI structures. Furthermore, we develop a novel time-frequency fusion (TFF) feature based on three types of TFIs by employing image preprocessing techniques, i.e., adaptive binarization, morphological processing, and feature fusion. Experiments demonstrate that CSANet achieves higher accuracy with our TFF compared to other TFIs. Meanwhile, CSANet outperforms cutting-edge networks across twelve radar signal datasets, providing an efficient solution for high-precision recognition in low-SNR scenarios.

A New Cross-Domain Motor Fault Diagnosis Method Based on Bimodal Inputs

Electric motors are indispensable electrical equipment in ships, with a wide range of applications. They can serve as auxiliary devices for propulsion, such as air compressors, anchor winches, and pumps, and are also used in propulsion systems; ensuring the safe and reliable operation of motors is crucial for ships. Existing deep learning methods typically target motors under a specific operating state and are susceptible to noise during feature extraction. To address these issues, this paper proposes a Resformer model based on bimodal input. First, vibration signals are transformed into time–frequency diagrams using continuous wavelet transform (CWT), and three-phase current signals are converted into Park vector modulus (PVM) signals through Park transformation. The time–frequency diagrams and PVM signals are then aligned in the time sequence to be used as bimodal input samples. The analysis of time–frequency images and PVM signals indicates that the same fault condition under different loads but at the same speed exhibits certain similarities. Therefore, data from the same fault condition under different loads but at the same speed are combined for cross-domain motor fault diagnosis. The proposed Resformer model combines the powerful spatial feature extraction capabilities of the Swin-t model with the excellent fine feature extraction and efficient training performance of the ResNet model. Experimental results show that the Resformer model can effectively diagnose cross-domain motor faults and maintains performance even under different noise conditions. Compared with single-modal models (VGG-11, ResNet, ResNeXt, and Swin-t), dual-modal models (MLP-Transformer and LSTM-Transformer), and other large models (Swin-s, Swin-b, and VGG-19), the Resformer model exhibits superior overall performance. This validates the method’s effectiveness and accuracy in the intelligent recognition of common cross-domain motor faults.