Time-frequency Image Research Articles

CNN for a Regression Machine Learning Algorithm for Predicting Cognitive Impairment using qEEG

AbstractBackgroundElectroencephalogram (EEG) signals give detailed information on the electrical brain activities occurring in the cerebral cortex. They are used to study brain‐related disorders such as mild cognitive impairment (MCI), and dementia. Brain signals obtained using an EEG machine can be a neurophysiological biomarker for early diagnosis of dementia through quantitative EEG (qEEG) analysis. This paper proposes a machine learning methodology to detect MCI and dementia from qEEG time‐frequency (TF) images of the subjects in an eyes‐closed resting state (ECR).MethodThe dataset consisted of 16,910 TF images from 890 subjects: 269 healthy controls (HC), 356 MCI, and 265 dementia. First, EEG signals were transformed into TF images using a Fast Fourier Transform (FFT) containing different event‐rated changes of frequency sub‐bands preprocessed from the EEGlab toolbox in the MATLAB R2021a environment software. The preprocessed TF images were applied in a convolutional neural network (CNN) with adjusted parameters. For classification, the computed image features were concatenated with age data and went through the feed‐forward neural network (FFN).ResultThe trained models’, HC vs. MCI, HC vs. dementia, and HC vs. CASE (MCI + dementia), performance metrics were evaluated based on the test dataset of the subjects. The accuracy, sensitivity, and specificity were evaluated: HC vs. MCI was 83%, 93%, and 73%, HC vs. dementia was 81%, 80%, and 83%, and HC vs. CASE (MCI + dementia) was 88%, 80%, and 90%, respectively.ConclusionThe proposed models trained with TF images and age can be used to assist clinicians as a biomarker in detecting cognitively impaired subjects at an early stage in clinical sectors.

Deep residual learning for acoustic emission source localization in A steel-concrete composite slab

Large errors can be introduced in traditional acoustic emission (AE) source localization methods using extracted signal features such as arrival time difference. This issue is obvious in the case of irregular structural geometries, complex composite structure types or presence of cracks in wave travel paths. In this study, based on a novel deep learning algorithm called deep residual network (DRN), a structural health monitoring (SHM) strategy is proposed for AE source localization through classifying and recognizing the AE signals generated in different sub-regions of critical areas in structures. Hammer hits and pencil-leak break (PLB) tests were carried out on a steel-concrete composite slab specimen to register time-domain AE signals under multiple structural damage conditions. The obtained time-domain AE signals were then converted into time-frequency images as inputs for the proposed DRN architecture using the continuous wavelet transform (CWT). The DRNs were trained, validated and tested by AE signals generated from different source types at various damage states of the slab specimen. The proposed DRN architecture shows an effective potential for AE source localization. The results show that the DRN models pre-trained by the AE signals obtained in the undamaged specimen are able to accurately classify and identify the locations of different types of AE sources with 3–4.5 cm intervals even when multiple cracks with widths up to 4–6 mm are present in the wave travel paths. Moreover, the influence factors on the model performance are investigated, including structural damage conditions, sensor-to-source distances and AE sensor mounting positions; in accordance with the parametric analyses, recommendations are proposed for the engineering application of the proposed SHM strategy.

YFDM: YOLO for detecting Morse code

With the increasing complexity of the shortwave communication environment, the efficiency and accuracy of the manual detection of Morse code no longer meet actual needs. Therefore, this paper proposes a Morse code detection algorithm called YFDM. For the time–frequency image of the received signal, a combination module of deformable convolution and C3 is used to enhance the backbone network’s attention to the abstract semantics and location information of Morse code. GSConv and VOV-GSCSP modules are used to build a lightweight neck network. Finally, the confidence propagation cluster (CP-Cluster) algorithm is used to filter the detection frame. In an ablation experiment, the parameters and giga floating-point operations per second (GFLOPs) of YFDM were 5.961 M and 9.74 G, respectively, 15.11% and 38.9% less than those of YOLOv5. Moreover, when WIoUv1 was used as the loss function of the bounding box, the AP0.5:0.95 and frames per second (FPS) values of the algorithm reached the highest values, 0.68 and 72.4. The experimental results indicate that the algorithm can effectively reduce the weight of the model while ensuring the detection accuracy and inference speed.

Wavelet transform and deep learning-based obstructive sleep apnea detection from single-lead ECG signals.

Sleep apnea is a common sleep disorder. Traditional testing and diagnosis heavily rely on the expertise of physicians, as well as analysis and statistical interpretation of extensive sleep testing data, resulting in time-consuming and labor-intensive processes. To address the problems of complex feature extraction, data imbalance, and low model capacity, we proposed an automatic sleep apnea classification model (CA-EfficientNet) based on the wavelet transform, a lightweight neural network, and a coordinated attention mechanism. The signal is converted into a time-frequency image by wavelet transform and put into the proposed model for classification. The effects of input time window, wavelet transform type and data balancing on the classification performance are considered, and a cost-sensitive algorithm is introduced to more accurately distinguish between normal and abnormal breathing events. PhysioNet apnea ECG database was used for training and evaluation. The 3-min Frequency B-Spline wavelets transform of ECG signal was carried out, and Dice Loss was used to train the classification model of sleep breathing. The classification accuracy was 93.44%, sensitivity was 88.9%, specificity was 96.2% and most indexes were better than other related work.

Hierarchical dynamic aggregation graph modeling of vibration for semi-supervised fault diagnosis of power transformer

Power equipment fault diagnostics is a critical aspect of ensuring the stability of the power grid system. However, it presents substantial challenges in obtaining labeled data that is spatial-temporal, multi-scale, and multi-domain, low noise for effective fault analysis and diagnosis. To address this issue, we propose a novel approach called hierarchical dynamic aggregation graph (HDAG) modeling for self-supervised fault diagnosis of power transformers using vibration data. Firstly, HDAG focuses on modeling the spatial and temporal correlations within the fault vectors, before converting them into time–frequency images for visualization. Secondly, our proposed fault diagnosis approach, (comprising the ST-sparse swin-transformer and multi-domain transformer fusion module), is integrated into the methodology. The ST-sparse swin transformer incorporates soft threshold modules, enabling the retention of relevant information while discarding irrelevant information. The multi-domain transformer fusion is designed to utilize the intra-domain and inter-domain signal characteristics to achieve a comprehensive feature representation. Finally, we present case studies based on experimental data that demonstrate the feasibility and effectiveness of our approach. Comparative evaluations against eight state-of-the-art techniques validate the improved information representation and diagnostic capabilities of our recommended strategy.

A Fault Diagnosis Strategy for Analog Circuits with Limited Samples Based on the Combination of the Transformer and Generative Models.

As a pivotal integral component within electronic systems, analog circuits are of paramount importance for the timely detection and precise diagnosis of their faults. However, the objective reality of limited fault samples in operational devices with analog circuitry poses challenges to the direct applicability of existing diagnostic methods. This study proposes an innovative approach for fault diagnosis in analog circuits by integrating deep convolutional generative adversarial networks (DCGANs) with the Transformer architecture, addressing the problem of insufficient fault samples affecting diagnostic performance. Firstly, the employment of the continuous wavelet transform in combination with Morlet wavelet basis functions serves as a means to derive time-frequency images, enhancing fault feature recognition while converting time-domain signals into time-frequency representations. Furthermore, the augmentation of datasets utilizing deep convolutional GANs is employed to generate synthetic time-frequency signals from existing fault data. The Transformer-based fault diagnosis model was trained using a mixture of original signals and generated signals, and the model was subsequently tested. Through experiments involving single and multiple fault scenarios in three simulated circuits, a comparative analysis of the proposed approach was conducted with a number of established benchmark methods, and its effectiveness in various scenarios was evaluated. In addition, the ability of the proposed fault diagnosis technique was investigated in the presence of limited fault data samples. The outcome reveals that the proposed diagnostic method exhibits a consistently high overall accuracy of over 96% in diverse test scenarios. Moreover, it delivers satisfactory performance even when real sample sizes are as small as 150 instances in various fault categories.

Rolling bearing fault diagnosis method by using feature extraction of convolutional time-frequency image

The vibration signal of the bearing is mostly used in the fault diagnosis research of rolling bearings, and the vibration signal has the characteristics of strong non-smoothness and is easy to be disturbed by noise. This characteristic of the vibration signal has a strong adverse effect on the fault diagnosis of bearings. To improve the accuracy of bearing fault diagnosis, a rolling bearing fault diagnosis method based on the combination of a short-time Fourier transform and a convolutional neural network is proposed. Firstly, a short-time Fourier transform is performed on the original vibration signal of the bearing to obtain the time-frequency image of the vibration signal. Then, the dual model of 2D convolutional neural network which can obtain abundant fault information is built. Secondly, the obtained time-frequency image is image compressed to fit the proposed model. Finally, the images are input into the network for fault diagnosis, and the fault classification accuracy reaches 100%. The proposed model is tested by two methods: adding Gaussian noise to the time-frequency image and testing the model using bearing data from other working conditions, and the results show that the proposed method has good noise immunity and generalization, which can provide an effective diagnostic scheme for bearing fault diagnosis.

TFF-CNN: Distributed optical fiber sensing intrusion detection framework based on two-dimensional multi-features

Distributed fiber optic vibration sensing system has been widely used in safety monitoring with distinct advantages, and the feature extraction and classification methods of fiber optic signals directly determine the real-time performance and reliability of the monitoring system. The existing research feature extraction methods are unidimensional and time consuming, which cannot balance the goals of high accuracy and low time consumption for safety monitoring systems. In this paper, we propose an efficient recognition framework for fusing signal time–frequency features, called TFF-CNN, based on Gramian Angular Difference Fields(GADF) and FFT co-generation matrix(FFTT) to manually extract two-dimensional time–frequency image features of fiber optic signals, taking the significant advantages of CNN in image processing and combining a two-channel model and a fusion module that simulates human decision-making behavior. The accuracy of TFF-CNN is 99.30 % and the detection response time is only 0.6 s. Compared with other methods in this field, TFF-CNN has the advantages of low false alarm rate and short time consumption, which is more suitable for deployment in security monitoring field with distributed fiber optic sensing system.Index Terms: Distributed optical fiber vibration sensing system(DVS), Two-dimensional multi-feature, CNN, intrusion detection, real-time monitoring.

A novel framework for the diagnosis of Parkinson’s disease using transfer learning with RESNET50 and SVM classifier

<span>Most Parkinson’s patients exhibit voice disorders in the early phases of the condition. Recent study on Parkinson’s disease (PD) has concentrated on identifying speech problems from pronunciation of vowels with people affected by this disease. Proposed algorithm offers analysis of time-frequency images using transfer learning methods with support vector machine (SVM) for classification of PD affected with healthy controls using residual network 50. PD morphology is preserved in 2-dimensional time-frequency graphs <br /> that were helpful in implementing the proposed approach. The technique employs a hybrid HT/Wigner-Ville distribution to convert one-dimensional PD soundtracks to two-dimensional time-frequency (TF) graphs. After implementation of the proposed approach, classification of healthy and unhealthy controls is done using a pre-trained ResNet50 and the result is further improved through transfer learning. The features are extracted by passing preprocessed 2-D time-frequency diagrams through ResNet50’s FC1000 layer and trained using a binary nonlinear SVM classifier. The training process with 5-fold cross-validation (CV) got accuracy of 95.07% and in testing; it reached 92.13%, attaining better results.</span>

ELoran signal message prediction algorithm based on Alexnet‐ECA

AbstractEnhanced Loran (eLoran) is the most important backup system for Global Navigation Satellite System (GNSS). However, most current eLoran demodulation methods are based on digital signal processing, which may produce large errors under the influence of strong interference or noise. The authors propose an eLoran signal message prediction algorithm based on deep learning. Using time‐frequency (TF) analysis convert one‐dimensional signal into two‐dimensional high‐resolution image. It improves Alexnet by adjusting the convolutional layer structure, adding an attention module, and reducing the pooling layer window to construct the Alexnet‐ECA network. The network is used to identify TF image features and predict signal message. The experimental results show that novel method for eLoran signal message prediction based on Alexnet‐ECA can achieve high accuracy and robustness in various SNR scenarios. The accuracy is more than 90% at SNR greater than 10db and not less than 80% even in the complex environments. Compared with Resnet34 and VGG16 networks, Alexnet‐ECA demonstrates its efficiency and effectiveness using real measurement data. This method can enhance the performance and reliability of eLoran receivers and applications.

Deep Learning-Based Classification of Epileptic Electroencephalography Signals Using a Concentrated Time-Frequency Approach.

ConceFT (concentration of frequency and time) is a new time-frequency (TF) analysis method which combines multitaper technique and synchrosqueezing transform (SST). This combination produces highly concentrated TF representations with approximately perfect time and frequency resolutions. In this paper, it is aimed to show the TF representation performance and robustness of ConceFT by using it for the classification of the epileptic electroencephalography (EEG) signals. Therefore, a signal classification algorithm which uses TF images obtained with ConceFT to feed the transfer learning structure has been presented. Epilepsy is a common neurological disorder that millions of people suffer worldwide. Daily lives of the patients are quite difficult because of the unpredictable time of seizures. EEG signals monitoring the electrical activity of the brain can be used to detect approaching seizures and make possible to warn the patient before the attack. GoogLeNet which is a well-known deep learning model has been preferred to classify TF images. Classification performance is directly related to the TF representation accuracy of the ConceFT.The proposed method has been tested for various classification scenarios and obtained accuracies between 95.83% and 99.58% for two and three-class classification scenarios. High results show that ConceFT is a successful and promising TF analysis method for non-stationary biomedical signals.

Voice pathology detection using optimized convolutional neural networks and explainable artificial intelligence-based analysis

This article proposes a noninvasive computer-aided assessment approach based on optimized convolutional neural network for healthy and pathological voice detection. Firstly, the input voice samples are first transformed into mel-spectrogram time-frequency visual representations and fed for training the CNN model. The time-frequency image captures inherent speech variations beneficial for healthy and pathological voice sample detection. The weights and biases of trained CNN network are further optimized using artificial bee colony (ABC) optimization algorithm resulting in optimum CNN network employed for testing unseen data. The proposed approach is evaluated using three popular and publicly available datasets: SVD, AVPD and VOICED. Experimental results emphasize that proposed ABC optimized CNN model shows improved accuracy performance by 1.02% compared to conventional CNN network illustrating data-independent discriminative representation ability. Finally, gradient-weighted class activation mapping (Grad-CAM) explainable artificial intelligence (XAI) is utilized to make the decision understandable.

Fault identification of rolling bearings under linear varying speed based on the slope features of time–frequency ridges

Under the condition of linear varying speed regulation, the vibration signal of the rolling bearing is non-stationary, and its fault frequency is time-varying, which makes it difficult to extract the bearing fault characteristics. In order to better improve the fault identification accuracy of rolling bearing, two slope characteristic indicators of time–frequency ridges (TFRs) based on time–frequency images (TFIs) are proposed, which are called pseudo-slope and pseudo-angle. The intra-class state-aware stability and inter-class state-aware sensitivity of the characteristic indicators are verified by simulation and experiment. At the same time, the robustness of the slope features of TFRs and the Tamura features under the influence of time-varying, noise and image cropping are compared, and it is proved that the stability of the slope features of TFRs are better than that of the Tamura features. In the case of one single-index identification, the fault recognition rate of pseudo-slope and pseudo-angle features is higher than that of Tamura single-index features. On the basis of single-index identification, a new feature fusion by numerical summation of pseudo-slope feature and Tamura contrast feature is proposed, and the fault identification accuracy of the fused feature indicator is 98.61%, which realizes high-precision identification of rolling bearing faults.

Deep transfer learning rolling bearing fault diagnosis method based on convolutional neural network feature fusion

Current deep-learning methods are often based on significantly large quantities of labeled fault data for supervised training. In practice, it is difficult to obtain samples of rolling bearing failures. In this paper, a transfer learning-based feature fusion convolutional neural network approach for bearing fault diagnosis is proposed. Specifically, the raw vibration signal features and the corresponding time-frequency image features of the input data are extracted by a one-dimensional convolutional neural network and a pre-trained ConvNeXt, respectively, and connected by a feature fusion strategy. Then, the fine-tuning method based on transfer learning can effectively reduce the reliance on labeled samples in the target domain. A wide convolution kernel is introduced in the time-domain signal feature extraction to increase the receptive field, which is combined with the channel attention mechanism to further optimize the feature quality. Finally, two common bearing datasets are utilized for fault diagnosis experiments. The experimental results show that the proposed model achieves an average accuracy of more than 98.63% in both cross-working conditions and cross-device diagnosis tasks. Meanwhile, anti-noise experiments and ablation experiments further validate the accuracy and robustness of the proposed method.

Toward practical tool wear prediction paradigm with optimized regressive Siamese neural network

Data-driven models, such as deep learning and transfer learning algorithms, have achieved leading results in essential tool condition monitoring (TCM) during manufacturing process. However, these models perform differently under available dataset with various sample scales in these factories, leading to poor application extensiveness in engineering. Besides, Siamese neural network (SNN) has been applied in machine condition identification, but how to apply the model on tool wear regression has not attracted much attention. Therefore, a novel SNN framework, called global frequency context channel optimized regressive SNN — support vector regression (GFCC-RSNN-SVR), is proposed and optimized to predict tool wear values practically. Firstly, Stockwell transform is utilized for time–frequency image generation from raw signal. Then, regressive SNN framework is constructed based on convolutional neural blocks and enhanced by novel attention mechanism layer. After model training, support vector regression (SVR) submodel is constructed for final tool wear prediction. Moreover, milling experiments has been designed and conducted, and the dataset and public ones have been chosen for the following cases study. Further comparison experiments have validated the brilliant performance of the proposed model, and the effectiveness of Stockwell transform, RSNN framework, and the attention mechanism, regardless of training sample scales.

Aggregated-attention deformable convolutional network for few-shot SAR jamming recognition

This work simultaneously addresses the challenges of unseen classes and low-data problems on synthetic aperture radar jamming recognition (SAR-JR). Currently, very few studies have tackled both challenges. Inspired by the success of few-shot learning, which learns a robust model from a few instances, we formulate SAR-JR as a few-shot task in a metric-learning framework to alleviate the above challenges. Against the jamming features with significant dispersion and complex geometric transformations, as well as feature obscuration in time–frequency images (TF), we propose an aggregated-attention deformable convolutional network (A2-DCNet) framework consisting of an aggregated-attention deformable convolutional module (A2-DC-Module) and a prototype classification module based on polynomial loss (PolyLoss-PC-Module). The former learns informative and refined embeddings from the TF images, while the latter performs the SAR-JR in an embedding space by calculating distances to prototypes of each class. Specifically, the modulated deformable convolution of the A2-DC-Module can capture long-range spatial contextual information from a global perspective, while the aggregated attention is designed to refine the representations of obscured features in the TF images. To further optimize the framework, we introduce a novel PolyLoss and customize the optimal form for our model to learn an embedding space with robust inter-class separability. Finally, to realize few-shot SAR-JR tasks, we simulate a novel dataset called JamSet. Extensive experimental results on our dataset have demonstrated substantial improvement of our proposed A2-DCNet method over the benchmarks.

Time-series modelling of harmful cyanobacteria blooms by convolutional neural networks and wavelet generated time-frequency images of environmental driving variables

Early warning systems for harmful cyanobacterial blooms (HCBs) that enable precautional control measures within water bodies and in water works are largely based on inferential time-series modelling. Among deep learning techniques, convolutional neural networks (CNNs) are widely applied for recognition of pictorial, acoustic and thermal images. Time-frequency images of environmental drivers generated by wavelets may provide crucial signals for modelling of HCBs to be recognized by CNNs. This study applies CNNs for time-series modelling of HCBs of Microcystis sp. in four South Korean rivers between 2016 and 2022 by means of time-frequency images of environmental drivers within the lead time of HCBs. After estimating the cardinal dates of beginning, peak, and ending of HCBs, wavelet analysis identified key drivers by phase analysis and generated time-frequency images of the drivers within the cardinal dates for 3, 4 and 5 years. Performances of CNNs were compared in terms of four determinants of input images: methods of estimating critical timings, the number of segments, time-series continuity, and image size. The resulting CNNs predicted high or low intensities of HCBs with a mean accuracy of 97.79 ± 0.06% and F1-score 97.49 ± 0.06% for training dataset, and a mean accuracy of 95.01 ± 0.06% and F1-score 93.30 ± 0.07% for testing dataset. Predictions of Microcystis abundances by CNNs achieved a mean MSE of 2.58 ± 2.46 and a mean R2 of 0.78 ± 0.20 for training, and a mean MSE of 2.76 ± 2.42 and a mean R2 of 0.55 ± 0.20 for testing dataset. Precipitation and discharge appeared to be the best performing drivers for qualitative and quantitative predictions of HCBs pointing at the nonstationary nature of river habitats. This study highlights the opportunities of time-series modelling by CNNs driven by wavelet generated time-frequency images of key environmental variables for forecasting of HCBs.

Bearing fault diagnosis based on CNN-BiLSTM and residual module

Bearings are key components of rotating machinery, and their fault diagnosis is essential for machinery operation. Bearing vibration signals belong to time series data, but traditional convolutional neural networks (CNNs) or recurrent neural networks cannot fully extract the fault features from these signals. To address the insufficient feature extraction and poor noise resistance, this paper proposes a fault diagnosis model based on continuous wavelet transform (CWT), CNN with channel attention, bidirectional long short-term memory network (BiLSTM) and residual module. Firstly, a parallel dual-path feature extraction mechanism is constructed which takes time-domain signals and time–frequency images transformed via CWT as the input respectively. Then BiLSTM extracts the time features of the signal as one path, and the CNN with efficient channel attention extracts the spatial features as the other path. This parallel neural network contributes to better feature extraction. Then, the residual module is applied to extract the global features to further improve the feature extraction ability and noise immunity. The experimental results demonstrate that the proposed model on the Case Western Reserve University dataset has better diagnostic accuracy under different working conditions and different signal-to-noise ratios than other methods. In addition, the model shows good generalization performance on Jiangnan University dataset.

Power quality recognition in noisy environment employing deep feature extraction from cross stockwell spectrum time–frequency images

Power quality recognition in noisy environment employing deep feature extraction from cross stockwell spectrum time–frequency images

MultiChannelSleepNet: A Transformer-Based Model for Automatic Sleep Stage Classification With PSG.

Automatic sleep stage classification plays an essential role in sleep quality measurement and sleep disorder diagnosis. Although many approaches have been developed, most use only single-channel electroencephalogram signals for classification. Polysomnography (PSG) provides multiple channels of signal recording, enabling the use of the appropriate method to extract and integrate the information from different channels to achieve higher sleep staging performance. We present a transformer encoder-based model, MultiChannelSleepNet, for automatic sleep stage classification with multichannel PSG data, whose architecture is implemented based on the transformer encoder for single-channel feature extraction and multichannel feature fusion. In a single-channel feature extraction block, transformer encoders extract features from time-frequency images of each channel independently. Based on our integration strategy, the feature maps extracted from each channel are fused in the multichannel feature fusion block. Another set of transformer encoders further capture joint features, and a residual connection preserves the original information from each channel in this block. Experimental results on three publicly available datasets demonstrate that our method achieves higher classification performance than state-of-the-art techniques. MultiChannelSleepNet is an efficient method to extract and integrate the information from multichannel PSG data, which facilitates precision sleep staging in clinical applications.