Time-frequency Feature Extraction Research Articles

EEG Emotion Recognition Network Based on Attention and Spatiotemporal Convolution.

Human emotions are complex psychological and physiological responses to external stimuli. Correctly identifying and providing feedback on emotions is an important goal in human-computer interaction research. Compared to facial expressions, speech, or other physiological signals, using electroencephalogram (EEG) signals for the task of emotion recognition has advantages in terms of authenticity, objectivity, and high reliability; thus, it is attracting increasing attention from researchers. However, the current methods have significant room for improvement in terms of the combination of information exchange between different brain regions and time-frequency feature extraction. Therefore, this paper proposes an EEG emotion recognition network, namely, self-organized graph pesudo-3D convolution (SOGPCN), based on attention and spatiotemporal convolution. Unlike previous methods that directly construct graph structures for brain channels, the proposed SOGPCN method considers that the spatial relationships between electrodes in each frequency band differ. First, a self-organizing map is constructed for each channel in each frequency band to obtain the 10 most relevant channels to the current channel, and graph convolution is employed to capture the spatial relationships between all channels in the self-organizing map constructed for each channel in each frequency band. Then, pseudo-three-dimensional convolution combined with partial dot product attention is implemented to extract the temporal features of the EEG sequence. Finally, LSTM is employed to learn the contextual information between adjacent time-series data. Subject-dependent and subject-independent experiments are conducted on the SEED dataset to evaluate the performance of the proposed SOGPCN method, which achieves recognition accuracies of 95.26% and 94.22%, respectively, indicating that the proposed method outperforms several baseline methods.

Tunnel indirect damage identification algorithm based on service train dynamic response time series

In this study, a tunnel indirect damage identification algorithm based on a service train dynamic response time series was proposed. First, the principles and processes of the proposed algorithm are introduced. The instantaneous frequency and spectral entropy of the trend and seasonal terms of the train dynamic response decomposed by Seasonal and Trend decomposition using Loess are calculated, respectively, to further extract the time-frequency features of the signal. The calculated sequence is input into the Bi-directional Long Short-Term Memory network, and the type of damage case is the output. Second, the feasibility of the algorithm is verified using a model test. Finally, the key links in the algorithm are discussed. The results indicate that the proposed algorithm can identify damage cases with a small number of training and test sets, and the accuracy of the test set can reach 83.33%. Through time-frequency feature extraction and normalization processing, the length of the network input sequence is reduced, the training speed is increased, and damage identification can be realized without a deep network.

Communication signal detection based on high-order cumulants time-frequency analysis: on the application of deep learning YOLOV5 network

The detection of communication signals in heterogeneous electromagnetic environments currently relies primarily on a one-dimensional statistical feature threshold method. However, this approach is highly sensitive to dynamic changes in the environment, fluctuations in signal-to-noise ratios, and complex noise. To address these limitations, this paper proposes a novel time-frequency diagram based on high-order accumulation for signal detection. Traditional time-frequency diagrams suffer from poor noise suppression ability and unclear features. However, higher-order cumulants can effectively overcome these shortcomings. Currently, methods based on higher-order cumulants are typically limited to one-dimensional signals. Yet, two-dimensional time-frequency signal diagrams can represent a broader array of features. This paper employs higher-order accumulation to extract time-frequency features from the received signal, thereby transforming the conventional radio detection problem into an image recognition challenge. By merging the advantages of higher-order accumulations and time-frequency diagrams, we propose the use of higher-order accumulation time-frequency diagrams for signal detection. Extensive experimental simulations demonstrate that the proposed time-frequency diagram exhibits strong anti-noise performance and effectively suppresses frequency bias from multiple perspectives. The performance of the Higher-Order Cumulant-Time Frequency (HOC-TF) indicated lower Root Mean Square Error (RMSE) compared with the Short-Time Fourier Transform-Time Frequency (STFT-TF) and Wavelet Transform-Time Frequency (WT-TF). Additionally, compared to the STFT-TF and WT-TF methodologies, the novel time-frequency diagram introduced demonstrates superior stability using the Singular Value Decomposition (SVD) method. Moreover, by combining the new time-frequency diagram with the deep learning YOLOV5 network, signal detection and modulation identification of communication signals can be achieved.

Automated identification of atrial fibrillation from single-lead ECGs using multi-branching ResNet.

Introduction: Atrial fibrillation (AF) is the most common cardiac arrhythmia, which is clinically identified with irregular and rapid heartbeat rhythm. AF puts a patient at risk of forming blood clots, which can eventually lead to heart failure, stroke, or even sudden death. Electrocardiography (ECG), which involves acquiring bioelectrical signals from the body surface to reflect heart activity, is a standard procedure for detecting AF. However, the occurrence of AF is often intermittent, costing a significant amount of time and effort from medical doctors to identify AF episodes. Moreover, human error is inevitable, as even experienced medical professionals can overlook or misinterpret subtle signs of AF. As such, it is of critical importance to develop an advanced analytical model that can automatically interpret ECG signals and provide decision support for AF diagnostics. Methods: In this paper, we propose an innovative deep-learning method for automated AF identification using single-lead ECGs. We first extract time-frequency features from ECG signals using continuous wavelet transform (CWT). Second, the convolutional neural networks enhanced with residual learning (ReNet) are employed as the functional approximator to interpret the time-frequency features extracted by CWT. Third, we propose to incorporate a multi-branching structure into the ResNet to address the issue of class imbalance, where normal ECGs significantly outnumber instances of AF in ECG datasets. Results and Discussion: We evaluate the proposed Multi-branching Resnet with CWT (CWT-MB-Resnet) with two ECG datasets, i.e., PhysioNet/CinC challenge 2017 and ECGs obtained from the University of Oklahoma Health Sciences Center (OUHSC). The proposed CWT-MB-Resnet demonstrates robust prediction performance, achieving an F1 score of 0.8865 for the PhysioNet dataset and 0.7369 for the OUHSC dataset. The experimental results signify the model's superior capability in balancing precision and recall, which is a desired attribute for ensuring reliable medical diagnoses.

Hybrid Multimodal Feature Fusion with Multi-Sensor for Bearing Fault Diagnosis.

Aiming at the traditional single sensor vibration signal cannot fully express the bearing running state, and in the high noise background, the traditional algorithm is insufficient for fault feature extraction. This paper proposes a fault diagnosis algorithm based on multi-sensor and hybrid multimodal feature fusion to achieve high-precision fault diagnosis by leveraging the operating state information of bearings in a high-noise environment to the fullest extent possible. First, the horizontal and vertical vibration signals from two sensors are fused using principal component analysis, aiming to provide a more comprehensive description of the bearing's operating condition, followed by data set segmentation. Following fusion, time-frequency feature maps are generated using a continuous wavelet transform for global time-frequency feature extraction. A first diagnostic model is then developed utilizing a residual neural network. Meanwhile, the feature data is normalized, and 28 time-frequency feature indexes are extracted. Subsequently, a second diagnostic model is constructed using a support vector machine. Lastly, the two diagnosis models are integrated to derive the final model through an ensemble learning algorithm fused at the decision level and complemented by a genetic algorithm solution to improve the diagnosis accuracy. Experimental results demonstrate the effectiveness of the proposed algorithm in achieving superior diagnostic performance with a 97.54% accuracy rate.

Reduced Biquaternion Windowed Linear Canonical Transform: Properties and Applications

The quaternion windowed linear canonical transform is a tool for processing multidimensional data and enhancing the quality and efficiency of signal and image processing; however, it has disadvantages due to the noncommutativity of quaternion multiplication. In contrast, reduced biquaternions, as a special case of four-dimensional algebra, possess unique advantages in computation because they satisfy the multiplicative exchange rule. This paper proposes the reduced biquaternion windowed linear canonical transform (RBWLCT) by combining the reduced biquaternion signal and the windowed linear canonical transform that has computational efficiency thanks to the commutative property. Firstly, we introduce the concept of a RBWLCT, which can extract the time local features of an image and has the advantages of both time-frequency analysis and feature extraction; moreover, we also provide some fundamental properties. Secondly, we propose convolution and correlation operations for RBWLCT along with their corresponding generalized convolution, correlation, and product theorems. Thirdly, we present a fast algorithm for RBWLCT and analyze its computational complexity based on two dimensional Fourier transform (2D FTs). Finally, simulations and examples are provided to demonstrate that the proposed transform effectively captures the local RBWLCT-frequency components with enhanced degrees of freedom and exhibits significant concentrations.

Multi-modal physiological time-frequency feature extraction network for accurate sleep stage classification

Sleep stage classification is essential for clinical disease diagnosis and sleep quality assessment. Most of the existing methods for sleep stage classification are based on single-channel or single-modal signal, and extract features using a single-branch, deep convolutional network, which not only hinders the capture of the diversity features related to sleep and increase the computational cost, but also has a certain impact on the accuracy of sleep stage classification. To solve this problem, this paper proposes an end-to-end multi-modal physiological time-frequency feature extraction network (MTFF-Net) for accurate sleep stage classification. First, multi-modal physiological signal containing electroencephalogram (EEG), electrocardiogram (ECG), electrooculogram (EOG) and electromyogram (EMG) are converted into two-dimensional time-frequency images containing time-frequency features by using short time Fourier transform (STFT). Then, the time-frequency feature extraction network combining multi-scale EEG compact convolution network (Ms-EEGNet) and bidirectional gated recurrent units (Bi-GRU) network is used to obtain multi-scale spectral features related to sleep feature waveforms and time series features related to sleep stage transition. According to the American Academy of Sleep Medicine (AASM) EEG sleep stage classification criterion, the model achieved 84.3% accuracy in the five-classification task on the third subgroup of the Institute of Systems and Robotics of the University of Coimbra Sleep Dataset (ISRUC-S3), with 83.1% macro F1 score value and 79.8% Cohen's Kappa coefficient. The experimental results show that the proposed model achieves higher classification accuracy and promotes the application of deep learning algorithms in assisting clinical decision-making.

Detection of Alzheimer’s Dementia by Using Deep Time–Frequency Feature Extraction

Detection of Alzheimer’s Dementia by Using Deep Time–Frequency Feature Extraction

Logistic regression-time–frequency algorithm for α/β discrimination in silicon photomultiplier–based Phoswich detectors

Phoswich (phosphor sandwich) systems composed of ZnS(Ag) and plastic dual scintillators have the ability to distinguish α and β particles via digital pulse-shape discrimination. In this work, a novel logistic regression algorithm based on time–frequency features (LR-TF) is presented and evaluated for its ability to digitally discriminate α and β particles in a mixed field. The LR-TF algorithm employs the charge comparison method (CCM) and Fourier gradient analysis (FGA) to extract time–frequency features from pulse shapes, thereby constructing a logistic regression model. Training of the model is shown to enable the discrimination α and β particles. The CCM, FGA, and LR-TF were applied to thousands of pulses obtained with a data acquisition system based on a Phoswich coupled with a silicon photomultiplier. The experimental results show that the LR-TF algorithm provided the best overall performance with misidentification ratios of 1.9 % and 0.63 % for α particles and β particles, respectively, achieving a figure of merit of 4.30, and an area under the receiver operating characteristic of 0.996.

Abnormal Heart Sound Detection using Time-Frequency Analysis and Machine Learning Techniques

Phonocardiogram (PCG) signals contains valuable information pertaining to heart valve functionality, rendering them potentially useful for early detection of cardiovascular diseases. Automated classification of heart sounds harbors great promise for identifying cardiac pathologies. This paper introduces a novel automated approach to classify normal and abnormal heart sounds. Our methodology involves partitioning heart sounds into four segments: S1, S2, systolic, and diastolic, followed by extraction of time–frequency and time-statistical features. Prior to data classification, we employ two techniques - particle Swarm optimization (PSO) and Sequential Forward Feature Selection (SFFS) - for efficient feature selection. We assess the performance of the proposed method on the Physionet Challenge 2016 database, utilizing the 10-fold cross-validation method. To address the issue of dataset imbalance, we apply the synthetic minority over-sampling technique (SMOTE) to create balanced datasets. Our approach surpasses existing methods in the literature, as evidenced by its superior accuracy, sensitivity, and specificity metrics. Specifically, our method achieves an accuracy of 98.03%, a sensitivity of 97.64%, and a specificity of 98.43% in distinguishing normal from abnormal heart sounds on the Physionet database. These findings outperform the results obtained by previously established methods evaluated on the Physionet 2016 challenge database.

A Partial Discharge Signal Separation Method Applicable for Various Sensors Based on Time–Frequency Feature Extraction of t-SNE

A Partial Discharge Signal Separation Method Applicable for Various Sensors Based on Time–Frequency Feature Extraction of t-SNE

A unified deep learning framework for water quality prediction based on time-frequency feature extraction and data feature enhancement

Deep learning methods exhibited significant advantages in mapping highly nonlinear relationships with acceptable computational speed, and have been widely used to predict water quality. However, various model selection and construction methods resulted in differences in prediction accuracy and performance. Hence, a unified deep learning framework for water quality prediction was established in the paper, including data processing module, feature enhancement module, and data prediction module. In the established model, the data processing module based on wavelet transform method was applied to decomposing complex nonlinear meteorology, hydrology, and water quality data into multiple frequency domain signals for extracting self characteristics of data cyclic and fluctuations. The feature enhancement module based on Informer Encoder was used to enhance feature encoding of time series data in different frequency domains to discover global time dependent features of variables. Finally, the data prediction module based on the stacked bidirectional long and short term memory network (SBiLSTM) method was employed to strengthen the local correlation of feature sequences and predict the water quality. The established model framework was applied in Lijiang River in Guilin, China. The maximum relative errors between the predicted and observed values for dissolved oxygen (DO), chemical oxygen demand (CODMn) were 12.4% and 20.7%, suggesting a satisfactory prediction performance of the established model. The validation results showed that the established model was superior to all other models in terms of prediction accuracy with RMSE values 0.329, 0.121, MAE values 0.217, 0.057, SMAPE values 0.022, 0.063 for DO and CODMn, respectively. Ablation tests confirmed the necessity and rationality of each module for the established model framework. The established method provided a unified deep learning framework for water quality prediction.

Precise disturbance localization of long distance fiber interferometer vibration senor based on an improved time–frequency variation feature extraction scheme

Locating intrusions with high accuracy is crucial for prompting long distance distributed optical fiber vibration sensing systems to practical applications in engineering filed. To achieve the high measurement accuracy and real-time performance simultaneously, in this paper, an intensity-modulated ultra-long distance distributed optical fiber vibration sensing system using an asymmetric dual Mach-Zehnder interferometer is experimentally established. More importantly, a positioning approach based on an ameliorated time–frequency feature extraction scheme is proposed with both theoretically analysis and experimental demonstration. In particular, an endpoint detection algorithm based on differential operation is firstly utilized to rapidly and accurately locate the endpoints of the received vibration signals. Thus the large-band width signals can be successfully acquired by setting an approximate window. Then, the ameliorated time–frequency feature extraction scheme based on a maximal overlap discrete wavelet packet transform is applied to eliminate the asymmetry of the large-band width signals, as it could lead to an erroneous positioning result when directly using a cross-correlation time delay estimation algorithm. Finally, the vibration positions along the sensing fiber link of the established sensing system can be successfully demodulated with the help of the extracted time–frequency features. To validate the effectiveness of the proposed scheme, vibration positioning tests were conducted in a real perimeter security scenario. And the results show that a mean positioning error of 11.12 m and a mean processing time of 0.276 s have been realized in a sensing length of 101 km. Thus, it is believed that through continuous technological advancements and refine algorithm optimization, the long distance distributed optical fiber vibration sensor founded on the proposed positioning strategy is poised to play an important role in the practical engineering applications.

Time–space–frequency feature Fusion for 3-channel motor imagery classification

Low-channel EEG devices are crucial for portable and entertainment applications. However, the low spatial resolution of EEG presents challenges in decoding low-channel motor imagery. This study introduces TSFF-Net, a novel network architecture that integrates time–space–frequency features, effectively compensating for the limitations of single-mode feature extraction networks based on time-series or time–frequency modalities. TSFF-Net comprises four main components: time–frequency representation, time–frequency feature extraction, time–space feature extraction, and feature fusion and classification. Time–frequency representation and feature extraction transform raw EEG signals into time–frequency spectrograms and extract relevant features. The time–space network processes time-series EEG trials as input and extracts temporal–spatial features. Feature fusion employs Maximum Mean Discrepancy (MMD) loss to constrain the distribution of time–frequency and time–space features in the Reproducing Kernel Hilbert Space, subsequently combining these features using a weighted fusion approach to obtain effective time–space–frequency features. Moreover, few studies have explored the decoding of three-channel motor imagery based on time–frequency spectrograms. This study proposes a shallow, lightweight decoding architecture (TSFF-img) based on time–frequency spectrograms and compares its classification performance in low-channel motor imagery with other methods using two publicly available datasets. Experimental results demonstrate that TSFF-Net not only compensates for the shortcomings of single-mode feature extraction networks in EEG decoding, but also outperforms other state-of-the-art methods. Overall, TSFF-Net offers considerable advantages in decoding low-channel motor imagery and provides valuable insights for algorithmically enhancing low-channel EEG decoding.

Time–frequency feature extraction based on multivariable synchronization index for training-free SSVEP-based BCI

Time–frequency feature extraction based on multivariable synchronization index for training-free SSVEP-based BCI

Wavelet-domain human activity recognition utilizing convolutional neural networks

Human activity recognition (HAR) is a crucial area of research in human-computer interaction. Despite previous efforts in this field, there is still a need for more accurate and robust methods that can handle time-series data from different sensors. In this study, we propose a novel method that generates an image using wavelet transform to extract time-frequency features of the recorded signal. Our method employs convolutional neural networks (CNNs) for feature extraction and activity recognition, and a new loss function that produces denser representations for samples, improving the model’s generalization on unseen samples. To evaluate the effectiveness of our proposed method, we conducted experiments on multiple publicly available data sets. Our results demonstrate that our method outperforms previous methods in terms of activity classification accuracy. Specifically, our method achieves higher accuracy rates and demonstrates improved robustness in real-world settings. Overall, our proposed method addresses the research gap of accurate and robust activity recognition from time-series data recorded from different sensors. Our findings have the potential to improve the accuracy and robustness of human activity recognition systems in real-world applications.

A novel crude oil futures trading strategy based on volume-price time-frequency decomposition with ensemble deep reinforcement learning

With the deepening financialization of crude oil, an increasing number of investors are becoming involved in trading crude oil futures. However, various factors influence crude oil futures price fluctuations, presenting complex characteristics. For this reason, a trading strategy plays a crucial role in investment. In this paper, we construct an ensemble trading strategy by combining time-frequency feature extraction of crude oil volume-price series with deep reinforcement learning. We decompose the original oil volume-price series by using the optimized variational mode decomposition. Additionally, taking into account the long memory characteristics of the crude oil time series, we employ the Sharpe ratio to filter the appropriate agents corresponding to market volatility. Moreover, the best features of the three agents are effectively combined and used in the next phase of trading, thus robustly adapting to different market volatility situations. Finally, we tested our strategy using Brent crude oil futures. Compared with other benchmark models, the proposed OVMD(V–P)-DRL-Ensemble strategy is suitable for the highly volatile crude oil market and performs well in terms of the corresponding performance evaluation metrics, showing excellent return performance and stability.

A Welch-EWT-SVD time–frequency feature extraction model for deformation monitoring data

It is a considerable challenge to accurately extract time–frequency features from non-linear and non-stationary deformation monitoring data. In this contribution, a time–frequency feature extraction model that integrates the Welch algorithm, empirical wavelet transform (EWT), and singular value decomposition (SVD) (Welch-EWT-SVD) is proposed. To avoid the impact of unreasonable spectral segmentation, the signal is decomposed at multiple levels based on the Welch power spectrum. The decomposed signal is then processed by EWT to obtain Intrinsic Mode Functions (IMFs). Finally, SVD is adopted to further denoise the IMFs to finely extract time–frequency features. Simulation tests and a field test at the Great Wall are conducted to validate the proposed method. The results demonstrate that Welch-EWT-SVD outperforms EWT, EMD, and Welch-EWT methods in terms of accuracy for time–frequency feature extraction. According to the simulation results, the relative error rate of its extracted frequency is kept within a maximum of 0.10%.

Material Classification and Aging Time Prediction of Structural Metals Using Laser-Induced Breakdown Spectroscopy Combined with Probabilistic Neural Network.

In this paper, laser-induced breakdown spectroscopy (LIBS) combined with a probabilistic neural network (PNN) was applied to classify engineering structural metal samples (valve stem, welding material, and base metal). Additionally, utilizing data from the plasma emission spectrum generated by laser ablation of samples with different aging times, an aging time prediction model based on a firefly optimized probabilistic neural network (FA-PNN) was established, which can effectively evaluate the service performance of structural materials. The problem of insufficient features obtained by principal component analysis (PCA) for predicting the aging time of materials is addressed by the proposal of a time-frequency feature extraction method based on short-time Fourier transform (STFT). The classification accuracy (ACC) of time-frequency features and principal component features was compared under PNN. The results indicate that, in comparison to the PCA feature extraction approach, the time-frequency feature extraction method based on STFT demonstrates higher accuracy in predicting the time of aging materials. Then, the relationship between classification accuracy (ACC) and settings of PNN was discussed. The ACC of the PNN model for both the material classification test set and the aging time test set achieved 100% with Firefly (FA) optimization algorithms. This result was also compared with the ACC of ANN, KNN, PLS-DA, and SIMCA for the aging time test set (95%, 87.5%, 85%, and 62.5%, respectively). The experimental results demonstrated that the classification model using LIBS combined with FA-PNN could realize better classification accuracy.

CSI-based human behavior segmentation and recognition using commodity Wi-Fi

In recent years, the behavior recognition technology based on Wi-Fi devices has been favored by many researchers. Existing Wi-Fi-based human behavior recognition technology mainly uses classification algorithms to construct classification models, which has problems such as inaccurate behavior segmentation, failure to extract deep-level features from the original data and design classification models matching the proposed features in the process of behavior recognition. In order to solve above problems, this paper proposes a window variance comparison method by combining the adaptive thresholds calculated to achieve effective segmentation of multiple discontinuous human behaviors, then uses the short-time Fourier algorithm to extract time-frequency features of individual behavior, and extracts the graph structure data from the autocorrelation matrix of time-frequency features and the features themselves. A graph neural network is built for behavior recognition. The experimental results show that the segmentation accuracy of the behavior segmentation method in the two scenes is 0.964 and 0.993, which is better than the existing threshold-based behavior segmentation methods. In addition, this paper extracts graph structure data by spectral energy change method and builds behavior recognition model by using graph neural network, and the recognition accuracy is significantly improved compared with the traditional classification algorithm.