Time-frequency Spectrogram Research Articles

Pen-based vibrotactile feedback rendering of surface textures under unconstrained acquisition conditions

Haptic rendering of surface textures enhances user immersion of human–computer interaction. However, strict input conditions and measurement methods limit the diversity of rendering algorithms. In this regard, we propose a neural network-based approach for vibrotactile haptic rendering of surface textures under unconstrained acquisition conditions. The method first encodes the interactions based on human perception characteristics, and then utilizes an autoregressive-based model to learn a non-linear mapping between the encoded data and haptic features. The interactions consist of normal forces and sliding velocities, while the haptic features are time–frequency amplitude spectrograms by short-time Fourier transform of the accelerations corresponding to the interactions. Finally, a generative adversarial network is employed to convert the generated time–frequency amplitude spectrograms into the accelerations. The effectiveness of the proposed approach is confirmed through numerical calculations and subjective experiences. This approach enables the rendering of a wide range of vibrotactile data for surface textures under unconstrained acquisition conditions, achieving a high level of haptic realism.

Deep sediment heterogeneity inferred using very low-frequency features from merchant shipsa).

The very low-frequency noise from merchant ships provides a good broadband sound source to study the deep layers of the seabed. The nested striations that characterize ship time-frequency spectrograms contain unique acoustic features corresponding to where the waveguide invariant β becomes infinite. In this dataset, these features occur at frequencies between 20 and 80 Hz, where pairs of modal group velocities become equal. The goal of this study is to identify these β = ∞ frequencies in ship noise spectrograms and use them to perform statistical inference for the deep layer sound speeds and thicknesses in the New England Mudpatch for a larger number of ships and acoustic arrays over a larger geographical region than previously studied. Marginal probability distributions of the data indicate that using singular points for a feature-based inversion yields an estimate of the sound speed and a limiting value for the thickness of the first deep layer. Heterogeneity is examined by correlating spatial variability of the deep layer sound speeds with ship tracks.

Spectrum Sensing Method Based on STFT-RADN in Cognitive Radio Networks.

To address the common issues in traditional convolutional neural network (CNN)-based spectrum sensing algorithms in cognitive radio networks (CRNs), including inadequate signal feature representation, inefficient utilization of feature map information, and limited feature extraction capabilities due to shallow network structures, this paper proposes a spectrum sensing algorithm based on a short-time Fourier transform (STFT) and residual attention dense network (RADN). Specifically, the RADN model improves the basic residual block and introduces the convolutional block attention module (CBAM), combining residual connections and dense connections to form a powerful deep feature extraction structure known as residual in dense (RID). This significantly enhances the network's feature extraction capabilities. By performing STFT on the received signals and normalizing them, the signals are converted into time-frequency spectrograms as network inputs, better capturing signal features. The RADN is trained to extract abstract features from the time-frequency images, and the trained RADN serves as the final classifier for spectrum sensing. Experimental results demonstrate that the STFT-RADN spectrum sensing method significantly improves performance under low signal-to-noise ratio (SNR) conditions compared to traditional deep-learning-based methods. This method not only adapts to various modulation schemes but also exhibits high detection probability and strong robustness.

Lightweight deep neural network for radio frequency interference detection and segmentation in synthetic aperture radar

Radio frequency interference (RFI) poses challenges in the analysis of synthetic aperture radar (SAR) images. Existing RFI suppression systems rely on prior knowledge of the presence of RFI. This paper proposes a lightweight neural network-based algorithm for detecting and segmenting RFI (LDNet) in the time-frequency domain. The network accurately delineates RFI pixel regions in time-frequency spectrograms. To mitigate the impact on the operational speed of the entire RFI suppression system, lightweight modules and pruning operations are introduced. Compared to threshold-based RFI detection algorithms, deep learning-based segmentation networks, and AC-UNet specifically designed for RFI detection, LDNet achieves improvements in mean intersection over union (MIoU) by 24.56%, 13.29%, and 7.54%, respectively.Furthermore, LDNet reduces model size by 99.03% and inference latency by 24.53% compared to AC-UNet.

A dual-region speech enhancement method based on voiceprint segmentation

Single-channel speech enhancement primarily relies on deep learning models to recover clean speech signals from noise-contaminated speech. These models establish a mapping relationship between noisy and clean speech. However, considering the sparse distribution characteristics of speech energy across the entire time–frequency spectrogram, constructing the mapping relationship from noisy to clean speech exhibits significant differences in regions where speech energy is concentrated and non-concentrated. Utilizing one deep model to simultaneously address these two distinct regression tasks increases the complexity of the mapping relationships, consequently restricting the model’s performance. To validate our hypothesis, we propose a dual-region speech enhancement model based on voiceprint region segmentation. Specifically, we first train a voiceprint segmentation model to classify noisy speech into two regions. Subsequently, we establish dedicated speech enhancement models for each region, with the dual-region models concurrently constructing mapping relationships for noise-corrupted speech to clean speech in distinct regions. Finally, by merging the results, the complete restored speech can be obtained. Experimental results on public datasets demonstrate that our method achieves competitive speech enhancement performance, outperforming the state-of-the-art. Ablation study results confirm the effectiveness of the proposed approach in enhancing model performance.

Radar Based Secure Contactless Fall Detection Using Hybrid Optimizer with Convolution Neural Network

Introduction: senior citizens can lead to severe injuries. Existing wearable fall-alert sensors are often ineffective as seniors tend to avoid using them, highlighting the need for non-contact sensor applications in smart homes. This study proposes a CNN-based fall detection system using time-frequency analyses. A unique hybrid optimizer, GWO-ABC, combining Artificial Bee Colony (ABC) and Grey Wolf Optimizer (GWO), is employed to optimize CNN architectures. Radar return signals are transformed into spectrograms and binary images for training the HOCNN with fall and non-fall data.Methods: Radar signals are processed using short-time Fourier transformation to create time-frequency spectrograms, converted into binary images. These images are fed into a CNN optimized by the GWO-ABC algorithm. The CNN is trained on labelled fall and non-fall instances, focusing on high-level feature extraction.Results: The HOCNN showed superior accuracy in fall detection, successfully extracting critical high-level features from radar signals. Performance metrics, including precision, recall, and F1-score, demonstrated significant improvements over traditional methods.Conclusion: This study introduces a non-contact, automatic fall detection system for smart homes using GWO-ABC optimized CNNs, offering a promising solution for enhancing geriatric care and ensuring senior citizen safety. Index Terms—Grey Wolf Optimizer, Artificial Bee Colony algorithm, Convolutional neural network, fall detection, time-frequency analysis, ultra-wideband (UWB) radar

Segmentation of MHD modes using Fourier transform, wavelets and computer vision algorithms

Abstract Magnetohydrodynamic (MHD) activity in fusion devices is typically analyzed by examining time-frequency spectrograms obtained from various diagnostics. MHD modes often co-exist with various types of noise and complex patterns generated by other events like pellet injection or active diagnostics. Traditionally, identifying MHD modes has been a manual task, making it labor-intensive. To overcome this issue, this study proposes the use of computer vision (CV) algorithms for noise removal and automatic feature extraction. First, the automatic detection of straight-line patterns is achieved by applying the Hough transform. Then, the discrete wavelet transform is proposed to break down spectrograms into sub-images of different scales, removing broadband noise and pellet injection signatures. The multiscale decomposition is subsequently extended to multiple directions using either 2D Fourier transforms or curvelets, achieving a high signal-to-noise ratio in spectrograms and eliminating undesired frequency sweeps of toroidal Alfvén eigenmodes antenna. Once MHD activity is successfully enhanced, a pipeline of algorithms for ridge detection, thresholding and labeling perform a segmentation of the image, automatically labeling individual modes. This study demonstrates the effectiveness of CV algorithms for the identification of MHD modes. The use of such algorithms may potentially help in the analysis process and the creation of large databases of modes.

Realizing Small UAV Targets Recognition via Multi-Dimensional Feature Fusion of High-Resolution Radar

For modern radar systems, small unmanned aerial vehicles (UAVs) belong to a typical types of targets with ‘low, slow, and small’ characteristics. In complex combat environments, the functional requirements of radar systems are not only limited to achieving stable detection and tracking performance but also to effectively complete the recognition of small UAV targets. In this paper, a multi-dimensional feature fusion framework for small UAV target recognition utilizing a small-sized and low-cost high-resolution radar is proposed, which can fully extract and combine the geometric structure features and the micro-motion features of small UAV targets. For the performance analysis, the echo data of different small UAV targets was measured and collected with a millimeter-wave radar, and the dataset consists of high-resolution range profiles (HRRP) and micro-Doppler time–frequency spectrograms was constructed for training and testing. The effectiveness of the proposed method was demonstrated by a series of comparison experiments, and the overall accuracy of the proposed method can reach 98.5%, which demonstrates that the proposed multi-dimensional feature fusion method can achieve better recognition performance than that of classical algorithms and higher robustness than that of single features for small UAV targets.

Multi-rolling element faults diagnosis of rolling bearing based on time-frequency analysis and multi-curves extraction

Abstract In industrial production, rolling bearings are widely used as key mechanical components in all types of rotating machinery. Fault diagnosis is essential for predicting bearing damage in advance, avoiding sudden equipment downtime and reducing economic losses. However, rolling element fault diagnosis of rolling bearings continues to be a challenge, especially with multi-rolling element faults. In view of the characteristics of randomness, weakness, and coupling in the vibration signal generated by multi-rolling element faults in rolling bearings, a multi-rolling element fault detection method is proposed by combination time-frequency (TF) analysis (TFA) with multi-curves extraction methods. The pre-processing method combined autoregressive model with maximum correlated kurtosis deconvolution is employed to enhance the weak periodic fault impulses in the raw vibration signals of the rolling bearing. Then an improved dynamic path multi-curves extraction method is proposed to extract multiple TF curves from the TF spectrogram (TFS) constructed via short-time Fourier transform. According to the proposed classification criteria, the TF curves are classified as homologous faults. The TF masking (TFM) method is employed to keep TF information closely associated with the fault impulse. Finally, the fault signals are reconstructed sequentially based on the TFS processed by TFM, and precise identification of multi-rolling element faults is achieved by envelope analysis. Experimental results demonstrate the effectiveness of the proposed method in extracting the weak fault features of multi-rolling elements and accomplishing fault separation and diagnosis.

Decoding nonlinear ultrasonic time-frequency characteristics for fatigue crack quantification and localisation via CNN

ABSTRACT Early detection of fatigue cracks is crucial to guarantee the safe operation of engineering structures. Nonlinear ultrasonic technique (NUT) is widely used for closed crack characterisation as it breaks through the detection sensitivity limit upon closed defects that are usually invisible to linear ultrasonic techniques. The nonlinearity parameter is generally defined as the damage index (DI) to qualitatively detect these cracks. However, DI-based methods are underdetermined for quantifying and localising cracks in most circumstances. In this study, a deep learning method for intelligently decoding nonlinear ultrasonic time-frequency characteristics is developed to quantify and localise fatigue cracks. The nonlinear ultrasonic time-frequency spectrogram, which includes features characterising crack severity and location, is obtained by cleverly selecting the excitation frequency. The convolutional neural networks (CNNs) establish a mapping relationship between the nonlinear ultrasonic time-frequency characteristics and the crack length and location. It is worth mentioning that frequency-dependent convolutional kernels are proposed to more competitively decode nonlinear ultrasonic time-frequency characteristics. The results indicate that the CNNs with the frequency-correlation kernels are particularly robust and effective in both noiseless and noisy environments. The study demonstrated the potential of the proposed nonlinear ultrasonic time-frequency characteristic decoding method for accurate and efficient characterisation of damage.

Variable data structures and customized deep learning surrogates for computationally efficient and reliable characterization of buried objects

In this study, in order to characterize the buried object via deep-learning-based surrogate modeling approach, 3-D full-wave electromagnetic simulations of a GPR model have been used. The task is to independently predict characteristic parameters of a buried object of diverse radii allocated at different positions (depth and lateral position) in various dispersive subsurface media. This study has analyzed variable data structures (raw B-scans, extracted features, consecutive A-scans) with respect to computational cost and accuracy of surrogates. The usage of raw B-scan data and the applications for processing steps on B-scan profiles in the context of object characterization incur high computational cost so it can be a challenging issue. The proposed surrogate model referred to as the deep regression network (DRN) is utilized for time frequency spectrogram (TFS) of consecutive A-scans. DRN is developed with the main aim being computationally efficient (about 13 times acceleration) compared to conventional network models using B-scan images (2D data). DRN with TFS is favorably benchmarked to the state-of-the-art regression techniques. The experimental results obtained for the proposed model and second-best model, CNN-1D show mean absolute and relative error rates of 3.6 mm, 11.8 mm and 4.7%, 11.6% respectively. For the sake of supplementary verification under realistic scenarios, it is also applied for scenarios involving noisy data. Furthermore, the proposed surrogate modeling approach is validated using measurement data, which is indicative of suitability of the approach to handle physical measurements as data sources.

Hybrid vehicle scanning techniques for detection of damaged hangers in tied-arch railway bridges

Regular monitoring of the hanger system is essential for preserving structural integrity in tied-arch bridges. This proactive monitoring enables preventive maintenance and early detection of hanger damage. To confront this issue, we propose a novel scanning method for assessing the impact of damaged hangers on the dynamic behaviour of single-span tied-arch railway bridges. Using each hanger as a positional reference, this method employs an instrumented vehicle as a mobile scanner to indirectly acquire vibration data from the traversed bridge deck. The hybrid approach integrates vehicle-bridge interaction (VBI) dynamics, vehicle scanning method (VSM), and moving windowed Fourier transform (mw-FT) technique to generate time–frequency spectrograms from the collected signals. In this representation, the time axis indicates the duration of the inspection vehicle traversing the bridge deck. By analysing frequency shifts within the spectrogram, we can identify damaged hangers through observed variations in frequency content over time. Numerical studies demonstrate the efficacy of the proposed hybrid vehicle scanning technique in detecting potential hanger damage in tied-arch railway bridges.

Advancing deep learning-based acoustic leak detection methods towards application for water distribution systems from a data-centric perspective

Against the backdrop of severe leakage issue in water distribution systems (WDSs), numerous researchers have focused on the development of deep learning-based acoustic leak detection technologies. However, these studies often prioritize model development while neglecting the importance of data. This research explores the impact of data augmentation techniques on enhancing deep learning-based acoustic leak detection methods. Five random transformation-based methods—jittering, scaling, warping, iterated amplitude adjusted Fourier transform (IAAFT), and masking—are proposed. Jittering, scaling, warping, and IAAFT directly process original signals, while masking operating on time-frequency spectrograms. Acoustic signals from a real-world WDS are augmented, and the efficacy is validated using convolutional neural network classifiers to identify the spectrograms of acoustic signals. Results indicate the importance of implementing data augmentation before data splitting to prevent data leakage and overly optimistic outcomes. Among the techniques, IAAFT stands out, significantly increasing data volume and diversity, improving recognition accuracy by over 7%. Masking enhances performance mainly by compelling the classifier to learn global features of the spectrograms. Sequential application of IAAFT and masking further strengthens leak detection performance. Furthermore, when applying a complex model to acoustic leakage detection through transfer learning, data augmentation can also enhance the effectiveness of transfer learning. These findings advance artificial intelligence-driven acoustic leak detection technology from a data-centric perspective towards more mature applications.

Research on Internal Damage Identification of Wire Rope Based on Improved VGG Network.

In order to solve the problem of great difficulty in detecting the internal damage of wire rope, this paper proposes a method to improve the VGG model to identify the internal damage of wire rope. The short-time Fourier transform method is used to transform the wire rope damage signal into a time-frequency spectrogram as the model input, and then the traditional VGG model is improved from three aspects: firstly, the attention mechanism module is introduced to increase the effective feature weights, which effectively improves the recognition accuracy; and then, the batch normalization layer is added to carry out a uniform normalization of the data, so as to make the model easier to converge. At the same time, the pooling layer and the fully connected layer are improved to solve the redundancy problem of the traditional VGG network model, which makes the model structure more lightweight, greatly saves the computational cost, shortens the training time, and finally adopts the joint-sample uniformly distributed cross-entropy as the loss function to solve the overfitting problem and further improve the recognition rate. The experimental results show that the improved VGG model has an identification accuracy of up to 98.84% for the internal damage spectrogram of the wire rope, which shows a good identification ability. Not only that, but the model is also superior, with less time-consuming training and stronger generalization ability.

Non-stationary vibration fatigue life prediction of automotive components based on long short-term memory network

Automotive components are prone to fatigue failure as a result of the long-term effects of vibration loads. Due to the significant non-stationarity of irregular excitations from various road surfaces, the classical frequency-domain method struggles to accurately estimate the fatigue life of automotive components. Based on long short-term memory (LSTM) networks, an efficient time-domain method for non-stationary vibration fatigue life prediction is proposed. Firstly, the data augmentation method for simulating long-time non-stationary loads is studied. Short-time histories are transformed into time–frequency spectrograms, and then the time–frequency spectrums are warped and masked to reconstruct the long-time non-stationary loads. Furthermore, employing only short-time loads and responses as training samples, the LSTM network is trained to construct a surrogate model for calculating structural stress time histories. Finally, the responses of varying-length long-time loads are calculated, and fatigue life is predicted by the combination of rainflow counting and Miner rule. Additionally, the representative response durations required for the fatigue analysis are estimated. Numerical simulation of control arms shows that the fatigue life prediction results using the LSTM surrogate model are within 1.9% difference compared to transient dynamics analysis results based on finite element method, and the calculation efficiency is improved by orders of magnitude.

Ensemble learning model for concrete delamination depth detection using impact echo

Delamination in concrete structures typically arises from the expansion of steel bar corrosion and the separation of the bonding layer. The presence of these delamination defects leads to a reduction in overall structural integrity, safety, and longevity. However, the delamination depth localization remains challenging. To address this issue, this paper introduces a predictive approach that combines time-frequency features extracted from vibration signals with ensemble learning models. The innovation presented in this paper can be summarized in two key aspects: (1) Signal time-frequency spectrograms are transformed into image features using the Histogram of Oriented Gradients (HOG) method, which serves as input attributes. The utilization of ensemble learning models facilitates both rapid and precise depth estimation of delamination signals, especially those with shallow defects. (2) By analyzing the time-frequency spectrogram of the vibration signals, a double exponential model is established between the depth-to-thickness ratio of delamination and the pixel matrix Eulerian distance. This may provide a new perspective for the depth estimation of delamination defects whose corresponding main frequencies are difficult to locate due to the influence of geometry and boundary effects. Laboratory experiments were conducted on concrete slab with simulated delamination defects at different depths to confirm the proposed method's effectiveness.

477 Identifying the Neural Correlates of the Triple Code Model in Numerical Cognition Using Stereotactic Electroencephalography in Epilepsy Patients

INTRODUCTION: The capacity to process numerical quantities is an essential human trait. The “Triple Code Model” (TCM) in numerical cognition, theorizes that different neural substrates encode the processing of visual, auditory, and nonsymbolic numerical representations formats. Limited data exists examining the direct electrophysiological activity involved in numerical cognition. METHODS: Thirteen patients with medically refractory epilepsy that underwent sEEG for the localization of seizures performed a passive numerical recognition task. Subjects were presented with a number quantity from 1 to 9 in one of five different representation formats: two were auditory (spoken numbers, sequential beeps) and three were visual (Arabic numeral, written word, assortment of dots). A total of 2,482 electrode contacts were analyzed. Time-frequency spectrograms were reduced with principal component analysis (PCA) and passed into a linear support vector machine (SVM) classification algorithm to identify neural correlates of number processing. RESULTS: The highest classification accuracy in number processing, irrespective of representation format occurred in the bilateral parietal lobes, right lingual cortex, and bilateral putamen. A total of 407, 776, 854, 475, and 513 contacts had significant classification values during Arabic, beep, spoken, written, and dots, respectively. Bilateral superior temporal cortices demonstrated the best classification value for the processing of auditory formats, whereas visual formats (Arabic and written number) showed greatest involvement in the frontal lobe and inferior precentral gyrus. Classification of dots indicated preferential engagement of the left parietal cortex. Spectral analyses revealed that non-gamma frequency bands held greater than chance classification values to characterize format-specific number representations. CONCLUSIONS: Using a linear SVM classifier with PCA, we examined the Triple Code Model in numerical cognition and identified several neural structures with high classification values involved in numerical processing.

A limited labeled data bearing fault diagnosis method based on self-supervised learning

The key focus of this study revolves around several crucial issues concerning bearing fault diagnosis, and presents an innovative solution. Conventional labeled approaches for bearing fault diagnosis often necessitate labeled data sets, which can be time-consuming or infeasible to obtain. To address this problem, an increasing amount of research has started to explore fault diagnosis methods that utilize limited labeled data. In our study, we introduce a framework for bearing fault diagnosis that incorporates wavelet transform and self-supervised learning techniques. The framework leverages vibration signals and transforms them into time-frequency spectrograms as inputs. To extract features, we employ the Swin Transformer as an encoder. Furthermore, we present a self-supervised learning approach named MoBy to address the challenge of limited labeled samples. Encouragingly, our approach achieves a diagnostic accuracy of 96.4% by utilizing only 1% labeled samples, through a well-trained encoder and a simple linear classification layer. This demonstrates outstanding performance in utilizing limited labeled data. To validate the superiority of our proposed approach, we conducted experiments on two rolling bearing fault datasets and achieved significant results.

Neurotoxic effects of home radon exposure on oscillatory dynamics serving attentional orienting in children and adolescents

Radon is a naturally occurring gas that contributes significantly to radiation in the environment and is the second leading cause of lung cancer globally. Previous studies have shown that other environmental toxins have deleterious effects on brain development, though radon has not been studied as thoroughly in this context. This study examined the impact of home radon exposure on the neural oscillatory activity serving attention reorientation in youths. Fifty-six participants (ages 6–14 years) completed a classic Posner cuing task during magnetoencephalography (MEG), and home radon levels were measured for each participant. Time-frequency spectrograms indicated stronger theta (3–7 Hz, 300–800 ms), alpha (9–13 Hz, 400–900 ms), and beta responses (14–24 Hz, 400–900 ms) during the task relative to baseline. Source reconstruction of each significant oscillatory response was performed, and validity maps were computed by subtracting the task conditions (invalidly cued – validly cued). These validity maps were examined for associations with radon exposure, age, and their interaction in a linear regression design. Children with greater radon exposure showed aberrant oscillatory activity across distributed regions critical for attentional processing and attention reorientation (e.g., dorsolateral prefrontal cortex, and anterior cingulate cortex). Generally, youths with greater radon exposure exhibited a reverse neural validity effect in almost all regions and showed greater overall power relative to peers with lesser radon exposure. We also detected an interactive effect between radon exposure and age where youths with greater radon exposure exhibited divergent developmental trajectories in neural substrates implicated in attentional processing (e.g., bilateral prefrontal cortices, superior temporal gyri, and inferior parietal lobules). These data suggest aberrant, but potentially compensatory neural processing as a function of increasing home radon exposure in areas critical for attention and higher order cognition.

Carpal Tunnel Syndrome Automated Diagnosis: A Motor vs. Sensory Nerve Conduction-Based Approach.

The objective of this study was to evaluate the effectiveness of machine learning classification techniques applied to nerve conduction studies (NCS) of motor and sensory signals for the automatic diagnosis of carpal tunnel syndrome (CTS). Two methodologies were tested. In the first methodology, motor signals recorded from the patients' median nerve were transformed into time-frequency spectrograms using the short-time Fourier transform (STFT). These spectrograms were then used as input to a deep two-dimensional convolutional neural network (CONV2D) for classification into two categories: patients and controls. In the second methodology, sensory signals from the patients' median and ulnar nerves were subjected to multilevel wavelet decomposition (MWD), and statistical and non-statistical features were extracted from the decomposed signals. These features were utilized to train and test classifiers. The classification target was set to three categories: normal subjects (controls), patients with mild CTS, and patients with moderate to severe CTS based on conventional electrodiagnosis results. The results of the classification analysis demonstrated that both methodologies surpassed previous attempts at automatic CTS diagnosis. The classification models utilizing the motor signals transformed into time-frequency spectrograms exhibited excellent performance, with average accuracy of 94%. Similarly, the classifiers based on the sensory signals and the extracted features from multilevel wavelet decomposition showed significant accuracy in distinguishing between controls, patients with mild CTS, and patients with moderate to severe CTS, with accuracy of 97.1%. The findings highlight the efficacy of incorporating machine learning algorithms into the diagnostic processes of NCS, providing a valuable tool for clinicians in the diagnosis and management of neuropathies such as CTS.