Frequency Representation Research Articles

High-resolution ground motion generation with time–frequency representation

High-resolution ground motion generation with time–frequency representation

Potential of using CO2 observations over India in a regional carbon budget estimation by improving the modelling system

Abstract. Devising effective national-level climate action plans requires a more detailed understanding of the regional distribution of sources and sinks of greenhouse gases. Due to insufficient observations and modelling capabilities, India's current carbon source–sink estimates are uncertain. This study uses a high-resolution Lagrangian transport model to examine the potential of available CO2 observations over India for inverse estimation of regional carbon fluxes. We use four different sites in India that vary in the measurement technique, frequency and spatial representation. These observations exhibit substantial seasonal (7.5 to 9.2 ppm) and intra-seasonal (2 to 12 ppm) variability. Our modelling framework, a high-resolution Weather Research and Forecasting Model combined with the Stochastic Time-Inverted Lagrangian Transport model (WRF–STILT), performs better in simulating seasonal (R2=0.50 to 0.96) and diurnal (R2=0.96) variability (for the Mohali station) of observed CO2 than the current-generation global models (CarboScope, CarbonTracker and ECMWF EGG4). The seasonal CO2 concentration variability in Mohali, associated with crop residue burning, is largely underestimated by the models. WRF–STILT captures the seasonal biospheric variability over Nainital better than the global models but underestimates the strength of the CO2 uptake by crops. The choice of emission inventory in the modelling framework alone leads to significant biases in simulations (5 to 10 ppm), endorsing the need for accounting for emission fluxes, especially for non-background sites. Our study highlights the possibility of using the CO2 observations from these Indian stations for deducing carbon flux information at regional (Nainital) and suburban to urban (Mohali, Shadnagar and Nagpur) scales with the help of a high-resolution model. On accounting for observed variability in CO2, the global carbon data assimilation system can benefit from the measurements from the Indian subcontinent.

A study of oil–water two-phase flow pattern recognition based on thermal sensors

To understand the flow characteristics and kinetic modelling of oil–water two-phase flow(OWTPF), this paper employs thermal array sensors to identify the flow pattern (FP) in OWTPF horizontal pipelines. Firstly, the thermal array sensor installation was constructed. Secondly, the time-varying characteristics of multi-feature signals were qualitatively analyzed using the adaptive optimal kernel time–frequency representation (AOK-TFR) method and unthresholded recurrence plot (URP) method. Finally, the sparrow search algorithm-support vector machine (SSA-SVM) machine learning algorithm is used for quantitative prediction of multi-feature classification, which leads to high prediction accuracy. The experiment proves that the correct rate of the thermal method in recognizing six types of OWTPF is above 91%, the correct rate of five types of flow is above 97.33%. This paper adopts the thermal method to identify the multiphase FP, which is a major innovation in this paper and also lays the foundation for recognizing the flow characteristics of multiphase flow.

A novel spectrogram based lightweight deep learning for IoT spectrum monitoring

The integration of cognitive radio network (CRNet) with internet of things (IoT) holds tremendous potential for creating more intelligent and advanced technological ecosystems. By allowing IoT devices to dynamically access and utilize the available radio spectrum, CRNet-based IoT systems can enable transparent connectivity, faster data transmission, and efficient resource utilization. The successful deployment of CRNet-based IoT systems highly depends on spectrum sensing implementation, which makes selecting the appropriate spectrum sensing technique extremely critical. However, several challenges are associated with the current spectrum sensing techniques, including limited representation of signal features, susceptibility to noise, inadequate hardware compatibility, and significant performance degradation using deep learning models. To deal with these issues, a novel approach is proposed integrating spectrogram representation and lightweight convolutional neural network models. Unlike raw digital representation, spectrograms allow efficient and consistent feature extraction using time–frequency representation of the received signals. Lightweight convolutional neural network models leverage advanced deep learning techniques to achieve fast and accurate predictions from spectrograms. They require few parameters to process while reduce the computational burden. The proposed approach is a promising solution for performing spectrum sensing in resource-constrained environments paving the way for efficient use of radio spectrum in IoT networks. The performance of the proposed approach is evaluated and compared to the existing techniques. Spectrogram based lightweight deep learning achieved high efficiency while required less computational resources.

Topography and Ensemble Activity in the Auditory Cortex of a Mouse Model of Fragile X Syndrome.

Autism spectrum disorder (ASD) is often associated with social communication impairments and specific sound processing deficits, for example, problems in following speech in noisy environments. To investigate underlying neuronal processing defects located in the auditory cortex (AC), we performed two-photon Ca2+ imaging in FMR1 (fragile X messenger ribonucleoprotein 1) knock-out (KO) mice, a model for fragile X syndrome (FXS), the most common cause of hereditary ASD in humans. For primary AC (A1) and the anterior auditory field (AAF), topographic frequency representation was less ordered compared with control animals. We additionally analyzed ensemble AC activity in response to various sounds and found subfield-specific differences. In A1, ensemble correlations were lower in general, while in secondary AC (A2), correlations were higher in response to complex sounds, but not to pure tones. Furthermore, sound specificity of ensemble activity was decreased in AAF. Repeating these experiments 1 week later revealed no major differences regarding representational drift. Nevertheless, we found subfield- and genotype-specific changes in ensemble correlation values between the two times points, hinting at alterations in network stability in FMR1 KO mice. These detailed insights into AC network activity and topography in FMR1 KO mice add to the understanding of auditory processing defects in FXS.

FF-ViT: probe orientation regression for robot-assisted endomicroscopy tissue scanning

PurposeProbe-based confocal laser endomicroscopy (pCLE) enables visualization of cellular tissue morphology during surgical procedures. To capture high-quality pCLE images during tissue scanning, it is important to maintain close contact between the probe and the tissue, while also keeping the probe perpendicular to the tissue surface. Existing robotic pCLE tissue scanning systems, which rely on macroscopic vision, struggle to accurately place the probe at the optimal position on the tissue surface. As a result, the need arises for regression of longitudinal distance and orientation via endomicroscopic vision.MethodThis paper introduces a novel method for automatically regressing the orientation between a pCLE probe and the tissue surface during robotic scanning, utilizing the fast Fourier vision transformer (FF-ViT) to extract local frequency representations and use them for probe orientation regression. Additionally, the FF-ViT incorporates a blur mapping attention (BMA) module to refine latent representations, which is combined with the pyramid angle regressor (PAR) to precisely estimate probe orientation.ResultA first of its kind dataset for pCLE probe-tissue orientation (pCLE-PTO) has been created. The performance evaluation demonstrates that our proposed network surpasses other top regression networks in accuracy, stability, and generalizability, while maintaining low computational complexity (1.8G FLOPs) and high inference speed (90 fps).ConclusionThe performance evaluation study verifies the clinical value of the proposed framework and its potential to be integrated into surgical robotic platforms for intraoperative tissue scanning.

Stacked U-Net with Time–Frequency Attention and Deep Connection Net for Single Channel Speech Enhancement

Deep neural networks have significantly promoted the progress of speech enhancement technology. However, a great number of speech enhancement approaches are unable to fully utilize context information from various scales, hindering performance enhancement. To tackle this issue, we introduce a method called TFADCSU-Net (Stacked U-Net with Time-Frequency Attention (TFA) and Deep Connection Layer (DCL)) for enhancing noisy speech in the time–frequency domain. TFADCSU-Net adopts an encoder-decoder structure with skip links. Within TFADCSU-Net, a multiscale feature extraction layer (MSFEL) is proposed to effectively capture contextual data from various scales. This allows us to leverage both global and local speech features to enhance the reconstruction of speech signals. Moreover, we incorporate deep connection layer and TFA mechanisms into the network to further improve feature extraction and aggregate utterance level context. The deep connection layer effectively captures rich and precise features by establishing direct connections starting from the initial layer to all subsequent layers, rather than relying on connections from earlier layers to subsequent layers. This approach not only enhances the information flow within the network but also avoids a significant rise in computational complexity as the number of network layers increases. The TFA module consists of two attention branches operating concurrently: one directed towards the temporal dimension and the other towards the frequency dimension. These branches generate distinct forms of attention — one for identifying relevant time frames and another for selecting frequency wise channels. These attention mechanisms assist the models in discerning “where” and “what” to prioritize. Subsequently, the TA and FA branches are combined to produce a comprehensive attention map in two dimensions. This map assigns specific attention weights to individual spectral components in the time–frequency representation, enabling the networks to proficiently capture the speech characteristics in the T-F representation. The results confirm that the proposed method outperforms other models in terms of objective speech quality as well as intelligibility.

DeepDet: YAMNet with BottleNeck Attention Module (BAM) for TTS synthesis detection

Spoofed speeches are becoming a big threat to society due to advancements in artificial intelligence techniques. Therefore, there must be an automated spoofing detector that can be integrated into automatic speaker verification (ASV) systems. In this study, we recommend a novel and robust model, named DeepDet, based on deep-layered architecture, to categorize speech into two classes: spoofed and bonafide. DeepDet is an improved model based on Yet Another Mobile Network (YAMNet) employing a customized MobileNet combined with a bottleneck attention module (BAM). First, we convert audio into mel-spectrograms that consist of time–frequency representations on mel-scale. Second, we trained our deep layered model using the extracted mel-spectrograms on a Logical Access (LA) set, including synthesized speeches and voice conversions of the ASVspoof-2019 dataset. In the end, we classified the audios, utilizing our trained binary classifier. More precisely, we utilized the power of layered architecture and guided attention that can discern the spoofed speech from bonafide samples. Our proposed improved model employs depth-wise linearly separate convolutions, which makes our model lighter weight than existing techniques. Furthermore, we implemented extensive experiments to assess the performance of the suggested model using the ASVspoof 2019 corpus. We attained an equal error rate (EER) of 0.042% on Logical Access (LA), whereas 0.43% on Physical Access (PA) attacks. Therefore, the performance of the proposed model is significant on the ASVspoof 2019 dataset and indicates the effectiveness of the DeepDet over existing spoofing detectors. Additionally, our proposed model is robust enough that can identify the unseen spoofed audios and classifies the several attacks accurately.

Wavelet-based photoacoustic viscoelastographic microscopy using photoacoustic damped oscillation effect

Viscoelastic characterization of biological tissue has significant biomedical value. In this study, we propose a wavelet-based photoacoustic viscoelastographic microscopy to characterize the viscoelasticity of tissues beneath the optical scattering depth from photoacoustic oscillation signals. Irradiated by laser pulses, biological tissue absorbs energy, expands, continues oscillating, and emits damped transient ultrasonic waves, i.e., photoacoustic oscillation signals, for a short time. A damped oscillation wavelet is applied to map photoacoustic oscillation signals into a representation of time, frequency, and decay rate. We can evaluate the frequency and decay rate of the photoacoustic oscillation signals. By considering the interrelationships among frequency, decay rate, shear modulus, and shear viscosity coefficient, we can generate images that depict the shear modulus image and shear viscosity coefficient image within the tissue. These images of the shear modulus and shear viscosity coefficient collectively form the viscoelastogram of the tissue, offering a comprehensive portrayal of its viscoelastic properties. Our study has potential biomedical value and may improve the accurate diagnosis of diseases and achieve more effective early interventions.

Generalizable Fourier Augmentation for Unsupervised Video Object Segmentation

The performance of existing unsupervised video object segmentation methods typically suffers from severe performance degradation on test videos when tested in out-of-distribution scenarios. The primary reason is that the test data in real- world may not follow the independent and identically distribution (i.i.d.) assumption, leading to domain shift. In this paper, we propose a generalizable fourier augmentation method during training to improve the generalization ability of the model. To achieve this, we perform Fast Fourier Transform (FFT) over the intermediate spatial domain features in each layer to yield corresponding frequency representations, including amplitude components (encoding scene-aware styles such as texture, color, contrast of the scene) and phase components (encoding rich semantics). We produce a variety of style features via Gaussian sampling to augment the training data, thereby improving the generalization capability of the model. To further improve the cross-domain generalization performance of the model, we design a phase feature update strategy via exponential moving average using phase features from past frames in an online update manner, which could help the model to learn cross-domain-invariant features. Extensive experiments show that our proposed method achieves the state-of-the-art performance on popular benchmarks.

Defect detection in additively manufactured AlSi10Mg and Ti6Al4V samples using laser ultrasonics and phase shift migration

Laser ultrasonics (LU) is a non-contact and non-destructive method with a high data acquisition rate, making it a promising candidate for in-situ monitoring of defects in different additive manufacturing (AM) processes, including laser powder bed fusion (LPBF) and directed energy deposition, as well as final part inspection. In order to see the effect of various artificial defect types on an LU sub-surface reconstruction, AlSi10Mg samples with side through-holes, as well as Ti6Al4V samples with bottom blind holes and trapped powder were printed using LPBF, and then ultrasound B-scans of the samples were obtained using an LU system. The resulting scan data was processed using a custom frequency domain phase shift migration (PSM) algorithm, to reconstruct the defects and their locations. Novel ways of pre-processing the B-scan, used as an input to PSM, and taking advantage of its frequency representation, are demonstrated. Newton’s method was used to find a stationary phase approximation, used to account in the frequency domain for the fixed offset emitter-receiver arrangement within the PSM calculation. The Newton’s method calculation time was reduced by 33%, by using an approximation of the phase function to find an initial guess. The smallest defects that were detected using this method were in the size range between 200 to 300μm for the bottom hole defects, using an 8 ns laser pulse duration. The effect of the laser on the surface of a part being built, and the challenges and further work needed to integrate LU in a LPBF machine for in-situ inspection are discussed.

Heart Rate Variability Monitoring Based on Doppler Radar Using Deep Learning.

The potential of microwave Doppler radar in non-contact vital sign detection is significant; however, prevailing radar-based heart rate (HR) and heart rate variability (HRV) monitoring technologies often necessitate data lengths surpassing 10 s, leading to increased detection latency and inaccurate HRV estimates. To address this problem, this paper introduces a novel network integrating a frequency representation module and a residual in residual module for the precise estimation and tracking of HR from concise time series, followed by HRV monitoring. The network adeptly transforms radar signals from the time domain to the frequency domain, yielding high-resolution spectrum representation within specified frequency intervals. This significantly reduces latency and improves HRV estimation accuracy by using data that are only 4 s in length. This study uses simulation data, Frequency-Modulated Continuous-Wave radar-measured data, and Continuous-Wave radar data to validate the model. Experimental results show that despite the shortened data length, the average heart rate measurement accuracy of the algorithm remains above 95% with no loss of estimation accuracy. This study contributes an efficient heart rate variability estimation algorithm to the domain of non-contact vital sign detection, offering significant practical application value.

An improved tracking method for bearing characteristic frequencies in the time-frequency representation of vibration signal

Rolling bearing is one of the most critical components for support and energy conversion in machines. The fault characteristic frequency (FCF) of time–frequency representation has received increasing attention in bearing diagnosis under variable speed conditions. However, FCF-extracted methods have poor adaptability to amplitude attenuation and noise interference due to local distortions or even transitions in the estimated instantaneous frequency ridges. Consequently, this paper proposes an improved FCF tracking method for variable speed bearing diagnosis. A strategy for locating distortion intervals is first developed using exponential smoothing and residual distribution. Subsequently, an advanced fast path optimization method, including peak map renewal and curve search optimization, is proposed to extract the ridges of interest. Finally, the probability density function of curve-to-curve ratios is designed to accurately identifying bearing faults. Simulation and experimental results demonstrate the effectiveness of the proposed method.

Noninvasive Monitoring of Tissue Temperature Changes Induced by Focused Ultrasound Exposure using Sparse Expression of Ultrasonic Radio Frequency Echo Signals.

Noninvasive therapies such as focused ultrasound were developed to be used for cancer therapies, vessel bleeding, and drug delivery. The main purpose of focused ultrasound therapy is to affect regions of interest (ROI) of tissues without any injuries to surrounding tissues. In this regard, an appropriate monitoring method is required to control the treatment. This study is aimed to develop a noninvasive monitoring technique of focused ultrasound (US) treatment using sparse representation of US radio frequency (RF) echo signals. To this end, reasonable results in temperature change estimation in the tissue under focused US radiation were obtained by utilizing algorithms related to sparse optimization as orthogonal matching pursuit (OMP) and accompanying Shannon's entropy. Consequently, ex vivo tissue experimental tests yielded two datasets, including low-intensity focused US (LIFU) and high-intensity focused US (HIFU) data. The proposed processing method analyzed the ultrasonic RF echo signal and expressed it as a sparse signal and calculated the entropy of each frame. The results indicated that the suggested approach could noninvasively estimate temperature changes between 37°C and 47°C during LIFU therapy. In addition, it represented temperature changes during HIFU ablation at various powers, ranging from 10 to 130 W. The normalized mean square error of the proposed method is 0.28, approximately 2.15 on previous related methods. These results demonstrated that this novel proposed approach, including the combination of sparsity and Shanoon's entropy, is more feasible and effective in temperature change estimation than its predecessors.

Generalization of Deep-Learning Models for Classification of Local Distance Earthquakes and Explosions across Various Geologic Settings

Abstract Although accurately classifying signals from earthquakes and explosions at local distance (<250 km) remains an important task for seismic network operations, the growing volume of available seismic data presents a challenge for analysts using traditional source discrimination techniques. In recent years, deep-learning models have proven effective at discriminating between low-magnitude earthquakes and explosions measured at local distances, but it is not clear how well these models are capable of generalizing across different geological settings. To address the issue of generalization between regions, we train deep-learning models (convolutional neural networks [CNNs]) on time–frequency representations (scalograms) of three-component earthquake and explosion signals from eight different regions in the continental United States. We explore scenarios where models are trained on data from all regions, individual regions, or all but one region. We find that although CNN models trained on individual regions do not necessarily generalize well across different settings, models trained on multiple regions that include diverse path coverage generalize to new regions, with station-level accuracy of up to 90% or more for data sets from unseen regions. In general, CNN-based discrimination models significantly outperform models based on uncorrected P/S ratio (measured in the 10–18 Hz frequency band), even when CNN models are tested on data from entirely unseen regions.

Method for Fault Diagnosis of Track Circuits Based on a Time–Frequency Intelligent Network

In response to the limitations posed by noise interference in complex environments and the narrow focus of existing diagnosis methods for jointless track circuit faults, an innovative approach is put forward in this study. It involves the application of the continuous wavelet transform (CWT) for signal preprocessing, along with the integration of a deep belief network (DBN) and a genetic algorithm (GA) to improve the least-squares support vector machine (LSSVM) model for intelligent time–frequency fault diagnosis. Initially, the raw induced voltage signals are transformed using continuous wavelet transformation resulting in wavelet time–frequency representations that combine temporal and spectral information. Subsequently, these time–frequency representations are fed into the deep belief networks, which perform semi-supervised dimensionality reduction and feature extraction, thereby uncovering distinct fault characteristics in the track circuit. Finally, the genetic algorithms are employed to improve the kernel function and penalty factor parameters of the least-squares support vector machine, thus establishing an optimal DBN-GA-LSSVM diagnostic model. Experimental validation demonstrates the effectiveness of the proposed time–frequency intelligent network model by leveraging the advantages of deep belief networks in hierarchical feature extraction and the superior performance of the least-squares support vector machine in addressing high-dimensional pattern recognition problems with limited samples. The achieved accuracy rate on the testing dataset reaches an impressive 99.6%. Consequently, this comprehensive approach provides a viable solution for data-driven track circuit fault diagnosis.

A maximum projective stratification criterion for selecting space-filling designs

Space-filling designs are commonly used in computer experiments. In this paper, we propose a maximum projective stratification criterion for classifying and ranking space-filling designs. We first introduce a stratification metric, which is proved to be a variance decomposition of the frequency representation of a design. Based on this metric, we define the new criterion to sequentially minimize a projective stratification pattern, that quantifies the stratification property of a design when projected into one up to full dimensions. Numerical results show that, when the space-filling property in cluster of projections with respect to dimensions is paid attention on, our criterion is more rational than existing criteria. We further study the intimate connections between our criterion and existing criteria. The extension of this criterion for a design with prime power levels to general levels are also discussed.

Optimizing Speech Emotion Recognition with Deep Learning and Grey Wolf Optimization: A Multi-Dataset Approach

Machine learning and speech emotion recognition are rapidly evolving fields, significantly impacting human-centered computing. Machine learning enables computers to learn from data and make predictions, while speech emotion recognition allows computers to identify and understand human emotions from speech. These technologies contribute to the creation of innovative human–computer interaction (HCI) applications. Deep learning algorithms, capable of learning high-level features directly from raw data, have given rise to new emotion recognition approaches employing models trained on advanced speech representations like spectrograms and time–frequency representations. This study introduces CNN and LSTM models with GWO optimization, aiming to determine optimal parameters for achieving enhanced accuracy within a specified parameter set. The proposed CNN and LSTM models with GWO optimization underwent performance testing on four diverse datasets—RAVDESS, SAVEE, TESS, and EMODB. The results indicated superior performance of the models compared to linear and kernelized SVM, with or without GWO optimizers.

Effective time-series Data Augmentation with Analytic Wavelets for bearing fault diagnosis

In the realm of rotary machine maintenance, rolling bearings emerge as crucial yet frequently vulnerable components. Ensuring their operational integrity is pivotal for the overall safety and performance of the machinery. Traditional diagnostic methods have utilized a variety of signal processing techniques, ranging from blind source separation to wavelet analysis, and more recently, deep neural networks (DNNs). However, a significant challenge in this domain is the scarcity of labeled failure data in real-world applications, leading to a reliance on data augmentation (DA) techniques to enhance the quality and quantity of the training data. Compounding this problem is the use of DA techniques tailored for images, which are not suitable for the time–frequency representations. This study proposes an innovative Data Augmentation with Analytic Wavelets (DAAW) approach, tailored specifically for time-series (TS) data classification in bearing fault diagnosis. The essence of DAAW lies in its ability to generate synthetic scalogram samples that closely mirror the properties of original samples. The proposed DAAW is directly applied at the input stage of a learning model by successively adjusting predefined parameters to generate scalograms, without the need for auxiliary DA algorithms. This technique is versatile in generating synthetic time–frequency data samples, and the parameterization enables it to be tailored to different time series datasets. Through experimentation with publicly available datasets, we show that combining a simple Convolutional Neural Network (CNN) with the proposed approach results in improved performance. It outperforms existing methods, achieving a mean classification accuracy of 99.49%. The critical distance for average accuracy, as determined by the Nemenyi post-hoc test, is 0.4619. Our results also show enhanced performance with accuracy rates of 90.14%, 78.94%, and 62.86% even under low signal-to-noise ratio (SNR) conditions, using 80%, 60%, and 50% of the dataset, respectively, for model training. Notably, its efficacy remains robust even under challenging low SNR conditions. The proposed DAAW significantly improves classification accuracy in bearing fault diagnosis, simultaneously obviating the need for proxy DA tasks.

Instantaneous frequency identification for nonstationary signals of time-varying structures using enhanced synchroextracting wavelet transform and dynamic optimization

Civil engineering structures under ambient excitations are essentially time-varying and nonlinear structural systems and the resultant response signals exhibit non-stationarity. To reveal the time-varying characteristics of non-stationary signals, time frequency analysis methods with high resolutions are required. Although synchroextracting short time Fourier transform (SESTFT) is able to generate more energy concentrated time frequency representations and allow signal reconstruction, its major disadvantage is the window function is fixed. To address this issue, an enhanced synchroextracting wavelet transform (ESEWT) method is proposed to refine frequency bands by combing synchroextracting and continuous wavelet transform. After that, dynamic optimization (DO) is used to extract instantaneous frequency (IF) curves within the refined frequency bands. Two numerical examples and an experimental study case are investigated to illustrate the effectiveness and accuracy of the proposed method. The results demonstrate that the proposed ESEWT is capable of extracting frequency bands more accurately and its combination with DO identifies IFs of non-stationary signals better than current time frequency analysis methods such as SESTFT.