Time-frequency Representation Research Articles

CWT-ViT: A time–frequency representation and vision transformer-based framework for automated robotic surgical skill assessment

Surgical skill assessment currently hinges on the manual observations of senior surgeons, and the assessment process is inherently time-consuming and subjective. Hence, there is a need to develop machine learning-based automated robotic surgical skill assessment. However, the existing machine learning-based works are only built in either the time domain or frequency domain but have never considered the investigation on the time–frequency domain. To fill the research gap, we explore the representation of the surgery motion data in the time–frequency domain. In this study, we propose a novel automated robotic surgical skill assessment framework called Continuous Wavelet Transform-Vision Transformer (CWT-ViT). We apply continuous wavelet transform, i.e., a time–frequency representation method, to convert robotic surgery kinematic data to synthesis images. Furthermore, by taking advantage of the prior knowledge of the da Vinci surgical system, we design a four branches-based architecture, each branch representing a robotic manipulator. We have conducted extensive experiments and achieved comparable results on the benchmark robotic surgical skill dataset JHU-ISI Gesture and Skill Assessment Working Set (JIGSAWS). Our proposed CWT-ViT framework has demonstrated the feasibility of applying time–frequency representation for automated robotic surgical skill assessment using kinematic data. The code is available at https://github.com/yiming95/CWT-ViT-Surgery.

Extraction of instantaneous frequencies for signals with intersecting and intermittent trajectories

A multicomponent signal usually presents multiple trajectories with time-varying frequencies and amplitudes in a time–frequency distribution (TFD). One can extract the ridges corresponding to true signal components and then reconstruct them to recover signal signatures. Most current practices for ridge extraction assume that each trajectory runs throughout the entire time axis without cross-terms. However, this hypothesis is inconsistent with the truth of many measured signals. The increasing application occasions require further consideration of complicated intersecting and intermittent cases. This study addresses this issue and proposes a novel intersecting and intermittent trajectory tracking (IITT) approach. We first develop a data-driven method to effectively isolate peaks from noises in a TFD and generate a dependable peak spectrum. Then, we propose a dynamic optimization tracking function to decide upon the acceptance of the peaks corresponding to an individual component based on the purified spectrum. The IITT approach fully exploits the information from the raw signal without any prior knowledge while promising robustness to the variations of ridge numbers, ridges’ births and deaths, and its continuation and discontinuation. Two simulated and three measured signals are utilized to assess the performance of the proposed IITT. The success elements of the IITT are revealed and discussed in detail at the end of the paper.

AI-Assisted Sensor System for the Acetone and Ethanol Detection Using Commercial Metal Oxide-Based Sensor Arrays and Convolutional Neural Network

In recent decades, the traditional landscape of volatile organic compound (VOC) sensing has adopted a new perspective in enhancing the detection of useful VOCs using data intelligence to extract constructive insights of the sensor behaviour towards multiple gases. In the domain of gas sensing, VOCs such as acetone and ethanol have been widely used in sensor testing due to their closely related chemical properties, which poses a challenge in discrimination. Therefore, this study aims to discriminate acetone from ethanol with the use of readily available commercial metal oxide (MOx) sensors through the implementation of Deep Learning (DL) techniques. The data set obtained after exposing a sensing array comprising various MOx sensors to acetone and ethanol was converted to a time-frequency representation known as a scalogram to train and test a multi-input convolutional neural network (CNN). The results show that training the CNN model on the sensor array data set yields better results than with an individual sensor data set. The findings of this research substantiated the ability of DL models to better capture the dynamic interaction of the sensors with acetone and ethanol, leading to the implication of the DL classifier having the capacity to reject sensor inconsistencies and variations in the responses. This research holds promise for advancing health monitoring and disease detection, as the combination of MOx sensors and DL techniques is expected to make significant future contributions in these areas.

Refined linear chirplet transform for time–frequency analysis of non-stationary signals

Time-Frequency Analysis (TFA) stands as a pivotal technique for unraveling the inherent properties of signals, which are omnipresent across natural phenomena. Current methodologies encounter significant challenges in the analysis of non-stationary signals, especially those characterized by closely-spaced or intersecting instantaneous frequencies. In this study, we present the refined linear chirplet transform (RLCT), inspired by the concept of the generalized linear chirplet transform (GLCT), to derive the time–frequency representation of non-stationary signals. By increasing the number of chirp rate searches in GLCT, one can derive higher time–frequency concentration, however, this costs much higher computational resources. In the proposed RLCT, the capacity in searching the optimal chirp rates, or equivalently the rotation angles on the time–frequency plane, is dramatically improved by adopting a local search strategy for mono-component signals. As a sequence, the resolution for the multi-component signals, even with overlapping instantaneous frequency, is greatly enhanced by adopting the idea of the adaptive linear chirplet transform (ALCT), where the components with high energy concentration are consecutively detected and subtracted from consideration. Two critical metrics are employed to assess the performance (accuracy and resolution) of the TFA methods: total energy and energy ratio. The latter is defined as the proportion of energy within a narrowly defined frequency band centered around the actual instantaneous frequency, relative to the total energy. The validation examples demonstrate that the LCT-based methods dramatically enhance the accuracy compared to short-time Fourier transform, in terms of total energy. Furthermore, among the LCT family investigated, the proposed method offers marked improvements in resolution, in terms of energy ratio, while maintaining algorithmic simplicity. The seismic responses of a lighthouse during a severe earthquake are investigated by the RLCT to reveal its frequency drop and recovery due to the opening and closing of the existing cracks.

A Positioning and Recognition Method Based on Bilinear Time–Frequency Transform and CNN for Long-Range Asymmetric Interferometer Vibration Sensor

A Positioning and Recognition Method Based on Bilinear Time–Frequency Transform and CNN for Long-Range Asymmetric Interferometer Vibration Sensor

Horizontal rearrangement frequency domain chirplet transform: algorithm and applications

Abstract Based on the idea of optimal time–frequency analysis (TFA) theory, a time–frequency post-processing method called horizontal rearrangement frequency domain chirplet transform (HRFCT) is proposed. The proposed method is designed to analyze non-stationary transient signals with nonlinear group delay (GD), aiming at obtaining energy-concentrated and accurate time–frequency representation (TFR). First, based on the image characteristics of the constructed amplitude function, the selection criteria of GD are defined, that is, the local maximum of amplitude function and the local minimum of absolute value of derivative of amplitude function are calculated. Next, according to the calculated GD, the coefficients on the GD trajectory are adaptively extracted, and the coefficients that cause the TFR to be blurred are discarded. Finally, the analysis result of a two-component simulation signal demonstrates the satisfactory performance of the HRFCT, and compared with four advanced TFA methods, the competitiveness of the HRFCT is verified. Additionally, the application of the HRFCT in the processing of constant and variable speed bearing signals highlights its prospects in transient signal analysis and rotating machinery condition monitoring.

Separation of overlapping non-stationary signals and compressive sensing-based reconstruction using instantaneous frequency estimation

Compressive sensing uses an incomplete sample set to consider signal reconstruction with sparse basis representations over a specific transform domain. This work examines the difficulties encountered in various signal processing applications, such as data overlap with the target signal in time and frequency domains. The conventional filtering and windowing processes fail to promote better results in recovering the desired signal. The non-stationary signal with time-frequency representations is highly significant for certain valuable applications. An efficient signal separation and reconstruction method based on instantaneous frequency estimate is proposed to improve the non-stationary signal outcome. The proposed research includes speech signal data acquisition, signal component decomposition, instantaneous frequency estimation, overlapped signal separation, and reconstruction. Two datasets, Dacon Overlapped Speech and LJ Speech Dataset, are utilized to generate a new overlapped signal dataset. Initially, the generated speech signals are collected and decomposed into several monocomponents using Synchrosqueezing Wavelet Transform (SynWav), Renyi's quadratic entropy (RQuad), Narrow bandpass filtering (NBand) and single frequency filtering (SFreq). The instantaneous frequencies are estimated using the Hilbert-Huang transform (HilHuT) to generate a time-frequency representation of multi-component signals. The overlapped signals are separated and reconstructed to a desired signal using a Compressive sensing-based Discrete Cosine transform with amended intrinsic chirp separation (CDcT_AIChirS). The proposed model is simulated using PYTHON to examine the performances. In comparison, the mean square error is calculated to be 2.71, the root mean square error is calculated to be 1.64, and the signal to noise ratio is calculated to be 32dB.

An LDoS attack detection method based on FSWT time–frequency distribution

The Low-rate Denial-of-Service (LDoS) attack is a stealthy and periodic attack, which belongs to the category of DoS attacks. The LDoS attack maliciously preempts and consumes target resources, causing the targeted network performance to decline and affecting the service quality. The LDoS attack is highly destructive and difficult to detect and defend against due to its abnormal attack behavior. In this paper, we discuss the network traffic behavior in the time domain, frequency domain, and time–frequency domain, and we find that the time–frequency domain contains more detailed information than the time domain and frequency domain alone. Considering the limitations of the existing time–frequency domain transformation methods, an LDoS attack detection method based on Frequency Slice Wavelet Transformation (FSWT) is presented in this paper. The entropy, ratio of energy, contrast, and correlation are extracted from the time–frequency distribution to depict the network traffic, and a decision tree is trained to detect the LDoS attack in this paper. According to the experimental results on NS2, testbed, and the comparative experiment, we conclude that the method presented in this paper performs well.

Synchronous match-reassigning transform: A method for extracting time-varying features of planetary gearbox

Instantaneous frequency (IF) time-varying closely spaced and severe background noise can introduce fake components into the time-frequency representation (TFR) of most time-frequency analysis (TFA) methods, especially for the vibration signals of variable-speed planetary gearboxes. These fake components lead to a decrease in the readability of the TFR, making it difficult to extract the true IF. To address this issue, a new TFA method called Synchronous Match-Reassigning Transform (SMRT) is proposed. Firstly, a polynomial chirp rate with a single parameter adjustment is constructed to achieve precise matching of the IF. Secondly, a frequency reassignment operator is constructed for the fine refinement of the IF ridge of Synchronous Match-Reassigning Operator (SMRO). Finally, SMRO is combined with VKF to achieve adaptive reconstruction of the true IF and to remove fake components. The TFR of SMRT has high readability and low energy leakage. The superiority and effectiveness of this method have been verified through synthetic signals and actual signals.

Unveil sleep spindles with concentration of frequency and time (ConceFT)

Objective. Sleep spindles contain crucial brain dynamics information. We introduce the novel non-linear time-frequency (TF) analysis tool ‘Concentration of Frequency and Time’ (ConceFT) to create an interpretable automated algorithm for sleep spindle annotation in EEG data and to measure spindle instantaneous frequencies (IFs). Approach. ConceFT effectively reduces stochastic EEG influence, enhancing spindle visibility in the TF representation. Our automated spindle detection algorithm, ConceFT-Spindle (ConceFT-S), is compared to A7 (non-deep learning) and SUMO (deep learning) using Dream and Montreal Archive of Sleep Studies (MASS) benchmark databases. We also quantify spindle IF dynamics. Main results. ConceFT-S achieves F1 scores of 0.765 in Dream and 0.791 in MASS, which surpass A7 and SUMO. We reveal that spindle IF is generally nonlinear. Significance. ConceFT offers an accurate, interpretable EEG-based sleep spindle detection algorithm and enables spindle IF quantification.

Pre-trained Bi-LSTM model for automated classification of ventricular arrhythmias using 1-D and 2-D ECG

Number of cardiac conditions have been associated with abnormal heartbeat (arrhythmia) such as ventricular fibrillation (Vfib), ventricular flutter (Vfl), and ventricular tachycardia (Vta). This is a difficult and essential job for timely clinical assessment and identification of these potentially life-threatening heart arrhythmias. With the aid of a one-dimensional electrocardiogram (ECG) signal and its associated two-dimensional image, the suggested method provides a strategy for the detection of time-frequency interpretation (Vfib, Vfl, and Vta). A four-stage cascaded Savitzky-Golay (SG) filter is used after a 2-stage median filter to preprocess the ECG signal. This technique employs z-score normalisation after brief (2 sec) ECG readings. The classification of these ECG segments (1-D) and associated time-frequency representation pictures (2-D) was explored separately using a bi-directional long short-term memory-based network. Eight distinct categorization scenarios were examined, and then an average accuracy of 99.67% for 1-D ECG and 99.87% for 2-D ECG signal was attained.

Optimal Time-Frequency Distribution for Instantaneous Frequency Estimation of Signals With Known IF Patterns

Optimal Time-Frequency Distribution for Instantaneous Frequency Estimation of Signals With Known IF Patterns

A Novel Time-Frequency Parameterization Method for Oscillations in Specific Frequency Bands and Its Application on OPM-MEG.

Time-frequency parameterization for oscillations in specific frequency bands reflects the dynamic changes in the brain. It is related to cognitive behavior and diseases and has received significant attention in neuroscience. However, many studies do not consider the impact of the aperiodic noise and neural activity, including their time-varying fluctuations. Some studies are limited by the low resolution of the time-frequency spectrum and parameter-solved operation. Therefore, this paper proposes super-resolution time-frequency periodic parameterization of (transient) oscillation (STPPTO). STPPTO obtains a super-resolution time-frequency spectrum with Superlet transform. Then, the time-frequency representation of oscillations is obtained by removing the aperiodic component fitted in a time-resolved way. Finally, the definition of transient events is used to parameterize oscillations. The performance of this method is validated on simulated data and its reliability is demonstrated on magnetoencephalography. We show how it can be used to explore and analyze oscillatory activity under rhythmic stimulation.

Local structure and effective dimensionality of time series data sets

The goal of this paper is to develop novel tools for understanding the local structure of systems of functions, e.g. time-series data points. The proposed tools include a total correlation function, the Cohen class of the data set, the data operator and the average lack of concentration. The Cohen class of the data operator gives a time-frequency representation of the data set. Furthermore, we show that the von Neumann entropy of the data operator captures local features of the data set and that it is related to the notion of effective dimensionality. The accumulated Cohen class of the data operator gives us a low-dimensional representation of the data set and we quantify this in terms of the average lack of concentration and the von Neumann entropy of the data operator and an improvement of the Berezin-Lieb inequality using the projection functional of the data augmentation operator. The framework for our approach is provided by quantum harmonic analysis.

Benchmarking Time-Frequency Representations of Phonocardiogram Signals for Classification of Valvular Heart Diseases Using Deep Features and Machine Learning

Heart sounds and murmur provide crucial diagnosis information for valvular heart diseases (VHD). A phonocardiogram (PCG) combined with modern digital processing techniques provides a complementary tool for clinicians. This article proposes a benchmark different time–frequency representations, which are spectograms, mel-spectograms and cochleagrams for obtaining images, in addition to the use of two interpolation techniques to improve the quality of the images, which are bicubic and Lanczos. Deep features are extracted from a pretrained model called VGG16, and for feature reduction, the Boruta algorithm is applied. To evaluate the models and obtain more precise results, nested cross-validation is used. The best results achieved in this study were for the cochleagram with 99.2% accuracy and mel-spectogram representation with the bicubic interpolation technique, which reached 99.4% accuracy, both having a support vector machine (SVM) as a classifier algorithm. Overall, this study highlights the potential of time–frequency representations of PCG signals combined with modern digital processing techniques and machine learning algorithms for accurate diagnosis of VHD.

Low-dimensional signal representations for massive black hole binary signals analysis from LISA data

The space-based gravitational wave observatory LISA will provide a wealth of information to analyze massive black hole binaries with high chirp masses, beyond $10^5$ solar masses. The large number of expected MBHBs (one event a day on average) increases the risk of overlapping between events. As well, the data will be contaminated with non-stationary artifacts, such as glitches and data gaps, which are expected to strongly impact the MBHB analysis, which mandates the development of dedicated detection and retrieval methods on long time intervals. Building upon a methodological approach we introduced for galactic binaries, in this article we investigate an original non-parametric recovery of MBHB signals from measurements with instrumental noise typical of LISA in order to tackle detection and signal reconstruction tasks on long time intervals. We investigated different approaches based on sparse signal modeling and machine learning. In this framework, we focused on recovering MBHB waveforms on long time intervals, which is a building block to further tackling more general signal recovery problems, from gap mitigation to unmixing overlapped signals. To that end, we introduced a hybrid method called SCARF (sparse chirp adaptive representation in Fourier), which combines a deep learning modeling of the merger of the MBHB with a specific adaptive time-frequency representation of the inspiral. Numerical experiments have been carried out on simulations of single MBHB events that account for the LISA response and with realistic realizations of noise. We checked the performances of the proposed hybrid method for the fast detection and recovery of the MBHB.

A Multidimensional Health Indicator Based on Autoregressive Power Spectral Density for Machine Condition Monitoring.

Condition monitoring (CM) is the basis of prognostics and health management (PHM), which is gaining more and more importance in the industrial world. CM, which refers to the tracking of industrial equipment's state of health during operations, plays, in fact, a significant role in the reliability, safety, and efficiency of industrial operations. This paper proposes a data-driven CM approach based on the autoregressive (AR) modeling of the acquired sensor data and their analysis within frequency subbands. The number and size of the bands are determined with negligible human intervention, analyzing only the time-frequency representation of the signal of interest under normal system operating conditions. In particular, the approach exploits the synchrosqueezing transform to improve the signal energy distribution in the time-frequency plane, defining a multidimensional health indicator built on the basis of the AR power spectral density and the symmetric Itakura-Saito spectral distance. The described health indicator proved capable of detecting changes in the signal spectrum due to the occurrence of faults. After the initial definition of the bands and the calculation of the characteristics of the nominal AR spectrum, the procedure requires no further intervention and can be used for online condition monitoring and fault diagnosis. Since it is based on the comparison of spectra under different operating conditions, its applicability depends neither on the nature of the acquired signal nor on a specific system to be monitored. As an example, the effectiveness of the proposed method was favorably tested using real data available in the Case Western Reserve University (CWRU) Bearing Data Center, a widely known and used benchmark.

An adaptive chatter detection method for micro milling based on variational mode extraction

Current chatter detection methods using variational mode decomposition (VMD) hardly select appropriate intrinsic mode functions (IMFs) for characterizing chatter in micro milling. This article introduces a novel variational mode extraction (VME) method to isolate a single IMF representing chatter. An adaptive filter reduces chatter-independent components, enabling VME to identify the desired IMF with adaptive calculations for center frequency and penalty factor. The Hilbert–Huang transform is then applied to obtain the time–frequency distribution of the obtained IMF, and various features are computed. Finally, two methods are employed: streaming feature selection considers feature interaction, and radial basis function support vector machine selects pertinent features to establish the chatter detection model. Extensive micro milling tests validate the method, achieving an average detection accuracy of 99.34%.

Identification of end-milling chatter based on comprehensive feature fusion

The main barrier impeding the advancement of high-speed milling is chatter, which has a detrimental effect on the dimensional accuracy and quality of the finished workpiece. A reliable and precise chatter identification method is essential to improving the quality of machining. This paper presents a novel method for chatter identification using a comprehensive feature fusion of the Short-Time Fourier Transform (STFT) and the Fourier Synchrosqueezing Transform (FSST). The Wavelet Packet Transform (WPT) was used to pre-process the collected vibration and force signals. Wavelet packets with rich chatter information were then selected and reconstructed for further analysis. To reduce the effects of the rotating frequency and generate a hybrid spectrum with high resolution, a Gabor time-frequency filter is employed. As chatter indicators, standard deviation, skewness, and root mean square are computed. With a higher Time-Frequency Representation (TFR) resolution and a shorter computation time of 0.46 and 0.97 s across vibration and force signals, the proposed method outperforms conventional STFT and FSST. As a result, it can be used to reliably identify chatter from the onset, which is beneficial for machining monitoring.

Graph constrained empirical wavelet transform and its application in bearing fault diagnosis

The signal decomposition based on frequency domain distribution is a fundamental methodology for mechanical component fault diagnosis. However, existing methods face challenges such as susceptibility to noise interference and limited adaptability. Therefore, this paper proposes the graph constrained empirical wavelet transform (GCEWT) method. This method introduces structured information, such as the interrelationships among different parts of the frequency domain distribution of vibration signals, into the boundary detection process of empirical wavelet transform. The high-dimensional connectivity among different parts of the time-frequency distribution is utilized to construct an adjacency matrix. By constructing an adjacent graph, the proposed method encodes the adjacency relationships among frequency bands to constrain the low-dimensional spatial relationships between them. In conjunction with spectral clustering algorithms, the GCEWT method determines the boundaries for empirical wavelet transformation in the frequency domain. This approach achieves structured and adaptive decomposition of vibration signals from components of critical equipment, facilitating the structured and adaptive extraction of fault features. The effectiveness of the proposed method is validated using vibration data from both wind turbine drivetrain systems and aircraft engines. The experimental results demonstrate that the proposed method yields more reasonable signal decomposition results compared to traditional algorithms. Additionally, the proposed method proves to be more effective in extracting weak fault features of bearings in the presence of noise.