Time Domain Information Research Articles

RPPG-Based Heart Rate Estimation Using Spatial-Temporal Attention Network

Remote photoplethysmography (rPPG) based on the computer vision technology is widely used to calculate the heart rate (HR) from facial videos. Existing rPPG techniques have been subject to some limitations [e.g., highly redundant spatial information, head movement noise, and region of interest (ROI) selection]. To address these limitations, this article introduces an effective spatial-temporal attention network. A temporal fusion module is first proposed to fully exploit the time-domain information, aiming to reduce the redundant video information and strengthen the association relationships of long-range videos. Furthermore, a spatial attention mechanism is designed in the backbone net to precisely target the skin ROIs. Finally, to assist the network in learning the weights between channels, we project the RGB images using the plane orthogonal to skin (POS) algorithm and add motion representation to complement physiological signals’ extraction. The extensive experiments on the public PURE, MMSE-HR, and UBFC-rPPG datasets demonstrate that our model achieves competitive results compared with other methods.

Ball screw fault diagnosis based on continuous wavelet transform and two-dimensional convolution neural network

Due to extreme operating conditions such as high-speed and heavy loads, ball screws are prone to damages, that affect the accuracy and operational safety of the mechanical equipment. As strong background noise and weak fault characteristics, it is difficult to capture the inherent fault state only depending on the time-domain or frequency-domain information from the vibration signal. In this paper, a fault diagnosis method for the ball screw based on continuous wavelet transform (CWT) and two-dimensional convolutional neural network (2DCNN) is proposed. The noise-reducing vibration signal is obtained via CWT. The time-frequency graph of the noise reduction signal can more comprehensively reflect the fault information of the ball screw. The time-frequency graph is used as the input to train and test the 2DCNN. Finally, diagnosis results of different types of faults reveal that the proposed CWT-2DCNN fault diagnosis method can achieve an average recognition rate of 99.67%. Compared with one-dimensional convolutional neural network (1DCNN) and traditional BP neural network, the proposed method has fast network convergence and high recognition accuracy. Time-frequency graphs of the noise-reduced signal used as fault features for classification can effectively avoid the problem of uncertainty due to the manual extraction of features. The proposed method has high application potential in the field of ball screw pair fault diagnosis.

Cross-validation of time-depth conversion and evaluation of different approaches in the Mesopotamian Basin, Iraq

The time-depth conversion process is a significant task in seismic interpretation to establish the link between geophysical information in the time domain and geological information in the depth domain at/away from well locations. Selecting the suitable velocity model for time-depth conversion to generate an accurate depth map is difficult if the accuracy of these models is unknown. In the current study, the cross-validation technique is used as a tool to diagnose and evaluate the performance of time-depth conversion at/away from well controls to predict the depth of Top Hartha and Zubair reservoirs using the dataset of East Baghdad Oil Field. To test this technique, four common velocity model approaches used for time-depth conversion with different scenarios of velocity parameters (initial velocity V 0 and depth gradient (K)) were applied to produce ten velocity models (1–10). According to the gradient variation of velocity with depth, check shot analysis, the velocity models (1–10) include three key velocities layer-cakes: Layer 1 (Middle Miocene-Upper Cretaceous), Layer 2 (Upper Cretaceous), and Layer 3 (Lower Cretaceous) with 18 horizons from Middle Miocene down to Lower Cretaceous. The cross-validation analysis reveals that the velocity model with a variable surface initial velocity and constant depth gradient (Model 9) was the most accurate with fewer mistie between actual and predicted depth. Consequently, this model is used to construct the depth map of the Hartha and Zubair reservoirs. Finally, this study progresses a workflow that can be applied to the region with any geological setting to investigate time-depth conversion uncertainty.

Dynamic Load Identification of Unspecified Metal Structures by Measuring Their Response

Many engineering structures are made of metal composite materials. External load information is a key issue for the design and condition monitoring of the structures. Due to the limitation of measurement technology and the external environment, it is difficult to directly measure dynamic loads on structures in many circumstances. This paper focuses on evaluating the external load applied on a structure with unknown dynamic properties. We proposed a novel dynamic load identification method that is based on the Bayesian principle coupled with the extended Kalman filter method. Firstly, the modal parameters are identified under ambient excitation using the Bayesian fast Fourier transform method (FFT). The posterior probability density function (PDF) and covariance of the modal parameters are obtained by the Fourier transform of the response data, and then the modal parameters of the structure are obtained based on unconstrained optimization. Next, the extended Kalman filter method in the modal space is used to update the modal parameters and identify the time-domain information of dynamic loads. The accuracy of the proposed theory was evaluated experimentally using a Bernoulli−Euler beam. The results showed that the method is feasible and efficient.

Modulation Recognition Using Signal Enhancement and Multistage Attention Mechanism

Robustness against noise is critical for modulation recognition (MR) approaches deployed in real-world communication systems. In MR systems, a corrupted signal is normally enhanced using low-level signal enhancement (SE) before signal classification (SC). Many existing approaches address signal distortion problems by compartmentalizing SE from SC. While those approaches allow for efficient development, they also dictate compartmentalized performance metrics, without feedback from the SC module. For example, SE modules are designed using perceptual signal quality metrics but not with SC in mind. To improve the effectiveness of SE on MR, this paper proposes a joint learning framework consisting of three cascaded modules: dual-channel spectrum fusion, SE, and SC. Instead of separately processing SE and SC, these three modules are integrated into one framework and jointly trained with a single recognition loss. In contrast to estimating clean signals, the SE module in the proposed joint learning framework is trained to predict a ratio mask and find important time-frequency bins for the SC module. We integrate a multistage attention mechanism into the framework to further increase the robustness. The multistage attention mechanism is deployed to strengthen the recognition-related features learned from context information in channel, time, and frequency domains. We evaluate the recognition performance of the proposed framework and its modules on two benchmark datasets: RadioML2016.10a and RadioML2016.10b. The experiment results show that the proposed joint learning framework outperforms the separate learning framework. Moreover, comparisons are performed with several existing learning-based MR methods in the literature. The proposed joint learning framework leads to significant performance improvement, especially for modulated signals corrupted by channel noise.

Multiscale entropy analysis of astronomical time series

Context.The multiscale entropy assesses the complexity of a signal across different timescales. It originates from the biomedical domain and was recently successfully used to characterize light curves as part of a supervised machine learning framework to classify stellar variability.Aims.We aim to explore the behavior of the multiscale entropy in detail by studying its algorithmic properties in a stellar variability context and by linking it with traditional astronomical time series analysis methods and metrics such as the Lomb-Scargle periodogram. We subsequently use the multiscale entropy as the basis for an interpretable clustering framework that can distinguish hybrid pulsators with bothp- and g-modes from stars with onlyp-mode pulsations, such asδScuti (δSct) stars, or from stars with onlyg-mode pulsations, such asγDoradus (γDor) stars.Methods.We calculate the multiscale entropy for a set ofKeplerlight curves and simulated sine waves. We link the multiscale entropy to the type of stellar variability and to the frequency content of a signal through a correlation analysis and a set of simulations. The dimensionality of the multiscale entropy is reduced to two dimensions and is subsequently used as input to the HDBSCAN density-based clustering algorithm in order to find the hybrid pulsators within sets ofδSct andγDor stars that were observed byKepler.Results.We find that the multiscale entropy is a powerful tool for capturing variability patterns in stellar light curves. The multiscale entropy provides insights into the pulsation structure of a star and reveals how short- and long-term variability interact with each other based on time-domain information only. We also show that the multiscale entropy is correlated to the frequency content of a stellar signal and in particular to the near-core rotation rates ofg-mode pulsators. We find that our new clustering framework can successfully identify the hybrid pulsators with bothp- andg-modes in sets ofδSct andγDor stars, respectively. The benefit of our clustering framework is that it is unsupervised. It therefore does not require previously labeled data and hence is not biased by previous knowledge.

Automatic Liver Tumor Segmentation on Dynamic Contrast Enhanced MRI Using 4D Information: Deep Learning Model Based on 3D Convolution and Convolutional LSTM.

Accurate segmentation of liver tumors, which could help physicians make appropriate treatment decisions and assess the effectiveness of surgical treatment, is crucial for the clinical diagnosis of liver cancer. In this study, we propose a 4-dimensional (4D) deep learning model based on 3D convolution and convolutional long short-term memory (C-LSTM) for hepatocellular carcinoma (HCC) lesion segmentation. The proposed deep learning model utilizes 4D information on dynamic contrast enhanced (DCE) magnetic resonance imaging (MRI) images to assist liver tumor segmentation. Specifically, a shallow U-net based 3D CNN module was designed to extract 3D spatial domain features from each DCE phase, followed by a 4-layer C-LSTM network module for time domain information exploitation. The combined information of multi-phase DCE images and the manner by which tissue imaging features change on multi-contrast images allow the network to more effectively learn the characteristics of HCC, resulting in better segmentation performance. The proposed model achieved a Dice score of 0.825± 0.077, a Hausdorff distance of 12.84± 8.14 mm, and a volume similarity of 0.891± 0.080 for liver tumor segmentation, which outperformed the 3D U-net model, RA-UNet model and other models in the ablation study in both internal and external test sets. Moreover, the performance of the proposed model is comparable to the nnU-Net model, which showed state-of-the-art performance in many segmentation tasks, with significantly reduced prediction time. The proposed 3D convolution and C-LSTM based model can achieve accurate segmentation of HCC lesions.

Influence of particle characteristics on dynamic characteristics of tail beam under coal rock caving impact

In order to improve the accuracy of simulation parameters used in the discrete element simulation test of a fully mechanized top-coal caving process and further explore the intelligent fully mechanized coal caving technology, this research work studies the influence of particle characteristics on the dynamic response of tail beams under the impact of caving coal rock in the process of coal caving. Based on the interface technology, the EDEM–RecurDyn–AMESim multi-domain collaborative simulation top-coal caving support is a built-model of a hydraulic mechanical integration system for scraper conveyors, which is used to simulate the coal caving process of the top-coal caving support to obtain the vibration signal of the tail beam of the top-coal caving support. This model can also be used to convert it into a two-dimensional time-spectrum image using the short-time Fourier transform (STFT) algorithm. Several groups of simulation tests were carried out on different particle radii, standard deviation of particle normal distribution, and particle slenderness ratio. The time-domain information and frequency-domain information obtained from the simulation were analyzed and compared. Combined with the vibration signal of the tail beam measured on the spot, the optimal setting parameters of the multi-field collaborative virtual prototype simulation were obtained. Compared with the data measured in the coal mine, the relative error of the maximum vibration value of the tail beam is only 3.8%, the minimum relative error is 5.5%, and the relative error of the root mean square value is 14%, which verifies the method and simulation results. This method solves the problems of difficult on-site sampling, high risk coefficient, and high test cost and promotes the development of an intelligent process of coal mining.

Dynamic memory network with spatial-temporal feature fusion for visual tracking

Recently, single object tracking has demonstrated great success. However, due to various problems caused by fast motion, occlusion, and deformation, it is still intractable for traditional trackers to adapt to changes in object appearance. In this work, a dynamic memory network is proposed for visual tracking to handle the template matching process globally. More specifically, we first build a memory model, which consists of a memory feature fusion module and a memory bank. By the memory model, the network not only accepts the first frame as initial information but also memorizes the selective frames in the video sequence to provide rich time-domain information. Second, an innovative sampling strategy is adopted in the tracking process. By updating the template and guiding the selection of memory frames, our model can output higher-quality features. In addition, a spatial-channel fused attention module that effectively improves the representational capability and discriminability of the model is introduced. Our proposed method obtains compelling results on six challenging tracking benchmarks, including the OTB100, VOT2019, UAV123, NFS, GOT-10k, and LaSOT datasets. Extensive experiments demonstrate that our approach shows satisfactory robustness and leading application potential in real-time speed.

Temporal convolution network‐based time frequency domain integrated model of multiple arch dam deformation and quantification of the load impact

Deformation is an intuitive reflection of the safety status of a dam. The construction of a dam deformation prediction model can predict the deformation and interpret the effects of environmental loads. The current research mainly focuses on the predictive ability of the model and rarely involves the interpretation of the load impact on deformation. Meanwhile, the selection of the model factors, such as water pressure factors and temperature factors, mostly relies on prior knowledge. In addition, the complex structure of multiple arch dams makes it difficult to capture the relationship between deformation and environmental loads. Consequently, the performance of conventional models based only on time domain information may be insufficient. In this paper, a deformation prediction model is established by integrating time frequency domain information. First, the deformation and load monitoring data are decomposed and regrouped according to the frequency characteristic of signals via kurtosis index-based VMD. Second, the sequence relationship between the dam deformation and loads under different frequency characteristics is automatically captured based on the temporal convolution network (TCN). Finally, a quantitative method of the load impact is proposed based on the network parameters. The case results show that the proposed modeling paradigm has significantly improved the prediction accuracy. The quantification result of the load impact on the horizontal displacement change of the dam conforms to the actual state of the project during the analysis period. The work effectively supplements the research on the prediction of ML-based models and interpretation of the load impact on deformation.

Application of optical measurement techniques during various stages of pregnancy

Optical measurement techniques have evolved since the mid-2000's due to the advancement in computer processing and digital camera sensors. These techniques allow for the digitization of full three-dimensional (3D) surfaces and provide time-domain surface information data with high details not previously possible with more traditional measurement techniques. A variety of industries are using these technologies with a focus on high accuracy industrial applications including aerospace and automotive. This paper introduces optical measurements techniques used to provide quantifiable data and visual information of a woman's abdomen during various stages of pregnancy. Two non-invasive techniques are used for this study including structured light (SL) scanning and digital image correlation (DIC). An overview of these techniques, the process for data collection, results of the data, and opportunities for future studies are presented in this paper. The structured light scanning was used to capture a 3D surface of the woman's abdomen at four to six week intervals starting at week 15 of maternity and ending six days prior to birth. The results quantify growth during each stage of pregnancy along with full surface data in addition to postnatal results. The DIC data was collected during week 30 when the baby was very active in the womb and external deformations on the woman's abdomen were observed visually. The results from these studies were not intended to be a formal medical study but to provide feasibility of the SL and DIC process during the stages of pregnancy and encourage unique uses of otherwise industrial measurements.

A 96.9-dB-Resolution 109-μW Second-Order Robust Closed-Loop VCO-Based Sensor Interface for Multiplexed Single-Ended Resistance Readout in 180-nm CMOS

This article presents a highly digital robust voltage-controlled oscillator (VCO)-based front end for multiplexed single-ended resistive sensor readout applications. The architecture features a modified digital phase-locked loop (DPLL) structure that enables second-order noise shaping without any operational transconductance amplifiers (OTAs) and a single feedback digital-to-analog converter (DAC). The direct conversion of the sensor input resistance to time-domain information obviates the need for any conditioning circuit, thus resulting in a highly digital and area-compact architecture. The closed loop substantially reduces the amplitude of the signal at the VCO input, thereby achieving high linearity. To ensure high robustness against supply and temperature variations, a double measurement approach is employed. Fabricated in a 180-nm CMOS process with a 0.1-mm <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> active area, the sensor readout consumes 109 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mu \text{W}$ </tex-math></inline-formula> of power and achieves a resolution of 96.9 dB for 0.5-ms conversion time, resulting in a Schreier figure of merit (FoM) of 166.5 dB. The circuit only requires a cheap and efficient single-point trimming calibration at room temperature. The low conversion time enables multiplexing among multiple sensor readouts. The supply and temperature sensitivities are 0.4%/V and 36 ppm/°C, respectively.

Patient behavior detection based on time-frequency fusion of FMCW radar

This paper proposes a novel time-frequency feature fusion method to recognise patients’ behaviours based on the Frequency Modulated Continuous Wave (FMCW) radar system, which can locate patients as well as recognise their current actions and thus is expected to solve the shortage of medical staff caused by the novel coronavirus pneumonia (COVID-19). To recognise the patient’s behaviour, the FMCW radar is utilised to acquire point clouds reflected by the human body, and the micro-Doppler spectrogram is generated by human motion. Then features are extracted and fused from the time-domain information of point clouds and the frequency-domain information of the micro-Doppler spectrogram respectively. According to the fused features, the patient’s behaviour is recognised by a Bayesian optimisation random forest algorithm, where the role of Bayesian optimisation is to select the best hyper-parameters for the random forest, i.e. the number of random forest decision trees, the depth of leaves, and the number of features. The experimental results show that an average accuracy of 99.3% can be achieved by using the time-frequency fusion with the Bayesian optimisation random forest model to recognise six actions.

Attention-based Frequency-aware Multi-scale Network for Sequential Recommendation

The key problem of sequential recommendation is how to capture user sequential patterns and enrich user sequential representations from historical interactions, mainly due to the uncertainty of user behavior and the limited information. The RNN-based methods capture the long- and short-term level patterns. The CNN-based methods treat the representation of the user’s historical interaction as an “image”, and discover the point-level patterns, union-level patterns and union-level with skip once. The attention-based methods mine the focus-level patterns. However, all the previous methods have only studied how to capture users’ sequential patterns in the time domain. In many cases, if we only consider the time domain information, these methods will have trouble in mining the user’s sequential patterns. To solve this problem, we consider the frequency domain to capture frequency-level patterns for the first time. Because a non-periodic historical behavior sequence in the time domain may be brutal to reflect the user’s intention but much more accessible in the frequency domain. In light of this, we propose a novel Attention-based Frequency-aware Multi-scale Network (AFMN) for Sequential Recommendation. We introduce Fourier transform to decompose the simple embedding vector, the representation of the user’s historical interaction, into a multi-frequency embedding vector to enrich the user’s behavior sequence representation. The frequency-aware attention layers adaptively focus on the important frequency components and output a refined multi-frequency embedding vector. Given the multi-frequency embedding vector, we develop a non-local attention module to aggregate attribute-level and item-level features of the previous L items. Empirical results on four public benchmark datasets show that our method can achieve a significant improvement over the state-of-the-art baselines.

A comparison between classical and new proposed feature selection methods for attention level recognition in disordered children

Lack of attention is a chronic behavior in ADHD (Attention Deficit Hyperactivity Disorder) and ASD (Autism Spectrum Disorder). Our goal is to develop a reliable method for the detection of inattention with high accuracy and low time consumption to be used in real time neurofeedback. The new applied methods for inattention in children are EMD (Empirical Mode Decomposition) with difference time series (Dt) and MRA (Multi Resolution Analysis). EMD is a method of breaking down a signal into ‘modes’ (IMFs) representing its different frequency components. Furthermore, MRA strikes balance between temporal and frequency resolution through localizing the EEG signal in frequency domain of interest (beta range) by wavelet decomposition or EMD and then retains time domain information using FD. As the results demonstrate, in intermediate and severe level cases of inattention, EMD_Dt technique is the most accurate. In mild level cases of inattention MRA (wavelet + FD) technique performance is better than EMD_Dt. However, the time consumption of the MRA (wavelet + FD) technique is fifteen times larger than EMD_Dt technique. EMD_Dt is the best technique as it requires less processing time which is the most important factor in neurofeedback, furthermore, clinician concerned more with severe and intermediate level of inattention to be treated.

Stress inversion in waveguides with arbitrary cross sections with acoustoelastic guided waves

Acoustoelasticity or the change in elastic wave speeds with stress is promising for prestress measurements in waveguides. The theory of guided wave propagation in initially isotropic materials with arbitrary cross sections and under homogeneous biaxial stresses is developed using Semi-Analytical Finite Element (SAFE) modeling in this article. Based on the anisotropic effect induced by the applied biaxial load, an inversion method for biaxial force was developed. The acoustoelastic response for a particular mode and frequency is described by only two constants, which can be determined from known uniaxial loading experiments. The magnitude and direction of the biaxial force can be obtained by further coefficient fitting. Stress inversion can be obtained without considering the shape of the cross section and applies to multiple guided wave modes. The inversion has been verified by the results of SAFE and 3D Sweeping Frequency Finite Element Modeling (SFFEM) method, and the Mean Absolute Errors of stresses obtained by different methods are all within 1%. The 3D SFFEM was combined with the Matrix Pencil Method using the time domain information to extract the dispersion curve. Unlike previous finite element modeling, here the inheritance of the solution between the two solvers was set instead of approximating static load conditions by shortening the guided wave travel time. It guarantees the steady state of the force in the time-variant study, ensuring the high precision required for the study of the acoustoelastic effect.

Impulsive feature extraction with improved singular spectrum decomposition and sparsity-closing morphological analysis

The singular spectrum decomposition (SSD) is an effective signal denoising tool and has been attracted much attention in fault diagnosis. However, the filtering effect and calculation efficiency of SSD are seriously affected by the embedding dimension of trajectory matrix. To overcome these disadvantages, the improved SSD (ISSD) is proposed in this paper. A length factor is designed to optimize the construction of trajectory matrix, which considers the fault information in both time-domain and frequency-domain. A series of analysis, including impulse response analysis and multi-components signal decomposition, demonstrate the ISSD lifts the performance of SSD. After that, a new indicator, named singular Gini index, is applied to select the optimal singular spectrum components (SSCs) decomposed by the ISSD. To further supplement the impulses extraction effect of the ISSD, the sparsity operation is improved by combing the morphological analysis to process the vibration and sound signals. The sparsity factor is updated in each iteration and the structure element in morphological analysis is determined adaptively. Benefiting from the virtues of ISSD and sparsity analysis, the fault impulses in the processed signal are more prominent. Finally, according to the information of bearing characteristic frequencies in the spectrum, the fault type of bearing is determined. The reliability and feasibility of the proposed method is identified by analyzing the different simulation and experimental cases.

Research on Sports Video Content Analysis Method Based on Clustering Extraction Algorithm

The algorithm first uses the time domain information of the sports video image to segment the foreground image containing multiple sports targets through sample variance for background modeling. Then the spatial connectivity rate of pixels is defined, and the initial clusters are adaptively split and merged. The self-organizing iterative clustering algorithm can complete the segmentation of multiple moving targets without setting the number of clustering divisions in advance. Experimental results prove that the algorithm has a good segmentation effect on multiple moving targets, and the segmentation results are consistent with the judgment of human vision. The use of spatial connectivity information makes the algorithm iteratively converge fast and has good real-time performance.

Identifying microseismic events using a dual-channel CNN with wavelet packets decomposition coefficients

Microseismic monitoring is a widely used technique in coal mine safe production. Microseismic events classification is one of the most important steps in microseismic monitoring. CNN has been proven to be the potential tool to identify microseismic events automatically. When the noise is serious and the signal is weak, the recognition ability of CNN is limited. Some scholars use denoising methods, such as wavelet transform, to enhance the signal-to-noise ratio (SNR) of the input data. However, the effective signal will be broken if the incorrect threshold parameters are set in this wavelet transform-based denoising method. In this paper, we intend to make the data volume of the time-frequency graph obtained by WPT as the input of CNN. But it may lead to a dimensional disaster if taking the time-frequency domain signals as input directly. To utilize the effective information extraction and reduce the data dimension, we propose a dual-channel CNN model by combining time domain information and wavelet packet decomposition coefficients (T-WPD CNN). The wavelet packet decomposition coefficients highlight the characteristics of the signal and suppress the characteristics of noise. In addition, the coefficients have the same dimension as the original signal. These advantages are useful for enhancing the performance of CNN. Two field datasets are used to test the network. One from Australia and another from Jiayang coal mine, Sichuan, China. The feature map of the convolutional layer with different inputs are shown to illustrate the influence effect of raw time domain signal and WPD coefficient on the classification performance. The final results show that the T-WPD CNN is superior to the traditional CNN method in accuracy and robustness.

Terahertz spatio-temporal deep learning computed tomography.

Terahertz computed tomography (THz CT) has drawn significant attention because of its unique capability to bring multi-dimensional object information from invisible to visible. However, current physics-model-based THz CT modalities present low data use efficiency on time-resolved THz signals and low model fusion extensibility, limiting their application fields' practical use. In this paper, we propose a supervised THz deep learning computed tomography (THz DL-CT) framework based on time-domain information. THz DL-CT restores superior THz tomographic images of 3D objects by extracting features from spatio-temporal THz signals without any prior material information. Compared with conventional and machine learning based methods, THz DL-CT delivers at least 50.2%, and 52.6% superior in root mean square error (RMSE) and structural similarity index (SSIM), respectively. Additionally, we have experimentally demonstrated that the pretrained THz DL-CT model can generalize to reconstruct multi-material systems with no prerequisite information. THz CT through the DL data fusion approach provides a new pathway for non-invasive functional imaging in object investigation.