Time Series Processes Research Articles

Self-Powered Dye-Sensitized Solar-Cell-Based Synaptic Devices for Multi-Scale Time-Series Data Processing in Physical Reservoir Computing.

Physical reservoir computing (PRC) using synaptic devices has attracted attention as a promising edge artificial intelligence device. To handle time-series data on various time scales, it is necessary to fabricate devices with the desired time scale. In this study, we fabricated a dye-sensitized solar-cell-based synaptic device with controllable time constants by changing the light intensity. This device showed synaptic features, such as paired-pulse facilitation and paired-pulse depression, in response to light intensity. Moreover, we found that the high computational performance of the time-series data processing task was achieved by changing the light intensity, even when the input pulse width was varied. In addition, the fabricated device can be used for motion recognition tasks. This study paves the way for realizing multiple time-scale PRC.

Remaining useful life prediction of turbofan engines based on dual attention mechanism guided parallel CNN-LSTM

Abstract Remaining useful life (RUL) prediction is an valuable research task for predictive health management of aero engines, which is crucial to enhance the safety, dependability and economy of the motor. Accurate prediction of remaining engine service life is an important means for the effectively monitoring of engine operating conditions. The forecasting accuracy of turbofan RUL is inadequate with the traditional single-parameter, single-stage mode. To boost the precision of aero-engine RUL anticipation, a novel mode is recommended, based on a parallel convolutional neural network (CNN) with a long and short-term memory neural network (LSTM) and a dual attention mechanism, named PCLD. The degradation information directly from time series sensor data. The advantages of convolutional neural network (CNN) networks and long short-term memory (LSTM) networks in feature mining and time series processing, respectively, are employed to process time series data, which is conducive to preventing the loss of important element in the data. At last, the experimental results on the aero-engine performance recession dataset C-MAPSS demonstrate that the method outperforms the currently popular models with better robustness and higher prediction accuracy.

Advanced power quality monitoring with dilated convolutional neural networks for smart nanogrids

This study presents the development of an advanced power quality monitoring (PQM) system using dilated convolutional deep neural networks (DCDNN) for smart nanogrids. Traditional convolutional neural networks (CNNs) face challenges in efficiently handling high-dimensional data and identifying nuanced features due to their limited receptive field. To overcome these limitations, our DCDNN model employs a novel use of dilated convolutions to expand the receptive field without increasing network complexity, enabling the capture of long-range dependencies essential for accurate classification of power quality disturbances (PQDs). In addition, the integration of multi-channel time-series data processing, including voltage, current, and frequency measurements, is unique to our approach. The model processes multi-channel time-series data, including voltage, current, and frequency measurements, achieving an average accuracy of 98.9% across 29 PQD classes and 100% accuracy in 6 main classes, with a reduced detection time of 47 µs per segment. This approach not only improves real-time disturbance detection but also enhances the reliability and efficiency of electrical systems. The results demonstrate significant improvements over existing methods, underscoring the potential of DCDNNs in advancing PQM practices. Future work will focus on validating the model in real-world conditions and optimizing its computational efficiency.

Dynamical predictive coding with reservoir computing performs noise-robust multi-sensory speech recognition.

Multi-sensory integration is a perceptual process through which the brain synthesizes a unified perception by integrating inputs from multiple sensory modalities. A key issue is understanding how the brain performs multi-sensory integrations using a common neural basis in the cortex. A cortical model based on reservoir computing has been proposed to elucidate the role of recurrent connectivity among cortical neurons in this process. Reservoir computing is well-suited for time series processing, such as speech recognition. This inquiry focuses on extending a reservoir computing-based cortical model to encompass multi-sensory integration within the cortex. This research introduces a dynamical model of multi-sensory speech recognition, leveraging predictive coding combined with reservoir computing. Predictive coding offers a framework for the hierarchical structure of the cortex. The model integrates reliability weighting, derived from the computational theory of multi-sensory integration, to adapt to multi-sensory time series processing. The model addresses a multi-sensory speech recognition task, necessitating the management of complex time series. We observed that the reservoir effectively recognizes speech by extracting time-contextual information and weighting sensory inputs according to sensory noise. These findings indicate that the dynamic properties of recurrent networks are applicable to multi-sensory time series processing, positioning reservoir computing as a suitable model for multi-sensory integration.

Sparse learned kernels for interpretable and efficient medical time series processing

Sparse learned kernels for interpretable and efficient medical time series processing

Internal deformation of the North Andean Sliver in Ecuador-southern Colombia observed by InSAR

Summary In the Northern Andes, partitioning of oblique subduction of the Nazca plate beneath the South American continent induces a northeastward motion of the North Andean Sliver. The strain resulting from this motion is absorbed by crustal faults, which have produced magnitude 7 + earthquakes historically in the Andean Cordillera of Ecuador and southern Colombia. In order to quantify the strain in that area, we derive a high-resolution surface velocity map using InSAR time-series processing. We analyzed 6 to 8 years of Sentinel-1 data and combined different satellite line-of-sight directions to produce a reliable velocity map in the East direction. We use interpolated GNSS data to express the velocity map with respect to Stable South America and remove the long-wavelength pattern due to the post-seismic deformation following the 2016 Mw 7.8 Pedernales earthquake. The InSAR velocity map finds high E-W shortening strain rates along N-S trending structures within the Western Cordillera and the Interandean valley, with little deformation taking place east of them. This result strengthens the previous proposition of a ∼350 km long Quito-Latacunga tectonic block, forming a restraining bend in the overall right-lateral strike-slip fault system accommodating the northeastward escape motion of the North Andean Sliver. However, the high spatial resolution provided by InSAR indicates that previously proposed boundaries for this block need to be revised. In particular, InSAR results highlight high strain rate (>300 nstrain/yr) along undescribed active structures, south and west of the proposed limits for the Quito-Latacunga block, respectively in Peltetec and Ibarra regions. Interestingly, the two areas with the largest strain rates spatially correlate with the proposed areas of large historical earthquakes. Modeling of the InSAR and GNSS velocities in these areas suggests shallow coupling and high slip rates on structures which, previously, were not identified as active. We also demonstrate a slow-down of the shallow aseismic slip on the Quito fault after the Pedernales earthquake, suggesting that stress changes following large megathrust events might trigger transient slip behaviors on crustal faults. The high-resolution strain map provided by this work provides a new basis for future tectonic models in the Ecuadorian and southern Colombian Andes, and will contribute to the seismic hazard assessment in this highly populated area of the Andes.

Challenges and Opportunities of Sentinel-1 InSAR for Transport Infrastructure Monitoring

Monitoring displacement at transport infrastructure using Sentinel‑1 Interferometric Synthetic Aperture Radar (InSAR) faces challenges due to the sensor’s medium spatial resolution, which limits the pixel coverage over the infrastructure. Therefore, carefully selecting coherent pixels is crucial to achieve a high density of reliable measurement points and to minimize noisy observations. This study evaluates the effectiveness of various pixel selection methods for displacement monitoring within transport infrastructures. We employ a two-step InSAR time series processing approach. First, high-quality first-order pixels are selected using temporal phase coherence (TPC) to estimate and correct atmospheric contributions. Then, a combination of different pixel selection methods is applied to identify coherent second-order pixels for displacement analysis. These methods include amplitude dispersion index (ADI), TPC, phase linking coherence (PLC), and top eigenvalue percentage (TEP), targeting both point-like scatterer (PS) and distributed scatterer (DS) pixels. Experiments are conducted in two case studies: one in Germany, characterized by dense vegetation, and one in Spain, with sparse vegetation. In Germany, the density of measurement points was approximately 30 points/km², with the longest segment of the infrastructure without any coherent pixels being 2.8 km. In Spain, the density of measurement points exceeded 500 points/km², with the longest section without coherent pixels being 700 meters. The results indicate that despite the challenges posed by medium-resolution data, the sensor is capable of providing adequate measurement points when suitable pixel selection methods are employed. However, careful consideration is necessary to exclude noisy pixels from the analysis. The findings highlight the importance of choosing a proper method tailored to infrastructure characteristics. Specifically, combining TPC and PLC methods offers a complementary set of pixels suitable for displacement measurements, whereas ADI and TEP are less effective in this context. This study demonstrates the potential of Sentinel‑1 InSAR for capturing both regional-scale and localized displacements at transport infrastructure.

Application of wavelet and seasonal-based emotional ANN (EANN) models to predict solar irradiance

This study models solar irradiance at six stations in Iran and the USA on an hourly scale. We explored two seasonal emotional artificial neural networks (EANN): sequence-EANN (SEANN) and wavelet EANN (WEANN). Analyzing ten years of climatic and solar data, we evaluated uncertainty using prediction intervals (PIs) computed via the bootstrap method based on artificial neural networks (ANNs). Unlike standalone EANNs, the proposed seasonal models effectively captured seasonal information and leveraged time series processing advantages. Utilizing Wavelet and Fourier transforms, these models captured long-short autoregressive dependencies in solar irradiance, addressing extended seasonal dependencies. Results showed that the seasonal EANN models outperformed the classic EANN model by approximately 15 % and the classic feed-forward neural network (FFNN) by about 25 % in both training and testing. The WEANN model demonstrated the highest performance in PIs, with an average normalized mean PI width (NMPIW) of 0.8 and an average PI coverage probability (PICP) of 0.96.

Physically-guided temporal diffusion transformer for long-term time series forecasting

Transformer has shown excellent performance in long-term time series forecasting because of its capability to capture long-term dependencies. However, existing Transformer-based approaches often overlook the unique characteristics inherent to time series, particularly multi-scale periodicity, which leads to a gap in inductive biases. To address this oversight, the temporal diffusion Transformer (TDT) was developed in this study to reveal the intrinsic evolution processes of time series. First, to uncover the connections among the periods of multi-periodic time series, the series are transformed into various types of patches using a multi-scale Patch method. Inspired by the principles of heat conduction, TDT conceptualizes the evolution of a time series as a diffusion process. TDT aims to achieve global consistency by minimizing energy constraints, which is accomplished through the iterative updating of patches. Finally, the results of these iterations across multiple periods are aggregated to form the TDT output. Compared to previous advanced models, TDT achieved state-of-the-art predictive performance in our experiments. In most datasets, TDT outperformed the baseline model by approximately 2% in terms of mean square error (MSE) and mean absolute error (MAE). Its effectiveness was further validated through ablation, efficiency, and hyperparameter analyses. TDT offers intuitive explanations by elucidating the diffusion process of time series patches throughout the iterative procedure.

Micromagnetic analysis of magnetic vortex dynamics for reservoir computing

Reservoir computing (RC) has generated significant interest for its ability to reduce computational costs compared to traditional neural networks. The performance of the RC element is quantified by its memory capacity (MC) and prediction capability. In this study, we utilize micromagnetic simulations to investigate a magnetic vortex based on a permalloy ferromagnetic layer and its dynamics in RC. The nonlinear dynamics of the vortex core (VC), driven by continuous oscillating magnetic fields and binary digit data as spin-polarized current pulses, are analyzed. The highest MC observed is 4.1, corresponding to the nonlinear VC dynamics. Additionally, the prediction capability is evaluated using the Nonlinear Auto-Regressive Moving Average 2 task, demonstrating a normalized mean squared error of 0.0241 highlighting the time-series data prediction performance of the vortex as a reservoir.

Revealing Hidden Features of Chaotic Systems Using High-Performance Bifurcation Analysis Tools Based on CUDA Technology

Bifurcation analysis is an essential tool in nonlinear dynamics. Bifurcation diagrams help to discover subtle features of investigated dynamics such as chaotic and periodic regimes, hidden attractors and fixed points. However, plotting high-resolution bifurcation diagrams can be a computationally challenging task, especially in multiparametric evaluation. It should be noted that the bifurcation analysis is a task with natural parallelism and thus can be efficiently solved using hybrid central-graphics processing architectures. In this paper, we propose an advanced algorithm and special software for plotting bifurcation diagrams using calculations accelerated by graphics processing unit in combination with a highly efficient semi-implicit ordinary differential equation solver. Time series processing is based on the extraction of amplitude and phase features for the density-based spatial clustering of applications with noise to determine oscillation periodicity. We showcase the features of the application of proposed solutions on a set of test chaotic systems. The performance of the analysis algorithms is investigated in comparison with conventional solutions based on central processing unit and several approaches known from the literature. We explicitly show that the proposed algorithm outperforms known solutions in both calculation speed and precision.

A new method of rock type identification based on transformer by utilizing acoustic emission.

The characterization and analysis of rock types based on acoustic emission (AE) signals have long been focal points in earth science research. However, traditional analysis methods struggle to handle the influx of big data. While signal processing methods combined with deep learning have found widespread use in various process analyses and state identification, effective feature extraction using progressive fusion technology still faces challenges in the field of intelligent rock type identification. To address this issue, our study proposes a novel framework for rock type identification based on AE and introduces a new signal identification model called 3CTNet. This model integrates convolutional neural networks (CNNs) and Transformer encoder, intelligently identifying AE of different rock fractures by establishing dependencies between adjacent positions within the data and gradually extracting advanced features. Furthermore, we experimentally compare five oversampling methods, ultimately selecting the adaptive synthetic sampling method (ADASYN) to balance the dataset and enhance the model's robustness and generalization ability. Comparison of the internal structure of our model with a series of time series processing models demonstrates the effectiveness of the proposed model structure. Experimental results showcase the high identification accuracy of the intelligent rock type identification model based on 3CTNet, with an overall identification accuracy reaching 98.780%. Our proposed method lays a solid foundation for the efficient and accurate identification of formation rock types in geological exploration and oil and gas development endeavors.

EHOPN: A novel enhanced high-order pooling-based network for industrial fault detection

Recently, deep learning algorithms have been successfully applied to industrial fault detection because they are better at automatically extracting complex features and processing high-dimensional data than traditional methods. However, most existing deep learning-based fault detection methods only concentrate on extracting features from industrial process data without considering the crucial long-term temporal features and higher-order statistical information. To address this challenge, we proposed a novel enhanced higher-order pooling-based network (EHOPN) for industrial fault detection. First, the data pre-processing of the network is presented to capture the dynamic features of time-series process data and unify the high-dimensional data scale. Second, the EHOPN utilizes channel and temporal second-order pooling techniques to gather temporal and channel statistics information, facilitating the backbone network’s ability to capture complex inter-dependencies and long-term dynamics. Additionally, the high-order feature aggregation module is presented to aggregate global and local features, enhancing the network’s generalization ability. The proposed industrial fault detection approach is evaluated on the Tennessee Eastman benchmark and a real-world heavy-plate production process. Experimental results show that the proposed method is significantly better than comparison models in four evaluation metrics: accuracy, precision, recall, and F1-score, further proving the effectiveness of EHOPN.

MIMO Signal Detection Based on IM-LSTMNet Model

Signal detection is crucial in multi-input multi-output orthogonal frequency division multiplexing (MIMO-OFDM) systems, yet classical detection methods often struggle with nonlinear issues in wireless channels. To handle this challenge, we propose a novel signal detection method for MIMO-OFDM system based on the fractional Fourier transform (FrFT), leveraging the robust time series processing capabilities of long short-term memory (LSTM) networks. Our innovative approach, termed IM-LSTMNet, integrates LSTM with convolutional neural networks (CNNs) and incorporates a Squeeze and Excitation Network to emphasize critical information, enhancing neural network performance. The proposed IM-LSTMNet is applied to the FrFT-based MIMO-OFDM system to improve signal detection performance. We compare the detection results of IM-LSTMNet with zero forcing (ZF), minimum mean square error (MMSE), simple LSTM neural network, and CNN–LSTM network by evaluating the bit error rate. Experimental results demonstrate that IM-LSTMNet outperforms ZF, MMSE, LSTM, and other methods, significantly enhancing system signal detection performance. This work offers a promising advancement in MIMO-OFDM signal detection, presenting a deep learning-based solution that effectively improves the system signal detection performance.

A Probabilistic Approach to Characterizing Drought Using Satellite Gravimetry

AbstractIn the recent past, the Gravity Recovery and Climate Experiment (GRACE) satellite mission and its successor GRACE Follow‐On (GRACE‐FO), have become invaluable tools for characterizing drought through measurements of Total Water Storage Anomaly (TWSA). However, the existing approaches have often overlooked the uncertainties in TWSA that stem from GRACE orbit configuration, background models, and intrinsic data errors. Here we introduce a fresh view on this problem which incorporates the uncertainties in the data: the Probabilistic Storage‐based Drought Index (PSDI). Our method leverages Monte Carlo simulations to yield realistic realizations for the stochastic process of the TWSA time series. These realizations depict a range of plausible drought scenarios that later on are used to characterize drought. This approach provides probability for each drought category instead of selecting a single final category at each epoch. We have compared PSDI with the deterministic approach (Storage‐based Drought Index, SDI) over major global basins. Our results show that the deterministic approach often leans toward an overestimation of storage‐based drought severity. Furthermore, we scrutinize the performance of PSDI across diverse hydrologic events, spanning continents from the United States to Europe, the Middle East, Southern Africa, South America, and Australia. In each case, PSDI emerges as a reliable indicator for characterizing drought conditions, providing a more comprehensive perspective than conventional deterministic indices. In contrast to the common deterministic view, our probabilistic approach provides a more realistic characterization of the TWS drought, making it more suited for adaptive strategies and realistic risk management.

Response spectra of earthquakes using ARMA model

This study is concerned with calculating hysteretic energy using ARMA coefficient models. We know that: For any random time series we can write a model of ARMA(p,q) detailed by Box and Jenkins [2]. Any movement generates energy, regardless of its form or type, negative or positive, so how do we deal with it or reduce its danger? This is possible through ARMA models that have proven their efficiency through several procedures such as earthquakes and accelerometer time series that are generated from random models in the time domain or frequency domain. Extensive reviews of time series process models for acceleration in the frequency and times domain have been presented by [1], [12], and [4]. A number of papers have reported on ARMA models. ARMA models are discussed in detail by Box and Jenkins [4]. Here we generally found the hysteretic energy NH in terms of the ARMA(2,1) coefficients, but the ARMA(2,1) model can be considered the most appropriate in this case for the regions of northern Algeria, from which we took the Boumerdes earthquake. When taking the AIC RITERIA criterion, we found that it is in fact ARMA. (2,1) which is what we studied The use of mathematical relationships, including some results of nodal integration, such as relationship [7] , which conforms to the definition of a stationary time series, which is that the mean and standard deviation are constant.

BRIDGING DATA GAPS: A FEDERATED LEARNING APPROACH TO HEAT EMISSION PREDICTION IN LASER POWDER BED FUSION

Abstract Deep learning has impacted defect prediction in Additive Manufacturing (AM), which is important to ensure process stability and part quality. However, its success depends on extensive training, requiring large, homogenous datasets–remaining a challenge for the AM industry, particularly for small- and medium-sized enterprises (SMEs). The unique and varied characteristics of AM parts, along with the limited resources of SMEs, hampers data collection, posing difficulties in the independent training of deep learning models. Addressing these concerns requires enabling knowledge sharing from the similarities in the physics of the AM process and defect formation mechanisms while carefully handling privacy concerns. Federated learning (FL) offers a solution to allow collaborative model training across multiple entities without sharing local data. This paper introduces an FL framework to predict section-wise heat emission during Laser Powder Bed Fusion (LPBF), a vital process signature. It incorporates a customized Long Short-Term Memory (LSTM) model for each client, capturing the dynamic AM process's time series properties without sharing sensitive information. Three advanced FL algorithms are integrated–FedAvg, FedProx, and FedAvgM–to aggregate model weights rather than raw datasets. Experiments demonstrate that the FL framework ensures convergence and maintains prediction performance comparable to individually trained models. This work demonstrates the potential of FL-enabled AM modeling and prediction where SMEs can improve their product quality without compromising data privacy.

Halide perovskite photovoltaics for in-sensor reservoir computing

As intelligent electronics get laden with multimodal sensors, the data transfer and computation increase their energy expenditure. Consequently, researchers aim to develop efficient computing paradigms or integrate energy harvesting from ambient sources. Halide perovskites possess unique photophysics and coupled ionic-electronic dynamics that actualise memory devices for brain-inspired computing. Synergising the computing capability with their conventional light harvesting efficacy could aptly address the aforementioned problem. In a novel approach, the transient open circuit voltage (VOC) of a methylammonium lead bromide-based solar cell was exploited to serve as a self-powered volatile short-term memory for optoelectronic in-sensor reservoir computing. The origin of the memory is attributed to the influence of mobile ions on the carrier generation and recombination in the device. The system's versatility and task specificity were shown by engineering the volatility of the memory. The benchmarking task of MNIST handwritten digit recognition was performed with the highly reproducible and robust transformation of optical inputs into unique reservoir states. To demonstrate the high nonlinearity, second-order time-series prediction (NARMA2) was performed. Finally, an exemplary cardiac health-monitoring application was showcased by monolithic reading and processing of a physiological time series known as photoplethysmography (PPG) to identify atrial fibrillation with increased computational efficiency.

Symmetric spatiotemporal learning network with sparse meter graph for short-term energy-consumption prediction in manufacturing systems

Short-term energy-consumption prediction is the basis of anomaly detection, real-time scheduling, and energy-saving control in manufacturing systems. Most existing methods focus on single-node energy-consumption prediction and suffer from difficult parameter collection and modelling. Although several methods have been presented for multinode energy-consumption prediction, their prediction performance needs to be improved owing to a lack of appropriate knowledge guidance and learning networks for complex spatiotemporal relationships. This study presents a symmetric spatiotemporal learning network (SSTLN) with a sparse meter graph (SMG) (SSTLN-SMG) that aims to predict multiple nodes based on energy-consumption time series and general process knowledge. The SMG expresses process knowledge by abstracting production nodes, material flows, and energy usage, and provides initial guidance for the SSTLN to extract spatial features. SSTLN, a symmetrical stack of graph convolutional networks (GCN) and gated linear units (GLU), is devised to achieve a trade-off not only between spatial and temporal feature extraction but also between detail capture and noise suppression. Extensive experiments were performed using datasets from an aluminium profile plant. The experimental results demonstrate that the proposed method allows multinode energy-consumption prediction with less prediction error than state-of-the-art methods, methods with deformed meter graphs, and methods with deformed learning networks.

Multivariate data binning and examples generation to build a Diabetic Retinopathy classifier based on temporal clinical and analytical risk factors

In this paper, we explore the possibility of exploiting retrospective clinical data from Electronic Health Records (EHR) for classification tasks in chronic patients. The different intervals, short length and high class imbalance make it unfeasible to use traditional time series techniques. The first contribution of the paper is a preprocessing method to construct a multivariate time series dataset using EHR data, which infers missing data and regularizes the data frequency. The second contribution addresses class imbalance by using domain knowledge and existing short EHR series. We synthetically extrapolate patients’ data by using similar long time series and a fuzzy-based approach. The paper addresses the problem of detection of Diabetic Retinopathy (DR). Expert domain knowledge from ophthalmologists has been used in the proposed techniques to guide the processing of time series. The novelty in that case study consists in not using eye-fundus image analysis. Instead, the proposed methods are based solely on EHR data. Several multivariate multiclass time series classifiers are used to detect the four levels of DR severity from the pre-processed data sequences. Experiments prove the quality of the sequence preprocessing techniques proposed for EHR data. Results indicate that the TapNet classifier is the best one for DR grading. Despite being tested for DR detection, the proposed data preparation methods are applicable to other diseases with similar characteristics.