Real-world Time Series Research Articles

An adaptive network with consecutive and intertwined slices for real-world time-series forecasting

Real-world systems are chiefly comprised of multiple sensors and devices, whose instantaneous data and upcoming tendencies are demanded by the decision makers to formulate future schemes. Accuracy and timeliness are both of paramount significance. Thereby, although the deep forecasting models are promising, the space and resources are often limited in reality, rendering their adaptability requisite. Unfortunately, the majority of current deep forecasting models are far from adaptive and acquire heavy hyper-parameter tuning processes to obtain the satisfactory performances. Moreover, their strategies to reduce the model complexities sometimes bring extra hyper-parameters. To tackle these issues, we propose AdaNS: an Adaptive Network with consecutive and intertwined Slices. The data-driven features of time-series in frequency domain are used in AdaNS to adaptively determine the model structure, as well as hyper-parameters, and categorize variables. In accordance with these frequency-based features, input sequences are first sliced in a consecutive way to extract the universal features and then in an intertwined way to extract the local features, for hierarchical and comprehensive forecasting. Extensive experiments show that AdaNS achieves state-of-the-art performances on fourteen benchmarks, virtually without any hyper-parameter tuning. The source code is available at https://github.com/OrigamiSL/AdaNS.

Multi-Step-Ahead Time Series Forecasting using Deep Learning and Fuzzy Time Series-based Error Correction Method

Recently time series forecasting has become one of the prime application areas of climatology, economics and industries. Many research works are conducted to forecast the time series more accurately. But few of them are concentrated on predicting the time series over an extended future horizon, and there is also a scope to improve their forecasting accuracy. This work proposes a multi-step-ahead foresting method to produce a stable and accurate forecasting result for the extended future horizon. Firstly, a deep learning-based forecasting model is proposed to predict the time series. Secondly, a fuzzy time series-based error correction model is implemented to enhance the prediction performance of the deep learning model. Here to optimize all the fuzzy time series (FTS) parameters in an integrated way, an integrated butterfly optimization (FTS-IBO) algorithm is proposed. In this study, two different types of real-world multivariate time series datasets are used to analyze the forecasting performance of the proposed model. The performance of the proposed FTS-IBO algorithm is compared with the traditional butterfly optimization (FTS-BO) algorithm. The experimental results show that the FTS-IBO technique is superior to the FTS-BO technique. The forecasting performance of the proposed model has also compared the other state-of-the-art models, and the simulation results exhibit that the proposed model produces a more accurate prediction performance for multi-step-ahead time series forecasting problems compared to other models.

Analog reservoir computing via ferroelectric mixed phase boundary transistors

Analog reservoir computing (ARC) systems have attracted attention owing to their efficiency in processing temporal information. However, the distinct functionalities of the system components pose challenges for hardware implementation. Herein, we report a fully integrated ARC system that leverages material versatility of the ferroelectric-to-mixed phase boundary (MPB) hafnium zirconium oxides integrated onto indium–gallium–zinc oxide thin-film transistors (TFTs). MPB-based TFTs (MPBTFTs) with nonlinear short-term memory characteristics are utilized for physical reservoirs and artificial neuron, while nonvolatile ferroelectric TFTs mimic synaptic behavior for readout networks. Furthermore, double-gate configuration of MPBTFTs enhances reservoir state differentiation and state expansion for physical reservoir and processes both excitatory and inhibitory pulses for neuronal functionality with minimal hardware burden. The seamless integration of ARC components on a single wafer executes complex real-world time-series predictions with a low normalized root mean squared error of 0.28. The material-device co-optimization proposed in this study paves the way for the development of area- and energy-efficient ARC systems.

Machine Learning Models Informed by Connected Mixture Components for Short- and Medium-Term Time Series Forecasting

This paper presents a new approach in the field of probability-informed machine learning (ML). It implies improving the results of ML algorithms and neural networks (NNs) by using probability models as a source of additional features in situations where it is impossible to increase the training datasets for various reasons. We introduce connected mixture components as a source of additional information that can be extracted from a mathematical model. These components are formed using probability mixture models and a special algorithm for merging parameters in the sliding window mode. This approach has been proven effective when applied to real-world time series data for short- and medium-term forecasting. In all cases, the models informed by the connected mixture components showed better results than those that did not use them, although different informed models may be effective for various datasets. The fundamental novelty of the research lies both in a new mathematical approach to informing ML models and in the demonstrated increase in forecasting accuracy in various applications. For geophysical spatiotemporal data, the decrease in Root Mean Square Error (RMSE) was up to 27.7%, and the reduction in Mean Absolute Percentage Error (MAPE) was up to 45.7% compared with ML models without probability informing. The best metrics values were obtained by an informed ensemble architecture that fuses the results of a Long Short-Term Memory (LSTM) network and a transformer. The Mean Squared Error (MSE) for the electricity transformer oil temperature from the ETDataset had improved by up to 10.0% compared with vanilla methods. The best MSE value was obtained by informed random forest. The introduced probability-informed approach allows us to outperform the results of both transformer NN architectures and classical statistical and machine learning methods.

Comparative Analysis of ARIMA, SARIMA and Prophet Model in Forecasting

Machine learning has become a powerful tool in forecasting, offering greater accuracy than traditional human predictions in today’s data-driven world. The capability of machine learning to predict future trends has significant implications for key sectors such as finance, healthcare, and supply chain management. In this study, ARIMA/SARIMA (AutoRegressive Integrated Moving Average/Seasonal AutoRegressive Integrated Moving Average), alongside Prophet, a scalable forecasting tool developed by Facebook based on a generalized additive model, are considered. These models are applied to predict the demand for antidiabetic drugs. The records were collected by the Australian Health Insurance Commission. This dataset was sourced from Medicare Australia. The study evaluates the performance of these models based on their Mean Absolute Error (MAE), a key metric for assessing forecast accuracy. Root Mean Squared Error (RMSE) and Mean Absolute Percentage Error (MAPE) are also considered. The outcome of the comparative analysis shows that the Prophet model outperformed both ARIMA and SARIMA models, achieving an MAE of 0.74, which is significantly lower than the MAE values of 2.18 and 3.02 obtained by SARIMA and ARIMA, respectively. Prophet's superior performance shows its effectiveness in handling complex, non-linear trends and seasonal patterns often observed in real-world time series data. This research contributes to the growing knowledge of machine learning-based forecasting and shows the importance of advanced models like Prophet in optimizing business operations and driving innovation. The findings from this research offer valuable guidance for data experts, analysts, and researchers in selecting the best forecasting methods for reliable predictions.

Granular computing-based time series anomaly pattern detection with semantic interpretation

Time series analysis may suffer from the “curse of dimensionality” due to its high-dimensionality characteristics. In terms of this issue, information granulation offers an effective vehicle to process time series at a higher level of abstraction. To take this advantage, this study conducts an interpretable time series anomaly pattern detection under the framework of granular computing, where each time series is granulated into a series of semantics according to its patterns and the anomaly ones are identified based on the obtained semantics. First, the time series is partitioned into a predefined number of segments and trend-based information granules are formed for the data points in each time interval. Guided by the principle of justifiable granularity, the granular results maintaining the main features of the original time series realize informative feature extraction and meaningful dimensionality reduction. Then, to realize semantic anomaly pattern detection, the Axiomatic Fuzzy Set (AFS) theory is generalized to construct and compute with semantic time series, and an AFS-based anomaly score is proposed to discover anomaly patterns. In the experiments, the proposed method is conducted on both UCR data sets and real-world time series, where the detected anomaly patterns are equipped with well-defined semantics.

Multi-Way adaptive Time Aware LSTM for irregularly collected sequential ICU data

In personalized predictive medicine, accurately modeling a patient’s illness and care processes is essential, given their inherent long-term temporal dependencies. However, electronic medical records contain episodic and irregularly timed data due to patients visiting hospitals based on treatment needs, resulting in unique patterns for each hospital stay. Consequently, when constructing a personalized predictive model, it is crucial to consider these factors in order to accurately capture the patient’s health journey.To address this challenge, we present a novel deep dynamic memory neural network called Multi-Way adaptive Time Aware LSTM (MWTA-LSTM). The primary objective of MWTA-LSTM is to leverage medical records, memorize illness trajectories and care processes, estimate current illness states, and predict future risks, thereby providing a high level of precision and predictive power. To enhance its capabilities, MWTA-LSTM extends the conventional Long Short-Term Memory (LSTM) model in two key ways. Firstly, it incorporates frequency measurement and the most recent observation(last observation) to enhance personalized predictive modeling of patient illnesses, enabling a more accurate understanding of the patient’s condition. Secondly, it parameterizes the cell state to handle irregular timing effectively, utilizing both frequency measurement and elapsed times. Furthermore, the model capitalizes on both to comprehend the impact of interventions on the course of illness on the cell state, facilitating the memorization of illness courses and improving its ability to capture the temporal dynamics of healthcare data, accommodating variations and irregularities in event and observation timing. Lastly, we introduce a novel adaptive pooling strategy that specifically targets and resolves outlier issues that can potentially occur during the analysis of EHR data. By incorporating these features, MWTA-LSTM significantly improves its ability to capture the temporal dynamics of healthcare data, accommodating variations and irregularities in event and observation timing.We validate the effectiveness of our proposed model through empirical experiments conducted on two real-world clinical datasets and three real-world time series datasets. Our results demonstrate the superiority of MWTA-LSTM over current state-of-the-art models and other robust baselines. This showcases the potential of MWTA-LSTM in advancing personalized predictive medicine by offering a more accurate and comprehensive approach to modeling patient health trajectories.

Meta Koopman decomposition for time series forecasting under temporal distribution shifts

Time series forecasting facilitates various real-world applications and has attracted great research interests. In real-world scenarios, time series forecasting models confront a fundamental issue of temporal distribution shifts (TDS), i.e., the statistical properties of time series are evolving over time. In this paper, we utilize Koopman theory to address temporal distribution shifts. Koopman theory states any time series can be mapped into a Koopman space by proper measurement functions with their dynamics modeled by the infinite-dimensional linear Koopman operator. Therefore, time series under different distributions can be modeled by different Koopman operators. Considering the linearity of Koopman operators, the Koopman operators for representing time series under different distributions can be decomposed as a linear combination of a set of Koopman operators, which we termed as meta Koopman operators. We further theoretically show the infinite-dimensional Koopman operators can be approximated by finite matrix multiplications and the meta Koopman operators are equivalent to a set of matrices. Based on the analysis, we propose an auto-encoder framework for implementing the meta Koopman decomposition of time series, which is theoretically capable of handling TDS. Extensive experiments conducted on four real-world time series datasets demonstrate the superiority of the proposed model in tackling temporal distribution shifts.

Efficient time series adaptive representation learning via Dynamic Routing Sparse Attention

Time series prediction plays a crucial role in various fields but also faces significant challenges. Converting original 1D time series data into 2D data through dimension transformation allows capturing more hidden features but incurs high memory consumption and low time efficiency. We have designed a sparse attention mechanism with dynamic routing perception called Dynamic Routing Sparse Attention (DRSA) to address these issues. Specifically, DRSA can effectively handle variations of complex time series data. Meanwhile, under memory constraints, the Dynamic Routing Filter (DRF) module further refines it by filtering the blocked 2D time series data to identify the most relevant feature vectors in the local context. We conducted predictive experiments on six real-world time series datasets with fine granularity and long sequence dependencies. Compared to eight state-of-the-art (SOTA) models, DRSA demonstrated relative improvements ranging from 4.18% to 81.02%. Furthermore, its time efficiency is 2 to 5 times higher than the baseline. Our code and dataset will be available at https://github.com/wwy8/DRSA_main.

Information Criteria for Signal Extraction Using Singular Spectrum Analysis: White and Red Noise

In singular spectrum analysis, which is applied to signal extraction, it is of critical importance to select the number of components correctly in order to accurately estimate the signal. In the case of a low-rank signal, there is a challenge in estimating the signal rank, which is equivalent to selecting the model order. Information criteria are commonly employed to address these issues. However, singular spectrum analysis is not aimed at the exact low-rank approximation of the signal. This makes it an adaptive, fast, and flexible approach. Conventional information criteria are not directly applicable in this context. The paper examines both subspace-based and information criteria, proposing modifications suited to the Hankel structure of trajectory matrices employed in singular spectrum analysis. These modifications are initially developed for white noise, and a version for red noise is also proposed. In the numerical comparisons, a number of scenarios are considered, including the case of signals that are approximated by low-rank signals. This is the most similar to the case of real-world time series. The criteria are compared with each other and with the optimal rank choice that minimizes the signal estimation error. The results of numerical experiments demonstrate that for low-rank signals and noise levels within a region of stable rank detection, the proposed modifications yield accurate estimates of the optimal rank for both white and red noise cases. The method that considers the Hankel structure of the trajectory matrices appears to be a superior approach in many instances. Reasonable model orders are obtained for real-world time series. It is recommended that a transformation be applied to stabilize the variance before estimating the rank.

Sparse compressed deep echo state network with improved arithmetic optimization algorithm for chaotic time series prediction

A sparse compressed deep echo state network (SCDESN) incorporating sparse input units, compressed sampling and principal component analysis (PCA) units is proposed in this paper, and theoretically proved the sufficient and necessary conditions to ensure the echo state characteristics of the proposed scheme. In addition, a hybrid arithmetic optimization algorithm based on matrix design strategy and boundary selection strategy (SVD-HAOA) was proposed to optimize the hyper-parameters of the SCDESN model, taking into account the characteristics of the SCDESN model. The steps and process for optimizing the hyper parameters of the SCDESN model using SVD-HAOA were presented. Finally, the SCDESN model optimized by SVD-HAOA was experimentally and summarized on a classic benchmark dataset and two real-world chaotic time series. The outcomes of the simulation indicated that the suggested model surpassed alternative models in performance and enhanced computational efficiency while maintaining prediction accuracy. This model may serve as a viable alternative for real-world applications. Noteworthy, the source code of SCDESN-SVD-HAOA is publicly available at https://github.com/qiuqiuyaaa/SCDESN-SVD-HAOA.

Rainforest: A three-stage distribution adaptation framework for unsupervised time series domain adaptation

Solving the unsupervised domain adaptation (UDA) task in time series is of great significance for practical applications, such as human activity recognition and machine fault diagnosis. Compared to UDA for computer vision, UDA in time series is more challenging due to the dynamics of time series data and the complex dependencies among different time steps. Existing UDA methods for time series fail to adequately capture the temporal dependencies, limiting their ability to learn domain-invariant temporal patterns. Furthermore, most UDA methods only focus on distribution adaptation on the backbone network without considering how the classifier adapts to the data distribution of the target domain. In this paper, we propose Rainforest, a three-stage UDA framework for time series. We first pre-train the backbone network through a self-supervised method called bidirectional autoregression, so that the model can comprehensively learn the temporal dependencies in time series. Next, we propose a novel meta-learning-based distribution adaptation method to achieve the joint alignment of the global and local distributions while encouraging the model to adaptively reduce the temporal dynamic differences among different domains. Finally, we design a pseudo-label-guided fine-tuning strategy to help the classifier estimate the data distribution of the target domain more accurately. Extensive experiments on four real-world time series datasets show that our Rainforest outperforms state-of-the-art methods, with an average improvement of 2.19% in accuracy and 2.41% in MF1-score.

A novel stochastic ProFiVaS model based on decomposition of stochastic Vasicek differential equation for modeling and simulating of financial indicators

The main purpose of this study is to develop a new discrete time stochastic ProFiVaS model based on decomposition of stochastic Vasicek differential equation which follows bouncing autoregressive AR1, with fixed variance and trend for modeling and simulating of international financial time series forecasting.This study offers the four discrete time stochastic models. The first model is determined as autoregressive integrated moving average-Arima1,1,1, and the parameter estimates of this model are found using the maximum likelihood method (MLM). Monte Carlo simulation is applied to the obtained parameters. This model is named MoCArima for short. Second, the MoCArima model whose parameters are estimated with particle swarm optimization (PSO) is determined as Proposed Monte Carlo (MC) Arima1,1,1, and is briefly named as ProMoCArima. The third model is determined as Fixed Variance Stochastic Model which has a bouncing autoregressive AR1 structure with constant variance and trend component. This model is briefly named as FiVaS. The parameter estimates are found using the MLM. The fourth model is determined as the Proposed Fixed Variance Stochastic Model. This model is briefly named as ProFiVaS. Structurally the same as the third model. The parameter estimates of both trend and stochastic parts of the FiVaS model are found by the PSO.Proposed models are applied to forecasting international different financial indicators to show its superiority and applicability. The performance and usage efficiency of the proposed four stochastic models are examined on a total of six different international financial data types, five of which are stock index data BIST100 (Türkiye), S&P500 (USA), DAX (Germany), FTSE100 (UK), NIKKEI225 (Japan), and one of which is the exchange rate EURO/USD. The ProFiVaS among all models provides the best performance on the all indices. The ProFiVaS can easily be extended to any real-world time series prediction requirements.The proposed models can be extended to all stationary and non-stationary linear or nonlinear autoregression models opening the door to modeling data types obtained from different sector fields. A part of the future plan of this work is to design a decision support system software that assigns a value of 1 when an investor needs to invest in a financial indicator, and 0 otherwise.

Synthesis and quality assessment of combined time-series and static medical data using a real-world time-series generative adversarial network

This study addresses challenges related to privacy issues in utilizing medical data, particularly the protection of personal information. To overcome this obstacle, the research focuses on data synthesis using real-world time-series generative adversarial networks (RTSGAN). A total of 53,005 data were synthesized using the dataset of 15,799 patients with colorectal cancer. The results of the quantitative evaluation of the synthetic data’s quality are as follows: the Hellinger distance ranged from 0 to 0.25; the train on synthetic, test on real (TSTR) and train on real, test on synthetic (TRTS) results showed an average area under the curve of 0.99 and 0.98; a propensity mean squared error was 0.223. The synthetic and real data were similar in the qualitative methods including t-SNE and histogram analyses. The application of synthetic data in predicting five-year survival in colorectal cancer patients demonstrates comparable performance to models based on real data. This study employs distance to closest records and membership inference test to assess potential privacy exposure, revealing minimal risk. This study demonstrated that it is feasible to synthesize medical data, including time-series data, using the RTSGAN, and the synthetic data can be evaluated to accurately reflect the characteristics of real data through quantitative and qualitative methods as well as by utilizing real-world artificial intelligence models.

Seasonality in synthetic average wind speed

There is a growing demand for computer-generated realistic high-fidelity wind speed data for various applications in the wind industry. Such data should capture the non-stationary dynamics of real-world wind time series, as well as be consistent with the statistical descriptors – the probability density function and power spectral density – of the observed wind speed. However, complying with the statistical descriptors is not a guarantee that the seasonality will be correctly reproduced in synthetic data. The seasonality, characterized by the average diurnal and seasonal variations, is driven by the periodicities embedded in diurnal and annual harmonic series respectively. Those periodicities are determined by the long-term orbital forcing components, which establish the insolation for a given latitude and longitude. We show that average diurnal and seasonal variations can be visualized as the output of comb filters, whose fundamental frequencies match the diurnal and annual fundamental frequency respectively. The aforementioned theoretical findings are readily reproduced in synthetic wind speed, generated by a non-parametric data-driven statistical model, based on the phase-randomized Fourier transform. The model, tested on both 10-min and 1-min resolution real-world datasets, yields average non-stationarities in synthetic wind speed with the accuracy close to the computing precision.

DeepSketch: A Query Sketching Interface for Deep Time Series Similarity Search

By empowering domain experts to perform interactive exploration of large time series datasets, sketch-based query interfaces have revitalized interest in the well-studied problem of time series similarity search. In this new interaction paradigm, recent similarity algorithms (e.g., Qetch, Peax, LineNet) that attempt to capture perceptually relevant features have supplanted older, more straightforward distance measures (e.g., Euclidean, DTW). However, the downside of these algorithms is the resulting difficulty in designing corresponding index structures to support efficient similarity search over large datasets, thus necessitating brute-force search. This demo will showcase Deep Time Series Similarity Search (DTS3), our pluggable indexing pipeline for arbitrary distance measures. DTS3 can automatically train a foundation model for any custom, user-supplied distance measure with no strict constraints (e.g., differentiability), thus enabling fast retrieval via an off-the-shelf vector DBMS. Using our DeepSketch web interface, participants can compare DTS3 to the baseline brute-force versions of several similarity algorithms to see that our approach can achieve much lower latency without sacrificing accuracy when searching over large, real-world time series datasets.

A variational approach to quantum gated recurrent units

Quantum Recurrent Neural Networks are receiving an increased attention thanks to their enhanced generalization capabilities in time series analysis. However, their performances were bottlenecked by long training times and unscalable architectures. In this paper, we propose a novel Quantum Recurrent Neural Network model based on Quantum Gated Recurrent Units. It uses a learnable Variational Quantum Layer to process temporal data, interspersed with two classical layers to properly match the dimensionality of the input and output vectors. Such an architecture has fewer quantum parameters than existing Quantum Long Short-Term Memory models. Both the quantum networks were evaluated on periodic and real-world time series datasets, together with the classical counterparts. The quantum models exhibited superior performances compared to the classical ones in all the test cases. The Quantum Gated Recurrent Units outperformed the Quantum Long Short-Term Memory network despite having a simpler internal configuration. Moreover, the Quantum Gated Recurrent Units network demonstrated to be about 25% faster during the training and inference procedure over the Quantum Long Short-Term Memory. This improvement in speed comes with one less quantum circuit to be executed, suggesting that our model may offer a more efficient alternative for implementing Quantum Recurrent Neural Networks on both simulated and real quantum hardware.

Assessing the Impact of Seasonal Decomposition on the Time Series Analysis Accuracy: A Comprehensive Study

Abstract: Time series forecasting plays a pivotal role in decision-making across various domains, ranging from finance to healthcare and weather prediction. The accurate prediction of future values in a time series is vital for informed planning and resource allocation. The aim of this study is to explore whether the utilization of seasonal decomposition techniques, such as classical decomposition, X-12-ARIMA, and seasonal decomposition of time series (STL), can improve the effectiveness of time series forecasting models by separating the data into its distinct components, including trend and seasonality. We conduct a comprehensive analysis using real-world time series data, employing popular forecasting models like ARIMA, exponential smoothing, and machine learning-based approaches. By comparing the forecasting accuracy of these models with and without the application of seasonal decomposition techniques, we provide empirical evidence to support the hypothesis. Our research results provide valuable insights into the tangible ad- vantages of integrating seasonal decomposition techniques in time series forecasting, potentially contributing to enhanced decision support systems across diverse application domains.

Clustering time series by extremal dependence

AbstractThe goal of this paper is to characterize the temporal dependence structure on the extremes of time series and use such dependency to group them. In particular, three similarity measures to capture extremal dependence are proposed, being their performance assessed in different scenarios. This will involve the use of classical time series clustering algorithms, as well as rigorous evaluation of their performance in both simulated scenarios and real-world time series data sets. The focus will be on comparing the performance of these similarity measures with different clustering methods, and illustrate the efficacy of extremal dependence-based clustering in meteorological data. To achieve this, we will consider a dataset consisting of daily maximum temperatures recorded at 500 stations across Europe.

Excitatory and inhibitory neuronal synapse unit: A novel recurrent cell for time series prediction

Time series prediction is one of the most challenging topics in real-world applications. Most studies undertook advanced mathematics and computer techniques above benchmark methods but often unable to guarantee forecasting models’ generalization and reliability. This paper proposed a biologically motivated recurrent unit based on neuronal synaptic activity mechanism and chaotic behaviors in deep learning domain called Excitatory and Inhibitory Neural Synapse unit (EINS). The major contribution of this research is to create a flexible and robust recurrent unit based on neuroscience theory. It conducted experiments on three real-world time series forecast examples including finance, household power, weather for generalization and robustness evaluation using eight state-of-the-art recurrent architecture units as baseline models to compare convergence rate and forecasting accuracy. Experimental results showed that EINS achieved satisfactory performances in time series prediction reliability and effectiveness.