Nonstationary Time Series Prediction Research Articles

A New Algorithm for Predicting Dam Deformation Using Grey Wolf-Optimized Variational Mode Long Short-Term Neural Network

To solve the problems of difficult-to-model parameter selections, useful-signal extraction and improper-signal decomposition in nonlinear, non-stationary dam displacement time series prediction methods, we propose a new predictive model for grey wolf optimization and variational mode decomposition and long short-term memory (GVLSTM). Firstly, we used the grey wolf optimization (GWO) algorithm to optimize the parameters of variable mode decomposition (VMD), obtaining the optimal parameter combination. Secondly, we used multiscale permutation entropy (MPE) as a standard to select signal screening, determining and recon-structing the effective modal components. Finally, the long short-term memory neural network (LSTM) was used to learn the dam deformation characteristics. The result shows that the GVLSTM model can effectively reduce the estimation deviation of the prediction model. Compared with VMDGRU and VMDANN, the average RMSE and MAE value of each station is increased by 19.11%~28.58% and 27.66%~29.63%, respectively. We used determination (R2) coefficient to judge the performance of the prediction model, and the value of R2 was 0.95~0.97, indicating that our method has good performance in predicting dam deformation. The proposed method has outstanding advantages of high accuracy, reliability, and stability for dam deformation prediction.

Forecasting Non-stationary Time Series Using Deep Learning in a Fuzzy Time Series Framework and its Application to Stock Markets

Non-stationary time series prediction is challenging due to its dynamic and complex nature. Fuzzy time series models offer a promising solution for forecasting such data, but a key challenge lies in partitioning the universe of discourse, which significantly impacts forecasting accuracy. Traditional fuzzy time series models often use equal-length interval partitioning, which is more suited for stationary data and limits their adaptability to non-stationary time series. This paper introduces a novel variable-length interval partitioning method designed specifically for non-stationary time series. The developed method combines a Long Short-Term Memory (LSTM) Autoencoder with K-means clustering, enabling dynamic, data-driven partitioning that adapts to the changing characteristics of the data. The LSTM Autoencoder encodes the time series, which is clustered using K-means, and intervals are defined based on cluster centers. Furthermore, the Variable Length Interval Partitioning-based Fuzzy Time Series model (VLIFTS) is developed by incorporating this partitioning method and the concepts of Markov chain and transition probability matrix. In this model, fuzzy sets are viewed as states of a Markov chain, and transition probabilities are used in the forecasting phase. The model is validated on stock market indices Nifty 50, NASDAQ, S&P 500, and Dow Jones. Stationarity and heteroscedasticity are tested using Augmented Dickey-Fuller (ADF) and Levene's tests respectively. Statistical forecast accuracy metrics Root Mean Squared Error (RMSE) and Mean Absolute Percent Error (MAPE) show that VLIFTS significantly improves forecasting accuracy over traditional models. This hybrid approach enhances fuzzy time series modelling and can be applied to various non-stationary time series forecasting problems.

Singular spectrum analysis (SSA) based hybrid models for emergency ambulance demand (EAD) time series forecasting

Abstract Accepted by: Konstantinos Nikolopoulos One of the challenges of emergency ambulance demand (EAD) time series prediction lies in their non-stationary nature. We study this important problem and propose two hybrid forecasting models, which combine the singular spectrum analysis (SSA) time-series technique with autoregressive integrated moving average (ARIMA) parameterized multivariate forecasting. Both daily and hourly time series are studied. The non-stationary time series are decomposed into three eigentriples by SSA: trends, periodic components and residuals. Selection of the group boundary point of the periodic component is a key issue in the SSA method. We use spectrum analysis to compute a threshold for maximum information content of periodic components. ARIMA mean value prediction models are employed to forecast the trends, periodic components and residuals sub-series. Our research compares ARIMA and SSA-based hybrid models by considering the emergency dispatching departure records of six core districts in Guangzhou city from 1 January 2021 to 31 December 2021. Results show that the integrated SSA-ARIMA model performs best. SSA is a very effective pre-processing method for non-stationary time series prediction. The predictive accuracy of using a hybrid model for hourly EAD time series is higher than that for daily ones. Our discussion should be useful for improving EAD prediction in contexts others than that considered in our research.

Transfer learning-based adaptive recursive neural network for short-term non-stationary building heating load prediction

Building energy consumption is a non-stationary time series, and its distribution law changes over time. Traditional machine-learning models are prone to model shift, which leads to a reduction of their prediction accuracy when applied to building energy consumption forecasting. Therefore, in this paper, an adaptive neural network framework is proposed for non-stationary building energy consumption prediction based on transfer learning, in which the training datasets are divided into the most dissimilar periods, and based on the transfer learning mechanism, the different periods of data are learned to obtain the minimum overall loss to improve the model generalization. Taking LSTM and GRU models as examples, adaptive long short-term memory (adaLSTM) and adaptive gated recurrent unit (adaGRU) building energy consumption prediction models are established. The models were trained and verified using heating load data from Xi'an, China. The results show that compared with LSTM model, the coefficient of determination, root mean square error, the coefficient of variation of the root mean squared error and mean absolute error of the adaLSTM model were improved by 0.61%, 37.78%, 38.05% and 30.69%, respectively, and the over-fitting degree was reduced by 227.7%. Compared with the traditional GRU model, the corresponding evaluation indexes of adaGRU were improved by 2.50%, 70.58%, 70.64% and 68.83%, respectively, and the over-fitting degree was improved by 505.7% points. The adaptive recurrent neural network framework proposed in this paper is a generalized approach which can be applied to other non-stationary time series prediction models.

A 2-phase prediction of a non-stationary time-series by Taylor series and reinforcement learning

Prediction of a non-stationary time-series is hard as the frequency components and their amplitudes in the series vary randomly over time. This paper proposes a 2-phase approach for prediction of such non-stationary time-series. The first phase employs Taylor series to approximately predict the next time-point value in the series from its current and last two preceding sample values. The Taylor series based prediction, however, presumes that the time-series is locally stationary. The second phase employs reinforcement learning to refine the Taylor series based prediction further, particularly at the juncture of structural changes in the time-series. The reinforcement learning based prediction is realized with the help of an adaptive probabilistic learning matrix that evolves to encode the mapping between current prediction error and the error-compensation for the next sample. On convergence of the matrix, the saved probabilities are used to determine the error-compensation for the next sample from the estimated error at the current sample. The additive error-compensation is then utilized to rectify the results of prediction obtained by Taylor series. Experiments undertaken confirm that the proposed 2-phase prediction outperforms the state-of-the-art prediction techniques by a significant margin of prediction accuracy.

AE-DIL: A double incremental learning algorithm for non-stationary time series prediction via adaptive ensemble

Many dynamic processes in the real world can be modeled as time series, so time series prediction is significant for social and economic development. The inherent non-stationarity of time series obtained from actual projects may make it difficult to predict accurately. To alleviate this problem, in this paper, a Double Incremental Learning algorithm via Adaptive Ensemble, termed as AE-DIL for short, is proposed for non-stationary time series prediction. AE-DIL provides a general online prediction framework consisting of two modules. The first detects changes based on the statistical hypothesis test and self-adaptive sliding window technology. The second updates the prediction model based on double incremental learning and adaptive ensemble learning. The effectiveness of the proposed algorithm is empirically underpinned by the experiments conducted on seven benchmark time series datasets, compared with several baselines and state-of-the-art models.

Nonstationary Time Series Prediction Based on Deep Echo State Network Tuned by Bayesian Optimization

The predictions from time series data can help us sense development trends and make scientific decisions in advance. The commonly used forecasting methods with backpropagation consume a lot of computational resources. The deep echo state network (DeepESN) is an advanced prediction method with a deep neural network structure and training algorithm without backpropagation. In this paper, a Bayesian optimization algorithm (BOA) is proposed to optimize DeepESN to address the problem of increasing parameter scale. Firstly, the DeepESN was studied and constructed as the basic prediction model for the time series data. Secondly, the BOA was reconstructed, based on the DeepESN, for optimal parameter searching. The algorithm is proposed within the framework of the DeepESN. Thirdly, an experiment was conducted to verify the DeepESN with a BOA within three datasets: simulation data generated from computer programs, a real humidity dataset collected from Beijing, and a power load dataset obtained from America. Compared with the models of BP (backpropagation), LSTM (long short-term memory), GRU (gated recurrent unit), and ESN (echo state network), DeepESN obtained optimal results, which were 0.0719, 18.6707, and 764.5281 using RMSE evaluation. While getting better accuracy, the BOA optimization time was only 323.4 s, 563.2 s, and 9854 s for the three datasets. It is more efficient than grid search and grey wolf optimizer.

Decomposition-Based Multi-Step Forecasting Model for the Environmental Variables of Rabbit Houses.

To improve prediction accuracy and provide sufficient time to control decision-making, a decomposition-based multi-step forecasting model for rabbit house environmental variables is proposed. Traditional forecasting methods for rabbit house environmental parameters perform poorly because the coupling relationship between sequences is ignored. Using the STL algorithm, the proposed model first decomposes the non-stationary time series into trend, seasonal, and residual components and then predicts separately based on the characteristics of each component. LSTM and Informer are used to predict the trend and residual components, respectively. The aforementioned two predicted values are added together with the seasonal component to obtain the final predicted value. The most important environmental variables in a rabbit house are temperature, humidity, and carbon dioxide concentration. The experimental results show that the encoder and decoder input sequence lengths in the Informer model have a significant impact on the model's performance. The rabbit house environment's multivariate correlation time series can be effectively predicted in a multi-input and single-output mode. The temperature and humidity prediction improved significantly, but the carbon dioxide concentration did not. Because of the effective extraction of the coupling relationship among the correlated time series, the proposed model can perfectly perform multivariate multi-step prediction of non-stationary time series.

Novel hybrid multi-head self-attention and multifractal algorithm for non-stationary time series prediction

Traditional time series prediction methods have shown their outstanding capabilities in time series prediction. However, due to essential differences in volatility characteristics among diverse types of non-stationary multivariate time series (NSMTS), it is difficult for traditional methods to maintain robust prediction performance. This study proposes a novel dynamic recurrent neural network to achieve stable and robust prediction performance. First, a multifractal gated recurrent unit (MF-GRU) based on the multifractal method is proposed to extract volatility characteristics. Meanwhile, to strengthen the parameters of the historical hidden layer state that has a more significant impact on the output, a self-attention mechanism is introduced into the MF-GRU, leading to a multifractal gated recurrent unit multi-head self-attention model. The efficiency of the proposed model was verified on public datasets. The experimental results show that the proposed model outperforms the traditional methods, such as long short-term memory (LSTM), the gated recurrent unit (GRU), and the minimal gated unit (MGU). etc.

Stochastic configuration network based on improved whale optimization algorithm for nonstationary time series prediction

AbstractThe stochastic configuration network (SCN), a type of randomized learning algorithm, can solve the infeasible problem in random vector functional link (RVFL) by establishing a supervisory mechanism. The advantages of fast learning, convergence and not easily falling into local optima make SCN popular. However, the prediction effect of SCN is affected by the parameter settings and the nonstationarity of input data. In this paper, a hybrid model based on variational mode decomposition (VMD), improved whale optimization algorithm (IWOA), and SCN is proposed. The SCN will predict relatively stable data after decomposition by VMD, and parameters of SCN are optimized by IWOA. The IWOA diversifies the initial population by employing logistic chaotic map based on bit reversal and improves the search ability by using Lévy flight. The exploration and exploitation of IWOA are superior to those of other optimization algorithms in multiple benchmark functions and CEC2020. Moreover, the proposed model is applied to the prediction of the nonstationary wind speeds in four seasons. We evaluate the performance of the proposed model using four evaluation indicators. The results show that the R2 of the proposed model under four seasons are more than 0.999, and the root mean square error, mean absolute error, and symmetric mean absolute percentage error are less than 0.3, 0.17, and 13%, respectively, which are almost 1/10, 1/10, and 1/4 those of SCN, respectively.

Broad Echo State Network with Reservoir Pruning for Nonstationary Time Series Prediction.

The nonstationary time series is generated in various natural and man-made systems, of which the prediction is vital for advanced control and management. The neural networks have been explored in the time series prediction, but the problem remains in modeling the data's nonstationary and nonlinear features. Referring to the time series feature and network property, a novel network is designed with dynamic optimization of the model structure. Firstly, the echo state network (ESN) is introduced into the broad learning system (BLS). The broad echo state network (BESN) can increase the training efficiency with the incremental learning algorithm by removing the error backpropagation. Secondly, an optimization algorithm is proposed to reduce the redundant information in the training process of BESN units. The number of neurons in BESN with a fixed step size is pruned according to the contribution degree. Finally, the improved network is applied in the different datasets. The tests in the time series of natural and man-made systems prove that the proposed network performs better on the nonstationary time series prediction than the typical methods, including the ESN, BLS, and recurrent neural network.

Fractional stochastic configuration networks-based nonstationary time series prediction and confidence interval estimation

Time series prediction is an important topic in the field of data analytics for real industrial production. However, the time series from real system usually has strong nonstationarity, which affects the generalization ability of the prediction model. An improved forecasting technique, named as fractional stochastic configuration networks (FSCN), is proposed for the prediction of nonstationary time series. FSCN is built on the basis of traditional stochastic configuration network by introducing fractional differential operator. The fractional differential technique avoids the challenge of infinite variance caused by modeling nonstationary series and the over-difference problem caused by traditional integer order difference. First, several data analysis methods are introduced to find the tendency, periodicity and probability density distribution characteristics hidden in the raw industrial data. Hurst exponent is calculated to determine the order of fractional difference to eliminate the nonstationarity of the raw data. Then FSCN network is constructed to model and forecast the sequential data. An explicit prediction uncertainty is derived to provide the confidence interval for the FSCN prediction. The proposed method is tested on a nonstationary time series benchmark dataset and a real cooling system. The experiment result demonstrates that it has a good potential prediction performance compared with several traditional prediction methods.

DWE-IL: a new incremental learning algorithm for non-stationary time series prediction via dynamically weighting ensemble learning

In this work, an Incremental Learning Algorithm via Dynamically Weighting Ensemble Learning (DWE-IL) is proposed to solve the problem of Non-Stationary Time Series Prediction (NS-TSP). The basic principle of DWE-IL is to track real-time data changes by dynamically establishing and maintaining a knowledge base composed of multiple basic models. It trains the base model for each non-stationary time series subset, and finally combine each base model with dynamically weighting rules. The emphasis of the DWE-IL algorithm lies in the update of data weights and base model weights and the training of the base model. Finally, the experimental results of the DWE-IL algorithm on six non-stationary time series datasets are presented and compared with those of several other excellent algorithms. It can be concluded from the experimental results that the DWE-IL algorithm provides a good solution to the challenges of the NS-TSP tasks and has significantly superior performance over other comparative algorithms.

Evolutionary artificial intelligence model via cooperation search algorithm and extreme learning machine for multiple scales nonstationary hydrological time series prediction

Reliable and stable hydrological prediction plays a vitally crucial role in the scientific operation of water resources system. As a famous artificial intelligence method for hydrological forecasting, extreme learning machine (ELM) has the virtues of fast training efficiency and strong generalization performance but is easily trapped into local optima because the preset computation parameters often remain unchanged in the learning process. In order to overcome this shortcoming, a practical evolutionary artificial intelligence model is developed for multiple scales nonstationary hydrological time series prediction. In the proposed method, an emerging evolutionary method called cooperation search algorithm (CSA) is used to search for the optimal input-hidden weights and hidden biases of the ELM model for the first time. The proposed method is used to forecast the runoff time series of three real-world hydrological stations in China. The experimental results show that the CSA approach can effectively determine satisfying network parameters of the ELM model, while our method can produce better results than the traditional ELM method in terms of all the performance evaluation indexes. Taking 1-step-ahead runoff forecasting at station B as an example, our method betters the ELM method with 15.76% and 42.35% improvements in both root mean squared error and mean absolute percentage error at the testing phase. Thus, a novel multiscale nonstationary hydrological prediction tool is developed to support the decision-making of water resource system.

Mechanism analysis of non-inertial particle swarm optimization for Internet of Things in edge computing

This paper carries out the mechanism analysis of non-inertial particle swarm optimization. It is a modified algorithm based on a new kinetic equation which is applied to the prediction of non-stationary time series for the internet of things in the edge computing. In order to avoid premature convergence and accelerate convergence rate, different from standard particle swarm optimization, the modified algorithm uses a new kinetic equation to lead particles motion direction, besides generalized opposition-based learning (GOBL) and adaptive elite mutation (AEM) strategies. The work presents the second order standardized recurrence equation for the new kinetic equation whose corresponding characteristic equations can be analyzed via the difference functions to obtain the boundaries of coefficients and its convergence region. Besides, GOBL and AEM strategies are also analyzed to boost global and local convergence of the algorithm as an interpretation. It is illustrated that performance analysis of the algorithm with two well-known test functions. The good performance is further validated from application in edge computing.

NON-STATIONARY TIME SERIES PREDICTION USING ONE-DIMENSIONAL CONVOLUTIONAL NEURAL NETWORK MODELS

The main goal of non-stationary time series prediction is the construction, identification, configuration and verification of their models. The efficiency of using machine learning technologies for the analysis of non-stationary time series is shown due to their ability to model complex nonlinear dependencies in the behaviourof the time series from both depending on previous values and external factors, to analysefeatures, relationships and complex interactions. The features of time series prediction using a one-dimensional convolutional neural network are discussed. The features of the architecture and the training process when using a one-dimensional convolutional neural network are considered on the example of solving the problems to predictsales and build a forecast of company stock prices. To improve the quality of the prediction, the initial time series were pre-processed by the moving average method in the window. Computer modellingof the predicting problem using the one-dimensional convolutional neural network was performed in the Python programming language. The sales prediction using the proposed one-dimensional convolutional neural network model predicted volume sale of cars and commercial vehicles in Vietnam from two thousand and eleven to two thousand and eighteen. The one-dimensional convolutional neural network model has given a high accuracy of prediction with seasonal trend data. In stock prices prediction, another architecture of one-dimensional convolutional neural network model was launched, which corresponds to non-stationary data with large lengths of data series with small intervals between minutes, such as stock trading statistics per minute. In this project, data is taken from Amazon Nasdaq one hundred for forty thousand five hundred and sixty data points. The data is divided into training and test sets. The test suite is used to verify the actual performance of the model. It is shown that the model of a one-dimensional convolutional neural network gives good results in the presence of both seasonal and trend components of the time series with large data sizes.

Improved Kernel Recursive Least Squares Algorithm Based Online Prediction for Nonstationary Time Series

We present an improved kernel recursive least squares (KRLS) algorithm for the online prediction of nonstationary time series. In order to adaptively sparsify a selected kernel dictionary for the KRLS algorithm, the approximate linear dependency (ALD) criterion based KRLS algorithm is combined with the quantized kernel recursive least squares algorithm to provide an initial framework. In order to sufficiently track the strongly changeable dynamic characteristics due to nonstationarity, a forgetting factor is further inserted into the proposed combined algorithm. It is shown that our proposed algorithm, referred to as the FFIKRLS algorithm, provides a clearly compatible algorithm structure and can be improved by the existing modeling techniques from both mapping and weights updating perspectives. Numerical simulations using benchmark Lorenz time series in comparison with existing methods have demonstrated that the proposed algorithm has superior performance in terms of both predictive accuracy and kernel dictionary size.

Fast Adaptive Gradient RBF Networks For Online Learning of Nonstationary Time Series

For a learning model to be effective in online modeling of nonstationary data, it must not only be equipped with high adaptability to track the changing data dynamics but also maintain low complexity to meet online computational restrictions. Based on these two important principles, in this paper, we propose a fast adaptive gradient radial basis function (GRBF) network for nonlinear and nonstationary time series prediction. Specifically, an initial compact GRBF model is constructed on the training data using the orthogonal least squares algorithm, which is capable of modeling variations of local mean and trend in the signal well. During the online operation, when the current model does not perform well, the worst performing GRBF node is replaced by a new node, whose structure is optimized to fit the current data. Owing to the local one-step predictor property of GRBF node, this adaptive node replacement can be done very efficiently. Experiments involving two chaotic time series and two real-world signals are used to demonstrate the superior online prediction performance of the proposed fast adaptive GRBF algorithm over a range of benchmark schemes, in terms of prediction accuracy and real-time computational complexity.

Online Sequential Model for Multivariate Time Series Prediction With Adaptive Forgetting Factor

In the process of online prediction of multivariable non-stationary time series by kernel extreme learning machine (KELM), the dynamic characteristics of the system which are difficult to determine have always posed a big problem. We propose an online sequential prediction model with an adaptive forgetting factor (AFF) for multivariable time series to solve this problem. The multivariable time series instead of variable itself is reconstructed firstly. AFF is introduced into the objective function and can be adjusted iteratively and adaptively with the system changes. As a result, higher weight can be allocated for the fresh and more important samples while the old failure samples can be quickly forgotten. The model sparsification uses a fast leave-one-out cross-validation (FLOO-CV) method to set a prediction error threshold so that samples can be selected conditionally to form a dictionary. Besides, the dictionary parameters, including AFF and kernel parameters, are recursively updated simultaneously without increasing calculation complexity. The experimental results show that, compared with four fashionable KLEM methods, the proposed AFF-OSKELM has a better dynamic tracking ability and adaptability. Moreover, compared with single variable prediction, the spatial reconstructed multivariable has higher prediction accuracy and stability.

A new optimization algorithm for non-stationary time series prediction based on recurrent neural networks

Deep neural network with recurrent structure was proposed in the recent years and has been applied to time series forecasting. Many optimization algorithms are developed under the assumption of invariant and stationary data distributions, which is invalid for the non-stationary data. A novel optimization algorithm for modeling non-stationary time series is proposed in this paper. A moving window and exponential decay weights are used in this algorithm to eliminate the effects of the history gradients. The regret bound of the new algorithm is analyzed to ensure the convergency of the calculation. Simulations are done on short-term power load data sets, which are typically non-stationary. The results are superior to the existing optimization algorithms.