Univariate Time Series Prediction Research Articles

Dynamic adaptive graph convolutional transformer with broad learning system for multi-dimensional chaotic time series prediction

Chaotic time series data is extensively applied in financial stocks, climate monitoring, and sea clutter, in which data fusion from various sources and multi-sensor information make accurate predictions of chaotic time series challenging under complex nonlinear conditions. Previous works focus on designing different model frameworks to capture the temporal dependence and extract richer nonlinear features to improve the accuracy of univariate chaotic time series prediction, which ignores the spatial dependence of multivariable. However, in this paper, we argue that spatial correlation among multiple variables is essential to improve the prediction accuracy of chaotic time series. To fill the gap, we innovatively propose a Dynamic Adaptive Graph Convolutional Transformer with a Broad Learning System (DAGCT-BLS), a GCN and Transformer-based model utilizing multivariate spatial dependence for multi-dimensional chaotic time series forecasting. In DAGCT-BLS, the multivariate chaotic time series are reconstructed into the phase space, and the reconstructed data are rapidly feature-extracted using a cascade network BLS with frozen weights to maximize the retention of chaotic properties and nonlinear relationships. Then, the Dynamic Adaptive Graph Convolutional Network (DAGCN) is proposed to capture the spatial correlation among the multiple variables. Finally, improved multi-head attention of the Transformer Encoder is used to capture the temporal dependence of the phase point sequence. Experiments of our proposed model on three datasets (Lorenz, Rossler, and Sea clutter) show that DAGCT-BLS can achieve the best prediction performance and have strong interpretability, and multivariate-based joint modeling of chaotic time series helps to improve the prediction performance.

A process knowledge-based hybrid method for univariate time series prediction with uncertain inputs in process industry

In process industry, accurate and efficient univariate time series prediction (TSP) of key process parameters is crucial for optimizing production processes. However, noisy data and varying working conditions introduce uncertainties that hinder the modeling process. Data-driven methods have been proposed to address input-uncertain TSP models, but they often fail to obtain sufficient and high-quality industrial data for model establishment in complex and dynamic production process. The paper presents a process knowledge-based hybrid method for univariate TSP that handles noisy inputs in different working conditions. Primarily, a knowledge-embedded deep kernel network (KE-DKN) model is proposed that approximates the outputs to the knowledge space in noisy industrial environment, by embedding instantiated process knowledge as soft constraints into the conventional DKN network. Further, a two-stage training schema is designed for KE-DKN to reduce the interference superposition on the predicted variable from random noisy data and operational interferences. Moreover, the pre-trained parameters transferring is conducted from stable working conditions to mitigate the low modeling efficiency in unstable working conditions. In two cases from process industry, the proposed method delivers higher prediction accuracy and robustness than baseline methods in both working conditions, and the knowledge transfer significantly enhances the modeling efficiency in unstable working conditions.

AI and semantic ontology for personalized activity eCoaching in healthy lifestyle recommendations: a meta-heuristic approach

BackgroundAutomated coaches (eCoach) can help people lead a healthy lifestyle (e.g., reduction of sedentary bouts) with continuous health status monitoring and personalized recommendation generation with artificial intelligence (AI). Semantic ontology can play a crucial role in knowledge representation, data integration, and information retrieval.MethodsThis study proposes a semantic ontology model to annotate the AI predictions, forecasting outcomes, and personal preferences to conceptualize a personalized recommendation generation model with a hybrid approach. This study considers a mixed activity projection method that takes individual activity insights from the univariate time-series prediction and ensemble multi-class classification approaches. We have introduced a way to improve the prediction result with a residual error minimization (REM) technique and make it meaningful in recommendation presentation with a Naïve-based interval prediction approach. We have integrated the activity prediction results in an ontology for semantic interpretation. A SPARQL query protocol and RDF Query Language (SPARQL) have generated personalized recommendations in an understandable format. Moreover, we have evaluated the performance of the time-series prediction and classification models against standard metrics on both imbalanced and balanced public PMData and private MOX2-5 activity datasets. We have used Adaptive Synthetic (ADASYN) to generate synthetic data from the minority classes to avoid bias. The activity datasets were collected from healthy adults (n = 16 for public datasets; n = 15 for private datasets). The standard ensemble algorithms have been used to investigate the possibility of classifying daily physical activity levels into the following activity classes: sedentary (0), low active (1), active (2), highly active (3), and rigorous active (4). The daily step count, low physical activity (LPA), medium physical activity (MPA), and vigorous physical activity (VPA) serve as input for the classification models. Subsequently, we re-verify the classifiers on the private MOX2-5 dataset. The performance of the ontology has been assessed with reasoning and SPARQL query execution time. Additionally, we have verified our ontology for effective recommendation generation.ResultsWe have tested several standard AI algorithms and selected the best-performing model with optimized configuration for our use case by empirical testing. We have found that the autoregression model with the REM method outperforms the autoregression model without the REM method for both datasets. Gradient Boost (GB) classifier outperforms other classifiers with a mean accuracy score of 98.00%, and 99.00% for imbalanced PMData and MOX2-5 datasets, respectively, and 98.30%, and 99.80% for balanced PMData and MOX2-5 datasets, respectively. Hermit reasoner performs better than other ontology reasoners under defined settings. Our proposed algorithm shows a direction to combine the AI prediction forecasting results in an ontology to generate personalized activity recommendations in eCoaching.ConclusionThe proposed method combining step-prediction, activity-level classification techniques, and personal preference information with semantic rules is an asset for generating personalized recommendations.

Decomposition strategy and attention-based long short-term memory network for multi-step ultra-short-term agricultural power load forecasting

Accurate forecasting of the agricultural power load is important for rural electricity networks’ safe and stable operation. But it is more uncertain and difficult to forecast than industrial, commercial, and residential loads. Multivariate time series prediction methods have problems such as poor refinement of weather services and difficulty in quantifying the influencing factors. Univariate time series prediction methods should be emphasized, and the characteristic information in the historical load data should be fully exploited. Therefore, this paper systematically investigates the decomposition strategy and attention-based recurrent neural network for univariate time-series multi-step prediction. The decomposition strategy is employed to generate derived variable sequences from the target sequence as inputs for the model, while the dual-stage attention mechanism is used for dynamically selecting features. A one-year real-world dataset of agricultural power loads in southern China is used for validation. The effectiveness of two different decomposition algorithms on dual-stage attention-based recurrent neural network (DARNN) is discussed, then the proposed ensemble empirical mode decomposition and dual-stage attention-based recurrent neural network (EEMD-DARNN) is compared with four univariate time series prediction models and three ablation models. The proposed model is optimal in four evaluation indicators, with R2, RMSE, MAPE, and MAE of 0.970, 12.298, 0.040, and 7.335 respectively in the shortest forecast horizon (15 min), and 0.941, 17.305, 0.065 and 11.347 respectively in the longest forecast horizon (1 h). It is a significant improvement over the existing models. In conclusion, EEMD-DARNN can achieve more accurate ultra-short-term agricultural power load forecasting.

Deep and Machine Learning Models to Forecast Photovoltaic Power Generation

The integration and management of distributed energy resources (DERs), including residential photovoltaic (PV) production, coupled with the widespread use of enabling technologies such as artificial intelligence, have led to the emergence of new tools, market models, and business opportunities. The accurate forecasting of these resources has become crucial to decision making, despite data availability and reliability issues in some parts of the world. To address these challenges, this paper proposes a deep and machine learning-based methodology for PV power forecasting, which includes XGBoost, random forest, support vector regressor, multi-layer perceptron, and LSTM-based tuned models, and introduces the ConvLSTM1D approach for this task. These models were evaluated on the univariate time-series prediction of low-volume residential PV production data across various forecast horizons. The proposed benchmarking and analysis approach considers technical and economic impacts, which can provide valuable insights for decision-making tools with these resources. The results indicate that the random forest and ConvLSTM1D model approaches yielded the most accurate forecasting performance, as demonstrated by the lowest RMSE, MAPE, and MAE across the different scenarios proposed.

Multivariate Deep Reconstruction Neural Network for Multi-step-ahead Prediction of Industrial Process Quality Variables

In industrial processes, deep learning has been widely used to solve the soft sensing problem. Multi-step-ahead prediction is one of the most challenging problems in the field of soft sensors. Recently, N-Beats has been proposed as a promising deep neural architecture for multi-step prediction, but it can only be used for univariate time series prediction, not for industrial soft sensor modeling. Inspired by N-Beats, a novel deep learning model, multivariate deep reconstruction neural network (MDRNN), is proposed for multivariate time series prediction in this work. MDRNN is designed on the basis of a doubly residual structure with a deep stack of fully-connect layers. MDRNN inherits the merits of N-Beats and incorporates the “doubly residual stacking” idea into the industrial soft sensor modeling to improve the prediction accuracy. To evaluate the feasibility and effectiveness of the proposed MDRNN, it is applied to the quality prediction task and validated with two real-world industrial processes. The application results demonstrated that the proposed MDRNN can achieve higher prediction accuracy compared to the existing N-Beats and Multilayer Perceptron (MLP)-based methods.

Forecasting Landfilling Indicators Of Municipal Waste In Greece Using Univariate Time-series Model

Implementation of environmental policies, is one of the European Commission's key priorities, as confirmed by its proposal for the 7th Environment Action Programme and the Roadmap to a resource-efficient Europe. Subsequently, the European Union revised its waste legislation to address these challenges more adequately. As a result, the EU's Waste Framework Directive and Landfill Directive set binding targets for recycling municipal waste and diverting biodegradable municipal waste from landfills. This paper deals with the monitoring of municipal waste management operations in Greece, in terms of technical content and ability to reach the targets of the initiatives. In addition, a more detailed report on waste indicators published by Eurostat will be based on an Exploratory Data Analysis and on a Univariate time-series prediction model. Finally, the overall aim of this study is to gain maximum insight into the data and understand its underlying structure through graphical representation, to achieve the most accurate forecast for the landfilling operation.

Implementation of Particle Swarm Optimization (PSO) to Improve Neural Network Performance in Univariate Time Series Prediction

One of the oldest known predictive analytics techniques is time series prediction. The target in time series prediction is use historical data about a specific quantity to predicts value of the same quantity in the future. Multivariate time series (MTS) data has been widely used in time series prediction research because it is considered better than univariate time series (UTS) data. However, in reality MTS data sets contain various types of information which makes it difficult to extract information to predict the situation. Therefore, UTS data still has a chance to be developed because it is actually simpler than MTS data. UTS prediction treats forecasts as a single variable problem, whereas MTS may employ a large number of time-concurred series to make predictions. Neural Network (NN) model could be built to predict the target variable given the other (predictor) variables. In this study, we used Particle Swarm Optimization (PSO) algorithm to optimize performance of NN on a UTS dataset. Our proposed model is validated using x-validation and and use RMSE to measure its performance. The experimental results show that NN performance after optimization using PSO produces good results compared to classical NN performance. This is evidenced by the value of RMSE = 0.410 which is the smallest RMSE value produced. The smaller the RMSE value, the better the model performance. It can be concluded that the proposed method can improve NN performance on UTS data.

CDA-LSTM: an evolutionary convolution-based dual-attention LSTM for univariate time series prediction

CDA-LSTM: an evolutionary convolution-based dual-attention LSTM for univariate time series prediction

Univariate water consumption time series prediction using deep learning in neural network (DLNN)

Univariate water consumption time series prediction using deep learning in neural network (DLNN)

Univariate Time Series Prediction of Wind speed with a case study of Yanbu, Saudi Arabia

Wind energy is a promising alternativefor renewable source of energy pursued world-wide to reduce carbon emissions for a green future. The prediction of wind speed is a challenging subject and plays an instrumental role in development of wind power systems (particularly grid connected renewable energy systems where predicting wind speed facilitates manipulation of the load on the grid). Modern machine learning techniques including neural networks have been widely utilized for this purpose. Literature indicates availability of several models for estimation of the wind speed one hour ahead and the hourly wind speed data profile one day ahead. This paper considers the prediction of wind energy as a univariate time series (UVT) prediction problem and employs major prediction algorithms including the K-Nearest Neighbors (kNN), Random Forest (RF), Support Vector Regression (SVR), Holt-Winter and ARIMA method. Forecasting a univariate time series depends only on past wind speed data values, rather than use of external data attributes like wind direction or weather forecast for prediction algorithm. In the present study (as a case-study), 13 years of hourly average wind speed data (of the period 1970-1982) of Yanbu, Saudi Arabia has been utilized to evaluate the performance of selected algorithms. Yanbu is an industrial city that plays a major role in the economy of Saudi Arabia. The findings showed that SVR, RF and ARIMA methods exhibit a better forecastingperformance in relation to four evaluation parameters of Mean Absolute Percentage Error(MAPE),Symmetric Mean Absolute Percentage Error (sMAPE),Mean Absolute Error (MAE) and Mean Absolute Scaled Error (MASE).

Differentiable Neural Architecture Search for High-Dimensional Time Series Forecasting

This study applies neural architecture search (NAS) techniques to the modeling of high-dimensional time series data such as multi-variate stock indices. It is known that traditional NAS method applies fully connected directed acyclic graph (DAG) for searching cell structures that requires high computational cost and cannot include the two-input operations such as the Hadamard product. To address the drawback of the DAG backbone, a novel two-input backbone cell architecture for recurrent neural networks is proposed, in which each candidate operation is also carried out with two inputs. Instead of using DAG, we simplify the backbone by considering the prior knowledge as an effective backbone such as preserving identity mappings. The cell structures will be incorporated in different types of model architectures including stacked long short-term memory (LSTM), gated recurrent unit (GRU) and attention-based encoder-decoder models. The experimental results on BRICS, G7 and G20 indices indicate that models with recurrent neural network (RNN) cells searched by the proposed backbone structure can significantly outperform baseline models including autoregressive integrated moving average model (ARIMA), vector autoregression (VAR), and stacked LSTM/GRUs. For neural architecture search, the proposed backbone is shown to be more effective compared to the classic differentiable architecture search (DARTS) in both uni-variate and multi-variate time series prediction tasks. Further analysis demonstrates that the pruned cells of the proposed backbone usually contains the Hadamard product introduced as a two-input operation, while the number of parameters involved in these pruned cells is on the same order with the baseline cells.

Time Series Prediction Based on Facebook Prophet: A Case Study, Temperature Forecasting in Myintkyina

Temperature forecasting is a progressive and time series analysis process to forecast the state of the temperature for a certain location in coming time. Nowadays, agriculture and manufacturing sectors are mostly dependent on temperature so forecasting is important to be precise because temperature warnings can save life and property. In this work, the Prophet Forecasting Model is used for Myitkyina's annual temperature forecasting using historical (2010 to 2017) time series data. Myitkyina is the capital city of the northernmost state (Kachin) in Myanmar, located 1480 kilometers from Yangon. Prophet is a modular regression model for time series predictions with high accuracy by using simple interpretable parameters that consider the effect of custom seasonality and holidays. In this study, the temperature forecasting model is proposed by using weather dataset provided by an International institution, National Oceanic and Atmospheric Administration (NOAA). This work implements the multi-step univariate time series prediction model and compares the forecasted value against the actual data. Such findings check that the proposed forecasting model provides an efficient and accurate prediction for temperature in Myitkyina.

A Time Series Sustainability Assessment of a Partial Energy Portfolio Transition

Energy portfolios are overwhelmingly dependent on fossil fuel resources that perpetuate the consequences associated with climate change. Therefore, it is imperative to transition to more renewable alternatives to limit further harm to the environment. This study presents a univariate time series prediction model that evaluates sustainability outcomes of partial energy transitions. Future electricity generation at the state-level is predicted using exponential smoothing and autoregressive integrated moving average (ARIMA). The best prediction results are then used as an input for a sustainability assessment of a proposed transition by calculating carbon, water, land, and cost footprints. Missouri, USA was selected as a model testbed due to its dependence on coal. Of the time series methods, ARIMA exhibited the best performance and was used to predict annual electricity generation over a 10-year period. The proposed transition consisted of a one-percent annual decrease of coal’s portfolio share to be replaced with an equal share of solar and wind supply. The sustainability outcomes of the transition demonstrate decreases in carbon and water footprints but increases in land and cost footprints. Decision makers can use the results presented here to better inform strategic provisioning of critical resources in the context of proposed energy transitions.

Real-Time Prediction of Large-Scale Ship Model Vertical Acceleration Based on Recurrent Neural Network

In marine environments, ships are bound to be disturbed by several external factors, which can cause stochastic fluctuations and strong nonlinearity in the ship motion. Predicting ship motion is pivotal to ensuring ship safety and providing early warning of risks. This report proposes a real-time ship vertical acceleration prediction algorithm based on the long short-term memory (LSTM) and gated recurrent units (GRU) models of a recurrent neural network. The vertical acceleration time history data at the bow, middle, and stern of a large-scale ship model were obtained by performing a self-propulsion test at sea, and the original data were pre-processed by resampling and normalisation via Python. The prediction results revealed that the proposed algorithm could accurately predict the acceleration time history data of the large-scale ship model, and the root mean square error between the predicted and real values was no greater than 0.1. The optimised multivariate time series prediction program could reduce the calculation time by approximately 55% compared to that of a univariate time series prediction program, and the run time of the GRU model was better than that of the LSTM model.

Comparison between support vector machines and K-nearest neighbor for time series forecasting

This paper aims to use the Support Vector Machine (SVM) and K-Nearest Neighbor (K-NN) for univariate time series prediction. The main goal of this study is to compare between Support Vector Machine and K-Nearest Neighbor to predict time series data. The dataset for the monthly gold prices was used during the period from Nov -1989 – Dec-2019, which represents 362 observations. SVM and K-NN models were fitted based on 90% of data as training set, and then their accuracy was compared using the statistical measure RMSE. The results indicated that SVM was better than K-NN in predicting future gold prices, based on RMSE= 33.77.

Univariate Financial Time Series Prediction using Clonal Selection Algorithm

The ability to predict the financial market is beneficial not only to the individual but also to the organization and country. It is not only beneficial in terms of financial but also in terms of making a short-term and long-term decision. This paper presents an experimental study to perform univariate financial time series prediction using a clonal selection algorithm (CSA). CSA is an optimization algorithm that is based on clonal selection theory. It is a subset of the artificial immune system, a class of evolutionary algorithms inspired by the immune system of a vertebrate. Since CSA is an optimization algorithm, the univariate financial time series prediction problem was modeled into an optimization problem using a weighted regression model. CSA was used to search for the optimal set of weights for the regression model to generate prediction with the lowest error. Three data sets from the financial market were chosen for the experiments of this study namely S&P500 price, Gold price, and EUR-USD exchange rate. The performance of CSA is measured using RMSE. The value of RMSE for a problem is related to the maximum and minimum value of the data set. Therefore, the results were not compared to other data sets. Instead, it is compared to the range of values of the data sets. The result of the experiments shows that CSA can make decent predictions for financial time series despite being inferior to ARIMA. Hence, this finding implies that CSA can be implemented on a univariate financial time series prediction problem given that the problem is modeled as an optimization problem.

Medium-term wind power forecasting based on multi-resolution multi-learner ensemble and adaptive model selection

Wind energy is a potential and prospective renewable energy resource. Wind power forecasting can enhance the stability and security of power integration. This study focuses on the univariate time series prediction of wind power. A hybrid wind power forecasting model based on multi-resolution multi-learner ensemble (MRMLE) and adaptive model selection (AMS), namely the MRMLE-AMS model is proposed. The proposed model adopts heterogeneous base learners and datasets with different resolutions to guarantee diversity. According to the multi-resolution ensemble scheme, the rapid fluctuation of original high-resolution data can be effectively considered, and a long prediction time scale is also achievable. A total of 16 sub-models are developed to generate preliminary forecasting results. To make the forecasting model adaptive and fit for different datasets, butterfly optimization algorithm (BOA) is employed to select these sub-models, and minimal-redundancy-maximal-relevance (mRMR) criterion is set as optimization objective. The selected sub-models are combined by support vector regression (SVR) to obtain final forecasting result. The proposed model is thoroughly analyzed and evaluated by four actual wind power series collected from Canada. Case study shows that the proposed MRMLE-AMS model has a rational framework and significantly outperforms the existing models. Finally, its advantages and limitations are given in the conclusion.

Comparison between support vector machines and K-nearest neighbor for time series forecasting

This paper aims to use the Support Vector Machine (SVM) and K-Nearest Neighbor (K-NN) for univariate time series prediction. The main goal of this study is to compare between Support Vector Machine and K-Nearest Neighbor to predict time series data. The dataset for the monthly gold prices was used during the period from Nov -1989 – Dec-2019, which represents 362 observations. SVM and K-NN models were fitted based on 90% of data as training set, and then their accuracy was compared using the statistical measure RMSE. The results indicated that SVM was better than K-NN in predicting future gold prices, based on RMSE= 33.77.

Fast Univariate Time Series Prediction of Solar Power for Real-Time Control of Energy Storage System

In this paper, super-short-term prediction of solar power generation for applications in dynamic control of energy system has been investigated. In order to follow and satisfy the dynamics of the controller, the deployed prediction method should have a fast response time. To this end, this paper proposes fast prediction methods to provide the control system with one step ahead of solar power generation. The proposed methods are based on univariate time series prediction. That is, instead of using external data such as the weather forecast as the input of prediction algorithms, they solely rely on past values of solar power data, hence lowering the volume and acquisition time of input data. In addition, the selected algorithms are able to generate the forecast output in less than a second. The proposed methods in this paper are grounded on four well-known prediction algorithms including Autoregressive Integrated Moving Average (ARIMA), K-Nearest Neighbors (kNN), Support Vector Regression (SVR), and Random Forest (RF). The speed and accuracy of the proposed algorithms have been compared based on two different error measures, Mean Absolute Error (MAE) and Symmetric Mean Absolute Percentage Error (SMAPE). Real world data collected from the PV installation at the University of California, Riverside (UCR) are used for prediction purposes. The results show that kNN and RF have better predicting performance with respect to SMAPE and MAE criteria.