Multistep-ahead Prediction Research Articles

A Data-Driven Approach for Forecasting Current Direction With a Hybrid Model of Empirical Mode Decomposition and Warped Gaussian Process

Abstract Ocean current forecasting is essential for tidal renewable energy generation and operation. However, forecasting the current direction at multiple points along the water depth are still lacking of comprehensive studies. In this study, a data-driven approach was developed to attain short-term prediction in the current direction with reasonable uncertainty quantification. The developed approach employed empirical mode decomposition (EMD) and the warped Gaussian process (WGP) in the forecasting process. The ocean current data, which were measured by a seabed-mounted acoustic Doppler current profiler in the Haitian Strait, were used to illustrate the developed approach. The measured current direction data were preprocessed with the average shifting method to obtain the principal and random components for the improvement of the forecasting accuracy. The random components were decomposed into intrinsic mode functions (IMFs) and residuals. The principal components, IMFs, and residuals of the current direction were then forecasted by the WGP approach. The forecasting performance of the developed approach was investigated through comparisons with those of single standard GP, single WGP, and EMD + GP models. The effects of the kernel function and training input on the forecasting efficiency and precision were investigated. The extrapolation performances of the proposed model for a 1-step prediction and multistep-ahead prediction were also examined.

Mass Conservative Time-Series GAN for Synthetic Extreme Flood-Event Generation: Impact on Probabilistic Forecasting Models

The lack of data on flood events poses challenges in flood management. In this paper, we propose a novel approach to enhance flood-forecasting models by utilizing the capabilities of Generative Adversarial Networks (GANs) to generate synthetic flood events. We modified a time-series GAN by incorporating constraints related to mass conservation, energy balance, and hydraulic principles into the GAN model through appropriate regularization terms in the loss function and by using mass conservative LSTM in the generator and discriminator models. In this way, we can improve the realism and physical consistency of the generated extreme flood-event data. These constraints ensure that the synthetic flood-event data generated by the GAN adhere to fundamental hydrological principles and characteristics, enhancing the accuracy and reliability of flood-forecasting and risk-assessment applications. PCA and t-SNE are applied to provide valuable insights into the structure and distribution of the synthetic flood data, highlighting patterns, clusters, and relationships within the data. We aimed to use the generated synthetic data to supplement the original data and train probabilistic neural runoff model for forecasting multi-step ahead flood events. t-statistic was performed to compare the means of synthetic data generated by TimeGAN with the original data, and the results showed that the means of the two datasets were statistically significant at 95% level. The integration of time-series GAN-generated synthetic flood events with real data improved the robustness and accuracy of the autoencoder model, enabling more reliable predictions of extreme flood events. In the pilot study, the model trained on the augmented dataset with synthetic data from time-series GAN shows higher NSE and KGE scores of NSE = 0.838 and KGE = 0.908, compared to the NSE = 0.829 and KGE = 0.90 of the sixth hour ahead, indicating improved accuracy of 9.8% NSE in multistep-ahead predictions of extreme flood events compared to the model trained on the original data alone. The integration of synthetic training datasets in the probabilistic forecasting improves the model’s ability to achieve a reduced Prediction Interval Normalized Average Width (PINAW) for interval forecasting, yet this enhancement comes with a trade-off in the Prediction Interval Coverage Probability (PICP).

Multistep-Ahead Prediction of Logging-While-Drilling Resistivity Curves Based on Seismic-Guided Seq2Seq-Long Short-Term Memory

Summary High-temperature and high-pressure reservoirs in complex geological conditions present primary targets and significant challenges in deepsea oil and gas exploration. Limited offshore drilling operations and lack of detailed geological data hinder accurate formation pressure prediction using geoguided and offset imaging, increasing drilling hazards. Logging-while-drilling (LWD) technology provides timely and accurate subsurface information. Resistivity closely correlates with formation pressure and lithology, aiding pressure prediction. Therefore, in this study, we developed an ahead prediction workflow for LWD curves using the resistivity curve (RD) as an example. A seismic-guided sequence-to-sequence framework with the long short-term memory model (Seq2Seq-LSTM) is used to predict the RD curve at a constant depth ahead of the drill bit, utilizing the RD curve of drilled sections and seismic attributes. The network structure incorporates a direct-recursive hybrid multistep prediction strategy based on update threshold control (Dir-Rec-Update), aligning with real-time LWD data acquisition for ahead curve prediction. Using real well data cross-prediction, baseline models such as multilayer perceptron (MLP) and extreme gradient boosting (XGBoost) were compared while also investigating the impact of different configurations on the proposed Seq2Seq-LSTM. The results demonstrate that the method outperforms conventional models, with an average performance across multiple wells under a 5-m update threshold: root mean square error (RMSE) of 0.15, correlation coefficient of 0.88, and coefficient of determination of 0.77. The Seq2Seq-LSTM model and Dir-Rec-Update strategy provide accurate LWD curves ahead of the drill bit, enabling advanced drilling decisions and preventing hazards. Advanced technologies such as empirical wavelet transform (EWT) and feature selection enhance the method’s potential for curve ahead-of-time prediction.

A Meta-learner approach to multistep-ahead time series prediction

AbstractThe utilization of machine learning has become ubiquitous in addressing contemporary challenges in data science. Moreover, there has been significant interest in democratizing the decision-making process for selecting machine learning algorithms, achieved through the incorporation of meta-features and automated machine learning techniques for both classification and regression tasks. However, this paradigm has not been readily applied to multistep-ahead time series prediction problems. Unlike regression and classification problems, which utilize independent variables not derived from the target variable, time series models typically rely on past values of the series to forecast future outcomes. The structure of a time series is often characterized by features such as trend, seasonality, cyclicality and irregularity. In our study, we illustrate how time series metrics representing these features, in conjunction with an ensemble-based regression Meta-Learner, were employed to predict the standardized mean square error of candidate time series prediction models. Our experiments utilized datasets covering a broad feature space, facilitating the selection of the most effective model by researchers. A rigorous evaluation was conducted to assess the performance of the Meta-Learner on both synthetic and real time series data.

Machine learning analysis of pressure fluctuations in a gas-solid fluidized bed

Pressure fluctuations in a gas-solid fluidized bed involve much information of the dynamic system. To uncover the value and significance of the pressure fluctuations time series, two meaningful tasks, i.e., the fluidization regime classification and the future state prediction, are investigated using Machine Learning (ML) algorithms, including Back Propagation neural network (BP), Convolutional Neural Network (CNN), Long Short-Term Memory network (LSTM), Radial Basis Function network (RBF), Random Forest (RF) and Support Vector Machine (SVM). Findings indicate that the six ML methods rank their performance for the classification task from high to low as: BP ∼ RF > SVM > CNN ∼ LSTM ∼ RBF. For the one-step-ahead future state prediction, the accuracy goes from high to low as: BP ∼ RBF > CNN ∼ LSTM > SVM ∼ RF, whilst for the multistep-ahead prediction the LSTM approach shows the best performance. Additionally, the sampling frequency of pressure time series has significant effects on both tasks. Overall, this study demonstrates the capability of machine learning methods for the value analysis of nonlinear time series and the better operation of gas-solid fluidization systems.

KSLD-TNet: Key Sample Location and Distillation Transformer Network for Multistep Ahead Prediction in Industrial Processes

KSLD-TNet: Key Sample Location and Distillation Transformer Network for Multistep Ahead Prediction in Industrial Processes

Short-term solar energy forecasting: Integrated computational intelligence of LSTMs and GRU.

Problems with erroneous forecasts of electricity production from solar farms create serious operational, technological, and financial challenges to both Solar farm owners and electricity companies. Accurate prediction results are necessary for efficient spinning reserve planning as well as regulating inertia and power supply during contingency events. In this work, the impact of several climatic conditions on solar electricity generation in Amherst. Furthermore, three machine learning models using Lasso Regression, ridge Regression, ElasticNet regression, and Support Vector Regression, as well as deep learning models for time series analysis include long short-term memory, bidirectional LSTM, and gated recurrent unit along with their variants for estimating solar energy generation for every five-minute interval on Amherst weather power station. These models were evaluated using mean absolute error root means square error, mean square error, and mean absolute percentage error. It was observed that horizontal solar irradiance and water saturation deficiency had a highly proportional relationship with Solar PV electricity generation. All proposed machine learning models turned out to perform well in predicting electricity generation from the analyzed solar farm. Bi-LSTM has performed the best among all models with 0.0135, 0.0315, 0.0012, and 0.1205 values of MAE, RMSE, MSE, and MAPE, respectively. Comparison with the existing methods endorses the use of our proposed RNN variants for higher efficiency, accuracy, and robustness. Multistep-ahead solar energy prediction is also carried out by exploiting hybrids of LSTM, Bi-LSTM, and GRU.

Grey-box and ANN-based building models for multistep-ahead prediction of indoor temperature to implement model predictive control

Model-based predictive control (MPC) strategies for heating, ventilation, and air-conditioning (HVAC) systems present an opportunity to lower building energy consumption and operational costs. Such approaches rely on the development of a model to precisely forecast building thermal dynamics, such as room air temperature or heating/cooling rate, and make control-related decisions. The control-oriented modeling of building energy systems should be accurate in predicting indoor conditions and present low computational complexity. These features are the key challenge of implementing advanced control methods such as MPC. Extant studies on building modeling for MPC have focused on step-ahead forecasting techniques to forecast building thermal dynamics, while multistep-ahead forecasting is essential. Moreover, machine learning model suitable in case of the domain-based engineering expertise are also not available. To this aim, we perform a comparative analysis of the grey-box model based on a resistance-capacitance (RC) thermal network and a machine learning model composed of an artificial neural network (ANN) for multistep-ahead prediction of building thermal dynamics using current and historical data. Actual experimental data obtained from the Flexible Research Platform (FRP) in Oak Ridge National Laboratory (US) are used for estimation and validation purposes. The average root mean squared error (RMSE) of the grey-box and ANN models are 0.89 °C and 1.02 °C, respectively. The results indicate that the grey-box model outperforms the ANN model in the considered validation periods in terms of accuracy and prediction stability.

Enhancing multistep-ahead bike-sharing demand prediction with a two-stage online learning-based time-series model: insight from Seoul

Enhancing multistep-ahead bike-sharing demand prediction with a two-stage online learning-based time-series model: insight from Seoul

Simultaneous multistep transformer architecture for model predictive control

Transformer neural networks have revolutionized natural language processing by effectively addressing the vanishing gradient problem. This study focuses on applying Transformer models to time-series forecasting and customizing them for a simultaneous multistep-ahead prediction model in surrogate model predictive control (MPC). The proposed method showcases improved control performance and computational efficiency compared to LSTM-based MPC and one-step-ahead prediction models using both LSTM and Transformer networks. The study introduces three key contributions: (1) a new MPC system based on a Transformer time-series architecture, (2) a training method enabling multistep-ahead prediction for time-series machine learning models, and (3) validation of the enhanced time performance of multistep-ahead Transformer MPC compared to one-step-ahead LSTM networks. Case studies demonstrate a significant fifteen-fold improvement in computational speed compared to one-step-ahead LSTM, although this improvement may vary depending on MPC factors like the lookback window and prediction horizon.

RUMP: Resource Usage Multi-Step Prediction in Extreme Edge Computing

Extreme Edge Computing that leverages the copious yet underutilized computational resources of Extreme Edge Devices (EEDs) has gained significant momentum lately. Estimating the computational capabilities of EEDs can be strenuously challenging since EEDs are user-owned devices, and are thus subject to a highly dynamic user access behavior (i.e., dynamic resource usage). In this paper, we propose the Resource Usage Multi-step Prediction (RUMP) scheme. RUMP is the first scheme that strives to enable multistep-ahead prediction of the dynamic resource usage of EEDs (i.e., workers) in a computationally efficient way, while providing a relatively high prediction accuracy. Towards that end, RUMP exploits the use of the Hierarchical Dirichlet Process-Hidden Semi-Markov Model (HDP-HSMM). In addition, RUMP uses the Simple and Exponential Moving Average (SMA&EMA) and Savitzky-Golay (SG) filters to improve the prediction accuracy. We scrupulously study the trade-off between computational efficiency, prediction accuracy, practicality, and adaptability of the underlying prediction model by conducting complexity analysis, performing various experiments on a testbed of heterogeneous workers, and comparing the HDP-HSMM model used in RUMP to three other prominent prediction models in different dynamic resource usage scenarios. Extensive evaluations show that RUMP achieves a 91% categorical multi-step prediction accuracy and renders a small performance gap of 6% on average in terms of the Mean Absolute Error (MAE) compared to representatives of state-of-the-art prediction models, while yielding a low computational complexity.

Self-Evolving Offset-Free Model Predictive Control with Model–Plant Mismatch for Dynamic Working-Point Change Tasks in Industrial Processes

This study proposes a self-evolving offset-free model predictive control (MPC) algorithm for dynamic working-point change (DWPC) tasks in industrial processes. The algorithm mitigates disturbance impacts caused by model–plant mismatch (MPM) and enhances the dynamic performance of MPC by locating sequences (scenarios) similar to the current operational scenario from historical DWPC tasks and using them for multistep-ahead disturbance prediction. First, a disturbance-augmented state–space model guarantees the basic offset-free control behavior of MPC with MPM. Next, to enhance the MPC performance, a direct multistep-ahead disturbance prediction approach is proposed by combining historically similar DWPC task scenarios. Specifically, a dynamic autoencoder is constructed to extract private features from process scenarios and locate similar scenarios from historical DWPC tasks. Based on the located scenarios, the multistep-ahead disturbance and its uncertainty are directly predicted through multioutput Gaussian process regression. Finally, the obtained disturbance results are incorporated into the MPC framework, which continuously enhances MPC performance in DWPC tasks. Two case studies demonstrate the effectiveness of the proposed MPC.

Predicting chaotic dynamics from incomplete input via reservoir computing with (D+1)-dimension input and output.

Predicting future evolution based on incomplete information of the past is still a challenge even though data-driven machine learning approaches have been successfully applied to forecast complex nonlinear dynamics. The widely adopted reservoir computing (RC) can hardly deal with this since it usually requires complete observations of the past. In this paper, a scheme of RC with (D+1)-dimension input and output (I/O) vectors is proposed to solve this problem, i.e., the incomplete input time series or dynamical trajectories of a system, in which certain portion of states are randomly removed. In this scheme, the I/O vectors coupled to the reservoir are changed to (D+1)-dimension, where the first D dimensions store the state vector as in the conventional RC, and the additional dimension is the corresponding time interval. We have successfully applied this approach to predict the future evolution of the logistic map and Lorenz, Rössler, and Kuramoto-Sivashinsky systems, where the inputs are the dynamical trajectories with missing data. The dropoff rate dependence of the valid prediction time (VPT) is analyzed. The results show that it can make forecasting with much longer VPT when the dropoff rate θ is lower. The reason for the failure at high θ is analyzed. The predictability of our RC is determined by the complexity of the dynamical systems involved. The more complex they are, the more difficult they are to predict. Perfect reconstructions of chaotic attractors are observed. This scheme is a pretty good generalization to RC and can treat input time series with regular and irregular time intervals. It is easy to use since it does not change the basic architecture of conventional RC. Furthermore, it can make multistep-ahead prediction just by changing the time interval in the output vector into a desired value, which is superior to conventional RC that can only do one-step-ahead forecasting based on complete regular input data.

Multi-Step Ahead Prediction for Anomaly Detection of Geomagnetic Observation in HVDC Transmission

Multi-Step Ahead Prediction for Anomaly Detection of Geomagnetic Observation in HVDC Transmission

Multistep Ahead Prediction of Vehicle Lateral Dynamics Based on Echo State Model

Lateral dynamics are critical for high-level autonomous driving, and multistep forecasting can significantly improve vehicle safety. However, the dynamics prediction method is rarely announced due to the difficulty of accurate prediction. In contrast to conventional time-series prediction, this article proposes a novel method for predicting a dynamical system with implicit parameters. Specifically, we construct an echo state model (ESM) incorporating a rational data organization and an efficient neural network structure. We observe the fading information property (FIP) of dynamic vehicle parameters and use temporal data to construct equivalent states for implicit parameters. Finally, a gated recurrent unit (GRU) neural network is designed and trained to accomplish the complex mapping of a vehicle dynamical system. Its performance is compared to that of several benchmark networks. The proposed method is capable of performing four-step-ahead prediction with high accuracy in both dry and slippery road conditions, implying the increased potential for some safe and comfortable driving applications of autonomous vehicles.

A Stacked Machine Learning Algorithm for Multi-Step Ahead Prediction of Soil Moisture

A trustworthy assessment of soil moisture content plays a significant role in irrigation planning and in controlling various natural disasters such as floods, landslides, and droughts. Various machine learning models (MLMs) have been used to increase the accuracy of soil moisture content prediction. The present investigation aims to apply MLMs with novel structures for the estimation of daily volumetric soil water content, based on the stacking of the multilayer perceptron (MLP), random forest (RF), and support vector regression (SVR). Two groups of input variables were considered: the first (Model A) consisted of various meteorological variables (i.e., daily precipitation, air temperature, humidity, and wind speed), and the second (Model B) included only daily precipitation. The stacked model (SM) had the best performance (R2 = 0.962) in the prediction of daily volumetric soil water content for both categories of input variables when compared with the MLP (R2 = 0.957), RF (R2 = 0.956) and SVR (R2 = 0.951) models. Overall, the SM, which, in general, allows the weaknesses of the individual basic algorithms to be overcome while still maintaining a limited number of parameters and short calculation times, can lead to more accurate predictions of soil water content than those provided by more commonly employed MLMs.

Predicting time series by data-driven spatiotemporal information transformation

Making an accurate prediction of an unknown system only from a short-term time series is difficult due to the lack of sufficient information, especially in a multistep-ahead manner. However, a high-dimensional short-term time series still contains rich dynamical information and is increasingly available in many fields. In this work, we exploit a spatiotemporal information (STI) scheme that transforms high-dimensional/spatial information into temporal information and develop a new method called multitask Gaussian process regression machine (MT-GPRM) to achieve accurate predictions from short-term time series. We first construct a specific multitask GPR comprising multiple linked STI mappings to transform high-dimensional/spatial information into temporal/dynamical information of any given target variable and then make multistep-ahead predictions of the target variable by solving those STI mappings. The multistep-ahead prediction results on various synthetic and real-world datasets show that MT-GPRM outperforms other existing approaches.

A Hybrid Method for Prediction of Ash Fouling on Heat Transfer Surfaces

Soot blowing optimization is a key, but challenging question in the health management of coal-fired power plant boiler. The monitoring and prediction of ash fouling for heat transfer surfaces is an important way to solve this problem. This study provides a hybrid data-driven model based on advanced machine-learning techniques for ash fouling prediction. First, the cleanliness factor is utilized to represent the level of ash fouling, which is the original data from the distributed control system. The wavelet threshold denoising algorithm is employed as the data preprocessing approach. Based on the empirical mode decomposition (EMD), the denoised cleanliness factor data is decoupled into a series of intrinsic mode functions (IMFs) and a residual component. Second, the support vector regression (SVR) model is used to fit the residual, and the Gaussian process regression (GPR) model is applied to estimate the IMFs. The cleanliness factor data of ash accumulation on the heat transfer surface of diverse devices are deployed to appraise the performance of the proposed SVR + GPR model in comparison with the sole SVR, sole GPR, SVR + EDM and GPR + EDM models. The illustrative results prove that the hybrid SVR + GPR model is superior to other models and can obtain satisfactory effects both in one-step- and the multistep-ahead cleanliness factor predictions.

A Hybrid Data-Driven Approach for Multistep Ahead Prediction of State of Health and Remaining Useful Life of Lithium-Ion Batteries.

In this paper, a novel multistep ahead predictor based upon a fusion of kernel recursive least square (KRLS) and Gaussian process regression (GPR) is proposed for the accurate prediction of the state of health (SoH) and remaining useful life (RUL) of lithium-ion batteries. The empirical mode decomposition is utilized to divide the battery capacity into local regeneration (intrinsic mode functions) and global degradation (residual). The KRLS and GPR submodels are employed to track the residual and intrinsic mode functions. For RUL, the KRLS predicted residual signal is utilized. The online available experimental battery aging data are used for the evaluation of the proposed model. The comparison analysis with other methodologies (i.e., GPR, KRLS, empirical mode decomposition with GPR, and empirical mode decomposition with KRLS) reveals the distinctiveness and superiority of the proposed approach. For 1-step ahead prediction, the proposed method tracks the trajectory with the root mean square error (RMSE) of 0.2299, and the increase of only 0.2243 RMSE is noted for 30-step ahead prediction. The RUL prediction using residual signal shows an increase of 3 to 5% in accuracy. This proposed methodology is a prospective approach for an efficient battery health prognostic.

Minimizing multistep-ahead prediction error for piecewise ARX model identification

This paper presents a new approach for piecewise auto-regressive with exogenous inputs (ARX) model identification by minimizing the multistep-ahead prediction error. A piecewise ARX model can approximate a nonlinear process within a wide operating range, and thus minimizing its long-term prediction error is of importance for the successful application of model predictive control (MPC). The traditional MPC relevant identification (MRI) methods rely on tuning a noise model with structures known a priori, but its application for the piecewise ARX model is non-trivial. Instead, we design an algorithm to directly minimize the multistep-ahead prediction error by solving a mixed-integer nonlinear program (MINLP) without incorporating a noise model. The proposed solution method is tested on datasets from a simulated fermenter and a real heat exchanger, respectively. The results show that our approach yields models with smaller prediction error than the compared method on both training and testing datasets.