Data-driven Forecasting Models Research Articles

A new framework for water quality forecasting coupling causal inference, time-frequency analysis and uncertainty quantification

Accurate forecasting of water quality variables in river systems is crucial for relevant administrators to identify potential water quality degradation issues and take countermeasures promptly. However, pure data-driven forecasting models are often insufficient to deal with the highly varying periodicity of water quality in today's more complex environment. This study presents a new holistic framework for time-series forecasting of water quality parameters by combining advanced deep learning algorithms (i.e., Long Short-Term Memory (LSTM) and Informer) with causal inference, time-frequency analysis, and uncertainty quantification. The framework was demonstrated for total nitrogen (TN) forecasting in the largest artificial lakes in Asia (i.e., the Danjiangkou Reservoir, China) with six-year monitoring data from January 2017 to June 2022. The results showed that the pre-processing techniques based on causal inference and wavelet decomposition can significantly improve the performance of deep learning algorithms. Compared to the individual LSTM and Informer models, wavelet-coupled approaches diminished well the apparent forecasting errors of TN concentrations, with 24.39%, 32.68%, and 41.26% reduction at most in the average, standard deviation, and maximum values of the errors, respectively. In addition, a post-processing algorithm based on the Copula function and Bayesian theory was designed to quantify the uncertainty of predictions. With the help of this algorithm, each deterministic prediction of our model can correspond to a range of possible outputs. The 95% forecast confidence interval covered almost all the observations, which proves a measure of the reliability and robustness of the predictions. This study provides rich scientific references for applying advanced data-driven methods in time-series forecasting tasks and a practical methodological framework for water resources management and similar projects.

Simulation of reservoir outflows using regression tree and support vector machine

Water stored in reservoirs has a lot of crucial function, including generating hydropower, supporting water supply, and relieving lasting droughts. During floods, water deliveries from reservoirs must be acceptable, so as to guarantee that the gross volume of water is at a safe level and any release from reservoirs will not trigger flooding downstream. This study aims to develop a well-versed assessment method for managing reservoirs and pre-releasing water outflows by using the machine learning technology. As a new and exciting AI area, this technology is regarded as the most valuable, time-saving, supervised and cost-effective approach. In this study, two data-driven forecasting models, i.e., Regression Tree (RT) and Support Vector Machine (SVM), were employed for approximately 30 years’ hydrological records, so as to simulate reservoir outflows. The SVM and RT models were applied to the data, accurately predicting the fluctuations in the water outflows of a Bhakra reservoir. Different input combinations were used to determine the most effective release. For cross-validation, the number of folds varied. It is found that quadratic SVM for 10 folds with seven different parameters would give the minimum RMSE, maximum R2, and minimum MAE; therefore, it can be considered as the best model for the dataset used in this study.

Long-term stability and generalization of observationally-constrained stochastic data-driven models for geophysical turbulence

Abstract Recent years have seen a surge in interest in building deep learning-based fully data-driven models for weather prediction. Such deep learning models, if trained on observations can mitigate certain biases in current state-of-the-art weather models, some of which stem from inaccurate representation of subgrid-scale processes. However, these data-driven models, being over-parameterized, require a lot of training data which may not be available from reanalysis (observational data) products. Moreover, an accurate, noise-free, initial condition to start forecasting with a data-driven weather model is not available in realistic scenarios. Finally, deterministic data-driven forecasting models suffer from issues with long-term stability and unphysical climate drift, which makes these data-driven models unsuitable for computing climate statistics. Given these challenges, previous studies have tried to pre-train deep learning-based weather forecasting models on a large amount of imperfect long-term climate model simulations and then re-train them on available observational data. In this article, we propose a convolutional variational autoencoder (VAE)-based stochastic data-driven model that is pre-trained on an imperfect climate model simulation from a two-layer quasi-geostrophic flow and re-trained, using transfer learning, on a small number of noisy observations from a perfect simulation. This re-trained model then performs stochastic forecasting with a noisy initial condition sampled from the perfect simulation. We show that our ensemble-based stochastic data-driven model outperforms a baseline deterministic encoder–decoder-based convolutional model in terms of short-term skills, while remaining stable for long-term climate simulations yielding accurate climatology.

Optimized Data-Driven Models for Short-Term Electricity Price Forecasting Based on Signal Decomposition and Clustering Techniques

In recent decades, the traditional monopolistic energy exchange market has been replaced by deregulated, competitive marketplaces in which electricity may be purchased and sold at market prices like any other commodity. As a result, the deregulation of the electricity industry has produced a demand for wholesale organized marketplaces. Price predictions, which are primarily meant to establish the market clearing price, have become a significant factor to an energy company’s decision making and strategic development. Recently, the fast development of deep learning algorithms, as well as the deployment of front-end metaheuristic optimization approaches, have resulted in the efficient development of enhanced prediction models that are used for electricity price forecasting. In this paper, the development of six highly accurate, robust and optimized data-driven forecasting models in conjunction with an optimized Variational Mode Decomposition method and the K-Means clustering algorithm for short-term electricity price forecasting is proposed. In this work, we also establish an Inverted and Discrete Particle Swarm Optimization approach that is implemented for the optimization of the Variational Mode Decomposition method. The prediction of the day-ahead electricity prices is based on historical weather and price data of the deregulated Greek electricity market. The resulting forecasting outcomes are thoroughly compared in order to address which of the two proposed divide-and-conquer preprocessing approaches results in more accuracy concerning the issue of short-term electricity price forecasting. Finally, the proposed technique that produces the smallest error in the electricity price forecasting is based on Variational Mode Decomposition, which is optimized through the proposed variation of Particle Swarm Optimization, with a mean absolute percentage error value of 6.15%.

Machine-Learning-Based Near-Surface Ozone Forecasting Model with Planetary Boundary Layer Information.

Surface ozone is one of six air pollutants designated as harmful by National Ambient Air Quality Standards because it can adversely impact human health and the environment. Thus, ozone forecasting is a critical task that can help people avoid dangerously high ozone concentrations. Conventional numerical approaches, as well as data-driven forecasting approaches, have been studied for ozone forecasting. Data-driven forecasting models, in particular, have gained momentum with the introduction of machine learning advancements. We consider planetary boundary layer (PBL) height as a new input feature for data-driven ozone forecasting models. PBL has been shown to impact ozone concentrations, making it an important factor in ozone forecasts. In this paper, we investigate the effectiveness of utilization of PBL height on the performance of surface ozone forecasts. We present both surface ozone forecasting models, based on multilayer perceptron (MLP) and bidirectional long short-term memory (LSTM) models. These two models forecast hourly ozone concentrations for an upcoming 24-h period using two types of input data, such as measurement data and PBL height. We consider the predicted values of PBL height obtained from the weather research and forecasting (WRF) model, since it is difficult to gather actual PBL measurements. We evaluate two ozone forecasting models in terms of index of agreement (IOA), mean absolute error (MAE), and root mean square error (RMSE). Results showed that the MLP-based and bidirectional LSTM-based models yielded lower MAE and RMSE when considering forecasted PBL height, but there was no significant changes in IOA when compared with models in which no forecasted PBL data were used. This result suggests that utilizing forecasted PBL height can improve the forecasting performance of data-driven prediction models for surface ozone concentrations.

Solar and wind power data from the Chinese State Grid Renewable Energy Generation Forecasting Competition

Accurate solar and wind generation forecasting along with high renewable energy penetration in power grids throughout the world are crucial to the days-ahead power scheduling of energy systems. It is difficult to precisely forecast on-site power generation due to the intermittency and fluctuation characteristics of solar and wind energy. Solar and wind generation data from on-site sources are beneficial for the development of data-driven forecasting models. In this paper, an open dataset consisting of data collected from on-site renewable energy stations, including six wind farms and eight solar stations in China, is provided. Over two years (2019–2020), power generation and weather-related data were collected at 15-minute intervals. The dataset was used in the Renewable Energy Generation Forecasting Competition hosted by the Chinese State Grid in 2021. The process of data collection, data processing, and potential applications are described. The use of this dataset is promising for the development of data-driven forecasting models for renewable energy generation and the optimization of electricity demand response (DR) programs for the power grid.

Reduced-Dimension, Biophysical Neuron Models Constructed From Observed Data

Using methods from nonlinear dynamics and interpolation techniques from applied mathematics, we show how to use data alone to construct discrete time dynamical rules that forecast observed neuron properties. These data may come from simulations of a Hodgkin-Huxley (HH) neuron model or from laboratory current clamp experiments. In each case, the reduced-dimension, data-driven forecasting (DDF) models are shown to predict accurately for times after the training period. When the available observations for neuron preparations are, for example, membrane voltage V(t) only, we use the technique of time delay embedding from nonlinear dynamics to generate an appropriate space in which the full dynamics can be realized. The DDF constructions are reduced-dimension models relative to HH models as they are built on and forecast only observables such as V(t). They do not require detailed specification of ion channels, their gating variables, and the many parameters that accompany an HH model for laboratory measurements, yet all of this important information is encoded in the DDF model. As the DDF models use and forecast only voltage data, they can be used in building networks with biophysical connections. Both gap junction connections and ligand gated synaptic connections among neurons involve presynaptic voltages and induce postsynaptic voltage response. Biophysically based DDF neuron models can replace other reduced-dimension neuron models, say, of the integrate-and-fire type, in developing and analyzing large networks of neurons. When one does have detailed HH model neurons for network components, a reduced-dimension DDF realization of the HH voltage dynamics may be used in network computations to achieve computational efficiency and the exploration of larger biological networks.

Deep Federated Adaptation: An Adaptative Residential Load Forecasting Approach with Federated Learning.

Residential-level short-term load forecasting (STLF) is significant for power system operation. Data-driven forecasting models, especially machine-learning-based models, are sensitive to the amount of data. However, privacy and security concerns raised by supervision departments and users limit the data for sharing. Meanwhile, the limited data from the newly built houses are not sufficient to support building a powerful model. Another problem is that the data from different houses are in a non-identical and independent distribution (non-IID), which makes the general model fail in predicting accurate load for the specific house. Even though we can build a model corresponding to each house, it costs a large computation time. We first propose a federated transfer learning approach applied in STLF, deep federated adaptation (DFA), to deal with the aforementioned problems. This approach adopts the federated learning architecture to train a global model without undermining privacy, and then the model leverage multiple kernel variant of maximum mean discrepancies (MK-MMD) to fine-tune the global model, which makes the model adapted to the specific house’s prediction task. Experimental results on the real residential datasets show that DFA has the best forecasting performance compared with other baseline models and the federated architecture of DFA has a remarkable superiority in computation time. The framework of DFA is extended with alternative transfer learning methods and all of them achieve good performances on STLF.

An improved pollution forecasting model with meteorological impact using multiple imputation and fine-tuning approach

Air pollution forecasting is a significant step for air quality pollution management to mitigate pollution’s negative impact on the environment and people’s health. The data-driven forecasting model can help a better understanding of environmental air quality. The existing data-driven forecasting models usually ignore missing values, the correlations between the pollutant and meteorological factors and fail to perform temporal modeling effectively, affecting prediction accuracy. In response to these issues, we present a deep learning-based Convolutional LSTM–SDAE (CLS) model to forecast the particulate matter level, revealing the correlation between particulate matter and meteorological factors. In the proposed architecture, the k nearest neighbor (KNN) imputation technique is employed to recover the air quality dataset’s missing values. The Convolutional Long Short Term Memory (CNN–LSTM) unit identifies the vast dataset’s hidden features and performs pollutants’ temporal modeling. In addition, Bidirectional Gatted Recurrent Unit (BIGRU) is implemented as both encoder and decoder in Sparse Denoising Autoencoder, which reconstructs the CNN–LSTM model’s output in the dynamic fine-tuning layer to get robust prediction results. The experimental results in Talcher, India, and Beijing, China indicate that the model can improve forecasting accuracy and outperforms the other state of art and baseline models.

A survey of human judgement and quantitative forecasting methods.

This paper's top-level goal is to provide an overview of research conducted in the many academic domains concerned with forecasting. By providing a summary encompassing these domains, this survey connects them, establishing a common ground for future discussions. To this end, we survey literature on human judgement and quantitative forecasting as well as hybrid methods that involve both humans and algorithmic approaches. The survey starts with key search terms that identified more than 280 publications in the fields of computer science, operations research, risk analysis, decision science, psychology and forecasting. Results show an almost 10-fold increase in the application-focused forecasting literature between the 1990s and the current decade, with a clear rise of quantitative, data-driven forecasting models. Comparative studies of quantitative methods and human judgement show that (1) neither method is universally superior, and (2) the better method varies as a function of factors such as availability, quality, extent and format of data, suggesting that (3) the two approaches can complement each other to yield more accurate and resilient models. We also identify four research thrusts in the human/machine-forecasting literature: (i) the choice of the appropriate quantitative model, (ii) the nature of the interaction between quantitative models and human judgement, (iii) the training and incentivization of human forecasters, and (iv) the combination of multiple forecasts (both algorithmic and human) into one. This review surveys current research in all four areas and argues that future research in the field of human/machine forecasting needs to consider all of them when investigating predictive performance. We also address some of the ethical dilemmas that might arise due to the combination of quantitative models with human judgement.

Time Series Analysis and Forecasting Using a Novel Hybrid LSTM Data-Driven Model Based on Empirical Wavelet Transform Applied to Total Column of Ozone at Buenos Aires, Argentina (1966–2017)

Total column of ozone (TCO) time series analysis and accurate forecasting is of great significance in monitoring the status of the Chapman Mechanism in the stratosphere, which prevents harmful UV radiation from reaching the Earth’s surface. In this study, we performed a detailed time series analysis of the TCO data measured in Buenos Aires, Argentina. Moreover, hybrid data-driven forecasting models, based on long short-term memory networks (LSTM) recurrent neural networks (RNNs), are developed. We extracted the updated trend of the TCO time series by utilizing the singular spectrum analysis (SSA), empirical wavelet transform (EWT), empirical mode decomposition (EMD), and Mann-Kendall. In general, the TCO has been stable since the mid-1990s. The trend analysis shows that there is a recovery of ozone during the period from 2010 to 2017, apart from the decline of ozone observed during 2015, which is presumably associated with the Calbuco volcanic event. The EWT trend method seems to have effective power for trend identification, compared with others. In this study, we developed a robust data-driven hybrid time series-forecasting model (named EWT-LSTM) for the TCO time series forecasting. Our model has the advantage of utilizing the EWT technique in the decomposition stage of the LSTM process. We compared our model with (1) an LSTM model that uses EMD, namely EMD-LSTM; (2) an LSTM model that uses wavelet denoising (WD) (WD-LSTM); (3) a wavelet denoising EWT-LSTM (WD-EWT-LSTM); and (4) a wavelet denoising noise-reducing sequence called EMD-LSTM (WD-EMD-LSTM). The model that uses the EWT decomposition process (EWT-LSTM) outperformed the other five models developed here in terms of various forecasting performance evaluation criteria, such as the root mean square error (RMSE), mean absolute error (MAE), mean absolute percentage error (MAPE), and correlation coefficient (R).

Adaptive Fast Orthogonal Search (FOS) algorithm for forecasting streamflow

Data-driven models for streamflow forecasting have attracted considerable attention, as they are independent of physical system features. The physical features of the river basin are extremely hard to collect, especially for large rivers. Empirical data-driven models, such as stochastic and regression models, have been widely used in the field of streamflow forecasting. However, they suffered limited accuracy in predicting extreme streamflow. They also required raw data pre-processing prior to the modeling process, especially for lengthy data records and for large time-scale increments (e.g. monthly resolution). To overcome these challenges, data-driven forecasting models based on Artificial Intelligence (AI) have been widely used and resulted in enhancing the forecasting accuracy. Nevertheless, AI-based models required augmentation with proper optimization schemes to adjust the model parameters for optimal accuracy. Furthermore, in some cases, due to unsuitability of the optimization model, there is high possibility for overfitting of the AI model, which might cause poor prediction of input patterns that were not adequately mimicked. This study introduces a new approach to streamflow forecasting based on nonlinear system identification. The proposed technique employs Fast Orthogonal Search (FOS) to develop a nonlinear model of stream flow. The main advantage of using FOS is eliminating the requirement of raw data pre-processing and the need for an optimization scheme for model parameter adjustment since the FOS algorithm takes this into account while building the model. In addition, the FOS algorithm includes a pole-zero cancellation procedure that can detect and avoid the over-fitted models. The FOS-based nonlinear modeling approach was adopted in this research for the development of a streamflow forecasting model at Aswan High Dam using monthly basis natural streamflow records for 130 years. The results indicated that the proposed FOS algorithm outperformed the previously developed AI models of streamflow forecasting for large data records and for large time-scale increment (monthly resolution).

Data-Driven Short-Term Solar Irradiance Forecasting Based on Information of Neighboring Sites

Short-term forecasting of solar irradiance at a targeted site with consideration of its time series and measurements at neighboring sites is studied in this paper. A data-driven framework for forecasting solar irradiance based on fusing spatial and temporal information is proposed. In the framework, data-driven approaches including boosted regression trees (BRT), artificial neural network, support vector machine, and least absolute shrinkage and selection operator are applied to model spatial dependence among solar irradiance time series thereby to generate forecasts of solar irradiance at the targeted site. A comprehensive comparison among data-driven forecasting models is performed using 30-min averaged data of recent two years. Moreover, benchmarking models including scaled persistence model, autoregressive model, and autoregressive exogenous model are employed to further validate the effectiveness of data-driven forecasting models. Computational results of multiple-steps ahead forecasting demonstrate that the BRT model gives the best performance with the lowest normalized root mean squared error of 18.4%, 24.3%, 27.9%, and 30.6% for forecasting horizons of 30, 60, 90, and 120-min, respectively.

Adaptive Power Transformer Lifetime Predictions Through Machine Learning and Uncertainty Modeling in Nuclear Power Plants

The remaining useful life (RUL) of transformer insulation paper is largely determined by the winding hot-spot temperature (HST). Frequently the HST is not directly monitored and it is inferred from other measurements. However, measurement errors affect prediction models and if uncertain variables are not taken into account this can lead to incorrect maintenance decisions. Additionally, existing analytic models for HST calculation are not always accurate because they cannot generalize the properties of transformers operating in different contexts. In this context, this paper presents a novel transformer condition assessment approach integrating uncertainty modeling, data-driven forecasting models, and model-based experimental models to increase the prediction accuracy and handle uncertainty. The proposed approach quantifies the effect of measurement errors on transformer RUL predictions and confirms that temperature and load measurement errors affect the RUL estimation. Forecasting results show that the extreme gradient boosting (XGB) algorithm best captures the nonlinearities of the thermal model and improves the prediction accuracy among a number of forecasting approaches. Accordingly, the XGB model is integrated with experimental models in a particle filtering framework to improve thermal modeling and RUL prediction tasks. Models are tested and validated using a real dataset from a power transformer operating in a nuclear power plant.

Exploring geometrical patterns in streamflow time series: utility for forecasting?

Abstract Streamflow time series often provide valuable insights into the underlying physical processes that govern responses of any watershed to storm events. Patterns derived from time series based on repeated structures within these series can be beneficial for developing new or improved data-driven forecasting models. Data-driven models, artificial neural networks (ANN), are developed in the current study for streamflow prediction using input structures that are classified by geometrically similar patterns. A new modular and integrated ANN architecture that combines multiple ANN models, referred to as pattern-classified neural network (PCNN), is proposed, developed and investigated in this study. The PCNN relies on the development of several independent local models instead of one global data-driven prediction model. The PCNN models are evaluated for one step-ahead prediction of daily streamflows for Reed Creek and Little River, Virginia, and Elkhorn Creek, Kentucky in the United States. Results obtained from this study suggest that the use of these patterns has improved the performance of the neural networks in prediction. The improved performance of the PCNN models can be attributed to prior classification of data benefiting generalization abilities. PCNN model outputs can also provide an ensemble of forecasts that help quantify forecast uncertainty.

Concept and benchmark results for Big Data energy forecasting based on Apache Spark

The present article describes a concept for the creation and application of energy forecasting models in a distributed environment. Additionally, a benchmark comparing the time required for the training and application of data-driven forecasting models on a single computer and a computing cluster is presented. This comparison is based on a simulated dataset and both R and Apache Spark are used. Furthermore, the obtained results show certain points in which the utilization of distributed computing based on Spark may be advantageous.

A new measure for assessing the efficiency of hydrological data-driven forecasting models

There is a lack of consistency and generality in assessing the performance of hydrological data-driven forecasting models, and this paper presents a new measure for evaluating that performance. Despite the fact that the objectives of hydrological data-driven forecasting models differ from those of the conventional hydrological simulation models, criteria designed to evaluate the latter models have been used until now to assess the performance of the former. Thus, the objectives of this paper are, firstly, to examine the limitations in applying conventional methods for evaluating the data-driven forecasting model performance, and, secondly, to present new performance evaluation methods that can be used to evaluate hydrological data-driven forecasting models with consistency and objectivity. The relative correlation coefficient (RCC) is used to estimate the forecasting efficiency relative to the naïve model (unchanged situation) in data-driven forecasting. A case study with 12 artificial data sets was performed to assess the evaluation measures of Persistence Index (PI), Nash-Sutcliffe coefficient of efficiency (NSC) and RCC. In particular, for six of the data sets with strong persistence and autocorrelation coefficients of 0.966–0.713 at correlation coefficients of 0.977–0.989, the PIs varied markedly from 0.368 to 0.930 and the NSCs were almost constant in the range 0.943–0.972, irrespective of the autocorrelation coefficients and correlation coefficients. However, the RCCs represented an increase of forecasting efficiency from 2.1% to 37.8% according to the persistence. The study results show that RCC is more useful than conventional evaluation methods as the latter do not provide a metric rating of model improvement relative to naïve models in data-driven forecasting. Editor D. Koutsoyiannis, Associate editor D. Yang Citation Hwang, S.H., Ham, D.H., and Kim, J.H., 2012. A new measure for assessing the efficiency of hydrological data-driven forecasting models. Hydrological Sciences Journal, 57 (7), 1257–1274.

Artificial neural network for tsunami forecasting

This paper presents a data-driven approach for effective and efficient forecasting of tsunami generated by underwater earthquakes. Based on pre-computed tsunami scenarios as training data sets the Artificial Neural Network (ANN) is used for the construction of data-driven forecasting models. The training data comprised spatial values of maximum tsunami heights and tsunami arrival times (snapshots), computed with process-based TUNAMI-N2-NUS model for the most probable ocean floor rupture scenarios. Validation tests demonstrated that with a given earthquake size and location, the ANN method provides accurate and near instantaneous forecasting of the maximum tsunami heights and arrival times for the entire computational domain.

Neural network approach for modeling the performance of reverse osmosis membrane desalting

A neural network-based modeling approach with back-propagation and support vector regression algorithms was investigated as a mean of developing data-driven models for forecasting reverse osmosis (RO) plant performance and for potential use for operational diagnostics. The concept of plant “short-term memory” time-interval was introduced to capture the time-variability of plant performance since both a state of the plant model and standard time-series analyses for both flux decline and salt passage did not result in realistic predictive horizons for practical purposes. Past information of normalized permeate flux and salt passage were introduced as unique input variables along with process operating parameters to capture short-term plant performance variability. Sequential models, where the time-variation within each forecasting time-interval was also taken as input information, and marching forecasting models, where target values were predicted at fixed future times from past plant information, were developed. Models were trained, with normalized permeate flux and salt passage, for various model architectures, memory time-intervals and forecasting times using both back-propagation and support vector regression approaches. State of the plant models (without forecasting) were able to describe the relatively small permeate flux variations but were unable to capture salt passage trends (for any present time condition) since unsteady state phenomena could not be properly described without plant memory information. Forecasting of plant performance, with both sequential and marching models, yielded good predictive accuracy for short-term memory time-intervals in the range of 8–24 h for permeate flux and salt passage for forecasting times up to 24 h. Current work is ongoing to extend the approach for longer time scales and to incorporate data-driven forecasting models of RO plant into control strategies and process diagnostics.

Baseflow separation techniques for modular artificial neural network modelling in flow forecasting

In hydrological sciences there is an increasing tendency to explore and improve artificial neural network (ANN) and other data-driven forecasting models. Attempts to improve such models relate, to a large extent, to the recognized problems of their physical interpretation. The present paper deals with the problem of incorporating hydrological knowledge into the modelling process through the use of a modular architecture that takes into account the existence of various flow regimes. Three different partitioning schemes were employed: automatic classification based on clustering, temporal segmentation of the hydrograph based on an adapted baseflow separation technique, and an optimized baseflow separation filter. Three different model performance measures were analysed. Three case studies were considered. The modular models incorporating hydrological knowledge were shown to be more accurate than the traditional ANN-based models.