Reliable Prediction Intervals Research Articles

The impact of natural constraints in linear regression of log transformed response variables

Abstract In linear regression, log transforming the response variable is the usual workaround regarding departures from the assumption of normality. However, the response variable is often subject to natural constraints, which can result in a truncated distribution of the residual errors on the log scale. In forestry, allometric relationships and tree growth are two typical examples a natural constraint; the response variable cannot be negative. Traditional least squares estimators do not account for constrained response variables. For this study, a modified maximum likelihood (MML) estimator that takes natural constraints into account was developed. This estimator was tested through a simulation study and showcased with black spruce tree diameter increment data. Results show that the ordinary least squares estimator underestimated large conditional expectations of the response variable on the original scale. In contrast, the MML estimator showed no evidence of bias for large sample sizes. Departures from distributional assumptions cannot be overlooked when the model is used for predictive purposes. Both Monte Carlo error propagation and prediction intervals rely on these assumptions. In this context, the MML estimator developed for this study can be used to properly propagate the errors and produce reliable prediction intervals.

A parsimonious, computationally efficient machine learning method for spatial regression

AbstractWe introduce the modified planar rotator method (MPRS), a physically inspired machine learning method for spatial/temporal regression. MPRS is a non-parametric model which incorporates spatial or temporal correlations via short-range, distance-dependent “interactions” without assuming a specific form for the underlying probability distribution. Predictions are obtained by means of a fully autonomous learning algorithm which employs equilibrium conditional Monte Carlo simulations. MPRS is able to handle scattered data and arbitrary spatial dimensions. We report tests on various synthetic and real-word data in one, two and three dimensions which demonstrate that the MPRS prediction performance (without hyperparameter tuning) is competitive with standard interpolation methods such as ordinary kriging and inverse distance weighting. MPRS is a particularly effective gap-filling method for rough and non-Gaussian data (e.g., daily precipitation time series). MPRS shows superior computational efficiency and scalability for large samples. Massive datasets involving millions of nodes can be processed in a few seconds on a standard personal computer. We also present evidence that MPRS, by avoiding the Gaussian assumption, provides more reliable prediction intervals than kriging for highly skewed distributions.

AF-OS-ELM-MVE: A new online sequential extreme learning machine of dam safety monitoring model for structure deformation estimation

Dam deformation monitoring is an important task in hydraulic engineering projects. Traditional statistical methods and machine learning algorithms have been widely applied in structural safety prediction due to the application’s convenience. However, most machine learning algorithms are offline models and are not suitable for simulating the complex dynamic process of dam displacement. In addition, dam safety monitoring should quantify the uncertainty of the time-dependent displacement and establish reliable prediction intervals. As a result, an online learning model that can qualify prediction intervals is necessary. In this paper, an online learning algorithm named AF-OS-ELM-MVE is proposed to solve the problems. It is based on the OS-ELM algorithm and optimized using the adaptive forgetting factor mechanism. Through selective forgetting for parameter tuning, it achieves self-adaptation to the underlying physical mechanism between the factors and the displacement data. Mean Variance Estimation (MVE) is also adapted to establish the new hybrid model. It can be used to estimate the variance of the prediction process, and the model can generate prediction intervals based on the minimum probability theory. The AF-OS-ELM-MVE model is verified using long-term monitoring data and the performance comparisons were made with a number of state-of-the-art models. The correlation coefficients of the proposed model in point prediction are above 0.99 for measuring points of different elevations, and at a 95% confidence level, it achieves 100% coverage of real displacements. The results demonstrate the effectiveness of the proposed model in predicting dam deformations and provide a more solid basis for dam safety analysis.

Novel deterministic and probabilistic forecasting methods for crude oil price employing optimized deep learning, statistical and hybrid models

In this paper, individual and hybrid methods are proposed employing optimized statistical and deep learning (DL) models for deterministic (point) and probabilistic (interval) forecasting of crude oil price time series. The statistical models are optimized using the Forecast package of R. To enhance the performance of DL models, a novel pruning DE-DL method is proposed, which employs the differential evolution (DE) algorithm to optimize architecture and continuous and discrete-valued hyper-parameters. The proposed DE-DL method is so generic that it can be applied to optimize different DL models for any supervised learning problem. Five DL models (LSTM, BiLSTM, GRU, CNN, and ConvLSTM) are optimized for forecasting monthly crude oil prices and hybridized with an optimized ARIMA model for developing optimized additive and multiplicative hybrid forecasting models. The effectiveness of the proposed methods is evaluated through deterministic and probabilistic forecasting measures, comparing the results with six optimized statistical models, thirteen machine learning models, five optimized DL models, and ten optimized hybrid models. It is observed from the simulation results that the proposed optimized Additive-ARIMA-GRU hybrid model provides statistically superior forecasts, and the t Location Scale distribution is more suitable than the Gaussian distribution for computing reliable prediction intervals with different significance levels.

Reliable prediction intervals with directly optimized inductive conformal regression for deep learning

By generating prediction intervals (PIs) to quantify the uncertainty of each prediction in deep learning regression, the risk of wrong predictions can be effectively controlled. High-quality PIs need to be as narrow as possible, whilst covering a preset proportion of real labels. At present, many approaches to improve the quality of PIs can effectively reduce the width of PIs, but they do not ensure that enough real labels are captured. Inductive Conformal Predictor (ICP) is an algorithm that can generate effective PIs which is theoretically guaranteed to cover a preset proportion of data. However, typically ICP is not directly optimized to yield minimal PI width. In this study, we propose Directly Optimized Inductive Conformal Regression (DOICR) for neural networks that takes only the average width of PIs as the loss function and increases the quality of PIs through an optimized scheme, under the validity condition that sufficient real labels are captured in the PIs. Benchmark experiments show that DOICR outperforms current state-of-the-art algorithms for regression problems using underlying Deep Neural Network structures for both tabular and image data.

Tropical Cyclone Intensity Probabilistic Forecasting System Based on Deep Learning

Tropical cyclones (TC) are one of the extreme disasters that have the most significant impact on human beings. Unfortunately, intensity forecasting of TC has been a difficult and bottleneck in weather forecasting. Recently, deep learning-based intensity forecasting of TC has shown the potential to surpass traditional methods. However, due to the Earth system’s complexity, nonlinearity, and chaotic effects, there is inherent uncertainty in weather forecasting. Besides, previous studies have not quantified the uncertainty, which is necessary for decision-making and risk assessment. This study proposes an intelligent system based on deep learning, PTCIF, to quantify this uncertainty based on multimodal meteorological data, which, to our knowledge, is the first study to assess the uncertainty of TC based on a deep learning approach. In this study, probabilistic forecasts are made for the intensity of 6–24 hours. Experimental results show that our proposed method is comparable to the forecast performance of weather forecast centers in terms of deterministic forecasts. Moreover, reliable prediction intervals and probabilistic forecasts can be obtained, which is vital for disaster warning and is expected to be a complement to operational models.

A deep-learning wind speed interval forecasting architecture based on modified scaling approach with feature ranking and two-output gated recurrent unit

Wind power is already an indispensable part of the power supply worldwide, for which wind speed forecasting is technically required as essential work to guarantee its safe and economical generation. To date, the majority of wind speed forecasting models only provide deterministic predictions, which, however, fails in accounting for the uncertainty associated with the wind speed that is critical to practice; the current few models attempting to use interval forecasting to address the uncertainty are always plagued by rigorous hypotheses along with the used interval construction approaches and poor interval quality. To fill the gap, a novel interval forecasting architecture has been developed to better capture the uncertainty. It comprises a modified scaling approach, a two-output Gated Recurrent Unit (GRU), and the efficient feature ranking (FR), getting rid of rigorous hypotheses and constructing reliable prediction intervals. The case study based on real wind farm data sheds light on the advantage of the modified scaling approach, as it outperforms many benchmark approaches and conventional scaling approaches by enhancing the coverage width criterion by 4.07% to 97.76% on average. Further embedding the FR and the deep-learning GRU helps in further narrowing the interval width and reducing the interval deviation.

Wind Turbine Power Curve Monitoring Based on Environmental and Operational Data

The power produced by a wind turbine depends on environmental conditions, working parameters, and interactions with nearby turbines. However, these aspects are often neglected in the design of data-driven models for wind farms’ performance analysis. In this article, we propose to predict the active power and to provide reliable prediction intervals via ensembles of multivariate polynomial regression models that exploit a higher number of inputs (compared to most approaches in the literature), including operational and thermal variables. We present two main strategies: the former considers the environmental measurements collected at the other wind turbines in the farm as additional modeling information for the turbine under analysis; the latter combines multiple models relative to different operative conditions. We validate our approach on real data from the SCADA system of a wind farm in Italy and obtain a MAE of the order of 1.0% of the rated power of the turbine. Moreover, due to the structure of our approach, we can gain quantitative insights on the covariates most frequently selected depending on the working region of the wind turbines.

A deep asymmetric Laplace neural network for deterministic and probabilistic wind power forecasting

Accurate forecasting of wind power faces two challenges: 1) extracting more effective information on power fluctuations from limited input features, and 2) constructing a suitable loss function for model training. This paper proposes a novel deep asymmetric Laplace neural network named AL-MCNN-BiLSTM for wind power forecasting. The maximal information coefficient, which can describe the linear and nonlinear relationships between targeted and historical wind power data, is employed to determine the optimal inputs. Then, a novel multi-convolutional neural network (MCNN) is designed with a multi-scale information fusion block, which helps make full use of the multi-scale information in different convolutional layers. The MCNN extracts local information from inputs, then bidirectional long-short-term memory (BiLSTM) is employed to extract temporal information. An asymmetric Laplace distribution is assumed to characterize the uncertainty in wind power forecasts, such that an asymmetric Laplace-based loss function can be used in the model. The forecasting results on four datasets demonstrate that AL-MCNN-BiLSTM not only generates more precise deterministic wind power forecasts with a maximum coefficient of determination of 0.9803, but also produces more reliable prediction intervals at 85%, 90%, 95%, and 99% confidence levels with minimum values of pinball loss reaching 6.8948, 5.1895, 3.0189, and 0.7645, respectively.

Machine Learning Forecasts to Reduce Risk of Entrainment Loss of Endangered Salmonids at Large-Scale Water Diversions in the Sacramento–San Joaquin Delta, California

Incidental entrainment of fishes at large-scale state and federal water diversion facilities in the Sacramento-San Joaquin Delta, California, can trigger protective management actions when limits imposed by environmental regulations are approached or exceeded. These actions can result in substantial economic costs, and likewise they can affect the status of vulnerable species. Here, we examine data relevant to water management actions during January–June; the period when juvenile salmonids are present in the Delta. We use a quantile regression forest approach to create a risk forecasting tool, which can inform adjustments of diversions based on near real-time predictions. Models were trained using historical entrainment data (Water Years 1999–2019) for Sacramento River winter-run Chinook Salmon or Central Valley Steelhead and a suite of environmental and water operations metrics. A range of models was developed; their performance was evaluated by comparison of a quantile loss metric. The models were validated through examination of partial dependence plots, cross-validation procedures, and further evaluated through WY 2019 pilot testing, which integrated real-world uncertainty in environmental parameters into model predictions. For both species, the strongest predictor of loss was the previous week’s entrainment loss. In addition, risk increased with higher water exports and more negative Old and Middle Rivers (OMR) flows. Point estimates of loss were modestly correlated with observations (R2 0.4 to 0.6), but the use of a quantile regression approach provided reliable prediction intervals. For both species, the predicted 75th quantile appears to be a robust and conservative estimator of entrainment risk, with overprediction occurring in fewer than 20% of cases. This quantile balances the magnitude of over- and under-prediction and results in a low probability (< 5% of predictions) of unexpected high-take events. These models, and the web-based application through which they are made accessible to non-technical users, can provide a useful and complementary approach to the current system of managing entrainment risk.

Applying regression conformal prediction with nearest neighbors to time series data

In this paper, we apply conformal prediction to time series data. Conformal prediction is a method that produces predictive regions given a confidence level. The regions outputs are always valid under the exchangeability assumption. However, this assumption does not hold for the time series data because there is a link among past, current, and future observations. Consequently, the challenge of applying conformal predictors to the problem of time series data lies in the fact that observations of a time series are dependent and therefore do not meet the exchangeability assumption. This paper aims to present a way of constructing reliable prediction intervals by using conformal predictors in the context of time series. We use the nearest neighbors method based on the fast parameters tuning technique in the weighted nearest neighbors (FPTO–WNN) approach as the underlying algorithm. Data analysis demonstrates the effectiveness of the proposed approach.

Multi-quantile recurrent neural network for feeder-level probabilistic energy disaggregation considering roof-top solar energy

The purpose of feeder-level energy disaggregation is to decouple the net load measured at the feeder-head into various components. This technology is vital for power system utilities since increased visibility of controllable loads enables the realization of demand-side management strategies. However, energy disaggregation at the feeder level is difficult to realize since the high penetration of embedded generation masks the actual demand and different loads are highly aggregated. In this paper, the solar energy at the grid supply point is separated from the net load at first via either an unsupervised upscaling method or the supervised gradient boosting regression tree (GBRT) method. To deal with the uncertainty of the load components, the probabilistic energy disaggregation models based on multi-quantile recurrent neural network model (multi-quantile long short-term memory (MQ-LSTM) model and multi-quantile gated recurrent unit (MQ-GRU) model) are proposed to disaggregate the demand load into thermostatically controlled loads (TCLs), non-thermostatically controlled loads (non-TCLs), and non-controllable loads. A variety of relevant information, including feeder measurements, meteorological measurements, calendar information, is adopted as the input features of the model. Instead of providing point prediction, the probabilistic model estimates the conditional quantiles and provides prediction intervals. A comprehensive case study is implemented to compare the proposed model with other state-of-the-art models (multi-quantile convolutional neural network (MQ-CNN), quantile gradient boosting regression tree (Q-GBRT), Quantile Light gradient boosting machine (Q-LGB)) from training time, reliability, sharpness, and overall performance aspects. The result shows that the MQ-LSTM can estimate reliable and sharp Prediction Intervals for target load components. And it shows the best performance among all algorithms with the shortest training time. Finally, a transfer learning algorithm is proposed to overcome the difficulty to obtain enough training data, and the model is pre-trained via synthetic data generated from a public database and then tested on the local dataset. The result confirms that the proposed energy disaggregation model is transferable and can be applied to other feeders easily.

A data-driven approach based on quantile regression forest to forecast cooling load for commercial buildings

• Data-driven approach based on quantile regression forest for forecasting cooling load. • Input variables selection by applying mutual information and recursive feature elimination. • Uncertainty quantification by computing Prediction Intervals (PIs). Reliable prediction of thermal load is essential for implementing an efficient and economic energy management plan in commercial buildings. While previous research has been concerned with point forecasts , in this study we focus on forecasting prediction intervals for building thermal load. Prediction Intervals (PIs) are more useful than point forecasts as they can quantify the uncertainty associated with energy consumption in the buildings and enable the market participants to exploit current energy market benefits more effectively. We present a data-driven approach for forecasting PIs for building cooling load which first uses machine learning feature selection methods to identify a small but informative set of variables. It then applies quantile regression forest as the prediction algorithm that uses the selected features as inputs to compute the upper and lower boundaries of PIs. We evaluate the performance of the proposed approach using real data sets from two commercial buildings: a large shopping centre and an office building. The results show that the proposed approach can generate narrow and reliable PIs while satisfying the pre-specified coverage probabilities. The approach is fast to train and significantly outperforms traditional Gradient Boosting Regression (GBR) model in terms of reliability of the generated PIs.

Allometric Models for Estimating Aboveground Biomass in Short Rotation Crops of Acacia Species in Two Different Sites in Chile

We evaluated the ability of different allometric models to estimate the biomass production of short-rotation woody crops of Acacia dealbata, A. mearnsii and A. melanoxylon. Models considered the adjustment and validation of biomass functions and biological restrictions, such as the use of additive components of the biomass (stem, branches, and leaves). Adjustments of linear and nonlinear models of the three acacia species—established in two locations and of three densities in southern Chile—were utilized. Systems of equations were adjusted to guarantee the addition of the biomass components and the trees’ total biomass. The selection of models was performed based on their goodness of fit and predictive quality. Methods that accounted for the correlation between biomass components granted an additively consistent equations system with efficient estimates and reliable prediction intervals.

POINT AND INTERVAL FORECASTS OF DEATH RATES USING NEURAL NETWORKS

AbstractThe Lee–Carter model has become a benchmark in stochastic mortality modeling. However, its forecasting performance can be significantly improved upon by modern machine learning techniques. We propose a convolutional neural network (NN) architecture for mortality rate forecasting, empirically compare this model as well as other NN models to the Lee–Carter model and find that lower forecast errors are achievable for many countries in the Human Mortality Database. We provide details on the errors and forecasts of our model to make it more understandable and, thus, more trustworthy. As NN by default only yield point estimates, previous works applying them to mortality modeling have not investigated prediction uncertainty. We address this gap in the literature by implementing a bootstrapping-based technique and demonstrate that it yields highly reliable prediction intervals for our NN model.

Clustering model responses in the frequency space for improved simulation‐based flood risk studies: The role of a cluster number

AbstractHydrologic models are often employed in flood risk studies to simulate possible hydrologic responses. They are, however, linked with uncertainty that is commonly represented with uncertainty intervals constructed based on a simulation ensemble. This work adapts an alternative clustering‐based approach to first, learn about hydrological responses in the frequency space, and second, select an optimal number of clusters and corresponding representative parameters sets for a hydrologic model. Each cluster is described with three parameter sets, which enable percentile and prediction intervals to be constructed. Based on a small Swiss catchment with 10,000 years of daily pseudo‐discharge simulations, it was found that clustering the ensemble of 1000 members into 5–7 groups is optimal to derive reliable flood prediction intervals in the frequency space. This lowers the computational costs of using a hydrological model by 98%. The developed approach is suitable for probabilistic flood risk analysis with current or future climate conditions to assess hydrologic changes.

Point and interval forecasting of electricity supply via pruned ensembles

This paper develops a new ensemble-based approach to point and interval forecasting, and focus on total electricity supply. The proposed approach combines Bootstrap Aggregation (Bagging), time series methods and a novel pruning routine that performs feature selection before the aggregation of forecasts. Monthly time series of the total electricity supplied between January 2000 and September 2020 in 16 countries are considered. Forecasting performance in different horizons is examined. As the data includes the COVID-19 pandemic that affected countries in different ways, with some visible changes in electricity demand, the likely impact of unusual observations on this proposal is also examined. A comparative, multi-step-ahead forecasting with out-of-sample evaluation is conducted using several forecasting accuracy metrics and detailed robustness checks. The results endorse the strength and resilience of the proposed approach in delivering not only accurate point forecasts, but also reliable prediction intervals under different economic settings. Moreover, the methodology presented herein is flexible, in the sense that it can be used to generate reliable point and interval forecasts for any time series in short and medium horizons.

Point and Interval Forecasts of Death Rates Using Neural Networks

The Lee-Carter model has become a benchmark in stochastic mortality modeling. However, its forecasting performance can be significantly improved upon by modern machine learning techniques. We propose a convolutional neural network architecture for mortality rate forecasting, empirically compare this model as well as other neural network models to the Lee-Carter model and find that lower forecast errors are achievable for many countries in the Human Mortality Database. We provide details on the errors, forecasts and global behavior of our model to make it more understandable and, thus, more trustworthy. As neural networks by default only yield point estimates, previous works applying them to mortality modeling have not investigated prediction uncertainty. We address this gap in the literature by implementing a bootstrapping-based technique and demonstrate that it yields highly reliable prediction intervals for our neural network model.

Conformal prediction interval estimation and applications to day-ahead and intraday power markets

In this study, we investigated the application of the conformal prediction (CP) concept in the context of short-term electricity price forecasting. In particular, we determined the most important aspects related to the utility of CP, as well as explaining why this simple but highly effective idea has proved useful in other application areas and why its characteristics make it promising for short-term power applications. We compared the performance of CP with various state-of-the-art electricity price forecasting models, such as quantile regression averaging, in an empirical out-of-sample study of three short-term electricity time series. We combined CP with various underlying point forecast models to demonstrate its versatility and behavior under changing conditions. Our findings suggest that CP yields sharp and reliable prediction intervals in short-term power markets. We also inspected the effects of each of the model components to provide path-based guideline regarding how to find the best CP model for each market.

A novel method based on lower–upper bound approximation to predict the wind energy

Recently, the prediction of wind power generation has served as a prerequisite problem for determining the objective of the reliability and low–cost power system performance. The difference between conventional deterministic prediction and probabilistic prediction is that the first types ignore the uncertainty of the wind speeds. In this paper, a novel probabilistic prediction method as improved lower–upper bound approximation is applied for building the reliable prediction intervals, which is based on the system that uses the mathematical models of the environment to prognosticate the weather based on existing meteorology (Numeral Weather Prediction System). To verify the worth of the proposed method, forecasting modes in this study are divided as hourly and daily predictions, which are applied to seven wind farms of Taiwan. In addition, adapting parameters in improved lower–upper bound approximation method is performed by resorting to the charged search system algorithm. Also, to increase the prediction speed, the probability density function of wind speed is used, and to decrease the estimation model volume, the kernel density estimation model is applied into the proposed method. The simulation results in comparison with other methods indicate that the prediction intervals produced by the proposed method outperformed and its indices are better. The average error between prediction and real data in the proposed method obtained as about 11%, while other methods error is more than 15%. In addition, three prediction interval indices are used in order to evaluate the different studied manners as: prediction interval overlay probability, normalized average of prediction interval length, and overlay width scale. As the results shown, the best performance is related to the proposed method.