Probability Density Forecasts Research Articles

Estimation of time-varying kernel densities and chronology of the impact of COVID-19 on financial markets

The time-varying kernel density estimation relies on two free parameters: the bandwidth and the discount factor. We propose to select these parameters so as to minimize a criterion consistent with the traditional requirements of the validation of a probability density forecast. These requirements are both the uniformity and the independence of the so-called probability integral transforms, which are the forecast time-varying cumulated distributions applied to the observations. We thus build a new numerical criterion incorporating both the uniformity and independence properties by the mean of an adapted Kolmogorov–Smirnov statistic. We apply this method to financial markets during the onset of the COVID-19 crisis. We determine the time-varying density of daily price returns of several stock indices and, using various divergence statistics, we are able to describe the chronology of the crisis as well as regional disparities. For instance, we observe a more limited impact of COVID-19 on financial markets in China, a strong impact in the US, and a slow recovery in Europe.

Day-Ahead Peak Load Probability Density Forecasting Based on QRLSTM-DF Considering Exogenous Factors

High-precision peak load forecasting is the guarantee of dispatching and decision-making in power systems. It is difficult to predict the peak power demand accurately and robustly due to many unstable factors that affect power loads. In order to solve this problem, this paper proposes quantile regression long-short term memory based on decoupling features (QRLSTM-DF) for probability density forecasting of the day-ahead peak load. This method reduces the possibility of mutual influence among characteristics by decoupling factors in different branches of neural network. Then, kernel density estimation (KDE) is used as a post-processing technique to generate probability density curves, which can give the comprehensive probability distribution of future peak loads and effectively quantify the uncertainty. Experimental results on three datasets show that the model outperforms several existing prediction models. In particular, the prediction results of the maximum daily peak power load and the peak load beyond the sample maximum boundary are also accurate and robust.

Data-driven probability density forecast for stochastic dynamical systems

In this paper, a data-driven non-parametric approach is presented for forecasting the probability density evolution of stochastic dynamical systems. The method is based on stochastic Koopman operator and extended dynamic mode decomposition (EDMD). To approximate the finite-dimensional eigendecomposition of the stochastic Koopman operator, EDMD is applied to the training data set sampled from the stationary distribution of the underlying stochastic dynamical system. The family of the Koopman operators form a semigroup, which is generated by the infinitesimal generator of the stochastic dynamical system. A significant connection between the generator and Fokker-Planck operator is leveraged to construct an orthonormal basis of a weighted Hilbert space, enabling a spectral decomposition of the probability density function to be performed in this space. This approach is a data-driven method and used to predict the probability density evolution and real-time moment estimation. In the limit of the large number of snapshots and observables, the data-driven probability density approximation converges to the Galerkin projection of the semigroup solution of Fokker-Planck equation on a basis adapted to an invariant measure. This method shares similarities with diffusion forecasting but is capable of producing more accurate probability density forecasts than diffusion forecasting. A few numerical examples are presented to illustrate the performance of the proposed data-driven probability density forecast.

A novel similar-day based probability density forecasting framework for residential loads

Probabilistic load forecasting can provide much richer information about uncertainties in residential loads than point load forecasting, and is gaining increasing attention in building efficient and economical demand response programs. However, obtaining satisfactory forecasting results for residential loads is challenging due to their intrinsic nonlinearity and volatility. Furthermore, existing probabilistic forecasting methods are limited in their ability to directly generate probability density forecasting in an efficient and accurate way. To address these issues, a new probability density forecasting framework for residential loads is proposed in this paper. Firstly, a similar-day choosing method is proposed by combining weighted external features and internal load curves to build a similar-day dataset. Then, point forecasting is achieved by training an exclusive predictor on the corresponding similar-day dataset for each forecasting target. Finally, probability density forecasting is realized by conducting kernel density estimation on the similar-day dataset and calibrating it according to the point forecasting result. Comparisons with other state-of-the-art methods carried out on the HUE dataset demonstrate the efficiency and superiority of the proposed framework in both point and probability density forecasting.

Probability Density Forecasting of Wind Power Based on Transformer Network with Expectile Regression and Kernel Density Estimation

A comprehensive and accurate wind power forecast assists in reducing the operational risk of wind power generation, improves the safety and stability of the power system, and maintains the balance of wind power generation. Herein, a hybrid wind power probabilistic density forecasting approach based on a transformer network combined with expectile regression and kernel density estimation (Transformer-ER-KDE) is methodically established. The wind power prediction results of various levels are exploited as the input of kernel density estimation, and the optimal bandwidth is achieved by employing leave-one-out cross-validation to arrive at the complete probability density prediction curve. In order to more methodically assess the predicted wind power results, two sets of evaluation criteria are constructed, including evaluation metrics for point estimation and interval prediction. The wind power generation dataset from the official website of the Belgian grid company Elia is employed to validate the proposed approach. The experimental results reveal that the proposed Transformer-ER-KDE method outperforms mainstream recurrent neural network models in terms of point estimation error. Further, the suggested approach is capable of more accurately capturing the uncertainty in the forecasting of wind power through the construction of accurate prediction intervals and probability density curves.

Adaptive data decomposition based quantile-long-short-term memory probabilistic forecasting framework for power demand side management of energy system

Load management can improve the overall benefit of power system through peak cutting and valley filling, whose performance depends on the accuracy of load forecasting. However, how to efficiently utilize load data to uprate forecasting performance is urgent. Therefore, a peak cutting and valley filling forecasting algorithm based on load operation state is proposed. Firstly, adaptive data decomposition based quantile-long-short-term memory (QLSTM) probabilistic forecasting framework is proposed to reflect the future load information more comprehensively. The method combines long-short-term memory (LSTM) with pinball loss function to provide deterministic forecasting and probability density forecasting. Then, a variable power peak cutting and valley filling algorithm is proposed to suppress the peak–valley difference and optimize the power structure. Finally, the case study of typical working conditions of actual power grid is performed to confirm that the proposed method can better reflect the future load information and reduce peak–valley difference.

Ultra-short-term probabilistic wind power forecasting with spatial-temporal multi-scale features and K-FSDW based weight

As a potential cleaner energy technology, wind power is a pollution-free and inexhaustible energy, which make a significant contribution to the global energy transformation. Most studies have focused on the accurate forecasting to help the management of the wind power grid-tied. Considering the need for the quantitative modeling of the endogenous random fluctuations and uncertainties involved, a novel ultra-short-term probabilistic wind power forecasting with spatial–temporal multi-scale features and K-FSDW based weight is proposed, which includes data decomposition, multi-scale feature selection, individual model training, dynamic weighting using K-FSDW, error correction modeling by RF and probability density forecasting by KDE, to create uncertainty quantification for power grid dispatching and operation. Firstly, the normalized target wind speed is decomposed into multiple subsequences through VMD, the subsequence and adjacent spatial wind speed series are reconstructed into spatiotemporal candidate features, and spatial–temporal multi-scale feature selection is carried out. Secondly, different quantile regression models are used to predict each subsequence, and a quantile dynamic sparse weighted combination algorithm based on K-forward nearest neighbor is proposed to combine the prediction results of each model, and then reorganize the prediction results of subsequences. Finally, the RF model is used for error correction, and the probability density function is obtained by kernel density estimation. In the experimental comparison and analysis, taking the actual data of an offshore wind farm in Penglai District, Shandong Province, China as an example, the feasibility and effectiveness of the model are verified.

Deep learning-based neural networks for day-ahead power load probability density forecasting.

Energy efficiency is crucial to greenhouse gas (GHG) emission pathways reported by the Intergovernmental Panel on Climate Change. Electrical overload frequently occurs and causes unwanted outages in distribution networks, which reduces energy utilization efficiency and raises environmental risks endangering public safety. Electrical load, however, has a dynamically fluctuating behavior with notoriously nonlinear hourly, daily, and seasonal patterns. Accurate and reliable load forecasting plays an important role in scheduling power generation processes and preventing electrical systems from overloading; nevertheless, such forecasting is fundamentally challenging, especially under highly variable power load and climate conditions. This study proposed a deep learning-based monotone composite quantile regression neural network (D-MCQRNN) model to extract the multiple non-crossing and nonlinear quantile functions while conquering the drawbacks of error propagation and accumulation encountered in multi-step-ahead probability density forecasting. The constructed models were assessed by an hourly power load series collected at the electric grid center of Henan Province in China in two recent years, along with the corresponding meteorological data collected at 16 monitoring stations. The results demonstrated that the proposed D-MCQRNN model could significantly alleviate the time-lag and biased-prediction phenomena and noticeably improve the accuracy and reliability of multi-step-ahead probability density forecasts on power load. Consequently, the proposed model can significantly reduce the risk and impact of overload faults and effectively promote energy utilization efficiency, thereby mitigating GHG emissions and moving toward cleaner energy production.

Probability density forecasts for natural gas demand in China: Do mixed-frequency dynamic factors matter?

Due to the expensive infrastructures and the difficulties in storage, the supply and demand for natural gas in China has experienced periodic fluctuations and geographical imbalance. The primary purpose of this study is to provide satisfying probability density forecasts using mixed-frequency dynamic factors to alleviate the imbalance risks between the supply and the demand. To this end, this paper develops a novel hybrid model named EEMD-MIDAS-SVR, based on ensemble empirical mode decomposition (EEMD) procedure, mixed data sampling (MIDAS) framework and support vector machine regression (SVR). The empirical results reveal that the mixed-frequency dynamic factors could provide powerful predictive information for natural gas demand. By the model confidence set (MCS) test, we identify the most predictive dynamic factors: gas industry index, electricity industry index, Daqing crude oil prices, Qinhuangdao steam coal prices, West Texas Intermediate crude oil prices, and Australia steam coal prices. They could improve forecasting accuracy by about 70% than benchmark models, particularly considering the nonlinearity of natural gas demand contributes to 21% roughly. Based on the MCS test, we provide the non-parametric bootstrap probability density forecasts for natural gas demand, which performs well in capturing the uncertainty of natural gas demand. These findings have several policy implications and practical value for natural gas pricing and carbon neutrality achieving in emerging markets.

Point, interval, and density forecasts: Differences in bias, judgment noise, and overall accuracy

There are three main ways in which judgmental predictions are expressed: point forecasts; interval forecasts; probability density forecasts. Do these approaches differ solely in terms of their simplicity of elicitation and the detail they provide? We examined error in values of the central tendency extracted from these three types of forecast in a domain in which all of them are used: lay forecasts of inflation. A first experiment using a between-participant design showed that the mean level of forecasts and the bias in them are unaffected by the type of forecast but that judgment noise (and, hence, overall error) is higher in point forecasts than in interval or density forecasts. A second experiment replicated the difference between point and interval forecasts in a within-participant design (of the sort used in inflation surveys) and showed no effect of the order in which different types of forecast are made but revealed that people are more overconfident in interval than in point forecasts. A third experiment showed that volatility in past data increases bias in point but not interval forecasts, and that taking the average of two point forecasts made by an individual reduces judgment noise to the level found in interval forecasting.

EEMD-CNN-BiLSTM-QR Enabled Probability Density Forecasts for Crude Oil Price

EEMD-CNN-BiLSTM-QR Enabled Probability Density Forecasts for Crude Oil Price

Probabilistic load forecasting with a non-crossing sparse-group Lasso-quantile regression deep neural network

In this paper, a non-crossing sparse-group Lasso-quantile regression deep neural network (SGLQRDNN) model is proposed to address electricity load forecasting. Different from the traditional deep learning for point forecasting, the SGLQRDNN model realizes the probability density forecasting of the load. SGLQRDNN modelling integrates two strategies to alleviate the dilemma of quantile crossing and structural complexity. The SGLQRDNN model mitigates the deficiency of quantile crossing by a joint estimation of non-crossing constraints. It also realizes the shrinkage of the network and the selection of critical features with the sparse-group Lasso algorithm. The proposed model is trained and tested using the residential daily electricity consumption data. The experimental results show that SGLQRDNN has advantages in interpretability, sparsity, and performance criteria. Specifically, the monotonicity of its internal quantiles is 4.18%–9.96% higher than that of the unconstrained model. Compared with three sparse regularization networks, SGLQRDNN can shrink 88.47% of the connection weights and 19.32% of neurons. Meanwhile, its performance improvement ranges from 5.76% to 18.28%. Additionally, its training speed is 2.73–7.01 times faster than the model trained on individual quantiles. Finally, two non-parametric tests verify that SGLQRDNN significantly outperforms the comparison models at the 10% level.

Short-term flood probability density forecasting using a conceptual hydrological model with machine learning techniques

Making accurate and reliable probability density forecasts of flood processes is fundamentally challenging for machine learning techniques, especially when prediction targets are outside the range of training data. Conceptual hydrological models can reduce rainfall-runoff modelling errors with efficient quasi-physical mechanisms. The Monotone Composite Quantile Regression Neural Network (MCQRNN) is used for the first time to make probability density forecasts of flood processes and serves as a benchmark model, whereas it confronts the drawbacks of overfitting and biased-prediction. Here we propose an integrated model (i.e. XAJ-MCQRNN) that incorporates Xinanjiang conceptual model (XAJ) and MCQRNN to overcome the phenomena of error propagation and accumulation encountered in multi-step-ahead flood probability density forecasts. We consider flood forecasts as a function of rainfall factors and runoff data. The models are evaluated by long-term (2009–2015) 3-hour streamflow series of the Jianxi River catchment in China and rainfall products of the European Centre for Medium-Range Weather Forecasts. Results demonstrated that the proposed XAJ-MCQRNN model can not only outperform the MCQRNN model but also prominently enhance the accuracy and reliability of multi-step-ahead probability density forecasts of flood process. Regarding short-term forecasts in testing stages at four horizons, the XAJ-MCQRNN model achieved higher Nash-Sutcliffe Efficiency but lower Root Mean Square Error values, while improving Coverage Ratio and Relative Bandwidth values in comparison to the MCQRNN model. Consequently, the improvement can benefit the mitigation of the impacts associated with uncertainties of extreme flood and rainfall events as well as promote the accuracy and reliability of flood forecasting and early warning.

Day-ahead load probability density forecasting using monotone composite quantile regression neural network and kernel density estimation

Due to the aggregation of renewable resources and the fluctuation of demand side, exploring more advanced approaches to maintain the accuracy of short-term load forecasting has become a momentous task. The neural network-based quantile regression probability density forecasting can reveal the prediction uncertainty, which is critical in building energy-saving and reliable power grid. However, quantile curves estimated by previous quantile regression probability density forecasting model may cross each other, violating the basic properties of quantile and jeopardizing the flexibility of modeling. For counteracting this disadvantage, a probability density prediction method based on monotone composite quantile regression neural network and Gaussian kernel function (MCQRNNG) for day-ahead load is proposed. The proposed method guarantees non-crossing by simultaneously estimating multiple nonlinear quantile regression functions to obtain better estimation accuracy. Besides, three-dimensional grid search is adopted to adaptively determine the optimal network structure. Taking real load data carrying quantile crossing from Ottawa in Canada, Baden-Württemberg in Germany and a certain region in China, the performance of the MCQRNNG model is verified from three aspects of point, interval and probability prediction. Results show that the proposed model significantly outperforms the comparison model on the basis of effectively extricating the quantile crossing for power load forecasting.

Probability density forecasts for steam coal prices in China: The role of high-frequency factors

Abstract Coal plays a key role in China’s economy as a dominant primary energy resource. In this paper, we provide probability density forecasts for weekly steam coal prices in China based on daily factors such as renewable energy source, Daqing oil, Japanese natural gas, Australia steam coal prices, coal mining industry index, A-share power sector index, A-share index, coal industry index, and temperature. The empirical results show that the influence of temperature lasts longer than other factors, while the Australia steam coal prices, renewable energy source and A-share index are the three best predictors for steam coal prices. It is also shown that the high-frequency factors are useful to forecast steam coal prices and that considering the nonlinearity of coal prices can improve the forecast accuracy by about 22%. We further provide the probability density forecasts for steam coal prices based on the influence of all the selected factors, the results suggest that our proposed method can provide accurate and satisfying probability density forecasts. Given these results, the policy-makers can make effective strategies which can not only adjust the energy structure but also ensure economic growth. • High-frequency factors are useful to forecast coal prices. • C-MIDAS-X model considers the high-frequency factors and the nonlinearity of coal prices. • The probability density forecasts are satisfying.

Probability Density Forecasting of Wind Speed Based on Quantile Regression and Kernel Density Estimation

Based on quantile regression (QR) and kernel density estimation (KDE), a framework for probability density forecasting of short-term wind speed is proposed in this study. The empirical mode decomposition (EMD) technique is implemented to reduce the noise of raw wind speed series. Both linear QR (LQR) and nonlinear QR (NQR, including quantile regression neural network (QRNN), quantile regression random forest (QRRF), and quantile regression support vector machine (QRSVM)) models are, respectively, utilized to study the de-noised wind speed series. An ensemble of conditional quantiles is obtained and then used for point and interval predictions of wind speed accordingly. After various experiments and comparisons on the real wind speed data at four wind observation stations of China, it is found that the EMD-LQR-KDE and EMD-QRNN-KDE generally have the best performance and robustness in both point and interval predictions. By taking conditional quantiles obtained by the EMD-QRNN-KDE model as the input, probability density functions (PDFs) of wind speed at different times are obtained by the KDE method, whose bandwidth is optimally determined according to the normal reference criterion. It is found that most actual wind speeds lie near the peak of predicted PDF curves, indicating that the probabilistic density prediction by EMD-QRNN-KDE is believable. Compared with the PDF curves of the 90% confidence level, the PDF curves of the 80% confidence level usually have narrower wind speed ranges and higher peak values. The PDF curves also vary with time. At some times, they might be biased, bimodal, or even multi-modal distributions. Based on the EMD-QRNN-KDE model, one can not only obtain the specific PDF curves of future wind speeds, but also understand the dynamic variation of density distributions with time. Compared with the traditional point and interval prediction models, the proposed QR-KDE models could acquire more information about the randomness and uncertainty of the actual wind speed, and thus provide more powerful support for the decision-making work.

Probability density forecasting of wind power based on multi-core parallel quantile regression neural network

The large-scale utilization of wind energy brings severe challenges to the dispatching operation of power systems. Currently, the probability density prediction method combining quantile regression neural network (QRNN) with Epanechnikov kernel function is an excellent algorithm for wind power prediction, which can give the comprehensive probability distribution of future wind power and effectively quantify the uncertainty of wind power generation. However, existing probability density prediction methods process data sequentially in different quantiles, and computational time costs multiply with the increase of training data. It affects the practicality of the probability density prediction method. To overcome this issue, this paper proposes a multi-core parallel quantile regression neural network (MPQRNN) based on parallel master–slave (MS) model. The algorithm divides the complex prediction tasks at all quantiles into multiple parallel sub-tasks, which are independently run on different cores, so that performance advantages of the multi-core CPU can be fully utilized for improving the computational efficiency of the joint operation model. We compare four different scale sample sets under different process numbers. The influence of different CPU core numbers on the parallel performance of MPQRNN are analyzed by the algorithmic nature of speedup and parallel efficiency. To demonstrate the effectiveness of the proposed model, comparative experiments of other four traditional models are carried out on data sets. The simulation results demonstrate that the MPQRNN can not only improve the training efficiency of QRNN, but also obtain precise results of wind power forecasting, showing potential value and utility for complex power system.

Load probability density forecasting by transforming and combining quantile forecasts

Compared with traditional deterministic load forecasting, probabilistic load forecasting (PLF) help us understand the potential risks in the power system operation by providing more information about future uncertainties of the loads. Quantile forecasting, as a kind of non-parametric probabilistic forecasting method, has been well developed and widely used in PLF. However, the results of quantile forecasts are discrete, which contain fewer details than density forecasts which provide the most comprehensive information. This paper proposes a novel day-ahead load probability density forecasting method by transforming and combining multiple quantile forecasts. The proposed method includes two main steps: transformation and combination. In the first step, the kernel density estimation method is used to transform the individual quantile forecast into the probability density curve; in the second step, an optimization problem is established to obtain the weighted combination of different probability density forecasts. The perturbation search method is applied to determine the optimal weight of each individual forecast. We demonstrate the effectiveness and superiority of our proposed method using comprehensive case studies on the real-world load data from Guangdong province in China, ISO New England (ISO-NE) in the US and Irish smart meter data. Case studies show that the combined model is robust to kernel function selection in the transformation step and has better forecasting performance. Compared with the best individual model, the purposed combined model has an accuracy improvement of 1.54% in the Guangdong dataset and 2.9% in the ISO-NE dataset in terms of the continuous ranked probability score. The proposed combination forecasting method can be robust in high volatility scenarios.

Deep-Based Conditional Probability Density Function Forecasting of Residential Loads

This paper proposes a direct model for conditional probability density forecasting of residential loads, based on a deep mixture network. Probabilistic residential load forecasting can provide comprehensive information about future uncertainties in demand. An end-to-end composite model comprising convolution neural networks (CNNs) and gated recurrent unit (GRU) is designed for probabilistic residential load forecasting. Then, the designed deep model is merged into a mixture density network (MDN) to directly predict probability density functions (PDFs). In addition, several techniques, including adversarial training, are presented to formulate a new loss function in the direct probabilistic residential load forecasting (PRLF) model. Several state-of-the-art deep and shallow forecasting models are also presented in order to compare the results. Furthermore, the effectiveness of the proposed deep mixture model in characterizing predicted PDFs is demonstrated through comparison with kernel density estimation, Monte Carlo dropout, a combined probabilistic load forecasting method and the proposed MDN without adversarial training.

Forecasting Hourly Power Load Considering Time Division: A Hybrid Model Based on K-means Clustering and Probability Density Forecasting Techniques

In comparison with traditional point forecasting method, probability density forecasting can reflect the load fluctuation more effectively and provides more information. This paper proposes a hybrid hourly power load forecasting model, which integrates K-means clustering algorithm, Salp Swarm Algorithm (SSA), Least Square Support Vector Machine (LSSVM), and kernel density estimation (KDE) method. Firstly, the loads at 24 times a day are grouped into three categories according to the K-means clustering algorithm, which correspond to the valley period, flat period, and peak period of the load, respectively. Secondly, the load point forecasting value is obtained by LSSVM method optimized by SSA algorithm. Furthermore, the kernel density estimation method is employed to fit the forecasting error of SSA-LSSVM in different time periods, and the probability density function of the error distribution is obtained. The final load probability density forecasting result is obtained by combining the point forecasting value and the error fitting result, and then the upper and lower limits of the confidence interval under the given confidence level are solved. In this paper, the performance of the model is evaluated by two indicators named interval coverage and interval average width. Meanwhile, in comparison with several other models, it can be concluded that the proposed model can effectively improve the forecasting effect.