Predictive Probability Density Research Articles

A short-term load forecasting method based on QRLSTM considering multiple factors analysis

Abstract Complex characteristics of power loads terribly affect the safety of the power grid. Thus, accurate short-term load forecasting is regarded as a solution to mitigate the effect of load complications. In this regard, this work is putting forward a Long Short-Term Memory Network Quantile Regression (LSTMQR) for power load, obtaining the probability density function (PDF). Firstly, the quantile of future power load was obtained using the LSTMQR model. The conditional quantile obtained by the LSTMQR model is used as the input of the Kernel Density Estimation model (KDE) to predict the future probability density distribution of short-term power load at different prediction times. Furthermore, an assessment method is proposed to quantify the effect of different factors on power load to improve prediction accuracy. Finally, a numerical simulation based on actual data in China is established to validate the effectiveness of the proposed forecasting algorithm, indicating the proposed forecasting algorithm can effectively evaluate the relationship between different factors and power load, and probability density prediction can accurately quantify the uncertainty of load information. The proposed algorithm results in a reduction of 37.6% in forecast error compared to the traditional forecasting algorithm.

Probability density prediction for carbon allowance prices based on TS2Vec and distribution Transformer

Carbon allowance price is an important tool to reduce carbon emissions and achieve carbon neutrality. It is necessary to establish a predictive model to provide accurate and reliable information to managers and participants in the carbon trading market. Therefore, a novel probability density prediction model, called TS2Vec-based distribution Transformer (TDT), is proposed. TDT consists of two stages: contrastive unsupervised pre-training and supervised training. In the contrastive unsupervised training stage, time series to vector (TS2Vec) is used to represent the dynamic trends and unique features of the data. Then, these representations are fed into the distribution Transformer (DT) to fit the hypothetical probability distribution. Experimental results show that the prediction results of the proposed TDT are more accurate and reliable than other benchmark models. In addition, our research indicates reliable probability density predictions provide enterprises with opportunities to control carbon emission costs and increase economic returns, thereby improving the competitiveness of enterprises and promoting carbon emission reduction.

Enhanced predictive PDF control of stochastic distribution systems with neural network compensation and its application

Compared to the conventional control algorithms that use mean and variance as indicators, predictive probability density function (PDF) control can effectively handle the output PDF control problem of non-Gaussian stochastic distribution systems. However, the existing predictive PDF control method does not consider the construction error between output PDF and weight, thus the control performance is still unsatisfactory. Therefore, this paper proposes a new Enhanced Predictive PDF control method (En-PDF) to improve the output PDF control performance of stochastic distribution systems. The proposed method mainly consists of two parts: the predictive PDF control part and the neural network compensation control part aiming to reduce the bias of output PDF. First, the Radical Basis Functions (RBFs) are used to approximate the PDF of the stochastic systems output, and then a prediction model representing the relationship between input and weight is established using the subspace identification algorithm to design the predictive PDF control for the stochastic systems. Next, the Kullback-Leibler (KL) divergence is used to measure the similarity between the output PDF and the set PDF, combined with the weight error and compensation to design a new performance index. Based on this, the parameters of the neural network are adjusted using the gradient descent algorithm to obtain the optimal compensation, and the stability and tracking performance of the proposed algorithm are analyzed using inductive reasoning method. Finally, the predictive PDF control input with the compensation work together on the controlled plant to achieve high-performance control of the output PDF of non-Gaussian stochastic distribution systems. Both simulation experiments and physical control experiments validate the effectiveness and superiority of the proposed method.

Short-Term Photovoltaic Power Probabilistic Forecasting Based on Temporal Decomposition and Vine Copula

With the large-scale development of solar power generation, highly uncertain photovoltaic (PV) power output has an increasing impact on distribution networks. PV power generation has complex correlations with various weather factors, while the time series embodies multiple temporal characteristics. To more accurately quantify the uncertainty of PV power generation, this paper proposes a short-term PV power probabilistic forecasting method based on the combination of decomposition prediction and multidimensional variable dependency modeling. First, a seasonal and trend decomposition using a Loess (STL)-based PV time series feature decomposition model is constructed to obtain periodic, trend, and residual components representing different characteristics. For different components, this paper develops a periodic component prediction model based on TimeMixer for multi-scale temporal feature mixing, a long short-term memory (LSTM)-based trend component extraction and prediction model, and a multidimensional PV residual probability density prediction model optimized by Vine Copula optimized with Q-Learning. These components’ results form a short-term PV probabilistic forecasting method that considers both temporal features and multidimensional variable correlations. Experimentation with data from the Desert Knowledge Australia Solar Center (DKASC) demonstrates that the proposed method reduced root mean square error (RMSE) and mean absolute percentage error (MAPE) by at least 14.8% and 22%, respectively, compared to recent benchmark models. In probability interval prediction, while improving accuracy by 4% at a 95% confidence interval, the interval width decreased by 19%. The results show that the proposed approach has stronger adaptability and higher accuracy, which can provide more valuable references for power grid planning and decision support.

Study on wind power prediction based on improved double wavelet transform and quantile regression forest

Abstract The random fluctuation of a wind power system will bring significant challenges to its stable operation when connected to the grid[1]. To reduce the impact of microgrid wind power integration on electric power systems and in response to the certain prediction inaccuracy of BP, LSSVM, and ARIMA-based prediction models in case of rapid fluctuation of actual power, a method of decomposing the original wind power sequence into several characteristic subcomponents through the improved double wavelet transform algorithm was proposed in this study to weaken the fluctuation of the wind power sequence. Specifically, a short-term wind power probability density prediction model based on the improved double wavelet algorithm and quantile regression method was established. Then, the original wind power sequence was decomposed into a series of components with different frequencies using the double wavelet transform method. Suitable components were chosen to construct a QRF prediction model to acquire the prediction results.

FILTERING OF AN INTERRUPTED SIGNAL ON APPROXIMATION BY GAMMA DISTRIBUTION UNDER THE INFLUENCE OF ADDITIVE MULTIPLICATIVE INTERFERENCE

The problem of constructing an algorithm for recognizing and filtering a continuous-valued Markov process measured in a mixture with random additive and multiplicative interference under conditions of random information interruptions is considered. The Bayes principle and the well-known two-stage filtering method “forecast-correction” are applied, supplemented by procedures for identifying the number of the structure, in which the system is located in the current and next time intervals. At the forecast stage, identification is based on calculating the probabilities of the future state of the system based on a priori information about the probabilities of its transitions from structure to structure, as well as on the current values of estimates of these probabilities. At the correction stage, identification is based on the use of probability prediction results and a priori known information about the reliability of the information of the system status indicator. In addition, the values of the probability densities of the measuring information in each structure are used, the expressions for which are obtained on the basis of a priori known laws of distribution of additive and multiplicative interference, as well as the predicted probability density of the Markov process in each structure at the next step of calculations. To obtain a practically realizable algorithm, expressions for the predicted and a posteriori densities of the Markov process probabilities are obtained based on their approximation by the probability density of the gamma distribution, completely determined by the first two initial moments. This made it possible to obtain a closed system of recurrent equations for estimating the probability of information interruption, mathematical expectations and variances of the Markov process in each structure.

Technical note: Accurate, reliable, and high-resolution air quality predictions by improving the Copernicus Atmosphere Monitoring Service using a novel statistical post-processing method

Abstract. Starting from the regional air quality forecasts produced by the Copernicus Atmosphere Monitoring Service (CAMS), we propose a novel post-processing approach to improve and downscale results on a finer scale. Our approach is based on the combination of ensemble model output statistics (EMOS) with a spatio-temporal interpolation process performed through the stochastic partial differential equation–integrated nested laplace approximation (SPDE-INLA). Our interpolation approach includes several spatial and spatio-temporal predictors, including meteorological variables. A use case is provided that scales down the CAMS forecasts on the Italian peninsula. The calibration is focused on the concentrations of several air quality pollutants (PM10, PM2.5, NO2, and O3) at a daily resolution from a set of 750 monitoring sites, distributed throughout the Italian country. Our results show the key role that conditioning variables play in improving the forecast capabilities of ensemble predictions, thus allowing for a net improvement in the calibration with respect to ordinary EMOS strategies. From a deterministic point of view, the performance of the predictive model shows a significant improvement in the performance of the raw ensemble forecast, with an almost-zero bias, significantly reduced root mean square errors, and correlations that are almost always higher than 0.9 for each pollutant; moreover, the post-processing approach is able to significantly improve the prediction of exceedances, even for very low thresholds, such as those recently recommended by the World Health Organisation. This is particularly significant if a forecasting approach is used to predict air quality conditions and plan adequate human health protection measures, even for low alert thresholds. From a probabilistic point of view, the quality of the forecast was verified in terms of reliability and credible intervals. After post-processing, the predictive probability density functions were sharp and much better calibrated than the raw ensemble forecast. Finally, we present some additional results based on a set of gridded (4 km × 4 km) maps covering the entire Italian country for the detection of areas where pollution peaks occur (exceedances of the current and/or proposed regulatory thresholds).

Probability density prediction of peak load based on mixed frequency noise-assisted multivariate empirical mode decomposition

Probability density prediction of peak load based on mixed frequency noise-assisted multivariate empirical mode decomposition

ForecastNet Wind Power Prediction Based on Spatio-Temporal Distribution

The integration of large-scale wind power into the power grid threatens the stable operation of the power system. Traditional wind power prediction is based on time series without considering the variability between wind turbines in different locations. This paper proposes a wind power probability density prediction method based on a time-variant deep feed-forward neural network (ForecastNet) considering a spatio-temporal distribution. First, the outliers in the wind turbine data are detected based on the isolated forest algorithm and repaired through Lagrange interpolation. Then, based on the graph attention mechanism, the features of the proximity node information of the individual wind turbines in the wind farm are extracted and the input feature matrix is constructed. Finally, the wind power probability density prediction results are obtained using the ForecastNet model based on three different hidden layer variants. The experimental results show that the ForecastNet model with a hidden layer as a dense network based on the attention mechanism (ADFN) predicts better. The average width of the prediction intervals at achieved confidence levels for all interval coverage is reduced by 34.19%, 35.41%, and 35.17%, respectively, when compared to the model with the hidden layer as a multilayer perceptron. For different categories of wind turbines, ADFN also achieves relatively narrow interval average widths of 368.37 kW, 315.87 kW, and 299.13 kW, respectively.

Fault detection in automated production systems based on a long short-term memory autoencoder

Abstract In this paper, a hybrid model of regularized Long Short-Term Memory (LSTM) and autoencoder for fault detection in automated production systems is proposed. The presented LSTM autoencoder is used as a stochastic process model, which captures the normal behavior of a production system and allows to predict the probability distribution of sensor data. Discrepancies between the observed sensor data and the predicted probability density distribution are detected as potential faults. The approach combines the advantages of LSTMs and autoencoders: The correlations between individual sensor signals are exploited by an autoencoder, while the temporal dependencies are captured by LSTM neurons. A key challenge in training such a process model from historical data is to control the information passed through the latent space of the autoencoder. Different regularization methods are investigated for this purpose. Fault detection with the proposed LSTM autoencoder has been evaluated on the use case of an industrial penicillin production, achieving significantly improved results in comparison to the baseline LSTM.

A Novel Daily Runoff Probability Density Prediction Model Based on Simplified Minimal Gated Memory–Non-Crossing Quantile Regression and Kernel Density Estimation

Reliable and accurate daily runoff predictions are critical to water resource management and planning. Probability density predictions of daily runoff can provide decision-makers with comprehensive information by quantifying the uncertainty of forecasting. Models based on quantile regression (QR) have been proven to achieve good probabilistic prediction performance, but the predicted quantiles may crossover with each other, seriously reducing the reliability of the prediction. This paper proposes non-crossing quantile regression (NCQR), which guarantees that the intervals between adjacent quantiles are greater than 0, which avoids the occurrence of quantile crossing. In order to apply NCQR to the prediction of nonlinear runoff series, this paper combines NCQR with recurrent neural network (RNN) models. In order to reduce the model training time and further improve the model accuracy, this paper simplifies the minimal gated memory (MGM) model and proposes a new RNN model, called the simplified minimal gated memory (SMGM) model. Kernel density estimation (KDE) is used to transform the discrete quantiles predicted using SMGM-NCQR into a continuous probability density function (PDF). This paper proposes a novel daily density prediction model that combines SMGM-NCQR and KDE. Three daily runoff datasets in the Yangtze River Basin in China are taken as examples and compared with the advanced models in current research in terms of five aspects: point prediction evaluation, interval prediction evaluation, probability density prediction evaluation, the degree of quantile crossing and training time. The experimental results show that the model can provide high-quality and highly reliable runoff probability density predictions.

Student’s t-Based Robust Poisson Multi-Bernoulli Mixture Filter under Heavy-Tailed Process and Measurement Noises

A novel Student’s t-based robust Poisson multi-Bernoulli mixture (PMBM) filter is proposed to effectively perform multi-target tracking under heavy-tailed process and measurement noises. To cope with the common scenario where the process and measurement noises possess different heavy-tailed degrees, the proposed filter models this noise as two Student’s t-distributions with different degrees of freedom. Furthermore, this method considers that the scale matrix of the one-step predictive probability density function is unknown and models it as an inverse-Wishart distribution to mitigate the influence of heavy-tailed process noise. A closed-form recursion of the PMBM filter for propagating the approximated Gaussian-based PMBM posterior density is derived by introducing the variational Bayesian approach and a hierarchical Gaussian state-space model. The overall performance improvement is demonstrated through three simulations.

High-frequency monitoring of China's green growth-at-risk.

With industrialization and urbanization, China faces enormous challenges from energy security and environmental issues. To address these challenges, it is imperative to establish a green accounting system for economic growth and to measure the uncertainty of China's green GDP (GGDP) growth from a risk management perspective. With this in mind, we follow the idea of growth-at-risk (GaR) to propose the concept of green GaR (GGaR) and extend it to the mixed-frequency data environment. Specifically, we first measure China's annual GGDP using the System of Environmental Economic Accounting (SEEA), then construct China's monthly green financial index by a mixed-frequency dynamic factor model (MF-DFM), and finally monitor China's GGaR from 2008M1 to 2021M12 with the mixed data sampling-quantile regression (MIDAS-QR) method. The main findings are as follows: First, the proportion of China's GGDP to traditional GDP gradually increases from 81.97% in 2008 to 89.34% in 2021, which illustrates that the negative environmental externalities caused by China's economic growth are gradually decreasing. Second, the high-frequency GGaR has favorable predictive performance and is significantly superior to the common-frequency GGaR at most quantiles. Third, the high-frequency GGaR has good nowcasting performance, and its 90% and 95% confidence intervals include true value for all prediction horizons. Furthermore, it can provide early warning of economic downturns through probability density prediction. Overall, our main contribution lies in constructing a quantitative assessment and high-frequency monitoring of China's GGDP growth risk, which provides an effective tool for investors and companies to predict risk, and a reference for the Chinese government to better formulate sustainable development strategies.

Uncertainty propagation of turbulence parameters for typhoon and Non-typhoon winds in buffeting analysis of Long-span bridges

Wind turbulence-induced buffeting vibration affects the serviceability of long-span bridges and lead to fatigue problems. The randomness of turbulence parameters in the power spectrum density function is always disregarded, failing to well reproduce the dispersion of structural buffeting responses and achieve the extreme vibration amplitude. This study proposes a probabilistic buffeting analysis approach based on multi-point estimation, dimension reduction techniques, and the principle of maximum entropy accounting for the uncertainty of turbulence parameters. The uncertainties in turbulence power spectral density functions are quantified based on the long-term real observational wind data captured by the sensors installed at Xihoumen Bridge. These uncertainties are propagated into the buffeting response of the bridge using the proposed method. The results are compared with those of Monte Carlo simulation, suggesting a good agreement with substantially lower computational costs can be attained by the proposed method. Then, the probability density evolution of buffeting response versus wind speed is calculated. Results reveal that typhoon-induced buffeting response is greater than non-typhoon’s under low mean wind speeds (e.g., 25 m/s), whereas it is the opposite at high wind speeds (e.g., 30 m/s). The predicted probability density evolution of buffeting response exhibits a great consistency with real observations. By introducing the mean wind speed hazard curves for both typhoon and non-typhoon winds, the exceeding probability of vibration within a given period can be easily attained. It allows the estimation of return periods of structural responses and development of the uniform-risk approach for structure design rather than the conventional uniform-hazard design method.

A novel electricity load forecasting based on probabilistic least absolute shrinkage and selection operator-Quantile regression neural network

Predicting energy consumption is challenging since forecast accuracy is heavily impacted by several external elements, including the economy, community, environment, and sustainable energy, including wind, solar, and hydro energy. The most crucial aspect in successfully predicting electrical power demand is to gather crucial information about those external factors in the vast amount of data provided by the smart grid. The Least Absolute Shrinkage and Selection Operator-Quantile Regression Neural Network (LASSO-QRNN) is a strategy for probability density prediction proposed in this article. First, using LASSO regression, the primary features are determined from external factors impacting electrical power utilization. The LASSO-Quantile regression NN approach is then developed to forecast annual electrical power consumption. In the years that followed, various quantiles were employed to gauge electrical power demand, and the outcomes were assessed. The LASSO-QRNN approach also includes a kernel density evaluation that, in the event of a one-value forecast, will display a probability distribution. The experimental assessments from the Guangdong database in China and the California real-world databases in the US have evaluated the forecasting precision. The results of the numerical study demonstrate that, in comparison to Quantile Regression and Kernel Density, Box-Cox transformation, the provided model provides superior accuracy for the prediction of electrical power demand. Deep Generative Architecture, K-means Clustering, a Quantile Regression Neural Network, expert prediction, and fuzzy Bayesian theory. LASSO-Quantile regression neural network can handle the higher dimension information in electrical power usage prediction besides the more accurate outcomes.

Wind Power Probability Density Prediction Based on Quantile Regression Model of Dilated Causal Convolutional Neural Network

Aiming at the wind power prediction problem, a wind power probability prediction method based on the quantile regression of a dilated causal convolutional neural network is proposed. With the developed model, the Adam stochastic gradient descent technique is utilized to solve the cavity parameters of the causal convolutional neural network under different quantile conditions and obtain the probability density distribution of wind power at various times within the following 200 hours. The presented method can obtain more useful information than conventional point and interval predictions. Moreover, a prediction of the future complete probability distribution of wind power can be realized. According to the actual data forecast of wind power in the PJM network in the United States, the proposed probability density prediction approach can not only obtain more accurate point prediction results, it also obtains the complete probability density curve prediction results for wind power. Compared with two other quantile regression methods, the developed technique can achieve a higher accuracy and smaller prediction interval range under the same confidence level.

Probability Density Forecasting of Wind Power Based on Transformer Network with Expectile Regression and Kernel Density Estimation

A comprehensive and accurate wind power forecast assists in reducing the operational risk of wind power generation, improves the safety and stability of the power system, and maintains the balance of wind power generation. Herein, a hybrid wind power probabilistic density forecasting approach based on a transformer network combined with expectile regression and kernel density estimation (Transformer-ER-KDE) is methodically established. The wind power prediction results of various levels are exploited as the input of kernel density estimation, and the optimal bandwidth is achieved by employing leave-one-out cross-validation to arrive at the complete probability density prediction curve. In order to more methodically assess the predicted wind power results, two sets of evaluation criteria are constructed, including evaluation metrics for point estimation and interval prediction. The wind power generation dataset from the official website of the Belgian grid company Elia is employed to validate the proposed approach. The experimental results reveal that the proposed Transformer-ER-KDE method outperforms mainstream recurrent neural network models in terms of point estimation error. Further, the suggested approach is capable of more accurately capturing the uncertainty in the forecasting of wind power through the construction of accurate prediction intervals and probability density curves.

Design of robust Gaussian approximate filter and smoother with latency probability identification

The Kalman filter is a linear optimal estimator when it can accurately receive measurements and the noise obeys a Gaussian distribution. However, the measurement noise may have heavy-tailed characteristics because of outlier interference. Actually, real-world measurement information may have one-step randomly delayed measurements (ORDMs) with unknown latency probability (LP) due to exogenous disturbances. Moreover, systems are usually not linear. To overcome the influence of the state estimator of nonlinear systems from heavy-tailed measurement noise (HMN) and the unknown LP, this paper focuses on the robust Gaussian approximate (GA) filter and smoother for the above issues. First, the one-step predicted probability density function (PDF) is modeled as a Gaussian distribution, and the likelihood PDF is modeled as a Normal–Gamma–Beta (NGBM) distribution. Furthermore, novel robust Gaussian approximate filter and smoother are proposed based on the variational Bayesian technique. They provide general estimation frameworks for estimating state of nonlinear systems with HMN and ORDMs, and different nonlinear estimators can be implemented using different numerical techniques. In addition, the computational complexity of the proposed algorithms is calculated. Finally, taking a univariate nonstationary growth model, a passive ranging problem, and target tracking as examples, the effectiveness and superiority of the proposed robust algorithms contrasted with the existing algorithms are shown. The outcomes show that the proposed methods have higher estimation precision; however, the computational burden is marginally increased.

Density integration approach for probabilistic prediction of wind power generation based on ensemble weather forecast

Prediction of wind power output becomes further important to realize compensation for output fluctuation through operation of reserve power sources and energy storage systems. In particular, the probability density prediction (PDP), which derives a conditional probability distribution of the forthcoming wind power output based on the latest observable information, is attractive from the perspective of providing rich information not only on the expected value of power generation output but also on its uncertainty. This study discusses a density integration approach for the PDP task utilizing ensemble weather forecast (EWF) results derived with the individual ensemble members. The approach focuses on sequences of probability densities for the wind power plant (WPP) output derived by individual ensemble members; these sequences are dynamically weighted and integrated according to its current plausibility by matching them with the latest actual power generation at the required prediction timing. The framework enables us to realize frequently updatable short-term PDP by focusing on fresh observations, while taking advantage of ensemble weather forecasts, which are computationally expensive and not easily updated frequently. The usefulness of the proposed framework is evaluated through benchmark experiments on data sets collected at several real-world WPPs.

Information-based seismic amplitude inversion

Amplitude variation with offset (AVO) inversion, particularly for more than two model parameters, is a highly ill-posed problem and, hence, regularization is indispensable. For common regularization such as norm-regularization, learning can be forged ahead via gradient descent. Alternatively, learning can be achieved by performing gradient descent on the objective and projects the result back into the space of the regularizer; this allows the use of atypical regularization functions. Here, we propose a regularized inverse problem to mitigate the Ill-posedness of the amplitude inversion. The regularization is added to measure the difference in information between the a priori probability density function and the predicted probability density of the inverted parameters, and therefore enhances the consistency between the inversion results and the a priori geological information. Information theory provides a collection of contrast functions which quantify the divergence from one probability distribution to another, such as the relative entropy. The a priori density is approximated by a Gaussian mixture model, obtained from well logs and rock physics data. The mixture model is a density estimator, providing the statistical properties of the model parameters of interest. The likelihood of the data and the divergence are combined in an augmented Lagrangian scheme, to obtain a stable solution that best generate the recorded seismic data and satisfy the geological constraints conveyed by the a priori probability density function. The proposed inversion scheme is applied to the anisotropy AVO inversion, for estimating the elastic and seismic anisotropy parameters of shale formations. Compared to the unconstrained minimization, the P- and S-wave velocity, and ε are better recovered, moreover, density and Thomsen's δ are well-constrained.