Estimation Of Probability Distribution Research Articles

On fractional moment estimation from polynomial chaos expansion

Fractional statistical moments are utilized for various tasks of uncertainty quantification, including the estimation of probability distributions. However, an estimation of fractional statistical moments of costly mathematical models by statistical sampling is challenging since it is typically not possible to create a large experimental design due to limitations in computing capacity. This paper presents a novel approach for the analytical estimation of fractional moments, directly from polynomial chaos expansions. Specifically, the first four statistical moments obtained from the deterministic coefficients of polynomial chaos expansion are used for an estimation of arbitrary fractional moments via Hölder’s inequality. The proposed approach is utilized for an estimation of statistical moments and probability distributions in four numerical examples of increasing complexity. Obtained results show that the proposed approach achieves a superior performance in estimating the distribution of the response, in comparison to a standard Latin hypercube sampling in the presented examples.

Parameter estimation of Weibull probability distribution by seven methods – a wind regime of the city of Nitra, Slovakia

Parameter estimation of Weibull probability distribution by seven methods – a wind regime of the city of Nitra, Slovakia

Development of a High-Resolution Indoor Radon Map Using a New Machine Learning-Based Probabilistic Model and German Radon Survey Data.

Radon is a carcinogenic, radioactive gas that can accumulate indoors and is undetected by human senses. Therefore, accurate knowledge of indoor radon concentration is crucial for assessing radon-related health effects or identifying radon-prone areas. Indoor radon concentration at the national scale is usually estimated on the basis of extensive measurement campaigns. However, characteristics of the sampled households often differ from the characteristics of the target population owing to the large number of relevant factors that control the indoor radon concentration, such as the availability of geogenic radon or floor level. Furthermore, the sample size usually does not allow estimation with high spatial resolution. We propose a model-based approach that allows a more realistic estimation of indoor radon distribution with a higher spatial resolution than a purely data-based approach. A multistage modeling approach was used by applying a quantile regression forest that uses environmental and building data as predictors to estimate the probability distribution function of indoor radon for each floor level of each residential building in Germany. Based on the estimated probability distribution function, a probabilistic Monte Carlo sampling technique was applied, enabling the combination and population weighting of floor-level predictions. In this way, the uncertainty of the individual predictions is effectively propagated into the estimate of variability at the aggregated level. The results show an approximate lognormal distribution of indoor radon in dwellings in Germany with an arithmetic mean of , a geometric mean of , and a 95th percentile of . The exceedance probabilities for 100 and are 12.5% (10.5 million people affected) and 2.2% (1.9 million people affected), respectively. In large cities, individual indoor radon concentration is generally estimated to be lower than in rural areas, which is due to the different distribution of the population on floor levels. The advantages of our approach are that is yields a) an accurate estimation of indoor radon concentration even if the survey is not fully representative with respect to floor level and radon concentration in soil, and b) an estimate of the indoor radon distribution with a much higher spatial resolution than basic descriptive statistics. https://doi.org/10.1289/EHP14171.

Battery state-of-health estimation incorporating model uncertainty based on Bayesian model averaging

Accurately estimating the state-of-health (SOH) of lithium-ion batteries is crucial for efficient, reliable, and safe use. However, the degradation of these batteries involves complex and intricate failure mechanisms that cannot be fully captured by a single model. To tackle this challenge, we propose an SOH estimation method based on Bayesian Model Averaging (BMA), which effectively accounts for both parameter and model uncertainties by combining estimations from different model implementations. It provides both point-value estimates and the probability distribution estimates, delivering results in a fraction of a second. Evaluated on three open-source datasets, the maximum absolute error of SOH estimation is within 0.03, and the Continuous Ranked Probability Score is within 0.015. Compared to optimal individual models, the proposed BMA method reduces the prediction error of point estimates by half to two-thirds. Additionally, the prediction error for probability estimation decreases by an order of magnitude. Moreover, the comparison studies with respect to the Gaussian Process Regression model and the Quantile Regression Forests model demonstrate the applicability and superiority of proposed method. These results highlight the potential of the BMA method to advance battery SOH estimation and facilitate reliable battery management.

Short term hydropower scheduling considering cumulative forecasting deviation of wind and photovoltaic power

The cumulative forecasting errors of wind and photovoltaic (PV) power pose serious challenges to the short-term scheduling of hydropower stations that connected to the same regional power grid. The variable and intermittent nature of wind and PV power necessitate a flexible power source to achieve power balance. Consequently, hydropower, with its flexible adjustment capability, plays a crucial role in supporting the consumption of wind and PV power stations in short-term scheduling. Meanwhile, to address the grid's peak shaving requirement, the scheduling of cascaded hydropower system becomes increasingly complex. Given the limitations of current wind and PV power output prediction technologies, achieving precise forecast remains challenging. In practice, the actual outputs of wind and PV power stations often diverge considerably from predicted values over multiple consecutive periods. This variability may result in the underutilization of wind and PV power if the reserve capacity of hydropower stations is insufficient. Therefore, it is essential to account for the cumulative deviation in electricity from wind and PV sources. By computing probability distribution and establishing boundary condition of the cumulative deviation, the operation of the cascaded hydropower system can be controlled more effectively, thus enhancing the stability of a multi-energy complementary system. This study employs Gaussian Mixture Model (GMM) to characterize the probability distributions of cumulative electricity deviation across multiple periods of wind and PV power and takes it as restriction to regulate the operation of hydropower stations. In order to balance the deviation electricity, the downward electricity reserve, upward electricity reserve and the storage energy of the cascaded reservoirs are introduced as restrictive conditions. The proposed Hydro-wind-PV peak shaving model is then applied to the short-term optimal scheduling of the Lancang River cascaded hydropower system to validate the feasibility and efficacy.

Efficient First-Order Algorithms for Large-Scale, Non-Smooth Maximum Entropy Models with Application to Wildfire Science.

Maximum entropy (MaxEnt) models are a class of statistical models that use the maximum entropy principle to estimate probability distributions from data. Due to the size of modern data sets, MaxEnt models need efficient optimization algorithms to scale well for big data applications. State-of-the-art algorithms for MaxEnt models, however, were not originally designed to handle big data sets; these algorithms either rely on technical devices that may yield unreliable numerical results, scale poorly, or require smoothness assumptions that many practical MaxEnt models lack. In this paper, we present novel optimization algorithms that overcome the shortcomings of state-of-the-art algorithms for training large-scale, non-smooth MaxEnt models. Our proposed first-order algorithms leverage the Kullback-Leibler divergence to train large-scale and non-smooth MaxEnt models efficiently. For MaxEnt models with discrete probability distribution of n elements built from samples, each containing m features, the stepsize parameter estimation and iterations in our algorithms scale on the order of O(mn) operations and can be trivially parallelized. Moreover, the strong ℓ1 convexity of the Kullback-Leibler divergence allows for larger stepsize parameters, thereby speeding up the convergence rate of our algorithms. To illustrate the efficiency of our novel algorithms, we consider the problem of estimating probabilities of fire occurrences as a function of ecological features in the Western US MTBS-Interagency wildfire data set. Our numerical results show that our algorithms outperform the state of the art by one order of magnitude and yield results that agree with physical models of wildfire occurrence and previous statistical analyses of wildfire drivers.

Rotating machine bearing health prognosis using a data driven approach based on KS‐density and BiLSTM

AbstractRolling element bearings are vital components within rotating machinery, making them a central focus of maintenance in the prognostics and health management sector. This involves closely monitoring their condition to accurately predict the remaining useful life, increasing reliability while minimizing unexpected breakdowns, thereby enabling cost savings through planned maintenance, and enhancing operational stability and security. To achieve this goal, it is necessary to build an online intelligent system for degradation monitoring and failure prognosis by the construction of a robust health indicator and making quantitative measure for bearing degradation. In this paper, an efficient and reliable approach is proposed to estimate the remaining useful life of bearing. A new prediction method is presented by the combination of kernel smoothing density (KS‐density) and bidirectional long short‐term memory (BiLSTM). Firstly, KS‐density smoothens the preliminarily estimated probability distribution function using machinery degradation data. Secondly, the obtained KS‐density is used in feed deep learning technique based on BiLSTM models. On this basis, the variation of the signal distribution models between the current faulty state and the normal conditions state is quantified for bearing health assessment. The effective recognition of bearing degradation by the proposed Weibull‐based health index is demonstrated through experimental validations utilizing run‐to‐failure datasets, provided by the centre for intelligent maintenance systems. The comparison with the literature's review show that the prediction results of the proposed approach are more accurate.

Bayesian neural network and Bayesian physics-informed neural network via variational inference for seismic petrophysical inversion

Bayesian neural network via variational inference (BNN-VI) aims to estimate probability distribution of the training parameters of the neural network, which are difficult to compute with traditional methods, by proposing a family of densities and finding the candidates that are close to the target. Variational inference is a Bayesian method that provides a faster tool than Markov chain Monte Carlo sampling algorithms. However, to provide an accurate estimation of the most likely models, BNN-VI requires sufficient training data, which is not always available in geophysical inverse problems. We propose a method that combines a Bayesian approach with a physics-informed neural networks algorithms via variational inference, namely Bayesian physics-informed neural networks via variational inference (BPINN-VI) and apply it to a geophysical inverse problem for the estimation of petrophysical properties from seismic data. The method is based on a Bayesian network approach in which the objective function is defined by the Kullback–Leibler (KL) divergence. In addition, the physical relations between data (seismic measurements) and model variables (petrophysical properties) are embedded into neural networks through a physics-dependent loss term. We tested the proposed method on a pre-stack seismic dataset from the North Sea with limited direct measurements of the target petrophysical properties from well logs. The outcome is the most likely model of petrophysical properties. Compared to BNN-VI, the BPINN-VI results show higher correlation and R2 coefficients, and lower mean square error as well as lower uncertainty and higher lateral continuity.

Identification of illicit discharges in sewer networks by an SWMM-Bayesian coupled approach.

Illicit discharges into sewer systems are a widespread concern within China's urban drainage management. They can result in unforeseen environmental contamination and deterioration in the performance of wastewater treatment plants. Consequently, pinpointing the origin of unauthorized discharges in the sewer network is crucial. This study aims to evaluate an integrative method that employs numerical modeling and statistical analysis to determine the locations and characteristics of illicit discharges. The Storm Water Management Model (SWMM) was employed to track water quality variations within the sewer network and examine the concentration profiles of exogenous pollutants under a range of scenarios. The identification technique employed Bayesian inference fused with the Markov chain Monte Carlo sampling method, enabling the estimation of probability distributions for the position of the suspected source, the discharge magnitude, and the commencement of the event. Specifically, the cases involving continuous release and multiple sources were examined. For single-point source identification, where all three parameters are unknown, concentration profiles from two monitoring sites in the path of pollutant transport and dispersion are necessary and sufficient to characterize the pollution source. For the identification of multiple sources, the proposed SWMM-Bayesian strategy with improved sampling is applied, which significantly improves the accuracy.

StocIPNet: A novel probabilistic interpretable network with affine-embedded reparameterization layer for high-dimensional stochastic inverse problems

The stochastic inverse problem (StocIP), which aims to align push-forward and observed output distributions by estimating probability distributions of unknown system inputs, often faces optimization challenges and the curse of dimensionality. A novel deep network called StocIPNet which comprises an affine-embedded reparameterization subnetwork (ReparNet) and a complex system metamodeling subnetwork (MetaNet) is proposed to alleviate these issues. The ReparNet subnetwork embeds the affine transformation to convert the statistical parameters of the physical random vector into learnable weights and biases, effectively implementing the reparameterization trick by separating random and deterministic elements in the stochastic sampling operation to preserve differentiability. In parallel, the MetaNet subnetwork offers a computationally efficient alternative to time-consuming forward solvers, facilitating the generation of push-forward distributions. The entire StocIPNet utilizes the kernel maximum mean discrepancy (MMD) as a distribution-free loss function, quantifying the discrepancy between push-forward and observed output distributions. By leveraging the ReparNet’s advantage of reformulating the sampling process as a differentiable transformation and combining two subnetworks seamlessly, the StocIP is reconfigured into a pure network training paradigm preserving differentiability perfectly, which allows for direct modeling and efficient inference of uncertainty within the network using automatic differentiation, backpropagation and gradient-based optimization methods, enabling ease of scaling to high-dimensional problems. The proposed framework has been theoretically demonstrated to be equivalent to the maximum likelihood method, ensuring its solid probabilistic interpretable foundation. The proposed framework is applied to perform stochastic model updating on a numerical and an experimental structure, which effectively demonstrates the framework’s remarkable effectiveness and high efficiency in treating high-dimensional StocIPs.

Exponential expansions for approximation of probability distributions

AbstractThis work analyses expansions of exponential form for approximating probability density functions, through the utilization of diverse orthogonal polynomial bases. Notably, exponential expansions ensure the maintenance of positive probabilities regardless of the degree of skewness and kurtosis inherent in the true density function. In particular, we introduce novel findings concerning the convergence of this series towards the true density function, employing mathematical tools of functional statistics. In particular, we show that the exponential expansion is a Fourier series of the true probability with respect to a given orthonormal basis of the so called Bayesian Hilbert space. Furthermore, we present a numerical technique for estimating the coefficients of the expansion, based on the first n exact moments of the corresponding true distribution. Finally, we provide numerical examples that effectively demonstrate the efficiency and straightforward implementability of our proposed approach.

Extracting Latent Variables From Forecast Ensembles and Advancements in Similarity Metric Utilizing Optimal Transport

AbstractThis study presents a novel methodology for extracting latent variables from high‐dimensional sparse data, particularly emphasizing spatial distributions such as precipitation distribution. This approach utilizes multidimensional scaling with a distance matrix derived from a new similarity metric, the Unbalanced Optimal Transport Score (UOTS). UOTS effectively captures discrepancies in spatial distributions while preserving physical units. This is similar to mean absolute error, however it considers location errors, providing a more robust measure crucial for understanding differences between observations, forecasts, and ensembles. Probability distribution estimation of these latent variables enhances the analytical utility, quantifying ensemble characteristics. The adaptability of the method to spatiotemporal data and its ability to handle errors suggest its potential as a promising tool for diverse research applications.

Distributionally robust chance-constrained optimization with Gaussian mixture ambiguity set

Conventional chance-constrained programming methods suffer from the inexactness of the estimated probability distribution of the underlying uncertainty from data. To this end, a distributionally robust approach to the problem allows for a level of ambiguity considered around a reference distribution. In this work, we propose a novel formulation for the distributionally robust chance-constrained programming problem using an ambiguity set constructed from a variant of optimal transport distance that was developed for Gaussian Mixture Models. We show that for multimodal process uncertainty, our proposed method provides an effective way to incorporate statistical moment information into the ambiguity set construction step, thus leading to improved optimal solutions. We illustrate the performance of our method on a numerical example as well as a chemical process case study. We show that our proposed methodology leverages the multimodal characteristics from the uncertainty data to give superior performance over the traditional Wasserstein distance-based method.

Elastic Shape Analysis of Tree-Like 3D Objects Using Extended SRVF Representation.

How can one analyze detailed 3D biological objects, such as neuronal and botanical trees, that exhibit complex geometrical and topological variation? In this paper, we develop a novel mathematical framework for representing, comparing, and computing geodesic deformations between the shapes of such tree-like 3D objects. A hierarchical organization of subtrees characterizes these objects - each subtree has a main branch with some side branches attached - and one needs to match these structures across objects for meaningful comparisons. We propose a novel representation that extends the Square-Root Velocity Function (SRVF), initially developed for Euclidean curves, to tree-shaped 3D objects. We then define a new metric that quantifies the bending, stretching, and branch sliding needed to deform one tree-shaped object into the other. Compared to the current metrics such as the Quotient Euclidean Distance (QED) and the Tree Edit Distance (TED), the proposed representation and metric capture the full elasticity of the branches (i.e., bending and stretching) as well as the topological variations (i.e., branch death/birth and sliding). It completely avoids the shrinkage that results from the edge collapse and node split operations of the QED and TED metrics. We demonstrate the utility of this framework in comparing, matching, and computing geodesics between biological objects such as neuronal and botanical trees. We also demonstrate its application to various shape analysis tasks such as (i) symmetry analysis and symmetrization of tree-shaped 3D objects, (ii) computing summary statistics (means and modes of variations) of populations of tree-shaped 3D objects, (iii) fitting parametric probability distributions to such populations, and (iv) finally synthesizing novel tree-shaped 3D objects through random sampling from estimated probability distributions.

Polynomial approaches in improving accuracy of probability distribution estimation using the method of moments

AbstractBACKGROUNDDetermination of a probability density function (PDF) is an area of active research in engineering sciences as it can improve process systems. A previously developed polynomial method‐of‐moments‐based PDF estimation model has been applied in the research to produce accurate approximations to both standard and more complex PDF. A model with a different polynomial basis than a monomial is still to be developed and evaluated. This is the work that is undertaken in this study.RESULTSA set of standard PDF (Normal, Weibull, Log Normal and Bimodal) and more complex distributions (solutions to the Smoluchowski coagulation equation and Population Balance equation) were approximated by the method‐of‐moments using Chebyshev, Hermite and Lagrange polynomial‐based density functions. Results show that Lagrange polynomial‐based models improve the fit compared to monomial based‐modeling in terms of RMSE and Kolmogorov–Smirnov test statistic estimates. The Kolmogorov–Smirnov test‐statistics decreased by 19% and the RMSE values were improved by around 85% compared to the standard monomial basis when using Lagrange polynomial basis.CONCLUSIONThis study indicates that the procedure using Lagrange polynomials with method‐of‐moments is a more reliable reconstruction procedure that calculates the approximate distribution using lesser number of moments, which is desirable. © 2024 The Authors. Journal of Chemical Technology and Biotechnology published by John Wiley & Sons Ltd on behalf of Society of Chemical Industry (SCI).

이산확률분포의 표본 분위수 계산 방법에 관한 연구

In this paper, we studied the methods of computing sample quantiles for the case of discrete probability distributions. There are two main methods which are introduced in statistics textbooks. We compared these two methods by simulation from practical and educational necessity. We considered some cases of Poisson, binomial, geometric, negative binomial, and discrete uniform distributions by setting up the parameters, and for each distribution we performed 10,000 times of simulation of drawing random samples of size 20 and 50. At each time of simulation, we computed the difference between the population quantile and the sample quantile obtained by each method. We compared the two methods by using two criteria: one is the mean square error over 10,000 times of simulation, and the other is the frequency of obtaining closer sample quantile to the population quantile than the competing method. We also obtained the estimated probabilities of exact estimation of population quantiles for the two methods, and performed the tests of homogeneity on the probability distribution of under-estimation, exact estimation, and over-estimation of the population quantiles.

Estimation of Circular Statistics in the Presence of Measurement Bias.

Circular statistics and Rayleigh tests are important tools for analyzing cyclic events. However, current methods are not robust to significant measurement bias, especially incomplete or otherwise non-uniform sampling. One example is studying 24-cyclicity but having data not recorded uniformly over the full 24-hour cycle. Our objective is to present a robust method to estimate circular statistics and their statistical significance in the presence of incomplete or otherwise non-uniform sampling. Our method is to solve the underlying Fredholm Integral Equation for the more general problem, estimating probability distributions in the context of imperfect measurements, with our circular statistics in the presence of incomplete/non-uniform sampling being one special case. The method is based on linear parameterizations of the underlying distributions. We simulated the estimation error of our approach for several toy examples as well as for a real-world example: analyzing the 24-hour cyclicity of an electrographic biomarker of epileptic tissue controlled for states of vigilance. We also evaluated the accuracy of the Rayleigh test statistic versus the direct simulation of statistical significance. Our method shows a very low estimation error. In the real-world example, the corrected moments had a root mean square error of [Formula: see text]. In contrast, the Rayleigh test statistic overestimated the statistical significance and was thus not reliable. The presented methods thus provide a robust solution to computing circular moments even with incomplete or otherwise non-uniform sampling. Since Rayleigh test statistics cannot be used in this circumstance, direct estimation of significance is the preferable option for estimating statistical significance.

Exponential Expansions for Approximation of Probability Distributions

Exponential Expansions for Approximation of Probability Distributions

Land use and land cover change simulation enhanced by asynchronous communicating cellular automata

Land use and land cover change (LUCC) modeling is crucial to urban planning and policy making. A well-used and effective paradigm for LUCC is the ANN-CA model which employs an artificial neural network (ANN) to calculate a transition probability over each land cell from various driving factors, and then uses a cellular automaton (CA) to evolve all cells consecutively to update their land usage and coverage according to the estimated probability distribution. This paper focuses on the effect of delays or perturbations possibly taking place between land cells on the LUCC modeling. To this end, a new ANN-CA model is proposed which adopts an asynchronous communicating cellular automaton (ACCA), rather than the conventional synchronous CA. Especially, the ACCA allows every cell to communicate with its neighbors independently at random times via a specific protocol, which offers a natural way to include stochastic delays in exchanging the current land usages between cells. As a result, every change of a cell's land use in the ACCA may not affect its neighbors immediately, but is subject to delays that might play an important role in modeling the practical LUCC. Numerical analysis of the new model are carried out over three regions of Chongqing city, China with different scales: Yongchuan District, Sanjiao Town, and Huangguashan Village, and experimental results demonstrate that the proposed ANN-ACCA model can achieve a higher accuracy for LUCC simulation as compared to conventional ANN-CA models.

Estimation of site-specific multivariate probability distribution of soil properties using a mixed sampling technique

Estimation of multivariate probability density function (PDF) of soil properties based on limited site investigation data plays an important role in data driven-based geotechnical engineering analysis and design. However, this estimation is often intractable because the site-specific investigation data generally are multivariate, uncertain and unique, sparse, incomplete, sometimes corrupt and spatially variable. The estimation is more challenging when the uncertainties in model parameters of the distribution (i.e., mean, standard deviations and scale of fluctuation of soil properties) are taken into consideration. To address these difficulties, this study proposed a mixed sampling technique by combing the Gibbs sampler and transitional Markov Chain Monte Carlo sampler to effectively estimate the multivariate probability density function of soil properties. The proposed method is demonstrated through a virtual site example while the effect of statistical uncertainty in the scale of fluctuation on reliability analyses is illustrated in an infinite slope example. It is shown that the proposed method can not only handle the complex features of site investigation data but also effectively quantify the statistical uncertainties of the mean values, standard deviations and scale of fluctuation of the investigated soil properties. Ignoring the statistical uncertainties of scale of fluctuation can affect the estimation of mean and standard deviations of the soil parameters. When considering the statistical uncertainty of scale of fluctuation, the 95% confidence intervals for the profiles of the soil properties can rationally cover the actual soil data at the unobserved points. Ignoring the statistical uncertainty of scale of fluctuation can lead to the underestimation of the probability of failure of the infinite slope.