Estimation Of Probability Density Function Research Articles

Deep learning-based ensemble forecast and predictability analysis of the Kuroshio intrusion into the South China Sea

Abstract The Kuroshio intrusion (KI) into the South China Sea (SCS) significantly affects the environment, ecology, and climate change of the SCS. However, due to the nonlinearity of KI, its numerical prediction often requires large ensemble size to measure prediction uncertainty. The huge computational costs of large numbers of members and high-resolution numerical models pose significant challenges for KI prediction. We, therefore, construct a Kuroshio ensemble deep learning prediction system (KurNet) through taking different values of parameters to predict KI paths because the deep learning models have high prediction skills and low computational cost. The KurNet containing 64 ensemble members can not only output ensemble mean forecast result of the Kuroshio path, but also estimate probability density functions for the path types. The KurNet illustrates a high predictive ability for the KI with the mean classification accuracy of 71.9% and root mean square error of 0.913 on the testing set, which is superior to the single control prediction by ∼1.0–2.9%, although the control prediction model is selected as one of the ensemble members with the best predictive ability on the validation set. Furthermore, the predictability analysis of 10 KI events indicates that when the lead time is 3 days, the most important areas are in the east of Luzon Island due to the upstream Kuroshio transport. As the lead time increases, the most important area is in the Luzon Strait due to the eddy activity. Observing system simulation experiments reveal that the KI forecast skill can be enhanced by ∼12–18%, when uncertainties of the input data in these important regions are removed.

Polynomial approaches in improving accuracy of probability distribution estimation using the method of moments

AbstractBACKGROUNDDetermination of a probability density function (PDF) is an area of active research in engineering sciences as it can improve process systems. A previously developed polynomial method‐of‐moments‐based PDF estimation model has been applied in the research to produce accurate approximations to both standard and more complex PDF. A model with a different polynomial basis than a monomial is still to be developed and evaluated. This is the work that is undertaken in this study.RESULTSA set of standard PDF (Normal, Weibull, Log Normal and Bimodal) and more complex distributions (solutions to the Smoluchowski coagulation equation and Population Balance equation) were approximated by the method‐of‐moments using Chebyshev, Hermite and Lagrange polynomial‐based density functions. Results show that Lagrange polynomial‐based models improve the fit compared to monomial based‐modeling in terms of RMSE and Kolmogorov–Smirnov test statistic estimates. The Kolmogorov–Smirnov test‐statistics decreased by 19% and the RMSE values were improved by around 85% compared to the standard monomial basis when using Lagrange polynomial basis.CONCLUSIONThis study indicates that the procedure using Lagrange polynomials with method‐of‐moments is a more reliable reconstruction procedure that calculates the approximate distribution using lesser number of moments, which is desirable. © 2024 The Authors. Journal of Chemical Technology and Biotechnology published by John Wiley & Sons Ltd on behalf of Society of Chemical Industry (SCI).

Two-Phase Adaptive Kriging Model Based Importance Sampling Method for Estimating Time-Dependent Failure Probability

To improve the computational efficiency of the time-dependent failure probability (TDFP) to system with dynamic loads, materials degeneration and other time-varying factors, a two-phase adaptive Kriging model-based importance sampling (IS) method is proposed. The main contribution of this article is that a single Kriging model is established and adaptively refined to gradually approach the optimal IS probability density function (PDF) and estimate TDFP in two sequential phases. In the first phase, a generally approximate IS PDF is constructed by Kriging model based on its prediction characteristic and subsequently updated until convergence. Then, a simple rejection sampling technique is used to robustly generate IS samples for the estimation of TDFP. In the second phase, current Kriging model is subsequently refined for accurately identifying the classification of the IS samples generated in the last phase. Finally, the TDFP can be estimated using the products obtained in the above two phases. As the IS PDF established by Kriging model is used for directly approximating the optimal one, the proposed method has the capacity to deal with the complicated problem containing multiple failure regions. Additionally, since IS procedure can dramatically improve the sampling efficiency, the proposed method is an effective technique to the time-consuming TDFP analysis problems.

A scaled kernel density estimation prior for dynamic borrowing of historical information with application to clinical trial design.

Incorporating historical data into a current data analysis can improve estimation of parameters shared across both datasets and increase the power to detect associations of interest while reducing the time and cost of new data collection. Several methods for prior distribution elicitation have been introduced to allow for the data-driven borrowing of historical information within a Bayesian analysis of the current data. We propose scaled Gaussian kernel density estimation (SGKDE) prior distributions as potentially more flexible alternatives. SGKDE priors directly use posterior samples collected from a historical data analysis to approximate probability density functions, whose variances depend on the degree of similarity between the historical and current datasets, which are used as prior distributions in the current data analysis. We compare the performances of the SGKDE priors with some existing approaches using a simulation study. Data from a recently completed phase III clinical trial of a maternal vaccine for respiratory syncytial virus are used to further explore the properties of SGKDE priors when designing a new clinical trial while incorporating historical data. Overall, both studies suggest that the new approach results in improved parameter estimation and power in the current data analysis compared to the considered existing methods.

Data-based approach for time-correlated closures of turbulence models.

Developed turbulent motion of fluid still lacks an analytical description despite more than a century of active research. Nowadays, phenomenological ideas are widely used in practical applications, such as small-scale closures for numerical simulations of turbulent flows. In the present paper, we use a shell model of turbulence to construct a closure intended to have a solid theoretical background and to capture intrinsic probabilistic features of turbulence. Shell models of turbulence are dynamical deterministic systems used to model energy cascade and other key aspects of the Navier-Stokes such as intermittency. We rescale the variables of the Sabra model in a way which leads to hidden symmetries and universal distributions. We then use such fine distributions to write closures, i.e., missing expressions for some of the Sabra variables. Our closures rely on approximating probability density functions using a Gaussian mixture model, which makes them probabilistic by nature and allows us to write time-correlated closures. We also provide a framework where other machine learning tools can be employed with reduced black-box aspects.

Multivariate Density Estimation by Neural Networks.

We propose nonparametric methods to obtain the Probability Density Function (PDF) to assess the properties of the underlying data generating process (DGP) without imposing any assumptions on the DGP, using neural networks (NNs). The proposed NN has advantages compared to well-known parametric and nonparametric density estimators. Our approach builds on literature on cumulative distribution function (CDF) estimation using NN. We extend this literature by providing analytical derivatives of this obtained CDF. Our approach hence removes the numerical approximation error in differentiating the CDF output, leading to more accurate PDF estimates. The proposed solution applies to any NN model, i.e., for any number of hidden layers or hidden neurons in the multilayer perceptron (MLP) structure. The proposed solution applies the PDF estimation by NN to continuous distributions as well as discrete distributions. We also show that the proposed solution to obtain the PDF leads to good approximations when applied to correlated variables in a multivariate setting. We test the performance of our method in a large Monte Carlo simulation using various complex distributions. Subsequently, we apply our method to estimate the density of the number of vehicle counts per minute measured with road sensors for a time window of 24 h.

Entropy-metric estimation of the small data models with stochastic parameters

The formalization of dependencies between datasets, taking into account specific hypotheses about data properties, is a constantly relevant task, which is especially acute when it comes to small data. The aim of the study is to formalize the procedure for calculating optimal estimates of probability density functions of parameters of linear and nonlinear dynamic and static small data models, created taking into account specific hypotheses regarding the properties of the studied object. The research methodology includes probability theory and mathematical statistics, information theory, evaluation theory, and stochastic mathematical programming methods. The mathematical apparatus presented in the article is based on the principle of maximization of information entropy on sets determined as a result of a small number of censored measurements of “input" and “output" entities in the presence of noise. These data structures became the basis for the formalization of linear and nonlinear dynamic and static models of small data with stochastic parameters, which include both controlled and noise-oriented input and output measurement entities. For all variants of the above-mentioned small data models, the tasks of determining the optimal estimates of the probability density functions of the parameters were carried out. Formulated optimization problems are reduced to the forms canonical for the stochastic linear programming problem with probabilistic constraints.

Shape Constrained Kernel PDF and PMF Estimation

We consider shape constrained kernel-based probability density function (PDF) and probability mass function (PMF) estimation. Our approach is of widespread potential applicability and includes, separately or simultaneously, constraints on the PDF (PMF) function itself, its integral (sum), and derivatives (finite-differences) of any order. We also allow for pointwise upper and lower bounds (i.e., inequality constraints) on the PDF and PMF in addition to more popular equality constraints, and the approach handles a range of transformations of the PDF and PMF including, for example, logarithmic transformations (which allows for the imposition of log-concave or log-convex constraints that are popular with practitioners). Theoretical underpinnings for the procedures are provided. A simulation-based comparison of our proposed approach with those obtained using Grenander-type methods is favourable to our approach when the DGP is itself smooth. As far as we know, ours is also the only smooth framework that handles PDFs and PMFs in the presence of inequality bounds, equality constraints, and other popular constraints such as those mentioned above. An implementation in R exists that incorporates constraints such as monotonicity (both increasing and decreasing), convexity and concavity, and log-convexity and log-concavity, among others, while respecting finite-support boundaries via explicit use of boundary kernel functions.

Estimation of Representative Wave Period and Optimal Probability Density Function Using Wave Observed Data around Korean Western Coast

In this study, the peak wave period <i>T<sub>p</sub></i> and mean wave period <i>T</i><sub>02</sub> and <i>T</i><sub><i>m</i>-1, 0</sub>, which are major parameters for classifying ocean characteristics, were calculated using water surface elevation data observed from the second west coast oceanographic and meteorological observation tower. In addition, the ratio of abnormal data, correlation analysis, and optimal probability density function were estimated. In the case of <i>T<sub>p</sub></i> among the calculated representative periods, the proportion of abnormal data was 5.73% and 0.67% at each point, and <i>T</i><sub>02</sub> was 4.35% and 0.01%. <i>T</i><sub><i>m</i>-1, 0</sub> was found to be 2.82% and 0.03%. Meanwhile, as a result of analyzing the relationship between <i>T</i><sub>02</sub> and <i>T<sub>p</sub></i>, the relationship was calculated to be 0.53 and 0.63 for each point. The relationship between <i>T</i><sub><i>m</i>-1, 0</sub> and <i>T<sub>p</sub></i> was 1.15 and 1.32, respectively, and <i>T</i><sub>02</sub>, <i>T</i><sub><i>m</i>-1, 0</sub> was 1.18 and 1.22. As a result of estimating the optimal probability density function of the calculated representative period, <i>T<sub>p</sub></i> followed the ‘Log-normal’ and ‘Normal’ distributions at each point, and <i>T</i><sub>02</sub> was ‘Gamma’, ‘Normal’ distribution and <i>T</i><sub><i>m</i>-1, 0</sub> showed that ‘Log-normal’ and ‘Normal’ distribution were dominant, respectively. It is decided that these results can be used as basic data for wave analysis conducted on the west coast.

Probabilistic Load Flow Analysis Using Nonparametric Distribution

In the pursuit of sustainable energy solutions, this research addresses the critical need for accurate probabilistic load flow (PLF) analysis in power systems. PLF analysis is an essential tool for estimating the statistical behavior of power systems under uncertainty. It plays a vital part in power system planning, operation, and dependability studies. To perform accurate PLF analysis, this article proposes a Kernel density estimation with adaptive bandwidth for probability density function (PDF) estimation of power injections from sustainable energy sources like solar and wind, reducing errors in PDF estimation. To reduce the computational burden, a Latin hypercube sampling approach was incorporated. Input random variables are modeled using kernel density estimation (KDE) in conjunction with Latin hypercube sampling (LHS) for probabilistic load flow (PLF) analysis. To test the proposed techniques, IEEE 14 and IEEE 118 bus systems are used. Two benchmark techniques, the Monte Carlo Simulation (MCS) method and Hamiltonian Monte Carlo (HMC), were set side by side for validation of results. The results illustrate that an adaptive bandwidth kernel density estimation with the Latin hypercube sampling (AKDE-LHS) method provides better performance in terms of precision and computational efficiency. The results also show that the suggested technique is more feasible in reducing errors, uncertainties, and computational time while depicting arbitrary distributions of photovoltaic and wind farms for probabilistic load flow analysis. It can be a potential solution to tackle challenges posed by sustainable energy sources in power systems.

Performance estimation technique for solar-wind hybrid systems: A machine learning approach

This study introduces a novel method to assess solar-wind hybrid energy systems, utilizing data analysis for key metrics (Performance Yields YR, Capacity Factor CF, and Performance Ratio PR), focused on their uncertainties estimation. Employing a bootstrapped-based K-means algorithm on features from a local weather station in Universidad de La Salle, Bogotá, the approach evaluates the technical feasibility of a photovoltaic system on the campus. It discusses the clustering algorithm advantages, emphasizing its suitability for modeling continuous processes in renewable energy systems. The hypothesis assumes independence and identically distributed centroid coordinates, aligning with large-scale energy system characteristics. Methods include data normalization, feature selection, statistical calculations, bootstrapping, and Probability Density Functions (PDF) estimation through a Maximum Likelihood Estimator (MLE) to assess uncertainties. Results suggest the unfeasibility of a wind energy system due to technological constraints. The study evaluates system flexibility, and design considerations, providing valuable decision-making insights. The conclusion highlights the potential use for accurate estimations, enhancing the system reliability.

Gaussian process metamodel and Markov chain Monte Carlo-based Bayesian inference framework for stochastic nonlinear model updating with uncertainties

The estimation of the posterior probability density function (PDF) of unknown parameters remains a challenge in stochastic nonlinear model updating with uncertainties; thus, a novel Bayesian inference framework based on the Gaussian process metamodel (GPM) and the advanced Markov chain Monte Carlo (MCMC) method is proposed in this paper. The instantaneous characteristics (ICs) of the decomposed measurement response, calculated using the Hilbert transform and the discrete analytical mode decomposition methodology, are extracted as nonlinear indices and further used to construct the likelihood function. Then, the posterior PDFs of structural nonlinear model parameters are derived based on the Bayesian theorem. To precisely calculate the posterior PDF, an advanced MCMC approach, i.e., delayed rejection adaptive Metropolis-Hastings (DRAM) algorithm, is adopted with the advantages of a high acceptance ratio and strong robustness. However, as a common shortage in most MCMC methods, the resampling technology is still applied, and numerous iterations of nonlinear simulations are conducted to ensure accuracy, thus directly reducing the computational efficiency of the DRAM. To address the abovementioned issue, a mathematical regression metamodel of the GPM with a polynomial kernel function is adopted in this paper instead of the traditional finite element model (FEM) to simulate a nonlinear response for the reduction of computational cost, and the hyperparameters are further estimated using the conjugate gradient optimization methodology. Finally, numerical simulations concerning a Giuffré–Menegotto–Pinto modeled steel-frame structure and a seven-storey base-isolated structure are conducted. Furthermore, a shake-table experimental test of a nonlinear steel framework is investigated to validate the accuracy of the Bayesian inference method. Both simulations and experiment demonstrate that the proposed GPM and DRAM-based Bayesian method effectively estimates the posterior PDF of unknown parameters and is appropriate for stochastic nonlinear model updating even with multisource uncertainties.

Dynamic Risk Assessment of Voltage Violation in Distribution Networks with Distributed Generation.

In response to the growing demand for economic and social development, there has been a significant increase in the integration of distributed generation (DG) into distribution networks. This paper proposes a dynamic risk assessment method for voltage violations in distribution networks with DG. Firstly, considering the characteristics of random variables such as load and DG, a probability density function estimation method based on boundary kernel density estimation is proposed. This method accurately models the probability of random variables under different time and external environmental conditions, such as wind speed and global horizontal radiation. Secondly, to address the issue of correlated DG in the same region, an independent transformation method based on the Rosenblatt inverse transform is proposed, which enhances the accuracy of probabilistic load flow. Thirdly, a voltage violation severity index based on the utility function is proposed. This index, in combination with probabilistic load flow results, facilitates the quantitative assessment of voltage violation risks. Finally, the accuracy of the proposed method is verified on the IEEE-33 system.

Estimation of site-specific multivariate probability distribution of soil properties using a mixed sampling technique

Estimation of multivariate probability density function (PDF) of soil properties based on limited site investigation data plays an important role in data driven-based geotechnical engineering analysis and design. However, this estimation is often intractable because the site-specific investigation data generally are multivariate, uncertain and unique, sparse, incomplete, sometimes corrupt and spatially variable. The estimation is more challenging when the uncertainties in model parameters of the distribution (i.e., mean, standard deviations and scale of fluctuation of soil properties) are taken into consideration. To address these difficulties, this study proposed a mixed sampling technique by combing the Gibbs sampler and transitional Markov Chain Monte Carlo sampler to effectively estimate the multivariate probability density function of soil properties. The proposed method is demonstrated through a virtual site example while the effect of statistical uncertainty in the scale of fluctuation on reliability analyses is illustrated in an infinite slope example. It is shown that the proposed method can not only handle the complex features of site investigation data but also effectively quantify the statistical uncertainties of the mean values, standard deviations and scale of fluctuation of the investigated soil properties. Ignoring the statistical uncertainties of scale of fluctuation can affect the estimation of mean and standard deviations of the soil parameters. When considering the statistical uncertainty of scale of fluctuation, the 95% confidence intervals for the profiles of the soil properties can rationally cover the actual soil data at the unobserved points. Ignoring the statistical uncertainty of scale of fluctuation can lead to the underestimation of the probability of failure of the infinite slope.

Probabilistic assessment of seismic earth pressure against backfills with a nonlinear failure criterion

A novel approach for the probabilistic assessment of seismic earth pressure against nonlinear backfills is introduced in this study. To implement reliability-based design for the retaining structures under seismic loading condition, nonlinear upper bound analysis is adopted to obtain the seismic earth pressure through optimization procedure. Firstly, rigorous analytical function is formulated to reveal the normal-shear stress information along the failure surface in soil with nonlinear failure criterion. Subsequently, a kinematic equation is put forward to estimate the seismic earth pressures by balancing the external work rate and the internal energy dissipation rate. The solutions are verified by the Discontinuity Layout Optimization numerical modellings based on the derived stress data from the proposed method. To consider the probability analysis of nonlinear backfill properties, the moment method is presented by combining the adaptive dimension decomposition with the direct integral method. According to the estimated first four statistical moments, the approximated probability density function of the performance function is determined contently. Finally, the failure probability of seismic earth pressure is calculated based on the proposed moment method. Two numerical examples demonstrate that the moment method can be adapted to the characteristics of nonlinear backfills and further improve the accuracy of reliability estimation by introducing higher-order moments analytically.

A diffusion-based kernel density estimator (diffKDE, version 1) with optimal bandwidth approximation for the analysis of data in geoscience and ecological research

Abstract. Probability density functions (PDFs) provide information about the probability of a random variable taking on a specific value. In geoscience, data distributions are often expressed by a parametric estimation of their PDF, such as, for example, a Gaussian distribution. At present there is growing attention towards the analysis of non-parametric estimation of PDFs, where no prior assumptions about the type of PDF are required. A common tool for such non-parametric estimation is a kernel density estimator (KDE). Existing KDEs are valuable but problematic because of the difficulty of objectively specifying optimal bandwidths for the individual kernels. In this study, we designed and developed a new implementation of a diffusion-based KDE as an open source Python tool to make diffusion-based KDE accessible for general use. Our new diffusion-based KDE provides (1) consistency at the boundaries, (2) better resolution of multimodal data, and (3) a family of KDEs with different smoothing intensities. We demonstrate our tool on artificial data with multiple and boundary-close modes and on real marine biogeochemical data, and compare our results against other popular KDE methods. We also provide an example for how our approach can be efficiently utilized for the derivation of plankton size spectra in ecological research. Our estimator is able to detect relevant multiple modes and it resolves modes that are located closely to a boundary of the observed data interval. Furthermore, our approach produces a smooth graph that is robust to noise and outliers. The convergence rate is comparable to that of the Gaussian estimator, but with a generally smaller error. This is most notable for small data sets with up to around 5000 data points. We discuss the general applicability and advantages of such KDEs for data–model comparison in geoscience.

Estimation of Fracture Size and Probability Density Function by Setting Scanlines in Rectangular Sampling Window

Abstract Rock masses are very important materials in geotechnical engineering. In engineering rock mass, fracture is the relatively weak part of mechanical strength in rock mass and is the most important factor controlling the deformation, damage, and permeability of rock mass. Therefore, investigating fractures is very important for characterizing rock mass. This paper proposed a new approach by using uniformly equidistant orthogonal scanlines. Within the study context, the solution formula of fracture size is derived by establishing the space intersection model of arbitrary fracture and scanline, rectangular window, and a rectangular box with a rectangular window. Then, fractures were randomly generated in a certain size cube and compared with the traditional Kulatilake trace length integral evaluation method. The study results have shown that the proposed method is more reasonable and accurate. Then, this method was applied to an adit of Songta Hydropower Station. Finally, a new fracture diameter probability density estimation method was proposed, the fracture diameter of the normal distribution was verified, and the parameters of the probability density function obtained by the scanlines method were in agreement with the initial set parameters. In summary, the proposed scanlines method can well estimate the mean value of the fracture diameter and the probability density function of the fracture size.

Probabilistic and semantic descriptions of image manifolds and their applications

This paper begins with a description of methods for estimating probability density functions for images that reflects the observation that such data is usually constrained to lie in restricted regions of the high-dimensional image space—not every pattern of pixels is an image. It is common to say that images lie on a lower-dimensional manifold in the high-dimensional space. However, although images may lie on such lower-dimensional manifolds, it is not the case that all points on the manifold have an equal probability of being images. Images are unevenly distributed on the manifold, and our task is to devise ways to model this distribution as a probability distribution. In pursuing this goal, we consider generative models that are popular in AI and computer vision community. For our purposes, generative/probabilistic models should have the properties of (1) sample generation: it should be possible to sample from this distribution according to the modeled density function, and (2) probability computation: given a previously unseen sample from the dataset of interest, one should be able to compute the probability of the sample, at least up to a normalizing constant. To this end, we investigate the use of methods such as normalizing flow and diffusion models. We then show how semantic interpretations are used to describe points on the manifold. To achieve this, we consider an emergent language framework that makes use of variational encoders to produce a disentangled representation of points that reside on a given manifold. Trajectories between points on a manifold can then be described in terms of evolving semantic descriptions. In addition to describing the manifold in terms of density and semantic disentanglement, we also show that such probabilistic descriptions (bounded) can be used to improve semantic consistency by constructing defenses against adversarial attacks. We evaluate our methods on CelebA and point samples for likelihood estimation with improved semantic robustness and out-of-distribution detection capability, MNIST and CelebA for semantic disentanglement with explainable and editable semantic interpolation, and CelebA and Fashion-MNIST to defend against patch attacks with significantly improved classification accuracy. We also discuss the limitations of applying our likelihood estimation to 2D images in diffusion models.

Extropy based inaccuracy measure in order statistics

In this paper, we provide a measure based on inaccuracy between distributions of the ith order statistic and the parent random variable. This measure characterizes the distribution function of parent random variable uniquely. We demonstrate that the extropy of the parent random variable is the average of the accuracy measure. It is also shown that the measure of inaccuracy defined is invariant under scale but not under location transformation. Nonparametric estimators for the proposed measures are also obtained. A Monte Carlo simulation study is performed to verify the performance of the suggested estimators. Simulation results show that the estimator based on the reflection boundary technique for probability density function estimation and the empirical method for cumulative distribution function estimation has the best performance among estimators. Also, a real dataset is considered to show an application of the proposed estimators on model selection.

Optimal Bandwidth Selection Methods with Application to Wind Speed Distribution

Accurate estimation of the unknown probability density functions of critical variables, such as wind speed—which plays a pivotal role in harnessing clean energy—is essential for various scientific and practical applications. This research conducts a comprehensive comparative analysis of seven distinct bandwidth calculation techniques across various normal distributions, using simulation as the evaluation method in the context of Kernel Density Estimation (KDE). This analysis includes the calculation of the optimal bandwidth and assessment of the performance of these methods with respect to Mean Squared Error (MSE), bias, and the optimal bandwidth value. The findings reveal that among the various bandwidth methods evaluated, the Bandwidth bandwidth-based Cross-Validation (BCV), especially for small sample sizes, consistently provides the closest result to the optimal bandwidth across most of the applied normal distributions. These results provide valuable insights into the selection of optimal bandwidths for accurate and reliable density estimation in the context of normal distributions. Another key aspect of this work is the extension of these methods to wind speed data in a specific region. Monthly wind speed kernel density estimates obtained using all seven bandwidth selection techniques show that Smoothed Cross-Validation (SCV) is suited for this type of real-world data.