Conditional Probability Density Research Articles

Data Assimilation Networks

AbstractData Assimilation aims at estimating the posterior conditional probability density functions based on error statistics of the noisy observations and the dynamical system. State of the art methods are sub‐optimal due to the common use of Gaussian error statistics and the linearization of the non‐linear dynamics. To achieve a good performance, these methods often require case‐by‐case fine‐tuning by using explicit regularization techniques such as inflation and localization. In this paper, we propose a fully data driven deep learning framework generalizing recurrent Elman networks and data assimilation algorithms. Our approach approximates a sequence of prior and posterior densities conditioned on noisy observations using a log‐likelihood cost function. By construction our approach can then be used for general nonlinear dynamics and non‐Gaussian densities. As a first step, we evaluate the performance of the proposed approach by using fully and partially observed Lorenz‐95 system in which the outputs of the recurrent network are fitted to Gaussian densities. We numerically show that our approach, without using any explicit regularization technique, achieves comparable performance to the state‐of‐the‐art methods, IEnKF‐Q and LETKF, across various ensemble size.

Фильтрация сигналов при скачкообразных мультипликативных помехах

The problem of constructing the approximately optimal Bayesian algorithm of filtering the output signal for a linear stochastic dynamical system is solved. Non-linear mixture of the output signal and of the multiplicative interference was measured. This interference is a continuous-valued random process with the unknown distribution law within the given limits of [c(1), c(2)], c(1) > 0, c(2) > 0, and in the frequency range of [0, ∆ω]. The filtering algorithm is synthesized by the approximately optimal signal estimation method based on the theory of systems with random jump structure and the method of two-time parametric approximation of the phase coordinates probability distribution. The method consists in approximate replacement of the unknown probability densities of phase coordinates by the known distribution laws with the unknown mathematical expectations and covariances determined as a result of solving the problem. The proposed approximate-optimal filtering algorithm is based on replacement of the continuous-valued multiplicative interference by a random jump process, i.e., Markov chain with two states c(1), c(2) and equal intensities of transitions from one state to another q′ = 2∆ω. Conditional probability densities of the output signal for fixed states of the Markov chain c(1), c(2) are approximated by gamma distributions that depend on mathematical expectations and variances of the x(t) signal at the fixed c(1), c(2) and measurements x(t) in a mixture with the Markov jump interference. An example of constructing an algorithm for evaluating the distance from an aircraft to the object based on object irradiance measurement by the infrared direction finder was considered. Irradiance was equal to the radiation strength ratio to the square of the distance. Radiation strength, i.e., multiplicative interference, was the random process with unknown continuous distribution in a limited range. The approximate-optimal filter was constructed for distance evaluation by replacing this process with a two-state Markov chain and approximating the distance probability density by the Rayleigh distribution

The mechanical behaviors of random curved fiber networks by numerical simulations

Random curved fiber networks (RCFNs) are ubiquitous in biological materials and artificial composites, attracting much attention because of their unique mechanical properties and functionalities. Based on numerical simulations, the effects of fiber morphology, fiber aspect ratio and relative density on mechanical properties of two dimensional (2D) RCFNs are investigated in terms of the elastic modulus, yield strength as well as energy ratio of stretching to bending deformation. Based on the conditional probability density function (CPDF), a general theoretical model is extended for calculating the crosslink density of fiber networks, which elucidates the independence of crosslink density on fiber shape in RCFNs. According to numerical results, the elastic modulus and yield strength decrease with rising fiber curvature and are enhanced by increasing of aspect ratio and relative density. The influence of fiber aspect ratio in RCFNs exhibits thresholds, beyond which the mechanical properties will reach upper limits for constant curvature. The scaled yield strength to modulus of RCFNs versus curvature is found to collapse into a nonlinear curve regardless of aspect ratios. The energy ratio is influenced by fiber curvature more vigorously which is explained by deformations of curved segments and load paths deviations. This paper is expected to contribute to deep understanding of relationships between overall mechanical properties and structural parameters of RCFNs.

Time-variant reliability global sensitivity analysis with single-loop Kriging model combined with importance sampling

This paper develops a novel failure probability-based global sensitivity index by introducing the Bayes formula into the moment-independent global sensitivity index to approximate the effect of input random variables or stochastic processes on the time-variant reliability. The proposed global sensitivity index can estimate the effect of uncertain inputs on the time-variant reliability by comparing the difference between the unconditional probability density function of input variables and the conditional probability density function in failure state of input variables. Furthermore, a single-loop active learning Kriging method combined with metamodel-based importance sampling is employed to improve the computational efficiency. The accuracy of the results obtained by Kriging model is verified by the reference results provided by the Monte Carlo simulation. Four examples are investigated to demonstrate the significance of the proposed failure probability-based global sensitivity index and the effectiveness of the computational method.

Proximal Decoding for LDPC Codes

This paper presents a novel optimization-based decoding algorithm for LDPC codes. The proposed decoding algorithm is based on a proximal gradient method for solving an approximate maximum a posteriori (MAP) decoding problem. The key idea of the proposed algorithm is the use of a code-constraint polynomial to penalize a vector far from a codeword as a regularizer in the approximate MAP objective function. A code proximal operator is naturally derived from a code-constraint polynomial. The proposed algorithm, called proximal decoding, can be described by a simple recursive formula consisting of the gradient descent step for a negative log-likelihood function corresponding to the channel conditional probability density function and the code proximal operation regarding the code-constraint polynomial. Proximal decoding is experimentally shown to be applicable to several non-trivial channel models such as LDPC-coded massive MIMO channels, correlated Gaussian noise channels, and nonlinear vector channels. In particular, in MIMO channels, proximal decoding outperforms known massive MIMO detection algorithms, such as an MMSE detector with belief propagation decoding. The simple optimization-based formulation of proximal decoding allows a way for developing novel signal processing algorithms involving LDPC codes.

Two-stage conditional density estimation based on Bernstein polynomials

Two-stage conditional probability density function estimators are proposed and studied. Specifically, the Nadaraya-Watson (NW) and local linear (LL) conditional distribution function estimators have been smoothed using Bernstein polynomials in the first stage. Second, the proposed estimators are obtained by differentiating NW and LL estimators. The asymptotic properties of these estimators are established such as asymptotic bias, variance, and normality. Finally, a simulation study is carried out to assess the relative advantage of our estimators compared to other competitors. In addition, the well-known Old Faithful Geyser data is analyzed using the proposed estimators.

A Bivariate Extension to Exponentiated Inverse Flexible Weibull Distribution: Shock Model, Features, and Inference to Model Asymmetric Data

The primary objective of this article was to introduce a new probabilistic model for the discussion and analysis of random covariates. The introduced model was derived based on the Marshall–Olkin shock model. After proposing the mathematical form of the new bivariate model, some of its distributional properties, including joint probability distribution, joint reliability distribution, joint reversed (hazard) rate distribution, marginal probability density function, conditional probability density function, moments, and distributions for both Y=max{X1,X2} and W=min{X1,X2}, were investigated. This novel model can be applied to discuss and evaluate symmetric and asymmetric data under various kinds of dispersion. Moreover, it can be used as a probability approach to analyze different shapes of hazard rates. The maximum likelihood approach was utilized for estimating the parameters of the bivariate model. A simulation study was carried out to assess the performance of the parameters, and it was noted that the maximum likelihood technique can be used to generate consistent estimators. Finally, two real datasets were analyzed to illustrate the notability of the novel bivariate distribution, and it was found that the suggested distribution provided a better fit than the competitive bivariate models.

Efficient conditional probability theorem and importance sampling-based methods for global reliability sensitivity analysis

Global reliability sensitivity (GRS) analysis can measure the effect of random inputs on failure probability (FP). To efficiently solve GRS, two conditional probability theorem (CPT)-based methods are proposed by combining adaptive Kriging (AK) with importance sampling (IS) (CPT-AK-IS) and combining AK with Meta-IS (CPT-AK-Meta-IS) respectively. Firstly, differentiation approximation and CPT are used to convert the estimation of conditional probability density function (PDF), which is required by the existing Bayes theorem-based methods, into that of a series of probabilities. Secondly, GRS can be directly estimated by the failure samples of IS, while the existing Bayes theorem methods based on IS need to transform the failure samples of IS into those of original PDF. Both the first and second strategies can reduce the computational complexity of solving GRS. Thirdly, by selecting a suitable differentiation interval with a proposed adaptive strategy, the estimation of a series of probabilities can be accurately completed as a byproduct of one IS based simulation for solving FP without additional computational cost. Finally, by introducing AK into IS and Meta-IS, it can reduce the number of evaluating performance function and the size of candidate sample pool simultaneously. These novelties are sufficiently verified by the presented examples.

Analysis of transition path ensemble in the exactly solvable models via overdamped langevin equation

Transition of a system between two states is an important but difficult problem in natural science. In this article we study the transition problem in the framework of transition path ensemble. Using the overdamped Langevin method, we introduce the path integral formulation of the transition probability and obtain the equation for the minimum action path in the transition path space. For the effective sampling in the transition path ensemble, we derive a conditional overdamped Langevin equation. In two exactly solvable models, the free particle system and the harmonic system, we present the expression of the conditional probability density and the explicit solutions for the conditional Langevin equation and the minimum action path. The analytic results demonstrate the consistence of the conditional Langevin equation with the desired probability distribution in the transition. It is confirmed that the conditional Langevin equation is an effective tool to sample the transition path ensemble, and the stationary action principle actually leads to the most probable path.

Pressure effects on soot formation and evolution in turbulent jet flames

In this study, two series of pressurized turbulent jet sooting flames at 1, 3, and 5 bar with either fixed jet velocity or fixed Reynolds number are simulated to study the pressure effects on soot formation and evolution. Through a radiation flamelet progress variable approach with a conditional soot subfilter probability density function (PDF) model to consider the turbulence–chemistry–soot interactions, quantitatively good agreements are achieved for soot volume fraction (SVF) predictions compared with the experimental data, regardless different turbulent intensities and residence times. SVF source terms are then discussed to show the pressure effects on nucleation, condensation, surface growth, and oxidation at different axial positions in these flames. It is found that surface growth and oxidation increase by about three orders of magnitude from 1 to 5 bar, while nucleation and condensation only increase within one order of magnitude. The stronger SVF scaling on pressure than measured data is found to be attributed to the inaccurate surface growth and oxidation scaling on pressure. Further analysis indicates that (i) the uncertainty of C2H2 prediction at elevated pressures is likely a major reason for the too strong surface growth scaling; and (ii) taking account of pressure effects in the conditional subfilter PDF modeling for turbulence–soot–chemistry interactions is likely a key to improve oxidation prediction. The results in this study open up the possibilities for improving future turbulent sooting flame modeling by improving C2H2 chemistry and turbulence–chemistry–soot modeling at elevated pressures.

A Multirobot Person Search System for Finding Multiple Dynamic Users in Human-Centered Environments.

Multirobot coordination for finding multiple users in an environment can be used in numerous robotic applications, including search and rescue, surveillance/monitoring, and activities of daily living assistance. Existing approaches have limited coordination between robots when generating team plans or do not consider user location probability within these plans. This results in long searches and robots potentially revisiting the same locations in succession. In this article, we present a novel multirobot person search system to generate search plans for multirobot teams to find multiple dynamic users before a deadline. Our approach is unique in that it simultaneously considers the search actions of all robots and user location probabilities when generating team plans, where user location probabilities are represented as conditional spatial-temporal probability density functions. We model this multirobot person search problem as a two-stage optimization problem to maximize the expected number of users found before the deadline. Stage 1 solves the action selection problem to determine a set of team actions, and the second stage solves the action allocation problem to distribute these actions amongst the robots. Namely, in stage 1, a novel conditional multiperiod multiknapsack problem is modeled as a min-flow graph solved sequentially by the Bellman-Ford shortest path algorithm. Stage 2 is a variant of the min-max multitraveling salesperson problem which models the environment topology as a search region network and search times selected by the previous stage. This stage is solved by a novel fuzzy clustering method. Numerous experiments comparing our proposed method to other existing approaches with varying environment sizes, search durations, and the number of users showed that our approach was able to find more target users before a defined deadline.

Function applicability to class conditional probability density of hourly rainfall

暴雨强度公式在水文、气象、工程设计等各领域都是非常关注的问题,而常用降雨概率分布函数的适用性欠缺,理论分布函数一直处于争鸣之中。从逐时降雨概率密度函数的适用性分析入手,有利于发现普适且恰当的理论密度函数。本文从我国暴雨洪涝灾害易发区中沿30°N选取4个经纬度长方形区域(雅安附近、鄂西南、江汉平原南部、杭州湾西),并在其南、北各选一对比分区(海南岛、郑州),对6个分区内降雨资料直接采用全样本,统计逐时降雨的三类概率密度经验函数,对照这些函数的特性,从理论上分析了众多分布函数的适用性,筛选适用函数并进行拟合试验,优选出理论密度函数。研究结果表明:三参广义伽玛函数拟合误差最小,而两参广义正态函数更恰当、被首推为理论密度函数;拟合参数寻优时的目标函数综合了乘性与加性误差模型,能使拟合曲线兼顾头尾;本研究有别于极值降雨概率分布中仅采用极少部分样本的方法,采用降雨概率密度方法替代传统的年极值法,使重现期计算更准确有效,能提高暴雨强度公式的科学性,拟合的高精度与函数的普适性有望解决降雨概率分布模型统一的问题。;Due to the deficiency of international-unified theoretical function for rainfall probability distribution, it is necessary to analyze the applicability, appropriateness and universality of commonly used probability density function(PDF). On the basis of summarizing the characteristics of three kinds of class conditional probability density of hourly rainfall, the applicability of 20 kinds of functions is theoretically analyzed and compared. The generalized Gamma distribution(GΓD), generalized normal distribution(GND) and Weibull distributions are selected as reference functions, and the genetic algorithm and genetic algorithms are used to carry out fitting experiments. When the fitting deviation is equivalent to the observation error, a necessary condition for the appropriateness of the theoretical distribution function is used for discriminant analysis. The fitting results show that the Weibull distribution performs worst, the GΓD has the smallest fitting deviation, and the GND performs best. Therefore the GND is recommended for theoretical distribution fitting. In addition, the genetic algorithm can directly obtain the optimal approximate solution through multi-objective parameter optimization, which can simplify the derivation and calculation steps. The multiplicative and additive deviation models are both used as objective functions, which can take the beginning and the end of fitting curves, as well as the absolute and relative deviations into account.

Discriminant analysis of environmental data based on zero inflated spatial auto-beta models

In this paper, a supervised classification problem of classifying feature observation into one of two populations is considered. Classical supervised classification in this work is expanded for non-Gaussian spatial data specified by zero inflated auto-beta models. A classification rule based on the Bayes discriminant function (BDF) that uses conditional probability density functions in the expression is proposed. This rule is applied to real data set for identifying sea bottom type by using Black carrageen concentration data over the southeastern Baltic Sea. Different feature observation models are chosen and classification results are compared using a hold-out error rate measure.

Probabilistic Interval Predictor Based on Dissimilarity Functions

This work presents a new methodology to obtain probabilistic interval predictions of a dynamical system. The proposed strategy uses stored past system measurements to estimate the future evolution of the system. The method relies on the use of dissimilarity functions to estimate the conditional probability density function of the outputs. A family of empirical probability density functions, parameterized by means of two scalars, is introduced. It is shown that the proposed family encompasses the multivariable normal probability density function as a particular case. We show that the presented approach constitutes a generalization of classical estimation methods. A validation scheme is used to tune the two parameters on which the methodology relies. In order to prove the effectiveness of the presented methodology, some numerical examples and comparisons are provided.

Non-parametric data-driven background modelling using conditional probabilities

Background modelling is one of the main challenges in particle physics data analysis. Commonly employed strategies include the use of simulated events of the background processes, and the fitting of parametric background models to the observed data. However, reliable simulations are not always available or may be extremely costly to produce. As a result, in many cases uncertainties associated with the accuracy or sample size of the simulation are the limiting factor in the analysis sensitivity. At the same time, parametric models are limited by the a priori unknown functional form and parameter values of the background distribution. These issues become ever more pressing when large datasets become available, as it is already the case at the CERN Large Hadron Collider, and when studying exclusive signatures involving hadronic backgrounds.A widely applicable approach for non-parametric data-driven background modelling is proposed, which addresses these issues for a broad class of searches and measurements. It relies on a relaxed version of the event selection to estimate conditional probability density functions and two different techniques are discussed for its realisation. The first relies on ancestral sampling and uses data from a relaxed event selection to estimate a graph of conditional probability density functions of the variables used in the analysis, while accounting for significant correlations. A background model is then generated from events drawn from this graph, on which the full event selection is applied. In the second, a novel generative adversarial network is trained to estimate the joint probability density function of the variables used in the analysis. The training is performed on a relaxed event selection, which excludes the signal region, and the network is conditioned on a blinding variable. Subsequently, the conditional probability density function is interpolated into the signal region to model the background. The application of each method on a benchmark analysis and on ensemble tests is presented in detail, and the performance is discussed.

Missing data imputation using utility-based regression and sampling approaches

Data are often missing not at random (MNAR) in scientific experiments. We treat the MNAR problem as an imbalanced learning task. Standard predictive error measures of regression (e.g., mean squared error) are not suitable for imbalanced learning problems, such as in clinical trials where extreme values tend to be MNAR. We investigate hybrid imbalanced learning approaches that combine utility-based regression (UBR) with synthetic minority oversampling technique for regression (SMOTER) in cross-sectional trial settings. UBR optimizes the product of the conditional probability density (estimated by quantile regression forests) and a utility function which takes the relevance of the target variable value and the prediction error into account. SMOTER oversamples the relevant rare cases. Simulations show that the proposed method provides plausible predictions and reduces the bias for realistic missing data scenarios when compared with standard approaches like random forests and multiple imputation (systematic bias is observed in those methods, i.e., a tendency to underestimate the mean and standard deviation given the presence of MNAR in the area of high values of the target variable). The proposed method is implemented in a real dataset from an antidepressant clinical trial, and similar pattern of the systematic bias from commonly used methods is observed in the real data compare to the proposed method. Therefore, we encourage the integration of utility-based learning strategies for handling of missing data in the analysis of clinical trials.

Multivariate Estimator for Linear Dynamical Systems with Additive Laplace Measurement and Process Noises

Since uncertainties in many physical systems have impulsive properties poorly modeled by Gaussian distributions, heavier-tailed distributions, such as Laplace, may be used to improve model characteristics. From insights obtained through development of the scalar Laplace estimator, an algorithm is determined for the vector-state case. For a discrete-time vector linear system with scalar additive Laplace-distributed process and measurement noises, the a priori and a posteriori conditional probability density functions (pdfs) of the system states given the measurement history are propagated recursively and in closed form. The conditional pdfs are composed of signs and absolute values of affine functions, and a basis composed of signs of affine functions is constructed to simplify their representation. The a posteriori conditional mean and variance are derived analytically from the conditional pdf using characteristic functions. Generalization to independent vector measurement and process noises is straightforward. From the general method for deriving Laplace estimators in $n$-dimensions, a two-dimensional minimum variance estimator is explicitly developed, and a simulation is presented.

A statistical description method of global sub-grid topography for numerical models

Slope and aspect are important topographic elements for thermodynamics and dynamics of atmospheric circulation, especially for local radiation and topographic precipitation. We propose a simple realistic statistical method based on trigonometric function transformation to calculate sub-grid slope and aspect for describing the orographic characteristics of complex areas over the globe. It is found that the transformed conditional probability density function conforms to the Gaussian distribution in most of the global areas (~ 98%), and this feature is not eliminated with the increasing of horizontal resolution. The reasonability of this method is tested over the Tibetan Plateau. The results show that the improvement ratio of surface solar radiation downward (SSRD) over the Tibetan Plateau improved significantly compared with the results from the grid average scheme, especially in autumn. The improvement of root mean square error is approximately 18.2 W/m2, and the improvement ratio reached 38.4%. The improvements of maximum and regional-averaged SSRD over the whole Tibetan Plateau were ~ 130 W/m2 and ~ 44.3 W/m2 respectively. Although we only consider the effect of sub-grid slope and aspect on solar shortwave radiation, which has a certain bias with the observation data, it is sufficient to prove the rationality of the statistical method compared with the unobstructed horizontal surfaces scheme.

The Statistical Estimation Averaging Method to Express the Effective Electromagnetic Parameters over a Planar Information Meta-Surface

The electromagnetic scattering of a meta-surface, formed by a honeycomb substrate or periodical arranged meta-materials, has great meanings to communication technologies. In a conventional treatment to investigate the properties of these interfaces, either a variational approach or series expansions had been applied, instead of estimating the effective permittivity or permeability by the mean value of its spatial discretized statistics due to spatiotemporal fusion. Following this, this study has re-examined the problem by the Statistic Estimation Averaging method (SEAm), through the transferred conditional structural Probability Density Functions (PDFs) to realize the structural determinations by homogenization. The parameters estimated by SEAm, which exploited the concept of a homogenized medium to express properties of a structural complex medium, has been verified of validity and accuracy by comparing with the measured results of a honeycomb structure. The method can be extended to estimate the parameters of an equivalent surface, such as randomly scattering from information metamaterials. As a new wireless communication relay technology, considering that information metamaterials can modulate the electromagnetic characteristics of communication links and wireless channels simultaneously by means of spatiotemporal sequence coding, the study also gives a preliminary proposition on state estimation method of information meta-surface, which would interpret the modulation effect of wireless channels caused by its inhomogeneity of antenna wavefront by statistical estimation average information entropy.

Probabilistic inversion of seismic data for reservoir petrophysical characterization: Review and examples

The physics that describes the seismic response of an interval of saturated porous rocks with known petrophysical properties is relatively well understood and includes rock physics, petrophysics, and wave propagation models. The main goal of seismic reservoir characterization is to predict the rock and fluid properties given a set of seismic measurements by combining geophysical models and mathematical methods. This modeling challenge is generally formulated as an inverse problem. The most common geophysical inverse problem is the seismic (or elastic) inversion, i.e., the estimation of elastic properties, such as seismic velocities or impedances, from seismic amplitudes and traveltimes. The estimation of petrophysical properties, such as porosity, lithology, and fluid saturations, also can be formulated as an inverse problem and is generally referred to as rock-physics (or petrophysical) inversion. Several deterministic and probabilistic methods can be applied to solve seismic inversion problems. Deterministic algorithms predict a single solution, which is a “best” estimate or the most likely value of the model variables of interest. In probabilistic algorithms, on the other hand, the solution is the probability distribution of the model variables of interest, which can be expressed as a conditional probability density function or a set of model realizations conditioned on the data. The probabilistic approach provides a quantification of the uncertainty of the solution in addition to the most likely model. Our goal is to define the terminology, present an overview of probabilistic seismic and rock-physics inversion methods for the estimation of petrophysical properties, demonstrate the fundamental concepts with illustrative examples, and discuss the recent research developments.