Infer Probability Distributions Research Articles

The Place of Geostatistical Simulation through the Life Cycle of a Mineral Deposit

Geostatistical techniques are applied to examine the life cycle of a mineral deposit. There are two main classes of geostatistical techniques: (1) deterministic techniques that include kriging and cokriging for a single best estimate, and (2) probabilistic techniques that include simulation, which infer probability distributions and simulate realizations to transfer multivariable and multilocation uncertainty through to larger-scale resource and reserve uncertainty. Probabilistic techniques are newer and more powerful in that they provide access to quantitative measures of uncertainty and models with correct spatial variability; however, they have not seen widespread application in all aspects of the life cycle of mines. Workflows and methodologies for the appropriate use of deterministic and probabilistic techniques have been discussed. Software, engineering practices and management expectations limit some applications. Applications have been reviewed, and enhancements are required to realize the full potential of geostatistical techniques, which have been discussed with examples.

Topological models to infer multiphase interstellar medium properties

Context.Spectroscopic observations of high-redshift galaxies slowly reveal the same complexity of the interstellar medium (ISM) as expected from resolved observations in nearby galaxies. While providing, in principle, a wealth of diagnostics concerning galaxy evolution, star formation, or the nature and influence of compact objects, such high-zspectra are often spatially and spectrally unresolved, and inferring reliable diagnostics represents a major obstacle. Bright, nearby, unresolved galaxies observed in the optical and infrared domains provide many constraints to design methods to infer ISM properties, but they have so far been limited to deterministic methods and/or with simple topological assumptions (e.g., single 1D model).Aims.It is urgent to build upon previous ISM multiphase and multicomponent methods by using a probabilistic approach that makes it possible to derive probability density functions for relevant parameters while also enabling a large number of free parameters with potential priors. The goal is to provide a flexible statistical framework that is agnostic to the model grid and that considers either a few discrete components defined by their parameter values and/or statistical distributions of parameters. In this paper, we present a first application with the objective to infer probability distributions of several physical parameters (e.g., the mass of H0, H2, escape fraction of ionizing photons, and metallicity) for the star-forming regions of the metal-poor dwarf galaxy I Zw 18 in order to confirm the low molecular gas content and high escape fraction of ionizing photons from Hiiregions.Methods.We present a Bayesian approach to model a suite of spectral lines using a sequential Monte Carlo method provided by the Python package PyMC which combines several concepts such as tempered likelihoods, importance sampling, and independent Metropolis-Hastings chains. The algorithm, provided by the associated code MULTIGRIS, accepts a few components which can be represented as sectors around one or several stellar clusters, or continuous (e.g., power-law, normal) distributions for any given parameter. We applied this approach to a grid of models calculated with the photoionization and photodissociation code Cloudy in order to produce topological models of I Zw 18.Results.The statistical framework we present makes it possible to consider a large number of spectroscopic tracers, with the extinction and systematic uncertainties as potential additional random variables. We applied this technique to the galaxy I Zw 18 in order to reproduce and go beyond previous topological models specifically tailored to this object. While our grid is designed for global properties of low-metallicity star-forming galaxies, we were able to calculate accurate values for the metallicity, number of ionizing photons, masses of ionized and neutral hydrogen, as well as the dust mass and the dust-to-gas mass ratio in I Zw 18. We find a relatively modest amount of H2(~105M⊙) which is predominantly CO-dark and traced by C+rather than C0. Nevertheless, more than 90% of the [Cii] emission is associated with the neutral atomic gas. Our models confirm the necessity to include an X-ray source with an inferred luminosity in good agreement with direct X-ray observations. Finally, we investigate the escape fraction of ionizing photons for different energy ranges. While the escape fraction for the main Hiiregion lies around 50–65%, we show that most of the soft X-ray photons are able to escape and may play a role in the ionization and heating of the circumgalactic or intergalactic medium.Conclusions.Multicomponent ISM models associate a complex enough distribution of matter and phases with a simple enough topological description to be constrained with probabilistic frameworks. Despite ignoring effects such as reflected light, the diffuse radiation field, or ionization by several non-cospatial sources, they remain well adapted to individual Hiiregions and to star-forming galaxies dominated by one or a few Hiiregions, and the improvement due to the combination of several components largely compensates for other secondary effects.

Approximate Bayesian Computation as a New Tool for Partial Discharge Analysis of Partial Discharge Data

Partial Discharges are short breakdowns inside electrical equipment. As they indicate weaknesses of the insulation strength, they are seen as important precursors to a failure of the system. Therefore measurement and analysis of the patterns of instances in time and strength of the discharge are an important tool to analyze the insulation status of electric equipment, that has been addressed already using different methods in the past. In this work we explore how a physics-based stochastic process can be combined with Approximate Bayesian Computation (ABC) as a new way to analyze them. ABC is a method to infer probability distributions of model parameters in cases, where the likelihood is not tractable, but simulations can be done easily. As such it is of interest for complex phenomena or measurement systems, as often found in prognostics applications. Especially the ABC-SMC method was found to be useful here. Real Partial Discharge measurement data was used not only for parameter estimation, but also to do model comparison in order to compare different physical models proposed in the literature.

Imprecise subset simulation

The objective of this work is to quantify the uncertainty in probability of failure estimates resulting from incomplete knowledge of the probability distributions for the input random variables. We propose a framework that couples the widely used Subset simulation (SuS) with Bayesian/information theoretic multi-model inference. The process starts with data used to infer probability distributions for the model inputs. Often such data sets are small. Multi-model inference is used to assess uncertainty associated with the model-form and parameters of these random variables in the form of model probabilities and the associated joint parameter probability densities. A sampling procedure is used to construct a set of equally probable candidate probability distributions and an optimal importance sampling distribution is determined analytically from this set. Subset simulation is then performed using this optimal sampling density and the resulting conditional probabilities are re-weighted using importance sampling. The result of this process are empirical probability distributions of failure probabilities that provide direct estimates of the uncertainty in failure probability estimates that result from inference on small data sets. The method is demonstrated to be both computationally efficient – requiring only a single subset simulation and nominal cost of sample re-weighting – and to provide reasonable estimates of the uncertainty in failure probabilities.

A Stochastic Framework for Atomistic Fracture

We present a stochastic modeling framework for atomistic propagation of a Mode I surface crack, with atoms interacting according to the Lennard--Jones interatomic potential at zero temperature. Specifically, we invoke the Cauchy--Born rule and the maximum entropy principle to infer probability distributions for the parameters of the interatomic potential. We then study how uncertainties in the parameters propagate to the quantities of interest relevant to crack propagation, namely, the critical stress intensity factor and the lattice trapping range. For our numerical investigation, we rely on an automated version of the so-called numerical-continuation enhanced flexible boundary NCFlex algorithm.

An Object-Based Bayesian Framework for Top-Down Visual Attention

We introduce a new task-independent framework to model top-down overt visual attention based on graph-ical models for probabilistic inference and reasoning. We describe a Dynamic Bayesian Network (DBN) that infers probability distributions over attended objects and spatial locations directly from observed data. Probabilistic inference in our model is performed over object-related functions which are fed from manual annotations of objects in video scenes or by state-of-the-art object detection models. Evaluating over ∼3 hours (appx. 315,000 eye fixations and 12,600 saccades) of observers playing 3 video games (time-scheduling, driving, and flight combat), we show that our approach is significantly more predictive of eye fixations compared to: 1) simpler classifier-based models also developed here that map a signature of a scene (multi-modal information from gist, bottom-up saliency, physical actions, and events) to eye positions, 2) 14 state-of-the-art bottom-up saliency models, and 3) brute-force algorithms such as mean eye position. Our results show that the proposed model is more effective in employing and reasoning over spatio-temporal visual data.

Uncertainty quantifications of calibrating laser-induced incandescence intensity on sooting propensity in a wick-fed diffusion flame burner

Extensive research has been devoted to engineering analysis in the presence of parameter uncertainties. Meanwhile, parameter estimations with uncertainty quantifications facilitate the reduction of bias and physical unrealistic estimates on interpreting model predictions. In this study, the sooting propensity from wick-fed diffusion flames tested by Jet A-1, diesel and their blended fuels are interpreted, with Laser-induced incandescence (LII) diagnosis to quantitative calibrate the soot volume fraction fv. To make the calibration independent of optical properties, the fv is directly inferred from particle size distribution measured in flames by the Differential Mobility Spectrometer 500 (DMS500). Thus, the calibration parameter with its uncertainties is therefore qualified with errors that arise from measurements. This study refers to several methodologies with potential estimates of parameter uncertainties for proper interpretation of fv by LII diagnosis measurement. Bayesian regression method with Gaussian mixture functions are accounted for calibration parameter uncertainties derived from heteroscedastic measurement errors. And the principal component analysis (PCA) assisted statistical approach is responsible for projecting multivariable datasets into low-dimension space, therefore joint probability distribution would be inferred. As a consequence, probability interval from inferred probability distribution of the calibration parameter is associated with degree of uncertainties, which provides better guidance regarding the applicability and uncertainty of LII diagnosis on soot characteristics.

Uncertainty Analysis of Simulations of the Turn‐of‐the‐Century Drought in the Western United States

AbstractWe perform the first uncertainty quantification analysis of the turn‐of‐the‐century drought in the western United States using a large perturbed‐parameter ensemble of the Community Atmosphere Model version 4.0 (CAM4). We develop several metrics to characterize the aridity bias, spatial extent, and tropical forcing of the drought and use statistical models to ascertain that the modeled drought was mainly sensitive to CAM4 parameters related to deep convection and clouds. Deep convection parameters account for over half the variance across the drought metrics. We employ observed estimates of these drought metrics to infer probability distributions of the model parameters that lead to a better agreement with observations, thereby providing guidance on how to improve the simulation of drought in CAM4. We find that in some cases, the suggested parameter values that improve the simulation of one drought characteristic would degrade the simulation of another, suggesting that there is a complex relationship between the model parameters and drought in the western United States. We also demonstrate reductions in the uncertainty of the drought metrics of up to 30% by constraining with observations of all metrics. Furthermore, we demonstrate for the first time that improvements to the simulation of the spatial extent of the turn‐of‐the‐century drought also lead to improvements in the spatial extent of the Australian Millennium drought, suggesting that physics of drought as encoded by the model parameters may be generalizable.

The Bayesian optimist's guide to adaptive immune receptor repertoire analysis.

Probabilistic modeling is fundamental to the statistical analysis of complex data. In addition to forming a coherent description of the data-generating process, probabilistic models enable parameter inference about given datasets. This procedure is well developed in the Bayesian perspective, in which one infers probability distributions describing to what extent various possible parameters agree with the data. In this paper, we motivate and review probabilistic modeling for adaptive immune receptor repertoire data then describe progress and prospects for future work, from germline haplotyping to adaptive immune system deployment across tissues. The relevant quantities in immune sequence analysis include not only continuous parameters such as gene use frequency but also discrete objects such as B-cell clusters and lineages. Throughout this review, we unravel the many opportunities for probabilistic modeling in adaptive immune receptor analysis, including settings for which the Bayesian approach holds substantial promise (especially if one is optimistic about new computational methods). From our perspective, the greatest prospects for progress in probabilistic modeling for repertoires concern ancestral sequence estimation for B-cell receptor lineages, including uncertainty from germline genotype, rearrangement, and lineage development.

Hierarchical Bayesian inference for ion channel screening dose-response data.

Dose-response (or 'concentration-effect') relationships commonly occur in biological and pharmacological systems and are well characterised by Hill curves. These curves are described by an equation with two parameters: the inhibitory concentration 50% (IC50); and the Hill coefficient. Typically just the 'best fit' parameter values are reported in the literature. Here we introduce a Python-based software tool, PyHillFit , and describe the underlying Bayesian inference methods that it uses, to infer probability distributions for these parameters as well as the level of experimental observation noise. The tool also allows for hierarchical fitting, characterising the effect of inter-experiment variability. We demonstrate the use of the tool on a recently published dataset on multiple ion channel inhibition by multiple drug compounds. We compare the maximum likelihood, Bayesian and hierarchical Bayesian approaches. We then show how uncertainty in dose-response inputs can be characterised and propagated into a cardiac action potential simulation to give a probability distribution on model outputs.

Hierarchical Bayesian inference for ion channel screening dose-response data

Dose-response (or ‘concentration-effect’) relationships commonly occur in biological and pharmacological systems and are well characterised by Hill curves. These curves are described by an equation with two parameters: the inhibitory concentration 50% (IC50); and the Hill coefficient. Typically just the ‘best fit’ parameter values are reported in the literature. Here we introduce a Python-based software tool, PyHillFit , and describe the underlying Bayesian inference methods that it uses, to infer probability distributions for these parameters as well as the level of experimental observation noise. The tool also allows for hierarchical fitting, characterising the effect of inter-experiment variability. We demonstrate the use of the tool on a recently published dataset on multiple ion channel inhibition by multiple drug compounds. We compare the maximum likelihood, Bayesian and hierarchical Bayesian approaches. We then show how uncertainty in dose-response inputs can be characterised and propagated into a cardiac action potential simulation to give a probability distribution on model outputs.

Hierarchical Bayesian inference for ion channel screening dose-response data

Dose-response (or ‘concentration-effect’) relationships commonly occur in biological and pharmacological systems and are well characterised by Hill curves. These curves are described by an equation with two parameters: the inhibitory concentration 50% (IC50); and the Hill coefficient. Typically just the ‘best fit’ parameter values are reported in the literature. Here we introduce a Python-based software tool, PyHillFit, and describe the underlying Bayesian inference methods that it uses, to infer probability distributions for these parameters as well as the level of experimental observation noise. The tool also allows for hierarchical fitting, characterising the effect of inter-experiment variability. We demonstrate the use of the tool on a recently published dataset on multiple ion channel inhibition by multiple drug compounds. We compare the maximum likelihood, Bayesian and hierarchical Bayesian approaches. We then show how uncertainty in dose-response inputs can be characterised and propagated into a cardiac action potential simulation to give a probability distribution on model outputs.

Neural Variability and Sampling-Based Probabilistic Representations in the Visual Cortex

SummaryNeural responses in the visual cortex are variable, and there is now an abundance of data characterizing how the magnitude and structure of this variability depends on the stimulus. Current theories of cortical computation fail to account for these data; they either ignore variability altogether or only model its unstructured Poisson-like aspects. We develop a theory in which the cortex performs probabilistic inference such that population activity patterns represent statistical samples from the inferred probability distribution. Our main prediction is that perceptual uncertainty is directly encoded by the variability, rather than the average, of cortical responses. Through direct comparisons to previously published data as well as original data analyses, we show that a sampling-based probabilistic representation accounts for the structure of noise, signal, and spontaneous response variability and correlations in the primary visual cortex. These results suggest a novel role for neural variability in cortical dynamics and computations.

Context-Aware Abnormality Monitoring Service for Care-Needing Persons Using a Probabilistic Model

Background/Objectives: Care-needing persons usually have routine life patterns in their visiting places and routes. Probabilistic models are useful tools to capture the inherent patterns for care-needing persons from a collection of data and to make inference for uncertain situation. Methods/Statistical Analysis: A dynamic Bayesian network model is proposed to capture the traveling patterns for care-needing person. A collection of traveling data having location, date, and battery charge state information is used to train the dynamic Bayesian network model, in which the training travel data are preprocessed to filter out noises using a smoothing filter. The model makes probabilistic inference for the latent random variables on the fly with respect to the arriving captured sensor data. To conduct the probabilistic inference, the Rao-Blackwellized particle filter is used for fast execution. To take protective steps at potentially abnormal situations, the context-based service rules are defined that specify what to do in which situations the care-needing person is. Findings: The situations of interest are evaluated based on the deviation of the inferred probability distribution from the trained ones regarding the latent random variables. The service rules enable to realize value-added services such abnormal state alerting, abnormality notification to stakeholders, and situational information collection at abnormal states for future forensic or tracking. A real data set has been collected from a young student and the proposed method has been trained and tested in an implemented prototype system for young child care. In the experiments for a 6 weeks data set, the method detected all new test routes successfully when the deviation distance is greater than about 20 meters due to the limited precision of the GPS sensors. Improvements/Applications: The proposed model can be applied to context-aware abnormality service for care-needing persons like young children, mentally ill or disabled persons, and elderly persons are exposed to various dangers in daily life.

Discriminative Bayesian Dictionary Learning for Classification.

We propose a Bayesian approach to learn discriminative dictionaries for sparse representation of data. The proposed approach infers probability distributions over the atoms of a discriminative dictionary using a finite approximation of Beta Process. It also computes sets of Bernoulli distributions that associate class labels to the learned dictionary atoms. This association signifies the selection probabilities of the dictionary atoms in the expansion of class-specific data. Furthermore, the non-parametric character of the proposed approach allows it to infer the correct size of the dictionary. We exploit the aforementioned Bernoulli distributions in separately learning a linear classifier. The classifier uses the same hierarchical Bayesian model as the dictionary, which we present along the analytical inference solution for Gibbs sampling. For classification, a test instance is first sparsely encoded over the learned dictionary and the codes are fed to the classifier. We performed experiments for face and action recognition; and object and scene-category classification using five public datasets and compared the results with state-of-the-art discriminative sparse representation approaches. Experiments show that the proposed Bayesian approach consistently outperforms the existing approaches.

Maximum entropy inference of seabed attenuation parameters using ship radiated broadband noise.

The received acoustic field generated by a single passage of a research vessel on the New Jersey continental shelf is employed to infer probability distributions for the parameter values representing the frequency dependence of the seabed attenuation and the source levels of the ship. The statistical inference approach employed in the analysis is a maximum entropy methodology. The average value of the error function, needed to uniquely specify a conditional posterior probability distribution, is estimated with data samples from time periods in which the ship-receiver geometry is dominated by either the stern or bow aspect. The existence of ambiguities between the source levels and the environmental parameter values motivates an attempt to partially decouple these parameter values. The main result is the demonstration that parameter values for the attenuation (α and the frequency exponent), the sediment sound speed, and the source levels can be resolved through a model space reduction technique. The results of this multi-step statistical inference developed for ship radiated noise is then tested by processing towed source data over the same bandwidth and source track to estimate continuous wave source levels that were measured independently with a reference hydrophone on the tow body.

Reply to C. Tsallis’ “Conceptual Inadequacy of the Shore and Johnson Axioms for Wide Classes of Complex Systems”

In a recent PRL (2013, 111, 180604), we invoked the Shore and Johnson axioms which demonstrate that the least-biased way to infer probability distributions fpig from data is to maximize the Boltzmann-Gibbs entropy. We then showed which biases are introduced in models obtained by maximizing nonadditive entropies. A rebuttal of our work appears in entropy (2015, 17, 2853) and argues that the Shore and Johnson axioms are inapplicable to a wide class of complex systems. Here we highlight the errors in this reasoning.

What/Where to Look Next? Modeling Top-Down Visual Attention in Complex Interactive Environments

Several visual attention models have been proposed for describing eye movements over simple stimuli and tasks such as free viewing or visual search. Yet, to date, there exists no computational framework that can reliably mimic human gaze behavior in more complex environments and tasks such as urban driving. In addition, benchmark datasets, scoring techniques, and top-down model architectures are not yet well understood. In this paper, we describe new task-dependent approaches for modeling top-down overt visual attention based on graphical models for probabilistic inference and reasoning. We describe a dynamic Bayesian network that infers probability distributions over attended objects and spatial locations directly from observed data. Probabilistic inference in our model is performed over object-related functions that are fed from manual annotations of objects in video scenes or by state-of-the-art object detection/recognition algorithms. Evaluating over approximately 3 h (approximately 315 000 eye fixations and 12 000 saccades) of observers playing three video games (time-scheduling, driving, and flight combat), we show that our approach is significantly more predictive of eye fixations compared to: 1) simpler classifier-based models also developed here that map a signature of a scene (multimodal information from gist, bottom-up saliency, physical actions, and events) to eye positions; 2) 14 state-of-the-art bottom-up saliency models; and 3) brute-force algorithms such as mean eye position. Our results show that the proposed model is more effective in employing and reasoning over spatio-temporal visual data compared with the state-of-the-art.

BINARY POPULATIONS IN MILKY WAY SATELLITE GALAXIES: CONSTRAINTS FROM MULTI-EPOCH DATA IN THE CARINA, FORNAX, SCULPTOR, AND SEXTANS DWARF SPHEROIDAL GALAXIES

We introduce a likelihood analysis of multi-epoch stellar line-of-sight velocities to constrain the binary fractions and binary period distributions of dwarf spheroidal galaxies. This method is applied to multi-epoch data from the Magellan/MMFS survey of the Carina, Fornax, Sculptor and Sextans dSph galaxies, after applying a model for the measurement errors that accounts for binary orbital motion. We find that the Fornax, Sculptor, and Sextans dSphs are consistent with having binary populations similar to that of Milky Way field binaries to within 68% confidence limits, whereas the Carina dSph is remarkably deficient in binaries with periods less than ~10 years. If Carina is assumed to have a period distribution identical to that of the Milky Way field, its best-fit binary fraction is 0.14^{+0.28}_{-0.05}, and is constrained to be less than 0.5 at the 90% confidence level; thus it is unlikely to host a binary population identical to that of the Milky Way field. By contrast, the best-fit binary fraction of the combined sample of all four galaxies is 0.46^{+0.13}_{-0.09}, consistent with that of Milky Way field binaries. More generally, we infer probability distributions in binary fraction, mean orbital period, and dispersion of periods for each galaxy in the sample. Looking ahead to future surveys, we show that the allowed parameter space of binary fraction and period distribution parameters in dSphs will be narrowed significantly by a large multi-epoch survey. However, there is a degeneracy between the parameters that is unlikely to be broken unless the measurement error is of order ~0.1 km/s or smaller, presently attainable only by a high-resolution spectrograph.

Nonadditive Entropies Yield Probability Distributions with Biases not Warranted by the Data

Different quantities that go by the name of entropy are used in variational principles to infer probability distributions from limited data. Shore and Johnson showed that maximizing the Boltzmann-Gibbs form of the entropy ensures that probability distributions inferred satisfy the multiplication rule of probability for independent events in the absence of data coupling such events. Other types of entropies that violate the Shore and Johnson axioms, including nonadditive entropies such as the Tsallis entropy, violate this basic consistency requirement. Here we use the axiomatic framework of Shore and Johnson to show how such nonadditive entropy functions generate biases in probability distributions that are not warranted by the underlying data.