Bayesian Inference Procedure Research Articles

Variational learning for finite Dirichlet mixture models and applications.

In this paper, we focus on the variational learning of finite Dirichlet mixture models. Compared to other algorithms that are commonly used for mixture models (such as expectation-maximization), our approach has several advantages: first, the problem of over-fitting is prevented; furthermore, the complexity of the mixture model (i.e., the number of components) can be determined automatically and simultaneously with the parameters estimation as part of the Bayesian inference procedure; finally, since the whole inference process is analytically tractable with closed-form solutions, it may scale well to large applications. Both synthetic and real data, generated from real-life challenging applications namely image databases categorization and anomaly intrusion detection, are experimented to verify the effectiveness of the proposed approach.

Bayesian Approach to the Analysis of Fluorescence Correlation Spectroscopy Data I: Theory

Fluorescence correlation spectroscopy (FCS) is a powerful tool to infer the physical process of macromolecules including local concentration, binding, and transport from fluorescence intensity measurements. Interpretation of FCS data relies critically on objective multiple hypothesis testing of competing models for complex physical processes that are typically unknown a priori. Here, we propose an objective Bayesian inference procedure for testing multiple competing models to describe FCS data based on temporal autocorrelation functions. We illustrate its performance on simulated temporal autocorrelation functions for which the physical process, noise, and sampling properties can be controlled completely. The procedure enables the systematic and objective evaluation of an arbitrary number of competing, non-nested physical models for FCS data, appropriately penalizing model complexity according to the Principle of Parsimony to prefer simpler models as the signal-to-noise ratio decreases. In addition to eliminating overfitting of FCS data, the procedure dictates when the interpretation of model parameters are not justified by the signal-to-noise ratio of the underlying sampled data. The proposed approach is completely general in its applicability to transport, binding, or other physical processes, as well as spatially resolved FCS from image correlation spectroscopy, providing an important theoretical foundation for the automated application of FCS to the analysis of biological and other complex samples.

A Bayesian inference based two-stage support vector regression framework for soft sensor development in batch bioprocesses

Inherent process and measurement uncertainty has posed a challenging issue on soft sensor development of batch bioprocesses. In this paper, a new soft sensor modeling framework is proposed by integrating Bayesian inference strategy with two-stage support vector regression (SVR) method. The Bayesian inference procedure is first designed to identify measurement biases and misalignments via posterior probabilities. Then the biased input measurements are calibrated through Bayesian estimation and the first-stage SVR model is thus built for output measurement reconciliation. The inferentially calibrated input and output data can be further used to construct the second-stage SVR model, which serves as the main model of soft sensor to predict new output measurements. The Bayesian inference based two-stage support vector regression (BI-SVR) approach is applied to a fed-batch penicillin cultivation process and the obtained soft sensor performance is compared to that of the conventional SVR method. The results from two test cases with different levels of measurement uncertainty show significant improvement of the BI-SVR approach over the regular SVR method in predicting various output measurements.

Quantifying uncertainty in statistical distribution of small sample data using Bayesian inference of unbounded Johnson distribution

Probabilistic analysis of physical systems requires information on the distributions of random variables. Distributions are typically obtained from testing or field data. In engineering design where tests are expensive, the sample size of such data is small O(10). Identifying correct distributions with small number of samples is difficult. Furthermore, parameters of assumed distributions obtained from small sample data themselves contain some uncertainty. In this study a Johnson SU family distribution function is used to identify shape, location and scale parameters of distribution that can best fit small sample data. A Bayesian inference procedure is used to determine distributions of the parameters. We show that the procedure correctly bounds the tail regions of the distributions and is less conservative than bounds obtained using bootstrap methods.

Bayesian confidence intervals for means and variances of lognormal and bivariate lognormal distributions

The lognormal distribution is currently used extensively to describe the distribution of positive random variables. This is especially the case with data pertaining to occupational health and other biological data. One particular application of the data is statistical inference with regards to the mean of the data. Other authors, namely Zou et al. (2009), have proposed procedures involving the so-called “method of variance estimates recovery” (MOVER), while an alternative approach based on simulation is the so-called generalized confidence interval, discussed by Krishnamoorthy and Mathew (2003). In this paper we compare the performance of the MOVER-based confidence interval estimates and the generalized confidence interval procedure to coverage of credibility intervals obtained using Bayesian methodology using a variety of different prior distributions to estimate the appropriateness of each. An extensive simulation study is conducted to evaluate the coverage accuracy and interval width of the proposed methods. For the Bayesian approach both the equal-tail and highest posterior density (HPD) credibility intervals are presented. Various prior distributions (Independence Jeffreys' prior, Jeffreys'-Rule prior, namely, the square root of the determinant of the Fisher Information matrix, reference and probability-matching priors) are evaluated and compared to determine which give the best coverage with the most efficient interval width. The simulation studies show that the constructed Bayesian confidence intervals have satisfying coverage probabilities and in some cases outperform the MOVER and generalized confidence interval results. The Bayesian inference procedures (hypothesis tests and confidence intervals) are also extended to the difference between two lognormal means as well as to the case of zero-valued observations and confidence intervals for the lognormal variance. In the last section of this paper the bivariate lognormal distribution is discussed and Bayesian confidence intervals are obtained for the difference between two correlated lognormal means as well as for the ratio of lognormal variances, using nine different priors.

Automatic detection of atmospheric boundary layer height using ceilometer backscatter data assisted by a boundary layer model

AbstractThis work tackles the problem of the automated detection of the atmospheric boundary layer (BL) height h, from aerosol lidar/ceilometer observations. A new method, the Bayesian selective method (BSM), is presented. It implements a Bayesian statistical inference procedure which combines in a statistically optimal way different sources of information. Firstly, atmospheric stratification boundaries are located from discontinuities in the ceilometer backscattered signal. The BSM then identifies the discontinuity edge that has the highest probability to effectively mark the BL height. Information from the contemporaneous physical boundary layer model simulations and a climatological dataset of BL height evolution are combined in the assimilation framework to assist this choice.The BSM algorithm has been tested for 4 months of continuous ceilometer measurements collected during the BASE:ALFA project, and is shown to realistically diagnose the BL depth evolution in many different weather conditions. A standard one‐dimensional processing of the ceilometer signal without the a priori support of the dynamical and climatological BL models often fails to correctly detect h, with the greatest inaccuracies occurring at night‐time when residual layers can generate very strong signals, which are then classified by an automated application of the gradient or of the wavelet analysis as the most probable BL height. The BSM approach instead carries information on the low climatological probability to find elevated BL depths at night and penalizes the selection of these points. Moreover, this method is able to correctly convey information along the temporal dimension, thus filling data gaps using earlier and subsequent ceilometer information for the retrieval. Copyright © 2011 Royal Meteorological Society

P1-S6.06 What has been the effect of the Avahan HIV intervention on condom use among female sex workers? Insights from a fully Bayesian inference methodology

<h3>Background</h3> Considerable efforts have been invested to evaluate the impact of the Avahan project, the India AIDS initiative, targeted to high risk groups. One measure of impact is the number of HIV infections prevented (PF) due to increases in condom use between FSWs and clients following the start of Avahan. PF estimates hinge on knowledge of the frequency of condom use over time. As there is no data on condom use prior to the first round of data collection post-Avahan, these trends must be estimated indirectly. We aim, using a Bayesian framework and Monte Carlo Markov Chains (MCMC) methodology, to determine if the trends in HIV prevalence among FSWs and clients can be used to infer changes in condom use between FSW and clients before and after the start of Avahan. <h3>Methods</h3> We used serial rounds of cross-sectional behavioural and biological survey (IBBA) data and a deterministic compartmental model of HIV transmission among FSW/clients in Mysore and Belgaum districts coupled with Bayesian inference procedures. Condom use was modelled as the fraction of FSW commercial sex acts protected by condoms. IBBA data was used to specify the model prior parameter distributions and estimate HIV prevalence at three time points among FWS, at one time point among their clients. The Particle MCMC algorithm was used to explore the posterior density of our complex parameter distribution, and to derive estimates of the evolution of condom use over time using either 2 or 3 rounds of HIV prevalence data among FSW. <h3>Results</h3> For both districts, the results reveal a clear increase in condom use around the start of Avahan in 2004 (Abstract P1-S6.06 figure 1). Abstract P1-S6.06 figure 1 suggests condom use before the intervention (jan 1994) was lower in Mysore (15.7%, 95% CI 0 to 48%) than in Belgaum (19%, 95% CI 0 to 58%). In both districts, post intervention condom use stabilised at values above 80% (95% CI Mysore 65 to 100% and Belgaum 50 to 100%). Lastly, Abstract P1-S6.06 figure 1 A),C) vs B),D) show the information gained by using 3 rounds of data instead of 2. <h3>Conclusions</h3> This is the first application of Particle MCMC in an intervention monitoring context. Our results consolidate previous back projections suggesting that condom use significantly increased since the start of Avahan, which sheds additional light on the potential intervention impact. This study illustrates the use of flexible Bayesian inference methodology to estimate time-varying parameters. It also informs the design of prevalence surveys for intervention monitoring.

Variational Bayesian Learning for Mixture Autoregressive Models With Uncertain-Order

Autoregressive (AR) models are fundamental tools for modeling a variety of signals in many fields of study. Selecting the appropriate order for AR models is typically accomplished using an information criterion to compare the models learned for all orders under consideration. The use of an information criterion for model order selection becomes increasingly computationally demanding when AR models are used as part of larger statistical models, such as mixture models, that have their own model order selection issues. Statistical models utilizing Dirichlet process (DP) priors provide a mechanism for automatically selecting the number of components within a mixture model, and have previously been utilized with AR components. These previous investigations utilize different priors for the AR parameters to enable automatic selection of the AR order and each makes us of computationally expensive Markov chain Monte Carlo (MCMC) sampling. This paper develops and evaluates a variational Bayesian (VB) inference procedure for the parameters of DP mixtures of AR components with uncertain order, to enable rapid parameter inference for AR based statistical models that provide automatic model order selection and is suitable for large scale problems. The previously utilized priors for AR models with uncertain order are evaluated to determine which is more appropriate for VB inference and the ability of the VB inference procedure for the developed model to correctly determine the number of mixture components and the AR order of each of the mixture components is shown to be comparable to computationally intensive MCMC inference. The VB inference procedure is then applied to an acoustic signal classification problem to illustrate the efficacy of AR based statistical models utilizing automated model order selection for real-world signal processing tasks.

Self-Selectivity in Firm’s Decision to Withdraw IPO: Bayesian Inference for Hazard Models of Bankruptcy With Feedback

Examination on firm performance subsequent to a chosen event is widely used in finance studies to analyze the motivation behind managerial decisions. However, results are often subject to bias when the self-selectivity behind managerial decisions is ignored and unspecified. This study investigates a unique corporate event of initial public offering (IPO) withdrawal, where a firm’s subsequent likelihood of bankruptcy is specified in a system of switching hazard models, and the expected difference in post-IPO and postwithdrawal survival probabilities serves as a “feedback” on a firm’s decision to cancel its offering. Our Bayesian inference procedure generates strong evidence that incidence of withdrawal unfavorably affects the subsequent performance of a firm, and that the “feedback” is an important determinant in managerial decisions. The econometric and statistical model specification and the accompanying estimation procedure we used can be widely applicable to study self-selective corporate transactions.

A general framework for the analysis of vessel encoded arterial spin labeling for vascular territory mapping

Vessel encoded arterial spin labeling provides a way to perform non-invasive vascular territory imaging. By uniquely encoding the blood within feeding arteries over a number of images, the territories of each can be identified. Here, a new approach for the analysis of vessel encoded arterial spin labeling data is presented. The method includes a full description of how the geometry of arteries and spatial label modulation affects the measured signal. It also incorporates an artery-based classification that considers multiple arteries in each class, explicitly permitting a voxel to be supplied by multiple arteries. The developed framework is cast within a Bayesian inference procedure allowing both flow contributions and the locations of the arteries in the labeling plane to be inferred. By using simulated data, the method was shown to provide more accurate estimates of blood contribution in areas of mixed supply, such as would be found in watershed regions, than conventional methods. It was also able to estimate the location of arteries within the labeling plane, accounting for motion between sequence prescription and acquisition. Similar performance was found for data acquired using a pseudo-continuous labeling scheme both in the neck and above the Circle of Willis.

Bayesian experimental design of tracer studies to monitor wastewater leakage from sewer networks

Despite more than a decade of research, the magnitude of wastewater leakage from defective sewer systems into groundwater supplies is still largely unknown, partly because reliable measurement methods are lacking. Although recently suggested in‐sewer tracer studies present a promising solution, it is unclear how to optimally design such studies in network settings. In this study we present a formal experimental design procedure, which uses Bayesian data analysis to improve the diagnosis of sewer leakage by combining tracer test data with prior knowledge on network topology and condition. From a simulation study, we show that (1) if a single sewer section is expected to have high leakage, that section should be distinguished in measurement layouts through isolated tests or appropriate overlapping of multiple tests; (2) if multiple sections are expected to have high leakage, layouts with tests that cover more than one high‐leakage section should be avoided; and (3) if a robust experimental design is desired, a balanced layout of tests that overlap multiple sections of high leakage, yet minimizes stretch length, should be chosen. This design will have the additional benefit of inducing covariance in the posterior distribution of exfiltration estimates, which can be used to advantage in subsequent studies. We apply these guidelines to a case study of a catchment in Zurich, Switzerland, and find that optimal layout design can improve the anticipated gain of information substantially relative to designs based on practical considerations alone. Remaining concerns regarding the procedure include (1) the generally poor understanding of the mechanisms governing sewer leakage, which can hamper reliable prior information on exfiltration; (2) the currently low measurement precision of sewer tracer studies, which might only allow for the detection of large leaks; and (3) the need for numerical implementation of the Bayesian inference procedure, which requires careful tuning and long computation times. In general, we were able to demonstrate that the incorporation of prior information through a Bayesian procedure adds significant value to experimental design, especially in situations with few “hard” data but good site‐specific knowledge, which is common in water resources research.

Periodic multivariate normal hidden markov models for the analysis of water quality time series

The modelling of multivariate riverine water quality time series poses some challenging problems including: weak dependency between observations; nonlinearity; non-Normality; seasonality and missing data. We demonstrate that periodic multivariate Normal hidden Markov models (MNHMMs) are appropriate tools to analyse riverine water quality time series. We introduce a fully Bayesian inference procedure for this class of models, where the number of hidden states of the Markov process is unknown and reversible jump Markov chain Monte Carlo (RJMCMC) methods are developed. We present a case study using long-term dissolved inorganic nitrogen time series measured in three Scottish rivers. Our results show the strength of the hidden Markov multistate approach for analysing long-term multivariate riverine water quality time series. Copyright (C) 2010 John Wiley & Sons, Ltd.

Hierarchical Bayesian inference for Ill-posed problems via variational method

This paper investigates a novel approximate Bayesian inference procedure for numerically solving inverse problems. A hierarchical formulation which determines automatically the regularization parameter and the noise level together with the inverse solution is adopted. The framework is of variational type, and it can deliver the inverse solution and regularization parameter together with their uncertainties calibrated. It approximates the posteriori probability distribution by separable distributions based on Kullback–Leibler divergence. Two approximations are derived within the framework, and some theoretical properties, e.g. variance estimate and consistency, are also provided. Algorithms for their efficient numerical realization are described, and their convergence properties are also discussed. Extensions to nonquadratic regularization/nonlinear forward models are also briefly studied. Numerical results for linear and nonlinear Cauchy-type problems arising in heat conduction with both smooth and nonsmooth solutions are presented for the proposed method, and compared with that by Markov chain Monte Carlo. The results illustrate that the variational method can faithfully capture the posteriori distribution in a computationally efficient way.

Bayesian Blind Deconvolution From Differently Exposed Image Pairs

Photographs acquired under low-lighting conditions require long exposure times and therefore exhibit significant blurring due to the shaking of the camera. Using shorter exposure times results in sharper images but with a very high level of noise. In this paper we address the problem of utilizing two such images in order to obtain an estimate of the original scene and present a novel blind deconvolution algorithm for solving it. We formulate the problem in a hierarchical Bayesian framework by utilizing prior knowledge on the unknown image and blur, and also on the dependency between the two observed images. By incorporating a fully Bayesian analysis, the developed algorithm estimates all necessary model parameters along with the unknown image and blur, such that no user-intervention is needed. Moreover, we employ a variational Bayesian inference procedure, which allows for the statistical compensation of errors occurring at different stages of the restoration, and also provides uncertainties of the estimates. Experimental results with synthetic and real images demonstrate that the proposed method provides very high quality restoration results and compares favorably to existing methods even though no user supervision is needed.

Doubly sparse factor models for unifying feature transformation and feature selection

A number of unsupervised learning methods for high-dimensional data are largely divided into two groups based on their procedures, i.e., (1) feature selection, which discards irrelevant dimensions of the data, and (2) feature transformation, which constructs new variables by transforming and mixing over all dimensions. We propose a method that both selects and transforms features in a common Bayesian inference procedure. Our method imposes a doubly automatic relevance determination (ARD) prior on the factor loading matrix. We propose a variational Bayesian inference for our model and demonstrate the performance of our method on both synthetic and real data.

Bayesian spatial defect pattern recognition in semiconductor fabrication using support vector clustering

Defects generated during integrated circuit (IC) fabrication processes are classified into global defects and local defects according to their generation causes. Spatial patterns of locally clustered defects are likely to contain the information related to their defect generation mechanisms. In this paper, we propose a model-based clustering for spatial patterns of local defects to reflect real situations. A flexible two-step approach is proposed to classify the spatial defects patterns via support vector clustering and Bayesian method. Support vector clustering is employed to separate global defects from the local ones to improve both clustering accuracy and computational efficiency in further analysis. A new mixture model is proposed for modeling the distribution of local defects on the wafers. Local defect clusters with amorphous/linear, curvilinear, and ring patterns are modeled by multivariate normal distribution, principal curve, and spherical shell, respectively. A Bayesian inference procedure is then applied for parametric pattern recognition of the local defects. Results from both simulated data and real wafer map data demonstrate potential in applying our approach to analyze general defect patterns in IC manufacturing.

Measuring portfolio credit risk correctly: Why parameter uncertainty matters

Why should risk management systems account for parameter uncertainty? In addressing this question, the paper lets an investor in a credit portfolio face non-diversifiable uncertainty about two risk parameters – probability of default and asset-return correlation – and calibrates this uncertainty to a lower bound on estimation noise. In this context, a Bayesian inference procedure is essential for deriving and analyzing the main result, i.e. that parameter uncertainty raises substantially the tail risk perceived by the investor. Since a measure of tail risk that incorporates parameter uncertainty is computationally demanding, the paper also derives a closed-form approximation to such a measure.

Bayesian inference for the stereotype regression model: Application to a case–control study of prostate cancer

The stereotype regression model for categorical outcomes, proposed by Anderson (J. Roy. Statist. Soc. B. 1984; 46:1-30) is nested between the baseline-category logits and adjacent category logits model with proportional odds structure. The stereotype model is more parsimonious than the ordinary baseline-category (or multinomial logistic) model due to a product representation of the log-odds-ratios in terms of a common parameter corresponding to each predictor and category-specific scores. The model could be used for both ordered and unordered outcomes. For ordered outcomes, the stereotype model allows more flexibility than the popular proportional odds model in capturing highly subjective ordinal scaling which does not result from categorization of a single latent variable, but are inherently multi-dimensional in nature. As pointed out by Greenland (Statist. Med. 1994; 13:1665-1677), an additional advantage of the stereotype model is that it provides unbiased and valid inference under outcome-stratified sampling as in case-control studies. In addition, for matched case-control studies, the stereotype model is amenable to classical conditional likelihood principle, whereas there is no reduction due to sufficiency under the proportional odds model. In spite of these attractive features, the model has been applied less, as there are issues with maximum likelihood estimation and likelihood-based testing approaches due to non-linearity and lack of identifiability of the parameters. We present comprehensive Bayesian inference and model comparison procedure for this class of models as an alternative to the classical frequentist approach. We illustrate our methodology by analyzing data from The Flint Men's Health Study, a case-control study of prostate cancer in African-American men aged 40-79 years. We use clinical staging of prostate cancer in terms of Tumors, Nodes and Metastasis as the categorical response of interest.

Long-term forecast of eruption style and size at Campi Flegrei caldera (Italy)

The Campi Flegrei caldera is an active and restless volcano in the densely inhabited Neapolitan area of southern Italy. Because of the very high value (lives, properties, infrastructures, etc.) exposed to potential volcanic hazards, it is one of the areas at highest volcanic risk on Earth. In such a situation we have made an attempt to contribute to assessment of its volcanic hazards by providing a quantitative probabilistic long-term forecast of style and size of the next eruption. We have evaluated the most relevant physical parameters of the 22 explosive eruptions of the Campi Flegrei caldera over the past 5 ka. This time span has been taken as the reference period for volcanic hazards assessment on the basis of the volcanic and deformation history of the caldera. The evaluated parameters include dispersal, volume and density of the pyroclastic deposits, volume of erupted magma, total erupted mass, and eruption magnitude. The obtained results permit a size classification of the explosive eruptions, which are grouped into three sizes: small, medium, and large. On the basis of the reconstructed eruption dynamics, we have considered a type event(s) representative of each size class and hypothesized the style of the next event. An effusive eruption will likely generate a dome or very small lava flows, while an explosive event of any size very probably will produce particles fallout and flowage of pyroclastic density currents. Using a Bayesian inference procedure, we have assigned a conditional probability of occurrence to each of the eruption size classes. A small-size explosive eruption is the most likely event with a probability of about 60%; a large-size explosive eruption is the least likely event with a probability of about 4%; a medium-size explosive eruption has a probability of occurrence of about 25%; an effusive eruption has about 11% probability of occurrence.

Revision of the Dipodomys Merriami Complex in the Baja California Peninsula, Mexico

The kangaroo rat Dipodomys merriami occurs widely over the Baja California peninsula, inhabiting all the arid and sandy lowlands. Its range encompasses diverse climatic, edaphic, and vegetation zones, including 2 islands. The 11 taxa recognized in the merriami complex in this region almost 50 years ago remain valid to the present. The 2 island forms originally were described as different species, and have been treated in different ways by subsequent authors. In this investigation we used a genetic analysis of 2 mitochondrial genes, cytochrome b (Cytb) and cytochrome c oxidase subunit III (COIII), to study patterns of genetic structuring in this species throughout the peninsula. We supplemented existing morphometric and pelage color data with new data, and integrated these data sets into, a phylogenetic analysis. Finally, we explored how our results compared with the existing taxonomic arrangement of species and subspecies. Our phylogenetic analyses of molecular data used the concatenation of 1,140 base pairs (bp) of the Cytb gene and 690 bp of the COIII gene; cladograms were generated using maximum-parsimony, maximum-likelihood, and Bayesian inference procedures. A hierarchy of nested clades was produced, the highest level of which revealed 2 reciprocally monophyletic clades separated by 20 mutational steps. There is a Southern Clade extending north to the Vizcaíno Desert in the west and San José Island in the east, and a Northern Clade that includes all the populations farther to the north. At a shallower level, the Southern Clade contains 4 subclades, including the populations of San José and Margarita islands, whereas the Northern Clade is composed of 3 subclades. Adding the morphometric and pelage coloration data sets to the analyses resulted in support for the 11 previously recognized taxa arranged in a single species, Dipodomys merriami, but with improved understanding of how the subspecies relate to each other. As is increasingly appreciated, our investigation supports a comprehensive approach involving multiple data sets that are sensitive to a wide temporal range of evolutionary history for phylogenetic reconstruction.