Nonlocal Priors Research Articles

Scalable Bayesian Variable Selection Using Nonlocal Prior Densities in Ultrahigh-dimensional Settings.

Bayesian model selection procedures based on nonlocal alternative prior densities are extended to ultrahigh dimensional settings and compared to other variable selection procedures using precision-recall curves. Variable selection procedures included in these comparisons include methods based on g-priors, reciprocal lasso, adaptive lasso, scad, and minimax concave penalty criteria. The use of precision-recall curves eliminates the sensitivity of our conclusions to the choice of tuning parameters. We find that Bayesian variable selection procedures based on nonlocal priors are competitive to all other procedures in a range of simulation scenarios, and we subsequently explain this favorable performance through a theoretical examination of their consistency properties. When certain regularity conditions apply, we demonstrate that the nonlocal procedures are consistent for linear models even when the number of covariates p increases sub-exponentially with the sample size n. A model selection procedure based on Zellner's g-prior is also found to be competitive with penalized likelihood methods in identifying the true model, but the posterior distribution on the model space induced by this method is much more dispersed than the posterior distribution induced on the model space by the nonlocal prior methods. We investigate the asymptotic form of the marginal likelihood based on the nonlocal priors and show that it attains a unique term that cannot be derived from the other Bayesian model selection procedures. We also propose a scalable and efficient algorithm called Simplified Shotgun Stochastic Search with Screening (S5) to explore the enormous model space, and we show that S5 dramatically reduces the computing time without losing the capacity to search the interesting region in the model space, at least in the simulation settings considered. The S5 algorithm is available in an R package BayesS5 on CRAN.

Reweighted Infrared Patch-Tensor Model With Both Nonlocal and Local Priors for Single-Frame Small Target Detection

Many state-of-the-art methods have been proposed for infrared small target detection. They work well on the images with homogeneous backgrounds and high-contrast targets. However, when facing highly heterogeneous backgrounds, they would not perform very well, mainly due to: 1) the existence of strong edges and other interfering components, 2) not utilizing the priors fully. Inspired by this, we propose a novel method to exploit both local and non-local priors simultaneously. Firstly, we employ a new infrared patch-tensor (IPT) model to represent the image and preserve its spatial correlations. Exploiting the target sparse prior and background non-local self-correlation prior, the target-background separation is modeled as a robust low-rank tensor recovery problem. Moreover, with the help of the structure tensor and reweighted idea, we design an entry-wise local-structure-adaptive and sparsity enhancing weight to replace the globally constant weighting parameter. The decomposition could be achieved via the element-wise reweighted higher-order robust principal component analysis with an additional convergence condition according to the practical situation of target detection. Extensive experiments demonstrate that our model outperforms the other state-of-the-arts, in particular for the images with very dim targets and heavy clutters.

Germ-line biomarkers disrupting microRNA regulatory pathways to predict toxicity and response to anti-PD-1 and anti-PD-L1 therapies.

3040 Background: Identifying responders to developing immune therapies is of high priority, yet identifying patients who will suffer from toxicity, which represents a unique spectrum of side effects, termed immune-related adverse events (irAEs), is critical. The complex host-specific response to immune therapies led us to hypothesize that germ-line mutations, present in both the tumor and in the host’s immune system, may be a source of biomarkers. As there is compelling evidence that recently discovered germ-line mutations disrupting microRNA (miRNA) networks act as biomarkers of treatment response and baseline immune status, we evaluated such mutations in patients treated with these therapies. Methods: Patients treated with anti-PD1 or anti-PDL1 therapy alone and clinically documented response and toxicity were tested with a panel of over 150 vetted germ-line miRNA based biomarkers. Of 85 patients evaluated for response, 75 had melanoma, and 10 had NSCLC. For analysis of toxicity, an additional 4 patients with other cancer types were included. Subjects were classified as responders (Complete, Partial) versus non-responders (Progressive, Stable), or as experiencing low irAEs (grade 0 and 1) versus high irAEs (grade 2 and above). We used Chi-squared analysis for base evaluation and then compared three classification techniques including classification trees, random forests and Bayesian probit regression with non-local priors. We estimated classification performance via leave-one-out cross validation. Results: By Chi-Squared analysis we found over a dozen biomarkers associated with response, and a dozen biomarkers associated with toxicity. These included germ-line mutations in 3’ untranslated regions as well as in miRNA promoter regions. We found that both classification trees and Bayesian probit compared reasonably well with random forests. We found a response signature with a specificity of 89% by Random Forests, and a toxicity signature with a specificity of 76% by Random Forests. Conclusions: We have shown that a new class of germ-line mutations disrupting miRNA regulatory networks can act as biomarkers of response and toxicity to anti-PD1 and anti-PDL1 therapy.

Bayesian Wavelet Analysis Using Nonlocal Priors with an Application to fMRI Analysis

We propose a Bayesian hierarchical wavelet methodology for nonparametric regression based on nonlocal priors. Our methodology assumes for the wavelet coefficients a prior that is a mixture of a point mass at zero and a Johnson-Rossell nonlocal prior. We consider two choices of Johnson-Rossell nonlocal priors: the moment prior and the inverse moment prior. To borrow strength across wavelet coefficients, in addition to more traditional specifications from the wavelet literature, we consider a logit specification for the mixture probability and a polynomial decay specification for the scale parameter. To estimate these specifications’ hyperparameters, we propose an empirical Bayes methodology based on Laplace approximation that allows fast analytic posterior inference for the wavelet coefficients. To assess performance, we perform a simulation study to compare our methodology to several other wavelet-based methods available in the literature. In terms of mean squared error, our methodology with the inverse moment prior yields superior results for cases of low signal-to-noise ratio, as well as for moderate to large sample sizes. Finally, we illustrate the flexibility of our novel methodology with an application to a functional magnetic resonance imaging (fMRI) dataset from a study of brain activations associated with working memory.

Nonlocal Priors for High-Dimensional Estimation

ABSTRACTJointly achieving parsimony and good predictive power in high dimensions is a main challenge in statistics. Nonlocal priors (NLPs) possess appealing properties for model choice, but their use for estimation has not been studied in detail. We show that for regular models NLP-based Bayesian model averaging (BMA) shrink spurious parameters either at fast polynomial or quasi-exponential rates as the sample size n increases, while nonspurious parameter estimates are not shrunk. We extend some results to linear models with dimension p growing with n. Coupled with our theoretical investigations, we outline the constructive representation of NLPs as mixtures of truncated distributions that enables simple posterior sampling and extending NLPs beyond previous proposals. Our results show notable high-dimensional estimation for linear models with p > >n at low computational cost. NLPs provided lower estimation error than benchmark and hyper-g priors, SCAD and LASSO in simulations, and in gene expression data achieved higher cross-validated R2 with less predictors. Remarkably, these results were obtained without prescreening variables. Our findings contribute to the debate of whether different priors should be used for estimation and model selection, showing that selection priors may actually be desirable for high-dimensional estimation. Supplementary materials for this article are available online.

A New Family of Non-Local Priors for Chain Event Graph Model Selection

Chain Event Graphs (CEGs) are a rich and provenly useful class of graphical models. The class contains discrete Bayesian Networks as a special case and is able to depict directly the asymmetric context-specific statements in the model. But bespoke efficient algorithms now need to be developed to search the enormous CEG model space. In different contexts Bayes Factor scored search algorithm using non-local priors (NLPs) has recently proved very successful for searching other huge model spaces. Here we define and explore three different types of NLP that we customise to search CEG spaces. We demonstrate how one of these candidate NLPs provides a framework for search which is both robust and computationally efficient. It also avoids selecting an overfitting model as the standard conjugate methods sometimes do. We illustrate the efficacy of our methods with two examples. First we analyse a previously well-studied 5-year longitudinal study of childhood hospitalisation. The second much larger example selects between competing models of prisoners’ radicalisation in British prisons: because of its size an application beyond the scope of earlier Bayes Factor search algorithms.

Bayesian variable selection for binary outcomes in high-dimensional genomic studies using non-local priors.

Motivation: The advent of new genomic technologies has resulted in the production of massive data sets. Analyses of these data require new statistical and computational methods. In this article, we propose one such method that is useful in selecting explanatory variables for prediction of a binary response. Although this problem has recently been addressed using penalized likelihood methods, we adopt a Bayesian approach that utilizes a mixture of non-local prior densities and point masses on the binary regression coefficient vectors.Results: The resulting method, which we call iMOMLogit, provides improved performance in identifying true models and reducing estimation and prediction error in a number of simulation studies. More importantly, its application to several genomic datasets produces predictions that have high accuracy using far fewer explanatory variables than competing methods. We also describe a novel approach for setting prior hyperparameters by examining the total variation distance between the prior distributions on the regression parameters and the distribution of the maximum likelihood estimator under the null distribution. Finally, we describe a computational algorithm that can be used to implement iMOMLogit in ultrahigh-dimensional settings () and provide diagnostics to assess the probability that this algorithm has identified the highest posterior probability model.Availability and implementation: Software to implement this method can be downloaded at: http://www.stat.tamu.edu/∼amir/code.html.Contact: wwang7@mdanderson.org or vjohnson@stat.tamu.eduSupplementary information: Supplementary data are available at Bioinformatics online.

Super‐resolution reconstruction based on multisource bidirectional similarity and non‐local similarity matching

A novel super-resolution (SR) reconstruction algorithm based on multisource bidirectional similarity and non-local similarity matching for multi-exposure dynamic image sequences is proposed in this study. First, luminance compensation for multi-exposure dynamic images is implemented based on multisource bidirectional similarity. Then, combining a self-adaptive regional correlation evaluation strategy and a weighting strategy based on pseudo-Zernike moment feature similarity and structural similarity yields a novel and robust non-local similarity matching scheme (PZ-NSM) to learn non-local similarity priors between low- and high-resolution dynamic image patches at different spatio-temporal scales without additional training. Finally, SR reconstruction is implemented based on PZ-NSM and spatio-temporal trans-scale fusion of non-local similarities between dynamic images. The proposed algorithm does not rely on accurate estimation of subpixel motion and can therefore be adapted to more complex motion patterns. It has high rotation invariance effectiveness and is robust to noise and illumination. Experimental results demonstrate that the proposed algorithm outperforms existing algorithms in terms of both subjective and objective evaluations.

Salient Object Detection with Higher Order Potentials and Learning Affinity

In this paper, we propose a novel graph-based salient object detection algorithm which exploits higher order potential to capture the cross-scale grouping cues instead of using multi-scale graph model or naive multi-scale fusion (i.e. individually compute a saliency result for each scale and then combine them). And, we investigate the importance of graph affinities in graph labeling. We take both local (spatial distribution) and nonlocal (feature distribution) priors into account and learn the pairwise similarity values in a semi-supervised manner, thereby obtaining a faithful graph affinity model. With the guidance of foreground and background seeds, salient object detection is formulated as a labeling inference problem. Extensive experiments on two large benchmark datasets demonstrate the proposed method performs well when against the state-of-the-art methods in terms of accuracy.

Image deblurring with mixed regularization via the alternating direction method of multipliers

In image deblurring problems, both local and nonlocal regularization priors are well studied. Local regularization prior assumes piecewise smoothness and transform-based sparsity, while the nonlocal one exploits self-similarity of images. We proposed a mixed regularization model which incorporates the advantages of both local adaptive sparsity prior and nonlocal sparsity prior resulting from the nonlocal self-similarity, and thus encourages a solution to simultaneously express both the local and nonlocal natures of images. The deblurring problem with mixed regularization can be transformed into a constrained optimization problem with separable structure via the variable splitting. Then this constrained optimization problem is solved by the alternating direction method of multipliers. Experimental results with a set of images under varying conditions demonstrate that the proposed method achieves the state-of-the-art deblurring performance.

Sparse nonlocal priors based two-phase approach for mixed noise removal

Mixed noise removal is a challenging problem due to the complexity of statistical model of image noise. Additive white Gaussian noise (AWGN) combined with impulse noise (IN) is a representative among commonly encountered mixed noise. At present, nonlocal self-similarity (NSS) prior coupled with adaptive regularization have shown great potential in AWGN removal and led to satisfactory denoising performance. However, few studies unify these properties to remove mixture of AWGN and IN. In this paper, we propose a simple yet effective method, namely sparse nonlocal priors based two-phase approach (SNTP), for mixed noise removal. In SNTP, a median-type filter is used to detect outlier pixels which are likely to be corrupted by IN, and the remaining pixels are mainly corrupted by AWGN. We recover the image by encoding free-outlier pixels over a pre-learned dictionary to remove AWGN, and integrate the image sparse nonlocal priors as a regularization term. Meanwhile, adaptive regularization is used to further improve the denoising performance. Experimental results show that the proposed SNTP algorithm outperforms state-of-the-art mixed noise removal methods in terms of both quantitative measures and visual perception quality.

Wavelet Frame Based Image Restoration via Combined Sparsity and Nonlocal Prior of Coefficients

Owing to the good ability of sparsely approximating piece-wise smooth functions like images, the (tight) wavelet frame has been widely investigated and applied for image restoration and other image processing problems over the past few years. Most of the variational models based on wavelet frame proposed in the past utilize the $$l_{1}$$l1 norm of frame coefficients as a sparsity prior. Very recently, the variational model which penalizes the $$l_{0}$$l0 "norm" of frame coefficients was proposed for image restoration, and proved to outperform the commonly used $$l_{1}$$l1 minimization methods in the quality of restored images. Though the $$l_{0}$$l0 "norm" has the ability of preserving sharp edges and smooth regions, textures and small details may be mistakenly removed at the same time. Therefore, we introduce a $$l_0-l_2$$l0-l2 regularization model which contains a nonlocal prior of frame coefficients to avoid this issue in this paper. Meanwhile, a narrow-band technique is introduced to further improve the computational efficiency of the proposed algorithm. Numerical experiments demonstrate that the propose algorithm is superior to the recently proposed algorithm for $$l_{0}$$l0 "norm" minimization in both iterative time and recovery quality.

Power-Expected-Posterior Priors for Variable Selection in Gaussian Linear Models

In the context of the expected-posterior prior (EPP) approach to Bayesian variable selection in linear models, we combine ideas from power-prior and unit-information-prior methodologies to simultaneously (a) produce a minimally-informative prior and (b) diminish the effect of training samples. The result is that in practice our power-expected-posterior (PEP) methodology is sufficiently insensitive to the size n* of the training sample, due to PEP’s unit-information construction, that one may take n* equal to the full-data sample size n and dispense with training samples altogether. This promotes stability of the resulting Bayes factors, removes the arbitrariness arising from individual training-sample selections, and greatly increases computational speed, allowing many more models to be compared within a fixed CPU budget. We find that, under an independence Jeffreys (reference) baseline prior, the asymptotics of PEP Bayes factors are equivalent to those of Schwartz’s Bayesian Information Criterion (BIC), ensuring consistency of the PEP approach to model selection. Our PEP prior, due to its unit-information structure, leads to a variable-selection procedure that — in our empirical studies — (1) is systematically more parsimonious than the basic EPP with minimal training sample, while sacrificing no desirable performance characteristics to achieve this parsimony; (2) is robust to the size of the training sample, thus enjoying the advantages described above arising from the avoidance of training samples altogether; and (3) identifies maximum-a-posteriori models that achieve better out-of-sample predictive performance than that provided by standard EPPs, the g-prior, the hyper-g prior, non-local priors, the Least Absolute Shrinkage and Selection Operator (LASSO) and Smoothly-Clipped Absolute Deviation (SCAD) methods.

Diffuse visual attention for saliency detection

This paper presents a physical perspective to understand visual saliency on the scene and develops an anisotropic heat diffusion based formulation, named visual attention diffusion (VAD), to detect salient regions in natural images. Our idea is based on the assumption that the visual attention can be diffused from a part of the most representative salient elements to other salient regions for a given image. In particular, we first design a corner-points-driven technique to simultaneously identify candidate foreground and background regions on the image. Then both the local object location and the nonlocal background structure can be estimated accordingly. Finally, we design a geometry-driven anisotropic Poisson system with Dirichlet boundary for saliency detection which uses the representative salient elements in the candidate foreground as heat sources and both local and nonlocal priors as guidance. After heat diffusion reaches a stable state, salient regions can be easily detected based on the temperatures (i.e., score value) of the image elements. Experiments show that the proposed system can produce more precise and reliable results compared to state-of-the-art saliency detectors.

Remote Sensing Image Super-Resolution Reconstruction Based on Nonlocal Pairwise Dictionaries and Double Regularization

A nonlocal pairwise dictionary learning (NPDL) model that includes an estimated dictionary and a residual dictionary is applied to remote sensing image super-resolution (SR) reconstruction in this paper. The dictionary pair is trained from some low-resolution (LR) remote sensing images to deal with the lack of high-resolution component in remote sensing images. The reconstructed image has been shown to retain the structural information of the given LR image itself. Moreover, the local and nonlocal (NL) priors are used for image SR to enhance robustness of the pairwise dictionary. Improved NL self-similarity and local kernel constraint regularization terms are introduced to the image optimization process. Using this, the photometric, geometric, and feature information of the given LR image can be taken into consideration to improve the quality of reconstruction. Simulation results show that the proposed algorithm can achieve better visual effects and the average peak signal-to-noise ratio (PSNR) is improved by approximately 0.5 db compared with the state-of-the-art image SR methods.

Depth map super-resolution using stereo-vision-assisted model

Abstract In this paper, we propose a novel Stereo-Vision-Assisted (SVA) model for depth map super-resolution. Given a low-resolution depth map as input, we recover a high-resolution depth map using the registered high-resolution color stereo image pair. First, based on the mutual benefits between raw depth map and features of high resolution color image, we model the relationship with two constraint terms of local and non-local priors and sufficiently explore their complementary nature. Moreover, by considering reliable disparity pixels calculated from stereo matching algorithm, we formulate a stereo disparity regularization term to further reinforce the preservation of fine depth detail. Hence, our SVA objective function includes a non-local prior constraint, a local prior constraint and a stereo disparity prior constraint. In addition, we employ an efficient algorithm to optimize the objective function. Experimental results demonstrate that our approach can obtain superior performance in comparison with other methods.

Depth map super-resolution via local and nonlocal priors

We propose a method for solving the depth map super-resolution problem. Given a low-resolution depth map as input, we recover a high-resolution depth map using a registered and potentially high-resolution color image. We formulate the super-resolution process as a regularization and optimization framework. Specifically, taking into account that discontinuities in depth and color tend to coalign, we obtain the local and nonlocal regularization terms based on the raw depth map and also the features of high-resolution color images. Different from conventional methods, we model the relationship with two constraint terms of local and nonlocal priors to sufficiently explore the complementary nature. Experimental results demonstrate the effectiveness of our approach, which produces sharper edges and more faithful details compared with other state-of-the-art strategies.

The Whetstone and the Alum Block: Balanced Objective Bayesian Comparison of Nested Models for Discrete Data

When two nested models are compared, using a Bayes factor, from an objective standpoint, two seemingly conflicting issues emerge at the time of choosing parameter priors under the two models. On the one hand, for moderate sample sizes, the evidence in favor of the smaller model can be inflated by diffuseness of the prior under the larger model. On the other hand, asymptotically, the evidence in favor of the smaller model typically accumulates at a slower rate. With reference to finitely discrete data models, we show that these two issues can be dealt with jointly, by combining intrinsic priors and nonlocal priors in a new unified class of priors. We illustrate our ideas in a running Bernoulli example, then we apply them to test the equality of two proportions, and finally we deal with the more general case of logistic regression models.

D.E. Anderson, D.M. Rings, Current Veterinary Therapy Food Animal Practice, 5th ed., Saunders Elsevier, 2008, ISBN9781416035916, 736 pp.; £110.00 (hard)

In this paper, we propose a novel Stereo-Vision-Assisted (SVA) model for depth map super-resolution. Given a low-resolution depth map as input, we recover a high-resolution depth map using the registered high-resolution color stereo image pair. First, based on the mutual benefits between raw depth map and features of high resolution color image, we model the relationship with two constraint terms of local and non-local priors and sufficiently explore their complementary nature. Moreover, by considering reliable disparity pixels calculated from stereo matching algorithm, we formulate a stereo disparity regularization term to further reinforce the preservation of fine depth detail. Hence, our SVA objective function includes a non-local prior constraint, a local prior constraint and a stereo disparity prior constraint. In addition, we employ an efficient algorithm to optimize the objective function. Experimental results demonstrate that our approach can obtain superior performance in comparison with other methods.