Abstract
Shrinkage prior are becoming more and more popular in Bayesian modeling for high dimensional sparse problems due to its computational efficiency. Recent works show that a polynomially decaying prior leads to satisfactory posterior asymptotics under regression models. In the literature, statisticians have investigated how the global shrinkage parameter, i.e., the scale parameter, in a heavy tail prior affects the posterior contraction. In this work, we explore how the shape of the prior, or more specifically, the polynomial order of the prior tail affects the posterior. We discover that, under the sparse normal means models, the polynomial order does affect the multiplicative constant of the posterior contraction rate. More importantly, if the polynomial order is sufficiently close to 1, it will induce the optimal Bayesian posterior convergence, in the sense that the Bayesian contraction rate is sharply minimax, i.e., not only the order, but also the multiplicative constant of the posterior contraction rate are optimal. The above Bayesian sharp minimaxity holds when the global shrinkage parameter follows a deterministic choice which depends on the unknown sparsity $s$. Therefore, a Beta-prior modeling is further proposed, such that our sharply minimax Bayesian procedure is adaptive to unknown $s$. Our theoretical discoveries are justified by simulation studies.
Highlights
In Bayesian inference for high dimensional sparse models, the prior distribution needs to incorporate certain a priori knowledge of the structural sparsity
We study the performance of the adaptive prior proposed by Theorem 3.1 when s is unknown, and compare it to the adaptive horseshoe prior proposed by
Given a continuous posterior distribution induced by a shrinkage prior, one easy way to perform model selection is to do a threshold truncation, that is, a variable is selected if its posterior summary such as posterior mean is greater than some thresholding value
Summary
In Bayesian inference for high dimensional sparse models, the prior distribution needs to incorporate certain a priori knowledge of the structural sparsity. Given that π0 is polynomially decaying, existing literature focuses on how to (adaptively) choose the global shrinkage τ , i.e., the scale of the prior, such that the order of posterior contraction rate is (near-)optimal In Bayesian literature, the sharpness in term of multiplicative constant is barely investigated, and our work sharpens all existing results on Bayesian posterior convergence for normal means problem To attain such sharp minimaxity, the choice of τ will depend on true sparsity ratio (s/n) which in practice is unknown. 2. Sharp Bayesian minimaxity As discussed, we consider Bayesian inferences for the sparse normal means model under a general prior specification (1.1), where π0 has a polynomial tail; in other words, we assume the following conditions on the model sparsity and prior distribution π0:. Note here we only compare the upper bound of posterior contraction rates obtained by Theorems 2.1 and 2.2, it is not rigorous to claim that the prior specification in Theorem 2.2 leads to suboptimal posterior convergence
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
