Abstract
High-dimensional data, where the number of variables exceeds or is comparable to the sample size, is now pervasive in many scientific applications. In recent years, Bayesian shrinkage models have been developed as effective and computationally feasible tools to analyze such data, especially in the context of linear regression. In this paper, we focus on the Normal-Gamma shrinkage model developed by Griffin and Brown [7]. This model subsumes the popular Bayesian lasso model, and a three-block Gibbs sampling algorithm to sample from the resulting intractable posterior distribution has been developed in [7]. We consider an alternative two-block Gibbs sampling algorithm, and rigorously demonstrate its advantage over the three-block sampler by comparing specific spectral properties. In particular, we show that the Markov operator corresponding to the two-block sampler is trace class (and hence Hilbert-Schmidt), whereas the operator corresponding to the three-block sampler is not even Hilbert-Schmidt. The trace class property for the two-block sampler implies geometric convergence for the associated Markov chain, which justifies the use of Markov chain CLT’s to obtain practical error bounds for MCMC based estimates. Additionally, it facilitates theoretical comparisons of the two-block sampler with sandwich algorithms which aim to improve performance by inserting inexpensive extra steps in between the two conditional draws of the two-block sampler.
Highlights
In recent years, the explosion of data, due to advances in science and information technology, has left almost no field untouched
We show that when a > 0, the Markov operator corresponding to the three-block Gibbs sampler Φ, with Markov transition density kspecified in (1), is not Hilbert-Schmidt
The two-block Markov chain Φ can be interpreted as a Data Augmentation (DA) algorithm, with (β, σ2) as the parameter block of interest, and τ as the augmented block
Summary
The explosion of data, due to advances in science and information technology, has left almost no field untouched. The Markov operator corresponding to the three-block chain Φis not Hilbert-Schmidt for all values of a (Theorem 2) These results hold for all values of the sample size n and the number of independent variables p. If a self-adjoint Markov operator K (with stationary density π) has a countable spectrum {λi(K)}∞ i=0 (with λ0 = 1), and corresponding sequence of eigenfunctions {φi}∞ i=0, or any h ∈ L2(π), the asymptotic variance of the Markov chain based cumulative averages for estimating Eπ[h] is given by. The results in this paper rigorously establish one way in which blocking affects the properties of hte Normal-Gamma Markov chain, and indicates why one should expect the blocked chain to have better performance in terms of essential sample size and convergence than the original.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
