Variable Selection by Perfect Sampling

Yufei Huang,Petar M Djurić

doi:10.1155/s1110865702000409

Abstract

Variable selection is very important in many fields, and for its resolution many procedures have been proposed and investigated. Among them are Bayesian methods that use Markov chain Monte-Carlo (MCMC) sampling algorithms. A problem with MCMC sampling, however, is that it cannot guarantee that the samples are exactly from the target distributions. This drawback is overcome by related methods known as perfect sampling algorithms. In this paper, we propose the use of two perfect sampling algorithms to perform variable selection within the Bayesian framework. They are the sandwiched coupling from the past (CFTP) algorithm and the Gibbs coupler. We focus our attention to scenarios where the model coefficients and noise variance are known. We indicate the condition under which the sandwiched CFTP can be applied. Most importantly, we design a detailed scheme to adapt the Gibbs coupler algorithm to variable selection. In addition, we discuss the possibilities of applying perfect sampling when the model coefficients and noise variance are unknown. Test results that show the performance of the algorithms are provided.

Highlights

The problem of variable selection is of great importance in many science and engineering areas
We propose to perform variable selection by two perfect sampling algorithms, the sandwiched coupling from the past (CFTP) and the Gibbs coupler
We develop a detailed scheme to accommodate the Gibbs coupler algorithm to variable selection and discuss the possibilities of applying perfect sampling when the model coefficients and the noise variance are unknown

Summary

Introduction

The problem of variable selection is of great importance in many science and engineering areas. A natural and direct approach is to exhaust all the possible subsets This exhaustive search method become computationally burdensome if the available number of predictors is large. Bayesian signal processing methods that use Markov chain Monte-Carlo (MCMC) sampling algorithms [7, 8, 9, 10, 11] have drawn much attention. These methods usually provide better performance than deterministic approaches. The first perfect sampling algorithm was proposed by Propp and Wilson [14], and is called coupling from the past. Efforts have been made to extend it to accommodate perfect sampling from continuous variable spaces [18, 21]

Results

Discussion

Conclusion