New approach to Bayesian high-dimensional linear regression

Abstract

Abstract Consider the problem of estimating parameters $X^n \in \mathbb{R}^n $, from $m$ response variables $Y^m = AX^n+Z^m$, under the assumption that the distribution of $X^n$ is known. Lack of computationally feasible algorithms that employ generic prior distributions and provide a good estimate of $X^n$ has limited the set of distributions researchers use to model the data. To address this challenge, in this article, a new estimation scheme named quantized maximum a posteriori (Q-MAP) is proposed. The new method has the following properties: (i) In the noiseless setting, it has similarities to maximum a posteriori (MAP) estimation. (ii) In the noiseless setting, when $X_1,\ldots,X_n$ are independent and identically distributed, asymptotically, as $n$ grows to infinity, its required sampling rate ($m/n$) for an almost zero-distortion recovery approaches the fundamental limits. (iii) It scales favorably with the dimensions of the problem and therefore is applicable to high-dimensional setups. (iv) The solution of the Q-MAP optimization can be found via a proposed iterative algorithm that is provably robust to error (noise) in response variables.