Abstract
Priors constructed from scale mixtures of normal distributions have long played an important role in decision theory and shrinkage estimation. This paper demonstrates equivalence between the maximum aposteriori estimator constructed under one such prior and Zhang’s minimax concave penalization estimator. This equivalence and related multivariate generalizations stem directly from an intriguing representation of the minimax concave penalty function as the Moreau envelope of a simple convex function. Maximum aposteriori estimation under the corresponding marginal prior distribution, a generalization of the quasi-Cauchy distribution proposed by Johnstone and Silverman, leads to thresholding estimators having excellent frequentist risk properties.
Highlights
Most penalized likelihood estimators have a formal Bayesian interpretation
We focus on the implications of using the priors (1.3) and (1.4) in two maximum aposteriori (MAP) estimation problems when p = 1: assuming Z ∼ N (θ, 1), derive (i) the MAP estimator for (θ, γ) under the improper prior (1.4) by minimizing (1.5) for p = 1 in both θ and γ; and, (ii) the MAP estimator for θ under the proper marginal prior (1.3), or equivalently, (1.2)
This paper demonstrates that several interesting thresholding estimators with good frequentist properties are connected either directly or indirectly to this class of priors, including the minimax concave penalized estimator of [39] (i.e., derived as a MAP under the improper modification (1.4)), thresholding estimators based on generalizations of the quasi-Cauchy prior of [23] (i.e., MAP estimators derived using the marginal prior (1.2)), and a wide class of multivariate generalizations of these results
Summary
Most penalized likelihood estimators have a formal Bayesian interpretation. In particular, the estimates obtained from maximizing a penalized likelihood function may be viewed as the posterior mode, or maximum aposteriori (MAP). The double exponential prior implicit in the LASSO penalization has broader connections to estimation under hierarchical prior specifications involving scale mixtures of normal distributions. Treating λ as a fixed hyperparameter and considering the corresponding generalization of the LASSO penalization, computation of the resulting MAP estimator under the likelihood specification Z ∼ Np(θ, Ip) reduces to determining the value of θ that minimizes. The marginal prior on θ is again observed to correspond to a scale mixture of normal distributions. In contrast to the LASSO and grouped LASSO estimators above, the minimax estimator θJS+(Z) is obtained as a MAP estimator when the posterior distribution under the Takada prior is maximized jointly in both θ and κ, not in θ alone for a fixed value of κ. Considering (ii), we establish a new class of thresholding estimators with desirable frequentist properties, as developed in [2] and [16]
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
