A new Bayesian lasso and ridge regression with a practically meaningful parameterization and a simple weakly informative prior

Abstract

Bayesian lasso mimics the regular lasso penalty by placing a double‐exponential prior on the regression coefficients. It automatically provides integrated interval estimates that the regular lasso does not do. Bayesian lasso has found many applications from genetics and genomics to text categorization to traffic safety. The difficulty in specifying a sensible prior for the rate of the double exponential distribution for a particular application, however, is a significant barrier for the wider use of this methodology. This paper proposes a new Bayesian lasso formulation. Instead of using the rate of the double exponential distribution, the new formulation uses a standardized total effect size as the parameter that determines the level of shrinkage with several significant advantages. First, an informative prior is more effectively constructed and understood for this practically meaningful parameter. Second, a weakly informative prior for this new parameter is derived, which allows a practicing statistician to carry out the analysis in an automated way when an informative prior is difficult to elucidate. Third, it is more flexible in modelling prior distributions. A parallel new formulation for Bayesian ridge regression is also provided. A simple and efficient Stan implementation is supplied that can be readily used.