Sparsely restricted penalized estimators

Abstract

Parameters of a linear regression model might be estimated with Ordinary Least Squares (OLS). If there are linear restrictions on model parameters, it is possible to use Restricted Least Squares (RLS). Both OLS and RLS fail when the number of predictors is large and not suitable for model selection in a sparse model. In contrast to OLS and RLS, Least Absolute Shrinkage and Selection Operator (LASSO) and Bridge estimators perform well in sparse models and can be used for model selection and estimation, simultaneously. A Restricted LASSO (RLASSO) estimator is proposed in the literature recently but it doesn’t provide sparse solutions. In this paper, we propose new sparsely restricted penalized estimators called Sparsely Restricted LASSO (SRL) and Sparsely Restricted Bridge (SRB) based on RLASSO and Restricted Bridge (RBridge). We show that SRL and SRB produces sparse solutions while satisfying restrictions of model parameters. We compare aforementioned estimators with Monte Carlo simulations. We also use prostate cancer dataset that is widely used in the literature for a numerical application. Our comparisons show that SRL and SRB outperform the remaining estimators in terms of model selection performance and mean squared error.