Generalized Ridge Regression Estimator in High Dimensional Sparse Regression Models

Mahdi Roozbeh

doi:10.19139/soic.v6i3.581

Abstract

Modern statistical analysis often encounters linear models with the number of explanatory variables much larger than the sample size. Estimation in these high-dimensional problems needs some regularization methods to be employed due to rank deficiency of the design matrix. In this paper, the ridge estimators are considered and their restricted regression counterparts are proposed when the errors are dependent under a multicollinearity and high-dimensionality setting. The asymptotic distributions of the proposed estimators are exactly derived. Incorporating the information contained in the restricted estimator, a shrinkage type ridge estimator is also exhibited and its asymptotic risk is analyzed under some special cases. To evaluate the efficiency of the proposed estimators, a Monte-Carlo simulation along with a real example are considered.

Highlights

Consider a linear regression model: yi = x⊤i β + εi, i = 1, 2, . . . , n, (1.1)where yis are responses, xi =⊤ is design points, β = (β1, . . . , βp)⊤ is vector denoting unknown coefficients, εis are unobservable random errors and the superscript (⊤) denotes the transpose of a vector or matrix
To achieve different degrees of collinearity, following [18, 8], the explanatory variables were generated using the following device for n = 25 and p = 30, 50 and 70 from the following model: xij
To illustrate the usefulness of the suggested strategies for high-dimensional data in the semiparametric regression model, the data set about riboflavin production is considered in Bacillus subtilis, which can be found in R package “hdi”

Summary

Introduction

Εn)⊤ has a cumulative distribution function F (ε); E(ε) = 0 and Var(ε) = σ2Vn, where σ2 is finite and Vn is a known matrix belonging to the space of all positive definite matrices of dimension n × n, denoted by S(n). Now-a-day, many data problems nowadays carry the structure that the number of covariates p may exceed sample size n, known as small n, large p problems. Such cases, that can be seen in the studies of genomics, financial markets, mobile phone communication, bioinformatics and risk management, regularization methods should be considered for inferring (see [9, 5]) about parameters of interest. To mention a few recent researches, see e.g., [1, 2, 3, 4, 14, 15, 16]

Methods

Results

Conclusion