Structural regularization in quadratic logistic regression model

Abstract

In statistical modeling, quadratic model including both main effects and interactions has drawn a large deal of attentions from researchers in many scientific fields. Researchers have found that it is extremely significant to maintain the heredity principle such as strong or weak heredity principle among variables when demanding sparsity in quadratic model. The reason why heredity principle is preferred is that model following logic structure is invariant to any scale transformation and is more stable when implementing forecasting and classification task. Although a large bore have studied quadratic models, most of them focus on the model performances in a regression problem and no systematically comparison are made in terms of classification accuracy. This paper investigates and studies group regularized estimation under structural hierarchy for classification (C-GRESH). In computation, a fast and simple-to-implement algorithm is designed with theoretical guarantee of its convergence. Furthermore, an accelerated gradient method is applied to speed up the convergence. Theoretically, we have shown the adaptive version of the proposed approach is able to achieve oracle property which includes asymptotic normality and model selection consistency. Simulation examples and real data examples including microarray gene expression datasets are shown to demonstrate the efficiency and superior performances of the proposed method over other existing competitors.