Sparse Regression Methods Research Articles

Sketch-Based Region Adaptive Sparse Unmixing Applied to Hyperspectral Image

Hyperspectral image (HSI) unmixing is an important issue of research due to its effect on the subsequent processing of HSIs. Recently, the sparse regression method with spatial information has been successfully applied in hyperspectral unmixing (HU). However, most sparse regression methods ignore the difference in spatial structure handling with only one sparse constraint. In fact, the pixels in detail regions are more likely to be severely mixed with more endmembers participated, and the sparsity degree of its corresponding abundances is relatively low. Considering the sparsity difference of abundances, a sketch-based region adaptive sparse unmixing applied to HSI is proposed in this article. Inspired by the vision computing theory, we use the region generation algorithm based on a sketch map to differentiate the homogeneous regions and detail regions. Then, the abundances of these two kind regions in HSIs are separately constrained by sparse regularizers of ${L}_{1/2}$ and ${L}_{1}$ with a proposed manifold constraint. Our method not only makes full use of the spatial information in HSIs but also exploits the latent structure of data. The encouraging experimental results on three data sets validate the effectiveness of our method for HU.

Unmixing of Hyperspectral Data Using Spectral Libraries

In hyperspectral images, pixels are found as a mixture of the spectral signatures of several materials, especially when there is an insufficient spatial resolution. In recent years, spectral libraries have provided spectral information of hundreds of materials that allow the development of techniques to solve the problem of hyperspectral unmixing in a semi-supervised fashion. These methods which are also known as sparse regression techniques assume that mixed pixels are a sparse linear combination of spectral signatures of materials in already available spectral libraries. In this paper, the spectral mixing problem has been solved via sparse separation methods. The United States Geological Survey (USGS) spectral library is used to generate simulated hyperspectral data. A comparative analysis is performed to determine which material signatures in the library are mixed in the pixels by using the convex-relaxation-based sparse regression methods. Root Mean Square Error (RMSE), Signal to Reconstruction Error (SRE) and processing time of the algorithms are used as comparing criterions. Moreover, Hinton diagrams are used to visualize which material signatures are found in the library and the proportions of these found material signatures.

Robust Low-Rank Discovery of Data-Driven Partial Differential Equations

Partial differential equations (PDEs) are essential foundations to model dynamic processes in natural sciences. Discovering the underlying PDEs of complex data collected from real world is key to understanding the dynamic processes of natural laws or behaviors. However, both the collected data and their partial derivatives are often corrupted by noise, especially from sparse outlying entries, due to measurement/process noise in the real-world applications. Our work is motivated by the observation that the underlying data modeled by PDEs are in fact often low rank. We thus develop a robust low-rank discovery framework to recover both the low-rank data and the sparse outlying entries by integrating double low-rank and sparse recoveries with a (group) sparse regression method, which is implemented as a minimization problem using mixed nuclear norms with ℓ1 and ℓ0 norms. We propose a low-rank sequential (grouped) threshold ridge regression algorithm to solve the minimization problem. Results from several experiments on seven canonical models (i.e., four PDEs and three parametric PDEs) verify that our framework outperforms the state-of-art sparse and group sparse regression methods. Code is available at https://github.com/junli2019/Robust-Discovery-of-PDEs

A novel method for feature learning and network intrusion classification

With the rapid advancement in technology, network systems are becoming prone to more sophisticated types of intrusions. However, machine learning (ML) based strategies are among the most efficient and popular methods to identify the network intrusions or attacks. In this study, we examined the important and discriminative features, in order to recognize the various attacks by applying the Structural Sparse Logistic Regression (SSPLR) and Support Vector Machine (SVMs) methods. The SVMs are standard ML-based techniques, which provide the reasonable performance, however, they have few shortcomings, such as, interpretability and huge computational cost. On the other hand, the sparse modeling (SSPLR) is considered as the advanced method for the data examination and processing through regularization. The structural sparse modeling can be used to simultaneously select the distinct features or the group of discriminative features from the repository of the data set to determine the coefficient of the linear classifier, where, prior information of the feature’s structure can be mapped on various sparsity-inducing regularizations. In this way, the particular group of features yielded by the most significant network attacks are selected and potentially identified. The experiments and discussion, show that the proposed techniques have improved performance compared to the most state-of-the-art techniques, used for the Intrusion Detection System (IDS).

Sparse hierarchical regression with polynomials

We present a novel method for sparse polynomial regression. We are interested in that degree r polynomial which depends on at most k inputs, counting at most $$\ell$$ monomial terms, and minimizes the sum of the squares of its prediction errors. Such highly structured sparse regression was denoted by Bach (Advances in neural information processing systems, pp 105–112, 2009) as sparse hierarchical regression in the context of kernel learning. Hierarchical sparse specification aligns well with modern big data settings where many inputs are not relevant for prediction purposes and the functional complexity of the regressor needs to be controlled as to avoid overfitting. We propose an efficient two-step approach to this hierarchical sparse regression problem. First, we discard irrelevant inputs using an extremely fast input ranking heuristic. Secondly, we take advantage of modern cutting plane methods for integer optimization to solve the remaining reduced hierarchical $$(k, \ell )$$-sparse problem exactly. The ability of our method to identify all k relevant inputs and all $$\ell$$ monomial terms is shown empirically to experience a phase transition. Crucially, the same transition also presents itself in our ability to reject all irrelevant features and monomials as well. In the regime where our method is statistically powerful, its computational complexity is interestingly on par with Lasso based heuristics. Hierarchical sparsity can retain the flexibility of general nonparametric methods such as nearest neighbors or regression trees (CART), without sacrificing much statistical power. The presented work hence fills a void in terms of a lack of powerful disciplined nonlinear sparse regression methods in high-dimensional settings. Our method is shown empirically to scale to regression problems with $$n\approx 10{,}000$$ observations for input dimension $$p\approx 1000$$.

A surrogate ℓ0 sparse Cox's regression with applications to sparse high-dimensional massive sample size time-to-event data.

Sparse high-dimensional massive sample size (sHDMSS) time-to-event data present multiple challenges to quantitative researchers as most current sparse survival regression methods and software will grind to a halt and become practically inoperable. This paper develops a scalable ℓ0 -based sparse Cox regression tool for right-censored time-to-event data that easily takes advantage of existing high performance implementation of ℓ2 -penalized regression method for sHDMSS time-to-event data. Specifically, we extend the ℓ0 -based broken adaptive ridge (BAR) methodology to the Cox model, which involves repeatedly performing reweighted ℓ2 -penalized regression. We rigorously show that the resulting estimator for the Cox model is selection consistent, oracle for parameter estimation, and has a grouping property for highly correlated covariates. Furthermore, we implement our BAR method in an R package for sHDMSS time-to-event data by leveraging existing efficient algorithms for massive ℓ2 -penalized Cox regression. We evaluate the BAR Cox regression method by extensive simulations and illustrate its application on an sHDMSS time-to-event data from the National Trauma Data Bank with hundreds of thousands of observations and tens of thousands sparsely represented covariates.

Sparse Hyperspectral Unmixing Using Spectral Library Adaptive Adjustment

Sparse regression methods based on the spectral libraries have shown promise for hyperspectral unmixing, which assumes that the observed image can be represented linearly by a small subset of a known spectral library. However, the spectral mismatches are often assumed to occur between the actual endmembers and the corresponding signatures in the spectral library, which greatly limits the application of the sparse unmixing method. In this article, we address the abovementioned problem by adaptively adjusting the spectral library to match the current hyperspectral image during the unmixing procedure and propose a spectral library adaptive collaborative sparse unmixing model. In our model, we introduce both the spectral scaling factors and the spectral residual variables, which can adjust the spectral library in terms of spectral intensity and local spectral variations, respectively. Moreover, the first-order sparse property of the spectral difference is utilized to improve the accuracy of the spectral library adjustment. We solve our model by the alternating direction method of multipliers. Simulations and real-data experiments show that the proposed method is effective for reducing the impact of spectral library mismatches compared with state-of-the-art algorithms.

Active Polynomial Chaos Expansion for Reliability-Based Design Optimization

The aim of the present paper is to develop a strategy for solving reliability-based design optimization (RBDO) problems by sparse polynomial chaos expansion (PCE). Because classic sparse regression methods cannot provide the surrogate error measure that can be employed to improve the sampling performance for failure probability estimation in RBDO, Bayesian compressed sensing with state-of-the-art performance is employed to build sparse PCE in the paper. In the meanwhile, a new active learning function is proposed to adaptively select new training points. Two goals can be achieved using this criterion; that is, most of the selected training points are selected from the desired regions to approximate limit state surfaces, and these points tend to be far away from the existing points in the current design to avoid the clustering problem. Because the sparse PCEs are built in an augmented space, it is made numerically affordable to solve the RBDO problem with no extra computational cost. The computation capability of the proposed method is demonstrated by several analytical RBDO problems. Meanwhile, the design optimization of a stiffened rib of the wing edge in a certain aircraft also verifies its good engineering applicability.

Abstract 3578: Comprehensive pharmacogenomic analysis establishes lncRNAs as protein-coding independent biomarker of drug response in human cancers

Abstract Somatic alterations in the cancer genome influence a patient’s response to anti-cancer therapeutics. Thus, identifying context-specific alterations associated with differential drug sensitivity can improve patient response prediction and therapeutic selection. The discovery of pervasive dysregulation of long non-coding RNAs (lncRNA) in human cancers and their perceived role in gene regulation has led to the speculation that lncRNAs may also play a role in determining cancer drug response. However, it is unclear whether lncRNAs can augment the existing, predominantly protein coding, repertoire of pharmacogenomic biomarkers of anti-cancer agents. We performed comprehensive analysis of whole genome and transcriptome data from over 1000 cancer cell lines and drug screening data for over 500 anti-cancer agents to systematically analyze over 5 million lncRNA-drug associations. We developed statistical and machine-learning frameworks to study these associations while controlling for the effects of confounding PCGs. Sparse regression and network-based methods demonstrated the predictive ability and biological relevance of literature-supported and novel lncRNAs biomarkers. Using regression analysis of lncRNA expression controlling for neighboring PCGs against drug response, we established that a large proportion of the lncRNAs candidates are PCG-independent biomarkers and potent predictors of cancer drug response. We further identified response-associated somatic alterations specifically in lncRNA genome that do not overlap PCGs. In addition, we demonstrated that collinear lncRNAs might be biologically relevant determinants of drug response. As an example, we identified EGFR-AS1 and MIR205HG as novel predictors of anti-EGFR therapy, which explain a significantly larger proportion of variability in erlotinib and gefitinib response as compared to EGFR mutations and copy-number alterations in drug screens and in patient data. We validated our findings in 16 non-small cell lung cancer and erlotinib-resistant cell lines. Our knockdown experiments revealed mechanisms of erlotinib sensitivity mediated by the two lncRNAs without influencing EGFR expression. In conclusion, our comprehensive analysis generated unprecedented insights into the role of lncRNAs in cancer drug response that reaches beyond PCGs. This study will serve as a foundation for future cancer pharmacogenomic studies and will be an invaluable resource for investigators seeking insights into mechanisms of drug response. Citation Format: Aritro Nath, Eunice Y. Lau, Adam M. Lee, Paul Geeleher, William C. Cho, R. Stephanie Huang. Comprehensive pharmacogenomic analysis establishes lncRNAs as protein-coding independent biomarker of drug response in human cancers [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2019; 2019 Mar 29-Apr 3; Atlanta, GA. Philadelphia (PA): AACR; Cancer Res 2019;79(13 Suppl):Abstract nr 3578.

Oracle inequalities for sign constrained generalized linear models

High-dimensional data have recently been analyzed because of advancements in data collection technology. Although many methods have been developed for sparse recovery in the past 20 years, most of these methods require the selection of tuning parameters. This requirement means that the results obtained with these methods heavily depend on tuning. Theoretical properties are developed for sign-constrained generalized linear models with convex loss function, which is one of the sparse regression methods that does not require tuning parameters. Recent studies on this subject have shown that, in the case of linear regression, sign-constraints alone could be as efficient as the oracle method if the design matrix enjoys a suitable assumption in addition to a traditional compatibility condition. This type of result is generalized to a model that encompasses logistic and quantile regressions. Some numerical experiments are performed to confirm the theoretical findings.

A Phylogeny-Regularized Sparse Regression Model for Predictive Modeling of Microbial Community Data.

Fueled by technological advancement, there has been a surge of human microbiome studies surveying the microbial communities associated with the human body and their links with health and disease. As a complement to the human genome, the human microbiome holds great potential for precision medicine. Efficient predictive models based on microbiome data could be potentially used in various clinical applications such as disease diagnosis, patient stratification and drug response prediction. One important characteristic of the microbial community data is the phylogenetic tree that relates all the microbial taxa based on their evolutionary history. The phylogenetic tree is an informative prior for more efficient prediction since the microbial community changes are usually not randomly distributed on the tree but tend to occur in clades at varying phylogenetic depths (clustered signal). Although community-wide changes are possible for some conditions, it is also likely that the community changes are only associated with a small subset of “marker” taxa (sparse signal). Unfortunately, predictive models of microbial community data taking into account both the sparsity and the tree structure remain under-developed. In this paper, we propose a predictive framework to exploit sparse and clustered microbiome signals using a phylogeny-regularized sparse regression model. Our approach is motivated by evolutionary theory, where a natural correlation structure among microbial taxa exists according to the phylogenetic relationship. A novel phylogeny-based smoothness penalty is proposed to smooth the coefficients of the microbial taxa with respect to the phylogenetic tree. Using simulated and real datasets, we show that our method achieves better prediction performance than competing sparse regression methods for sparse and clustered microbiome signals.

Approximate Residual Balancing: Debiased Inference of Average Treatment Effects in High Dimensions

SummaryThere are many settings where researchers are interested in estimating average treatment effects and are willing to rely on the unconfoundedness assumption, which requires that the treatment assignment be as good as random conditional on pretreatment variables. The unconfoundedness assumption is often more plausible if a large number of pretreatment variables are included in the analysis, but this can worsen the performance of standard approaches to treatment effect estimation. We develop a method for debiasing penalized regression adjustments to allow sparse regression methods like the lasso to be used for √n-consistent inference of average treatment effects in high dimensional linear models. Given linearity, we do not need to assume that the treatment propensities are estimable, or that the average treatment effect is a sparse contrast of the outcome model parameters. Rather, in addition to standard assumptions used to make lasso regression on the outcome model consistent under 1-norm error, we require only overlap, i.e. that the propensity score be uniformly bounded away from 0 and 1. Procedurally, our method combines balancing weights with a regularized regression adjustment.

Robust and Sparse Regression via γ-Divergence

In high-dimensional data, many sparse regression methods have been proposed. However, they may not be robust against outliers. Recently, the use of density power weight has been studied for robust parameter estimation, and the corresponding divergences have been discussed. One such divergence is the γ -divergence, and the robust estimator using the γ -divergence is known for having a strong robustness. In this paper, we extend the γ -divergence to the regression problem, consider the robust and sparse regression based on the γ -divergence and show that it has a strong robustness under heavy contamination even when outliers are heterogeneous. The loss function is constructed by an empirical estimate of the γ -divergence with sparse regularization, and the parameter estimate is defined as the minimizer of the loss function. To obtain the robust and sparse estimate, we propose an efficient update algorithm, which has a monotone decreasing property of the loss function. Particularly, we discuss a linear regression problem with L 1 regularization in detail. In numerical experiments and real data analyses, we see that the proposed method outperforms past robust and sparse methods.

Integrative regression network for genomic association study.

BackgroundThe increasing availability of multiple types of genomic profiles measured from the same cancer patients has provided numerous opportunities for investigating genomic mechanisms underlying cancer. In particular, association studies of gene expression traits with respect to multi-layered genomic features are highly useful for uncovering the underlying mechanism. Conventional correlation-based association tests are limited because they are prone to revealing indirect associations. Moreover, integration of multiple types of genomic features raises another challenge.MethodsIn this study, we propose a new framework for association studies called integrative regression network that identifies genomic associations on multiple high-dimensional genomic profiles by taking into account the associations between as well as within profiles. We employed high-dimensional regression techniques to first identify the associations between different genomic profiles. Based on the resulting regression coefficients, a regression network was constructed within each profile. For example, two methylation features having similar regression coefficients with respect to a number of gene expression traits are likely to be involved in the same biological process and therefore we define an edge between two methylation features in the regression network. To extract more reliable associations, multiple sparse structured regression techniques were applied and the resulting multiple networks were merged as the integrative regression network using a similarity network fusion technique.ResultsExperiments were carried out using four different sparse structured regression methods on five cancer types from TCGA. The advantages and disadvantages of each regression method were also explored. We find there was large inconsistency in the results from different regression methods, which supports the need to extract the proposed integrative regression network from multiple complimentary regression techniques. Fusing multiple regression networks by using similarity measurements led to the identification of significant gene pairs and a resulting network with better topological properties.ConclusionsWe developed and validated the integrative regression network scheme on multi-layered genomic profiles from TCGA. Our method facilitates identification of the strong signals as well as weaker signals by fusing information from different regression techniques. It could be extended to integrate results obtained from different cancer types as well.Electronic supplementary materialThe online version of this article (doi:10.1186/s12920-016-0192-7) contains supplementary material, which is available to authorized users.

Integrative approach for inference of gene regulatory networks using lasso-based random featuring and application to psychiatric disorders.

BackgroundInferring gene regulatory networks is one of the most interesting research areas in the systems biology. Many inference methods have been developed by using a variety of computational models and approaches. However, there are two issues to solve. First, depending on the structural or computational model of inference method, the results tend to be inconsistent due to innately different advantages and limitations of the methods. Therefore the combination of dissimilar approaches is demanded as an alternative way in order to overcome the limitations of standalone methods through complementary integration. Second, sparse linear regression that is penalized by the regularization parameter (lasso) and bootstrapping-based sparse linear regression methods were suggested in state of the art methods for network inference but they are not effective for a small sample size data and also a true regulator could be missed if the target gene is strongly affected by an indirect regulator with high correlation or another true regulator.ResultsWe present two novel network inference methods based on the integration of three different criteria, (i) z-score to measure the variation of gene expression from knockout data, (ii) mutual information for the dependency between two genes, and (iii) linear regression-based feature selection.Based on these criterion, we propose a lasso-based random feature selection algorithm (LARF) to achieve better performance overcoming the limitations of bootstrapping as mentioned above.ConclusionsIn this work, there are three main contributions. First, our z score-based method to measure gene expression variations from knockout data is more effective than similar criteria of related works. Second, we confirmed that the true regulator selection can be effectively improved by LARF. Lastly, we verified that an integrative approach can clearly outperform a single method when two different methods are effectively jointed. In the experiments, our methods were validated by outperforming the state of the art methods on DREAM challenge data, and then LARF was applied to inferences of gene regulatory network associated with psychiatric disorders.

Phenomic prediction of maize hybrids

Phenomic experiments are carried out in large-scale plant phenotyping facilities that acquire a large number of pictures of hundreds of plants simultaneously. With the aid of automated image processing, the data are converted into genotype-feature matrices that cover many consecutive days of development. Here, we explore the possibility of predicting the biomass of the fully grown plant from early developmental stage image-derived features. We performed phenomic experiments on 195 inbred and 382 hybrid maizes varieties and followed their progress from 16 days after sowing (DAS) to 48 DAS with 129 image-derived features. By applying sparse regression methods, we show that 73% of the variance in hybrid fresh weight of fully-grown plants is explained by about 20 features at the three-leaf-stage or earlier. Dry weight prediction explained over 90% of the variance. When phenomic features of parental inbred lines were used as predictors of hybrid biomass, the proportion of variance explained was 42 and 45%, for fresh weight and dry weight models consisting of 35 and 36 features, respectively. These models were very robust, showing only a small amount of variation in performance over the time scale of the experiment. We also examined mid-parent heterosis in phenomic features. Feature heterosis displayed a large degree of variance which resulted in prediction performance that was less robust than models of either parental or hybrid predictors. Our results show that phenomic prediction is a viable alternative to genomic and metabolic prediction of hybrid performance. In particular, the utility of early-stage parental lines is very encouraging.

Deep sparse multi-task learning for feature selection in Alzheimer's disease diagnosis.

Recently, neuroimaging-based Alzheimer's disease (AD) or mild cognitive impairment (MCI) diagnosis has attracted researchers in the field, due to the increasing prevalence of the diseases. Unfortunately, the unfavorable high-dimensional nature of neuroimagingdata, but a limited small number of samples available, makes it challenging to build a robust computer-aided diagnosis system. Machine learning techniques have been considered as a useful tool in this respect and, among various methods, sparse regression has shown its validity in the literature. However, to our best knowledge, the existing sparse regression methods mostly try to select features based on the optimal regression coefficients in one step. We argue that since the training feature vectors are composed of both informative and uninformative or less informative features, the resulting optimal regression coefficients are inevidently affected by the uninformative or less informative features. To this end, we first propose a novel deep architecture to recursively discard uninformative features by performing sparse multi-task learning in a hierarchical fashion. We further hypothesize that the optimal regression coefficients reflect the relative importance of features in representing the target response variables. In this regard, we use the optimal regression coefficients learned in one hierarchy as feature weighting factors in the following hierarchy, and formulate a weighted sparse multi-task learning method. Lastly, we also take into account the distributional characteristics of samples per class and use clustering-induced subclass label vectors as target response values in our sparse regression model. In our experiments on the ADNI cohort, we performed both binary and multi-class classification tasks in AD/MCI diagnosis and showed the superiority of the proposed method by comparing with the state-of-the-art methods.

Parallel Gaussian Process Regression for Big Data: Low-Rank Representation Meets Markov Approximation

The expressive power of a Gaussian process (GP) model comes at a cost of poor scalability in the data size. To improve its scalability, this paper presents a low-rank-cum-Markov approximation (LMA) of the GP model that is novel in leveraging the dual computational advantages stemming from complementing a low-rank approximate representation of the full-rank GP based on a support set of inputs with a Markov approximation of the resulting residual process; the latter approximation is guaranteed to be closest in the Kullback-Leibler distance criterion subject to some constraint and is considerably more refined than that of existing sparse GP models utilizing low-rank representations due to its more relaxed conditional independence assumption (especially with larger data). As a result, our LMA method can trade off between the size of the support set and the order of the Markov property to (a) incur lower computational cost than such sparse GP models while achieving predictive performance comparable to them and (b) accurately represent features/patterns of any scale. Interestingly, varying the Markov order produces a spectrum of LMAs with PIC approximation and full-rank GP at the two extremes. An advantage of our LMA method is that it is amenable to parallelization on multiple machines/cores, thereby gaining greater scalability. Empirical evaluation on three real-world datasets in clusters of up to 32 computing nodes shows that our centralized and parallel LMA methods are significantly more time-efficient and scalable than state-of-the-art sparse and full-rank GP regression methods while achieving comparable predictive performances.

Abstract T P45: Automated CSF Segmentation to Quantify Cerebral Edema Following Large Hemispheric Ischemic Stroke

Introduction: Cerebral edema with resultant mass effect is a potentially fatal consequence of ischemic stroke, but early and sensitive biomarkers of brain tissue compression are lacking. To quantify brain mass effect, we developed a novel, automated segmentation method to delineate CSF spaces in CT images from ischemic stroke patients. Methods: CTs from sixteen acute ischemic stroke patients (median NIHSS 16.5, median age 61.5 yrs, 14-92 hrs after stroke onset) were included after informed consent was obtained. After infarction, conventional CSF segmentation using Hounsfield unit (HU) thresholding is suboptimal due to infarct hypodensity. Utilizing manually delineated infarct and CSF spaces as training samples, we augmented conventional HU threshold segmentation with level sets, sparse regression and random forest segmentation methods. Using leave-one-out cross-validation, the combined approach was compared to HU thresholding using Dice ratios (a measure of the overlap between the segmented and the ground-truth CSF spaces). Results: Shown is an example of a CT brain slice segmented by HU thresholding and the combined strategy: false negative (red), false positive (green), and true positive (yellow). The Dice ratios for HU thresholding and the combined approaches were 58.2±16.3% and 68.9±14.6%, respectively, demonstrating the significantly improved performance for the combined strategy (p=0.0014). Conclusions: We have developed an advanced image segmentation strategy to delineate CSF spaces which outperforms conventional HU thresholding. An automated CSF segmentation strategy will permit quantification of cerebral edema in a large population of stroke patients, as required for genetic studies, for example.

Properties of sparse penalties on inferring gene regulatory networks from time-course gene expression data.

Genes regulate each other and form a gene regulatory network (GRN) to realise biological functions. Elucidating GRN from experimental data remains a challenging problem in systems biology. Numerous techniques have been developed and sparse linear regression methods become a promising approach to infer accurate GRNs. However, most linear methods are either based on steady-state gene expression data or their statistical properties are not analysed. Here, two sparse penalties, adaptive least absolute shrinkage and selection operator and smoothly clipped absolute deviation, are proposed to infer GRNs from time-course gene expression data based on an auto-regressive model and their Oracle properties are proved under mild conditions. The effectiveness of those methods is demonstrated by applications to in silico and real biological data.