Least Angle Regression Algorithm Research Articles

Pendugaan Koefisien Regresi Logistik Biner Menggunakan Algoritma Least Angle Regression

Binary logistic regression is a statistical analysis method that aims to determine the relationship between variable which has two categories with the predictor variable that have categorical or continuous scale. The method that used to estimate logistic regression parameters is Maximum Likelihood Estimation (MLE) method. In estimating parameters, Least Angle Regression (LAR) algorithm is used to select the significant variables in order to get the best model from the estimation results of binary logistic regression coefficients. This LAR algorithm is applied to the risko of stunting data in two-year-old-babies at Buntu Batu Health Center working area, Enrekang Regency, South Sulawesi in 2019. This results obtained in the estimation of binary logistic regression prediction model using LAR algorithm, the standard error value is 0.018 smaller than the standard error value of binary logistic regression, which is 0.025. This shows that the binary logistic regression model using LAR algorithm is better than the usual binary logistic regression model on the risk of stunting data. Based on the results obtained, the variables that significantly affect the risk of stunting in two-year-old-babies on 2019 are father’s height, body length of birth, exclusive breastfeeding, history of infectious diseases, and history of immunization.

NIR Spectral Inversion of Soil Physicochemical Properties in Tea Plantations under Different Particle Size States.

Soil fertility is vital for the growth of tea plants. The physicochemical properties of soil play a key role in the evaluation of soil fertility. Thus, realizing the rapid and accurate detection of soil physicochemical properties is of great significance for promoting the development of precision agriculture in tea plantations. In recent years, spectral data have become an important tool for the non-destructive testing of soil physicochemical properties. In this study, a support vector regression (SVR) model was constructed to model the hydrolyzed nitrogen, available potassium, and effective phosphorus in tea plantation soils of different grain sizes. Then, the successful projections algorithm (SPA) and least-angle regression (LAR) and bootstrapping soft shrinkage (BOSS) variable importance screening methods were used to optimize the variables in the soil physicochemical properties. The findings demonstrated that soil particle sizes of 0.25-0.5 mm produced the best predictions for all three physicochemical properties. After further using the dimensionality reduction approach, the LAR algorithm (R2C = 0.979, R2P = 0.976, RPD = 6.613) performed optimally in the prediction model for hydrolytic nitrogen at a soil particle size of 0.25~0.5. The models using data dimensionality reduction and those that used the BOSS method to estimate available potassium (R2C = 0.977, R2P = 0.981, RPD = 7.222) and effective phosphorus (R2C = 0.969, R2P = 0.964, RPD = 5.163) had the best accuracy. In order to offer a reference for the accurate detection of soil physicochemical properties in tea plantations, this study investigated the modeling effect of each physicochemical property under various soil particle sizes and integrated the regression model with various downscaling strategies.

Investigation of a vertical closed‐loop geothermal system for heating an educational building

AbstractThis article is related to the thermal engineering aspect of the hybrid heating unit that can be operated either by shallow geothermal heat or natural gas. The thermal investigation is related to determining the fluctuation in the energy requirement if the unit is coupled with the gas boilers. Therefore, the given system was independently driven by both sources, and the conclusion was based on the comparative merit/demerit of renewable energy over the conventional power supply. The analysis assumed the steady‐state energy flow processes. The heat capacity of the source and sink was assumed to be infinitely large as compared to the working fluid. The leastangle regression algorithm was adopted to establish a mathematical relationship between thermodynamic parameters. The average COP of the closed‐loop geothermal unit was estimated to be 2.07. The overall exergy efficiency of the installed geothermal facility is 3%, whereas it is 33.55% for the gas boilers operated by natural gas. A square meter of the area of the building will get 26.25 W·h of useful convective heat energy from an installed geothermal unit and 28.59% of the total received will be dissipated through the structure. Similarly, the system driven by natural gas provided 21.88 W·h of the net useful convective heat energy, while the convective loss amounted to 23.06%. The net annual energy saving of 502.99 kW·h was expected from the geothermal site.

Multiple testing and variable selection along the path of the least angle regression

Abstract We investigate multiple testing and variable selection using the Least Angle Regression (LARS) algorithm in high dimensions under the assumption of Gaussian noise. LARS is known to produce a piecewise affine solution path with change points referred to as the knots of the LARS path. The key to our results is an expression in closed form of the exact joint law of a $K$-tuple of knots conditional on the variables selected by LARS, the so-called post-selection joint law of the LARS knots. Numerical experiments demonstrate the perfect fit of our findings. This paper makes three main contributions. First, we build testing procedures on variables entering the model along the LARS path in the general design case when the noise level can be unknown. These testing procedures are referred to as the Generalized $t$-Spacing tests and we prove that they have an exact non-asymptotic level (i.e. the Type I error is exactly controlled). This extends work of [ 31] where the spacing test works for consecutive knots and known variance. Second, we introduce a new exact multiple testing procedure after model selection in the general design case when the noise level may be unknown. We prove that this testing procedure has exact non-asymptotic level for general design and unknown noise level. Third, we prove exact control of the false discovery rate under orthogonal design assumption. Monte-Carlo simulations and a real data experiment are provided to illustrate our results in this case. Of independent interest, we introduce an equivalent formulation of the LARS algorithm based on a recursive function.

Penerapan Regresi Least Absolute Shrinkage And Selection Operator (LASSO) Untuk Mengidentifikasi Variabel Yang Berpengaruh Terhadap Kejadian Stunting di Indonesia

Linear regression analysis is an analytical method that can be used to analyze data and draw meaningful conclusions about the dependence of one variable on another variable. In linear regression analysis there are several assumptions that must be met, namely normal distribution, there is no correlation between errors. There are several obstacles that cause the assumption to be unfulfilled, for example the occurrence of correlations between independent variables (multicollinearity). The analysis in this study uses the Least Absolute Shrinkage And Selection Operator (LASSO) regression method with the Least Angle Regression (LAR) algorithm because the stunting data in Indonesia has multicollinearity problems among the independent variables used. LASSO which can solve the case of multicollinearity in the regression at the same time it is possible to reduce the regression coefficient from the highly correlated independent variable to exactly zero. The LASSO coefficient obtained uses quadratic so that the LAR algorithm is used which is more efficient in LASSO computing. Based on the analysis that has been carried out, it is concluded that the variables of exclusive breastfeeding (X1), protein consumption (X2), DPT-HB exercise (X5), maternal height (X8) and diarrhea (X9) had an effect on stunting in Indonesia in 2018.

The variable selection methods and algorithms in the multiple linear model

The Lasso and Ridge estimators are effective variable selection methods for a simple linear model. While the multiple linear model is often used in practice, only few studies have examined its problem of variable selection. To address this gap, we study such problem by taking advantage of the Lasso and Ridge estimate. We propose two variable selection methods in the multiple linear model and introduce the multiple random simulation (MRS) algorithm, whose efficiency is similar to that of the least angle regression (LARS) algorithm in a simple linear model. We verify the performance of these two algorithms by applying them to real diabetes data and find that the LARS algorithm cannot be easily applied in a multiple linear model. We also illustrate the excellent performance of MRS by applying this algorithm on a simulated dataset.

Comparison of the performance and reliability between improved sampling strategies for polynomial chaos expansion.

As uncertainty and sensitivity analysis of complex models grows ever more important, the difficulty of their timely realizations highlights a need for more efficient numerical operations. Non-intrusive Polynomial Chaos methods are highly efficient and accurate methods of mapping input-output relationships to investigate complex models. There is substantial potential to increase the efficacy of the method regarding the selected sampling scheme. We examine state-of-the-art sampling schemes categorized in space-filling-optimal designs such as Latin Hypercube sampling and L1-optimal sampling and compare their empirical performance against standard random sampling. The analysis was performed in the context of L1 minimization using the least-angle regression algorithm to fit the GPCE regression models. Due to the random nature of the sampling schemes, we compared different sampling approaches using statistical stability measures and evaluated the success rates to construct a surrogate model with relative errors of <0.1%, <1%, and <10%, respectively. The sampling schemes are thoroughly investigated by evaluating the y of surrogate models constructed for various distinct test cases, which represent different problem classes covering low, medium and high dimensional problems. Finally, the sampling schemes are tested on an application example to estimate the sensitivity of the self-impedance of a probe that is used to measure the impedance of biological tissues at different frequencies. We observed strong differences in the convergence properties of the methods between the analyzed test functions.

Evaluation of Kriging-NARX Modeling for Uncertainty Quantification of Nonlinear SDOF Systems with Degradation

Structural assessment for collapse is commonly approached by observing the failure or collapse of systems fully incorporating degradation. Challenges however exist in the performance indicator or damage measure due to compound impacts of uncertainties of external (seismic excitation) and internal (structural properties) characteristics with degradation behavior. To account for the impacts of uncertainties, the state-of-the-art kriging nonlinear autoregressive with exogenous (NARX) model is explored in this study to replicate the response of nonlinear single-degree-of-freedom systems. The generalized hysteretic Bouc-Wen model with internal uncertainties is selected to emulate the stiffness and strength degradation. A probabilistic stochastic ground motion model is introduced to represent the external uncertainties. The global terms of NARX model are selected by least-angle regression algorithm and the kriging model is utilized to surrogate uncertain parameters into corresponding NARX model coefficients. The predictions of kriging NARX models are further compared with that of the polynomial chaos nonlinear autoregressive with exogenous input form model as well as Monte Carlo simulation. The comparisons show that kriging NARX model presents an effective and efficient meta-model technique for uncertainty quantification of systems with degradation.

Parallel and Communication Avoiding Least Angle Regression

We are interested in parallelizing the least angle regression (LARS) algorithm for fitting linear regression models to high-dimensional data. We consider two parallel and communication avoiding versions of the basic LARS algorithm. The two algorithms have different asymptotic costs and practical performance. One offers more speedup and the other produces more accurate output. The first is bLARS, a block version of the LARS algorithm, where we update $b$ columns at each iteration. Assuming that the data are row-partitioned, bLARS reduces the number of arithmetic operations, latency, and bandwidth by a factor of $b$. The second is tournament-bLARS (T-bLARS), a tournament version of LARS where processors compete by running several LARS computations in parallel to choose $b$ new columns to be added in the solution. Assuming that the data are column-partitioned, T-bLARS reduces latency by a factor of $b$. Similarly to LARS, our proposed methods generate a sequence of linear models. We present extensive numerical experiments that illustrate speedups up to 4x compared to LARS without any compromise in solution quality.

Kronecker least angle regression for unsupervised unmixing of hyperspectral imaging data

Spectral unmixing is an important data processing task that is commonly applied to hyperspectral imaging data. It uses a set of spectral pixels, i.e., multichannel data, to separate each spectral pixel, a multicomponent signal comprised of a linear mixture of pure spectral signatures, into its individual spectral signatures commonly known as endmembers. When no prior information about the required endmembers is available, the resulting unsupervised unmixing problem will be underdetermined, and additional constraints become necessary. A recent approach to solving this problem required that these endmembers be sparse in some dictionary. Sparse signal recovery is commonly solved using a basis pursuit optimization algorithm that requires specifying a data-dependent regularization parameter. Least angle regression (LARS) is a very efficient method to simultaneously solve the basis pursuit optimization problem for all relevant regularization parameter values. However, despite this efficiency of LARS, it has not been applied to the spectral unmixing problem before. This is likely because the application of LARS to large multichannel data could be very challenging in practice, due to the need for generation and storage of extremely large arrays (~ 1010 bytes in a relatively small area of spectral unmixing problem). In this paper, we extend the standard LARS algorithm, using Kronecker products, to make it suitable for practical and efficient recovery of sparse signals from large multichannel data, i.e., without the need to construct or process very large arrays, or the need for trial and error to determine the regularization parameter value. We then apply this new Kronecker LARS (K-LARS) algorithm to successfully achieve spectral unmixing of both synthetic and AVIRIS hyperspectral imaging data. We also compare our results to ones obtained using an earlier basis pursuit-based spectral unmixing algorithm, generalized morphological component analysis (GMCA). We show that these two results are similar, albeit our results were obtained without trial and error, or arbitrary choices, in specifying the regularization parameter. More important, our K-LARS algorithm could be a very valuable research tool to the signal processing community, where it could be used to solve sparse least squares problems involving large multichannel data.

A new strategy of least absolute shrinkage and selection operator coupled with sampling error profile analysis for wavelength selection

A new strategy based on sampling error profile analysis (SEPA) combined with least absolute shrinkage and selection operator (SEPA-LASSO) was proposed. LASSO has been proven to be effective for multivariate calibration with automatic variable selection for high-dimensional data.However, in the previous research, the critical process of multivariate calibration by LASSO was an optimization of 1-norm turning parameter for a fixed sample set without considering the behaviors of variable selection by different subsets of samples. In the present work, Monte Carlo Sampling (MCS), the core of SEPA framework, is used to investigate various sub-models. Least angle regression (LAR) is used to solve LASSO, and various LAR iteration including certain number of variables could be obtained instead of choosing the numerical values of 1-norm turning parameters. SEPA-LASSO algorithm consists of plenty of loops. Under the SEPA framework and LAR algorithm, a number of LASSO sub-models with the same dimensions are built by MCS in each loop, the vote rule is used to determine the importance of variables and select them to build variable subsets. After running the loops, several subsets of variables are obtained and their error profile is used to choose the optimal subset of variables. The performance of SEPA-LASSO was evaluated by three near-infrared (NIR) datasets. The results show that the model built by SEPA-LASSO has excellent predictability and interpretability, compared with some commonly used multivariate calibration methods, such as principal component regression (PCR) and partial least squares (PLS), as well as some wavelength selection methods including LASSO, moving window partial least squares regression (MWPLSR), Monte Carlo uninformative variable elimination (MC-UVE), ordered homogeneity pursuit lasso (OHPL) and stability competitive adaptive reweighted sampling (SCARS).

Power of the spacing test for least-angle regression

Recent advances in Post-Selection Inference have shown that conditional testing is relevant and tractable in high-dimensions. In the Gaussian linear model, further works have derived unconditional test statistics such as the Kac–Rice Pivot for general penalized problems. In order to test the global null, a prominent offspring of this breakthrough is the Spacing test that accounts the relative separation between the first two knots of the celebrated least-angle regression (LARS) algorithm. However, no results have been shown regarding the distribution of these test statistics under the alternative. For the first time, this paper addresses this important issue for the Spacing test and shows that it is unconditionally unbiased. Furthermore, we provide the first extension of the Spacing test to the frame of unknown noise variance. More precisely, we investigate the power of the Spacing test for LARS and prove that it is unbiased: its power is always greater or equal to the significance level $\alpha$. In particular, we describe the power of this test under various scenarii: we prove that its rejection region is optimal when the predictors are orthogonal; as the level $\alpha$ goes to zero, we show that the probability of getting a true positive is much greater than $\alpha$; and we give a detailed description of its power in the case of two predictors. Moreover, we numerically investigate a comparison between the Spacing test for LARS, the Pearson’s chi-squared test (goodness of fit) and a numerical testing procedure based on the maximal correlation. When the noise variance is unknown, our analysis unleashes a new test statistic that can be computed in cubic time in the population size and which we refer to as the $t$-Spacing test for LARS. The $t$-Spacing test involves the first two knots of the LARS algorithm and we give its distribution under the null hypothesis. Interestingly, numerical experiments witness that the $t$-Spacing test for LARS enjoys the same aforementioned properties as the Spacing test.

A comparative study for least angle regression on NIR spectra analysis to determine internal qualities of navel oranges

Internal qualities of navel oranges are the key factors for their market value and of major concern to customers. Unlike traditional subjective quality assessment, near infrared (NIR) spectroscopy based techniques are quantitative, convenient and non-destructive. Various machine learning methods have been applied to NIR spectra analysis to determine the fruit qualities. NIR spectra are usually of very high dimension. Explicit or implicit variable selection is essential to ensure prediction performance. Least angle regression (LAR) is a relatively new and efficient machine learning algorithm for regression analysis and is good for variable selection. We investigate the potential of the LAR algorithm for NIR spectra analysis to determine the internal qualities of navel oranges. A total of 1535 navel orange samples from 15 origins were prepared for NIR spectra collection and quality parameters measurement. Spectra are of 1500 dimensions with wavelengths ranging from 1000 nm to 2499 nm. The LAR was compared with the most widely used linear and nonlinear methods in three aspects: prediction accuracy, computational efficiency, and model interpretability. The results showed that the prediction performance of LAR was better than that of PLS, while slightly inferior to that of least squares support vector machines (LS-SVM). LAR was computationally more efficient than both PLS and LS-SVM. By concentrating on the most important predictors, LAR is much easier to reveal the most relevant predictors than PLS; LS-SVM was hardly interpretable because of its nonlinear kernel.

Forward and backward least angle regression for nonlinear system identification

A forward and backward least angle regression (LAR) algorithm is proposed to construct the nonlinear autoregressive model with exogenous inputs (NARX) that is widely used to describe a large class of nonlinear dynamic systems. The main objective of this paper is to improve model sparsity and generalization performance of the original forward LAR algorithm. This is achieved by introducing a replacement scheme using an additional backward LAR stage. The backward stage replaces insignificant model terms selected by forward LAR with more significant ones, leading to an improved model in terms of the model compactness and performance. A numerical example to construct four types of NARX models, namely polynomials, radial basis function (RBF) networks, neuro fuzzy and wavelet networks, is presented to illustrate the effectiveness of the proposed technique in comparison with some popular methods.

Least angle regression for model selection

Model selection using least angle regression (LARS) is an interesting approach proposed by Efron B, Hastie T, Johnstone L, Tibshirani R. Least angle regression. The Annals of statistics 2004, 320:407–499. In this paper we first review the LARS algorithm and its relationships with other popular methods such as stagewise regression and the lasso. Then we conduct a survey of recent developments and extensions of LARS. WIREs Comput Stat 2014, 6:116–123. doi: 10.1002/wics.1288This article is categorized under: Statistical Learning and Exploratory Methods of the Data Sciences &gt; Modeling Methods Statistical Models &gt; Model Selection Statistical and Graphical Methods of Data Analysis &gt; Modeling Methods and Algorithms Statistical Models &gt; Linear Models

Probabilistic Modeling of Anticipation in Human Controllers

This paper presents a methodology for determining whether human operators anticipate future control needs in order to compensate for time delays when controlling remote vehicles. The approach utilizes techniques drawn from the machine learning community in order to learn statistical models of human decision making. Models are fit to an experimental data set generated by remote operations of a robot subjected to time delays between 0 and 2.5 s, using the least angle regression (LARS) and sparse multinomial logistic regression (SMLR) algorithms. These algorithms make use of regularization to reduce the effects of overparameterization due to redundant or noisy environmental features. Models learned by LARS achieve an average prediction rate between 81% and 98%, depending on time delay, while those learned by SMLR achieve average rates between 68% and 86%. A novel metric of feature “importance” is used to evaluate the relative contributions of environmental features to model performance, motivated by the structure of the LARS algorithm. The degree to which human operators rely on anticipation is determined by examining how “importance” scores for features representing different prediction horizons vary with increasing time delay.

EFFICIENT AUTOMATIC SELECTION AND COMBINATION OF EEG FEATURES IN LEAST SQUARES CLASSIFIERS FOR MOTOR IMAGERY BRAIN–COMPUTER INTERFACES

Discriminative features have to be properly extracted and selected from the electroencephalographic (EEG) signals of each specific subject in order to achieve an adaptive brain-computer interface (BCI) system. This work presents an efficient wrapper-based methodology for feature selection and least squares discrimination of high-dimensional EEG data with low computational complexity. Features are computed in different time segments using three widely used methods for motor imagery tasks and, then, they are concatenated or averaged in order to take into account the time course variability of the EEG signals. Once EEG features have been extracted, proposed framework comprises two stages. The first stage entails feature ranking and, in this work, two different procedures have been considered, the least angle regression (LARS) and the Wilcoxon rank sum test, to compare the performance of each one. The second stage selects the most relevant features using an efficient leave-one-out (LOO) estimation based on the Allen's PRESS statistic. Experimental comparisons with the state-of-the-art BCI methods shows that this approach gives better results than current state-of-the-art approaches in terms of recognition rates and computational requirements and, also with respect to the first ranking stage, it is confirmed that the LARS algorithm provides better results than the Wilcoxon rank sum test for these experiments.

Least angle regression

The purpose of model selection algorithms such as All Subsets, Forward Selection and Backward Elimination is to choose a linear model on the basis of the same set of data to which the model will be applied. Typically we have available a large collection of possible covariates from which we hope to select a parsimonious set for the efficient prediction of a response variable. Least Angle Regression (LARS), a new model selection algorithm, is a useful and less greedy version of traditional forward selection methods. Three main properties are derived: (1) A simple modification of the LARS algorithm implements the Lasso, an attractive version of ordinary least squares that constrains the sum of the absolute regression coefficients; the LARS modification calculates all possible Lasso estimates for a given problem, using an order of magnitude less computer time than previous methods. (2) A different LARS modification efficiently implements Forward Stagewise linear regression, another promising new model selection method; this connection explains the similar numerical results previously observed for the Lasso and Stagewise, and helps us understand the properties of both methods, which are seen as constrained versions of the simpler LARS algorithm. (3) A simple approximation for the degrees of freedom of a LARS estimate is available, from which we derive a Cp estimate of prediction error; this allows a principled choice among the range of possible LARS estimates. LARS and its variants are computationally efficient: the paper describes a publicly available algorithm that requires only the same order of magnitude of computational effort as ordinary least squares applied to the full set of covariates.