ПОВЫШЕНИЕ УСТОЙЧИВОСТИ ЛИНЕЙНОЙ РЕГРЕССИИ

Abstract

The paper considers a method for increasing the stability of a linear regression equation, the coefficientsof which are the solution of a system of normal equations. The instability of the equation isdetermined by the presence of a linear relationship between the experimental data; the quantitativecharacteristics of the dependence are determined from the correlation matrix, which can also serve as amatrix of the system of equations. To reduce the correlation coefficients, Ridge (ridge) regression istraditionally used, which involves an increase in the diagonal members of the matrix by the same positivenumber. As a result, the matrix condition number decreases and the regression equation becomesmore stable: a small change in the input results in a small change in the solution. The number by whichthe diagonal terms of the matrix increase is called the penalty imposed in ridge regression on all regressioncoefficients. In the proposed method, penalties, and different ones, are imposed only on those coefficientsthat correspond to data with high correlation. This leads to an increase in the stability of theequation due to a decrease in the values of the coefficients corresponding to correlated data. The choiceof elements to be increased is based on the analysis of the correlation matrix of the original data set bydecomposing it into diagonal matrices using the square root method. In addition to increasing the stabilityusing the proposed method, a reduction in the dimension of the regression model can be achieved -a decrease in the number of terms of the corresponding equation, for which the LASSO and LARS algorithmsare usually used. The effectiveness of the method is tested on a known data set, and a comparisonis made not only with Ridge regression, but also with the results of known dimensionality reductionalgorithms.