Model Selection Based on Residual Dependence Measure

Abstract

Choosing the most suitable model from a set of models is crucial. Based on the assumption that the noise and independent variables in the model are independent, the degree of fit of the model was determined by studying the correlation between the residuals of the fitting equation and independent variables. The dependence measure can measure the dependence relationship between random variables. Common dependence measures include the Canonical Correlation Coefficient (CCA) and Distance Correlation (DC). In this study, the feasibility of the standard correlation coefficient and distance correlation in a two-dimensional case is proven by numerical simulation experiments, and the robustness of different noises, noise intensity and model types are also demonstrated. On the Boston housing price dataset, ridge regression, lasso regression, Bayesian regression and other methods were used to obtain different fitting equations. By comparing the CCA and DC of the residuals and independent variables, the estimation equation of the ridge regression was found to be the best, which proved the feasibility of the method.