Nonlinear regression: a hybrid model

Abstract

Using traditional parametric methods of regression analysis, one must make assumption(s) about the form of the regression equation which may not be valid. In high-dimensional cases of a reasonable sample size, nonparametric techniques of regression analysis (kernel, nearest neighbor, and spline smoothing) do not perform well due to the “curse of dimensionality”. This paper proposes a nontraditional nonlinear regression model for cases in which the sample space is high dimensional and the relationship between the independent variables and dependent variable is arbitrary. This research suggests the combination of the linear regression analysis method with the self-organizing feature maps, algorithm for high-dimensional convex polytopes, and back-propagation neural networks. When the sample set is pre-processed by a linear regression function, the self-organizing feature maps can be used to detect clusters of misrepresented sample points when they exist. Using the algorithm for high-dimensional convex polytopes, the sample data points in each of these clusters are sorted into two classes, each of which is supposed to distribute on one of the two sides of the pursued regression function. These groups of data points are then used to train the back-propagation neural network. The function represented by the trained neural network, which represents the boundary between the two groups, is the nonlinear regression function for the original data set. Scope and purpose Neural networks have received a great deal of attention in many research and application areas. One of the advantages of neural networks is the adaptive ability of generalization of data from the real world. Taking this advantage, many researchers use neural networks for nonlinear regression analysis and have reported positive experimental results in their applications. However, little research has addressed two correlated issues concurrently: the unpredictability of a single neural network in nonlinear regression, and a possible way(s) of improving the performance of neural networks in nonlinear regression analysis. In this paper a hybrid model which combines the linear regression method, neural networks, and the algorithm for convex polytopes is developed for nonlinear regression. This paper contests that a neural network model could be a more useful nonlinear regression tool if it successfully incorporates human knowledge (heuristics) and other regression techniques.