Unification of neural and statistical methods as applied to materials structure-property mapping

Abstract

A wide variety of neural and statistical methods are available for nonlinear empirical modeling based on different modeling approaches. Selecting the best method for a given task requires deep understanding of their similarities and differences and a systematic approach to method selection. This paper presents a common framework for gaining insight into neural and statistical modeling methods. The framework is then used to unify methods that combine inputs by linear projection before applying the basis function. The result of this unification is a new method called nonlinear continuum regression (NLCR) that unifies ordinary least squares regression (OLS), partial least squares regression (PLS), principal components regression (PCR) and ridge regression (RR), and nonlinear methods such as, backpropagation networks (BPN) with a single hidden layer, projection pursuit regression (PPR), nonlinear partial least squares regression (NLPLS), and nonlinear principal component regression (NLPCR), by spanning the continuum between these methods. The unification is facilitated by developing a common objective function for all methods in this category, and an efficient hierarchical training algorithm, illustrative examples on synthetic data and materials structure-property prediction demonstrate the ability of NLCR to specialize to the best existing method based on linear projection, or to a method between existing methods, resulting in the most general model from this class of methods.